Nameexact_genome_1542727803_170_6410_1
Workunit4101337
Created20 Nov 2018, 20:17:21 UTC
Sent20 Nov 2018, 20:17:42 UTC
Report deadline23 Dec 2018, 19:46:30 UTC
Received23 Nov 2018, 6:48:42 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88573
Run time15 hours 37 min 27 sec
CPU time15 hours 37 min 27 sec
Validate stateValid
Credit8,244.59
Device peak FLOPS3.35 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size584.40 MB
Peak swap size583.18 MB
Peak disk usage12.47 MB

Stderr output

<core_client_version>7.6.22</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_170_6410.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_170_6410_1_r1643886273_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542727803_170_6410.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.459449
read mu: 0.459449
read mu_delta: 0.93604
read initial_learning_rate: 1e-05
read learning_rate: 1e-05
read learning_rate_delta: 0.905258
read initial_weight_decay: 0.000183789
read weight_decay: 0.000183789
read weight_decay_delta: 0.923984
read batch_size: 105
read epsilon: 1e-07
read alpha: 0.0507453
read input_dropout_probability: 0.0015858
read hidden_dropout_probability: 0.0788934
read velocity_reset: 1390
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6410
read normal distribution: '1 0 0'
generator_str: '1743330128'
read generator: 1743330128
read generated_by_map:
1 disable_node 1
reading 129 nodes.
reading 460 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 2030065275
post shuffle 1: 1668107381
epoch time: 2058.5263671875s, input_fired_time: 28.5798397064209, output_fired_time: 14.6868724822998, propagate_forward_time: 823.908874511719, propagate_backward_time: 1172.30444335938, weight_update_time: 0.578921556472778, other_time: 19.04638671875
epoch time: 192.761138916016s, input_fired_time: 2.5685338973999, output_fired_time: 0, propagate_forward_time: 186.610168457031, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.58244323730469
validation[          , genome  6410] predictions:    6372/  10000 (63.72%), best:    6372/10000 (63.72%), error:     10544.20117, best error:     10544.20117 on epoch:     0, epoch:    0/25, mu: 0.4594494998, learning_rate: 0.0000100000, weight_decay: 0.0001837892

epoch time: 2136.6267089844s, input_fired_time: 29.4023914337, output_fired_time: 15.3980140686, propagate_forward_time: 857.1514282227, propagate_backward_time: 1214.3406982422, weight_update_time: 0.6183666587, other_time: 20.3343505859
epoch time: 136.9183502197s, input_fired_time: 2.0661871433, output_fired_time: 0.0000000000, propagate_forward_time: 131.4321746826, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4199829102
validation[          , genome  6410] predictions:    6434/  10000 (64.34%), best:    6434/10000 (64.34%), error:     10506.36719, best error:     10506.36719 on epoch:     1, epoch:    1/25, mu: 0.4933833778, learning_rate: 0.0000090526, weight_decay: 0.0001698184

epoch time: 2001.1491699219s, input_fired_time: 28.2304687500, output_fired_time: 14.2279939651, propagate_forward_time: 799.2044677734, propagate_backward_time: 1140.6557617188, weight_update_time: 0.5651621819, other_time: 18.8304443359
epoch time: 133.4471740723s, input_fired_time: 1.9690468311, output_fired_time: 0.0000000000, propagate_forward_time: 128.3500213623, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1281127930
validation[          , genome  6410] predictions:    6461/  10000 (64.61%), best:    6434/10000 (64.34%), error:     10559.21875, best error:     10506.36719 on epoch:     1, epoch:    2/25, mu: 0.5251468420, learning_rate: 0.0000081949, weight_decay: 0.0001569096

epoch time: 1978.4943847656s, input_fired_time: 28.1410655975, output_fired_time: 14.1809215546, propagate_forward_time: 790.5062866211, propagate_backward_time: 1126.8642578125, weight_update_time: 0.5631703734, other_time: 18.8017578125
epoch time: 130.8281250000s, input_fired_time: 1.9494637251, output_fired_time: 0.0000000000, propagate_forward_time: 125.7658004761, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1128616333
validation[          , genome  6410] predictions:    6465/  10000 (64.65%), best:    6434/10000 (64.34%), error:     10647.31250, best error:     10506.36719 on epoch:     1, epoch:    3/25, mu: 0.5548787713, learning_rate: 0.0000074185, weight_decay: 0.0001449820

epoch time: 1958.7292480469s, input_fired_time: 28.0477104187, output_fired_time: 14.0073461533, propagate_forward_time: 783.4226684570, propagate_backward_time: 1114.5296630859, weight_update_time: 0.5561684370, other_time: 18.7218017578
epoch time: 206.9705352783s, input_fired_time: 2.7016286850, output_fired_time: 0.0000000000, propagate_forward_time: 200.6046905518, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6642150879
validation[          , genome  6410] predictions:    6470/  10000 (64.70%), best:    6434/10000 (64.34%), error:     10747.73242, best error:     10506.36719 on epoch:     1, epoch:    4/25, mu: 0.5827090144, learning_rate: 0.0000067157, weight_decay: 0.0001339611

epoch time: 1882.9406738281s, input_fired_time: 27.9040832520, output_fired_time: 13.5024948120, propagate_forward_time: 747.9005126953, propagate_backward_time: 1074.3273925781, weight_update_time: 0.5584633946, other_time: 19.3062744141
epoch time: 167.2792205811s, input_fired_time: 2.3108532429, output_fired_time: 0.0000000000, propagate_forward_time: 161.5258941650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4424743652
validation[          , genome  6410] predictions:    6465/  10000 (64.65%), best:    6434/10000 (64.34%), error:     10868.12109, best error:     10506.36719 on epoch:     1, epoch:    5/25, mu: 0.6087592840, learning_rate: 0.0000060794, weight_decay: 0.0001237780

epoch time: 1922.6531982422s, input_fired_time: 28.1676406860, output_fired_time: 13.8739833832, propagate_forward_time: 766.6212768555, propagate_backward_time: 1094.5921630859, weight_update_time: 0.5638564229, other_time: 19.3980712891
epoch time: 128.5783386230s, input_fired_time: 1.9360671043, output_fired_time: 0.0000000000, propagate_forward_time: 123.4660873413, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1761856079
validation[          , genome  6410] predictions:    6452/  10000 (64.52%), best:    6434/10000 (64.34%), error:     10994.24414, best error:     10506.36719 on epoch:     1, epoch:    6/25, mu: 0.6331433654, learning_rate: 0.0000055034, weight_decay: 0.0001143689

epoch time: 2003.8123779297s, input_fired_time: 28.7308788300, output_fired_time: 14.4760169983, propagate_forward_time: 801.1327514648, propagate_backward_time: 1139.6542968750, weight_update_time: 0.5861268044, other_time: 19.8183593750
epoch time: 209.8299102783s, input_fired_time: 2.7202804089, output_fired_time: 0.0000000000, propagate_forward_time: 203.4389648438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6706695557
validation[          , genome  6410] predictions:    6435/  10000 (64.35%), best:    6434/10000 (64.34%), error:     11095.02148, best error:     10506.36719 on epoch:     1, epoch:    7/25, mu: 0.6559678316, learning_rate: 0.0000049820, weight_decay: 0.0001056751

epoch time: 1999.5678710938s, input_fired_time: 28.3425006866, output_fired_time: 14.3107471466, propagate_forward_time: 801.0100097656, propagate_backward_time: 1136.8031005859, weight_update_time: 0.5652264953, other_time: 19.1014404297
epoch time: 144.3014221191s, input_fired_time: 2.0924983025, output_fired_time: 0.0000000000, propagate_forward_time: 138.8960266113, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3128967285
validation[          , genome  6410] predictions:    6437/  10000 (64.37%), best:    6434/10000 (64.34%), error:     11189.37891, best error:     10506.36719 on epoch:     1, epoch:    8/25, mu: 0.6773324609, learning_rate: 0.0000045100, weight_decay: 0.0000976422

epoch time: 2055.6520996094s, input_fired_time: 28.8419246674, output_fired_time: 14.7888002396, propagate_forward_time: 823.4548339844, propagate_backward_time: 1168.8553466797, weight_update_time: 0.5915773511, other_time: 19.7111816406
epoch time: 166.8893737793s, input_fired_time: 2.3083090782, output_fired_time: 0.0000000000, propagate_forward_time: 161.1539459229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4271240234
validation[          , genome  6410] predictions:    6430/  10000 (64.30%), best:    6434/10000 (64.34%), error:     11294.99121, best error:     10506.36719 on epoch:     1, epoch:    9/25, mu: 0.6973305941, learning_rate: 0.0000040827, weight_decay: 0.0000902199

epoch time: 1951.6337890625s, input_fired_time: 28.0566654205, output_fired_time: 13.9617519379, propagate_forward_time: 779.4914550781, propagate_backward_time: 1111.1821289063, weight_update_time: 0.5567601919, other_time: 18.9417724609
epoch time: 170.0082092285s, input_fired_time: 2.3678708076, output_fired_time: 0.0000000000, propagate_forward_time: 164.1348724365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5054626465
validation[          , genome  6410] predictions:    6401/  10000 (64.01%), best:    6434/10000 (64.34%), error:     11397.65430, best error:     10506.36719 on epoch:     1, epoch:   10/25, mu: 0.7160496712, learning_rate: 0.0000036959, weight_decay: 0.0000833617

epoch time: 1847.5202636719s, input_fired_time: 27.8484954834, output_fired_time: 13.3700351715, propagate_forward_time: 733.7991333008, propagate_backward_time: 1053.0992431641, weight_update_time: 0.5528565049, other_time: 19.4034423828
epoch time: 185.7852935791s, input_fired_time: 2.5615553856, output_fired_time: 0.0000000000, propagate_forward_time: 179.5186614990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7050781250
validation[          , genome  6410] predictions:    6388/  10000 (63.88%), best:    6434/10000 (64.34%), error:     11490.50098, best error:     10506.36719 on epoch:     1, epoch:   11/25, mu: 0.7335714698, learning_rate: 0.0000033458, weight_decay: 0.0000770250

epoch time: 1976.3107910156s, input_fired_time: 28.6802864075, output_fired_time: 14.2947473526, propagate_forward_time: 789.9429931641, propagate_backward_time: 1123.1580810547, weight_update_time: 0.5829380155, other_time: 20.2346191406
epoch time: 198.2601318359s, input_fired_time: 2.6861789227, output_fired_time: 0.0000000000, propagate_forward_time: 191.7853393555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7886199951
validation[          , genome  6410] predictions:    6365/  10000 (63.65%), best:    6434/10000 (64.34%), error:     11571.03711, best error:     10506.36719 on epoch:     1, epoch:   12/25, mu: 0.7499725819, learning_rate: 0.0000030288, weight_decay: 0.0000711699

epoch time: 1688.8226318359s, input_fired_time: 27.0608978271, output_fired_time: 12.2807550430, propagate_forward_time: 664.8900146484, propagate_backward_time: 965.3338623047, weight_update_time: 0.5170261264, other_time: 19.2570800781
epoch time: 130.3345031738s, input_fired_time: 1.9433020353, output_fired_time: 0.0000000000, propagate_forward_time: 125.2722473145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1189575195
validation[          , genome  6410] predictions:    6362/  10000 (63.62%), best:    6434/10000 (64.34%), error:     11637.80762, best error:     10506.36719 on epoch:     1, epoch:   13/25, mu: 0.7653247118, learning_rate: 0.0000027418, weight_decay: 0.0000657599

epoch time: 1909.2572021484s, input_fired_time: 27.7469177246, output_fired_time: 13.5332736969, propagate_forward_time: 762.4047241211, propagate_backward_time: 1086.9774169922, weight_update_time: 0.5461320281, other_time: 18.5948486328
epoch time: 148.4196777344s, input_fired_time: 2.1574950218, output_fired_time: 0.0000000000, propagate_forward_time: 142.9039764404, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3582000732
validation[          , genome  6410] predictions:    6351/  10000 (63.51%), best:    6434/10000 (64.34%), error:     11715.69043, best error:     10506.36719 on epoch:     1, epoch:   14/25, mu: 0.7796949148, learning_rate: 0.0000024821, weight_decay: 0.0000607611

epoch time: 2117.6867675781s, input_fired_time: 29.2747440338, output_fired_time: 15.2185459137, propagate_forward_time: 848.0357666016, propagate_backward_time: 1204.9067382813, weight_update_time: 0.6075817943, other_time: 20.2510986328
epoch time: 121.3617630005s, input_fired_time: 1.8525531292, output_fired_time: 0.0000000000, propagate_forward_time: 116.4611358643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0480728149
validation[          , genome  6410] predictions:    6324/  10000 (63.24%), best:    6434/10000 (64.34%), error:     11785.28125, best error:     10506.36719 on epoch:     1, epoch:   15/25, mu: 0.7931459546, learning_rate: 0.0000022469, weight_decay: 0.0000561423

epoch time: 2101.6752929688s, input_fired_time: 28.8386383057, output_fired_time: 15.0254077911, propagate_forward_time: 843.4711914063, propagate_backward_time: 1195.0917968750, weight_update_time: 0.5884861946, other_time: 19.2482910156
epoch time: 178.2917175293s, input_fired_time: 2.4091708660, output_fired_time: 0.0000000000, propagate_forward_time: 172.4184112549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4641418457
validation[          , genome  6410] predictions:    6328/  10000 (63.28%), best:    6434/10000 (64.34%), error:     11832.64844, best error:     10506.36719 on epoch:     1, epoch:   16/25, mu: 0.8057367206, learning_rate: 0.0000020340, weight_decay: 0.0000518746

epoch time: 1763.7928466797s, input_fired_time: 26.9258651733, output_fired_time: 12.5765361786, propagate_forward_time: 699.4096069336, propagate_backward_time: 1006.8154296875, weight_update_time: 0.5132324696, other_time: 18.0654296875
epoch time: 158.8986663818s, input_fired_time: 2.2169349194, output_fired_time: 0.0000000000, propagate_forward_time: 153.3625946045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3191375732
validation[          , genome  6410] predictions:    6336/  10000 (63.36%), best:    6434/10000 (64.34%), error:     11868.93164, best error:     10506.36719 on epoch:     1, epoch:   17/25, mu: 0.8175221682, learning_rate: 0.0000018413, weight_decay: 0.0000479313

epoch time: 2269.7519531250s, input_fired_time: 30.2651996613, output_fired_time: 16.3964996338, propagate_forward_time: 914.6989746094, propagate_backward_time: 1287.2597656250, weight_update_time: 0.6522902846, other_time: 21.1315917969
epoch time: 168.6117706299s, input_fired_time: 2.3835144043, output_fired_time: 0.0000000000, propagate_forward_time: 162.6147308350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6135253906
validation[          , genome  6410] predictions:    6292/  10000 (62.92%), best:    6434/10000 (64.34%), error:     11912.35840, best error:     10506.36719 on epoch:     1, epoch:   18/25, mu: 0.8285537958, learning_rate: 0.0000016669, weight_decay: 0.0000442878

epoch time: 1638.4439697266s, input_fired_time: 26.4386444092, output_fired_time: 11.8059206009, propagate_forward_time: 644.5050659180, propagate_backward_time: 937.5922241211, weight_update_time: 0.4910660982, other_time: 18.1021728516
epoch time: 187.4453735352s, input_fired_time: 2.4958491325, output_fired_time: 0.0000000000, propagate_forward_time: 181.4414978027, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5080261230
validation[          , genome  6410] predictions:    6302/  10000 (63.02%), best:    6434/10000 (64.34%), error:     11957.13281, best error:     10506.36719 on epoch:     1, epoch:   19/25, mu: 0.8388798833, learning_rate: 0.0000015089, weight_decay: 0.0000409212

epoch time: 1943.5000000000s, input_fired_time: 28.3686046600, output_fired_time: 14.0480747223, propagate_forward_time: 772.5646972656, propagate_backward_time: 1108.6232910156, weight_update_time: 0.5726519823, other_time: 19.8952636719
epoch time: 180.5302124023s, input_fired_time: 2.4372811317, output_fired_time: 0.0000000000, propagate_forward_time: 174.6093292236, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4835968018
validation[          , genome  6410] predictions:    6289/  10000 (62.89%), best:    6434/10000 (64.34%), error:     12008.04980, best error:     10506.36719 on epoch:     1, epoch:   20/25, mu: 0.8485454917, learning_rate: 0.0000013660, weight_decay: 0.0000378106

epoch time: 1836.4450683594s, input_fired_time: 27.3644256592, output_fired_time: 13.1565494537, propagate_forward_time: 729.1119384766, propagate_backward_time: 1048.3714599609, weight_update_time: 0.5294249654, other_time: 18.4407958984
epoch time: 200.5279541016s, input_fired_time: 2.6971740723, output_fired_time: 0.0000000000, propagate_forward_time: 194.0075683594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8232116699
validation[          , genome  6410] predictions:    6292/  10000 (62.92%), best:    6434/10000 (64.34%), error:     12015.47461, best error:     10506.36719 on epoch:     1, epoch:   21/25, mu: 0.8575928807, learning_rate: 0.0000012366, weight_decay: 0.0000349364

epoch time: 1873.6763916016s, input_fired_time: 27.5333194733, output_fired_time: 13.3903112411, propagate_forward_time: 745.3908691406, propagate_backward_time: 1068.8696289063, weight_update_time: 0.5399855375, other_time: 18.4923095703
epoch time: 151.5508270264s, input_fired_time: 2.1461906433, output_fired_time: 0.0000000000, propagate_forward_time: 146.1446838379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2599487305
validation[          , genome  6410] predictions:    6271/  10000 (62.71%), best:    6434/10000 (64.34%), error:     12072.35156, best error:     10506.36719 on epoch:     1, epoch:   22/25, mu: 0.8660616279, learning_rate: 0.0000011194, weight_decay: 0.0000322807

epoch time: 1928.9849853516s, input_fired_time: 27.9619216919, output_fired_time: 13.8596925735, propagate_forward_time: 767.9627685547, propagate_backward_time: 1100.3297119141, weight_update_time: 0.5539110303, other_time: 18.8708496094
epoch time: 188.6914825439s, input_fired_time: 2.5660071373, output_fired_time: 0.0000000000, propagate_forward_time: 182.4033660889, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7221069336
validation[          , genome  6410] predictions:    6279/  10000 (62.79%), best:    6434/10000 (64.34%), error:     12102.28223, best error:     10506.36719 on epoch:     1, epoch:   23/25, mu: 0.8739886880, learning_rate: 0.0000010134, weight_decay: 0.0000298269

epoch time: 1883.3669433594s, input_fired_time: 28.0283565521, output_fired_time: 13.6119470596, propagate_forward_time: 748.2010498047, propagate_backward_time: 1074.0284423828, weight_update_time: 0.5607168078, other_time: 19.4971923828
epoch time: 172.0850830078s, input_fired_time: 2.4185116291, output_fired_time: 0.0000000000, propagate_forward_time: 166.0544738770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6120910645
validation[          , genome  6410] predictions:    6252/  10000 (62.52%), best:    6434/10000 (64.34%), error:     12105.60352, best error:     10506.36719 on epoch:     1, epoch:   24/25, mu: 0.8814087510, learning_rate: 0.0000009173, weight_decay: 0.0000275596

epoch time: 2072.4072265625s, input_fired_time: 29.0573787689, output_fired_time: 14.9189586639, propagate_forward_time: 829.4050292969, propagate_backward_time: 1178.8430175781, weight_update_time: 0.6023310423, other_time: 20.1828613281
epoch time: 163.2489929199s, input_fired_time: 2.3269948959, output_fired_time: 0.0000000000, propagate_forward_time: 157.3545227051, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5674743652
validation[          , genome  6410] predictions:    6262/  10000 (62.62%), best:    6434/10000 (64.34%), error:     12133.01172, best error:     10506.36719 on epoch:     1, epoch:   25/25, mu: 0.8883542418, learning_rate: 0.0000008304, weight_decay: 0.0000254646

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 903.1093750000s, input_fired_time: 12.6174554825, output_fired_time: 0.0000000000, propagate_forward_time: 867.6412963867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 22.8506469727
best training[          , genome  6410] predictions:   36332/  50000 (72.66%), best:    6434/10000 (64.34%), error:     42634.60938, best error:     10506.36719 on epoch:     1, epoch:   26/25, mu: 0.8948554993, learning_rate: 0.0000007518, weight_decay: 0.0000235289
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 204.2369689941s, input_fired_time: 2.6980376244, output_fired_time: 0.0000000000, propagate_forward_time: 197.8771057129, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6618194580
   testing[          , genome  6410] predictions:    6434/  10000 (64.34%), best:    6434/10000 (64.34%), error:     10506.36719, best error:     10506.36719 on epoch:     1, epoch:   26/25, mu: 0.8948554993, learning_rate: 0.0000007518, weight_decay: 0.0000235289
backpropagation finished successfully!
00:46:09 (1780): called boinc_finish(0)

</stderr_txt>
]]>