Nameexact_genome_1532788802_163_9740_1
Workunit3887869
Created31 Jul 2018, 1:46:33 UTC
Sent31 Jul 2018, 2:16:14 UTC
Report deadline24 Jan 2019, 12:00:29 UTC
Received5 Aug 2018, 6:31:08 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID85440
Run time2 days 12 hours 15 min 11 sec
CPU time2 days 12 hours 12 min 56 sec
Validate stateInitial
Credit0.00
Device peak FLOPS5.01 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size574.64 MB
Peak swap size577.27 MB
Peak disk usage10.00 MB

Stderr output

<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
te_time: 0.0000000000, other_time: 2.1180725098
validation[          , genome  9740] predictions:    6210/  10000 (62.10%), best:    6304/10000 (63.04%), error:     10851.48047, best error:     10684.37598 on epoch:    16, epoch:   21/100, mu: 0.8502964377, learning_rate: 0.0001692661, weight_decay: 0.0001209080

epoch time: 2004.8856201172s, input_fired_time: 27.4360580444, output_fired_time: 9.2994108200, propagate_forward_time: 767.5343017578, propagate_backward_time: 1187.9525146484, weight_update_time: 0.5213132501, other_time: 12.6633300781
epoch time: 159.8968200684s, input_fired_time: 1.4375351667, output_fired_time: 0.0000000000, propagate_forward_time: 156.3618927002, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0973968506
validation[          , genome  9740] predictions:    6348/  10000 (63.48%), best:    6348/10000 (63.48%), error:     10571.02539, best error:     10571.02539 on epoch:    22, epoch:   22/100, mu: 0.8583322763, learning_rate: 0.0001580482, weight_decay: 0.0001137330

epoch time: 1938.0101318359s, input_fired_time: 27.2252159119, output_fired_time: 8.8493871689, propagate_forward_time: 735.7514038086, propagate_backward_time: 1154.0065917969, weight_update_time: 0.5069819093, other_time: 12.1774902344
epoch time: 148.4301300049s, input_fired_time: 1.4059062004, output_fired_time: 0.0000000000, propagate_forward_time: 144.9969482422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0272827148
validation[          , genome  9740] predictions:    6415/  10000 (64.15%), best:    6415/10000 (64.15%), error:     10221.08691, best error:     10221.08691 on epoch:    23, epoch:   23/100, mu: 0.8659058809, learning_rate: 0.0001475737, weight_decay: 0.0001069839

epoch time: 1978.5515136719s, input_fired_time: 27.3916740417, output_fired_time: 9.0500268936, propagate_forward_time: 755.6401977539, propagate_backward_time: 1173.8375244141, weight_update_time: 0.5225288868, other_time: 12.6319580078
epoch time: 162.1338195801s, input_fired_time: 1.4797669649, output_fired_time: 0.0000000000, propagate_forward_time: 158.4685974121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1854553223
validation[          , genome  9740] predictions:    6350/  10000 (63.50%), best:    6415/10000 (64.15%), error:     10542.33008, best error:     10221.08691 on epoch:    23, epoch:   24/100, mu: 0.8730438352, learning_rate: 0.0001377934, weight_decay: 0.0001006352

epoch time: 2047.4865722656s, input_fired_time: 27.6253528595, output_fired_time: 9.5684175491, propagate_forward_time: 786.3895263672, propagate_backward_time: 1210.7707519531, weight_update_time: 0.5343345404, other_time: 13.1325683594
epoch time: 161.1431732178s, input_fired_time: 1.4908441305, output_fired_time: 0.0000000000, propagate_forward_time: 157.4156951904, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2366333008
validation[          , genome  9740] predictions:    6460/  10000 (64.60%), best:    6415/10000 (64.15%), error:     10282.27539, best error:     10221.08691 on epoch:    23, epoch:   25/100, mu: 0.8797712326, learning_rate: 0.0001286612, weight_decay: 0.0000946633

epoch time: 2047.1276855469s, input_fired_time: 27.6255855560, output_fired_time: 9.5680189133, propagate_forward_time: 786.2160034180, propagate_backward_time: 1210.6251220703, weight_update_time: 0.5349327922, other_time: 13.0928955078
epoch time: 161.1260833740s, input_fired_time: 1.4716646671, output_fired_time: 0.0000000000, propagate_forward_time: 157.4259796143, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2284393311
validation[          , genome  9740] predictions:    6404/  10000 (64.04%), best:    6415/10000 (64.15%), error:     10450.70898, best error:     10221.08691 on epoch:    23, epoch:   26/100, mu: 0.8861116767, learning_rate: 0.0001201343, weight_decay: 0.0000890457

epoch time: 2043.8385009766s, input_fired_time: 27.5987396240, output_fired_time: 9.5522594452, propagate_forward_time: 784.8067626953, propagate_backward_time: 1208.7565917969, weight_update_time: 0.5360766053, other_time: 13.1241455078
epoch time: 160.6051025391s, input_fired_time: 1.4921777248, output_fired_time: 0.0000000000, propagate_forward_time: 156.8986358643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2142944336
validation[          , genome  9740] predictions:    6474/  10000 (64.74%), best:    6474/10000 (64.74%), error:     10208.96680, best error:     10208.96680 on epoch:    27, epoch:   27/100, mu: 0.8920874000, learning_rate: 0.0001121725, weight_decay: 0.0000837615

epoch time: 2037.5021972656s, input_fired_time: 27.5653495789, output_fired_time: 9.5246591568, propagate_forward_time: 783.5457763672, propagate_backward_time: 1203.8031005859, weight_update_time: 0.5336737037, other_time: 13.0633544922
epoch time: 160.4467468262s, input_fired_time: 1.4984787703, output_fired_time: 0.0000000000, propagate_forward_time: 156.7410278320, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2072448730
validation[          , genome  9740] predictions:    6519/  10000 (65.19%), best:    6519/10000 (65.19%), error:     10034.40137, best error:     10034.40137 on epoch:    28, epoch:   28/100, mu: 0.8977193832, learning_rate: 0.0001047384, weight_decay: 0.0000787909

epoch time: 2037.1169433594s, input_fired_time: 27.5490951538, output_fired_time: 9.5218486786, propagate_forward_time: 782.3606567383, propagate_backward_time: 1204.6354980469, weight_update_time: 0.5305042863, other_time: 13.0498046875
epoch time: 160.1166992188s, input_fired_time: 1.4935119152, output_fired_time: 0.0000000000, propagate_forward_time: 156.4079589844, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2152252197
validation[          , genome  9740] predictions:    6512/  10000 (65.12%), best:    6519/10000 (65.19%), error:     10086.52344, best error:     10034.40137 on epoch:    28, epoch:   29/100, mu: 0.9030274153, learning_rate: 0.0000977969, weight_decay: 0.0000741153

epoch time: 2034.8857421875s, input_fired_time: 27.5571708679, output_fired_time: 9.5105934143, propagate_forward_time: 782.3283691406, propagate_backward_time: 1202.4400634766, weight_update_time: 0.5302220583, other_time: 13.0495605469
epoch time: 160.4106903076s, input_fired_time: 1.4843986034, output_fired_time: 0.0000000000, propagate_forward_time: 156.7097015381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2165832520
validation[          , genome  9740] predictions:    6395/  10000 (63.95%), best:    6519/10000 (65.19%), error:     10268.99805, best error:     10034.40137 on epoch:    28, epoch:   30/100, mu: 0.9080301523, learning_rate: 0.0000913155, weight_decay: 0.0000697171

epoch time: 2032.6968994141s, input_fired_time: 27.5497303009, output_fired_time: 9.4947061539, propagate_forward_time: 782.4192504883, propagate_backward_time: 1200.1989746094, weight_update_time: 0.5270266533, other_time: 13.0341796875
epoch time: 160.2019348145s, input_fired_time: 1.4963582754, output_fired_time: 0.0000000000, propagate_forward_time: 156.5041351318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2014465332
validation[          , genome  9740] predictions:    6454/  10000 (64.54%), best:    6519/10000 (65.19%), error:     10097.24219, best error:     10034.40137 on epoch:    28, epoch:   31/100, mu: 0.9127451181, learning_rate: 0.0000852637, weight_decay: 0.0000655800

epoch time: 2034.2614746094s, input_fired_time: 27.5767154694, output_fired_time: 9.5030155182, propagate_forward_time: 782.9318847656, propagate_backward_time: 1201.2583007812, weight_update_time: 0.5292692780, other_time: 12.9915771484
epoch time: 160.8395690918s, input_fired_time: 1.5034649372, output_fired_time: 0.0000000000, propagate_forward_time: 157.1299285889, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2061767578
validation[          , genome  9740] predictions:    6520/  10000 (65.20%), best:    6520/10000 (65.20%), error:     10012.56055, best error:     10012.56055 on epoch:    32, epoch:   32/100, mu: 0.9171888828, learning_rate: 0.0000796129, weight_decay: 0.0000616883

epoch time: 2044.6134033203s, input_fired_time: 27.6715469360, output_fired_time: 9.5560817719, propagate_forward_time: 785.9609985352, propagate_backward_time: 1208.3500976562, weight_update_time: 0.5340033174, other_time: 13.0747070312
epoch time: 161.2127075195s, input_fired_time: 1.4891164303, output_fired_time: 0.0000000000, propagate_forward_time: 157.5064239502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2171630859
validation[          , genome  9740] predictions:    6507/  10000 (65.07%), best:    6520/10000 (65.20%), error:     10092.50195, best error:     10012.56055 on epoch:    32, epoch:   33/100, mu: 0.9213770032, learning_rate: 0.0000743366, weight_decay: 0.0000580276

epoch time: 2029.4433593750s, input_fired_time: 27.6578788757, output_fired_time: 9.4722852707, propagate_forward_time: 791.3399658203, propagate_backward_time: 1188.0814208984, weight_update_time: 0.5337594748, other_time: 12.8918457031
epoch time: 163.4252166748s, input_fired_time: 1.4991334677, output_fired_time: 0.0000000000, propagate_forward_time: 159.8286132812, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0974731445
validation[          , genome  9740] predictions:    6506/  10000 (65.06%), best:    6520/10000 (65.20%), error:     10027.87305, best error:     10012.56055 on epoch:    32, epoch:   34/100, mu: 0.9253242612, learning_rate: 0.0000694100, weight_decay: 0.0000545841

epoch time: 2045.5548095703s, input_fired_time: 27.7327136993, output_fired_time: 9.5632200241, propagate_forward_time: 785.3175048828, propagate_backward_time: 1209.8551025391, weight_update_time: 0.5329191685, other_time: 13.0863037109
epoch time: 161.2932128906s, input_fired_time: 1.4987794161, output_fired_time: 0.0000000000, propagate_forward_time: 157.5620574951, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2323760986
validation[          , genome  9740] predictions:    6550/  10000 (65.50%), best:    6550/10000 (65.50%), error:      9877.41602, best error:      9877.41602 on epoch:    35, epoch:   35/100, mu: 0.9290444255, learning_rate: 0.0000648100, weight_decay: 0.0000513449

epoch time: 2043.9420166016s, input_fired_time: 27.7357749939, output_fired_time: 9.5589103699, propagate_forward_time: 785.6972656250, propagate_backward_time: 1207.8592529297, weight_update_time: 0.5342316031, other_time: 13.0908203125
epoch time: 160.8165130615s, input_fired_time: 1.5110836029, output_fired_time: 0.0000000000, propagate_forward_time: 157.0687866211, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2366485596
validation[          , genome  9740] predictions:    6451/  10000 (64.51%), best:    6550/10000 (65.50%), error:     10193.49121, best error:      9877.41602 on epoch:    35, epoch:   36/100, mu: 0.9325506091, learning_rate: 0.0000605147, weight_decay: 0.0000482980

epoch time: 2044.3908691406s, input_fired_time: 27.7183551788, output_fired_time: 9.5504484177, propagate_forward_time: 784.8316650391, propagate_backward_time: 1209.1585693359, weight_update_time: 0.5321519971, other_time: 13.1318359375
epoch time: 160.9221496582s, input_fired_time: 1.5060200691, output_fired_time: 0.0000000000, propagate_forward_time: 157.2203979492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1957244873
validation[          , genome  9740] predictions:    6514/  10000 (65.14%), best:    6550/10000 (65.50%), error:      9996.98535, best error:      9877.41602 on epoch:    35, epoch:   37/100, mu: 0.9358551502, learning_rate: 0.0000565042, weight_decay: 0.0000454319

epoch time: 2040.2583007812s, input_fired_time: 27.6857299805, output_fired_time: 9.5467100143, propagate_forward_time: 785.2289428711, propagate_backward_time: 1204.6934814453, weight_update_time: 0.5323145390, other_time: 13.1033935547
epoch time: 160.7865753174s, input_fired_time: 1.5133019686, output_fired_time: 0.0000000000, propagate_forward_time: 157.0615997314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2116699219
validation[          , genome  9740] predictions:    6586/  10000 (65.86%), best:    6586/10000 (65.86%), error:      9827.30273, best error:      9827.30273 on epoch:    38, epoch:   38/100, mu: 0.9389696121, learning_rate: 0.0000527594, weight_decay: 0.0000427358

epoch time: 2040.2008056641s, input_fired_time: 27.7251300812, output_fired_time: 9.5361356735, propagate_forward_time: 784.9530029297, propagate_backward_time: 1204.9332275391, weight_update_time: 0.5317608118, other_time: 13.0533447266
epoch time: 160.8992614746s, input_fired_time: 1.5166691542, output_fired_time: 0.0000000000, propagate_forward_time: 157.1834411621, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1991577148
validation[          , genome  9740] predictions:    6527/  10000 (65.27%), best:    6586/10000 (65.86%), error:      9952.17480, best error:      9827.30273 on epoch:    38, epoch:   39/100, mu: 0.9419049025, learning_rate: 0.0000492628, weight_decay: 0.0000401998

epoch time: 2036.7811279297s, input_fired_time: 27.7294902802, output_fired_time: 9.5149669647, propagate_forward_time: 785.5562744141, propagate_backward_time: 1200.9632568359, weight_update_time: 0.5288153291, other_time: 13.0170898438
epoch time: 160.7177276611s, input_fired_time: 1.5081397295, output_fired_time: 0.0000000000, propagate_forward_time: 157.0005645752, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2090301514
validation[          , genome  9740] predictions:    6561/  10000 (65.61%), best:    6586/10000 (65.86%), error:     10020.72363, best error:      9827.30273 on epoch:    38, epoch:   40/100, mu: 0.9446713328, learning_rate: 0.0000459980, weight_decay: 0.0000378142

epoch time: 2048.1730957031s, input_fired_time: 27.8156204224, output_fired_time: 9.5541725159, propagate_forward_time: 786.0268554688, propagate_backward_time: 1211.6696777344, weight_update_time: 0.5326589942, other_time: 13.1066894531
epoch time: 160.7534637451s, input_fired_time: 1.5110331774, output_fired_time: 0.0000000000, propagate_forward_time: 157.0245361328, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2178955078
validation[          , genome  9740] predictions:    6676/  10000 (66.76%), best:    6676/10000 (66.76%), error:      9638.02246, best error:      9638.02246 on epoch:    41, epoch:   41/100, mu: 0.9472786784, learning_rate: 0.0000429495, weight_decay: 0.0000355703

epoch time: 2040.1353759766s, input_fired_time: 27.7190666199, output_fired_time: 9.5273675919, propagate_forward_time: 784.1183471680, propagate_backward_time: 1205.6796875000, weight_update_time: 0.5304707885, other_time: 13.0908203125
epoch time: 160.7290649414s, input_fired_time: 1.5069046021, output_fired_time: 0.0000000000, propagate_forward_time: 156.9963378906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2258300781
validation[          , genome  9740] predictions:    6636/  10000 (66.36%), best:    6676/10000 (66.76%), error:      9697.13281, best error:      9638.02246 on epoch:    41, epoch:   42/100, mu: 0.9497360587, learning_rate: 0.0000401031, weight_decay: 0.0000334594

epoch time: 2043.9196777344s, input_fired_time: 27.7880058289, output_fired_time: 9.5522680283, propagate_forward_time: 786.2213134766, propagate_backward_time: 1207.3186035156, weight_update_time: 0.5309228301, other_time: 13.0395507812
epoch time: 162.0920715332s, input_fired_time: 1.5162547827, output_fired_time: 0.0000000000, propagate_forward_time: 158.4160308838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1597900391
validation[          , genome  9740] predictions:    6670/  10000 (66.70%), best:    6676/10000 (66.76%), error:      9716.46191, best error:      9638.02246 on epoch:    41, epoch:   43/100, mu: 0.9520520568, learning_rate: 0.0000374453, weight_decay: 0.0000314739

epoch time: 2047.3161621094s, input_fired_time: 27.8760528564, output_fired_time: 9.5573797226, propagate_forward_time: 786.9259643555, propagate_backward_time: 1209.9215087891, weight_update_time: 0.5342870355, other_time: 13.0352783203
epoch time: 160.8284759521s, input_fired_time: 1.5350618362, output_fired_time: 0.0000000000, propagate_forward_time: 157.0753326416, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2180786133
validation[          , genome  9740] predictions:    6576/  10000 (65.76%), best:    6676/10000 (66.76%), error:      9779.48145, best error:      9638.02246 on epoch:    41, epoch:   44/100, mu: 0.9542348385, learning_rate: 0.0000349636, weight_decay: 0.0000296061

epoch time: 2027.5275878906s, input_fired_time: 27.7896118164, output_fired_time: 9.4392290115, propagate_forward_time: 779.6781005859, propagate_backward_time: 1197.7219238281, weight_update_time: 0.5295940042, other_time: 12.8988037109
epoch time: 160.9887695312s, input_fired_time: 1.5338022709, output_fired_time: 0.0000000000, propagate_forward_time: 157.2417297363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2132415771
validation[          , genome  9740] predictions:    6640/  10000 (66.40%), best:    6676/10000 (66.76%), error:      9767.84277, best error:      9638.02246 on epoch:    41, epoch:   45/100, mu: 0.9562920928, learning_rate: 0.0000326464, weight_decay: 0.0000278492

epoch time: 2037.5350341797s, input_fired_time: 27.8133659363, output_fired_time: 9.5165681839, propagate_forward_time: 785.6113281250, propagate_backward_time: 1201.5277099609, weight_update_time: 0.5327028036, other_time: 13.0660400391
epoch time: 161.1451110840s, input_fired_time: 1.5303364992, output_fired_time: 0.0000000000, propagate_forward_time: 157.3815765381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2332000732
validation[          , genome  9740] predictions:    6626/  10000 (66.26%), best:    6676/10000 (66.76%), error:      9731.88965, best error:      9638.02246 on epoch:    41, epoch:   46/100, mu: 0.9582309723, learning_rate: 0.0000304828, weight_decay: 0.0000261966

epoch time: 2021.9416503906s, input_fired_time: 27.7955856323, output_fired_time: 9.4096984863, propagate_forward_time: 773.2211303711, propagate_backward_time: 1198.6606445312, weight_update_time: 0.5298007727, other_time: 12.8547363281
epoch time: 151.2417602539s, input_fired_time: 1.4942250252, output_fired_time: 0.0000000000, propagate_forward_time: 147.7192687988, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0282592773
validation[          , genome  9740] predictions:    6623/  10000 (66.23%), best:    6676/10000 (66.76%), error:      9711.69238, best error:      9638.02246 on epoch:    41, epoch:   47/100, mu: 0.9600583315, learning_rate: 0.0000284626, weight_decay: 0.0000246420

epoch time: 1988.1086425781s, input_fired_time: 27.6847057343, output_fired_time: 9.0869131088, propagate_forward_time: 758.6039428711, propagate_backward_time: 1180.0512695312, weight_update_time: 0.5225533247, other_time: 12.6818847656
epoch time: 156.0759277344s, input_fired_time: 1.5000030994, output_fired_time: 0.0000000000, propagate_forward_time: 152.4270629883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1488647461
validation[          , genome  9740] predictions:    6616/  10000 (66.16%), best:    6676/10000 (66.76%), error:      9787.39844, best error:      9638.02246 on epoch:    41, epoch:   48/100, mu: 0.9617806077, learning_rate: 0.0000265763, weight_decay: 0.0000231797

epoch time: 1946.3222656250s, input_fired_time: 27.5731735229, output_fired_time: 8.7522010803, propagate_forward_time: 741.2202758789, propagate_backward_time: 1156.4176025391, weight_update_time: 0.5123929977, other_time: 12.3590087891
epoch time: 152.7404022217s, input_fired_time: 1.4800902605, output_fired_time: 0.0000000000, propagate_forward_time: 149.1898803711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0704345703
validation[          , genome  9740] predictions:    6620/  10000 (66.20%), best:    6676/10000 (66.76%), error:      9848.45215, best error:      9638.02246 on epoch:    41, epoch:   49/100, mu: 0.9634038210, learning_rate: 0.0000248149, weight_decay: 0.0000218042

epoch time: 2004.6273193359s, input_fired_time: 27.7289047241, output_fired_time: 9.2879867554, propagate_forward_time: 765.2815551758, propagate_backward_time: 1189.6295166016, weight_update_time: 0.5194054246, other_time: 12.6993408203
epoch time: 154.3285369873s, input_fired_time: 1.4934911728, output_fired_time: 0.0000000000, propagate_forward_time: 150.7330474854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1020050049
validation[          , genome  9740] predictions:    6686/  10000 (66.86%), best:    6686/10000 (66.86%), error:      9527.25781, best error:      9527.25781 on epoch:    50, epoch:   50/100, mu: 0.9649336338, learning_rate: 0.0000231704, weight_decay: 0.0000205103

epoch time: 1918.3690185547s, input_fired_time: 27.4978466034, output_fired_time: 8.6537761688, propagate_forward_time: 728.2202758789, propagate_backward_time: 1141.9265136719, weight_update_time: 0.5017374754, other_time: 12.0705566406
epoch time: 153.3180236816s, input_fired_time: 1.4896438122, output_fired_time: 0.0000000000, propagate_forward_time: 149.8045501709, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0238342285
validation[          , genome  9740] predictions:    6637/  10000 (66.37%), best:    6686/10000 (66.86%), error:      9649.27930, best error:      9527.25781 on epoch:    50, epoch:   51/100, mu: 0.9663754702, learning_rate: 0.0000216348, weight_decay: 0.0000192931

epoch time: 2035.8422851562s, input_fired_time: 27.8162746429, output_fired_time: 9.4810295105, propagate_forward_time: 780.6245727539, propagate_backward_time: 1205.0264892578, weight_update_time: 0.5292717218, other_time: 12.8939208984
epoch time: 159.8031463623s, input_fired_time: 1.5122184753, output_fired_time: 0.0000000000, propagate_forward_time: 156.1438903809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1470336914
validation[          , genome  9740] predictions:    6697/  10000 (66.97%), best:    6697/10000 (66.97%), error:      9527.17773, best error:      9527.17773 on epoch:    52, epoch:   52/100, mu: 0.9677343369, learning_rate: 0.0000202009, weight_decay: 0.0000181482

epoch time: 2029.4320068359s, input_fired_time: 27.8277950287, output_fired_time: 9.4563064575, propagate_forward_time: 781.3908081055, propagate_backward_time: 1197.7912597656, weight_update_time: 0.5309815407, other_time: 12.9658203125
epoch time: 161.1339111328s, input_fired_time: 1.5330556631, output_fired_time: 0.0000000000, propagate_forward_time: 157.4078521729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1930084229
validation[          , genome  9740] predictions:    6711/  10000 (67.11%), best:    6711/10000 (67.11%), error:      9452.57422, best error:      9452.57422 on epoch:    53, epoch:   53/100, mu: 0.9690150619, learning_rate: 0.0000188621, weight_decay: 0.0000170713

epoch time: 2021.0617675781s, input_fired_time: 27.7956676483, output_fired_time: 9.4055128098, propagate_forward_time: 775.7535400391, propagate_backward_time: 1195.2768554688, weight_update_time: 0.5272598267, other_time: 12.8302001953
epoch time: 151.1237945557s, input_fired_time: 1.4893062115, output_fired_time: 0.0000000000, propagate_forward_time: 147.6103973389, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0240936279
validation[          , genome  9740] predictions:    6704/  10000 (67.04%), best:    6711/10000 (67.11%), error:      9482.00293, best error:      9452.57422 on epoch:    53, epoch:   54/100, mu: 0.9702221155, learning_rate: 0.0000176121, weight_decay: 0.0000160582

epoch time: 1888.3615722656s, input_fired_time: 27.2448444366, output_fired_time: 8.5190277100, propagate_forward_time: 718.2103881836, propagate_backward_time: 1122.4234619141, weight_update_time: 0.4962812066, other_time: 11.9637451172
epoch time: 144.4743347168s, input_fired_time: 1.4649976492, output_fired_time: 0.0000000000, propagate_forward_time: 141.0654449463, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9438934326
validation[          , genome  9740] predictions:    6701/  10000 (67.01%), best:    6711/10000 (67.11%), error:      9553.10352, best error:      9452.57422 on epoch:    53, epoch:   55/100, mu: 0.9713597298, learning_rate: 0.0000164448, weight_decay: 0.0000151053

epoch time: 1844.0600585938s, input_fired_time: 27.0409965515, output_fired_time: 8.2816162109, propagate_forward_time: 700.3736572266, propagate_backward_time: 1096.7030029297, weight_update_time: 0.4869061708, other_time: 11.6607666016
epoch time: 106.5192489624s, input_fired_time: 1.3270124197, output_fired_time: 0.0000000000, propagate_forward_time: 103.7658081055, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4264297485
validation[          , genome  9740] predictions:    6707/  10000 (67.07%), best:    6711/10000 (67.11%), error:      9477.50879, best error:      9452.57422 on epoch:    53, epoch:   56/100, mu: 0.9724318981, learning_rate: 0.0000153550, weight_decay: 0.0000142089

epoch time: 1678.8449707031s, input_fired_time: 26.4143676758, output_fired_time: 7.2579312325, propagate_forward_time: 624.2702026367, propagate_backward_time: 1010.3267211914, weight_update_time: 0.4450534284, other_time: 10.5758056641
epoch time: 98.0053253174s, input_fired_time: 1.2960830927, output_fired_time: 0.0000000000, propagate_forward_time: 95.3978347778, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3114089966
validation[          , genome  9740] predictions:    6745/  10000 (67.45%), best:    6745/10000 (67.45%), error:      9429.02344, best error:      9429.02344 on epoch:    57, epoch:   57/100, mu: 0.9734424353, learning_rate: 0.0000143373, weight_decay: 0.0000133657

epoch time: 1678.6336669922s, input_fired_time: 26.4335193634, output_fired_time: 7.2384753227, propagate_forward_time: 626.0708618164, propagate_backward_time: 1008.4181518555, weight_update_time: 0.4414916039, other_time: 10.4726562500
epoch time: 145.5001983643s, input_fired_time: 1.4642034769, output_fired_time: 0.0000000000, propagate_forward_time: 142.0875396729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9484558105
validation[          , genome  9740] predictions:    6749/  10000 (67.49%), best:    6749/10000 (67.49%), error:      9358.56055, best error:      9358.56055 on epoch:    58, epoch:   58/100, mu: 0.9743948579, learning_rate: 0.0000133871, weight_decay: 0.0000125726

epoch time: 1669.7725830078s, input_fired_time: 26.4207668304, output_fired_time: 7.1798205376, propagate_forward_time: 622.5888061523, propagate_backward_time: 1003.1179809570, weight_update_time: 0.4409210682, other_time: 10.4652099609
epoch time: 105.1511993408s, input_fired_time: 1.3214954138, output_fired_time: 0.0000000000, propagate_forward_time: 102.4207229614, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4089813232
validation[          , genome  9740] predictions:    6692/  10000 (66.92%), best:    6749/10000 (67.49%), error:      9460.17578, best error:      9358.56055 on epoch:    58, epoch:   59/100, mu: 0.9752924442, learning_rate: 0.0000124999, weight_decay: 0.0000118265

epoch time: 1683.5013427734s, input_fired_time: 26.4535102844, output_fired_time: 7.2692646980, propagate_forward_time: 626.5385131836, propagate_backward_time: 1012.7383422852, weight_update_time: 0.4431498647, other_time: 10.5017089844
epoch time: 140.7867126465s, input_fired_time: 1.4603966475, output_fired_time: 0.0000000000, propagate_forward_time: 137.4460906982, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8802185059
validation[          , genome  9740] predictions:    6737/  10000 (67.37%), best:    6737/10000 (67.37%), error:      9310.14453, best error:      9310.14453 on epoch:    60, epoch:   60/100, mu: 0.9761384130, learning_rate: 0.0000116715, weight_decay: 0.0000111247

epoch time: 1761.7396240234s, input_fired_time: 26.6570892334, output_fired_time: 7.7710399628, propagate_forward_time: 665.3259887695, propagate_backward_time: 1050.9427490234, weight_update_time: 0.4632585049, other_time: 11.0428466797
epoch time: 153.3781433105s, input_fired_time: 1.5041444302, output_fired_time: 0.0000000000, propagate_forward_time: 149.7927703857, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0812225342
validation[          , genome  9740] predictions:    6770/  10000 (67.70%), best:    6737/10000 (67.37%), error:      9341.76758, best error:      9310.14453 on epoch:    60, epoch:   61/100, mu: 0.9769357443, learning_rate: 0.0000108980, weight_decay: 0.0000104645

epoch time: 1564.6926269531s, input_fired_time: 26.0055484772, output_fired_time: 6.5082521439, propagate_forward_time: 576.9111938477, propagate_backward_time: 945.4862060547, weight_update_time: 0.4181158841, other_time: 9.7814331055
epoch time: 107.4322433472s, input_fired_time: 1.3343936205, output_fired_time: 0.0000000000, propagate_forward_time: 104.6282196045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4696273804
validation[          , genome  9740] predictions:    6746/  10000 (67.46%), best:    6737/10000 (67.37%), error:      9395.61523, best error:      9310.14453 on epoch:    60, epoch:   62/100, mu: 0.9776871800, learning_rate: 0.0000101757, weight_decay: 0.0000098435

epoch time: 1635.6605224609s, input_fired_time: 26.1849346161, output_fired_time: 6.9652538300, propagate_forward_time: 609.2670288086, propagate_backward_time: 982.9771118164, weight_update_time: 0.4339761436, other_time: 10.2662353516
epoch time: 99.9136505127s, input_fired_time: 1.2989320755, output_fired_time: 0.0000000000, propagate_forward_time: 97.2689208984, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3457946777
validation[          , genome  9740] predictions:    6740/  10000 (67.40%), best:    6737/10000 (67.37%), error:      9404.72461, best error:      9310.14453 on epoch:    60, epoch:   63/100, mu: 0.9783954024, learning_rate: 0.0000095013, weight_decay: 0.0000092594

epoch time: 1578.4305419922s, input_fired_time: 25.8996601105, output_fired_time: 6.6322617531, propagate_forward_time: 583.4138183594, propagate_backward_time: 952.5836181641, weight_update_time: 0.4214819670, other_time: 9.9012451172
epoch time: 102.3205871582s, input_fired_time: 1.3119239807, output_fired_time: 0.0000000000, propagate_forward_time: 99.6444625854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3642044067
validation[          , genome  9740] predictions:    6783/  10000 (67.83%), best:    6737/10000 (67.37%), error:      9354.62891, best error:      9310.14453 on epoch:    60, epoch:   64/100, mu: 0.9790629148, learning_rate: 0.0000088717, weight_decay: 0.0000087099

epoch time: 1525.9973144531s, input_fired_time: 25.6664104462, output_fired_time: 6.2833890915, propagate_forward_time: 560.0507812500, propagate_backward_time: 924.5533447266, weight_update_time: 0.4082008600, other_time: 9.4433593750
epoch time: 113.3742980957s, input_fired_time: 1.3363268375, output_fired_time: 0.0000000000, propagate_forward_time: 110.5445480347, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4934234619
validation[          , genome  9740] predictions:    6755/  10000 (67.55%), best:    6737/10000 (67.37%), error:      9375.31543, best error:      9310.14453 on epoch:    60, epoch:   65/100, mu: 0.9796920419, learning_rate: 0.0000082837, weight_decay: 0.0000081930

epoch time: 1683.2576904297s, input_fired_time: 26.1010150909, output_fired_time: 7.2522125244, propagate_forward_time: 629.7523803711, propagate_backward_time: 1009.6528930664, weight_update_time: 0.4433335364, other_time: 10.4991455078
epoch time: 147.2207031250s, input_fired_time: 1.4704086781, output_fired_time: 0.0000000000, propagate_forward_time: 143.7308502197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0194396973
validation[          , genome  9740] predictions:    6720/  10000 (67.20%), best:    6737/10000 (67.37%), error:      9456.34375, best error:      9310.14453 on epoch:    60, epoch:   66/100, mu: 0.9802849889, learning_rate: 0.0000077347, weight_decay: 0.0000077069

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1532788802_163_9740.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1532788802_163_9740_1_r1497271369_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.505273
read mu: 0.980844
read mu_delta: 0.942479
read initial_learning_rate: 0.000714435
read learning_rate: 7.22209e-06
read learning_rate_delta: 0.933726
read initial_weight_decay: 0.000436909
read weight_decay: 7.24951e-06
read weight_decay_delta: 0.940658
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.110272
read input_dropout_probability: 0.0011958
read hidden_dropout_probability: 0.0955705
read velocity_reset: 1476
read epoch: 67
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 60
read number_validation_images: 10000
read best_validation_predictions: 6737
read best_validation_error: 9310.14
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 9740
read normal distribution: '1 0 0'
generator_str: '1692228052'
read generator: 1692228052
read generated_by_map:
1 disable_edge 1
reading 128 nodes.
reading 505 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1993.94946289062s, input_fired_time: 27.4586067199707, output_fired_time: 9.37475109100342, propagate_forward_time: 769.551025390625, propagate_backward_time: 1175.16186523438, weight_update_time: 0.530308604240417, other_time: 12.4031982421875
epoch time: 161.042541503906s, input_fired_time: 1.55344665050507, output_fired_time: 0, propagate_forward_time: 157.259017944336, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.23007202148438
validation[          , genome  9740] predictions:    6784/  10000 (67.84%), best:    6784/10000 (67.84%), error:      9268.29980, best error:      9268.29980 on epoch:    67, epoch:   67/100, mu: 0.9808437824, learning_rate: 0.0000072221, weight_decay: 0.0000072495

epoch time: 2040.4862060547s, input_fired_time: 27.9088020325, output_fired_time: 9.5449352264, propagate_forward_time: 782.9546508789, propagate_backward_time: 1206.9685058594, weight_update_time: 0.5343783498, other_time: 13.1093750000
epoch time: 160.7030334473s, input_fired_time: 1.5603730679, output_fired_time: 0.0000000000, propagate_forward_time: 156.8966217041, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2460327148
validation[          , genome  9740] predictions:    6798/  10000 (67.98%), best:    6784/10000 (67.84%), error:      9271.76270, best error:      9268.29980 on epoch:    67, epoch:   68/100, mu: 0.9813704491, learning_rate: 0.0000067434, weight_decay: 0.0000068193

epoch time: 2034.2165527344s, input_fired_time: 27.8827762604, output_fired_time: 9.5154628754, propagate_forward_time: 782.9405517578, propagate_backward_time: 1200.7557373047, weight_update_time: 0.5360614657, other_time: 13.1219482422
epoch time: 160.2626800537s, input_fired_time: 1.5470671654, output_fired_time: 0.0000000000, propagate_forward_time: 156.5064239502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2091827393
validation[          , genome  9740] predictions:    6797/  10000 (67.97%), best:    6797/10000 (67.97%), error:      9265.64648, best error:      9265.64648 on epoch:    69, epoch:   69/100, mu: 0.9818668365, learning_rate: 0.0000062965, weight_decay: 0.0000064146

epoch time: 2036.8286132812s, input_fired_time: 27.8717594147, output_fired_time: 9.5322656631, propagate_forward_time: 782.2149658203, propagate_backward_time: 1204.0052490234, weight_update_time: 0.5367076993, other_time: 13.2044677734
epoch time: 160.2664337158s, input_fired_time: 1.5471742153, output_fired_time: 0.0000000000, propagate_forward_time: 156.4853515625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2339019775
validation[          , genome  9740] predictions:    6790/  10000 (67.90%), best:    6790/10000 (67.90%), error:      9250.51855, best error:      9250.51855 on epoch:    70, epoch:   70/100, mu: 0.9823346734, learning_rate: 0.0000058792, weight_decay: 0.0000060340

epoch time: 2034.2362060547s, input_fired_time: 27.8519706726, output_fired_time: 9.5147266388, propagate_forward_time: 781.3756103516, propagate_backward_time: 1202.3970947266, weight_update_time: 0.5327531695, other_time: 13.0968017578
epoch time: 160.0449218750s, input_fired_time: 1.5498609543, output_fired_time: 0.0000000000, propagate_forward_time: 156.2997589111, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1952972412
validation[          , genome  9740] predictions:    6788/  10000 (67.88%), best:    6790/10000 (67.90%), error:      9266.30176, best error:      9250.51855 on epoch:    70, epoch:   71/100, mu: 0.9827755690, learning_rate: 0.0000054896, weight_decay: 0.0000056759

epoch time: 2031.4875488281s, input_fired_time: 27.8147201538, output_fired_time: 9.5025253296, propagate_forward_time: 780.2473144531, propagate_backward_time: 1200.8710937500, weight_update_time: 0.5328326225, other_time: 13.0518798828
epoch time: 159.9446105957s, input_fired_time: 1.5514451265, output_fired_time: 0.0000000000, propagate_forward_time: 156.1882019043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2049560547
validation[          , genome  9740] predictions:    6858/  10000 (68.58%), best:    6858/10000 (68.58%), error:      9191.46973, best error:      9191.46973 on epoch:    72, epoch:   72/100, mu: 0.9831911325, learning_rate: 0.0000051258, weight_decay: 0.0000053391

epoch time: 2031.9005126953s, input_fired_time: 27.8291034698, output_fired_time: 9.5032958984, propagate_forward_time: 780.3140869141, propagate_backward_time: 1201.0915527344, weight_update_time: 0.5376637578, other_time: 13.1624755859
epoch time: 159.7553863525s, input_fired_time: 1.5447237492, output_fired_time: 0.0000000000, propagate_forward_time: 156.0077056885, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2029571533
validation[          , genome  9740] predictions:    6830/  10000 (68.30%), best:    6858/10000 (68.58%), error:      9192.25586, best error:      9191.46973 on epoch:    72, epoch:   73/100, mu: 0.9835827947, learning_rate: 0.0000047861, weight_decay: 0.0000050222

epoch time: 2027.9117431641s, input_fired_time: 27.8149967194, output_fired_time: 9.4523220062, propagate_forward_time: 780.3060913086, propagate_backward_time: 1197.3519287109, weight_update_time: 0.5311499238, other_time: 12.9864501953
epoch time: 160.2453002930s, input_fired_time: 1.5520361662, output_fired_time: 0.0000000000, propagate_forward_time: 156.4784393311, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2148284912
validation[          , genome  9740] predictions:    6804/  10000 (68.04%), best:    6858/10000 (68.58%), error:      9309.23926, best error:      9191.46973 on epoch:    72, epoch:   74/100, mu: 0.9839519262, learning_rate: 0.0000044689, weight_decay: 0.0000047242

epoch time: 2030.8007812500s, input_fired_time: 27.8229751587, output_fired_time: 9.5029754639, propagate_forward_time: 781.3396606445, propagate_backward_time: 1199.0031738281, weight_update_time: 0.5367993116, other_time: 13.1320800781
epoch time: 160.1159820557s, input_fired_time: 1.5501364470, output_fired_time: 0.0000000000, propagate_forward_time: 156.3555450439, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2102966309
validation[          , genome  9740] predictions:    6815/  10000 (68.15%), best:    6858/10000 (68.58%), error:      9195.23828, best error:      9191.46973 on epoch:    72, epoch:   75/100, mu: 0.9842998385, learning_rate: 0.0000041727, weight_decay: 0.0000044439

epoch time: 2034.0981445312s, input_fired_time: 27.8541412354, output_fired_time: 9.5202465057, propagate_forward_time: 780.3885498047, propagate_backward_time: 1203.2573242188, weight_update_time: 0.5367638469, other_time: 13.0778808594
epoch time: 161.2451477051s, input_fired_time: 1.5538624525, output_fired_time: 0.0000000000, propagate_forward_time: 157.4991455078, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1921386719
validation[          , genome  9740] predictions:    6851/  10000 (68.51%), best:    6851/10000 (68.51%), error:      9183.95020, best error:      9183.95020 on epoch:    76, epoch:   76/100, mu: 0.9846277237, learning_rate: 0.0000038962, weight_decay: 0.0000041802

epoch time: 2003.9304199219s, input_fired_time: 27.7412490845, output_fired_time: 9.2793207169, propagate_forward_time: 766.2915039062, propagate_backward_time: 1187.6643066406, weight_update_time: 0.5309615731, other_time: 12.9541015625
epoch time: 151.3602600098s, input_fired_time: 1.5219786167, output_fired_time: 0.0000000000, propagate_forward_time: 147.7687683105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0695190430
validation[          , genome  9740] predictions:    6808/  10000 (68.08%), best:    6851/10000 (68.51%), error:      9204.34082, best error:      9183.95020 on epoch:    76, epoch:   77/100, mu: 0.9849367142, learning_rate: 0.0000036379, weight_decay: 0.0000039321

epoch time: 2019.9990234375s, input_fired_time: 27.7969703674, output_fired_time: 9.4159002304, propagate_forward_time: 776.8679199219, propagate_backward_time: 1192.8931884766, weight_update_time: 0.5336422920, other_time: 13.0250244141
epoch time: 156.0946197510s, input_fired_time: 1.5315643549, output_fired_time: 0.0000000000, propagate_forward_time: 152.4611358643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1019134521
validation[          , genome  9740] predictions:    6829/  10000 (68.29%), best:    6851/10000 (68.51%), error:      9212.10449, best error:      9183.95020 on epoch:    76, epoch:   78/100, mu: 0.9852279425, learning_rate: 0.0000033968, weight_decay: 0.0000036988

epoch time: 1904.5435791016s, input_fired_time: 27.4175453186, output_fired_time: 8.5855960846, propagate_forward_time: 721.0861816406, propagate_backward_time: 1135.3150634766, weight_update_time: 0.5024000406, other_time: 12.1391601562
epoch time: 145.3961486816s, input_fired_time: 1.4890164137, output_fired_time: 0.0000000000, propagate_forward_time: 141.8941345215, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0130004883
validation[          , genome  9740] predictions:    6814/  10000 (68.14%), best:    6851/10000 (68.51%), error:      9215.68457, best error:      9183.95020 on epoch:    76, epoch:   79/100, mu: 0.9855024219, learning_rate: 0.0000031717, weight_decay: 0.0000034793

epoch time: 2002.4007568359s, input_fired_time: 27.7907943726, output_fired_time: 9.3002347946, propagate_forward_time: 771.0952148438, propagate_backward_time: 1181.4387207031, weight_update_time: 0.5285957456, other_time: 12.7757568359
epoch time: 160.4405059814s, input_fired_time: 1.5585405827, output_fired_time: 0.0000000000, propagate_forward_time: 156.6993865967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1825714111
validation[          , genome  9740] predictions:    6825/  10000 (68.25%), best:    6851/10000 (68.51%), error:      9197.82715, best error:      9183.95020 on epoch:    76, epoch:   80/100, mu: 0.9857611060, learning_rate: 0.0000029615, weight_decay: 0.0000032728

epoch time: 2039.5832519531s, input_fired_time: 27.9559478760, output_fired_time: 9.5392780304, propagate_forward_time: 781.4675903320, propagate_backward_time: 1207.5245361328, weight_update_time: 0.5336697102, other_time: 13.0958251953
epoch time: 160.7201690674s, input_fired_time: 1.5612052679, output_fired_time: 0.0000000000, propagate_forward_time: 156.9468078613, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2121582031
validation[          , genome  9740] predictions:    6798/  10000 (67.98%), best:    6851/10000 (68.51%), error:      9250.63281, best error:      9183.95020 on epoch:    76, epoch:   81/100, mu: 0.9860049486, learning_rate: 0.0000027652, weight_decay: 0.0000030786

epoch time: 2039.1311035156s, input_fired_time: 28.0201168060, output_fired_time: 9.5368146896, propagate_forward_time: 783.9329833984, propagate_backward_time: 1204.5195312500, weight_update_time: 0.5379833579, other_time: 13.1215820312
epoch time: 160.6612396240s, input_fired_time: 1.5627083778, output_fired_time: 0.0000000000, propagate_forward_time: 156.8691406250, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2293853760
validation[          , genome  9740] predictions:    6830/  10000 (68.30%), best:    6851/10000 (68.51%), error:      9197.34375, best error:      9183.95020 on epoch:    76, epoch:   82/100, mu: 0.9862347245, learning_rate: 0.0000025820, weight_decay: 0.0000028959

epoch time: 2041.5922851562s, input_fired_time: 27.9787559509, output_fired_time: 9.5546789169, propagate_forward_time: 784.5734863281, propagate_backward_time: 1206.2869873047, weight_update_time: 0.5392900705, other_time: 13.1983642578
epoch time: 160.9243621826s, input_fired_time: 1.5638442039, output_fired_time: 0.0000000000, propagate_forward_time: 157.1147308350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2457885742
validation[          , genome  9740] predictions:    6859/  10000 (68.59%), best:    6859/10000 (68.59%), error:      9174.12891, best error:      9174.12891 on epoch:    83, epoch:   83/100, mu: 0.9864513278, learning_rate: 0.0000024109, weight_decay: 0.0000027240

epoch time: 2050.6330566406s, input_fired_time: 28.0846080780, output_fired_time: 9.5699205399, propagate_forward_time: 786.4315795898, propagate_backward_time: 1213.2949218750, weight_update_time: 0.5388503075, other_time: 13.2519531250
epoch time: 160.7689971924s, input_fired_time: 1.5655367374, output_fired_time: 0.0000000000, propagate_forward_time: 156.9833068848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2201538086
validation[          , genome  9740] predictions:    6860/  10000 (68.60%), best:    6860/10000 (68.60%), error:      9140.34570, best error:      9140.34570 on epoch:    84, epoch:   84/100, mu: 0.9866554737, learning_rate: 0.0000022511, weight_decay: 0.0000025624

epoch time: 2046.3360595703s, input_fired_time: 28.0455150604, output_fired_time: 9.5720653534, propagate_forward_time: 785.1141357422, propagate_backward_time: 1210.4025878906, weight_update_time: 0.5386252999, other_time: 13.2017822266
epoch time: 160.9272308350s, input_fired_time: 1.5684223175, output_fired_time: 0.0000000000, propagate_forward_time: 157.1196441650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2391662598
validation[          , genome  9740] predictions:    6838/  10000 (68.38%), best:    6838/10000 (68.38%), error:      9131.90137, best error:      9131.90137 on epoch:    85, epoch:   85/100, mu: 0.9868478775, learning_rate: 0.0000021019, weight_decay: 0.0000024103

epoch time: 2030.1308593750s, input_fired_time: 28.0385875702, output_fired_time: 9.4432582855, propagate_forward_time: 778.4572753906, propagate_backward_time: 1201.1821289062, weight_update_time: 0.5324730277, other_time: 13.0096435547
epoch time: 160.7561035156s, input_fired_time: 1.5588939190, output_fired_time: 0.0000000000, propagate_forward_time: 157.0359344482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1612701416
validation[          , genome  9740] predictions:    6848/  10000 (68.48%), best:    6848/10000 (68.48%), error:      9109.51172, best error:      9109.51172 on epoch:    86, epoch:   86/100, mu: 0.9870291948, learning_rate: 0.0000019626, weight_decay: 0.0000022673

epoch time: 2012.7999267578s, input_fired_time: 27.9361572266, output_fired_time: 9.3121356964, propagate_forward_time: 770.5178833008, propagate_backward_time: 1191.9967041016, weight_update_time: 0.5320405960, other_time: 13.0369873047
epoch time: 157.3298950195s, input_fired_time: 1.5403937101, output_fired_time: 0.0000000000, propagate_forward_time: 153.6121826172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1773223877
validation[          , genome  9740] predictions:    6846/  10000 (68.46%), best:    6848/10000 (68.48%), error:      9136.43848, best error:      9109.51172 on epoch:    86, epoch:   87/100, mu: 0.9872000813, learning_rate: 0.0000018325, weight_decay: 0.0000021328

epoch time: 1986.2722167969s, input_fired_time: 27.8470954895, output_fired_time: 9.1833677292, propagate_forward_time: 759.9110107422, propagate_backward_time: 1176.6075439453, weight_update_time: 0.5211507082, other_time: 12.7232666016
epoch time: 154.7239379883s, input_fired_time: 1.5418257713, output_fired_time: 0.0000000000, propagate_forward_time: 151.0327606201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1493530273
validation[          , genome  9740] predictions:    6842/  10000 (68.42%), best:    6848/10000 (68.48%), error:      9182.48633, best error:      9109.51172 on epoch:    86, epoch:   88/100, mu: 0.9873611331, learning_rate: 0.0000017111, weight_decay: 0.0000020062

epoch time: 1994.6663818359s, input_fired_time: 27.9145965576, output_fired_time: 9.1581954956, propagate_forward_time: 762.4244995117, propagate_backward_time: 1182.4388427734, weight_update_time: 0.5220881701, other_time: 12.7303466797
epoch time: 151.5337219238s, input_fired_time: 1.5282260180, output_fired_time: 0.0000000000, propagate_forward_time: 147.9356384277, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0698547363
validation[          , genome  9740] predictions:    6805/  10000 (68.05%), best:    6848/10000 (68.48%), error:      9194.66602, best error:      9109.51172 on epoch:    86, epoch:   89/100, mu: 0.9875129461, learning_rate: 0.0000015977, weight_decay: 0.0000018871

epoch time: 2006.2078857422s, input_fired_time: 27.9209690094, output_fired_time: 9.2663593292, propagate_forward_time: 765.8666992188, propagate_backward_time: 1190.3658447266, weight_update_time: 0.5262683034, other_time: 12.7879638672
epoch time: 148.3905639648s, input_fired_time: 1.5134403706, output_fired_time: 0.0000000000, propagate_forward_time: 144.8655548096, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0115661621
validation[          , genome  9740] predictions:    6857/  10000 (68.57%), best:    6857/10000 (68.57%), error:      9102.46387, best error:      9102.46387 on epoch:    90, epoch:   90/100, mu: 0.9876559973, learning_rate: 0.0000014918, weight_decay: 0.0000017752

epoch time: 1938.7851562500s, input_fired_time: 27.7333431244, output_fired_time: 8.8088579178, propagate_forward_time: 734.6887207031, propagate_backward_time: 1155.0070800781, weight_update_time: 0.5179224014, other_time: 12.5471191406
epoch time: 145.4696197510s, input_fired_time: 1.5045155287, output_fired_time: 0.0000000000, propagate_forward_time: 141.9745483398, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9905548096
validation[          , genome  9740] predictions:    6862/  10000 (68.62%), best:    6857/10000 (68.57%), error:      9115.83203, best error:      9102.46387 on epoch:    90, epoch:   91/100, mu: 0.9877908230, learning_rate: 0.0000013929, weight_decay: 0.0000016698

epoch time: 1961.5330810547s, input_fired_time: 27.8546924591, output_fired_time: 8.9987192154, propagate_forward_time: 746.7674560547, propagate_backward_time: 1165.3818359375, weight_update_time: 0.5174817443, other_time: 12.5302734375
epoch time: 161.0769653320s, input_fired_time: 1.5516407490, output_fired_time: 0.0000000000, propagate_forward_time: 157.2987365723, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2265930176
validation[          , genome  9740] predictions:    6845/  10000 (68.45%), best:    6857/10000 (68.57%), error:      9135.78809, best error:      9102.46387 on epoch:    90, epoch:   92/100, mu: 0.9879179001, learning_rate: 0.0000013006, weight_decay: 0.0000015707

epoch time: 2009.2259521484s, input_fired_time: 28.0477848053, output_fired_time: 9.2572078705, propagate_forward_time: 769.4329223633, propagate_backward_time: 1189.6005859375, weight_update_time: 0.5284144878, other_time: 12.8874511719
epoch time: 161.4749908447s, input_fired_time: 1.5873717070, output_fired_time: 0.0000000000, propagate_forward_time: 157.6761016846, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2115173340
validation[          , genome  9740] predictions:    6840/  10000 (68.40%), best:    6857/10000 (68.57%), error:      9151.94238, best error:      9102.46387 on epoch:    90, epoch:   93/100, mu: 0.9880376458, learning_rate: 0.0000012144, weight_decay: 0.0000014775

epoch time: 2052.1748046875s, input_fired_time: 28.2133827209, output_fired_time: 9.6012706757, propagate_forward_time: 787.1638183594, propagate_backward_time: 1214.0102539062, weight_update_time: 0.5387192965, other_time: 13.1860351562
epoch time: 162.3677368164s, input_fired_time: 1.5883123875, output_fired_time: 0.0000000000, propagate_forward_time: 158.6073303223, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1720886230
validation[          , genome  9740] predictions:    6850/  10000 (68.50%), best:    6857/10000 (68.57%), error:      9124.45312, best error:      9102.46387 on epoch:    90, epoch:   94/100, mu: 0.9881505370, learning_rate: 0.0000011339, weight_decay: 0.0000013898

epoch time: 2044.3355712891s, input_fired_time: 28.1662597656, output_fired_time: 9.5271167755, propagate_forward_time: 785.8950805664, propagate_backward_time: 1207.6674804688, weight_update_time: 0.5379080772, other_time: 13.0795898438
epoch time: 161.5832214355s, input_fired_time: 1.5882664919, output_fired_time: 0.0000000000, propagate_forward_time: 157.7583770752, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2365722656
validation[          , genome  9740] predictions:    6837/  10000 (68.37%), best:    6837/10000 (68.37%), error:      9100.99023, best error:      9100.99023 on epoch:    95, epoch:   95/100, mu: 0.9882569313, learning_rate: 0.0000010588, weight_decay: 0.0000013074

epoch time: 2071.1450195312s, input_fired_time: 28.1555995941, output_fired_time: 9.6257276535, propagate_forward_time: 791.0919799805, propagate_backward_time: 1229.2248535156, weight_update_time: 0.5380020738, other_time: 13.0468750000
epoch time: 161.6431732178s, input_fired_time: 1.5748877525, output_fired_time: 0.0000000000, propagate_forward_time: 157.8348541260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2334289551
validation[          , genome  9740] predictions:    6852/  10000 (68.52%), best:    6837/10000 (68.37%), error:      9117.88281, best error:      9100.99023 on epoch:    95, epoch:   96/100, mu: 0.9883571863, learning_rate: 0.0000009886, weight_decay: 0.0000012298

epoch time: 2066.8806152344s, input_fired_time: 28.1331520081, output_fired_time: 9.6268577576, propagate_forward_time: 791.7662963867, propagate_backward_time: 1224.4301757812, weight_update_time: 0.5366504192, other_time: 12.9240722656
epoch time: 161.8603057861s, input_fired_time: 1.5762296915, output_fired_time: 0.0000000000, propagate_forward_time: 158.0474090576, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2366638184
validation[          , genome  9740] predictions:    6870/  10000 (68.70%), best:    6837/10000 (68.37%), error:      9121.76367, best error:      9100.99023 on epoch:    95, epoch:   97/100, mu: 0.9884516597, learning_rate: 0.0000009231, weight_decay: 0.0000011568

epoch time: 2055.9160156250s, input_fired_time: 28.2072353363, output_fired_time: 9.6229553223, propagate_forward_time: 789.5800170898, propagate_backward_time: 1215.4034423828, weight_update_time: 0.5410199761, other_time: 13.1024169922
epoch time: 162.3070831299s, input_fired_time: 1.5914403200, output_fired_time: 0.0000000000, propagate_forward_time: 158.4744262695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2412109375
validation[          , genome  9740] predictions:    6847/  10000 (68.47%), best:    6837/10000 (68.37%), error:      9102.74902, best error:      9100.99023 on epoch:    95, epoch:   98/100, mu: 0.9885407090, learning_rate: 0.0000008619, weight_decay: 0.0000010881

epoch time: 2062.7099609375s, input_fired_time: 28.3175754547, output_fired_time: 9.6762075424, propagate_forward_time: 791.8150024414, propagate_backward_time: 1219.7670898438, weight_update_time: 0.5397878885, other_time: 13.1340332031
epoch time: 163.4517822266s, input_fired_time: 1.5859816074, output_fired_time: 0.0000000000, propagate_forward_time: 159.6702270508, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1955718994
validation[          , genome  9740] predictions:    6854/  10000 (68.54%), best:    6854/10000 (68.54%), error:      9090.38379, best error:      9090.38379 on epoch:    99, epoch:   99/100, mu: 0.9886246324, learning_rate: 0.0000008048, weight_decay: 0.0000010236

epoch time: 2061.5288085938s, input_fired_time: 28.3096103668, output_fired_time: 9.6802988052, propagate_forward_time: 793.8511352539, propagate_backward_time: 1216.6783447266, weight_update_time: 0.5401976705, other_time: 13.0093994141
epoch time: 162.5386047363s, input_fired_time: 1.5864188671, output_fired_time: 0.0000000000, propagate_forward_time: 158.7606811523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1914978027
validation[          , genome  9740] predictions:    6877/  10000 (68.77%), best:    6854/10000 (68.54%), error:      9117.08203, best error:      9090.38379 on epoch:    99, epoch:  100/100, mu: 0.9887037277, learning_rate: 0.0000007515, weight_decay: 0.0000009628

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 789.7521972656s, input_fired_time: 7.8576674461, output_fired_time: 0.0000000000, propagate_forward_time: 770.9653320312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.9291992188
best training[          , genome  9740] predictions:   37871/  50000 (75.74%), best:    6854/10000 (68.54%), error:     37187.57812, best error:      9090.38379 on epoch:    99, epoch:  101/100, mu: 0.9887782931, learning_rate: 0.0000007017, weight_decay: 0.0000009057
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 161.7007141113s, input_fired_time: 1.5874385834, output_fired_time: 0.0000000000, propagate_forward_time: 157.8782501221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2350311279
   testing[          , genome  9740] predictions:    6854/  10000 (68.54%), best:    6854/10000 (68.54%), error:      9090.38379, best error:      9090.38379 on epoch:    99, epoch:  101/100, mu: 0.9887782931, learning_rate: 0.0000007017, weight_decay: 0.0000009057
backpropagation finished successfully!
16:47:57 (126735): called boinc_finish(0)

</stderr_txt>
]]>