Nameexact_genome_1542531604_185_6254_1
Workunit4097168
Created18 Nov 2018, 11:21:03 UTC
Sent18 Nov 2018, 11:57:02 UTC
Report deadline26 Feb 2019, 17:57:33 UTC
Received22 Nov 2018, 17:56:15 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID84153
Run time1 days 19 hours 29 min 27 sec
CPU time1 days 16 hours 44 min 22 sec
Validate stateValid
Credit25,062.59
Device peak FLOPS4.74 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size514.02 MB
Peak swap size517.74 MB
Peak disk usage5.88 MB

Stderr output

<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
ate_forward_time: 113.8356018066, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7616958618
validation[          , genome  6254] predictions:    5535/  10000 (55.35%), best:    5658/10000 (56.58%), error:     14425.75195, best error:     13485.71680 on epoch:    20, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 1533.8514404297s, input_fired_time: 9.4993200302, output_fired_time: 7.4132633209, propagate_forward_time: 561.8133544922, propagate_backward_time: 938.5889892578, weight_update_time: 1.0821261406, other_time: 16.5366210938
epoch time: 118.8840637207s, input_fired_time: 1.2022900581, output_fired_time: 0.0000000000, propagate_forward_time: 114.6840133667, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9977569580
validation[          , genome  6254] predictions:    5716/  10000 (57.16%), best:    5716/10000 (57.16%), error:     13309.81250, best error:     13309.81250 on epoch:    22, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1553.6241455078s, input_fired_time: 10.6112222672, output_fired_time: 8.4289760590, propagate_forward_time: 579.5893554688, propagate_backward_time: 933.9638671875, weight_update_time: 1.1138350964, other_time: 21.0307617188
epoch time: 117.2980957031s, input_fired_time: 2.1416316032, output_fired_time: 0.0000000000, propagate_forward_time: 112.3621368408, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7943267822
validation[          , genome  6254] predictions:    5553/  10000 (55.53%), best:    5716/10000 (57.16%), error:     13787.90234, best error:     13309.81250 on epoch:    22, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 1545.2271728516s, input_fired_time: 10.4947128296, output_fired_time: 7.4977202415, propagate_forward_time: 563.9985351562, propagate_backward_time: 947.3871459961, weight_update_time: 1.1514333487, other_time: 15.8490600586
epoch time: 116.3427276611s, input_fired_time: 1.1564378738, output_fired_time: 0.0000000000, propagate_forward_time: 112.3585662842, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8277206421
validation[          , genome  6254] predictions:    5640/  10000 (56.40%), best:    5716/10000 (57.16%), error:     13587.78027, best error:     13309.81250 on epoch:    22, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 1535.1253662109s, input_fired_time: 9.6004629135, output_fired_time: 7.3723602295, propagate_forward_time: 572.7293701172, propagate_backward_time: 928.5068969727, weight_update_time: 1.1431367397, other_time: 16.9163208008
epoch time: 117.1558609009s, input_fired_time: 1.2059419155, output_fired_time: 0.0000000000, propagate_forward_time: 113.0165328979, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9333877563
validation[          , genome  6254] predictions:    5716/  10000 (57.16%), best:    5716/10000 (57.16%), error:     13166.07129, best error:     13166.07129 on epoch:    25, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 1539.3919677734s, input_fired_time: 9.5892210007, output_fired_time: 7.4635801315, propagate_forward_time: 564.0642089844, propagate_backward_time: 942.4238891602, weight_update_time: 1.0993539095, other_time: 15.8510131836
epoch time: 113.6346206665s, input_fired_time: 1.1172320843, output_fired_time: 0.0000000000, propagate_forward_time: 109.8527221680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6646652222
validation[          , genome  6254] predictions:    5814/  10000 (58.14%), best:    5814/10000 (58.14%), error:     13163.78516, best error:     13163.78516 on epoch:    26, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 1536.6582031250s, input_fired_time: 11.5960550308, output_fired_time: 9.4695920944, propagate_forward_time: 562.7617797852, propagate_backward_time: 935.9714355469, weight_update_time: 1.0812823772, other_time: 16.8593139648
epoch time: 117.4166641235s, input_fired_time: 1.2201726437, output_fired_time: 0.0000000000, propagate_forward_time: 113.3659286499, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8305664062
validation[          , genome  6254] predictions:    5617/  10000 (56.17%), best:    5814/10000 (58.14%), error:     13544.07324, best error:     13163.78516 on epoch:    26, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 1531.2656250000s, input_fired_time: 9.6252155304, output_fired_time: 7.4325819016, propagate_forward_time: 551.5112304688, propagate_backward_time: 946.9440917969, weight_update_time: 1.1262321472, other_time: 15.7524414062
epoch time: 118.6280364990s, input_fired_time: 2.1985578537, output_fired_time: 0.0000000000, propagate_forward_time: 113.5908508301, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8386306763
validation[          , genome  6254] predictions:    5695/  10000 (56.95%), best:    5814/10000 (58.14%), error:     13487.47852, best error:     13163.78516 on epoch:    26, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 1527.9792480469s, input_fired_time: 10.5810050964, output_fired_time: 8.4320363998, propagate_forward_time: 572.7753295898, propagate_backward_time: 920.4758300781, weight_update_time: 1.1217100620, other_time: 15.7150268555
epoch time: 116.0374069214s, input_fired_time: 2.1960310936, output_fired_time: 0.0000000000, propagate_forward_time: 111.0847015381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7566757202
validation[          , genome  6254] predictions:    5755/  10000 (57.55%), best:    5755/10000 (57.55%), error:     13025.72070, best error:     13025.72070 on epoch:    29, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1536.8355712891s, input_fired_time: 9.6826639175, output_fired_time: 7.4076089859, propagate_forward_time: 561.3117065430, propagate_backward_time: 942.4882202148, weight_update_time: 1.1275677681, other_time: 15.9454345703
epoch time: 117.3703765869s, input_fired_time: 1.1757791042, output_fired_time: 0.0000000000, propagate_forward_time: 113.4384155273, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7561798096
validation[          , genome  6254] predictions:    5875/  10000 (58.75%), best:    5875/10000 (58.75%), error:     12672.22949, best error:     12672.22949 on epoch:    30, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 1521.0155029297s, input_fired_time: 10.5097713470, output_fired_time: 7.2266168594, propagate_forward_time: 556.8732299805, propagate_backward_time: 929.9460449219, weight_update_time: 2.0715866089, other_time: 16.4598999023
epoch time: 114.7994461060s, input_fired_time: 1.1453760862, output_fired_time: 0.0000000000, propagate_forward_time: 110.9240722656, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7299957275
validation[          , genome  6254] predictions:    5779/  10000 (57.79%), best:    5875/10000 (58.75%), error:     13149.52441, best error:     12672.22949 on epoch:    30, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 1497.7985839844s, input_fired_time: 10.2761726379, output_fired_time: 10.0870923996, propagate_forward_time: 548.7108154297, propagate_backward_time: 913.7243652344, weight_update_time: 1.0338495970, other_time: 15.0002441406
epoch time: 111.0024871826s, input_fired_time: 2.1229052544, output_fired_time: 0.0000000000, propagate_forward_time: 106.2856521606, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5939331055
validation[          , genome  6254] predictions:    5780/  10000 (57.80%), best:    5875/10000 (58.75%), error:     12781.33105, best error:     12672.22949 on epoch:    30, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 1503.7808837891s, input_fired_time: 9.4004354477, output_fired_time: 7.1566157341, propagate_forward_time: 549.3181762695, propagate_backward_time: 921.7527465820, weight_update_time: 1.0557830334, other_time: 16.1529541016
epoch time: 117.8501205444s, input_fired_time: 1.1901291609, output_fired_time: 0.0000000000, propagate_forward_time: 113.8681488037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7918395996
validation[          , genome  6254] predictions:    5755/  10000 (57.55%), best:    5875/10000 (58.75%), error:     13176.16895, best error:     12672.22949 on epoch:    30, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 1549.9522705078s, input_fired_time: 9.9065332413, output_fired_time: 10.5692472458, propagate_forward_time: 567.3764648438, propagate_backward_time: 945.0961303711, weight_update_time: 1.2139126062, other_time: 17.0039672852
epoch time: 117.8236236572s, input_fired_time: 1.2151991129, output_fired_time: 0.0000000000, propagate_forward_time: 113.7569808960, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8514404297
validation[          , genome  6254] predictions:    6022/  10000 (60.22%), best:    6022/10000 (60.22%), error:     12351.01660, best error:     12351.01660 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1526.8107910156s, input_fired_time: 9.5461835861, output_fired_time: 7.2949080467, propagate_forward_time: 556.0725708008, propagate_backward_time: 937.3422851562, weight_update_time: 1.0830389261, other_time: 16.5548706055
epoch time: 114.5224914551s, input_fired_time: 1.1401216984, output_fired_time: 0.0000000000, propagate_forward_time: 109.7021865845, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6801834106
validation[          , genome  6254] predictions:    5919/  10000 (59.19%), best:    6022/10000 (60.22%), error:     12602.38867, best error:     12351.01660 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1542.0288085938s, input_fired_time: 9.6911220551, output_fired_time: 8.3223066330, propagate_forward_time: 560.6959838867, propagate_backward_time: 946.5948486328, weight_update_time: 1.0909082890, other_time: 16.7245483398
epoch time: 115.9730987549s, input_fired_time: 1.1727813482, output_fired_time: 0.0000000000, propagate_forward_time: 111.0403594971, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7599563599
validation[          , genome  6254] predictions:    5710/  10000 (57.10%), best:    6022/10000 (60.22%), error:     13208.70020, best error:     12351.01660 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1547.3641357422s, input_fired_time: 9.7109403610, output_fired_time: 7.4672403336, propagate_forward_time: 574.6512451172, propagate_backward_time: 936.6380004883, weight_update_time: 1.0985068083, other_time: 18.8966674805
epoch time: 118.4312133789s, input_fired_time: 1.2051651478, output_fired_time: 0.0000000000, propagate_forward_time: 114.4052429199, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8208084106
validation[          , genome  6254] predictions:    5899/  10000 (58.99%), best:    6022/10000 (60.22%), error:     12542.57812, best error:     12351.01660 on epoch:    34, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1556.2812500000s, input_fired_time: 10.8090772629, output_fired_time: 10.4794435501, propagate_forward_time: 567.2244873047, propagate_backward_time: 950.9607543945, weight_update_time: 2.1154363155, other_time: 16.8074340820
epoch time: 119.0703277588s, input_fired_time: 1.2282469273, output_fired_time: 0.0000000000, propagate_forward_time: 115.0192489624, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8228302002
validation[          , genome  6254] predictions:    5913/  10000 (59.13%), best:    6022/10000 (60.22%), error:     12871.47656, best error:     12351.01660 on epoch:    34, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 1515.0872802734s, input_fired_time: 10.5402355194, output_fired_time: 7.2893509865, propagate_forward_time: 553.9993286133, propagate_backward_time: 923.1299438477, weight_update_time: 2.0536363125, other_time: 20.1284179688
epoch time: 114.8998565674s, input_fired_time: 1.1303117275, output_fired_time: 0.0000000000, propagate_forward_time: 111.0755004883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6940460205
validation[          , genome  6254] predictions:    5744/  10000 (57.44%), best:    6022/10000 (60.22%), error:     13243.50391, best error:     12351.01660 on epoch:    34, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1502.3559570312s, input_fired_time: 10.4923839569, output_fired_time: 7.1044335365, propagate_forward_time: 556.1094360352, propagate_backward_time: 910.5612792969, weight_update_time: 1.0707887411, other_time: 18.0883178711
epoch time: 111.8783416748s, input_fired_time: 1.1345851421, output_fired_time: 0.0000000000, propagate_forward_time: 108.0921783447, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6515808105
validation[          , genome  6254] predictions:    5885/  10000 (58.85%), best:    6022/10000 (60.22%), error:     12734.97168, best error:     12351.01660 on epoch:    34, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1514.9589843750s, input_fired_time: 10.4779672623, output_fired_time: 7.2040939331, propagate_forward_time: 560.9229125977, propagate_backward_time: 920.1551513672, weight_update_time: 1.0338797569, other_time: 16.1987915039
epoch time: 115.4858169556s, input_fired_time: 1.1978234053, output_fired_time: 0.0000000000, propagate_forward_time: 111.6165008545, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6714935303
validation[          , genome  6254] predictions:    6008/  10000 (60.08%), best:    6008/10000 (60.08%), error:     12238.78320, best error:     12238.78320 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1500.9254150391s, input_fired_time: 12.4369363785, output_fired_time: 7.1143512726, propagate_forward_time: 554.3032836914, propagate_backward_time: 910.1765747070, weight_update_time: 1.0244413614, other_time: 16.8942871094
epoch time: 112.6144638062s, input_fired_time: 1.1338332891, output_fired_time: 0.0000000000, propagate_forward_time: 107.7131118774, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7675170898
validation[          , genome  6254] predictions:    6027/  10000 (60.27%), best:    6008/10000 (60.08%), error:     12374.44434, best error:     12238.78320 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1512.4113769531s, input_fired_time: 9.4370794296, output_fired_time: 8.2276115417, propagate_forward_time: 555.6735839844, propagate_backward_time: 923.8858642578, weight_update_time: 1.0321911573, other_time: 15.1871337891
epoch time: 117.4482498169s, input_fired_time: 1.1626276970, output_fired_time: 0.0000000000, propagate_forward_time: 112.6615753174, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6240463257
validation[          , genome  6254] predictions:    6015/  10000 (60.15%), best:    6008/10000 (60.08%), error:     12445.72754, best error:     12238.78320 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1500.5999755859s, input_fired_time: 10.5359630585, output_fired_time: 8.0881824493, propagate_forward_time: 551.6928100586, propagate_backward_time: 910.2859497070, weight_update_time: 2.0431323051, other_time: 19.9970703125
epoch time: 112.7730407715s, input_fired_time: 1.1414506435, output_fired_time: 0.0000000000, propagate_forward_time: 108.9276046753, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7039871216
validation[          , genome  6254] predictions:    6018/  10000 (60.18%), best:    6008/10000 (60.08%), error:     12359.09375, best error:     12238.78320 on epoch:    41, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1499.3785400391s, input_fired_time: 9.5749025345, output_fired_time: 8.1980781555, propagate_forward_time: 560.1400146484, propagate_backward_time: 905.5489501953, weight_update_time: 1.0448516607, other_time: 15.9165039062
epoch time: 111.6004714966s, input_fired_time: 2.1763727665, output_fired_time: 0.0000000000, propagate_forward_time: 106.8226089478, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6014862061
validation[          , genome  6254] predictions:    6082/  10000 (60.82%), best:    6082/10000 (60.82%), error:     12118.28613, best error:     12118.28613 on epoch:    45, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1505.0742187500s, input_fired_time: 9.5206642151, output_fired_time: 9.0539264679, propagate_forward_time: 556.7338867188, propagate_backward_time: 912.5473632812, weight_update_time: 1.0437202454, other_time: 17.2183837891
epoch time: 119.6580505371s, input_fired_time: 1.2082977295, output_fired_time: 0.0000000000, propagate_forward_time: 115.8241653442, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6255874634
validation[          , genome  6254] predictions:    6067/  10000 (60.67%), best:    6082/10000 (60.82%), error:     12222.93750, best error:     12118.28613 on epoch:    45, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1509.8664550781s, input_fired_time: 9.6594743729, output_fired_time: 8.2346963882, propagate_forward_time: 553.5991210938, propagate_backward_time: 921.3071289062, weight_update_time: 1.0358208418, other_time: 17.0660400391
epoch time: 112.6004791260s, input_fired_time: 1.1445592642, output_fired_time: 0.0000000000, propagate_forward_time: 107.8434753418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6124420166
validation[          , genome  6254] predictions:    6023/  10000 (60.23%), best:    6082/10000 (60.82%), error:     12330.78125, best error:     12118.28613 on epoch:    45, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1521.6049804688s, input_fired_time: 9.6533069611, output_fired_time: 7.2626376152, propagate_forward_time: 568.3941650391, propagate_backward_time: 919.1956787109, weight_update_time: 1.0522714853, other_time: 17.0991210938
epoch time: 113.2079696655s, input_fired_time: 1.1323041916, output_fired_time: 0.0000000000, propagate_forward_time: 109.3933792114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6822891235
validation[          , genome  6254] predictions:    6057/  10000 (60.57%), best:    6057/10000 (60.57%), error:     12105.67969, best error:     12105.67969 on epoch:    48, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 1518.8919677734s, input_fired_time: 9.6666841507, output_fired_time: 7.2385110855, propagate_forward_time: 566.4828491211, propagate_backward_time: 916.9160156250, weight_update_time: 1.0680373907, other_time: 18.5879516602
epoch time: 118.3522567749s, input_fired_time: 1.2164252996, output_fired_time: 0.0000000000, propagate_forward_time: 114.1923751831, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9434585571
validation[          , genome  6254] predictions:    6034/  10000 (60.34%), best:    6057/10000 (60.57%), error:     12187.71094, best error:     12105.67969 on epoch:    48, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1527.4392089844s, input_fired_time: 9.8496789932, output_fired_time: 8.4682893753, propagate_forward_time: 566.5747070312, propagate_backward_time: 924.6159667969, weight_update_time: 1.1068354845, other_time: 17.9305419922
epoch time: 116.8235321045s, input_fired_time: 1.2073903084, output_fired_time: 0.0000000000, propagate_forward_time: 112.8923492432, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7237930298
validation[          , genome  6254] predictions:    6040/  10000 (60.40%), best:    6057/10000 (60.57%), error:     12576.31250, best error:     12105.67969 on epoch:    48, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 1530.0124511719s, input_fired_time: 10.8935375214, output_fired_time: 8.4839067459, propagate_forward_time: 567.8930664062, propagate_backward_time: 926.8286132812, weight_update_time: 1.0991169214, other_time: 15.9133300781
epoch time: 114.0626220703s, input_fired_time: 1.2234979868, output_fired_time: 0.0000000000, propagate_forward_time: 109.0735321045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7655944824
validation[          , genome  6254] predictions:    6074/  10000 (60.74%), best:    6074/10000 (60.74%), error:     12032.69336, best error:     12032.69336 on epoch:    51, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 1546.4211425781s, input_fired_time: 10.2840356827, output_fired_time: 7.6564726830, propagate_forward_time: 573.6331787109, propagate_backward_time: 935.1742553711, weight_update_time: 1.1485981941, other_time: 19.6731567383
epoch time: 114.0175933838s, input_fired_time: 1.2555353642, output_fired_time: 0.0000000000, propagate_forward_time: 109.9639129639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7981414795
validation[          , genome  6254] predictions:    5993/  10000 (59.93%), best:    6074/10000 (60.74%), error:     12528.11914, best error:     12032.69336 on epoch:    51, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 1535.9085693359s, input_fired_time: 11.0864925385, output_fired_time: 7.6235017776, propagate_forward_time: 568.0516357422, propagate_backward_time: 931.4523315430, weight_update_time: 1.1678104401, other_time: 17.6945190430
epoch time: 118.4384918213s, input_fired_time: 2.3741669655, output_fired_time: 0.0000000000, propagate_forward_time: 113.0725936890, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9917297363
validation[          , genome  6254] predictions:    6079/  10000 (60.79%), best:    6074/10000 (60.74%), error:     12320.91699, best error:     12032.69336 on epoch:    51, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 1542.7850341797s, input_fired_time: 10.2770433426, output_fired_time: 7.7819967270, propagate_forward_time: 567.5686035156, propagate_backward_time: 937.3208618164, weight_update_time: 1.1760278940, other_time: 19.8364868164
epoch time: 118.6370162964s, input_fired_time: 1.3159627914, output_fired_time: 0.0000000000, propagate_forward_time: 114.3455657959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9754867554
validation[          , genome  6254] predictions:    6038/  10000 (60.38%), best:    6074/10000 (60.74%), error:     12504.52539, best error:     12032.69336 on epoch:    51, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 1540.8728027344s, input_fired_time: 13.0480136871, output_fired_time: 7.6407389641, propagate_forward_time: 563.0492553711, propagate_backward_time: 939.3635253906, weight_update_time: 2.1686544418, other_time: 17.7713012695
epoch time: 116.2938613892s, input_fired_time: 1.2723875046, output_fired_time: 0.0000000000, propagate_forward_time: 111.0774917603, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9439849854
validation[          , genome  6254] predictions:    6009/  10000 (60.09%), best:    6074/10000 (60.74%), error:     12525.74805, best error:     12032.69336 on epoch:    51, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 1537.0227050781s, input_fired_time: 10.2229022980, output_fired_time: 7.7751622200, propagate_forward_time: 565.0421142578, propagate_backward_time: 936.2830810547, weight_update_time: 1.1755741835, other_time: 17.6994628906
epoch time: 116.0808715820s, input_fired_time: 1.2929759026, output_fired_time: 0.0000000000, propagate_forward_time: 111.9056701660, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8822250366
validation[          , genome  6254] predictions:    6124/  10000 (61.24%), best:    6074/10000 (60.74%), error:     12124.72852, best error:     12032.69336 on epoch:    51, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 1538.1071777344s, input_fired_time: 11.1690473557, output_fired_time: 7.7141261101, propagate_forward_time: 568.2440795898, propagate_backward_time: 931.1118164062, weight_update_time: 1.1535596848, other_time: 19.8681030273
epoch time: 118.6864242554s, input_fired_time: 1.3160405159, output_fired_time: 0.0000000000, propagate_forward_time: 113.2731628418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0972213745
validation[          , genome  6254] predictions:    6077/  10000 (60.77%), best:    6074/10000 (60.74%), error:     12270.69531, best error:     12032.69336 on epoch:    51, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1543.4178466797s, input_fired_time: 10.1688070297, output_fired_time: 8.6655473709, propagate_forward_time: 568.6627197266, propagate_backward_time: 937.0642089844, weight_update_time: 1.1430724859, other_time: 18.8565673828
epoch time: 116.8807144165s, input_fired_time: 1.3031795025, output_fired_time: 0.0000000000, propagate_forward_time: 112.5732421875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0042953491
validation[          , genome  6254] predictions:    6144/  10000 (61.44%), best:    6144/10000 (61.44%), error:     12027.20410, best error:     12027.20410 on epoch:    58, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 1543.6760253906s, input_fired_time: 10.2439174652, output_fired_time: 10.8049554825, propagate_forward_time: 557.3358764648, propagate_backward_time: 947.4464721680, weight_update_time: 1.2117015123, other_time: 17.8448486328
epoch time: 116.2909240723s, input_fired_time: 1.2808054686, output_fired_time: 0.0000000000, propagate_forward_time: 112.1025619507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9075546265
validation[          , genome  6254] predictions:    6087/  10000 (60.87%), best:    6144/10000 (61.44%), error:     12366.73633, best error:     12027.20410 on epoch:    58, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 1535.4190673828s, input_fired_time: 10.3174304962, output_fired_time: 7.7260751724, propagate_forward_time: 571.6243286133, propagate_backward_time: 928.9923706055, weight_update_time: 1.1816020012, other_time: 16.7589111328
epoch time: 115.8983230591s, input_fired_time: 1.2870843410, output_fired_time: 0.0000000000, propagate_forward_time: 111.6418533325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9693832397
validation[          , genome  6254] predictions:    6163/  10000 (61.63%), best:    6163/10000 (61.63%), error:     12025.70215, best error:     12025.70215 on epoch:    60, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 1543.9431152344s, input_fired_time: 12.3095788956, output_fired_time: 7.6523437500, propagate_forward_time: 565.5466918945, propagate_backward_time: 939.4523315430, weight_update_time: 1.1869776249, other_time: 18.9821777344
epoch time: 115.4050292969s, input_fired_time: 1.3034131527, output_fired_time: 0.0000000000, propagate_forward_time: 111.1135330200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9880828857
validation[          , genome  6254] predictions:    6052/  10000 (60.52%), best:    6163/10000 (61.63%), error:     12448.94238, best error:     12025.70215 on epoch:    60, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 1529.5718994141s, input_fired_time: 11.1552886963, output_fired_time: 8.6615524292, propagate_forward_time: 544.6213989258, propagate_backward_time: 947.5332641602, weight_update_time: 1.1702636480, other_time: 17.6004638672
epoch time: 113.5014266968s, input_fired_time: 1.3009225130, output_fired_time: 0.0000000000, propagate_forward_time: 109.2923278809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9081726074
validation[          , genome  6254] predictions:    6172/  10000 (61.72%), best:    6172/10000 (61.72%), error:     11891.41113, best error:     11891.41113 on epoch:    62, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 1536.3183593750s, input_fired_time: 12.4295511246, output_fired_time: 10.7492713928, propagate_forward_time: 558.5213623047, propagate_backward_time: 936.8575439453, weight_update_time: 1.1469771862, other_time: 17.7606201172
epoch time: 115.3911437988s, input_fired_time: 1.3152921200, output_fired_time: 0.0000000000, propagate_forward_time: 111.1357650757, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9400863647
validation[          , genome  6254] predictions:    6128/  10000 (61.28%), best:    6172/10000 (61.72%), error:     12127.62695, best error:     11891.41113 on epoch:    62, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 1551.1427001953s, input_fired_time: 10.3795309067, output_fired_time: 8.7144699097, propagate_forward_time: 573.3107910156, propagate_backward_time: 938.6560058594, weight_update_time: 1.2022311687, other_time: 20.0819091797
epoch time: 116.7994613647s, input_fired_time: 1.2721977234, output_fired_time: 0.0000000000, propagate_forward_time: 112.5746307373, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9526367188
validation[          , genome  6254] predictions:    6152/  10000 (61.52%), best:    6172/10000 (61.72%), error:     12027.95215, best error:     11891.41113 on epoch:    62, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_185_6254.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_185_6254_1_r343612068_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.972533
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000445599
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.7824e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 65
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 62
read number_validation_images: 10000
read best_validation_predictions: 6172
read best_validation_error: 11891.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6254
read normal distribution: '1 1.13862085342407 -0.365068942308426'
generator_str: '791989400'
read generator: 791989400
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 384 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1544.31921386719s, input_fired_time: 10.247275352478, output_fired_time: 8.74146270751953, propagate_forward_time: 574.110412597656, propagate_backward_time: 932.221862792969, weight_update_time: 1.19019758701324, other_time: 18.9981689453125
epoch time: 117.540405273438s, input_fired_time: 1.31381034851074, output_fired_time: 0, propagate_forward_time: 112.354713439941, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.87187957763672
validation[          , genome  6254] predictions:    6156/  10000 (61.56%), best:    6172/10000 (61.72%), error:     12036.00391, best error:     11891.41113 on epoch:    62, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 1528.8310546875s, input_fired_time: 11.1499538422, output_fired_time: 7.7145538330, propagate_forward_time: 553.6830444336, propagate_backward_time: 938.4906005859, weight_update_time: 1.1747680902, other_time: 17.7929077148
epoch time: 120.2825546265s, input_fired_time: 1.3118647337, output_fired_time: 0.0000000000, propagate_forward_time: 115.8677902222, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1028976440
validation[          , genome  6254] predictions:    6075/  10000 (60.75%), best:    6172/10000 (61.72%), error:     12309.39062, best error:     11891.41113 on epoch:    62, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 1543.9138183594s, input_fired_time: 13.2856998444, output_fired_time: 7.8487839699, propagate_forward_time: 562.8782348633, propagate_backward_time: 939.8472900391, weight_update_time: 1.1976336241, other_time: 20.0538940430
epoch time: 115.3044052124s, input_fired_time: 1.3019220829, output_fired_time: 0.0000000000, propagate_forward_time: 111.0632553101, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9392242432
validation[          , genome  6254] predictions:    6139/  10000 (61.39%), best:    6172/10000 (61.72%), error:     12087.90723, best error:     11891.41113 on epoch:    62, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 1568.9871826172s, input_fired_time: 11.9668684006, output_fired_time: 9.2740659714, propagate_forward_time: 577.0687255859, propagate_backward_time: 951.4644165039, weight_update_time: 1.2710688114, other_time: 19.2130737305
epoch time: 119.5754470825s, input_fired_time: 2.4392695427, output_fired_time: 0.0000000000, propagate_forward_time: 113.8571243286, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2790527344
validation[          , genome  6254] predictions:    6210/  10000 (62.10%), best:    6172/10000 (61.72%), error:     11897.92188, best error:     11891.41113 on epoch:    62, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 2154.2114257812s, input_fired_time: 14.4997987747, output_fired_time: 12.3861656189, propagate_forward_time: 818.7774658203, propagate_backward_time: 1283.9482421875, weight_update_time: 1.6026000977, other_time: 24.5997314453
epoch time: 169.0945892334s, input_fired_time: 1.8788157701, output_fired_time: 0.0000000000, propagate_forward_time: 163.2657623291, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9500122070
validation[          , genome  6254] predictions:    6168/  10000 (61.68%), best:    6172/10000 (61.72%), error:     12085.72168, best error:     11891.41113 on epoch:    62, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 1980.0201416016s, input_fired_time: 14.0549383163, output_fired_time: 9.8866968155, propagate_forward_time: 737.9521484375, propagate_backward_time: 1194.9475097656, weight_update_time: 1.5775407553, other_time: 23.1788330078
epoch time: 117.4590072632s, input_fired_time: 1.4413039684, output_fired_time: 0.0000000000, propagate_forward_time: 112.8124465942, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2052536011
validation[          , genome  6254] predictions:    6229/  10000 (62.29%), best:    6229/10000 (62.29%), error:     11698.15332, best error:     11698.15332 on epoch:    70, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 1567.3923339844s, input_fired_time: 11.9571123123, output_fired_time: 9.3106365204, propagate_forward_time: 571.9822387695, propagate_backward_time: 955.9188842773, weight_update_time: 1.2875964642, other_time: 18.2233886719
epoch time: 118.9617156982s, input_fired_time: 1.4297145605, output_fired_time: 0.0000000000, propagate_forward_time: 114.3507080078, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1812896729
validation[          , genome  6254] predictions:    6214/  10000 (62.14%), best:    6229/10000 (62.29%), error:     11821.52930, best error:     11698.15332 on epoch:    70, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 1567.0894775391s, input_fired_time: 11.0311412811, output_fired_time: 8.2916135788, propagate_forward_time: 579.1319580078, propagate_backward_time: 950.2211914062, weight_update_time: 1.3080165386, other_time: 18.4135742188
epoch time: 116.0566101074s, input_fired_time: 1.3611294031, output_fired_time: 0.0000000000, propagate_forward_time: 110.6433410645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0521392822
validation[          , genome  6254] predictions:    6236/  10000 (62.36%), best:    6229/10000 (62.29%), error:     11787.60547, best error:     11698.15332 on epoch:    70, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 1583.3570556641s, input_fired_time: 12.2412824631, output_fired_time: 9.5379743576, propagate_forward_time: 584.2377929688, propagate_backward_time: 955.4475097656, weight_update_time: 1.3452833891, other_time: 21.8924560547
epoch time: 123.3675613403s, input_fired_time: 1.5358756781, output_fired_time: 0.0000000000, propagate_forward_time: 118.5316009521, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3000869751
validation[          , genome  6254] predictions:    6209/  10000 (62.09%), best:    6229/10000 (62.29%), error:     12020.21582, best error:     11698.15332 on epoch:    70, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 1597.1578369141s, input_fired_time: 11.4868745804, output_fired_time: 8.5700826645, propagate_forward_time: 580.3159790039, propagate_backward_time: 976.4186401367, weight_update_time: 1.4064165354, other_time: 20.3663330078
epoch time: 123.8253021240s, input_fired_time: 1.5388222933, output_fired_time: 0.0000000000, propagate_forward_time: 118.9426345825, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3438415527
validation[          , genome  6254] predictions:    6263/  10000 (62.63%), best:    6229/10000 (62.29%), error:     11786.44629, best error:     11698.15332 on epoch:    70, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 1617.9829101562s, input_fired_time: 12.6383819580, output_fired_time: 9.8012027740, propagate_forward_time: 608.9363403320, propagate_backward_time: 962.9180297852, weight_update_time: 1.3751945496, other_time: 23.6889648438
epoch time: 129.3146057129s, input_fired_time: 1.5535564423, output_fired_time: 0.0000000000, propagate_forward_time: 124.3100051880, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4510421753
validation[          , genome  6254] predictions:    6186/  10000 (61.86%), best:    6229/10000 (62.29%), error:     11926.12402, best error:     11698.15332 on epoch:    70, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 1579.5230712891s, input_fired_time: 11.3223953247, output_fired_time: 8.5413312912, propagate_forward_time: 577.9097900391, propagate_backward_time: 960.7914428711, weight_update_time: 1.3351932764, other_time: 20.9580688477
epoch time: 122.1657943726s, input_fired_time: 1.4959315062, output_fired_time: 0.0000000000, propagate_forward_time: 117.3181457520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3517150879
validation[          , genome  6254] predictions:    6249/  10000 (62.49%), best:    6229/10000 (62.29%), error:     11842.58691, best error:     11698.15332 on epoch:    70, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 1579.6368408203s, input_fired_time: 12.2457132339, output_fired_time: 8.4928808212, propagate_forward_time: 573.9138793945, propagate_backward_time: 964.3652954102, weight_update_time: 1.3629530668, other_time: 20.6190185547
epoch time: 121.7942962646s, input_fired_time: 1.5144262314, output_fired_time: 0.0000000000, propagate_forward_time: 116.9767837524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3030853271
validation[          , genome  6254] predictions:    6214/  10000 (62.14%), best:    6229/10000 (62.29%), error:     11997.15039, best error:     11698.15332 on epoch:    70, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 1592.1708984375s, input_fired_time: 13.3136386871, output_fired_time: 8.5898876190, propagate_forward_time: 586.9529418945, propagate_backward_time: 961.3720703125, weight_update_time: 3.3607215881, other_time: 21.9424438477
epoch time: 120.4151382446s, input_fired_time: 1.5081061125, output_fired_time: 0.0000000000, propagate_forward_time: 115.5530471802, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3539886475
validation[          , genome  6254] predictions:    6280/  10000 (62.80%), best:    6280/10000 (62.80%), error:     11592.97656, best error:     11592.97656 on epoch:    78, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1607.9885253906s, input_fired_time: 13.4830245972, output_fired_time: 8.7898693085, propagate_forward_time: 569.6460571289, propagate_backward_time: 994.1409912109, weight_update_time: 1.3566927910, other_time: 21.9285278320
epoch time: 122.3484191895s, input_fired_time: 1.4999785423, output_fired_time: 0.0000000000, propagate_forward_time: 117.4355010986, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4129409790
validation[          , genome  6254] predictions:    6185/  10000 (61.85%), best:    6280/10000 (62.80%), error:     12055.41211, best error:     11592.97656 on epoch:    78, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1600.5162353516s, input_fired_time: 11.4204483032, output_fired_time: 9.5774431229, propagate_forward_time: 584.4633789062, propagate_backward_time: 973.9133911133, weight_update_time: 1.3530135155, other_time: 21.1416625977
epoch time: 122.1024932861s, input_fired_time: 1.5083318949, output_fired_time: 0.0000000000, propagate_forward_time: 116.3128890991, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2812728882
validation[          , genome  6254] predictions:    6147/  10000 (61.47%), best:    6280/10000 (62.80%), error:     12007.42383, best error:     11592.97656 on epoch:    78, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1601.7492675781s, input_fired_time: 13.4784307480, output_fired_time: 9.5881538391, propagate_forward_time: 588.9075927734, propagate_backward_time: 970.8270263672, weight_update_time: 1.3466554880, other_time: 18.9481201172
epoch time: 121.7912826538s, input_fired_time: 1.5600517988, output_fired_time: 0.0000000000, propagate_forward_time: 116.8967285156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3345031738
validation[          , genome  6254] predictions:    6198/  10000 (61.98%), best:    6280/10000 (62.80%), error:     11987.50391, best error:     11592.97656 on epoch:    78, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1599.6446533203s, input_fired_time: 12.3599586487, output_fired_time: 8.5801668167, propagate_forward_time: 588.8373413086, propagate_backward_time: 968.7419433594, weight_update_time: 1.3471316099, other_time: 21.1251831055
epoch time: 124.5177230835s, input_fired_time: 1.4923020601, output_fired_time: 0.0000000000, propagate_forward_time: 119.7000732422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3253479004
validation[          , genome  6254] predictions:    6263/  10000 (62.63%), best:    6280/10000 (62.80%), error:     11652.28320, best error:     11592.97656 on epoch:    78, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1593.6735839844s, input_fired_time: 11.3564472198, output_fired_time: 9.4969673157, propagate_forward_time: 582.4664916992, propagate_backward_time: 970.2426757812, weight_update_time: 1.3613339663, other_time: 20.1110229492
epoch time: 120.7423095703s, input_fired_time: 2.4979970455, output_fired_time: 0.0000000000, propagate_forward_time: 114.9724502563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2718658447
validation[          , genome  6254] predictions:    6213/  10000 (62.13%), best:    6280/10000 (62.80%), error:     12001.57617, best error:     11592.97656 on epoch:    78, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1596.8111572266s, input_fired_time: 14.6162662506, output_fired_time: 8.8808803558, propagate_forward_time: 582.9144287109, propagate_backward_time: 966.7529907227, weight_update_time: 1.4030294418, other_time: 23.6466674805
epoch time: 125.0905380249s, input_fired_time: 1.6976398230, output_fired_time: 0.0000000000, propagate_forward_time: 119.8880386353, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5048599243
validation[          , genome  6254] predictions:    6213/  10000 (62.13%), best:    6280/10000 (62.80%), error:     11942.59570, best error:     11592.97656 on epoch:    78, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 1628.7718505859s, input_fired_time: 12.0443019867, output_fired_time: 10.2260437012, propagate_forward_time: 581.9968261719, propagate_backward_time: 1000.2202148438, weight_update_time: 1.4463089705, other_time: 24.2844238281
epoch time: 124.9457626343s, input_fired_time: 1.6595356464, output_fired_time: 0.0000000000, propagate_forward_time: 119.8106155396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4756088257
validation[          , genome  6254] predictions:    6198/  10000 (61.98%), best:    6280/10000 (62.80%), error:     11989.33594, best error:     11592.97656 on epoch:    78, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1630.5603027344s, input_fired_time: 14.0395212173, output_fired_time: 12.2142343521, propagate_forward_time: 594.6470336914, propagate_backward_time: 987.3724975586, weight_update_time: 1.4615136385, other_time: 22.2869873047
epoch time: 130.5781097412s, input_fired_time: 1.7410229445, output_fired_time: 0.0000000000, propagate_forward_time: 125.2210083008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6160736084
validation[          , genome  6254] predictions:    6236/  10000 (62.36%), best:    6280/10000 (62.80%), error:     11833.31055, best error:     11592.97656 on epoch:    78, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1639.6979980469s, input_fired_time: 13.1686077118, output_fired_time: 9.0647239685, propagate_forward_time: 588.6875610352, propagate_backward_time: 1008.5708618164, weight_update_time: 1.4271601439, other_time: 20.2062377930
epoch time: 124.6216278076s, input_fired_time: 1.6818397045, output_fired_time: 0.0000000000, propagate_forward_time: 118.4208984375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5188903809
validation[          , genome  6254] predictions:    6242/  10000 (62.42%), best:    6280/10000 (62.80%), error:     11723.73828, best error:     11592.97656 on epoch:    78, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1650.9086914062s, input_fired_time: 13.3666591644, output_fired_time: 10.2755975723, propagate_forward_time: 596.5694580078, propagate_backward_time: 1007.2641601562, weight_update_time: 2.4105136395, other_time: 23.4327392578
epoch time: 125.1307067871s, input_fired_time: 1.6328607798, output_fired_time: 0.0000000000, propagate_forward_time: 119.8354873657, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6623611450
validation[          , genome  6254] predictions:    6240/  10000 (62.40%), best:    6280/10000 (62.80%), error:     11785.24609, best error:     11592.97656 on epoch:    78, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1647.0240478516s, input_fired_time: 13.3185291290, output_fired_time: 9.2826566696, propagate_forward_time: 601.3770141602, propagate_backward_time: 1001.4302978516, weight_update_time: 1.4938414097, other_time: 21.6155395508
epoch time: 124.5093078613s, input_fired_time: 1.7204277515, output_fired_time: 0.0000000000, propagate_forward_time: 118.3001251221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4887542725
validation[          , genome  6254] predictions:    6252/  10000 (62.52%), best:    6280/10000 (62.80%), error:     11873.28906, best error:     11592.97656 on epoch:    78, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1645.9200439453s, input_fired_time: 13.1495151520, output_fired_time: 10.1506013870, propagate_forward_time: 595.8936767578, propagate_backward_time: 1005.2813110352, weight_update_time: 1.4491610527, other_time: 21.4448852539
epoch time: 125.4847412109s, input_fired_time: 2.6921904087, output_fired_time: 0.0000000000, propagate_forward_time: 118.2826080322, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5099411011
validation[          , genome  6254] predictions:    6190/  10000 (61.90%), best:    6280/10000 (62.80%), error:     11871.31348, best error:     11592.97656 on epoch:    78, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1640.8088378906s, input_fired_time: 15.1739845276, output_fired_time: 9.2358083725, propagate_forward_time: 588.4666137695, propagate_backward_time: 1007.5119628906, weight_update_time: 1.4873458147, other_time: 20.4204101562
epoch time: 121.2899169922s, input_fired_time: 1.6585538387, output_fired_time: 0.0000000000, propagate_forward_time: 116.0817642212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5495986938
validation[          , genome  6254] predictions:    6260/  10000 (62.60%), best:    6280/10000 (62.80%), error:     11756.59277, best error:     11592.97656 on epoch:    78, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1638.6665039062s, input_fired_time: 12.1818532944, output_fired_time: 9.1677169800, propagate_forward_time: 595.7577514648, propagate_backward_time: 999.2596435547, weight_update_time: 1.4433826208, other_time: 22.2994995117
epoch time: 125.5901641846s, input_fired_time: 1.6441264153, output_fired_time: 0.0000000000, propagate_forward_time: 120.3516082764, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5944290161
validation[          , genome  6254] predictions:    6254/  10000 (62.54%), best:    6280/10000 (62.80%), error:     11668.89551, best error:     11592.97656 on epoch:    78, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1640.2100830078s, input_fired_time: 12.1971902847, output_fired_time: 10.2066316605, propagate_forward_time: 590.7126464844, propagate_backward_time: 1003.4996948242, weight_update_time: 1.4374158382, other_time: 23.5939331055
epoch time: 125.6332168579s, input_fired_time: 1.6064367294, output_fired_time: 0.0000000000, propagate_forward_time: 120.4030685425, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6237106323
validation[          , genome  6254] predictions:    6206/  10000 (62.06%), best:    6280/10000 (62.80%), error:     12010.91309, best error:     11592.97656 on epoch:    78, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1625.5186767578s, input_fired_time: 12.1012248993, output_fired_time: 10.2419471741, propagate_forward_time: 594.5299072266, propagate_backward_time: 987.3092651367, weight_update_time: 2.4437983036, other_time: 21.3363647461
epoch time: 124.3979415894s, input_fired_time: 1.6254949570, output_fired_time: 0.0000000000, propagate_forward_time: 119.2263793945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5460662842
validation[          , genome  6254] predictions:    6214/  10000 (62.14%), best:    6280/10000 (62.80%), error:     11975.88770, best error:     11592.97656 on epoch:    78, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1629.7939453125s, input_fired_time: 13.0509948730, output_fired_time: 10.1411199570, propagate_forward_time: 592.5758666992, propagate_backward_time: 992.7281494141, weight_update_time: 1.4722896814, other_time: 21.2977905273
epoch time: 124.6736831665s, input_fired_time: 2.6993961334, output_fired_time: 0.0000000000, propagate_forward_time: 118.3918762207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5824127197
validation[          , genome  6254] predictions:    6236/  10000 (62.36%), best:    6280/10000 (62.80%), error:     11852.50488, best error:     11592.97656 on epoch:    78, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1639.5533447266s, input_fired_time: 12.2896890640, output_fired_time: 9.1839838028, propagate_forward_time: 596.5910034180, propagate_backward_time: 999.9985351562, weight_update_time: 1.4741690159, other_time: 21.4901733398
epoch time: 123.2089309692s, input_fired_time: 2.6422128677, output_fired_time: 0.0000000000, propagate_forward_time: 117.0275955200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5391235352
validation[          , genome  6254] predictions:    6243/  10000 (62.43%), best:    6280/10000 (62.80%), error:     11858.15430, best error:     11592.97656 on epoch:    78, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1644.3043212891s, input_fired_time: 12.0993452072, output_fired_time: 9.1667118073, propagate_forward_time: 601.0011596680, propagate_backward_time: 1000.5253295898, weight_update_time: 1.4606330395, other_time: 21.5117187500
epoch time: 129.1992797852s, input_fired_time: 1.7188715935, output_fired_time: 0.0000000000, propagate_forward_time: 123.8645553589, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6158523560
validation[          , genome  6254] predictions:    6232/  10000 (62.32%), best:    6280/10000 (62.80%), error:     11917.07715, best error:     11592.97656 on epoch:    78, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1638.0682373047s, input_fired_time: 12.2805576324, output_fired_time: 13.2725095749, propagate_forward_time: 580.8719482422, propagate_backward_time: 1007.2912597656, weight_update_time: 1.4529531002, other_time: 24.3520507812
epoch time: 123.4082870483s, input_fired_time: 1.6660276651, output_fired_time: 0.0000000000, propagate_forward_time: 118.2009429932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5413131714
validation[          , genome  6254] predictions:    6228/  10000 (62.28%), best:    6280/10000 (62.80%), error:     11821.50977, best error:     11592.97656 on epoch:    78, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1642.3581542969s, input_fired_time: 15.2528524399, output_fired_time: 9.3692598343, propagate_forward_time: 602.8139038086, propagate_backward_time: 991.3562622070, weight_update_time: 1.4831992388, other_time: 23.5659179688
epoch time: 122.7703399658s, input_fired_time: 1.5918948650, output_fired_time: 0.0000000000, propagate_forward_time: 117.6672668457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5111770630
validation[          , genome  6254] predictions:    6252/  10000 (62.52%), best:    6280/10000 (62.80%), error:     11797.16016, best error:     11592.97656 on epoch:    78, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1632.3117675781s, input_fired_time: 14.1646842957, output_fired_time: 10.1937599182, propagate_forward_time: 591.8537597656, propagate_backward_time: 993.5183715820, weight_update_time: 1.4906957150, other_time: 22.5812377930
epoch time: 124.6941604614s, input_fired_time: 1.7276202440, output_fired_time: 0.0000000000, propagate_forward_time: 119.4216613770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5448760986
validation[          , genome  6254] predictions:    6251/  10000 (62.51%), best:    6280/10000 (62.80%), error:     11805.12402, best error:     11592.97656 on epoch:    78, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 626.3314819336s, input_fired_time: 8.3143854141, output_fired_time: 0.0000000000, propagate_forward_time: 598.2377319336, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.7793579102
best training[          , genome  6254] predictions:   41604/  50000 (83.21%), best:    6280/10000 (62.80%), error:     26569.51758, best error:     11592.97656 on epoch:    78, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 122.2354812622s, input_fired_time: 1.6103107929, output_fired_time: 0.0000000000, propagate_forward_time: 117.2476577759, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3775100708
   testing[          , genome  6254] predictions:    6280/  10000 (62.80%), best:    6280/10000 (62.80%), error:     11592.97656, best error:     11592.97656 on epoch:    78, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
11:55:04 (27753): called boinc_finish(0)

</stderr_txt>
]]>