Nameexact_genome_1542069603_184_5799_2
Workunit4086282
Created1 Mar 2019, 17:55:34 UTC
Sent1 Mar 2019, 18:06:00 UTC
Report deadline16 Jun 2019, 1:15:36 UTC
Received3 Mar 2019, 13:33:10 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID84860
Run time1 days 18 hours 58 min 34 sec
CPU time1 days 15 hours 27 min 12 sec
Validate stateValid
Credit26,574.58
Device peak FLOPS4.84 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size516.98 MB
Peak swap size519.72 MB
Peak disk usage5.43 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
145760, propagate_forward_time: 490.2361145020, propagate_backward_time: 827.5215454102, weight_update_time: 0.6771737933, other_time: 9.1958618164
epoch time: 101.1959075928s, input_fired_time: 0.9350649714, output_fired_time: 0.0000000000, propagate_forward_time: 98.5842514038, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6765899658
validation[          , genome  5799] predictions:    5627/  10000 (56.27%), best:    5627/10000 (56.27%), error:     13431.67773, best error:     13431.67773 on epoch:    21, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 1297.4930419922s, input_fired_time: 9.1474084854, output_fired_time: 7.1591711044, propagate_forward_time: 476.4169311523, propagate_backward_time: 795.9783935547, weight_update_time: 0.6517384052, other_time: 8.7910766602
epoch time: 101.9008483887s, input_fired_time: 0.9429413676, output_fired_time: 0.0000000000, propagate_forward_time: 99.3012619019, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6566467285
validation[          , genome  5799] predictions:    5517/  10000 (55.17%), best:    5627/10000 (56.27%), error:     14325.82812, best error:     13431.67773 on epoch:    21, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1327.9575195312s, input_fired_time: 9.2116498947, output_fired_time: 7.3552165031, propagate_forward_time: 484.8906860352, propagate_backward_time: 817.5040283203, weight_update_time: 0.6627123952, other_time: 8.9959106445
epoch time: 96.7807083130s, input_fired_time: 0.9212921262, output_fired_time: 0.0000000000, propagate_forward_time: 94.2909698486, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5684432983
validation[          , genome  5799] predictions:    5586/  10000 (55.86%), best:    5627/10000 (56.27%), error:     13707.74121, best error:     13431.67773 on epoch:    21, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 1292.5139160156s, input_fired_time: 9.1372556686, output_fired_time: 7.1002230644, propagate_forward_time: 474.2078247070, propagate_backward_time: 793.2947998047, weight_update_time: 0.6539453864, other_time: 8.7738647461
epoch time: 92.0142517090s, input_fired_time: 0.9302357435, output_fired_time: 0.0000000000, propagate_forward_time: 89.6342620850, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4497528076
validation[          , genome  5799] predictions:    5618/  10000 (56.18%), best:    5627/10000 (56.27%), error:     13568.47852, best error:     13431.67773 on epoch:    21, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 1260.8338623047s, input_fired_time: 9.0815067291, output_fired_time: 6.9288482666, propagate_forward_time: 462.8196105957, propagate_backward_time: 773.4545898438, weight_update_time: 0.6349569559, other_time: 8.5493164062
epoch time: 96.7657470703s, input_fired_time: 0.9522849321, output_fired_time: 0.0000000000, propagate_forward_time: 94.3780899048, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4353713989
validation[          , genome  5799] predictions:    5600/  10000 (56.00%), best:    5627/10000 (56.27%), error:     13972.25098, best error:     13431.67773 on epoch:    21, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 1295.7967529297s, input_fired_time: 9.1638307571, output_fired_time: 7.1550049782, propagate_forward_time: 475.1672058105, propagate_backward_time: 795.5970458984, weight_update_time: 0.6482506394, other_time: 8.7136230469
epoch time: 93.5519943237s, input_fired_time: 0.9441797733, output_fired_time: 0.0000000000, propagate_forward_time: 91.1407852173, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4670257568
validation[          , genome  5799] predictions:    5755/  10000 (57.55%), best:    5755/10000 (57.55%), error:     13026.59082, best error:     13026.59082 on epoch:    26, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 1304.8784179688s, input_fired_time: 9.1822948456, output_fired_time: 7.2088780403, propagate_forward_time: 476.6205139160, propagate_backward_time: 802.9899902344, weight_update_time: 0.6592806578, other_time: 8.8768310547
epoch time: 96.2043457031s, input_fired_time: 0.9278147817, output_fired_time: 0.0000000000, propagate_forward_time: 93.7251739502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5513534546
validation[          , genome  5799] predictions:    5535/  10000 (55.35%), best:    5755/10000 (57.55%), error:     14050.12598, best error:     13026.59082 on epoch:    26, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 1329.9837646484s, input_fired_time: 9.2789421082, output_fired_time: 7.4096417427, propagate_forward_time: 483.0642700195, propagate_backward_time: 821.1830444336, weight_update_time: 0.6702016592, other_time: 9.0478515625
epoch time: 101.5673446655s, input_fired_time: 0.9497863650, output_fired_time: 0.0000000000, propagate_forward_time: 98.9377899170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6797714233
validation[          , genome  5799] predictions:    5679/  10000 (56.79%), best:    5755/10000 (57.55%), error:     13517.90430, best error:     13026.59082 on epoch:    26, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 1351.4936523438s, input_fired_time: 9.3574733734, output_fired_time: 7.5484900475, propagate_forward_time: 490.3906555176, propagate_backward_time: 835.0102539062, weight_update_time: 0.6729617119, other_time: 9.1868896484
epoch time: 101.5175552368s, input_fired_time: 0.9606035352, output_fired_time: 0.0000000000, propagate_forward_time: 98.8724670410, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6844863892
validation[          , genome  5799] predictions:    5746/  10000 (57.46%), best:    5746/10000 (57.46%), error:     12926.24707, best error:     12926.24707 on epoch:    29, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1289.3100585938s, input_fired_time: 9.2275581360, output_fired_time: 7.1286602020, propagate_forward_time: 473.0997314453, propagate_backward_time: 791.1840820312, weight_update_time: 0.6505789757, other_time: 8.6700439453
epoch time: 97.3278961182s, input_fired_time: 0.9577774405, output_fired_time: 0.0000000000, propagate_forward_time: 94.7827987671, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5873184204
validation[          , genome  5799] predictions:    5510/  10000 (55.10%), best:    5746/10000 (57.46%), error:     13997.31934, best error:     12926.24707 on epoch:    29, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 1361.9580078125s, input_fired_time: 9.3956661224, output_fired_time: 7.5968070030, propagate_forward_time: 493.8031921387, propagate_backward_time: 841.9435424805, weight_update_time: 0.6742913127, other_time: 9.2188110352
epoch time: 101.0146865845s, input_fired_time: 0.9673005342, output_fired_time: 0.0000000000, propagate_forward_time: 98.3801040649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6672821045
validation[          , genome  5799] predictions:    5901/  10000 (59.01%), best:    5901/10000 (59.01%), error:     12323.15918, best error:     12323.15918 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 1315.7290039062s, input_fired_time: 9.3201808929, output_fired_time: 7.2872772217, propagate_forward_time: 479.6526489258, propagate_backward_time: 810.5563354492, weight_update_time: 0.6605650783, other_time: 8.9125976562
epoch time: 101.4376449585s, input_fired_time: 0.9486618638, output_fired_time: 0.0000000000, propagate_forward_time: 98.8150863647, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6738967896
validation[          , genome  5799] predictions:    5686/  10000 (56.86%), best:    5901/10000 (59.01%), error:     13144.65723, best error:     12323.15918 on epoch:    31, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 1288.1499023438s, input_fired_time: 9.2165679932, output_fired_time: 7.1064496040, propagate_forward_time: 473.2123413086, propagate_backward_time: 789.9323730469, weight_update_time: 0.6516202688, other_time: 8.6821899414
epoch time: 101.7741928101s, input_fired_time: 0.9668090343, output_fired_time: 0.0000000000, propagate_forward_time: 99.1254272461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6819534302
validation[          , genome  5799] predictions:    5693/  10000 (56.93%), best:    5901/10000 (59.01%), error:     13264.45508, best error:     12323.15918 on epoch:    31, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 1310.3214111328s, input_fired_time: 9.2715988159, output_fired_time: 7.2377896309, propagate_forward_time: 478.9997253418, propagate_backward_time: 806.0156250000, weight_update_time: 0.6525729895, other_time: 8.7966308594
epoch time: 101.2502822876s, input_fired_time: 0.9678040147, output_fired_time: 0.0000000000, propagate_forward_time: 98.6295394897, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6529388428
validation[          , genome  5799] predictions:    5768/  10000 (57.68%), best:    5901/10000 (59.01%), error:     13046.25977, best error:     12323.15918 on epoch:    31, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1337.3081054688s, input_fired_time: 9.3806591034, output_fired_time: 7.4171304703, propagate_forward_time: 487.6273193359, propagate_backward_time: 823.7979736328, weight_update_time: 0.6746851802, other_time: 9.0850830078
epoch time: 91.3989105225s, input_fired_time: 0.9583075047, output_fired_time: 0.0000000000, propagate_forward_time: 89.1280517578, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3125534058
validation[          , genome  5799] predictions:    5760/  10000 (57.60%), best:    5901/10000 (59.01%), error:     13041.60742, best error:     12323.15918 on epoch:    31, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1239.8242187500s, input_fired_time: 9.1314382553, output_fired_time: 6.7746853828, propagate_forward_time: 459.8143310547, propagate_backward_time: 755.8615112305, weight_update_time: 0.6235770583, other_time: 8.2422485352
epoch time: 101.7180023193s, input_fired_time: 0.9739124775, output_fired_time: 0.0000000000, propagate_forward_time: 99.0635452271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6805419922
validation[          , genome  5799] predictions:    5840/  10000 (58.40%), best:    5901/10000 (59.01%), error:     12670.95410, best error:     12323.15918 on epoch:    31, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1318.2365722656s, input_fired_time: 9.3549594879, output_fired_time: 7.3013162613, propagate_forward_time: 482.3342895508, propagate_backward_time: 810.3944091797, weight_update_time: 0.6619100571, other_time: 8.8516235352
epoch time: 100.3156204224s, input_fired_time: 0.9762707949, output_fired_time: 0.0000000000, propagate_forward_time: 97.6912307739, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6481170654
validation[          , genome  5799] predictions:    5918/  10000 (59.18%), best:    5901/10000 (59.01%), error:     12555.94629, best error:     12323.15918 on epoch:    31, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1308.2437744141s, input_fired_time: 9.3153162003, output_fired_time: 7.2214021683, propagate_forward_time: 477.3089904785, propagate_backward_time: 805.5559692383, weight_update_time: 0.6504841447, other_time: 8.8421020508
epoch time: 101.5501708984s, input_fired_time: 0.9682323933, output_fired_time: 0.0000000000, propagate_forward_time: 98.9514694214, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6304702759
validation[          , genome  5799] predictions:    5886/  10000 (58.86%), best:    5901/10000 (59.01%), error:     12830.60742, best error:     12323.15918 on epoch:    31, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 1339.2701416016s, input_fired_time: 9.4176025391, output_fired_time: 7.4324197769, propagate_forward_time: 487.9858093262, propagate_backward_time: 825.4085693359, weight_update_time: 0.6667359471, other_time: 9.0257568359
epoch time: 100.8061599731s, input_fired_time: 0.9735509753, output_fired_time: 0.0000000000, propagate_forward_time: 98.1763534546, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6562576294
validation[          , genome  5799] predictions:    5880/  10000 (58.80%), best:    5901/10000 (59.01%), error:     12954.08691, best error:     12323.15918 on epoch:    31, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1296.1345214844s, input_fired_time: 9.3197736740, output_fired_time: 7.1450023651, propagate_forward_time: 474.5043945312, propagate_backward_time: 796.4323120117, weight_update_time: 0.6539415717, other_time: 8.7329711914
epoch time: 102.4125518799s, input_fired_time: 0.9985499382, output_fired_time: 0.0000000000, propagate_forward_time: 99.7845535278, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6294479370
validation[          , genome  5799] predictions:    5902/  10000 (59.02%), best:    5901/10000 (59.01%), error:     12646.33594, best error:     12323.15918 on epoch:    31, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1320.0953369141s, input_fired_time: 9.3970069885, output_fired_time: 7.2758188248, propagate_forward_time: 483.2477722168, propagate_backward_time: 811.3093261719, weight_update_time: 0.6604082584, other_time: 8.8653564453
epoch time: 95.1311111450s, input_fired_time: 0.9665463567, output_fired_time: 0.0000000000, propagate_forward_time: 92.6794052124, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4851608276
validation[          , genome  5799] predictions:    5791/  10000 (57.91%), best:    5901/10000 (59.01%), error:     13037.96289, best error:     12323.15918 on epoch:    31, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1310.8243408203s, input_fired_time: 9.4011754990, output_fired_time: 7.2401385307, propagate_forward_time: 479.2059326172, propagate_backward_time: 806.1351318359, weight_update_time: 0.6564011574, other_time: 8.8420410156
epoch time: 97.4642791748s, input_fired_time: 0.9806892872, output_fired_time: 0.0000000000, propagate_forward_time: 94.9165039062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5670852661
validation[          , genome  5799] predictions:    5961/  10000 (59.61%), best:    5901/10000 (59.01%), error:     12333.39355, best error:     12323.15918 on epoch:    31, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1290.1458740234s, input_fired_time: 9.3321018219, output_fired_time: 7.0757608414, propagate_forward_time: 471.6748352051, propagate_backward_time: 793.3908691406, weight_update_time: 0.6502010226, other_time: 8.6722412109
epoch time: 101.5487976074s, input_fired_time: 0.9841076732, output_fired_time: 0.0000000000, propagate_forward_time: 98.8821563721, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6825332642
validation[          , genome  5799] predictions:    5913/  10000 (59.13%), best:    5913/10000 (59.13%), error:     12216.21484, best error:     12216.21484 on epoch:    43, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1352.5206298828s, input_fired_time: 9.5147895813, output_fired_time: 7.5425500870, propagate_forward_time: 490.0928039551, propagate_backward_time: 836.1329345703, weight_update_time: 0.6764882803, other_time: 9.2375488281
epoch time: 101.5484619141s, input_fired_time: 0.9882250428, output_fired_time: 0.0000000000, propagate_forward_time: 98.8787918091, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6814422607
validation[          , genome  5799] predictions:    5951/  10000 (59.51%), best:    5913/10000 (59.13%), error:     12644.00684, best error:     12216.21484 on epoch:    43, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1330.9367675781s, input_fired_time: 9.4794559479, output_fired_time: 7.4243211746, propagate_forward_time: 485.0369567871, propagate_backward_time: 819.9495239258, weight_update_time: 0.6700745225, other_time: 9.0464477539
epoch time: 77.2115173340s, input_fired_time: 0.9345792532, output_fired_time: 0.0000000000, propagate_forward_time: 75.0089569092, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2679824829
validation[          , genome  5799] predictions:    5892/  10000 (58.92%), best:    5913/10000 (59.13%), error:     12393.97363, best error:     12216.21484 on epoch:    43, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1339.8814697266s, input_fired_time: 9.5022077560, output_fired_time: 7.4639754295, propagate_forward_time: 486.5962524414, propagate_backward_time: 827.2078247070, weight_update_time: 0.6684125066, other_time: 9.1112060547
epoch time: 101.4311141968s, input_fired_time: 0.9937759638, output_fired_time: 0.0000000000, propagate_forward_time: 98.7550048828, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6823348999
validation[          , genome  5799] predictions:    5928/  10000 (59.28%), best:    5913/10000 (59.13%), error:     12737.92188, best error:     12216.21484 on epoch:    43, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1356.2597656250s, input_fired_time: 9.5552148819, output_fired_time: 7.5553345680, propagate_forward_time: 492.2282714844, propagate_backward_time: 837.7589111328, weight_update_time: 0.6722083688, other_time: 9.1621093750
epoch time: 99.5973587036s, input_fired_time: 0.9925143719, output_fired_time: 0.0000000000, propagate_forward_time: 96.9886932373, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6161499023
validation[          , genome  5799] predictions:    5993/  10000 (59.93%), best:    5913/10000 (59.13%), error:     12434.04297, best error:     12216.21484 on epoch:    43, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1361.8723144531s, input_fired_time: 9.7893419266, output_fired_time: 7.4848098755, propagate_forward_time: 500.5503234863, propagate_backward_time: 834.7476806641, weight_update_time: 0.7030616999, other_time: 9.3001708984
epoch time: 105.3035888672s, input_fired_time: 1.0719251633, output_fired_time: 0.0000000000, propagate_forward_time: 102.5008621216, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7308044434
validation[          , genome  5799] predictions:    6042/  10000 (60.42%), best:    6042/10000 (60.42%), error:     11923.25293, best error:     11923.25293 on epoch:    48, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 1364.6347656250s, input_fired_time: 9.7924566269, output_fired_time: 7.5456790924, propagate_forward_time: 500.7143249512, propagate_backward_time: 837.2914428711, weight_update_time: 0.6779062152, other_time: 9.2908325195
epoch time: 96.7299194336s, input_fired_time: 1.0105730295, output_fired_time: 0.0000000000, propagate_forward_time: 94.0998916626, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6194534302
validation[          , genome  5799] predictions:    5874/  10000 (58.74%), best:    6042/10000 (60.42%), error:     12483.21191, best error:     11923.25293 on epoch:    48, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1346.6273193359s, input_fired_time: 9.6762704849, output_fired_time: 7.4281153679, propagate_forward_time: 494.9171752930, propagate_backward_time: 825.3814697266, weight_update_time: 0.6736384630, other_time: 9.2243041992
epoch time: 99.4389266968s, input_fired_time: 1.0000705719, output_fired_time: 0.0000000000, propagate_forward_time: 96.7886276245, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6502304077
validation[          , genome  5799] predictions:    5943/  10000 (59.43%), best:    6042/10000 (60.42%), error:     12212.86719, best error:     11923.25293 on epoch:    48, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 1328.8070068359s, input_fired_time: 9.6691598892, output_fired_time: 7.3389883041, propagate_forward_time: 486.4084167480, propagate_backward_time: 816.2467651367, weight_update_time: 0.6782863140, other_time: 9.1436157227
epoch time: 93.7716674805s, input_fired_time: 1.0117117167, output_fired_time: 0.0000000000, propagate_forward_time: 91.2250137329, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5349426270
validation[          , genome  5799] predictions:    5983/  10000 (59.83%), best:    6042/10000 (60.42%), error:     12354.24121, best error:     11923.25293 on epoch:    48, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 1344.3344726562s, input_fired_time: 9.7781925201, output_fired_time: 7.4511132240, propagate_forward_time: 492.8663024902, propagate_backward_time: 825.0501708984, weight_update_time: 0.6954996586, other_time: 9.1885986328
epoch time: 102.8152008057s, input_fired_time: 1.0377815962, output_fired_time: 0.0000000000, propagate_forward_time: 100.1021118164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6753082275
validation[          , genome  5799] predictions:    5965/  10000 (59.65%), best:    6042/10000 (60.42%), error:     12356.37598, best error:     11923.25293 on epoch:    48, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 1354.5266113281s, input_fired_time: 9.8455066681, output_fired_time: 7.4997844696, propagate_forward_time: 494.7841491699, propagate_backward_time: 833.1755371094, weight_update_time: 0.6880542636, other_time: 9.2216796875
epoch time: 101.0110321045s, input_fired_time: 0.9856413603, output_fired_time: 0.0000000000, propagate_forward_time: 98.3625259399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6628646851
validation[          , genome  5799] predictions:    5958/  10000 (59.58%), best:    6042/10000 (60.42%), error:     12498.02344, best error:     11923.25293 on epoch:    48, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 1673.3443603516s, input_fired_time: 12.5487098694, output_fired_time: 9.1513652802, propagate_forward_time: 623.4004516602, propagate_backward_time: 1015.9729003906, weight_update_time: 0.9614178538, other_time: 12.2709960938
epoch time: 191.5583953857s, input_fired_time: 2.1096851826, output_fired_time: 0.0000000000, propagate_forward_time: 186.2970428467, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1516723633
validation[          , genome  5799] predictions:    6093/  10000 (60.93%), best:    6042/10000 (60.42%), error:     12068.97656, best error:     11923.25293 on epoch:    48, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 2164.1442871094s, input_fired_time: 16.6091823578, output_fired_time: 11.8466691971, propagate_forward_time: 826.5622558594, propagate_backward_time: 1292.9603271484, weight_update_time: 1.2442969084, other_time: 16.1658935547
epoch time: 198.3961486816s, input_fired_time: 2.1401851177, output_fired_time: 0.0000000000, propagate_forward_time: 192.7917480469, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4642181396
validation[          , genome  5799] predictions:    5905/  10000 (59.05%), best:    6042/10000 (60.42%), error:     12730.78418, best error:     11923.25293 on epoch:    48, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 1623.5689697266s, input_fired_time: 11.8075647354, output_fired_time: 8.9674968719, propagate_forward_time: 603.6743164062, propagate_backward_time: 987.8296508789, weight_update_time: 0.8190808296, other_time: 11.2898559570
epoch time: 97.8065490723s, input_fired_time: 1.0045082569, output_fired_time: 0.0000000000, propagate_forward_time: 95.2256469727, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5763931274
validation[          , genome  5799] predictions:    6004/  10000 (60.04%), best:    6042/10000 (60.42%), error:     12373.26758, best error:     11923.25293 on epoch:    48, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 1274.0191650391s, input_fired_time: 9.5015869141, output_fired_time: 6.9273777008, propagate_forward_time: 480.2868347168, propagate_backward_time: 769.0021972656, weight_update_time: 0.6483107805, other_time: 8.3011474609
epoch time: 101.0485458374s, input_fired_time: 1.0165880919, output_fired_time: 0.0000000000, propagate_forward_time: 98.3809204102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6510391235
validation[          , genome  5799] predictions:    6028/  10000 (60.28%), best:    6042/10000 (60.42%), error:     12095.03320, best error:     11923.25293 on epoch:    48, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1411.3365478516s, input_fired_time: 10.0737800598, output_fired_time: 7.6067528725, propagate_forward_time: 532.3131103516, propagate_backward_time: 852.2026977539, weight_update_time: 0.7002349496, other_time: 9.1401977539
epoch time: 136.1582946777s, input_fired_time: 1.4321709871, output_fired_time: 0.0000000000, propagate_forward_time: 132.3924560547, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3336639404
validation[          , genome  5799] predictions:    6093/  10000 (60.93%), best:    6042/10000 (60.42%), error:     12207.91602, best error:     11923.25293 on epoch:    48, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 2433.7851562500s, input_fired_time: 19.0375709534, output_fired_time: 13.0617179871, propagate_forward_time: 929.7492065430, propagate_backward_time: 1453.5039062500, weight_update_time: 1.4615192413, other_time: 18.4326171875
epoch time: 187.1319885254s, input_fired_time: 2.0716974735, output_fired_time: 0.0000000000, propagate_forward_time: 181.9952697754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0650177002
validation[          , genome  5799] predictions:    6031/  10000 (60.31%), best:    6042/10000 (60.42%), error:     12242.78320, best error:     11923.25293 on epoch:    48, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 2536.5969238281s, input_fired_time: 19.6502475739, output_fired_time: 13.7805738449, propagate_forward_time: 974.2891235352, propagate_backward_time: 1509.4520263672, weight_update_time: 1.4275311232, other_time: 19.4251708984
epoch time: 216.6085205078s, input_fired_time: 2.4348444939, output_fired_time: 0.0000000000, propagate_forward_time: 210.4998931885, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6737823486
validation[          , genome  5799] predictions:    6151/  10000 (61.51%), best:    6151/10000 (61.51%), error:     11746.73242, best error:     11746.73242 on epoch:    60, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 1668.3302001953s, input_fired_time: 12.1296463013, output_fired_time: 9.0220985413, propagate_forward_time: 630.5647583008, propagate_backward_time: 1005.2982788086, weight_update_time: 0.8789027929, other_time: 11.3154296875
epoch time: 104.1189956665s, input_fired_time: 1.0662750006, output_fired_time: 0.0000000000, propagate_forward_time: 101.4273452759, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6253738403
validation[          , genome  5799] predictions:    6125/  10000 (61.25%), best:    6151/10000 (61.51%), error:     11960.74219, best error:     11746.73242 on epoch:    60, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 2417.1203613281s, input_fired_time: 18.4984683990, output_fired_time: 13.3001461029, propagate_forward_time: 933.3480834961, propagate_backward_time: 1434.4040527344, weight_update_time: 1.2406600714, other_time: 17.5695800781
epoch time: 220.6126556396s, input_fired_time: 2.6294300556, output_fired_time: 0.0000000000, propagate_forward_time: 214.3285827637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6546478271
validation[          , genome  5799] predictions:    6097/  10000 (60.97%), best:    6151/10000 (61.51%), error:     11919.03125, best error:     11746.73242 on epoch:    60, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 2702.5456542969s, input_fired_time: 20.4657211304, output_fired_time: 14.7202568054, propagate_forward_time: 1050.6381835938, propagate_backward_time: 1596.6979980469, weight_update_time: 1.5468506813, other_time: 20.0234375000
epoch time: 218.8479919434s, input_fired_time: 2.3621513844, output_fired_time: 0.0000000000, propagate_forward_time: 212.8587036133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6271362305
validation[          , genome  5799] predictions:    6086/  10000 (60.86%), best:    6151/10000 (61.51%), error:     12157.57227, best error:     11746.73242 on epoch:    60, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 2681.3957519531s, input_fired_time: 19.9926681519, output_fired_time: 14.5755624771, propagate_forward_time: 1045.5292968750, propagate_backward_time: 1582.1690673828, weight_update_time: 1.3990966082, other_time: 19.1290283203
epoch time: 206.2740783691s, input_fired_time: 2.1356987953, output_fired_time: 0.0000000000, propagate_forward_time: 200.9200439453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2183380127
validation[          , genome  5799] predictions:    6192/  10000 (61.92%), best:    6192/10000 (61.92%), error:     11687.12500, best error:     11687.12500 on epoch:    64, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 2299.8913574219s, input_fired_time: 17.1164875031, output_fired_time: 12.5060939789, propagate_forward_time: 893.3756713867, propagate_backward_time: 1360.3074951172, weight_update_time: 1.2818490267, other_time: 16.5855712891
epoch time: 107.8445816040s, input_fired_time: 1.0811628103, output_fired_time: 0.0000000000, propagate_forward_time: 105.1898803711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5735397339
validation[          , genome  5799] predictions:    6124/  10000 (61.24%), best:    6192/10000 (61.92%), error:     11929.75391, best error:     11687.12500 on epoch:    64, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 1336.4704589844s, input_fired_time: 9.9298038483, output_fired_time: 7.2962808609, propagate_forward_time: 501.3072509766, propagate_backward_time: 809.0633544922, weight_update_time: 0.6867820024, other_time: 8.8737792969
epoch time: 105.8483657837s, input_fired_time: 1.1038357019, output_fired_time: 0.0000000000, propagate_forward_time: 103.2001190186, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5444107056
validation[          , genome  5799] predictions:    6090/  10000 (60.90%), best:    6192/10000 (61.92%), error:     11949.21582, best error:     11687.12500 on epoch:    64, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 1329.2846679688s, input_fired_time: 9.8905324936, output_fired_time: 7.2890233994, propagate_forward_time: 496.4466552734, propagate_backward_time: 806.8261108398, weight_update_time: 0.6915547252, other_time: 8.8323364258
epoch time: 120.9926910400s, input_fired_time: 1.1318275928, output_fired_time: 0.0000000000, propagate_forward_time: 118.3094863892, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5513763428
validation[          , genome  5799] predictions:    6134/  10000 (61.34%), best:    6192/10000 (61.92%), error:     11882.97852, best error:     11687.12500 on epoch:    64, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 1357.1845703125s, input_fired_time: 9.9856281281, output_fired_time: 7.4315123558, propagate_forward_time: 515.0328979492, propagate_backward_time: 815.9787597656, weight_update_time: 0.6880164146, other_time: 8.7557983398
epoch time: 115.8715133667s, input_fired_time: 1.1591110229, output_fired_time: 0.0000000000, propagate_forward_time: 113.0209579468, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6914443970
validation[          , genome  5799] predictions:    6105/  10000 (61.05%), best:    6192/10000 (61.92%), error:     12349.39746, best error:     11687.12500 on epoch:    64, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 1404.0938720703s, input_fired_time: 10.3339519501, output_fired_time: 7.7025809288, propagate_forward_time: 531.2918090820, propagate_backward_time: 845.5616455078, weight_update_time: 0.7051675320, other_time: 9.2037963867
epoch time: 102.7564010620s, input_fired_time: 1.0918862820, output_fired_time: 0.0000000000, propagate_forward_time: 100.0588455200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6056671143
validation[          , genome  5799] predictions:    6209/  10000 (62.09%), best:    6209/10000 (62.09%), error:     11626.63574, best error:     11626.63574 on epoch:    69, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 1340.2476806641s, input_fired_time: 9.9076223373, output_fired_time: 7.3157629967, propagate_forward_time: 505.6752319336, propagate_backward_time: 808.5919799805, weight_update_time: 0.6895668507, other_time: 8.7570800781
epoch time: 91.5758666992s, input_fired_time: 1.0041242838, output_fired_time: 0.0000000000, propagate_forward_time: 89.1498413086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4218978882
validation[          , genome  5799] predictions:    6136/  10000 (61.36%), best:    6209/10000 (62.09%), error:     11885.40137, best error:     11626.63574 on epoch:    69, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 1436.0939941406s, input_fired_time: 10.0447921753, output_fired_time: 7.7819862366, propagate_forward_time: 547.4375000000, propagate_backward_time: 861.9636840820, weight_update_time: 0.6913851500, other_time: 8.8660278320
epoch time: 132.7199554443s, input_fired_time: 1.1737763882, output_fired_time: 0.0000000000, propagate_forward_time: 129.9692840576, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5768890381
validation[          , genome  5799] predictions:    6174/  10000 (61.74%), best:    6209/10000 (62.09%), error:     11815.84082, best error:     11626.63574 on epoch:    69, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 1406.1912841797s, input_fired_time: 10.0531682968, output_fired_time: 7.6311640739, propagate_forward_time: 537.0353393555, propagate_backward_time: 842.7150878906, weight_update_time: 0.6913949847, other_time: 8.7565307617
epoch time: 106.3201065063s, input_fired_time: 1.0568758249, output_fired_time: 0.0000000000, propagate_forward_time: 103.6136779785, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6495513916
validation[          , genome  5799] predictions:    6198/  10000 (61.98%), best:    6209/10000 (62.09%), error:     11688.78906, best error:     11626.63574 on epoch:    69, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 1313.3151855469s, input_fired_time: 9.6899642944, output_fired_time: 7.0767130852, propagate_forward_time: 499.4843444824, propagate_backward_time: 788.8265380859, weight_update_time: 0.6533337235, other_time: 8.2376708984
epoch time: 100.7358016968s, input_fired_time: 1.0607250929, output_fired_time: 0.0000000000, propagate_forward_time: 98.0816345215, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5934448242
validation[          , genome  5799] predictions:    6164/  10000 (61.64%), best:    6209/10000 (62.09%), error:     11816.38672, best error:     11626.63574 on epoch:    69, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 1321.7292480469s, input_fired_time: 9.8000764847, output_fired_time: 7.2790904045, propagate_forward_time: 494.9307861328, propagate_backward_time: 801.0344238281, weight_update_time: 0.6749763489, other_time: 8.6849365234
epoch time: 101.1800384521s, input_fired_time: 1.0396993160, output_fired_time: 0.0000000000, propagate_forward_time: 98.5181655884, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6221771240
validation[          , genome  5799] predictions:    6114/  10000 (61.14%), best:    6209/10000 (62.09%), error:     11998.54395, best error:     11626.63574 on epoch:    69, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 1330.9639892578s, input_fired_time: 9.8314838409, output_fired_time: 7.3950166702, propagate_forward_time: 492.2585449219, propagate_backward_time: 812.5953369141, weight_update_time: 0.6776598096, other_time: 8.8835449219
epoch time: 101.2291412354s, input_fired_time: 1.0340050459, output_fired_time: 0.0000000000, propagate_forward_time: 98.5421905518, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6529464722
validation[          , genome  5799] predictions:    6179/  10000 (61.79%), best:    6209/10000 (62.09%), error:     11941.13770, best error:     11626.63574 on epoch:    69, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 1355.1087646484s, input_fired_time: 9.8823385239, output_fired_time: 7.3668804169, propagate_forward_time: 513.8723754883, propagate_backward_time: 815.4237060547, weight_update_time: 0.6719000936, other_time: 8.5635375977
epoch time: 97.9229736328s, input_fired_time: 1.0308033228, output_fired_time: 0.0000000000, propagate_forward_time: 95.2966156006, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5955581665
validation[          , genome  5799] predictions:    6177/  10000 (61.77%), best:    6209/10000 (62.09%), error:     11862.00684, best error:     11626.63574 on epoch:    69, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542069603_184_5799.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542069603_184_5799_2_r1504590210_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.980561
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000240784
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 9.63135e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 77
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6209
read best_validation_error: 11626.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5799
read normal distribution: '1 0 0'
generator_str: '1124877649'
read generator: 1124877649
read generated_by_map:
1 crossover 1
reading 88 nodes.
reading 300 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1304.91223144531s, input_fired_time: 9.84389591217041, output_fired_time: 7.28436470031738, propagate_forward_time: 475.950866699219, propagate_backward_time: 802.915283203125, weight_update_time: 0.685498595237732, other_time: 8.91778564453125
epoch time: 93.7310943603516s, input_fired_time: 1.06209444999695, output_fired_time: 0, propagate_forward_time: 91.1592254638672, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.50977325439453
validation[          , genome  5799] predictions:    6225/  10000 (62.25%), best:    6209/10000 (62.09%), error:     11721.94824, best error:     11626.63574 on epoch:    69, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 1219.5014648438s, input_fired_time: 9.7083196640, output_fired_time: 6.8570785522, propagate_forward_time: 449.3050537109, propagate_backward_time: 745.2266845703, weight_update_time: 0.6643061638, other_time: 8.4042968750
epoch time: 101.1903305054s, input_fired_time: 1.0403940678, output_fired_time: 0.0000000000, propagate_forward_time: 98.5165100098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6334228516
validation[          , genome  5799] predictions:    6166/  10000 (61.66%), best:    6209/10000 (62.09%), error:     11794.61035, best error:     11626.63574 on epoch:    69, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1229.5582275391s, input_fired_time: 9.6329259872, output_fired_time: 6.7775549889, propagate_forward_time: 448.8273315430, propagate_backward_time: 755.8957519531, weight_update_time: 0.6506854892, other_time: 8.4246215820
epoch time: 100.6304397583s, input_fired_time: 1.0397933722, output_fired_time: 0.0000000000, propagate_forward_time: 97.9614639282, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6291809082
validation[          , genome  5799] predictions:    6259/  10000 (62.59%), best:    6259/10000 (62.59%), error:     11605.64160, best error:     11605.64160 on epoch:    79, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1271.5618896484s, input_fired_time: 9.7204580307, output_fired_time: 7.0979690552, propagate_forward_time: 464.2257080078, propagate_backward_time: 781.8908081055, weight_update_time: 0.6650784612, other_time: 8.6268920898
epoch time: 97.0351333618s, input_fired_time: 1.0411880016, output_fired_time: 0.0000000000, propagate_forward_time: 94.4194259644, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5745162964
validation[          , genome  5799] predictions:    6245/  10000 (62.45%), best:    6245/10000 (62.45%), error:     11602.69141, best error:     11602.69141 on epoch:    80, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1270.1950683594s, input_fired_time: 9.7120943069, output_fired_time: 7.0832271576, propagate_forward_time: 463.1976623535, propagate_backward_time: 781.5519409180, weight_update_time: 0.6673194170, other_time: 8.6502075195
epoch time: 91.2266311646s, input_fired_time: 1.0210825205, output_fired_time: 0.0000000000, propagate_forward_time: 88.7159347534, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4896163940
validation[          , genome  5799] predictions:    6216/  10000 (62.16%), best:    6245/10000 (62.45%), error:     11625.77930, best error:     11602.69141 on epoch:    80, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1269.1865234375s, input_fired_time: 9.7818794250, output_fired_time: 7.1151304245, propagate_forward_time: 465.1602172852, propagate_backward_time: 778.5219116211, weight_update_time: 0.6700264812, other_time: 8.6074218750
epoch time: 96.4063873291s, input_fired_time: 1.0269665718, output_fired_time: 0.0000000000, propagate_forward_time: 93.8088912964, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5705261230
validation[          , genome  5799] predictions:    6114/  10000 (61.14%), best:    6245/10000 (62.45%), error:     11955.35156, best error:     11602.69141 on epoch:    80, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1271.2408447266s, input_fired_time: 9.8154087067, output_fired_time: 7.1834263802, propagate_forward_time: 468.7733459473, propagate_backward_time: 776.7132568359, weight_update_time: 0.6904141903, other_time: 8.7553710938
epoch time: 96.1352996826s, input_fired_time: 1.0377521515, output_fired_time: 0.0000000000, propagate_forward_time: 93.5305786133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5669708252
validation[          , genome  5799] predictions:    6241/  10000 (62.41%), best:    6241/10000 (62.41%), error:     11532.33691, best error:     11532.33691 on epoch:    83, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1219.8460693359s, input_fired_time: 9.6159563065, output_fired_time: 6.7580070496, propagate_forward_time: 446.5915222168, propagate_backward_time: 748.4885864258, weight_update_time: 0.6590486169, other_time: 8.3919067383
epoch time: 101.2489929199s, input_fired_time: 1.0314038992, output_fired_time: 0.0000000000, propagate_forward_time: 98.5944366455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6231536865
validation[          , genome  5799] predictions:    6177/  10000 (61.77%), best:    6241/10000 (62.41%), error:     11825.48438, best error:     11532.33691 on epoch:    83, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 1279.3883056641s, input_fired_time: 9.8543243408, output_fired_time: 7.2236967087, propagate_forward_time: 471.0856018066, propagate_backward_time: 782.4999389648, weight_update_time: 0.6901649237, other_time: 8.7246704102
epoch time: 99.6205902100s, input_fired_time: 1.0513949394, output_fired_time: 0.0000000000, propagate_forward_time: 96.9698638916, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5993347168
validation[          , genome  5799] predictions:    6163/  10000 (61.63%), best:    6241/10000 (62.41%), error:     11734.36719, best error:     11532.33691 on epoch:    83, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1275.1318359375s, input_fired_time: 9.7747583389, output_fired_time: 7.1445899010, propagate_forward_time: 467.2093811035, propagate_backward_time: 782.3623046875, weight_update_time: 0.6703057885, other_time: 8.6408691406
epoch time: 93.2959289551s, input_fired_time: 1.0394120216, output_fired_time: 0.0000000000, propagate_forward_time: 90.7399597168, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5165557861
validation[          , genome  5799] predictions:    6218/  10000 (62.18%), best:    6241/10000 (62.41%), error:     11724.21191, best error:     11532.33691 on epoch:    83, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1249.8079833984s, input_fired_time: 9.8187589645, output_fired_time: 7.0768356323, propagate_forward_time: 461.6605529785, propagate_backward_time: 762.7256469727, weight_update_time: 0.6832906008, other_time: 8.5263061523
epoch time: 90.6515426636s, input_fired_time: 1.0214517117, output_fired_time: 0.0000000000, propagate_forward_time: 88.1540756226, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4760131836
validation[          , genome  5799] predictions:    6263/  10000 (62.63%), best:    6241/10000 (62.41%), error:     11594.84277, best error:     11532.33691 on epoch:    83, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1260.0620117188s, input_fired_time: 9.7195625305, output_fired_time: 7.0169534683, propagate_forward_time: 461.3381652832, propagate_backward_time: 773.4107055664, weight_update_time: 0.6655275226, other_time: 8.5765991211
epoch time: 94.7039031982s, input_fired_time: 1.0378187895, output_fired_time: 0.0000000000, propagate_forward_time: 92.1370544434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5290298462
validation[          , genome  5799] predictions:    6200/  10000 (62.00%), best:    6241/10000 (62.41%), error:     11669.85254, best error:     11532.33691 on epoch:    83, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1240.1365966797s, input_fired_time: 9.6816234589, output_fired_time: 6.9531540871, propagate_forward_time: 453.8944702148, propagate_backward_time: 761.1548461914, weight_update_time: 0.6581017971, other_time: 8.4525146484
epoch time: 100.8515396118s, input_fired_time: 1.0455610752, output_fired_time: 0.0000000000, propagate_forward_time: 98.1858978271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6200790405
validation[          , genome  5799] predictions:    6221/  10000 (62.21%), best:    6241/10000 (62.41%), error:     11753.86426, best error:     11532.33691 on epoch:    83, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1256.6300048828s, input_fired_time: 9.8274412155, output_fired_time: 7.1192593575, propagate_forward_time: 463.9444885254, propagate_backward_time: 767.1799926758, weight_update_time: 0.6853095889, other_time: 8.5588989258
epoch time: 79.9318771362s, input_fired_time: 0.9994039536, output_fired_time: 0.0000000000, propagate_forward_time: 77.6098861694, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3225860596
validation[          , genome  5799] predictions:    6204/  10000 (62.04%), best:    6241/10000 (62.41%), error:     11692.48047, best error:     11532.33691 on epoch:    83, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1245.6207275391s, input_fired_time: 9.6906137466, output_fired_time: 6.9279341698, propagate_forward_time: 455.4763793945, propagate_backward_time: 765.0527343750, weight_update_time: 0.6604250073, other_time: 8.4729614258
epoch time: 91.3551559448s, input_fired_time: 1.0193784237, output_fired_time: 0.0000000000, propagate_forward_time: 88.8391342163, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4966430664
validation[          , genome  5799] predictions:    6239/  10000 (62.39%), best:    6241/10000 (62.41%), error:     11603.85840, best error:     11532.33691 on epoch:    83, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1267.2524414062s, input_fired_time: 9.8006925583, output_fired_time: 7.1013269424, propagate_forward_time: 465.6953430176, propagate_backward_time: 776.0350952148, weight_update_time: 0.6761754751, other_time: 8.6200561523
epoch time: 100.2759628296s, input_fired_time: 1.0546315908, output_fired_time: 0.0000000000, propagate_forward_time: 97.6137161255, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6076126099
validation[          , genome  5799] predictions:    6264/  10000 (62.64%), best:    6241/10000 (62.41%), error:     11642.85645, best error:     11532.33691 on epoch:    83, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1226.0323486328s, input_fired_time: 9.7418107986, output_fired_time: 6.8764891624, propagate_forward_time: 453.1954956055, propagate_backward_time: 747.7524414062, weight_update_time: 0.6751453280, other_time: 8.4661254883
epoch time: 86.4018478394s, input_fired_time: 1.0187071562, output_fired_time: 0.0000000000, propagate_forward_time: 83.9781646729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4049758911
validation[          , genome  5799] predictions:    6264/  10000 (62.64%), best:    6241/10000 (62.41%), error:     11588.23828, best error:     11532.33691 on epoch:    83, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1259.9541015625s, input_fired_time: 9.8937911987, output_fired_time: 7.1239099503, propagate_forward_time: 466.3530883789, propagate_backward_time: 767.9265136719, weight_update_time: 0.6829904318, other_time: 8.6567993164
epoch time: 82.3337173462s, input_fired_time: 1.0214841366, output_fired_time: 0.0000000000, propagate_forward_time: 79.9556045532, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3566284180
validation[          , genome  5799] predictions:    6253/  10000 (62.53%), best:    6241/10000 (62.41%), error:     11658.27734, best error:     11532.33691 on epoch:    83, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1315.2449951172s, input_fired_time: 9.8030691147, output_fired_time: 7.3442792892, propagate_forward_time: 478.2753906250, propagate_backward_time: 811.0123291016, weight_update_time: 0.6699720025, other_time: 8.8099365234
epoch time: 93.6495208740s, input_fired_time: 1.0392885208, output_fired_time: 0.0000000000, propagate_forward_time: 91.0857391357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5244903564
validation[          , genome  5799] predictions:    6277/  10000 (62.77%), best:    6241/10000 (62.41%), error:     11634.76270, best error:     11532.33691 on epoch:    83, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1264.5483398438s, input_fired_time: 10.0000963211, output_fired_time: 7.0835371017, propagate_forward_time: 464.0647583008, propagate_backward_time: 774.7515258789, weight_update_time: 0.6744266748, other_time: 8.6484375000
epoch time: 81.9870910645s, input_fired_time: 1.0178877115, output_fired_time: 0.0000000000, propagate_forward_time: 79.6225967407, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3466033936
validation[          , genome  5799] predictions:    6261/  10000 (62.61%), best:    6241/10000 (62.41%), error:     11648.03711, best error:     11532.33691 on epoch:    83, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1245.9780273438s, input_fired_time: 9.6624870300, output_fired_time: 6.8979129791, propagate_forward_time: 452.5690612793, propagate_backward_time: 768.4268188477, weight_update_time: 0.6449981928, other_time: 8.4216918945
epoch time: 95.8639831543s, input_fired_time: 1.0608603954, output_fired_time: 0.0000000000, propagate_forward_time: 93.2496185303, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5535049438
validation[          , genome  5799] predictions:    6291/  10000 (62.91%), best:    6241/10000 (62.41%), error:     11577.23535, best error:     11532.33691 on epoch:    83, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1285.5161132812s, input_fired_time: 9.8287286758, output_fired_time: 7.1614022255, propagate_forward_time: 468.9738159180, propagate_backward_time: 790.8220825195, weight_update_time: 0.6710439920, other_time: 8.7301025391
epoch time: 95.9860916138s, input_fired_time: 1.0457115173, output_fired_time: 0.0000000000, propagate_forward_time: 93.3866195679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5537643433
validation[          , genome  5799] predictions:    6257/  10000 (62.57%), best:    6241/10000 (62.41%), error:     11593.68652, best error:     11532.33691 on epoch:    83, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1246.3947753906s, input_fired_time: 9.7805967331, output_fired_time: 6.9917392731, propagate_forward_time: 457.3891906738, propagate_backward_time: 763.7305297852, weight_update_time: 0.6669479012, other_time: 8.5027465820
epoch time: 97.8590698242s, input_fired_time: 1.0546253920, output_fired_time: 0.0000000000, propagate_forward_time: 95.2251663208, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5792770386
validation[          , genome  5799] predictions:    6251/  10000 (62.51%), best:    6241/10000 (62.41%), error:     11698.98438, best error:     11532.33691 on epoch:    83, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1251.9129638672s, input_fired_time: 9.8630304337, output_fired_time: 7.0769848824, propagate_forward_time: 461.9200134277, propagate_backward_time: 764.4663085938, weight_update_time: 0.6813153028, other_time: 8.5865478516
epoch time: 93.1953735352s, input_fired_time: 1.0732992887, output_fired_time: 0.0000000000, propagate_forward_time: 90.6322097778, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4898681641
validation[          , genome  5799] predictions:    6221/  10000 (62.21%), best:    6241/10000 (62.41%), error:     11736.37305, best error:     11532.33691 on epoch:    83, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 469.9779663086s, input_fired_time: 5.2338371277, output_fired_time: 0.0000000000, propagate_forward_time: 457.1231079102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.6210327148
best training[          , genome  5799] predictions:   40600/  50000 (81.20%), best:    6241/10000 (62.41%), error:     29063.73242, best error:     11532.33691 on epoch:    83, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 95.1178741455s, input_fired_time: 1.0390650034, output_fired_time: 0.0000000000, propagate_forward_time: 92.5483169556, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5304946899
   testing[          , genome  5799] predictions:    6241/  10000 (62.41%), best:    6241/10000 (62.41%), error:     11532.33691, best error:     11532.33691 on epoch:    83, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
08:32:59 (2517): called boinc_finish(0)

</stderr_txt>
]]>