Nameexact_genome_1542531604_188_6272_4
Workunit4097913
Created1 Mar 2019, 17:55:34 UTC
Sent1 Mar 2019, 18:06:00 UTC
Report deadline10 Jun 2019, 19:05:41 UTC
Received3 Mar 2019, 13:16:46 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID84860
Run time1 days 18 hours 10 min 55 sec
CPU time1 days 14 hours 41 min 11 sec
Validate stateValid
Credit25,260.36
Device peak FLOPS4.84 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size547.28 MB
Peak swap size550.02 MB
Peak disk usage8.10 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
h time: 93.5963745117s, input_fired_time: 0.9272087216, output_fired_time: 0.0000000000, propagate_forward_time: 91.0647506714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6044158936
validation[          , genome  6272] predictions:    6062/  10000 (60.62%), best:    6028/10000 (60.28%), error:     11331.82812, best error:     11276.41406 on epoch:    19, epoch:   21/100, mu: 0.8455072045, learning_rate: 0.0001736353, weight_decay: 0.0001469994

epoch time: 1271.5826416016s, input_fired_time: 18.1351547241, output_fired_time: 6.5652947426, propagate_forward_time: 466.3521423340, propagate_backward_time: 770.3485717773, weight_update_time: 0.3627857864, other_time: 10.1814575195
epoch time: 101.3941726685s, input_fired_time: 0.9429152608, output_fired_time: 0.0000000000, propagate_forward_time: 98.6834945679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7677612305
validation[          , genome  6272] predictions:    6220/  10000 (62.20%), best:    6220/10000 (62.20%), error:     11031.90527, best error:     11031.90527 on epoch:    22, epoch:   22/100, mu: 0.8536649346, learning_rate: 0.0001620985, weight_decay: 0.0001386890

epoch time: 1248.8651123047s, input_fired_time: 18.1160488129, output_fired_time: 6.4534859657, propagate_forward_time: 457.7466125488, propagate_backward_time: 756.4652709961, weight_update_time: 0.3589317501, other_time: 10.0836791992
epoch time: 98.2957839966s, input_fired_time: 0.9421432018, output_fired_time: 0.0000000000, propagate_forward_time: 95.6964187622, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6572189331
validation[          , genome  6272] predictions:    6217/  10000 (62.17%), best:    6220/10000 (62.20%), error:     11038.74609, best error:     11031.90527 on epoch:    22, epoch:   23/100, mu: 0.8613620996, learning_rate: 0.0001513282, weight_decay: 0.0001308485

epoch time: 1312.4645996094s, input_fired_time: 18.1834125519, output_fired_time: 6.8363313675, propagate_forward_time: 479.1246032715, propagate_backward_time: 797.7297363281, weight_update_time: 0.3762166500, other_time: 10.5904541016
epoch time: 98.9882354736s, input_fired_time: 0.9437131882, output_fired_time: 0.0000000000, propagate_forward_time: 96.2525177002, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7920074463
validation[          , genome  6272] predictions:    6114/  10000 (61.14%), best:    6220/10000 (62.20%), error:     11059.21777, best error:     11031.90527 on epoch:    22, epoch:   24/100, mu: 0.8686246872, learning_rate: 0.0001412735, weight_decay: 0.0001234512

epoch time: 1322.9890136719s, input_fired_time: 18.2152633667, output_fired_time: 6.9196643829, propagate_forward_time: 480.2321777344, propagate_backward_time: 806.9311523438, weight_update_time: 0.3761092722, other_time: 10.6907958984
epoch time: 99.1580047607s, input_fired_time: 0.9366101027, output_fired_time: 0.0000000000, propagate_forward_time: 96.4224548340, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7989425659
validation[          , genome  6272] predictions:    6150/  10000 (61.50%), best:    6220/10000 (62.20%), error:     11032.09082, best error:     11031.90527 on epoch:    22, epoch:   25/100, mu: 0.8754772544, learning_rate: 0.0001318869, weight_decay: 0.0001164721

epoch time: 1321.8565673828s, input_fired_time: 18.2229194641, output_fired_time: 6.9071331024, propagate_forward_time: 479.5008239746, propagate_backward_time: 806.4955444336, weight_update_time: 0.3797672391, other_time: 10.7301635742
epoch time: 99.1037673950s, input_fired_time: 0.9353531003, output_fired_time: 0.0000000000, propagate_forward_time: 96.3597030640, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8087081909
validation[          , genome  6272] predictions:    6166/  10000 (61.66%), best:    6166/10000 (61.66%), error:     11023.48633, best error:     11023.48633 on epoch:    26, epoch:   26/100, mu: 0.8819429278, learning_rate: 0.0001231239, weight_decay: 0.0001098876

epoch time: 1319.2952880859s, input_fired_time: 18.2086772919, output_fired_time: 6.9047207832, propagate_forward_time: 478.7799987793, propagate_backward_time: 804.5833740234, weight_update_time: 0.3834395111, other_time: 10.8186035156
epoch time: 99.0265731812s, input_fired_time: 0.9412747622, output_fired_time: 0.0000000000, propagate_forward_time: 96.2941360474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7911605835
validation[          , genome  6272] predictions:    6242/  10000 (62.42%), best:    6242/10000 (62.42%), error:     10905.47168, best error:     10905.47168 on epoch:    27, epoch:   27/100, mu: 0.8880435824, learning_rate: 0.0001149432, weight_decay: 0.0001036753

epoch time: 1324.2469482422s, input_fired_time: 18.2273960114, output_fired_time: 6.9200930595, propagate_forward_time: 480.6954345703, propagate_backward_time: 807.6025390625, weight_update_time: 0.3824929595, other_time: 10.8015136719
epoch time: 98.7821578979s, input_fired_time: 0.9467657208, output_fired_time: 0.0000000000, propagate_forward_time: 96.0739898682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7614059448
validation[          , genome  6272] predictions:    6326/  10000 (63.26%), best:    6326/10000 (63.26%), error:     10711.59473, best error:     10711.59473 on epoch:    28, epoch:   28/100, mu: 0.8937998414, learning_rate: 0.0001073061, weight_decay: 0.0000978142

epoch time: 1314.2526855469s, input_fired_time: 18.2180328369, output_fired_time: 6.8850231171, propagate_forward_time: 476.6958618164, propagate_backward_time: 801.7951049805, weight_update_time: 0.3819910586, other_time: 10.6586914062
epoch time: 98.7651748657s, input_fired_time: 0.9486132264, output_fired_time: 0.0000000000, propagate_forward_time: 96.0145187378, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8020401001
validation[          , genome  6272] predictions:    6227/  10000 (62.27%), best:    6326/10000 (63.26%), error:     10842.80176, best error:     10711.59473 on epoch:    28, epoch:   29/100, mu: 0.8992310762, learning_rate: 0.0001001763, weight_decay: 0.0000922844

epoch time: 1311.6881103516s, input_fired_time: 18.2382526398, output_fired_time: 6.8550105095, propagate_forward_time: 475.9277648926, propagate_backward_time: 800.0571899414, weight_update_time: 0.3841486573, other_time: 10.6099243164
epoch time: 96.4192810059s, input_fired_time: 0.9471123815, output_fired_time: 0.0000000000, propagate_forward_time: 93.7038574219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7683105469
validation[          , genome  6272] predictions:    6289/  10000 (62.89%), best:    6289/10000 (62.89%), error:     10676.52441, best error:     10676.52441 on epoch:    30, epoch:   30/100, mu: 0.9043557048, learning_rate: 0.0000935203, weight_decay: 0.0000870673

epoch time: 1298.8425292969s, input_fired_time: 18.2453212738, output_fired_time: 6.7631702423, propagate_forward_time: 472.8464965820, propagate_backward_time: 790.5102539062, weight_update_time: 0.3789431751, other_time: 10.4772338867
epoch time: 97.3522796631s, input_fired_time: 0.9514333010, output_fired_time: 0.0000000000, propagate_forward_time: 94.6419601440, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7588882446
validation[          , genome  6272] predictions:    6288/  10000 (62.88%), best:    6289/10000 (62.89%), error:     10690.79980, best error:     10676.52441 on epoch:    30, epoch:   31/100, mu: 0.9091910124, learning_rate: 0.0000873066, weight_decay: 0.0000821451

epoch time: 1233.1489257812s, input_fired_time: 18.0878067017, output_fired_time: 6.3190827370, propagate_forward_time: 452.8712463379, propagate_backward_time: 746.0760498047, weight_update_time: 0.3642805815, other_time: 9.7947998047
epoch time: 99.1072311401s, input_fired_time: 0.9590910673, output_fired_time: 0.0000000000, propagate_forward_time: 96.3489074707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7992324829
validation[          , genome  6272] predictions:    6197/  10000 (61.97%), best:    6289/10000 (62.89%), error:     10816.24219, best error:     10676.52441 on epoch:    30, epoch:   32/100, mu: 0.9137533307, learning_rate: 0.0000815057, weight_decay: 0.0000775012

epoch time: 1281.8544921875s, input_fired_time: 18.2026939392, output_fired_time: 6.6032552719, propagate_forward_time: 467.3187255859, propagate_backward_time: 779.3684082031, weight_update_time: 0.3753675520, other_time: 10.3614501953
epoch time: 98.8868179321s, input_fired_time: 0.9584536552, output_fired_time: 0.0000000000, propagate_forward_time: 96.1440048218, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7843627930
validation[          , genome  6272] predictions:    6305/  10000 (63.05%), best:    6305/10000 (63.05%), error:     10548.51855, best error:     10548.51855 on epoch:    33, epoch:   33/100, mu: 0.9180580378, learning_rate: 0.0000760902, weight_decay: 0.0000731198

epoch time: 1287.6656494141s, input_fired_time: 18.2182407379, output_fired_time: 6.6824059486, propagate_forward_time: 468.6730346680, propagate_backward_time: 783.6653442383, weight_update_time: 0.3787424266, other_time: 10.4266357422
epoch time: 97.6549682617s, input_fired_time: 0.9450331330, output_fired_time: 0.0000000000, propagate_forward_time: 94.9369583130, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7729797363
validation[          , genome  6272] predictions:    6210/  10000 (62.10%), best:    6305/10000 (63.05%), error:     10851.65430, best error:     10548.51855 on epoch:    33, epoch:   34/100, mu: 0.9221197367, learning_rate: 0.0000710345, weight_decay: 0.0000689861

epoch time: 1309.7917480469s, input_fired_time: 18.2611865997, output_fired_time: 6.8614902496, propagate_forward_time: 476.1460571289, propagate_backward_time: 797.8190307617, weight_update_time: 0.3863384128, other_time: 10.7039794922
epoch time: 99.0514907837s, input_fired_time: 0.9577321410, output_fired_time: 0.0000000000, propagate_forward_time: 96.2941741943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7995834351
validation[          , genome  6272] predictions:    6426/  10000 (64.26%), best:    6426/10000 (64.26%), error:     10437.01172, best error:     10437.01172 on epoch:    35, epoch:   35/100, mu: 0.9259520769, learning_rate: 0.0000663148, weight_decay: 0.0000650861

epoch time: 1323.5006103516s, input_fired_time: 18.2698650360, output_fired_time: 6.9240207672, propagate_forward_time: 480.4707641602, propagate_backward_time: 807.0844726562, weight_update_time: 0.3870870769, other_time: 10.7514038086
epoch time: 99.0140914917s, input_fired_time: 0.9611179233, output_fired_time: 0.0000000000, propagate_forward_time: 96.2462158203, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8067550659
validation[          , genome  6272] predictions:    6398/  10000 (63.98%), best:    6426/10000 (64.26%), error:     10478.56934, best error:     10437.01172 on epoch:    35, epoch:   36/100, mu: 0.9295680523, learning_rate: 0.0000619086, weight_decay: 0.0000614065

epoch time: 1312.7540283203s, input_fired_time: 18.2767391205, output_fired_time: 6.8478431702, propagate_forward_time: 479.3346557617, propagate_backward_time: 797.7257080078, weight_update_time: 0.3827100098, other_time: 10.5690307617
epoch time: 89.1522979736s, input_fired_time: 0.9248582125, output_fired_time: 0.0000000000, propagate_forward_time: 86.6227645874, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6046752930
validation[          , genome  6272] predictions:    6241/  10000 (62.41%), best:    6426/10000 (64.26%), error:     10747.57617, best error:     10437.01172 on epoch:    35, epoch:   37/100, mu: 0.9329798818, learning_rate: 0.0000577953, weight_decay: 0.0000579350

epoch time: 1296.1607666016s, input_fired_time: 18.2739486694, output_fired_time: 6.7347793579, propagate_forward_time: 472.1240539551, propagate_backward_time: 788.5216674805, weight_update_time: 0.3825984597, other_time: 10.5064086914
epoch time: 99.0233535767s, input_fired_time: 0.9569029808, output_fired_time: 0.0000000000, propagate_forward_time: 96.2652740479, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8011779785
validation[          , genome  6272] predictions:    6353/  10000 (63.53%), best:    6426/10000 (64.26%), error:     10463.46680, best error:     10437.01172 on epoch:    35, epoch:   38/100, mu: 0.9361991286, learning_rate: 0.0000539552, weight_decay: 0.0000546598

epoch time: 1259.0480957031s, input_fired_time: 18.2064723969, output_fired_time: 6.4732751846, propagate_forward_time: 461.7584533691, propagate_backward_time: 762.5008544922, weight_update_time: 0.3698618710, other_time: 10.1091308594
epoch time: 94.5280761719s, input_fired_time: 0.9600431323, output_fired_time: 0.0000000000, propagate_forward_time: 91.9591140747, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6089172363
validation[          , genome  6272] predictions:    6385/  10000 (63.85%), best:    6385/10000 (63.85%), error:     10423.08105, best error:     10423.08105 on epoch:    39, epoch:   39/100, mu: 0.9392365813, learning_rate: 0.0000503702, weight_decay: 0.0000515697

epoch time: 1227.3436279297s, input_fired_time: 18.1506557465, output_fired_time: 6.2676377296, propagate_forward_time: 450.1202087402, propagate_backward_time: 742.8614501953, weight_update_time: 0.3662838042, other_time: 9.9437255859
epoch time: 88.5409851074s, input_fired_time: 0.9414625168, output_fired_time: 0.0000000000, propagate_forward_time: 86.0698013306, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5297241211
validation[          , genome  6272] predictions:    6255/  10000 (62.55%), best:    6385/10000 (63.85%), error:     10663.27051, best error:     10423.08105 on epoch:    39, epoch:   40/100, mu: 0.9421025515, learning_rate: 0.0000470235, weight_decay: 0.0000486543

epoch time: 1280.1391601562s, input_fired_time: 18.2707023621, output_fired_time: 6.6332936287, propagate_forward_time: 467.6753845215, propagate_backward_time: 777.2215576172, weight_update_time: 0.3764471412, other_time: 10.3381347656
epoch time: 98.8831405640s, input_fired_time: 0.9749959707, output_fired_time: 0.0000000000, propagate_forward_time: 96.2518539429, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6562881470
validation[          , genome  6272] predictions:    6310/  10000 (63.10%), best:    6385/10000 (63.85%), error:     10592.25977, best error:     10423.08105 on epoch:    39, epoch:   41/100, mu: 0.9448067546, learning_rate: 0.0000438991, weight_decay: 0.0000459037

epoch time: 1295.3607177734s, input_fired_time: 18.3018932343, output_fired_time: 6.7492041588, propagate_forward_time: 473.0948486328, propagate_backward_time: 786.8040771484, weight_update_time: 0.3812513649, other_time: 10.4107666016
epoch time: 97.3393936157s, input_fired_time: 0.9628394246, output_fired_time: 0.0000000000, propagate_forward_time: 94.6179504395, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7586059570
validation[          , genome  6272] predictions:    6363/  10000 (63.63%), best:    6385/10000 (63.85%), error:     10542.60547, best error:     10423.08105 on epoch:    39, epoch:   42/100, mu: 0.9473582506, learning_rate: 0.0000409823, weight_decay: 0.0000433086

epoch time: 1293.7753906250s, input_fired_time: 18.2880439758, output_fired_time: 6.7120566368, propagate_forward_time: 471.0687561035, propagate_backward_time: 787.3095703125, weight_update_time: 0.3806768358, other_time: 10.3969726562
epoch time: 97.6460494995s, input_fired_time: 0.9624851942, output_fired_time: 0.0000000000, propagate_forward_time: 94.9085083008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7750549316
validation[          , genome  6272] predictions:    6440/  10000 (64.40%), best:    6440/10000 (64.40%), error:     10251.02344, best error:     10251.02344 on epoch:    43, epoch:   43/100, mu: 0.9497656822, learning_rate: 0.0000382593, weight_decay: 0.0000408602

epoch time: 1269.3317871094s, input_fired_time: 18.2575435638, output_fired_time: 6.5049262047, propagate_forward_time: 465.0868225098, propagate_backward_time: 769.2955322266, weight_update_time: 0.3748662770, other_time: 10.1870117188
epoch time: 96.6168365479s, input_fired_time: 0.9478425384, output_fired_time: 0.0000000000, propagate_forward_time: 93.9776153564, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6913757324
validation[          , genome  6272] predictions:    6310/  10000 (63.10%), best:    6440/10000 (64.40%), error:     10445.41406, best error:     10251.02344 on epoch:    43, epoch:   44/100, mu: 0.9520372152, learning_rate: 0.0000357173, weight_decay: 0.0000385503

epoch time: 1263.3588867188s, input_fired_time: 18.2792053223, output_fired_time: 6.4867253304, propagate_forward_time: 467.4427490234, propagate_backward_time: 761.1954345703, weight_update_time: 0.3704250753, other_time: 9.9548339844
epoch time: 99.1577682495s, input_fired_time: 0.9716672301, output_fired_time: 0.0000000000, propagate_forward_time: 96.3688049316, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8172988892
validation[          , genome  6272] predictions:    6443/  10000 (64.43%), best:    6440/10000 (64.40%), error:     10253.61719, best error:     10251.02344 on epoch:    43, epoch:   45/100, mu: 0.9541805387, learning_rate: 0.0000333441, weight_decay: 0.0000363709

epoch time: 1317.0341796875s, input_fired_time: 18.3539772034, output_fired_time: 6.8845829964, propagate_forward_time: 477.9465332031, propagate_backward_time: 803.1364135742, weight_update_time: 0.3892261386, other_time: 10.7125854492
epoch time: 99.0516433716s, input_fired_time: 0.9667609334, output_fired_time: 0.0000000000, propagate_forward_time: 96.2771377563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8077468872
validation[          , genome  6272] predictions:    6412/  10000 (64.12%), best:    6440/10000 (64.40%), error:     10342.55176, best error:     10251.02344 on epoch:    43, epoch:   46/100, mu: 0.9562028050, learning_rate: 0.0000311286, weight_decay: 0.0000343148

epoch time: 1320.0134277344s, input_fired_time: 18.7335243225, output_fired_time: 6.8423624039, propagate_forward_time: 482.1017761230, propagate_backward_time: 801.5804443359, weight_update_time: 0.3974446952, other_time: 10.7552490234
epoch time: 98.8242340088s, input_fired_time: 1.0193796158, output_fired_time: 0.0000000000, propagate_forward_time: 95.9935073853, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8113479614
validation[          , genome  6272] predictions:    6482/  10000 (64.82%), best:    6482/10000 (64.82%), error:     10146.41699, best error:     10146.41699 on epoch:    47, epoch:   47/100, mu: 0.9581109285, learning_rate: 0.0000290604, weight_decay: 0.0000323748

epoch time: 1326.8317871094s, input_fired_time: 18.9811668396, output_fired_time: 6.8807415962, propagate_forward_time: 487.2333679199, propagate_backward_time: 803.0155639648, weight_update_time: 0.3851686716, other_time: 10.7208862305
epoch time: 107.7678527832s, input_fired_time: 1.0611348152, output_fired_time: 0.0000000000, propagate_forward_time: 104.8481216431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8585968018
validation[          , genome  6272] predictions:    6484/  10000 (64.84%), best:    6482/10000 (64.82%), error:     10163.55469, best error:     10146.41699 on epoch:    47, epoch:   48/100, mu: 0.9599112868, learning_rate: 0.0000271295, weight_decay: 0.0000305446

epoch time: 1308.7521972656s, input_fired_time: 18.8851757050, output_fired_time: 6.7043452263, propagate_forward_time: 482.7739257812, propagate_backward_time: 789.8973999023, weight_update_time: 0.3916407824, other_time: 10.4913940430
epoch time: 98.3504028320s, input_fired_time: 0.9944221377, output_fired_time: 0.0000000000, propagate_forward_time: 95.5753250122, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7806549072
validation[          , genome  6272] predictions:    6442/  10000 (64.42%), best:    6482/10000 (64.82%), error:     10194.47754, best error:     10146.41699 on epoch:    47, epoch:   49/100, mu: 0.9616100192, learning_rate: 0.0000253269, weight_decay: 0.0000288178

epoch time: 1300.7775878906s, input_fired_time: 18.6460208893, output_fired_time: 6.6659221649, propagate_forward_time: 478.5982360840, propagate_backward_time: 786.3209228516, weight_update_time: 0.3793718219, other_time: 10.5465087891
epoch time: 98.2436447144s, input_fired_time: 0.9831097126, output_fired_time: 0.0000000000, propagate_forward_time: 95.4701004028, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7904357910
validation[          , genome  6272] predictions:    6476/  10000 (64.76%), best:    6482/10000 (64.82%), error:     10181.14648, best error:     10146.41699 on epoch:    47, epoch:   50/100, mu: 0.9632128477, learning_rate: 0.0000236441, weight_decay: 0.0000271886

epoch time: 1302.5745849609s, input_fired_time: 18.7634429932, output_fired_time: 6.6845655441, propagate_forward_time: 480.3249816895, propagate_backward_time: 786.2704467773, weight_update_time: 0.4007447660, other_time: 10.5311889648
epoch time: 101.8826370239s, input_fired_time: 1.0132968426, output_fired_time: 0.0000000000, propagate_forward_time: 99.0975189209, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7718200684
validation[          , genome  6272] predictions:    6506/  10000 (65.06%), best:    6506/10000 (65.06%), error:     10084.51367, best error:     10084.51367 on epoch:    51, epoch:   51/100, mu: 0.9647251964, learning_rate: 0.0000220732, weight_decay: 0.0000256516

epoch time: 1343.1373291016s, input_fired_time: 19.1151962280, output_fired_time: 6.9235391617, propagate_forward_time: 490.8571777344, propagate_backward_time: 815.4117431641, weight_update_time: 0.4098287821, other_time: 10.8295898438
epoch time: 99.0553512573s, input_fired_time: 0.9844964743, output_fired_time: 0.0000000000, propagate_forward_time: 96.3028564453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7679977417
validation[          , genome  6272] predictions:    6459/  10000 (64.59%), best:    6506/10000 (65.06%), error:     10130.33691, best error:     10084.51367 on epoch:    51, epoch:   52/100, mu: 0.9661521316, learning_rate: 0.0000206066, weight_decay: 0.0000242014

epoch time: 1472.1911621094s, input_fired_time: 21.2570343018, output_fired_time: 7.7824497223, propagate_forward_time: 543.7784423828, propagate_backward_time: 887.0269165039, weight_update_time: 0.4372235537, other_time: 12.3462524414
epoch time: 149.0519714355s, input_fired_time: 1.5640136003, output_fired_time: 0.0000000000, propagate_forward_time: 144.7645416260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7234191895
validation[          , genome  6272] predictions:    6428/  10000 (64.28%), best:    6506/10000 (65.06%), error:     10105.59082, best error:     10084.51367 on epoch:    51, epoch:   53/100, mu: 0.9674985409, learning_rate: 0.0000192374, weight_decay: 0.0000228332

epoch time: 1961.0458984375s, input_fired_time: 29.3993625641, output_fired_time: 10.0353946686, propagate_forward_time: 743.2472534180, propagate_backward_time: 1161.2132568359, weight_update_time: 0.5963285565, other_time: 17.1505126953
epoch time: 180.7765502930s, input_fired_time: 1.9059096575, output_fired_time: 0.0000000000, propagate_forward_time: 175.2897644043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5808715820
validation[          , genome  6272] predictions:    6477/  10000 (64.77%), best:    6506/10000 (65.06%), error:     10139.48242, best error:     10084.51367 on epoch:    51, epoch:   54/100, mu: 0.9687688947, learning_rate: 0.0000179592, weight_decay: 0.0000215424

epoch time: 1959.3050537109s, input_fired_time: 29.6521434784, output_fired_time: 10.1069898605, propagate_forward_time: 743.2042846680, propagate_backward_time: 1159.4631347656, weight_update_time: 0.6146875024, other_time: 16.8786621094
epoch time: 99.5653762817s, input_fired_time: 0.9932715297, output_fired_time: 0.0000000000, propagate_forward_time: 96.7967453003, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7753601074
validation[          , genome  6272] predictions:    6456/  10000 (64.56%), best:    6506/10000 (65.06%), error:     10146.06543, best error:     10084.51367 on epoch:    51, epoch:   55/100, mu: 0.9699675441, learning_rate: 0.0000167659, weight_decay: 0.0000203245

epoch time: 1257.9803466797s, input_fired_time: 18.5716056824, output_fired_time: 6.5822067261, propagate_forward_time: 468.7772216797, propagate_backward_time: 754.0216064453, weight_update_time: 0.3906143606, other_time: 10.0277099609
epoch time: 95.7557373047s, input_fired_time: 0.9886825681, output_fired_time: 0.0000000000, propagate_forward_time: 93.1813659668, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5856857300
validation[          , genome  6272] predictions:    6527/  10000 (65.27%), best:    6527/10000 (65.27%), error:     10034.44922, best error:     10034.44922 on epoch:    56, epoch:   56/100, mu: 0.9710985422, learning_rate: 0.0000156520, weight_decay: 0.0000191755

epoch time: 1307.5877685547s, input_fired_time: 18.4529056549, output_fired_time: 6.8110218048, propagate_forward_time: 479.0650634766, propagate_backward_time: 792.8135375977, weight_update_time: 0.3847676814, other_time: 10.4452514648
epoch time: 98.9238357544s, input_fired_time: 0.9861724973, output_fired_time: 0.0000000000, propagate_forward_time: 96.1475601196, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7901000977
validation[          , genome  6272] predictions:    6511/  10000 (65.11%), best:    6527/10000 (65.27%), error:     10094.97754, best error:     10034.44922 on epoch:    56, epoch:   57/100, mu: 0.9721657038, learning_rate: 0.0000146120, weight_decay: 0.0000180915

epoch time: 2029.2081298828s, input_fired_time: 30.1166801453, output_fired_time: 10.3527898788, propagate_forward_time: 769.4832763672, propagate_backward_time: 1202.0190429688, weight_update_time: 0.6257563233, other_time: 17.2363281250
epoch time: 184.6717987061s, input_fired_time: 1.9546053410, output_fired_time: 0.0000000000, propagate_forward_time: 179.1780395508, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5391540527
validation[          , genome  6272] predictions:    6516/  10000 (65.16%), best:    6527/10000 (65.27%), error:     10134.58984, best error:     10034.44922 on epoch:    56, epoch:   58/100, mu: 0.9731726050, learning_rate: 0.0000136411, weight_decay: 0.0000170687

epoch time: 2334.3256835938s, input_fired_time: 35.8564186096, output_fired_time: 12.1525945663, propagate_forward_time: 893.6271972656, propagate_backward_time: 1371.6011962891, weight_update_time: 0.8058178425, other_time: 21.0882568359
epoch time: 191.1741790771s, input_fired_time: 1.8577772379, output_fired_time: 0.0000000000, propagate_forward_time: 185.7127990723, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6036071777
validation[          , genome  6272] predictions:    6580/  10000 (65.80%), best:    6580/10000 (65.80%), error:      9952.98145, best error:      9952.98145 on epoch:    59, epoch:   59/100, mu: 0.9741226435, learning_rate: 0.0000127348, weight_decay: 0.0000161037

epoch time: 2199.7851562500s, input_fired_time: 33.7138214111, output_fired_time: 11.7225961685, propagate_forward_time: 844.4784545898, propagate_backward_time: 1290.8341064453, weight_update_time: 0.7234972119, other_time: 19.0360107422
epoch time: 112.8789138794s, input_fired_time: 1.0778037310, output_fired_time: 0.0000000000, propagate_forward_time: 110.0360641479, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7650451660
validation[          , genome  6272] predictions:    6521/  10000 (65.21%), best:    6580/10000 (65.80%), error:     10034.49609, best error:      9952.98145 on epoch:    59, epoch:   60/100, mu: 0.9750190377, learning_rate: 0.0000118886, weight_decay: 0.0000151933

epoch time: 1719.7133789062s, input_fired_time: 25.0177936554, output_fired_time: 8.7858095169, propagate_forward_time: 656.8649291992, propagate_backward_time: 1014.9199829102, weight_update_time: 0.5397814512, other_time: 14.1249389648
epoch time: 183.8033294678s, input_fired_time: 2.0382199287, output_fired_time: 0.0000000000, propagate_forward_time: 178.4149322510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3501739502
validation[          , genome  6272] predictions:    6465/  10000 (64.65%), best:    6580/10000 (65.80%), error:     10093.44922, best error:      9952.98145 on epoch:    59, epoch:   61/100, mu: 0.9758648276, learning_rate: 0.0000110987, weight_decay: 0.0000143344

epoch time: 2550.7990722656s, input_fired_time: 37.7460632324, output_fired_time: 12.6721935272, propagate_forward_time: 989.9923095703, propagate_backward_time: 1488.3237304688, weight_update_time: 0.7727061510, other_time: 22.0648193359
epoch time: 198.7795867920s, input_fired_time: 2.0386383533, output_fired_time: 0.0000000000, propagate_forward_time: 193.1330566406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6078948975
validation[          , genome  6272] predictions:    6532/  10000 (65.32%), best:    6580/10000 (65.80%), error:      9993.95410, best error:      9952.98145 on epoch:    59, epoch:   62/100, mu: 0.9766628742, learning_rate: 0.0000103613, weight_decay: 0.0000135240

epoch time: 2640.1953125000s, input_fired_time: 39.1085433960, output_fired_time: 13.1729688644, propagate_forward_time: 1034.0469970703, propagate_backward_time: 1531.0250244141, weight_update_time: 0.8862369657, other_time: 22.8417968750
epoch time: 211.7616271973s, input_fired_time: 2.1734905243, output_fired_time: 0.0000000000, propagate_forward_time: 205.7268524170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8612823486
validation[          , genome  6272] predictions:    6510/  10000 (65.10%), best:    6510/10000 (65.10%), error:      9946.28027, best error:      9946.28027 on epoch:    63, epoch:   63/100, mu: 0.9774158597, learning_rate: 0.0000096729, weight_decay: 0.0000127595

epoch time: 2618.7060546875s, input_fired_time: 38.8565979004, output_fired_time: 13.4023351669, propagate_forward_time: 1025.4099121094, propagate_backward_time: 1518.7026367188, weight_update_time: 0.8154621124, other_time: 22.3344726562
epoch time: 216.6944122314s, input_fired_time: 2.3356854916, output_fired_time: 0.0000000000, propagate_forward_time: 210.5346984863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8240356445
validation[          , genome  6272] predictions:    6512/  10000 (65.12%), best:    6510/10000 (65.10%), error:      9972.85254, best error:      9946.28027 on epoch:    63, epoch:   64/100, mu: 0.9781263471, learning_rate: 0.0000090302, weight_decay: 0.0000120382

epoch time: 1736.4155273438s, input_fired_time: 25.4558410645, output_fired_time: 8.6635465622, propagate_forward_time: 666.8878784180, propagate_backward_time: 1021.2843627930, weight_update_time: 0.5675477386, other_time: 14.1238403320
epoch time: 111.6165847778s, input_fired_time: 1.0604193211, output_fired_time: 0.0000000000, propagate_forward_time: 108.8598327637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6963348389
validation[          , genome  6272] predictions:    6505/  10000 (65.05%), best:    6510/10000 (65.10%), error:     10001.13477, best error:      9946.28027 on epoch:    63, epoch:   65/100, mu: 0.9787967205, learning_rate: 0.0000084302, weight_decay: 0.0000113576

epoch time: 1315.6755371094s, input_fired_time: 18.7942581177, output_fired_time: 6.8265743256, propagate_forward_time: 488.2705688477, propagate_backward_time: 791.3487548828, weight_update_time: 0.4035684764, other_time: 10.4353637695
epoch time: 100.9836349487s, input_fired_time: 1.0211898088, output_fired_time: 0.0000000000, propagate_forward_time: 98.2643432617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6981048584
validation[          , genome  6272] predictions:    6534/  10000 (65.34%), best:    6510/10000 (65.10%), error:     10014.87012, best error:      9946.28027 on epoch:    63, epoch:   66/100, mu: 0.9794292450, learning_rate: 0.0000078701, weight_decay: 0.0000107155

epoch time: 1341.6990966797s, input_fired_time: 18.9526157379, output_fired_time: 6.9044594765, propagate_forward_time: 509.5886230469, propagate_backward_time: 796.0941772461, weight_update_time: 0.4024315774, other_time: 10.1592407227
epoch time: 119.3138504028s, input_fired_time: 1.1022844315, output_fired_time: 0.0000000000, propagate_forward_time: 116.5577850342, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6537780762
validation[          , genome  6272] predictions:    6544/  10000 (65.44%), best:    6544/10000 (65.44%), error:      9926.58398, best error:      9926.58398 on epoch:    67, epoch:   67/100, mu: 0.9800260663, learning_rate: 0.0000073471, weight_decay: 0.0000101097

epoch time: 1380.2406005859s, input_fired_time: 19.1167907715, output_fired_time: 7.0178933144, propagate_forward_time: 523.4304199219, propagate_backward_time: 820.2822875977, weight_update_time: 0.4038604200, other_time: 10.3931274414
epoch time: 98.1804199219s, input_fired_time: 1.0113221407, output_fired_time: 0.0000000000, propagate_forward_time: 95.3792800903, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7898178101
validation[          , genome  6272] predictions:    6536/  10000 (65.36%), best:    6544/10000 (65.44%), error:     10020.64551, best error:      9926.58398 on epoch:    67, epoch:   68/100, mu: 0.9805891514, learning_rate: 0.0000068590, weight_decay: 0.0000095382

epoch time: 1370.2523193359s, input_fired_time: 19.2870426178, output_fired_time: 6.9850907326, propagate_forward_time: 518.9317016602, propagate_backward_time: 814.4995727539, weight_update_time: 0.4085884094, other_time: 10.5489501953
epoch time: 103.4696502686s, input_fired_time: 1.0302660465, output_fired_time: 0.0000000000, propagate_forward_time: 100.8871765137, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5522079468
validation[          , genome  6272] predictions:    6564/  10000 (65.64%), best:    6564/10000 (65.64%), error:      9906.97363, best error:      9906.97363 on epoch:    69, epoch:   69/100, mu: 0.9811204672, learning_rate: 0.0000064032, weight_decay: 0.0000089990

epoch time: 1310.5083007812s, input_fired_time: 18.7336635590, output_fired_time: 6.7251744270, propagate_forward_time: 493.5362854004, propagate_backward_time: 781.5776367188, weight_update_time: 0.3886458278, other_time: 9.9355468750
epoch time: 99.4159011841s, input_fired_time: 1.0023947954, output_fired_time: 0.0000000000, propagate_forward_time: 96.7082061768, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7052993774
validation[          , genome  6272] predictions:    6601/  10000 (66.01%), best:    6564/10000 (65.64%), error:      9926.99023, best error:      9906.97363 on epoch:    69, epoch:   70/100, mu: 0.9816218019, learning_rate: 0.0000059778, weight_decay: 0.0000084902

epoch time: 1284.6000976562s, input_fired_time: 18.6515502930, output_fired_time: 6.6408514977, propagate_forward_time: 482.9378662109, propagate_backward_time: 766.4299926758, weight_update_time: 0.3873431981, other_time: 9.9397583008
epoch time: 112.1989898682s, input_fired_time: 1.0560606718, output_fired_time: 0.0000000000, propagate_forward_time: 109.6121902466, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5307388306
validation[          , genome  6272] predictions:    6538/  10000 (65.38%), best:    6564/10000 (65.64%), error:      9996.54004, best error:      9906.97363 on epoch:    69, epoch:   71/100, mu: 0.9820948243, learning_rate: 0.0000055806, weight_decay: 0.0000080103

epoch time: 1293.0134277344s, input_fired_time: 18.7011432648, output_fired_time: 6.6595110893, propagate_forward_time: 488.3105468750, propagate_backward_time: 769.4633789062, weight_update_time: 0.3881945610, other_time: 9.8787841797
epoch time: 115.7871627808s, input_fired_time: 1.0723116398, output_fired_time: 0.0000000000, propagate_forward_time: 113.1083679199, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6064834595
validation[          , genome  6272] predictions:    6546/  10000 (65.46%), best:    6564/10000 (65.64%), error:      9913.97754, best error:      9906.97363 on epoch:    69, epoch:   72/100, mu: 0.9825411439, learning_rate: 0.0000052098, weight_decay: 0.0000075574

epoch time: 1261.9954833984s, input_fired_time: 18.6688823700, output_fired_time: 6.6179518700, propagate_forward_time: 472.4099731445, propagate_backward_time: 754.2752075195, weight_update_time: 0.3966206908, other_time: 10.0235595703
epoch time: 98.6372528076s, input_fired_time: 0.9924310446, output_fired_time: 0.0000000000, propagate_forward_time: 95.8808593750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7639617920
validation[          , genome  6272] predictions:    6564/  10000 (65.64%), best:    6564/10000 (65.64%), error:      9875.54395, best error:      9875.54395 on epoch:    73, epoch:   73/100, mu: 0.9829622507, learning_rate: 0.0000048637, weight_decay: 0.0000071302

epoch time: 1279.2122802734s, input_fired_time: 18.7057437897, output_fired_time: 6.7420778275, propagate_forward_time: 476.9720764160, propagate_backward_time: 766.6487426758, weight_update_time: 0.3953667879, other_time: 10.1437377930
epoch time: 94.2075347900s, input_fired_time: 0.9954006672, output_fired_time: 0.0000000000, propagate_forward_time: 91.5765457153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6355895996
validation[          , genome  6272] predictions:    6544/  10000 (65.44%), best:    6564/10000 (65.64%), error:      9902.65918, best error:      9875.54395 on epoch:    73, epoch:   74/100, mu: 0.9833595753, learning_rate: 0.0000045405, weight_decay: 0.0000067271

epoch time: 1281.6226806641s, input_fired_time: 18.6252613068, output_fired_time: 6.5271401405, propagate_forward_time: 485.1680908203, propagate_backward_time: 761.5406494141, weight_update_time: 0.3837303221, other_time: 9.7615966797
epoch time: 91.7623596191s, input_fired_time: 0.9866868258, output_fired_time: 0.0000000000, propagate_forward_time: 89.3457870483, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4298858643
validation[          , genome  6272] predictions:    6540/  10000 (65.40%), best:    6564/10000 (65.64%), error:      9940.92383, best error:      9875.54395 on epoch:    73, epoch:   75/100, mu: 0.9837344885, learning_rate: 0.0000042388, weight_decay: 0.0000063468

epoch time: 1351.5156250000s, input_fired_time: 18.7739048004, output_fired_time: 6.8495945930, propagate_forward_time: 511.6124267578, propagate_backward_time: 804.2946166992, weight_update_time: 0.3908467293, other_time: 9.9850463867
epoch time: 96.3694000244s, input_fired_time: 0.9969817996, output_fired_time: 0.0000000000, propagate_forward_time: 93.6698913574, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7025299072
validation[          , genome  6272] predictions:    6566/  10000 (65.66%), best:    6566/10000 (65.66%), error:      9845.22754, best error:      9845.22754 on epoch:    76, epoch:   76/100, mu: 0.9840882421, learning_rate: 0.0000039572, weight_decay: 0.0000059880

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_188_6272.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_188_6272_4_r1895994517_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.500381
read mu: 0.984422
read mu_delta: 0.943542
read initial_learning_rate: 0.000735664
read learning_rate: 3.69426e-06
read learning_rate_delta: 0.933557
read initial_weight_decay: 0.00049895
read weight_decay: 5.64945e-06
read weight_decay_delta: 0.943467
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0788928
read input_dropout_probability: 0.00107237
read hidden_dropout_probability: 0.0894228
read velocity_reset: 1651
read epoch: 77
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 76
read number_validation_images: 10000
read best_validation_predictions: 6566
read best_validation_error: 9845.23
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6272
read normal distribution: '1 -1.01736629009247 -0.5181605219841'
generator_str: '352327676'
read generator: 352327676
read generated_by_map:
1 crossover 1
reading 90 nodes.
reading 384 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1242.49731445312s, input_fired_time: 18.70654296875, output_fired_time: 6.49713659286499, propagate_forward_time: 456.631134033203, propagate_backward_time: 750.405517578125, weight_update_time: 0.37720137834549, other_time: 10.2569580078125
epoch time: 93.9284057617188s, input_fired_time: 0.986251890659332, output_fired_time: 0, propagate_forward_time: 91.2780914306641, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.6640625
validation[          , genome  6272] predictions:    6584/  10000 (65.84%), best:    6566/10000 (65.66%), error:      9853.60352, best error:      9845.22754 on epoch:    76, epoch:   77/100, mu: 0.9844220281, learning_rate: 0.0000036943, weight_decay: 0.0000056494

epoch time: 1236.3731689453s, input_fired_time: 18.4180583954, output_fired_time: 6.4152278900, propagate_forward_time: 450.4274902344, propagate_backward_time: 751.0046386719, weight_update_time: 0.3653736115, other_time: 10.1076660156
epoch time: 97.5861587524s, input_fired_time: 0.9928082824, output_fired_time: 0.0000000000, propagate_forward_time: 94.8464279175, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7469253540
validation[          , genome  6272] predictions:    6512/  10000 (65.12%), best:    6566/10000 (65.66%), error:      9982.84863, best error:      9845.22754 on epoch:    76, epoch:   78/100, mu: 0.9847369194, learning_rate: 0.0000034488, weight_decay: 0.0000053301

epoch time: 1266.6229248047s, input_fired_time: 18.5440006256, output_fired_time: 6.6663489342, propagate_forward_time: 462.0728149414, propagate_backward_time: 769.0350341797, weight_update_time: 0.3767925501, other_time: 10.3047485352
epoch time: 98.1194381714s, input_fired_time: 0.9984208345, output_fired_time: 0.0000000000, propagate_forward_time: 95.3496704102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7713470459
validation[          , genome  6272] predictions:    6580/  10000 (65.80%), best:    6566/10000 (65.66%), error:      9876.71289, best error:      9845.22754 on epoch:    76, epoch:   79/100, mu: 0.9850340486, learning_rate: 0.0000032197, weight_decay: 0.0000050287

epoch time: 1224.7357177734s, input_fired_time: 18.6607589722, output_fired_time: 6.5127897263, propagate_forward_time: 454.1199340820, propagate_backward_time: 735.3388671875, weight_update_time: 0.3849105537, other_time: 10.1033325195
epoch time: 94.6142425537s, input_fired_time: 0.9946962595, output_fired_time: 0.0000000000, propagate_forward_time: 91.9411926270, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6783523560
validation[          , genome  6272] predictions:    6600/  10000 (66.00%), best:    6600/10000 (66.00%), error:      9838.78711, best error:      9838.78711 on epoch:    80, epoch:   80/100, mu: 0.9853144288, learning_rate: 0.0000030057, weight_decay: 0.0000047444

epoch time: 1256.6883544922s, input_fired_time: 18.4325370789, output_fired_time: 6.5302529335, propagate_forward_time: 457.0189208984, propagate_backward_time: 764.4585571289, weight_update_time: 0.3698319197, other_time: 10.2481079102
epoch time: 90.1909484863s, input_fired_time: 0.9758113027, output_fired_time: 0.0000000000, propagate_forward_time: 87.5659179688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6492156982
validation[          , genome  6272] predictions:    6612/  10000 (66.12%), best:    6612/10000 (66.12%), error:      9821.07812, best error:      9821.07812 on epoch:    81, epoch:   81/100, mu: 0.9855789542, learning_rate: 0.0000028060, weight_decay: 0.0000044762

epoch time: 1225.9102783203s, input_fired_time: 18.4974422455, output_fired_time: 6.3994269371, propagate_forward_time: 450.7119750977, propagate_backward_time: 740.3062133789, weight_update_time: 0.3688117862, other_time: 9.9952392578
epoch time: 97.7865447998s, input_fired_time: 1.0011800528, output_fired_time: 0.0000000000, propagate_forward_time: 95.0168685913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7684936523
validation[          , genome  6272] predictions:    6584/  10000 (65.84%), best:    6612/10000 (66.12%), error:      9873.32520, best error:      9821.07812 on epoch:    81, epoch:   82/100, mu: 0.9858285785, learning_rate: 0.0000026196, weight_decay: 0.0000042232

epoch time: 1169.0833740234s, input_fired_time: 18.5409526825, output_fired_time: 6.1832723618, propagate_forward_time: 431.9453430176, propagate_backward_time: 702.7886352539, weight_update_time: 0.3713432550, other_time: 9.6253051758
epoch time: 94.7508239746s, input_fired_time: 0.9917138815, output_fired_time: 0.0000000000, propagate_forward_time: 92.0625762939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6965332031
validation[          , genome  6272] predictions:    6595/  10000 (65.95%), best:    6612/10000 (66.12%), error:      9831.13281, best error:      9821.07812 on epoch:    81, epoch:   83/100, mu: 0.9860640764, learning_rate: 0.0000024455, weight_decay: 0.0000039844

epoch time: 1228.7769775391s, input_fired_time: 18.6323165894, output_fired_time: 6.5223703384, propagate_forward_time: 453.6717224121, propagate_backward_time: 739.8411254883, weight_update_time: 0.3827663362, other_time: 10.1094360352
epoch time: 98.8647766113s, input_fired_time: 1.0003416538, output_fired_time: 0.0000000000, propagate_forward_time: 96.0884780884, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7759552002
validation[          , genome  6272] predictions:    6608/  10000 (66.08%), best:    6612/10000 (66.12%), error:      9823.31152, best error:      9821.07812 on epoch:    81, epoch:   84/100, mu: 0.9862862825, learning_rate: 0.0000022830, weight_decay: 0.0000037592

epoch time: 1223.4475097656s, input_fired_time: 18.5736789703, output_fired_time: 6.4269900322, propagate_forward_time: 451.3373718262, propagate_backward_time: 737.1135253906, weight_update_time: 0.3762588203, other_time: 9.9958496094
epoch time: 86.0147857666s, input_fired_time: 0.9743025899, output_fired_time: 0.0000000000, propagate_forward_time: 83.5401916504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5002899170
validation[          , genome  6272] predictions:    6605/  10000 (66.05%), best:    6605/10000 (66.05%), error:      9811.69629, best error:      9811.69629 on epoch:    85, epoch:   85/100, mu: 0.9864959717, learning_rate: 0.0000021313, weight_decay: 0.0000035467

epoch time: 1192.0118408203s, input_fired_time: 18.4436225891, output_fired_time: 6.1900968552, propagate_forward_time: 435.2408142090, propagate_backward_time: 722.3970947266, weight_update_time: 0.3652230501, other_time: 9.7402343750
epoch time: 92.2972640991s, input_fired_time: 0.9882997870, output_fired_time: 0.0000000000, propagate_forward_time: 89.6490097046, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6599578857
validation[          , genome  6272] predictions:    6601/  10000 (66.01%), best:    6601/10000 (66.01%), error:      9799.87305, best error:      9799.87305 on epoch:    86, epoch:   86/100, mu: 0.9866937995, learning_rate: 0.0000019897, weight_decay: 0.0000033462

epoch time: 1194.4567871094s, input_fired_time: 18.7700996399, output_fired_time: 6.4288802147, propagate_forward_time: 447.5831298828, propagate_backward_time: 711.8187866211, weight_update_time: 0.3906492591, other_time: 9.8558959961
epoch time: 86.9974288940s, input_fired_time: 0.9808269739, output_fired_time: 0.0000000000, propagate_forward_time: 84.4552612305, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5613403320
validation[          , genome  6272] predictions:    6601/  10000 (66.01%), best:    6601/10000 (66.01%), error:      9822.22559, best error:      9799.87305 on epoch:    86, epoch:   87/100, mu: 0.9868804812, learning_rate: 0.0000018575, weight_decay: 0.0000031570

epoch time: 1203.5775146484s, input_fired_time: 18.5042171478, output_fired_time: 6.3604307175, propagate_forward_time: 442.4786987305, propagate_backward_time: 726.4904785156, weight_update_time: 0.3675474524, other_time: 9.7435913086
epoch time: 89.2886276245s, input_fired_time: 0.9848527908, output_fired_time: 0.0000000000, propagate_forward_time: 86.7316436768, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5721282959
validation[          , genome  6272] predictions:    6599/  10000 (65.99%), best:    6601/10000 (66.01%), error:      9849.39844, best error:      9799.87305 on epoch:    86, epoch:   88/100, mu: 0.9870566130, learning_rate: 0.0000017341, weight_decay: 0.0000029785

epoch time: 1214.7004394531s, input_fired_time: 18.5688209534, output_fired_time: 6.3663101196, propagate_forward_time: 449.3238830566, propagate_backward_time: 730.5190429688, weight_update_time: 0.3811558187, other_time: 9.9223632812
epoch time: 98.8845214844s, input_fired_time: 1.0086747408, output_fired_time: 0.0000000000, propagate_forward_time: 96.0993041992, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7765426636
validation[          , genome  6272] predictions:    6605/  10000 (66.05%), best:    6601/10000 (66.01%), error:      9810.71484, best error:      9799.87305 on epoch:    86, epoch:   89/100, mu: 0.9872227907, learning_rate: 0.0000016189, weight_decay: 0.0000028101

epoch time: 1222.9992675781s, input_fired_time: 18.5098857880, output_fired_time: 6.3971028328, propagate_forward_time: 448.6878051758, propagate_backward_time: 739.3951416016, weight_update_time: 0.3750337958, other_time: 10.0093383789
epoch time: 82.1656341553s, input_fired_time: 0.9581240416, output_fired_time: 0.0000000000, propagate_forward_time: 79.7628097534, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4447021484
validation[          , genome  6272] predictions:    6636/  10000 (66.36%), best:    6601/10000 (66.01%), error:      9801.84668, best error:      9799.87305 on epoch:    86, epoch:   90/100, mu: 0.9873796105, learning_rate: 0.0000015113, weight_decay: 0.0000026513

epoch time: 1226.9429931641s, input_fired_time: 18.5999488831, output_fired_time: 6.4756865501, propagate_forward_time: 451.3721618652, propagate_backward_time: 740.5059814453, weight_update_time: 0.3790656626, other_time: 9.9891357422
epoch time: 92.0423889160s, input_fired_time: 0.9994800687, output_fired_time: 0.0000000000, propagate_forward_time: 89.4616165161, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5812911987
validation[          , genome  6272] predictions:    6609/  10000 (66.09%), best:    6609/10000 (66.09%), error:      9796.39941, best error:      9796.39941 on epoch:    91, epoch:   91/100, mu: 0.9875275493, learning_rate: 0.0000014109, weight_decay: 0.0000025014

epoch time: 1227.2757568359s, input_fired_time: 18.5768947601, output_fired_time: 6.4912743568, propagate_forward_time: 453.9933776855, propagate_backward_time: 738.1623535156, weight_update_time: 0.3816854954, other_time: 10.0517578125
epoch time: 91.1938323975s, input_fired_time: 0.9840958714, output_fired_time: 0.0000000000, propagate_forward_time: 88.5992507935, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6104888916
validation[          , genome  6272] predictions:    6609/  10000 (66.09%), best:    6609/10000 (66.09%), error:      9813.88965, best error:      9796.39941 on epoch:    91, epoch:   92/100, mu: 0.9876671433, learning_rate: 0.0000013172, weight_decay: 0.0000023600

epoch time: 1256.1397705078s, input_fired_time: 18.5734691620, output_fired_time: 6.5759019852, propagate_forward_time: 460.1132507324, propagate_backward_time: 760.6481323242, weight_update_time: 0.3769043386, other_time: 10.2289428711
epoch time: 97.2848358154s, input_fired_time: 1.0018830299, output_fired_time: 0.0000000000, propagate_forward_time: 94.5518798828, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7310714722
validation[          , genome  6272] predictions:    6624/  10000 (66.24%), best:    6624/10000 (66.24%), error:      9782.18457, best error:      9782.18457 on epoch:    93, epoch:   93/100, mu: 0.9877988696, learning_rate: 0.0000012297, weight_decay: 0.0000022265

epoch time: 1217.4250488281s, input_fired_time: 18.6977806091, output_fired_time: 6.4811067581, propagate_forward_time: 451.0869750977, propagate_backward_time: 731.1851806641, weight_update_time: 0.3839853108, other_time: 9.9740600586
epoch time: 92.7084121704s, input_fired_time: 0.9939301014, output_fired_time: 0.0000000000, propagate_forward_time: 90.0680084229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6464767456
validation[          , genome  6272] predictions:    6584/  10000 (65.84%), best:    6624/10000 (66.24%), error:      9816.40527, best error:      9782.18457 on epoch:    93, epoch:   94/100, mu: 0.9879231453, learning_rate: 0.0000011479, weight_decay: 0.0000021007

epoch time: 1196.7309570312s, input_fired_time: 18.5701503754, output_fired_time: 6.3155694008, propagate_forward_time: 441.1353149414, propagate_backward_time: 720.9317016602, weight_update_time: 0.3714230955, other_time: 9.7781982422
epoch time: 99.1282958984s, input_fired_time: 1.0057083368, output_fired_time: 0.0000000000, propagate_forward_time: 96.3468170166, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7757720947
validation[          , genome  6272] predictions:    6635/  10000 (66.35%), best:    6624/10000 (66.24%), error:      9786.78809, best error:      9782.18457 on epoch:    93, epoch:   95/100, mu: 0.9880403876, learning_rate: 0.0000010717, weight_decay: 0.0000019819

epoch time: 1227.7373046875s, input_fired_time: 18.4575996399, output_fired_time: 6.3503098488, propagate_forward_time: 447.3703918457, propagate_backward_time: 745.5562133789, weight_update_time: 0.3669688702, other_time: 10.0027465820
epoch time: 97.8933334351s, input_fired_time: 1.0030341148, output_fired_time: 0.0000000000, propagate_forward_time: 95.1355438232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7547531128
validation[          , genome  6272] predictions:    6609/  10000 (66.09%), best:    6624/10000 (66.24%), error:      9823.37891, best error:      9782.18457 on epoch:    93, epoch:   96/100, mu: 0.9881510139, learning_rate: 0.0000010005, weight_decay: 0.0000018699

epoch time: 1227.5134277344s, input_fired_time: 18.6495208740, output_fired_time: 6.4880347252, propagate_forward_time: 452.9109497070, propagate_backward_time: 739.4608764648, weight_update_time: 0.3771907687, other_time: 10.0040283203
epoch time: 96.3773040771s, input_fired_time: 0.9924985170, output_fired_time: 0.0000000000, propagate_forward_time: 93.6577301025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7270736694
validation[          , genome  6272] predictions:    6618/  10000 (66.18%), best:    6624/10000 (66.24%), error:      9805.64746, best error:      9782.18457 on epoch:    93, epoch:   97/100, mu: 0.9882553816, learning_rate: 0.0000009340, weight_decay: 0.0000017642

epoch time: 1198.3767089844s, input_fired_time: 18.6123504639, output_fired_time: 6.2831015587, propagate_forward_time: 444.6666564941, propagate_backward_time: 718.9235839844, weight_update_time: 0.3755978048, other_time: 9.8911132812
epoch time: 84.1191787720s, input_fired_time: 0.9756436348, output_fired_time: 0.0000000000, propagate_forward_time: 81.6635055542, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4800262451
validation[          , genome  6272] predictions:    6610/  10000 (66.10%), best:    6624/10000 (66.24%), error:      9786.27344, best error:      9782.18457 on epoch:    93, epoch:   98/100, mu: 0.9883539081, learning_rate: 0.0000008719, weight_decay: 0.0000016644

epoch time: 1206.1582031250s, input_fired_time: 18.6265716553, output_fired_time: 6.3668789864, propagate_forward_time: 445.9350280762, propagate_backward_time: 725.2942504883, weight_update_time: 0.3806305230, other_time: 9.9354858398
epoch time: 91.0482330322s, input_fired_time: 1.0092260838, output_fired_time: 0.0000000000, propagate_forward_time: 88.4543304443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5846786499
validation[          , genome  6272] predictions:    6619/  10000 (66.19%), best:    6619/10000 (66.19%), error:      9765.81641, best error:      9765.81641 on epoch:    99, epoch:   99/100, mu: 0.9884468317, learning_rate: 0.0000008140, weight_decay: 0.0000015703

epoch time: 1202.4962158203s, input_fired_time: 18.5567092896, output_fired_time: 6.3111243248, propagate_forward_time: 443.2680358887, propagate_backward_time: 724.4993286133, weight_update_time: 0.3738717139, other_time: 9.8609008789
epoch time: 91.8228378296s, input_fired_time: 0.9990303516, output_fired_time: 0.0000000000, propagate_forward_time: 89.2289276123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5948791504
validation[          , genome  6272] predictions:    6628/  10000 (66.28%), best:    6619/10000 (66.19%), error:      9788.10352, best error:      9765.81641 on epoch:    99, epoch:  100/100, mu: 0.9885345101, learning_rate: 0.0000007599, weight_decay: 0.0000014816

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 449.9054565430s, input_fired_time: 4.9361205101, output_fired_time: 0.0000000000, propagate_forward_time: 436.9486389160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.0206909180
best training[          , genome  6272] predictions:   36050/  50000 (72.10%), best:    6619/10000 (66.19%), error:     42587.22266, best error:      9765.81641 on epoch:    99, epoch:  101/100, mu: 0.9886172414, learning_rate: 0.0000007094, weight_decay: 0.0000013978
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 87.0131149292s, input_fired_time: 0.9912590384, output_fired_time: 0.0000000000, propagate_forward_time: 84.5044403076, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5174179077
   testing[          , genome  6272] predictions:    6619/  10000 (66.19%), best:    6619/10000 (66.19%), error:      9765.81641, best error:      9765.81641 on epoch:    99, epoch:  101/100, mu: 0.9886172414, learning_rate: 0.0000007094, weight_decay: 0.0000013978
backpropagation finished successfully!
08:16:34 (2525): called boinc_finish(0)

</stderr_txt>
]]>