Nameexact_genome_1535107803_159_11299_0
Workunit3924909
Created25 Aug 2018, 20:19:33 UTC
Sent25 Aug 2018, 21:50:23 UTC
Report deadline4 Jan 2019, 15:52:33 UTC
Received1 Sep 2018, 7:11:27 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID56670
Run time2 days 8 hours 26 min 20 sec
CPU time2 days 6 hours 30 min 33 sec
Validate stateInitial
Credit0.00
Device peak FLOPS3.24 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size542.68 MB
Peak swap size564.11 MB
Peak disk usage12.73 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
me: 3.0830459595, other_time: 15.1340332031
epoch time: 140.3326721191s, input_fired_time: 1.6654173136, output_fired_time: 0.0000000000, propagate_forward_time: 136.2859344482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3813171387
validation[          , genome 11299] predictions:    5586/  10000 (55.86%), best:    5670/10000 (56.70%), error:     13744.53027, best error:     13209.05176 on epoch:    21, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1808.9544677734s, input_fired_time: 13.5529670715, output_fired_time: 11.0020313263, propagate_forward_time: 682.5294189453, propagate_backward_time: 1086.7128906250, weight_update_time: 3.0648083687, other_time: 15.1571044922
epoch time: 139.8605499268s, input_fired_time: 1.6555209160, output_fired_time: 0.0000000000, propagate_forward_time: 135.7966918945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4083404541
validation[          , genome 11299] predictions:    5474/  10000 (54.74%), best:    5670/10000 (56.70%), error:     13750.29395, best error:     13209.05176 on epoch:    21, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 1806.2687988281s, input_fired_time: 13.5215301514, output_fired_time: 10.9966554642, propagate_forward_time: 682.0232543945, propagate_backward_time: 1084.6423339844, weight_update_time: 3.0598628521, other_time: 15.0849609375
epoch time: 141.1937255859s, input_fired_time: 1.6986682415, output_fired_time: 0.0000000000, propagate_forward_time: 137.1291503906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3659057617
validation[          , genome 11299] predictions:    5564/  10000 (55.64%), best:    5670/10000 (56.70%), error:     13365.72949, best error:     13209.05176 on epoch:    21, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 1806.9465332031s, input_fired_time: 13.6183309555, output_fired_time: 11.0219125748, propagate_forward_time: 681.0142822266, propagate_backward_time: 1086.1630859375, weight_update_time: 3.0755510330, other_time: 15.1289062500
epoch time: 139.8648986816s, input_fired_time: 1.6937105656, output_fired_time: 0.0000000000, propagate_forward_time: 135.7898559570, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3813323975
validation[          , genome 11299] predictions:    5867/  10000 (58.67%), best:    5867/10000 (58.67%), error:     12806.43164, best error:     12806.43164 on epoch:    25, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 1806.0556640625s, input_fired_time: 13.4648084641, output_fired_time: 10.9614171982, propagate_forward_time: 682.2477416992, propagate_backward_time: 1084.3710937500, weight_update_time: 3.0602827072, other_time: 15.0104980469
epoch time: 141.7364349365s, input_fired_time: 1.7145628929, output_fired_time: 0.0000000000, propagate_forward_time: 137.6227264404, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3991394043
validation[          , genome 11299] predictions:    5551/  10000 (55.51%), best:    5867/10000 (58.67%), error:     13541.33984, best error:     12806.43164 on epoch:    25, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 1859.1055908203s, input_fired_time: 13.9746990204, output_fired_time: 11.3059806824, propagate_forward_time: 699.6210937500, propagate_backward_time: 1118.5358886719, weight_update_time: 3.1493196487, other_time: 15.6678466797
epoch time: 144.4485473633s, input_fired_time: 1.7412134409, output_fired_time: 0.0000000000, propagate_forward_time: 140.2052459717, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5020904541
validation[          , genome 11299] predictions:    5769/  10000 (57.69%), best:    5867/10000 (58.67%), error:     13035.41113, best error:     12806.43164 on epoch:    25, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 1910.7210693359s, input_fired_time: 14.6128616333, output_fired_time: 11.6418218613, propagate_forward_time: 718.2060546875, propagate_backward_time: 1150.0603027344, weight_update_time: 3.2388286591, other_time: 16.1999511719
epoch time: 146.7866058350s, input_fired_time: 1.7762628794, output_fired_time: 0.0000000000, propagate_forward_time: 142.5158233643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4945220947
validation[          , genome 11299] predictions:    5799/  10000 (57.99%), best:    5867/10000 (58.67%), error:     13026.86523, best error:     12806.43164 on epoch:    25, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 1876.5201416016s, input_fired_time: 14.3122835159, output_fired_time: 11.4729251862, propagate_forward_time: 707.7966308594, propagate_backward_time: 1127.0855712891, weight_update_time: 3.1723163128, other_time: 15.8527832031
epoch time: 145.9331817627s, input_fired_time: 1.7903617620, output_fired_time: 0.0000000000, propagate_forward_time: 141.6180725098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5247497559
validation[          , genome 11299] predictions:    5756/  10000 (57.56%), best:    5867/10000 (58.67%), error:     13207.48730, best error:     12806.43164 on epoch:    25, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1925.7845458984s, input_fired_time: 14.6379699707, output_fired_time: 11.7819919586, propagate_forward_time: 724.7073364258, propagate_backward_time: 1157.7974853516, weight_update_time: 3.2057051659, other_time: 16.8597412109
epoch time: 146.3439636230s, input_fired_time: 1.7536224127, output_fired_time: 0.0000000000, propagate_forward_time: 141.9758148193, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6145324707
validation[          , genome 11299] predictions:    5754/  10000 (57.54%), best:    5867/10000 (58.67%), error:     13044.12012, best error:     12806.43164 on epoch:    25, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 1873.1519775391s, input_fired_time: 14.3349981308, output_fired_time: 11.5265941620, propagate_forward_time: 705.1359252930, propagate_backward_time: 1125.7122802734, weight_update_time: 3.2214763165, other_time: 16.4422607422
epoch time: 141.0300750732s, input_fired_time: 1.7331904173, output_fired_time: 0.0000000000, propagate_forward_time: 136.8061981201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4906921387
validation[          , genome 11299] predictions:    5818/  10000 (58.18%), best:    5818/10000 (58.18%), error:     12755.27246, best error:     12755.27246 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 1850.2757568359s, input_fired_time: 14.2150020599, output_fired_time: 11.3125638962, propagate_forward_time: 697.1490478516, propagate_backward_time: 1111.3951416016, weight_update_time: 3.1889460087, other_time: 16.2039794922
epoch time: 142.0798797607s, input_fired_time: 1.7737989426, output_fired_time: 0.0000000000, propagate_forward_time: 137.7977600098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5083160400
validation[          , genome 11299] predictions:    6031/  10000 (60.31%), best:    6031/10000 (60.31%), error:     12169.04785, best error:     12169.04785 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 1917.7178955078s, input_fired_time: 14.7629232407, output_fired_time: 11.8835544586, propagate_forward_time: 719.6321411133, propagate_backward_time: 1154.5032958984, weight_update_time: 3.2283482552, other_time: 16.9359130859
epoch time: 164.8097229004s, input_fired_time: 2.1442894936, output_fired_time: 0.0000000000, propagate_forward_time: 159.6512451172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0141906738
validation[          , genome 11299] predictions:    5764/  10000 (57.64%), best:    6031/10000 (60.31%), error:     12954.35449, best error:     12169.04785 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 1836.5606689453s, input_fired_time: 14.0502805710, output_fired_time: 11.1484479904, propagate_forward_time: 692.7736206055, propagate_backward_time: 1103.1245117188, weight_update_time: 3.1210951805, other_time: 15.4638671875
epoch time: 144.8088836670s, input_fired_time: 1.7545143366, output_fired_time: 0.0000000000, propagate_forward_time: 140.5978393555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4565277100
validation[          , genome 11299] predictions:    5721/  10000 (57.21%), best:    6031/10000 (60.31%), error:     13212.57422, best error:     12169.04785 on epoch:    32, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1810.3312988281s, input_fired_time: 13.6500949860, output_fired_time: 11.0398693085, propagate_forward_time: 683.3222656250, propagate_backward_time: 1087.1892089844, weight_update_time: 3.0685155392, other_time: 15.1297607422
epoch time: 140.3601837158s, input_fired_time: 1.6991540194, output_fired_time: 0.0000000000, propagate_forward_time: 136.2879333496, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3730926514
validation[          , genome 11299] predictions:    5902/  10000 (59.02%), best:    6031/10000 (60.31%), error:     12572.90625, best error:     12169.04785 on epoch:    32, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1807.9708251953s, input_fired_time: 13.7996234894, output_fired_time: 11.0489902496, propagate_forward_time: 682.4276733398, propagate_backward_time: 1085.5373535156, weight_update_time: 3.0709545612, other_time: 15.1572265625
epoch time: 140.8373260498s, input_fired_time: 1.7494717836, output_fired_time: 0.0000000000, propagate_forward_time: 136.7192993164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3685607910
validation[          , genome 11299] predictions:    5836/  10000 (58.36%), best:    6031/10000 (60.31%), error:     12932.25195, best error:     12169.04785 on epoch:    32, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1809.7282714844s, input_fired_time: 13.7315502167, output_fired_time: 11.0072736740, propagate_forward_time: 682.9290161133, propagate_backward_time: 1086.9176025391, weight_update_time: 3.0627660751, other_time: 15.1427001953
epoch time: 141.1103973389s, input_fired_time: 1.7437046766, output_fired_time: 0.0000000000, propagate_forward_time: 136.9642791748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4024200439
validation[          , genome 11299] predictions:    5963/  10000 (59.63%), best:    6031/10000 (60.31%), error:     12209.40039, best error:     12169.04785 on epoch:    32, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1823.6124267578s, input_fired_time: 13.9659271240, output_fired_time: 11.1626882553, propagate_forward_time: 687.4553833008, propagate_backward_time: 1095.8562011719, weight_update_time: 3.0672070980, other_time: 15.1721191406
epoch time: 147.1103820801s, input_fired_time: 1.8305240870, output_fired_time: 0.0000000000, propagate_forward_time: 142.7574768066, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5223846436
validation[          , genome 11299] predictions:    5985/  10000 (59.85%), best:    6031/10000 (60.31%), error:     12208.32422, best error:     12169.04785 on epoch:    32, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 2029.1607666016s, input_fired_time: 15.7272682190, output_fired_time: 12.4622879028, propagate_forward_time: 758.7926025391, propagate_backward_time: 1224.4785156250, weight_update_time: 3.5787847042, other_time: 17.7000732422
epoch time: 247.8775787354s, input_fired_time: 3.2846422195, output_fired_time: 0.0000000000, propagate_forward_time: 239.9351043701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6578369141
validation[          , genome 11299] predictions:    5698/  10000 (56.98%), best:    6031/10000 (60.31%), error:     13298.81738, best error:     12169.04785 on epoch:    32, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1902.8801269531s, input_fired_time: 14.6004800797, output_fired_time: 11.6985464096, propagate_forward_time: 716.0236816406, propagate_backward_time: 1143.8414306641, weight_update_time: 3.2895870209, other_time: 16.7160644531
epoch time: 141.3032989502s, input_fired_time: 1.7371973991, output_fired_time: 0.0000000000, propagate_forward_time: 137.0917968750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4743041992
validation[          , genome 11299] predictions:    5968/  10000 (59.68%), best:    6031/10000 (60.31%), error:     12636.32324, best error:     12169.04785 on epoch:    32, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1849.5333251953s, input_fired_time: 14.1344146729, output_fired_time: 11.3240938187, propagate_forward_time: 696.5509643555, propagate_backward_time: 1111.5668945313, weight_update_time: 3.1725034714, other_time: 15.9570312500
epoch time: 144.8052978516s, input_fired_time: 1.8403129578, output_fired_time: 0.0000000000, propagate_forward_time: 140.3976287842, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5673522949
validation[          , genome 11299] predictions:    5973/  10000 (59.73%), best:    6031/10000 (60.31%), error:     12387.94922, best error:     12169.04785 on epoch:    32, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1858.6187744141s, input_fired_time: 14.2984056473, output_fired_time: 11.3679857254, propagate_forward_time: 699.0973510742, propagate_backward_time: 1118.0664062500, weight_update_time: 3.1576116085, other_time: 15.7885742188
epoch time: 144.6708221436s, input_fired_time: 1.7762039900, output_fired_time: 0.0000000000, propagate_forward_time: 140.4356384277, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4589843750
validation[          , genome 11299] predictions:    5955/  10000 (59.55%), best:    6031/10000 (60.31%), error:     12561.12402, best error:     12169.04785 on epoch:    32, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1865.9072265625s, input_fired_time: 14.4094324112, output_fired_time: 11.4222850800, propagate_forward_time: 702.0892333984, propagate_backward_time: 1122.1904296875, weight_update_time: 3.1516363621, other_time: 15.7958984375
epoch time: 144.7030639648s, input_fired_time: 1.8014761209, output_fired_time: 0.0000000000, propagate_forward_time: 140.4191284180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4824523926
validation[          , genome 11299] predictions:    5981/  10000 (59.81%), best:    6031/10000 (60.31%), error:     12350.35742, best error:     12169.04785 on epoch:    32, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1868.8071289063s, input_fired_time: 14.2990541458, output_fired_time: 11.3995246887, propagate_forward_time: 703.4729614258, propagate_backward_time: 1123.9042968750, weight_update_time: 3.1849632263, other_time: 15.7312011719
epoch time: 145.3082275391s, input_fired_time: 1.8227318525, output_fired_time: 0.0000000000, propagate_forward_time: 141.0111846924, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4743041992
validation[          , genome 11299] predictions:    5994/  10000 (59.94%), best:    6031/10000 (60.31%), error:     12474.03516, best error:     12169.04785 on epoch:    32, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1926.2413330078s, input_fired_time: 15.0027647018, output_fired_time: 11.7730178833, propagate_forward_time: 725.8870849609, propagate_backward_time: 1157.2780761719, weight_update_time: 3.2721989155, other_time: 16.3002929688
epoch time: 145.3053894043s, input_fired_time: 1.8452693224, output_fired_time: 0.0000000000, propagate_forward_time: 140.9629364014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4971771240
validation[          , genome 11299] predictions:    5948/  10000 (59.48%), best:    6031/10000 (60.31%), error:     12868.04590, best error:     12169.04785 on epoch:    32, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 2602.2966308594s, input_fired_time: 19.7693271637, output_fired_time: 15.6058778763, propagate_forward_time: 968.5614624023, propagate_backward_time: 1575.3676757813, weight_update_time: 4.2267847061, other_time: 22.9921875000
epoch time: 188.4663848877s, input_fired_time: 2.3913011551, output_fired_time: 0.0000000000, propagate_forward_time: 182.5699157715, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5051727295
validation[          , genome 11299] predictions:    6112/  10000 (61.12%), best:    6112/10000 (61.12%), error:     12088.56738, best error:     12088.56738 on epoch:    46, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 2982.5917968750s, input_fired_time: 22.3562545776, output_fired_time: 16.8009662628, propagate_forward_time: 1098.4342041016, propagate_backward_time: 1813.8035888672, weight_update_time: 9.4651412964, other_time: 31.1967773438
epoch time: 486.5485229492s, input_fired_time: 7.8936071396, output_fired_time: 0.0000000000, propagate_forward_time: 470.2154846191, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.4394226074
validation[          , genome 11299] predictions:    6029/  10000 (60.29%), best:    6112/10000 (61.12%), error:     12254.98145, best error:     12088.56738 on epoch:    46, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1823.8764648438s, input_fired_time: 14.0535974503, output_fired_time: 11.1212177277, propagate_forward_time: 687.7973022461, propagate_backward_time: 1095.8920898438, weight_update_time: 3.0979907513, other_time: 15.0122070313
epoch time: 143.9523620605s, input_fired_time: 1.8080914021, output_fired_time: 0.0000000000, propagate_forward_time: 139.7494659424, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3948059082
validation[          , genome 11299] predictions:    5990/  10000 (59.90%), best:    6112/10000 (61.12%), error:     12481.97559, best error:     12088.56738 on epoch:    46, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 2002.6392822266s, input_fired_time: 15.2578992844, output_fired_time: 12.0426073074, propagate_forward_time: 749.6765136719, propagate_backward_time: 1209.0849609375, weight_update_time: 3.3806047440, other_time: 16.5772705078
epoch time: 140.0847930908s, input_fired_time: 1.7445666790, output_fired_time: 0.0000000000, propagate_forward_time: 136.0023345947, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3378906250
validation[          , genome 11299] predictions:    5991/  10000 (59.91%), best:    6112/10000 (61.12%), error:     12558.35156, best error:     12088.56738 on epoch:    46, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1843.0877685547s, input_fired_time: 14.3351621628, output_fired_time: 11.1963920593, propagate_forward_time: 694.9998779297, propagate_backward_time: 1107.3514404297, weight_update_time: 3.1050038338, other_time: 15.2048339844
epoch time: 147.9849395752s, input_fired_time: 1.8069491386, output_fired_time: 0.0000000000, propagate_forward_time: 143.7220611572, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4559326172
validation[          , genome 11299] predictions:    6010/  10000 (60.10%), best:    6112/10000 (61.12%), error:     12209.02344, best error:     12088.56738 on epoch:    46, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 1859.7263183594s, input_fired_time: 14.4425573349, output_fired_time: 11.2059078217, propagate_forward_time: 700.5175170898, propagate_backward_time: 1118.0869140625, weight_update_time: 3.1445739269, other_time: 15.4733886719
epoch time: 139.9647521973s, input_fired_time: 1.7256171703, output_fired_time: 0.0000000000, propagate_forward_time: 135.8953704834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3437652588
validation[          , genome 11299] predictions:    6093/  10000 (60.93%), best:    6112/10000 (61.12%), error:     12122.75781, best error:     12088.56738 on epoch:    46, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 1796.7435302734s, input_fired_time: 13.6999521255, output_fired_time: 10.8913125992, propagate_forward_time: 678.6879882813, propagate_backward_time: 1078.7706298828, weight_update_time: 3.0343384743, other_time: 14.6936035156
epoch time: 139.7057189941s, input_fired_time: 1.7197846174, output_fired_time: 0.0000000000, propagate_forward_time: 135.7014160156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2845153809
validation[          , genome 11299] predictions:    6010/  10000 (60.10%), best:    6112/10000 (61.12%), error:     12398.86230, best error:     12088.56738 on epoch:    46, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 1796.7983398438s, input_fired_time: 13.7636661530, output_fired_time: 10.9073934555, propagate_forward_time: 678.6431884766, propagate_backward_time: 1078.7542724609, weight_update_time: 3.0366914272, other_time: 14.7298583984
epoch time: 139.3984985352s, input_fired_time: 1.7424430847, output_fired_time: 0.0000000000, propagate_forward_time: 135.3638458252, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2922058105
validation[          , genome 11299] predictions:    6049/  10000 (60.49%), best:    6112/10000 (61.12%), error:     12398.03906, best error:     12088.56738 on epoch:    46, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 1797.4691162109s, input_fired_time: 13.7096967697, output_fired_time: 10.9600687027, propagate_forward_time: 678.5744018555, propagate_backward_time: 1079.1657714844, weight_update_time: 3.0300538540, other_time: 15.0590820313
epoch time: 139.7069396973s, input_fired_time: 1.7358433008, output_fired_time: 0.0000000000, propagate_forward_time: 135.5821075439, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3889923096
validation[          , genome 11299] predictions:    6050/  10000 (60.50%), best:    6112/10000 (61.12%), error:     12729.38867, best error:     12088.56738 on epoch:    46, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 1797.8262939453s, input_fired_time: 13.7837991714, output_fired_time: 10.9255685806, propagate_forward_time: 678.9336547852, propagate_backward_time: 1078.9943847656, weight_update_time: 3.0354926586, other_time: 15.1889648438
epoch time: 139.7936706543s, input_fired_time: 1.7191176414, output_fired_time: 0.0000000000, propagate_forward_time: 135.6960296631, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3785247803
validation[          , genome 11299] predictions:    6112/  10000 (61.12%), best:    6112/10000 (61.12%), error:     12103.86328, best error:     12088.56738 on epoch:    46, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 1796.5596923828s, input_fired_time: 13.7362689972, output_fired_time: 10.9306869507, propagate_forward_time: 678.1488647461, propagate_backward_time: 1078.6245117188, weight_update_time: 3.0460453033, other_time: 15.1193847656
epoch time: 140.2649230957s, input_fired_time: 1.7887945175, output_fired_time: 0.0000000000, propagate_forward_time: 136.1086578369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3674774170
validation[          , genome 11299] predictions:    6212/  10000 (62.12%), best:    6212/10000 (62.12%), error:     11626.75879, best error:     11626.75879 on epoch:    56, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 1797.7413330078s, input_fired_time: 13.8548545837, output_fired_time: 10.9538583755, propagate_forward_time: 678.8964233398, propagate_backward_time: 1079.0678710938, weight_update_time: 3.0276160240, other_time: 14.9682617188
epoch time: 139.6159667969s, input_fired_time: 1.7581990957, output_fired_time: 0.0000000000, propagate_forward_time: 135.5083007813, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3494720459
validation[          , genome 11299] predictions:    6145/  10000 (61.45%), best:    6212/10000 (62.12%), error:     11972.57715, best error:     11626.75879 on epoch:    56, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1796.4223632813s, input_fired_time: 13.8555927277, output_fired_time: 10.9479627609, propagate_forward_time: 677.9342041016, propagate_backward_time: 1078.8485107422, weight_update_time: 3.0388290882, other_time: 14.8360595703
epoch time: 139.8275451660s, input_fired_time: 1.7607973814, output_fired_time: 0.0000000000, propagate_forward_time: 135.7422790527, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3244628906
validation[          , genome 11299] predictions:    6153/  10000 (61.53%), best:    6212/10000 (62.12%), error:     11900.08594, best error:     11626.75879 on epoch:    56, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 1796.5037841797s, input_fired_time: 13.8860702515, output_fired_time: 10.8798408508, propagate_forward_time: 678.3607788086, propagate_backward_time: 1078.6118164063, weight_update_time: 3.0306665897, other_time: 14.7651367188
epoch time: 139.3450469971s, input_fired_time: 1.7618672848, output_fired_time: 0.0000000000, propagate_forward_time: 135.2812805176, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3018951416
validation[          , genome 11299] predictions:    6184/  10000 (61.84%), best:    6212/10000 (62.12%), error:     11939.09961, best error:     11626.75879 on epoch:    56, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 1796.3662109375s, input_fired_time: 13.8464002609, output_fired_time: 10.8965148926, propagate_forward_time: 678.1043090820, propagate_backward_time: 1078.8032226563, weight_update_time: 3.0345320702, other_time: 14.7158203125
epoch time: 139.5107727051s, input_fired_time: 1.7697149515, output_fired_time: 0.0000000000, propagate_forward_time: 135.3870086670, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3540496826
validation[          , genome 11299] predictions:    6114/  10000 (61.14%), best:    6212/10000 (62.12%), error:     11934.25879, best error:     11626.75879 on epoch:    56, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 1797.3428955078s, input_fired_time: 13.9355955124, output_fired_time: 10.9169225693, propagate_forward_time: 679.0323486328, propagate_backward_time: 1078.7315673828, weight_update_time: 3.0392286777, other_time: 14.7265625000
epoch time: 139.6022949219s, input_fired_time: 1.7729458809, output_fired_time: 0.0000000000, propagate_forward_time: 135.5332794189, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2960662842
validation[          , genome 11299] predictions:    6113/  10000 (61.13%), best:    6212/10000 (62.12%), error:     12072.42871, best error:     11626.75879 on epoch:    56, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 1796.5631103516s, input_fired_time: 13.9209518433, output_fired_time: 10.8941459656, propagate_forward_time: 678.7124633789, propagate_backward_time: 1078.4084472656, weight_update_time: 3.0395126343, other_time: 14.6271972656
epoch time: 139.7619476318s, input_fired_time: 1.7549304962, output_fired_time: 0.0000000000, propagate_forward_time: 135.7083129883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2987060547
validation[          , genome 11299] predictions:    6149/  10000 (61.49%), best:    6212/10000 (62.12%), error:     12011.66504, best error:     11626.75879 on epoch:    56, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 1796.1625976563s, input_fired_time: 13.9145660400, output_fired_time: 10.8875226974, propagate_forward_time: 678.1636962891, propagate_backward_time: 1078.5716552734, weight_update_time: 3.0344748497, other_time: 14.6251220703
epoch time: 139.9034271240s, input_fired_time: 1.7818210125, output_fired_time: 0.0000000000, propagate_forward_time: 135.8092803955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3123321533
validation[          , genome 11299] predictions:    6124/  10000 (61.24%), best:    6212/10000 (62.12%), error:     12191.22168, best error:     11626.75879 on epoch:    56, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 1796.5832519531s, input_fired_time: 13.9644918442, output_fired_time: 10.9023561478, propagate_forward_time: 678.6424560547, propagate_backward_time: 1078.3820800781, weight_update_time: 3.0374815464, other_time: 14.6918945313
epoch time: 139.5424652100s, input_fired_time: 1.7466244698, output_fired_time: 0.0000000000, propagate_forward_time: 135.5466003418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2492370605
validation[          , genome 11299] predictions:    6210/  10000 (62.10%), best:    6212/10000 (62.12%), error:     11805.87500, best error:     11626.75879 on epoch:    56, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 1798.1311035156s, input_fired_time: 13.9478807449, output_fired_time: 10.9014062881, propagate_forward_time: 679.2242431641, propagate_backward_time: 1079.3619384766, weight_update_time: 3.0484740734, other_time: 14.6956787109
epoch time: 139.7644042969s, input_fired_time: 1.8182883263, output_fired_time: 0.0000000000, propagate_forward_time: 135.6140289307, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3320922852
validation[          , genome 11299] predictions:    6163/  10000 (61.63%), best:    6212/10000 (62.12%), error:     12088.35547, best error:     11626.75879 on epoch:    56, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 1796.4351806641s, input_fired_time: 13.9533433914, output_fired_time: 10.8867387772, propagate_forward_time: 678.6431274414, propagate_backward_time: 1078.2730712891, weight_update_time: 3.0244286060, other_time: 14.6788330078
epoch time: 139.9883422852s, input_fired_time: 1.7958854437, output_fired_time: 0.0000000000, propagate_forward_time: 135.8816680908, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3107910156
validation[          , genome 11299] predictions:    6232/  10000 (62.32%), best:    6212/10000 (62.12%), error:     11953.27051, best error:     11626.75879 on epoch:    56, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 1797.5675048828s, input_fired_time: 13.9523153305, output_fired_time: 10.9241313934, propagate_forward_time: 678.9654541016, propagate_backward_time: 1078.7786865234, weight_update_time: 3.0345005989, other_time: 14.9470214844
epoch time: 139.6669311523s, input_fired_time: 1.7762472630, output_fired_time: 0.0000000000, propagate_forward_time: 135.5144042969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3762817383
validation[          , genome 11299] predictions:    6158/  10000 (61.58%), best:    6212/10000 (62.12%), error:     12033.24902, best error:     11626.75879 on epoch:    56, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 1797.0024414063s, input_fired_time: 13.9924592972, output_fired_time: 10.9144258499, propagate_forward_time: 678.1086425781, propagate_backward_time: 1078.8906250000, weight_update_time: 3.0362696648, other_time: 15.0963134766
epoch time: 139.7498474121s, input_fired_time: 1.7956360579, output_fired_time: 0.0000000000, propagate_forward_time: 135.5685119629, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3856964111
validation[          , genome 11299] predictions:    6211/  10000 (62.11%), best:    6212/10000 (62.12%), error:     11792.08691, best error:     11626.75879 on epoch:    56, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 1850.8173828125s, input_fired_time: 14.5342111588, output_fired_time: 11.3205184937, propagate_forward_time: 696.1235961914, propagate_backward_time: 1113.0766601563, weight_update_time: 3.1578457355, other_time: 15.7624511719
epoch time: 149.0468902588s, input_fired_time: 1.9023913145, output_fired_time: 0.0000000000, propagate_forward_time: 144.5287017822, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6157989502
validation[          , genome 11299] predictions:    6114/  10000 (61.14%), best:    6212/10000 (62.12%), error:     12059.42969, best error:     11626.75879 on epoch:    56, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535107803_159_11299.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535107803_159_11299_0_r316615232_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.976484
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000344796
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.37918e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 70
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 56
read number_validation_images: 10000
read best_validation_predictions: 6212
read best_validation_error: 11626.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11299
read normal distribution: '1 0 0'
generator_str: '40075019'
read generator: 40075019
read generated_by_map:
1 crossover 1
reading 119 nodes.
reading 457 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1899.57946777344s, input_fired_time: 15.047046661377, output_fired_time: 11.7515649795532, propagate_forward_time: 714.083862304688, propagate_backward_time: 1142.60021972656, weight_update_time: 3.21632981300354, other_time: 16.0968017578125
epoch time: 153.186431884766s, input_fired_time: 1.896568775177, output_fired_time: 0, propagate_forward_time: 148.767822265625, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.52203369140625
validation[          , genome 11299] predictions:    6132/  10000 (61.32%), best:    6212/10000 (62.12%), error:     12108.38770, best error:     11626.75879 on epoch:    56, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 1909.5637207031s, input_fired_time: 15.1840019226, output_fired_time: 11.6949424744, propagate_forward_time: 718.3073730469, propagate_backward_time: 1148.4926757813, weight_update_time: 3.1923027039, other_time: 15.8847656250
epoch time: 150.6724090576s, input_fired_time: 1.9443025589, output_fired_time: 0.0000000000, propagate_forward_time: 146.1686706543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5594329834
validation[          , genome 11299] predictions:    6150/  10000 (61.50%), best:    6212/10000 (62.12%), error:     11863.02637, best error:     11626.75879 on epoch:    56, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 1907.1752929688s, input_fired_time: 15.0310020447, output_fired_time: 11.5927095413, propagate_forward_time: 716.8889770508, propagate_backward_time: 1147.6013183594, weight_update_time: 3.2765042782, other_time: 16.0612792969
epoch time: 144.2597961426s, input_fired_time: 1.8644739389, output_fired_time: 0.0000000000, propagate_forward_time: 140.0030212402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3923034668
validation[          , genome 11299] predictions:    6231/  10000 (62.31%), best:    6212/10000 (62.12%), error:     11881.27441, best error:     11626.75879 on epoch:    56, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 2136.0549316406s, input_fired_time: 16.9756736755, output_fired_time: 12.9807014465, propagate_forward_time: 801.2774047852, propagate_backward_time: 1286.7668457031, weight_update_time: 3.5859520435, other_time: 18.0544433594
epoch time: 151.7645721436s, input_fired_time: 1.9452389479, output_fired_time: 0.0000000000, propagate_forward_time: 147.2253265381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5940093994
validation[          , genome 11299] predictions:    6175/  10000 (61.75%), best:    6212/10000 (62.12%), error:     11944.61523, best error:     11626.75879 on epoch:    56, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 1846.6307373047s, input_fired_time: 14.5384550095, output_fired_time: 11.3500204086, propagate_forward_time: 696.8685302734, propagate_backward_time: 1108.0599365234, weight_update_time: 3.1173384190, other_time: 15.8138427734
epoch time: 140.9086608887s, input_fired_time: 1.8080149889, output_fired_time: 0.0000000000, propagate_forward_time: 136.7091979980, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3914489746
validation[          , genome 11299] predictions:    6265/  10000 (62.65%), best:    6212/10000 (62.12%), error:     11785.69824, best error:     11626.75879 on epoch:    56, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 1834.8374023438s, input_fired_time: 14.4167003632, output_fired_time: 11.2391538620, propagate_forward_time: 691.9461059570, propagate_backward_time: 1101.7618408203, weight_update_time: 3.1145567894, other_time: 15.4735107422
epoch time: 140.4874420166s, input_fired_time: 1.8049900532, output_fired_time: 0.0000000000, propagate_forward_time: 136.3644866943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3179626465
validation[          , genome 11299] predictions:    6146/  10000 (61.46%), best:    6212/10000 (62.12%), error:     12221.17676, best error:     11626.75879 on epoch:    56, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 1797.0511474609s, input_fired_time: 14.0625171661, output_fired_time: 10.9291896820, propagate_forward_time: 678.5786743164, propagate_backward_time: 1078.5989990234, weight_update_time: 3.0299613476, other_time: 14.8817138672
epoch time: 139.6434173584s, input_fired_time: 1.7829473019, output_fired_time: 0.0000000000, propagate_forward_time: 135.5530242920, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3074493408
validation[          , genome 11299] predictions:    6212/  10000 (62.12%), best:    6212/10000 (62.12%), error:     11929.00977, best error:     11626.75879 on epoch:    56, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 1838.0424804688s, input_fired_time: 14.3987951279, output_fired_time: 11.2464370728, propagate_forward_time: 692.9690551758, propagate_backward_time: 1104.2052001953, weight_update_time: 3.1255424023, other_time: 15.2230224609
epoch time: 141.5847930908s, input_fired_time: 1.8148274422, output_fired_time: 0.0000000000, propagate_forward_time: 137.4137268066, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3562316895
validation[          , genome 11299] predictions:    6227/  10000 (62.27%), best:    6212/10000 (62.12%), error:     11931.69922, best error:     11626.75879 on epoch:    56, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 1850.0430908203s, input_fired_time: 14.6395854950, output_fired_time: 11.2257175446, propagate_forward_time: 696.2213745117, propagate_backward_time: 1112.6223144531, weight_update_time: 3.1461086273, other_time: 15.3342285156
epoch time: 150.3533020020s, input_fired_time: 1.9844590425, output_fired_time: 0.0000000000, propagate_forward_time: 145.8482971191, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5205383301
validation[          , genome 11299] predictions:    6237/  10000 (62.37%), best:    6212/10000 (62.12%), error:     11867.59961, best error:     11626.75879 on epoch:    56, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1880.4006347656s, input_fired_time: 14.8925619125, output_fired_time: 11.5038328171, propagate_forward_time: 705.9708251953, propagate_backward_time: 1132.4058837891, weight_update_time: 3.1319274902, other_time: 15.6275634766
epoch time: 139.4458465576s, input_fired_time: 1.7939229012, output_fired_time: 0.0000000000, propagate_forward_time: 135.3409423828, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3109741211
validation[          , genome 11299] predictions:    6239/  10000 (62.39%), best:    6212/10000 (62.12%), error:     11756.29004, best error:     11626.75879 on epoch:    56, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1867.6214599609s, input_fired_time: 14.7401771545, output_fired_time: 11.2734899521, propagate_forward_time: 702.5820312500, propagate_backward_time: 1123.4709472656, weight_update_time: 3.1705789566, other_time: 15.5548095703
epoch time: 192.6940155029s, input_fired_time: 2.6393568516, output_fired_time: 0.0000000000, propagate_forward_time: 186.5642395020, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4904174805
validation[          , genome 11299] predictions:    6237/  10000 (62.37%), best:    6212/10000 (62.12%), error:     11925.33887, best error:     11626.75879 on epoch:    56, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1907.4080810547s, input_fired_time: 15.2297468185, output_fired_time: 11.4194898605, propagate_forward_time: 716.5719604492, propagate_backward_time: 1148.3223876953, weight_update_time: 3.2747993469, other_time: 15.8646240234
epoch time: 147.7127380371s, input_fired_time: 1.8890851736, output_fired_time: 0.0000000000, propagate_forward_time: 143.2933654785, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5302886963
validation[          , genome 11299] predictions:    6177/  10000 (61.77%), best:    6212/10000 (62.12%), error:     12118.37109, best error:     11626.75879 on epoch:    56, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1842.8529052734s, input_fired_time: 14.6092939377, output_fired_time: 11.2651271820, propagate_forward_time: 693.4998168945, propagate_backward_time: 1107.9888916016, weight_update_time: 3.1246864796, other_time: 15.4898681641
epoch time: 144.0081634521s, input_fired_time: 1.8764483929, output_fired_time: 0.0000000000, propagate_forward_time: 139.7123413086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4193725586
validation[          , genome 11299] predictions:    6231/  10000 (62.31%), best:    6212/10000 (62.12%), error:     11917.52930, best error:     11626.75879 on epoch:    56, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1863.4044189453s, input_fired_time: 14.8338766098, output_fired_time: 11.4065942764, propagate_forward_time: 700.5661010742, propagate_backward_time: 1120.8262939453, weight_update_time: 3.1941609383, other_time: 15.7716064453
epoch time: 140.6807861328s, input_fired_time: 1.8129910231, output_fired_time: 0.0000000000, propagate_forward_time: 136.4841308594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3836669922
validation[          , genome 11299] predictions:    6266/  10000 (62.66%), best:    6212/10000 (62.12%), error:     11691.38965, best error:     11626.75879 on epoch:    56, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1806.8337402344s, input_fired_time: 14.2513628006, output_fired_time: 11.0109786987, propagate_forward_time: 681.8422851563, propagate_backward_time: 1084.5192871094, weight_update_time: 3.0590305328, other_time: 15.2098388672
epoch time: 141.6513519287s, input_fired_time: 1.8275846243, output_fired_time: 0.0000000000, propagate_forward_time: 137.4172668457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4064941406
validation[          , genome 11299] predictions:    6227/  10000 (62.27%), best:    6212/10000 (62.12%), error:     12158.18750, best error:     11626.75879 on epoch:    56, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 1807.6818847656s, input_fired_time: 14.1611652374, output_fired_time: 11.0411548615, propagate_forward_time: 682.5368041992, propagate_backward_time: 1084.6563720703, weight_update_time: 3.0572276115, other_time: 15.2864990234
epoch time: 141.0074462891s, input_fired_time: 1.8342785835, output_fired_time: 0.0000000000, propagate_forward_time: 136.7609710693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4122009277
validation[          , genome 11299] predictions:    6272/  10000 (62.72%), best:    6212/10000 (62.12%), error:     11790.14160, best error:     11626.75879 on epoch:    56, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1811.0678710938s, input_fired_time: 14.1944684982, output_fired_time: 11.0808763504, propagate_forward_time: 683.1215209961, propagate_backward_time: 1087.3066406250, weight_update_time: 3.0523424149, other_time: 15.3642578125
epoch time: 140.8282012939s, input_fired_time: 1.8416440487, output_fired_time: 0.0000000000, propagate_forward_time: 136.5749816895, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4115753174
validation[          , genome 11299] predictions:    6218/  10000 (62.18%), best:    6212/10000 (62.12%), error:     11897.52734, best error:     11626.75879 on epoch:    56, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1814.0784912109s, input_fired_time: 14.3013505936, output_fired_time: 11.1250152588, propagate_forward_time: 684.9842529297, propagate_backward_time: 1088.1079101563, weight_update_time: 3.0545928478, other_time: 15.5599365234
epoch time: 141.3895416260s, input_fired_time: 1.8433399200, output_fired_time: 0.0000000000, propagate_forward_time: 137.0710144043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4751892090
validation[          , genome 11299] predictions:    6248/  10000 (62.48%), best:    6212/10000 (62.12%), error:     11943.38477, best error:     11626.75879 on epoch:    56, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1814.3349609375s, input_fired_time: 14.3373289108, output_fired_time: 11.1031885147, propagate_forward_time: 684.1288452148, propagate_backward_time: 1089.0043945313, weight_update_time: 3.0688953400, other_time: 15.7612304688
epoch time: 141.5385742188s, input_fired_time: 1.8529394865, output_fired_time: 0.0000000000, propagate_forward_time: 137.1750793457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5105590820
validation[          , genome 11299] predictions:    6262/  10000 (62.62%), best:    6212/10000 (62.12%), error:     11799.74707, best error:     11626.75879 on epoch:    56, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1810.5821533203s, input_fired_time: 14.3146343231, output_fired_time: 11.0328044891, propagate_forward_time: 683.2775268555, propagate_backward_time: 1086.2994384766, weight_update_time: 3.0682139397, other_time: 15.6578369141
epoch time: 140.5672760010s, input_fired_time: 1.8217580318, output_fired_time: 0.0000000000, propagate_forward_time: 136.2648010254, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4807128906
validation[          , genome 11299] predictions:    6262/  10000 (62.62%), best:    6212/10000 (62.12%), error:     11909.86523, best error:     11626.75879 on epoch:    56, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1810.0565185547s, input_fired_time: 14.3046808243, output_fired_time: 11.0184259415, propagate_forward_time: 682.5842895508, propagate_backward_time: 1086.4416503906, weight_update_time: 3.0773084164, other_time: 15.7075195313
epoch time: 140.3282623291s, input_fired_time: 1.8248462677, output_fired_time: 0.0000000000, propagate_forward_time: 136.0455780029, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4578399658
validation[          , genome 11299] predictions:    6255/  10000 (62.55%), best:    6212/10000 (62.12%), error:     12046.69727, best error:     11626.75879 on epoch:    56, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1806.9973144531s, input_fired_time: 14.2824735641, output_fired_time: 11.0064134598, propagate_forward_time: 681.4837646484, propagate_backward_time: 1084.6494140625, weight_update_time: 3.0465762615, other_time: 15.5751953125
epoch time: 140.7869415283s, input_fired_time: 1.8375765085, output_fired_time: 0.0000000000, propagate_forward_time: 136.4702911377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4790802002
validation[          , genome 11299] predictions:    6193/  10000 (61.93%), best:    6212/10000 (62.12%), error:     12256.13184, best error:     11626.75879 on epoch:    56, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1814.6773681641s, input_fired_time: 14.3868303299, output_fired_time: 11.0639743805, propagate_forward_time: 685.1054077148, propagate_backward_time: 1088.4989013672, weight_update_time: 3.0784983635, other_time: 15.6221923828
epoch time: 141.8785400391s, input_fired_time: 1.8672493696, output_fired_time: 0.0000000000, propagate_forward_time: 137.5178222656, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4934692383
validation[          , genome 11299] predictions:    6251/  10000 (62.51%), best:    6212/10000 (62.12%), error:     11867.98730, best error:     11626.75879 on epoch:    56, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1809.0153808594s, input_fired_time: 14.2812175751, output_fired_time: 11.0031862259, propagate_forward_time: 682.1259155273, propagate_backward_time: 1086.1218261719, weight_update_time: 3.0402276516, other_time: 15.4831542969
epoch time: 140.7959442139s, input_fired_time: 1.8429867029, output_fired_time: 0.0000000000, propagate_forward_time: 136.4915771484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4613800049
validation[          , genome 11299] predictions:    6264/  10000 (62.64%), best:    6212/10000 (62.12%), error:     11926.12402, best error:     11626.75879 on epoch:    56, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1809.2248535156s, input_fired_time: 14.2990455627, output_fired_time: 11.0115137100, propagate_forward_time: 682.3201904297, propagate_backward_time: 1086.0874023438, weight_update_time: 3.0568773746, other_time: 15.5067138672
epoch time: 141.0019531250s, input_fired_time: 1.8393452168, output_fired_time: 0.0000000000, propagate_forward_time: 136.7237243652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4388885498
validation[          , genome 11299] predictions:    6307/  10000 (63.07%), best:    6212/10000 (62.12%), error:     11833.79199, best error:     11626.75879 on epoch:    56, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1812.9057617188s, input_fired_time: 14.3338670731, output_fired_time: 11.0970220566, propagate_forward_time: 683.6063842773, propagate_backward_time: 1088.2287597656, weight_update_time: 3.0790638924, other_time: 15.6396484375
epoch time: 141.6686706543s, input_fired_time: 1.8383884430, output_fired_time: 0.0000000000, propagate_forward_time: 137.3588714600, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4714050293
validation[          , genome 11299] predictions:    6229/  10000 (62.29%), best:    6212/10000 (62.12%), error:     12094.54980, best error:     11626.75879 on epoch:    56, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1815.1569824219s, input_fired_time: 14.3322629929, output_fired_time: 11.0915374756, propagate_forward_time: 685.1871948242, propagate_backward_time: 1088.7076416016, weight_update_time: 3.0675852299, other_time: 15.8382568359
epoch time: 142.0518646240s, input_fired_time: 1.8626343012, output_fired_time: 0.0000000000, propagate_forward_time: 137.6626129150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5266113281
validation[          , genome 11299] predictions:    6237/  10000 (62.37%), best:    6212/10000 (62.12%), error:     12036.05664, best error:     11626.75879 on epoch:    56, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1815.1113281250s, input_fired_time: 14.3894414902, output_fired_time: 11.0929965973, propagate_forward_time: 684.7255859375, propagate_backward_time: 1089.1125488281, weight_update_time: 3.0841417313, other_time: 15.7907714844
epoch time: 140.9828186035s, input_fired_time: 1.8161646128, output_fired_time: 0.0000000000, propagate_forward_time: 136.6672515869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4994049072
validation[          , genome 11299] predictions:    6204/  10000 (62.04%), best:    6212/10000 (62.12%), error:     12140.76758, best error:     11626.75879 on epoch:    56, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1807.7640380859s, input_fired_time: 14.2605590820, output_fired_time: 11.0816392899, propagate_forward_time: 681.9347534180, propagate_backward_time: 1084.7332763672, weight_update_time: 3.0678584576, other_time: 15.7537841797
epoch time: 141.0674133301s, input_fired_time: 1.8650060892, output_fired_time: 0.0000000000, propagate_forward_time: 136.7184600830, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4839477539
validation[          , genome 11299] predictions:    6305/  10000 (63.05%), best:    6212/10000 (62.12%), error:     11992.31836, best error:     11626.75879 on epoch:    56, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1811.5410156250s, input_fired_time: 14.3410730362, output_fired_time: 11.0738811493, propagate_forward_time: 682.7752075195, propagate_backward_time: 1087.5830078125, weight_update_time: 3.0678133965, other_time: 15.7678222656
epoch time: 141.9028167725s, input_fired_time: 1.8513789177, output_fired_time: 0.0000000000, propagate_forward_time: 137.5658416748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4855957031
validation[          , genome 11299] predictions:    6269/  10000 (62.69%), best:    6212/10000 (62.12%), error:     11894.34570, best error:     11626.75879 on epoch:    56, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1810.4940185547s, input_fired_time: 14.3311691284, output_fired_time: 11.0309715271, propagate_forward_time: 682.9757690430, propagate_backward_time: 1086.4970703125, weight_update_time: 3.0543234348, other_time: 15.6589355469
epoch time: 141.0812225342s, input_fired_time: 1.8302544355, output_fired_time: 0.0000000000, propagate_forward_time: 136.7612304688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4897308350
validation[          , genome 11299] predictions:    6240/  10000 (62.40%), best:    6212/10000 (62.12%), error:     11964.65918, best error:     11626.75879 on epoch:    56, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 707.9758300781s, input_fired_time: 8.8711538315, output_fired_time: 0.0000000000, propagate_forward_time: 686.5114135742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.5932617188
best training[          , genome 11299] predictions:   39018/  50000 (78.04%), best:    6212/10000 (62.12%), error:     31993.96289, best error:     11626.75879 on epoch:    56, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 141.1190338135s, input_fired_time: 1.7772833109, output_fired_time: 0.0000000000, propagate_forward_time: 136.8576507568, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4841003418
   testing[          , genome 11299] predictions:    6212/  10000 (62.12%), best:    6212/10000 (62.12%), error:     11626.75879, best error:     11626.75879 on epoch:    56, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
01:11:03 (6468): called boinc_finish(0)

</stderr_txt>
]]>