Nameexact_genome_1529063102_163_6778_1
Workunit3816711
Created20 Jun 2018, 22:02:18 UTC
Sent20 Jun 2018, 23:32:45 UTC
Report deadline17 Oct 2018, 8:30:44 UTC
Received27 Oct 2018, 2:01:46 UTC
Server stateOver
OutcomeComputation error
Client stateAborted by user
Exit status203 (0xcb) EXIT_ABORTED_VIA_GUI
Computer ID60660
Run time1 days 1 hours 30 min 5 sec
CPU time1 days 0 hours 44 min 49 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS3.49 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size556.61 MB
Peak swap size576.04 MB
Peak disk usage11.59 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
te_time: 0.0000000000, other_time: 2.3579940796
validation[          , genome  6778] predictions:    5782/  10000 (57.82%), best:    5867/10000 (58.67%), error:     12349.72363, best error:     12077.72461 on epoch:    11, epoch:   13/100, mu: 0.7463580966, learning_rate: 0.0001313397, weight_decay: 0.0002038646

epoch time: 1219.4019775391s, input_fired_time: 20.9976863861, output_fired_time: 7.2998003960, propagate_forward_time: 445.8791503906, propagate_backward_time: 732.4006958008, weight_update_time: 0.3428357244, other_time: 12.8246459961
epoch time: 87.5880889893s, input_fired_time: 1.1506079435, output_fired_time: 0.0000000000, propagate_forward_time: 84.4402694702, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9972152710
validation[          , genome  6778] predictions:    5897/  10000 (58.97%), best:    5867/10000 (58.67%), error:     12115.46582, best error:     12077.72461 on epoch:    11, epoch:   14/100, mu: 0.7591232061, learning_rate: 0.0001227540, weight_decay: 0.0001918742

epoch time: 1168.9387207031s, input_fired_time: 19.8397979736, output_fired_time: 6.9679646492, propagate_forward_time: 429.2520751953, propagate_backward_time: 700.6170043945, weight_update_time: 0.3342211246, other_time: 12.2617797852
epoch time: 87.5115966797s, input_fired_time: 1.1516082287, output_fired_time: 0.0000000000, propagate_forward_time: 84.3159790039, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0440063477
validation[          , genome  6778] predictions:    5808/  10000 (58.08%), best:    5867/10000 (58.67%), error:     12242.30664, best error:     12077.72461 on epoch:    11, epoch:   15/100, mu: 0.7712194920, learning_rate: 0.0001147296, weight_decay: 0.0001805890

epoch time: 1237.2340087891s, input_fired_time: 21.0534954071, output_fired_time: 7.2667312622, propagate_forward_time: 454.5852966309, propagate_backward_time: 741.5092773438, weight_update_time: 0.3552820385, other_time: 12.8192138672
epoch time: 89.1114959717s, input_fired_time: 1.1692116261, output_fired_time: 0.0000000000, propagate_forward_time: 85.8822021484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0600814819
validation[          , genome  6778] predictions:    5962/  10000 (59.62%), best:    5962/10000 (59.62%), error:     11864.63574, best error:     11864.63574 on epoch:    16, epoch:   16/100, mu: 0.7826820016, learning_rate: 0.0001072297, weight_decay: 0.0001699676

epoch time: 1174.0489501953s, input_fired_time: 20.0151386261, output_fired_time: 7.0000953674, propagate_forward_time: 431.6402893066, propagate_backward_time: 703.2866821289, weight_update_time: 0.3315168917, other_time: 12.1067504883
epoch time: 89.2902526855s, input_fired_time: 1.1773970127, output_fired_time: 0.0000000000, propagate_forward_time: 86.0716629028, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0411911011
validation[          , genome  6778] predictions:    5821/  10000 (58.21%), best:    5962/10000 (59.62%), error:     12066.19824, best error:     11864.63574 on epoch:    16, epoch:   17/100, mu: 0.7935439944, learning_rate: 0.0001002201, weight_decay: 0.0001599708

epoch time: 1161.7880859375s, input_fired_time: 19.8212356567, output_fired_time: 6.8946061134, propagate_forward_time: 426.7701110840, propagate_backward_time: 696.1086425781, weight_update_time: 0.3418029845, other_time: 12.1933593750
epoch time: 87.9178161621s, input_fired_time: 1.1655431986, output_fired_time: 0.0000000000, propagate_forward_time: 84.7345275879, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0177459717
validation[          , genome  6778] predictions:    5933/  10000 (59.33%), best:    5962/10000 (59.62%), error:     11908.30566, best error:     11864.63574 on epoch:    16, epoch:   18/100, mu: 0.8038368821, learning_rate: 0.0000936687, weight_decay: 0.0001505621

epoch time: 1154.9945068359s, input_fired_time: 19.7431144714, output_fired_time: 6.8738846779, propagate_forward_time: 424.0398254395, propagate_backward_time: 692.1664428711, weight_update_time: 0.3280862868, other_time: 12.1712036133
epoch time: 88.5147094727s, input_fired_time: 1.1672393084, output_fired_time: 0.0000000000, propagate_forward_time: 85.3275527954, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0199203491
validation[          , genome  6778] predictions:    5928/  10000 (59.28%), best:    5962/10000 (59.62%), error:     11926.50781, best error:     11864.63574 on epoch:    16, epoch:   19/100, mu: 0.8135904670, learning_rate: 0.0000875456, weight_decay: 0.0001417067

epoch time: 1158.7869873047s, input_fired_time: 19.8819713593, output_fired_time: 6.8751034737, propagate_forward_time: 425.4862365723, propagate_backward_time: 694.3466796875, weight_update_time: 0.3303050399, other_time: 12.1970214844
epoch time: 88.7969436646s, input_fired_time: 1.1723022461, output_fired_time: 0.0000000000, propagate_forward_time: 85.5600585938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0645828247
validation[          , genome  6778] predictions:    6024/  10000 (60.24%), best:    6024/10000 (60.24%), error:     11575.20215, best error:     11575.20215 on epoch:    20, epoch:   20/100, mu: 0.8228330612, learning_rate: 0.0000818228, weight_decay: 0.0001333721

epoch time: 1153.9223632813s, input_fired_time: 19.7719039917, output_fired_time: 6.8920731544, propagate_forward_time: 423.7947692871, propagate_backward_time: 691.3626708984, weight_update_time: 0.3301702440, other_time: 12.1009521484
epoch time: 87.9040298462s, input_fired_time: 1.1838701963, output_fired_time: 0.0000000000, propagate_forward_time: 84.6320419312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0881195068
validation[          , genome  6778] predictions:    5993/  10000 (59.93%), best:    6024/10000 (60.24%), error:     11680.61230, best error:     11575.20215 on epoch:    20, epoch:   21/100, mu: 0.8315914273, learning_rate: 0.0000764740, weight_decay: 0.0001255278

epoch time: 2330.8417968750s, input_fired_time: 19.8222541809, output_fired_time: 6.8993415833, propagate_forward_time: 1599.6024169922, propagate_backward_time: 692.2152709961, weight_update_time: 0.3349552453, other_time: 12.3024291992
epoch time: 88.3641357422s, input_fired_time: 1.1787602901, output_fired_time: 0.0000000000, propagate_forward_time: 85.0843582153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1010208130
validation[          , genome  6778] predictions:    6018/  10000 (60.18%), best:    6018/10000 (60.18%), error:     11573.57910, best error:     11573.57910 on epoch:    22, epoch:   22/100, mu: 0.8398908973, learning_rate: 0.0000714749, weight_decay: 0.0001181448

epoch time: 1159.4719238281s, input_fired_time: 19.8348464966, output_fired_time: 6.8592481613, propagate_forward_time: 425.9175415039, propagate_backward_time: 694.6176147461, weight_update_time: 0.3302729726, other_time: 12.2426757813
epoch time: 87.0432128906s, input_fired_time: 1.1435605288, output_fired_time: 0.0000000000, propagate_forward_time: 83.8814315796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0182189941
validation[          , genome  6778] predictions:    6014/  10000 (60.14%), best:    6018/10000 (60.18%), error:     11631.60059, best error:     11573.57910 on epoch:    22, epoch:   23/100, mu: 0.8477555513, learning_rate: 0.0000668026, weight_decay: 0.0001111960

epoch time: 1158.7398681641s, input_fired_time: 19.9468250275, output_fired_time: 6.8567271233, propagate_forward_time: 425.5631713867, propagate_backward_time: 694.1862182617, weight_update_time: 0.3295624852, other_time: 12.1870117188
epoch time: 87.2312164307s, input_fired_time: 1.1473243237, output_fired_time: 0.0000000000, propagate_forward_time: 84.0727615356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0111312866
validation[          , genome  6778] predictions:    5954/  10000 (59.54%), best:    6018/10000 (60.18%), error:     11656.32129, best error:     11573.57910 on epoch:    22, epoch:   24/100, mu: 0.8552081585, learning_rate: 0.0000624357, weight_decay: 0.0001046560

epoch time: 2401.5637207031s, input_fired_time: 20.1894798279, output_fired_time: 7.0227208138, propagate_forward_time: 433.5616455078, propagate_backward_time: 1928.3693847656, weight_update_time: 0.3466506302, other_time: 12.4205322266
epoch time: 89.0198593140s, input_fired_time: 1.1835949421, output_fired_time: 0.0000000000, propagate_forward_time: 85.7054443359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1308212280
validation[          , genome  6778] predictions:    6102/  10000 (61.02%), best:    6102/10000 (61.02%), error:     11384.24805, best error:     11384.24805 on epoch:    25, epoch:   25/100, mu: 0.8622702956, learning_rate: 0.0000583543, weight_decay: 0.0000985006

epoch time: 2149.1743164063s, input_fired_time: 20.2794113159, output_fired_time: 7.1029677391, propagate_forward_time: 1396.9291992188, propagate_backward_time: 712.4356689453, weight_update_time: 0.3345616460, other_time: 12.4271240234
epoch time: 93.5080413818s, input_fired_time: 1.3091899157, output_fired_time: 0.0000000000, propagate_forward_time: 90.0495071411, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1493453979
validation[          , genome  6778] predictions:    6011/  10000 (60.11%), best:    6102/10000 (61.02%), error:     11564.92188, best error:     11384.24805 on epoch:    25, epoch:   26/100, mu: 0.8689624071, learning_rate: 0.0000545397, weight_decay: 0.0000927072

epoch time: 1193.1051025391s, input_fired_time: 20.4830989838, output_fired_time: 7.0383186340, propagate_forward_time: 437.9261169434, propagate_backward_time: 715.2052001953, weight_update_time: 0.3340109885, other_time: 12.4522705078
epoch time: 91.1577682495s, input_fired_time: 1.2124371529, output_fired_time: 0.0000000000, propagate_forward_time: 87.8211746216, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1241531372
validation[          , genome  6778] predictions:    6046/  10000 (60.46%), best:    6102/10000 (61.02%), error:     11524.23926, best error:     11384.24805 on epoch:    25, epoch:   27/100, mu: 0.8753039241, learning_rate: 0.0000509744, weight_decay: 0.0000872546

epoch time: 1181.4014892578s, input_fired_time: 20.2099151611, output_fired_time: 7.0437116623, propagate_forward_time: 434.2067260742, propagate_backward_time: 707.5538940430, weight_update_time: 0.3348391354, other_time: 12.3872070313
epoch time: 2568.3352050781s, input_fired_time: 1.2684129477, output_fired_time: 0.0000000000, propagate_forward_time: 2564.8369140625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2299804688
validation[          , genome  6778] predictions:    6086/  10000 (60.86%), best:    6102/10000 (61.02%), error:     11439.80176, best error:     11384.24805 on epoch:    25, epoch:   28/100, mu: 0.8813132048, learning_rate: 0.0000476422, weight_decay: 0.0000821227

epoch time: 2324.6250000000s, input_fired_time: 20.1608333588, output_fired_time: 6.9154429436, propagate_forward_time: 431.3991088867, propagate_backward_time: 1853.7537841797, weight_update_time: 0.3623833358, other_time: 12.3956298828
epoch time: 89.6921157837s, input_fired_time: 1.1903829575, output_fired_time: 0.0000000000, propagate_forward_time: 86.4122161865, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0895156860
validation[          , genome  6778] predictions:    6115/  10000 (61.15%), best:    6115/10000 (61.15%), error:     11293.62305, best error:     11293.62305 on epoch:    29, epoch:   29/100, mu: 0.8870075941, learning_rate: 0.0000445278, weight_decay: 0.0000772926

epoch time: 1149.8471679688s, input_fired_time: 19.7443408966, output_fired_time: 6.8248801231, propagate_forward_time: 422.7393493652, propagate_backward_time: 688.2904052734, weight_update_time: 0.3354560733, other_time: 12.2481689453
epoch time: 91.2728729248s, input_fired_time: 1.2169424295, output_fired_time: 0.0000000000, propagate_forward_time: 88.0184555054, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0374755859
validation[          , genome  6778] predictions:    5943/  10000 (59.43%), best:    6115/10000 (61.15%), error:     11799.63281, best error:     11293.62305 on epoch:    29, epoch:   30/100, mu: 0.8924036622, learning_rate: 0.0000416171, weight_decay: 0.0000727466

epoch time: 1163.3839111328s, input_fired_time: 19.9756965637, output_fired_time: 6.8897476196, propagate_forward_time: 427.6036682129, propagate_backward_time: 696.7930908203, weight_update_time: 0.3316950798, other_time: 12.1217041016
epoch time: 3318.0441894531s, input_fired_time: 1.1823979616, output_fired_time: 0.0000000000, propagate_forward_time: 3314.7988281250, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0629882813
validation[          , genome  6778] predictions:    6022/  10000 (60.22%), best:    6115/10000 (61.15%), error:     11415.08984, best error:     11293.62305 on epoch:    29, epoch:   31/100, mu: 0.8975170255, learning_rate: 0.0000388966, weight_decay: 0.0000684680

epoch time: 1161.0705566406s, input_fired_time: 19.8985767365, output_fired_time: 6.8781957626, propagate_forward_time: 426.1617736816, propagate_backward_time: 695.9873046875, weight_update_time: 0.3351490498, other_time: 12.1447753906
epoch time: 87.2890319824s, input_fired_time: 1.1751786470, output_fired_time: 0.0000000000, propagate_forward_time: 84.1532287598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9606246948
validation[          , genome  6778] predictions:    6106/  10000 (61.06%), best:    6106/10000 (61.06%), error:     11239.81543, best error:     11239.81543 on epoch:    32, epoch:   32/100, mu: 0.9023624659, learning_rate: 0.0000363539, weight_decay: 0.0000644410

epoch time: 2152.7917480469s, input_fired_time: 19.9823265076, output_fired_time: 7.0179815292, propagate_forward_time: 429.6610412598, propagate_backward_time: 1683.9458007813, weight_update_time: 0.3371439278, other_time: 12.1844482422
epoch time: 87.9311065674s, input_fired_time: 1.1709365845, output_fired_time: 0.0000000000, propagate_forward_time: 84.7909164429, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9692535400
validation[          , genome  6778] predictions:    6132/  10000 (61.32%), best:    6132/10000 (61.32%), error:     11193.21387, best error:     11193.21387 on epoch:    33, epoch:   33/100, mu: 0.9069540501, learning_rate: 0.0000339774, weight_decay: 0.0000606508

epoch time: 1159.2299804688s, input_fired_time: 19.8599853516, output_fired_time: 6.8911395073, propagate_forward_time: 425.5055847168, propagate_backward_time: 694.7373046875, weight_update_time: 0.3285008669, other_time: 12.2359619141
epoch time: 87.9681549072s, input_fired_time: 1.1742324829, output_fired_time: 0.0000000000, propagate_forward_time: 84.8024063110, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9915161133
validation[          , genome  6778] predictions:    6158/  10000 (61.58%), best:    6158/10000 (61.58%), error:     11156.62207, best error:     11156.62207 on epoch:    34, epoch:   34/100, mu: 0.9113050699, learning_rate: 0.0000317563, weight_decay: 0.0000570836

epoch time: 1161.5341796875s, input_fired_time: 19.9282398224, output_fired_time: 6.8510465622, propagate_forward_time: 426.4260253906, propagate_backward_time: 696.0938110352, weight_update_time: 0.3359212577, other_time: 12.2350463867
epoch time: 87.3320770264s, input_fired_time: 1.1808915138, output_fired_time: 0.0000000000, propagate_forward_time: 84.0640945435, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0870895386
validation[          , genome  6778] predictions:    6089/  10000 (60.89%), best:    6158/10000 (61.58%), error:     11447.93262, best error:     11156.62207 on epoch:    34, epoch:   35/100, mu: 0.9154281020, learning_rate: 0.0000296804, weight_decay: 0.0000537262

epoch time: 2309.3066406250s, input_fired_time: 19.7423038483, output_fired_time: 6.8577580452, propagate_forward_time: 1579.4591064453, propagate_backward_time: 691.1901855469, weight_update_time: 0.3270757198, other_time: 12.0574951172
epoch time: 87.4339218140s, input_fired_time: 1.1680319309, output_fired_time: 0.0000000000, propagate_forward_time: 84.2936706543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9722213745
validation[          , genome  6778] predictions:    6181/  10000 (61.81%), best:    6181/10000 (61.81%), error:     11123.66895, best error:     11123.66895 on epoch:    36, epoch:   36/100, mu: 0.9193351269, learning_rate: 0.0000277402, weight_decay: 0.0000505663

epoch time: 1146.2894287109s, input_fired_time: 19.7349967957, output_fired_time: 6.8331646919, propagate_forward_time: 421.0177307129, propagate_backward_time: 686.6951904297, weight_update_time: 0.3259378076, other_time: 12.0084228516
epoch time: 87.5095748901s, input_fired_time: 1.1862549782, output_fired_time: 0.0000000000, propagate_forward_time: 84.2754821777, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0478363037
validation[          , genome  6778] predictions:    6184/  10000 (61.84%), best:    6184/10000 (61.84%), error:     11092.15723, best error:     11092.15723 on epoch:    37, epoch:   37/100, mu: 0.9230374694, learning_rate: 0.0000259268, weight_decay: 0.0000475922

epoch time: 5525.2524414063s, input_fired_time: 19.6848068237, output_fired_time: 6.8131098747, propagate_forward_time: 421.6289367676, propagate_backward_time: 4113.0190429688, weight_update_time: 0.3274120390, other_time: 964.1069335938
epoch time: 87.4110870361s, input_fired_time: 1.1795951128, output_fired_time: 0.0000000000, propagate_forward_time: 84.2383270264, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9931640625
validation[          , genome  6778] predictions:    6141/  10000 (61.41%), best:    6184/10000 (61.84%), error:     11147.00488, best error:     11092.15723 on epoch:    37, epoch:   38/100, mu: 0.9265457988, learning_rate: 0.0000242320, weight_decay: 0.0000447930

epoch time: 1145.7125244141s, input_fired_time: 19.7263298035, output_fired_time: 6.7917680740, propagate_forward_time: 421.3257751465, propagate_backward_time: 685.9700927734, weight_update_time: 0.3255351484, other_time: 11.8985595703
epoch time: 86.9828643799s, input_fired_time: 1.1672425270, output_fired_time: 0.0000000000, propagate_forward_time: 83.8369140625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9787063599
validation[          , genome  6778] predictions:    6141/  10000 (61.41%), best:    6184/10000 (61.84%), error:     11189.29004, best error:     11092.15723 on epoch:    37, epoch:   39/100, mu: 0.9298703671, learning_rate: 0.0000226480, weight_decay: 0.0000421585

epoch time: 2299.9230957031s, input_fired_time: 19.7599086761, output_fired_time: 6.8253769875, propagate_forward_time: 422.2143554688, propagate_backward_time: 1839.1636962891, weight_update_time: 0.3255118728, other_time: 11.9595947266
epoch time: 87.4273376465s, input_fired_time: 1.1886181831, output_fired_time: 0.0000000000, propagate_forward_time: 84.2288208008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0098953247
validation[          , genome  6778] predictions:    6202/  10000 (62.02%), best:    6202/10000 (62.02%), error:     10983.93652, best error:     10983.93652 on epoch:    40, epoch:   40/100, mu: 0.9330207109, learning_rate: 0.0000211675, weight_decay: 0.0000396789

epoch time: 1146.9541015625s, input_fired_time: 19.6556739807, output_fired_time: 6.8039245605, propagate_forward_time: 421.7825012207, propagate_backward_time: 686.7063598633, weight_update_time: 0.3275103867, other_time: 12.0056762695
epoch time: 88.2425918579s, input_fired_time: 1.2018638849, output_fired_time: 0.0000000000, propagate_forward_time: 84.9823379517, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0583877563
validation[          , genome  6778] predictions:    6176/  10000 (61.76%), best:    6202/10000 (62.02%), error:     11106.95410, best error:     10983.93652 on epoch:    40, epoch:   41/100, mu: 0.9360060096, learning_rate: 0.0000197837, weight_decay: 0.0000373452

epoch time: 2264.8671875000s, input_fired_time: 19.9557838440, output_fired_time: 6.8568344116, propagate_forward_time: 1534.4808349609, propagate_backward_time: 691.3157348633, weight_update_time: 0.3313248158, other_time: 12.2578735352
epoch time: 89.1075057983s, input_fired_time: 1.2120876312, output_fired_time: 0.0000000000, propagate_forward_time: 85.8079605103, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0874557495
validation[          , genome  6778] predictions:    6269/  10000 (62.69%), best:    6269/10000 (62.69%), error:     10929.56836, best error:     10929.56836 on epoch:    42, epoch:   42/100, mu: 0.9388349056, learning_rate: 0.0000184905, weight_decay: 0.0000351487

epoch time: 1147.7963867188s, input_fired_time: 19.7271480560, output_fired_time: 6.8216013908, propagate_forward_time: 421.8565979004, propagate_backward_time: 687.3146362305, weight_update_time: 0.3281217813, other_time: 12.0763549805
epoch time: 88.3249206543s, input_fired_time: 1.2010545731, output_fired_time: 0.0000000000, propagate_forward_time: 85.1185684204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0052947998
validation[          , genome  6778] predictions:    6265/  10000 (62.65%), best:    6265/10000 (62.65%), error:     10871.48438, best error:     10871.48438 on epoch:    43, epoch:   43/100, mu: 0.9415155649, learning_rate: 0.0000172818, weight_decay: 0.0000330814

epoch time: 1152.8009033203s, input_fired_time: 19.8963909149, output_fired_time: 6.8322281837, propagate_forward_time: 423.4581298828, propagate_backward_time: 690.3834228516, weight_update_time: 0.3348423243, other_time: 12.2307128906
epoch time: 88.8947296143s, input_fired_time: 1.2130613327, output_fired_time: 0.0000000000, propagate_forward_time: 85.6261520386, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0555191040
validation[          , genome  6778] predictions:    6243/  10000 (62.43%), best:    6265/10000 (62.65%), error:     10941.26855, best error:     10871.48438 on epoch:    43, epoch:   44/100, mu: 0.9440557957, learning_rate: 0.0000161521, weight_decay: 0.0000311357

epoch time: 2313.3024902344s, input_fired_time: 19.9166336060, output_fired_time: 6.8300147057, propagate_forward_time: 423.9339599609, propagate_backward_time: 1850.3920898438, weight_update_time: 0.3323110342, other_time: 12.2296142578
epoch time: 90.7605972290s, input_fired_time: 1.2422569990, output_fired_time: 0.0000000000, propagate_forward_time: 87.4428787231, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0754623413
validation[          , genome  6778] predictions:    6200/  10000 (62.00%), best:    6265/10000 (62.65%), error:     10972.12207, best error:     10871.48438 on epoch:    43, epoch:   45/100, mu: 0.9464629292, learning_rate: 0.0000150962, weight_decay: 0.0000293045

epoch time: 1165.0875244141s, input_fired_time: 20.0605831146, output_fired_time: 6.8997917175, propagate_forward_time: 428.0366821289, propagate_backward_time: 698.0146484375, weight_update_time: 0.3314366639, other_time: 12.0758666992
epoch time: 87.4297409058s, input_fired_time: 1.1774332523, output_fired_time: 0.0000000000, propagate_forward_time: 84.3187561035, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9335479736
validation[          , genome  6778] predictions:    6113/  10000 (61.13%), best:    6265/10000 (62.65%), error:     11095.36719, best error:     10871.48438 on epoch:    43, epoch:   46/100, mu: 0.9487439394, learning_rate: 0.0000141094, weight_decay: 0.0000275809

epoch time: 1164.9027099609s, input_fired_time: 20.0941963196, output_fired_time: 6.9140939713, propagate_forward_time: 428.0040283203, propagate_backward_time: 697.8223876953, weight_update_time: 0.3262580633, other_time: 12.0679931641
epoch time: 89.2799606323s, input_fired_time: 1.2114037275, output_fired_time: 0.0000000000, propagate_forward_time: 86.0457077026, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0228500366
validation[          , genome  6778] predictions:    6155/  10000 (61.55%), best:    6265/10000 (62.65%), error:     11003.57715, best error:     10871.48438 on epoch:    43, epoch:   47/100, mu: 0.9509054422, learning_rate: 0.0000131870, weight_decay: 0.0000259587

epoch time: 1159.3289794922s, input_fired_time: 19.9817237854, output_fired_time: 6.8571877480, propagate_forward_time: 425.8000488281, propagate_backward_time: 694.7297973633, weight_update_time: 0.3290926218, other_time: 11.9602661133
epoch time: 87.4084320068s, input_fired_time: 1.1960765123, output_fired_time: 0.0000000000, propagate_forward_time: 84.2672805786, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9450759888
validation[          , genome  6778] predictions:    6273/  10000 (62.73%), best:    6273/10000 (62.73%), error:     10843.92285, best error:     10843.92285 on epoch:    48, epoch:   48/100, mu: 0.9529536963, learning_rate: 0.0000123250, weight_decay: 0.0000244319

epoch time: 1169.0996093750s, input_fired_time: 20.0947246552, output_fired_time: 7.0006761551, propagate_forward_time: 429.7340698242, propagate_backward_time: 700.2554321289, weight_update_time: 0.3272065222, other_time: 12.0146484375
epoch time: 89.4688949585s, input_fired_time: 1.2194077969, output_fired_time: 0.0000000000, propagate_forward_time: 86.2173767090, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0321121216
validation[          , genome  6778] predictions:    6300/  10000 (63.00%), best:    6300/10000 (63.00%), error:     10785.15234, best error:     10785.15234 on epoch:    49, epoch:   49/100, mu: 0.9548946619, learning_rate: 0.0000115193, weight_decay: 0.0000229950

epoch time: 1173.2333984375s, input_fired_time: 20.2115745544, output_fired_time: 6.9591765404, propagate_forward_time: 431.2312622070, propagate_backward_time: 702.6083374023, weight_update_time: 0.3445726037, other_time: 12.2230224609
epoch time: 88.3143081665s, input_fired_time: 1.2067348957, output_fired_time: 0.0000000000, propagate_forward_time: 85.0661621094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0414123535
validation[          , genome  6778] predictions:    6270/  10000 (62.70%), best:    6300/10000 (63.00%), error:     10819.84180, best error:     10785.15234 on epoch:    49, epoch:   50/100, mu: 0.9567339420, learning_rate: 0.0000107663, weight_decay: 0.0000216425

epoch time: 1167.4721679688s, input_fired_time: 20.1646766663, output_fired_time: 6.9016642570, propagate_forward_time: 428.3961181641, propagate_backward_time: 699.8543701172, weight_update_time: 0.3368272781, other_time: 12.1553955078
epoch time: 89.2488784790s, input_fired_time: 1.2203714848, output_fired_time: 0.0000000000, propagate_forward_time: 85.9506378174, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0778656006
validation[          , genome  6778] predictions:    6302/  10000 (63.02%), best:    6302/10000 (63.02%), error:     10768.65039, best error:     10768.65039 on epoch:    51, epoch:   51/100, mu: 0.9584768414, learning_rate: 0.0000100625, weight_decay: 0.0000203696

epoch time: 1160.7601318359s, input_fired_time: 20.0182323456, output_fired_time: 6.8816118240, propagate_forward_time: 426.5536193848, propagate_backward_time: 695.3070068359, weight_update_time: 0.3282397389, other_time: 11.9997558594
epoch time: 88.5674591064s, input_fired_time: 1.2084970474, output_fired_time: 0.0000000000, propagate_forward_time: 85.3086471558, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0503158569
validation[          , genome  6778] predictions:    6250/  10000 (62.50%), best:    6302/10000 (63.02%), error:     10833.94434, best error:     10768.65039 on epoch:    51, epoch:   52/100, mu: 0.9601284266, learning_rate: 0.0000094047, weight_decay: 0.0000191715

epoch time: 1153.9891357422s, input_fired_time: 19.9597129822, output_fired_time: 6.8590869904, propagate_forward_time: 423.6639099121, propagate_backward_time: 691.3925170898, weight_update_time: 0.3297327459, other_time: 12.1138305664
epoch time: 86.8961105347s, input_fired_time: 1.1917045116, output_fired_time: 0.0000000000, propagate_forward_time: 83.7736129761, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9307937622
validation[          , genome  6778] predictions:    6318/  10000 (63.18%), best:    6318/10000 (63.18%), error:     10747.96387, best error:     10747.96387 on epoch:    53, epoch:   53/100, mu: 0.9616934657, learning_rate: 0.0000087899, weight_decay: 0.0000180440

epoch time: 1149.4941406250s, input_fired_time: 19.7621879578, output_fired_time: 6.7719573975, propagate_forward_time: 421.7340087891, propagate_backward_time: 689.3890380859, weight_update_time: 0.3247242868, other_time: 11.8369140625
epoch time: 89.0150299072s, input_fired_time: 1.2061557770, output_fired_time: 0.0000000000, propagate_forward_time: 85.8283309937, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9805450439
validation[          , genome  6778] predictions:    6304/  10000 (63.04%), best:    6304/10000 (63.04%), error:     10737.64648, best error:     10737.64648 on epoch:    54, epoch:   54/100, mu: 0.9631765485, learning_rate: 0.0000082153, weight_decay: 0.0000169827

epoch time: 1153.0048828125s, input_fired_time: 19.8928413391, output_fired_time: 6.8604855537, propagate_forward_time: 423.4340515137, propagate_backward_time: 690.8787841797, weight_update_time: 0.3261523843, other_time: 11.9387207031
epoch time: 88.3475265503s, input_fired_time: 1.2007648945, output_fired_time: 0.0000000000, propagate_forward_time: 85.1693191528, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9774398804
validation[          , genome  6778] predictions:    6317/  10000 (63.17%), best:    6317/10000 (63.17%), error:     10733.71191, best error:     10733.71191 on epoch:    55, epoch:   55/100, mu: 0.9645819068, learning_rate: 0.0000076783, weight_decay: 0.0000159838

epoch time: 1151.3607177734s, input_fired_time: 19.9014205933, output_fired_time: 6.8553495407, propagate_forward_time: 422.6645507813, propagate_backward_time: 689.9437866211, weight_update_time: 0.3283626437, other_time: 11.9956665039
epoch time: 87.0429763794s, input_fired_time: 1.1912610531, output_fired_time: 0.0000000000, propagate_forward_time: 83.8904190063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9612960815
validation[          , genome  6778] predictions:    6362/  10000 (63.62%), best:    6362/10000 (63.62%), error:     10690.18262, best error:     10690.18262 on epoch:    56, epoch:   56/100, mu: 0.9659136534, learning_rate: 0.0000071764, weight_decay: 0.0000150437

epoch time: 1154.0753173828s, input_fired_time: 19.9639911652, output_fired_time: 6.8522992134, propagate_forward_time: 423.5317687988, propagate_backward_time: 691.8044433594, weight_update_time: 0.3260970116, other_time: 11.9228515625
epoch time: 88.3588180542s, input_fired_time: 1.2237830162, output_fired_time: 0.0000000000, propagate_forward_time: 85.1358108521, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9992218018
validation[          , genome  6778] predictions:    6329/  10000 (63.29%), best:    6329/10000 (63.29%), error:     10669.43652, best error:     10669.43652 on epoch:    57, epoch:   57/100, mu: 0.9671756029, learning_rate: 0.0000067072, weight_decay: 0.0000141589

epoch time: 1187.4182128906s, input_fired_time: 20.6150264740, output_fired_time: 7.1143879890, propagate_forward_time: 434.5569152832, propagate_backward_time: 712.5789184570, weight_update_time: 0.3337852061, other_time: 12.5530395508
epoch time: 98.3266525269s, input_fired_time: 1.4394018650, output_fired_time: 0.0000000000, propagate_forward_time: 94.4012298584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4860229492
validation[          , genome  6778] predictions:    6265/  10000 (62.65%), best:    6329/10000 (63.29%), error:     10804.16797, best error:     10669.43652 on epoch:    57, epoch:   58/100, mu: 0.9683714509, learning_rate: 0.0000062688, weight_decay: 0.0000133262

epoch time: 1178.3160400391s, input_fired_time: 20.5271549225, output_fired_time: 7.0328507423, propagate_forward_time: 431.4810791016, propagate_backward_time: 706.6852416992, weight_update_time: 0.3691482842, other_time: 12.5897827148
epoch time: 88.9609527588s, input_fired_time: 1.2276128531, output_fired_time: 0.0000000000, propagate_forward_time: 85.7454452515, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9878921509
validation[          , genome  6778] predictions:    6337/  10000 (63.37%), best:    6337/10000 (63.37%), error:     10643.49805, best error:     10643.49805 on epoch:    59, epoch:   59/100, mu: 0.9695046544, learning_rate: 0.0000058590, weight_decay: 0.0000125424

epoch time: 1163.9351806641s, input_fired_time: 20.2556324005, output_fired_time: 6.8828029633, propagate_forward_time: 426.8403015137, propagate_backward_time: 697.9600830078, weight_update_time: 0.3410333395, other_time: 11.9963378906
epoch time: 90.7442779541s, input_fired_time: 1.2784221172, output_fired_time: 0.0000000000, propagate_forward_time: 87.3199386597, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1459197998
validation[          , genome  6778] predictions:    6385/  10000 (63.85%), best:    6385/10000 (63.85%), error:     10610.64258, best error:     10610.64258 on epoch:    60, epoch:   60/100, mu: 0.9705784917, learning_rate: 0.0000054760, weight_decay: 0.0000118047

epoch time: 1200.4676513672s, input_fired_time: 21.1608619690, output_fired_time: 7.2621812820, propagate_forward_time: 438.9562683105, propagate_backward_time: 719.8047485352, weight_update_time: 0.3506219387, other_time: 13.2835083008
epoch time: 89.1232757568s, input_fired_time: 1.2385041714, output_fired_time: 0.0000000000, propagate_forward_time: 85.7545166016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1302566528
validation[          , genome  6778] predictions:    6327/  10000 (63.27%), best:    6385/10000 (63.85%), error:     10646.58691, best error:     10610.64258 on epoch:    60, epoch:   61/100, mu: 0.9715960622, learning_rate: 0.0000051180, weight_decay: 0.0000111104

epoch time: 1163.4174804688s, input_fired_time: 20.2427692413, output_fired_time: 6.9933958054, propagate_forward_time: 425.9779052734, propagate_backward_time: 698.1095581055, weight_update_time: 0.3336896002, other_time: 12.0938110352
epoch time: 87.4555130005s, input_fired_time: 1.1868760586, output_fired_time: 0.0000000000, propagate_forward_time: 84.2991256714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9695129395
validation[          , genome  6778] predictions:    6330/  10000 (63.30%), best:    6385/10000 (63.85%), error:     10648.81250, best error:     10610.64258 on epoch:    60, epoch:   62/100, mu: 0.9725602865, learning_rate: 0.0000047835, weight_decay: 0.0000104569

epoch time: 1163.3720703125s, input_fired_time: 20.1716728210, output_fired_time: 6.9438772202, propagate_forward_time: 426.8840026855, propagate_backward_time: 697.1088867188, weight_update_time: 0.3318165541, other_time: 12.2636718750
epoch time: 89.2315063477s, input_fired_time: 1.2358255386, output_fired_time: 0.0000000000, propagate_forward_time: 85.9718322754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0238494873
validation[          , genome  6778] predictions:    6348/  10000 (63.48%), best:    6348/10000 (63.48%), error:     10609.54492, best error:     10609.54492 on epoch:    63, epoch:   63/100, mu: 0.9734740257, learning_rate: 0.0000044708, weight_decay: 0.0000098419

epoch time: 1179.4221191406s, input_fired_time: 20.3824958801, output_fired_time: 7.0164351463, propagate_forward_time: 433.0133666992, propagate_backward_time: 706.7856445313, weight_update_time: 0.3310056627, other_time: 12.2241821289
epoch time: 103.4404602051s, input_fired_time: 1.4508696795, output_fired_time: 0.0000000000, propagate_forward_time: 99.4696502686, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5199432373
validation[          , genome  6778] predictions:    6321/  10000 (63.21%), best:    6348/10000 (63.48%), error:     10705.57227, best error:     10609.54492 on epoch:    63, epoch:   64/100, mu: 0.9743398428, learning_rate: 0.0000041785, weight_decay: 0.0000092630

epoch time: 1211.1994628906s, input_fired_time: 21.0520763397, output_fired_time: 7.2651357651, propagate_forward_time: 442.6463317871, propagate_backward_time: 727.3107910156, weight_update_time: 0.3538694978, other_time: 12.9250488281
epoch time: 94.4054107666s, input_fired_time: 1.3392077684, output_fired_time: 0.0000000000, propagate_forward_time: 90.6509704590, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4152297974
validation[          , genome  6778] predictions:    6314/  10000 (63.14%), best:    6348/10000 (63.48%), error:     10679.88965, best error:     10609.54492 on epoch:    63, epoch:   65/100, mu: 0.9751603007, learning_rate: 0.0000039054, weight_decay: 0.0000087182

epoch time: 1215.0626220703s, input_fired_time: 21.2257499695, output_fired_time: 7.2263507843, propagate_forward_time: 445.5012817383, propagate_backward_time: 727.9429931641, weight_update_time: 0.3503626287, other_time: 13.1663208008
epoch time: 90.9743957520s, input_fired_time: 1.2780747414, output_fired_time: 0.0000000000, propagate_forward_time: 87.4423828125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2539367676
validation[          , genome  6778] predictions:    6341/  10000 (63.41%), best:    6348/10000 (63.48%), error:     10652.08984, best error:     10609.54492 on epoch:    63, epoch:   66/100, mu: 0.9759377837, learning_rate: 0.0000036501, weight_decay: 0.0000082055

epoch time: 1187.7415771484s, input_fired_time: 20.7858028412, output_fired_time: 7.1201109886, propagate_forward_time: 435.4505920410, propagate_backward_time: 711.4145507813, weight_update_time: 0.3429041505, other_time: 12.9705810547
epoch time: 93.7575683594s, input_fired_time: 1.2926717997, output_fired_time: 0.0000000000, propagate_forward_time: 90.2350463867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2298507690
validation[          , genome  6778] predictions:    6315/  10000 (63.15%), best:    6348/10000 (63.48%), error:     10630.05078, best error:     10609.54492 on epoch:    63, epoch:   67/100, mu: 0.9766745567, learning_rate: 0.0000034115, weight_decay: 0.0000077229

epoch time: 1149.7017822266s, input_fired_time: 19.8739643097, output_fired_time: 6.8100190163, propagate_forward_time: 422.1527099609, propagate_backward_time: 688.8971557617, weight_update_time: 0.3265253305, other_time: 11.9678344727
epoch time: 88.3846206665s, input_fired_time: 1.2350522280, output_fired_time: 0.0000000000, propagate_forward_time: 85.1656494141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9839172363
validation[          , genome  6778] predictions:    6361/  10000 (63.61%), best:    6361/10000 (63.61%), error:     10586.73926, best error:     10586.73926 on epoch:    68, epoch:   68/100, mu: 0.9773727059, learning_rate: 0.0000031885, weight_decay: 0.0000072686

epoch time: 1154.4162597656s, input_fired_time: 19.9543533325, output_fired_time: 6.8411364555, propagate_forward_time: 423.6520385742, propagate_backward_time: 691.9962768555, weight_update_time: 0.3264755905, other_time: 11.9724121094
epoch time: 90.1028366089s, input_fired_time: 1.2399395704, output_fired_time: 0.0000000000, propagate_forward_time: 86.7973556519, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0655441284
validation[          , genome  6778] predictions:    6315/  10000 (63.15%), best:    6361/10000 (63.61%), error:     10661.08105, best error:     10586.73926 on epoch:    68, epoch:   69/100, mu: 0.9780342579, learning_rate: 0.0000029800, weight_decay: 0.0000068411

epoch time: 1197.5007324219s, input_fired_time: 20.6236648560, output_fired_time: 7.1586546898, propagate_forward_time: 440.0278015137, propagate_backward_time: 717.3530273438, weight_update_time: 0.3346179724, other_time: 12.3375244141
epoch time: 90.1455001831s, input_fired_time: 1.2389763594, output_fired_time: 0.0000000000, propagate_forward_time: 86.7879714966, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1185531616
validation[          , genome  6778] predictions:    6397/  10000 (63.97%), best:    6397/10000 (63.97%), error:     10567.85156, best error:     10567.85156 on epoch:    70, epoch:   70/100, mu: 0.9786611795, learning_rate: 0.0000027852, weight_decay: 0.0000064388

epoch time: 1175.0117187500s, input_fired_time: 20.3008346558, output_fired_time: 6.9826192856, propagate_forward_time: 431.1329345703, propagate_backward_time: 704.4819335938, weight_update_time: 0.3304595947, other_time: 12.1134033203
epoch time: 89.5073699951s, input_fired_time: 1.2537434101, output_fired_time: 0.0000000000, propagate_forward_time: 86.2510147095, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0026092529
validation[          , genome  6778] predictions:    6374/  10000 (63.74%), best:    6397/10000 (63.97%), error:     10595.80664, best error:     10567.85156 on epoch:    70, epoch:   71/100, mu: 0.9792552590, learning_rate: 0.0000026032, weight_decay: 0.0000060601

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_1_r1982667377_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.979818
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 2.43299e-06
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 5.70364e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 72
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 70
read number_validation_images: 10000
read best_validation_predictions: 6397
read best_validation_error: 10567.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '377740764'
read generator: 377740764
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1184.24829101563s, input_fired_time: 20.6197738647461, output_fired_time: 7.17936944961548, propagate_forward_time: 433.906616210938, propagate_backward_time: 709.553405761719, weight_update_time: 0.364231616258621, other_time: 12.9891967773438
epoch time: 90.6942825317383s, input_fired_time: 1.28636884689331, output_fired_time: 0, propagate_forward_time: 87.2245407104492, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.18337249755859
validation[          , genome  6778] predictions:    6364/  10000 (63.64%), best:    6397/10000 (63.97%), error:     10578.12305, best error:     10567.85156 on epoch:    70, epoch:   72/100, mu: 0.9798182249, learning_rate: 0.0000024330, weight_decay: 0.0000057036



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFC89BFC902

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 10/26/18 21:33:07
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\Program Files\BOINC\\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files\BOINC\\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\8;C:\ProgramData\BOINC\projects\csgrid.org_csg


ModLoad: 0000000048010000 00000000001da000 C:\ProgramData\BOINC\projects\csgrid.org_csg\exact_client_0.34_windows_x86_64.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : C:\Users\travi\Code\boinc_2015\win_build\Build\x64\Release\exact.pdb

ModLoad: 000000008d590000 00000000001e1000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.165) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008bde0000 00000000000b2000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000089b50000 0000000000273000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.165) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000087820000 000000000008b000 C:\WINDOWS\SYSTEM32\apphelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008b430000 0000000000190000 C:\WINDOWS\System32\USER32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008a0d0000 0000000000020000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008b230000 0000000000028000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008a0f0000 0000000000192000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.17134.112 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.112

ModLoad: 0000000089e30000 000000000009f000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.137) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.17134.137 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.137

ModLoad: 0000000089fd0000 00000000000fa000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.165) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.17134.165 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.165

ModLoad: 000000008bb40000 00000000000a1000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008bed0000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.17134.1

ModLoad: 000000008b8f0000 000000000005b000 C:\WINDOWS\System32\sechost.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008af00000 0000000000124000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008c080000 0000000001440000 C:\WINDOWS\System32\SHELL32.dll (6.2.17134.137) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000089ed0000 0000000000049000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008bfd0000 00000000000a9000 C:\WINDOWS\System32\shcore.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008b5c0000 0000000000323000 C:\WINDOWS\System32\combase.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008a9a0000 000000000007a000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008a290000 000000000070d000 C:\WINDOWS\System32\windows.storage.dll (6.2.17134.137) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008bf70000 0000000000051000 C:\WINDOWS\System32\shlwapi.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 00000000898d0000 0000000000011000 C:\WINDOWS\System32\kernel.appcore.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.17134.112 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.112

ModLoad: 0000000089940000 000000000001f000 C:\WINDOWS\System32\profapi.dll (6.2.17134.1) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : profapi.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 00000000898f0000 000000000004c000 C:\WINDOWS\System32\powrprof.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : powrprof.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 00000000898c0000 000000000000a000 C:\WINDOWS\System32\FLTLIB.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : fltLib.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000008bea0000 000000000002d000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000071c40000 00000000001c9000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000084500000 000000000000a000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 55467, Write: 13310, Other 1245

- I/O Transfers Counters -
Read: 227142772, Write: 12088136, Other 50434

- Paged Pool Usage -
QuotaPagedPoolUsage: 158944, QuotaPeakPagedPoolUsage: 159104
QuotaNonPagedPoolUsage: 12912, QuotaPeakNonPagedPoolUsage: 13184

- Virtual Memory Usage -
VirtualSize: 604307456, PeakVirtualSize: 735797248

- Pagefile Usage -
PagefileUsage: 604307456, PeakPagefileUsage: 604307456

- Working Set Size -
WorkingSetSize: 584388608, PeakWorkingSetSize: 584388608, PageFaultCount: 171753

*** Dump of thread ID 7436 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FFC89BFC902

- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=0000000048169440 rdx=00000000e5cff7b0 rsi=0000000000000000 rdi=0000000000000000
r8=00000000e5cff7b0 r9=0000000048169430 r10=0000000000000001 r11=0000000000000fff r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000089bfc902 rsp=00000000e5cff788 rbp=0000000000000000
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
e5cff780 4809e421 00000001 e5cff7b0 e5cff7b0 48169430 KERNELBASE!DebugBreak+0x0 
e5cffbc0 4809f820 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
e5cffe20 8bdf3034 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
e5cffe50 8d601431 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
e5cffea0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 32764 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Unknown, , Kernel Time: 31.000000, User Time: 156250.000000, Wait Time: 3311706880.000000

- Registers -
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000000000000 rsp=0000000000000000 rbp=0000000000000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !+0x0 

*** Dump of thread ID 30698898 (state: Unknown): ***

- Information -
Status: Base Priority: Normal, Priority: Unknown, , Kernel Time: 17179869184.000000, User Time: 21474850816.000000, Wait Time: 0.000000

- Registers -
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000000000000 rsp=0000000000000000 rbp=0000000000000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>