Nameexact_genome_1533295503_162_10089_2
Workunit3896027
Created5 Aug 2018, 7:46:26 UTC
Sent5 Aug 2018, 8:25:12 UTC
Report deadline25 Jan 2019, 2:44:14 UTC
Received10 Aug 2018, 10:28:34 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60269
Run time2 days 23 hours 3 min 24 sec
CPU time2 days 22 hours 24 min 17 sec
Validate stateInitial
Credit0.00
Device peak FLOPS4.36 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size593.07 MB
Peak swap size612.87 MB
Peak disk usage16.47 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
52905273s, input_fired_time: 2.31373620033264, output_fired_time: 0, propagate_forward_time: 169.201538085938, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.59957885742188
validation[          , genome 10089] predictions:    6340/  10000 (63.40%), best:    6340/10000 (63.40%), error:     10268.74023, best error:     10268.74023 on epoch:    22, epoch:   22/100, mu: 0.8364537954, learning_rate: 0.0004349051, weight_decay: 0.0001470032

epoch time: 2177.4895019531s, input_fired_time: 31.8531131744, output_fired_time: 13.1280574799, propagate_forward_time: 845.7098388672, propagate_backward_time: 1266.4829101563, weight_update_time: 0.5249735117, other_time: 20.3155517578
epoch time: 173.4585723877s, input_fired_time: 2.3111376762, output_fired_time: 0.0000000000, propagate_forward_time: 167.4669952393, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6804351807
validation[          , genome 10089] predictions:    6431/  10000 (64.31%), best:    6340/10000 (63.40%), error:     10350.09863, best error:     10268.74023 on epoch:    22, epoch:   23/100, mu: 0.8439996839, learning_rate: 0.0004021227, weight_decay: 0.0001386539

epoch time: 2217.5312500000s, input_fired_time: 31.9306449890, output_fired_time: 13.3313159943, propagate_forward_time: 862.9359130859, propagate_backward_time: 1288.5059814453, weight_update_time: 0.5301802754, other_time: 20.8273925781
epoch time: 179.1984558105s, input_fired_time: 2.3333122730, output_fired_time: 0.0000000000, propagate_forward_time: 173.2085571289, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6565856934
validation[          , genome 10089] predictions:    6445/  10000 (64.45%), best:    6340/10000 (63.40%), error:     10328.66113, best error:     10268.74023 on epoch:    22, epoch:   24/100, mu: 0.8511747122, learning_rate: 0.0003718114, weight_decay: 0.0001307787

epoch time: 2229.6074218750s, input_fired_time: 31.9542121887, output_fired_time: 13.3513307571, propagate_forward_time: 865.5026855469, propagate_backward_time: 1298.2680664063, weight_update_time: 0.5319715142, other_time: 20.5312500000
epoch time: 181.8619995117s, input_fired_time: 2.3403148651, output_fired_time: 0.0000000000, propagate_forward_time: 175.4786987305, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0429840088
validation[          , genome 10089] predictions:    6290/  10000 (62.90%), best:    6340/10000 (63.40%), error:     10719.14160, best error:     10268.74023 on epoch:    22, epoch:   25/100, mu: 0.8579971194, learning_rate: 0.0003437849, weight_decay: 0.0001233509

epoch time: 2268.3432617188s, input_fired_time: 31.9250869751, output_fired_time: 13.5151386261, propagate_forward_time: 881.6090698242, propagate_backward_time: 1319.6390380859, weight_update_time: 0.5254477262, other_time: 21.6549072266
epoch time: 182.5780181885s, input_fired_time: 2.3619921207, output_fired_time: 0.0000000000, propagate_forward_time: 176.2378692627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9781494141
validation[          , genome 10089] predictions:    6275/  10000 (62.75%), best:    6340/10000 (63.40%), error:     10928.14063, best error:     10268.74023 on epoch:    22, epoch:   26/100, mu: 0.8644842505, learning_rate: 0.0003178710, weight_decay: 0.0001163449

epoch time: 2275.7536621094s, input_fired_time: 31.9394016266, output_fired_time: 13.5536956787, propagate_forward_time: 883.9141845703, propagate_backward_time: 1324.5142822266, weight_update_time: 0.5265879035, other_time: 21.8320312500
epoch time: 184.3229064941s, input_fired_time: 2.3451099396, output_fired_time: 0.0000000000, propagate_forward_time: 178.1289520264, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8488464355
validation[          , genome 10089] predictions:    6427/  10000 (64.27%), best:    6427/10000 (64.27%), error:     10225.64160, best error:     10225.64160 on epoch:    27, epoch:   27/100, mu: 0.8706525564, learning_rate: 0.0002939104, weight_decay: 0.0001097369

epoch time: 2280.0490722656s, input_fired_time: 31.9709930420, output_fired_time: 13.5519523621, propagate_forward_time: 884.3938598633, propagate_backward_time: 1328.1306152344, weight_update_time: 0.5294227004, other_time: 22.0017089844
epoch time: 183.4088134766s, input_fired_time: 2.3680038452, output_fired_time: 0.0000000000, propagate_forward_time: 177.1301422119, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9106597900
validation[          , genome 10089] predictions:    6484/  10000 (64.84%), best:    6484/10000 (64.84%), error:     10127.19922, best error:     10127.19922 on epoch:    28, epoch:   28/100, mu: 0.8765177727, learning_rate: 0.0002717559, weight_decay: 0.0001035041

epoch time: 2309.0532226563s, input_fired_time: 32.0142021179, output_fired_time: 13.6787528992, propagate_forward_time: 897.8458862305, propagate_backward_time: 1343.4233398438, weight_update_time: 0.5352598429, other_time: 22.0910644531
epoch time: 187.5122985840s, input_fired_time: 2.3700656891, output_fired_time: 0.0000000000, propagate_forward_time: 181.1288452148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0133819580
validation[          , genome 10089] predictions:    6564/  10000 (65.64%), best:    6564/10000 (65.64%), error:      9924.72754, best error:      9924.72754 on epoch:    29, epoch:   29/100, mu: 0.8820947409, learning_rate: 0.0002512714, weight_decay: 0.0000976254

epoch time: 2310.3447265625s, input_fired_time: 32.0779151917, output_fired_time: 13.6408386230, propagate_forward_time: 899.6497802734, propagate_backward_time: 1342.9027099609, weight_update_time: 0.5280111432, other_time: 22.0734863281
epoch time: 183.8123016357s, input_fired_time: 2.3514521122, output_fired_time: 0.0000000000, propagate_forward_time: 177.5004577637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9603881836
validation[          , genome 10089] predictions:    6461/  10000 (64.61%), best:    6564/10000 (65.64%), error:     10171.21973, best error:      9924.72754 on epoch:    29, epoch:   30/100, mu: 0.8873976469, learning_rate: 0.0002323310, weight_decay: 0.0000920806

epoch time: 2288.4543457031s, input_fired_time: 32.0391235352, output_fired_time: 13.5880746841, propagate_forward_time: 888.8601074219, propagate_backward_time: 1332.1307373047, weight_update_time: 0.5303089023, other_time: 21.8363037109
epoch time: 185.6090698242s, input_fired_time: 2.3895590305, output_fired_time: 0.0000000000, propagate_forward_time: 179.3344421387, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8850708008
validation[          , genome 10089] predictions:    6370/  10000 (63.70%), best:    6564/10000 (65.64%), error:     10564.23926, best error:      9924.72754 on epoch:    29, epoch:   31/100, mu: 0.8924399018, learning_rate: 0.0002148183, weight_decay: 0.0000868507

epoch time: 2282.1687011719s, input_fired_time: 32.0946006775, output_fired_time: 13.5387878418, propagate_forward_time: 887.4418334961, propagate_backward_time: 1327.6575927734, weight_update_time: 0.5343320966, other_time: 21.4359130859
epoch time: 182.0768585205s, input_fired_time: 2.3716721535, output_fired_time: 0.0000000000, propagate_forward_time: 175.9254150391, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7797698975
validation[          , genome 10089] predictions:    6576/  10000 (65.76%), best:    6564/10000 (65.64%), error:      9926.09473, best error:      9924.72754 on epoch:    29, epoch:   32/100, mu: 0.8972343802, learning_rate: 0.0001986256, weight_decay: 0.0000819178

epoch time: 2267.0776367188s, input_fired_time: 32.1638107300, output_fired_time: 13.4929151535, propagate_forward_time: 882.6093750000, propagate_backward_time: 1317.5635986328, weight_update_time: 0.5310061574, other_time: 21.2479248047
epoch time: 183.9104309082s, input_fired_time: 2.3819983006, output_fired_time: 0.0000000000, propagate_forward_time: 177.5102996826, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0181274414
validation[          , genome 10089] predictions:    6624/  10000 (66.24%), best:    6624/10000 (66.24%), error:      9755.18555, best error:      9755.18555 on epoch:    33, epoch:   33/100, mu: 0.9017932415, learning_rate: 0.0001836536, weight_decay: 0.0000772651

epoch time: 2291.6640625000s, input_fired_time: 32.2009925842, output_fired_time: 13.5527524948, propagate_forward_time: 890.3723754883, propagate_backward_time: 1334.1485595703, weight_update_time: 0.5250427723, other_time: 21.3895263672
epoch time: 186.4905242920s, input_fired_time: 2.3950583935, output_fired_time: 0.0000000000, propagate_forward_time: 180.2098999023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8855590820
validation[          , genome 10089] predictions:    6536/  10000 (65.36%), best:    6624/10000 (66.24%), error:     10047.71973, best error:      9755.18555 on epoch:    33, epoch:   34/100, mu: 0.9061280489, learning_rate: 0.0001698101, weight_decay: 0.0000728767

epoch time: 2309.6208496094s, input_fired_time: 32.2203903198, output_fired_time: 13.6537389755, propagate_forward_time: 900.5906372070, propagate_backward_time: 1341.3715820313, weight_update_time: 0.5327810049, other_time: 21.7844238281
epoch time: 185.4519195557s, input_fired_time: 2.3766701221, output_fired_time: 0.0000000000, propagate_forward_time: 179.1466522217, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9286041260
validation[          , genome 10089] predictions:    6523/  10000 (65.23%), best:    6624/10000 (66.24%), error:     10107.49316, best error:      9755.18555 on epoch:    33, epoch:   35/100, mu: 0.9102498293, learning_rate: 0.0001570101, weight_decay: 0.0000687375

epoch time: 2285.8364257813s, input_fired_time: 32.2552223206, output_fired_time: 13.5805864334, propagate_forward_time: 888.2854614258, propagate_backward_time: 1330.0357666016, weight_update_time: 0.5304135084, other_time: 21.6795654297
epoch time: 181.1910247803s, input_fired_time: 2.3863985538, output_fired_time: 0.0000000000, propagate_forward_time: 175.1218719482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6827545166
validation[          , genome 10089] predictions:    6598/  10000 (65.98%), best:    6624/10000 (66.24%), error:      9760.58496, best error:      9755.18555 on epoch:    33, epoch:   36/100, mu: 0.9141690731, learning_rate: 0.0001451749, weight_decay: 0.0000648334

epoch time: 2274.1594238281s, input_fired_time: 32.2071533203, output_fired_time: 13.5321283340, propagate_forward_time: 884.7863159180, propagate_backward_time: 1322.1298828125, weight_update_time: 0.5294011831, other_time: 21.5036621094
epoch time: 183.4821777344s, input_fired_time: 2.3766598701, output_fired_time: 0.0000000000, propagate_forward_time: 176.9594573975, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1460571289
validation[          , genome 10089] predictions:    6416/  10000 (64.16%), best:    6624/10000 (66.24%), error:     10212.87402, best error:      9755.18555 on epoch:    33, epoch:   37/100, mu: 0.9178956747, learning_rate: 0.0001342319, weight_decay: 0.0000611510

epoch time: 2295.6218261719s, input_fired_time: 32.2427978516, output_fired_time: 13.5719337463, propagate_forward_time: 893.9473876953, propagate_backward_time: 1334.0084228516, weight_update_time: 0.5280801654, other_time: 21.8510742188
epoch time: 185.6235198975s, input_fired_time: 2.3951630592, output_fired_time: 0.0000000000, propagate_forward_time: 179.2944183350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9339447021
validation[          , genome 10089] predictions:    6703/  10000 (67.03%), best:    6703/10000 (67.03%), error:      9651.73438, best error:      9651.73438 on epoch:    38, epoch:   38/100, mu: 0.9214391708, learning_rate: 0.0001241137, weight_decay: 0.0000576778

epoch time: 2269.5795898438s, input_fired_time: 32.1739196777, output_fired_time: 13.5016336441, propagate_forward_time: 883.7522583008, propagate_backward_time: 1319.0054931641, weight_update_time: 0.5273132920, other_time: 21.1463623047
epoch time: 174.5506744385s, input_fired_time: 2.3458251953, output_fired_time: 0.0000000000, propagate_forward_time: 168.6448974609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5599517822
validation[          , genome 10089] predictions:    6574/  10000 (65.74%), best:    6703/10000 (67.03%), error:      9876.42871, best error:      9651.73438 on epoch:    38, epoch:   39/100, mu: 0.9248085022, learning_rate: 0.0001147582, weight_decay: 0.0000544019

epoch time: 2226.8496093750s, input_fired_time: 32.0999450684, output_fired_time: 13.3319129944, propagate_forward_time: 866.0403442383, propagate_backward_time: 1294.3671875000, weight_update_time: 0.5308467150, other_time: 21.0102539063
epoch time: 181.0541381836s, input_fired_time: 2.4029145241, output_fired_time: 0.0000000000, propagate_forward_time: 174.7311401367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9200897217
validation[          , genome 10089] predictions:    6556/  10000 (65.56%), best:    6703/10000 (67.03%), error:      9915.68262, best error:      9651.73438 on epoch:    38, epoch:   40/100, mu: 0.9280122519, learning_rate: 0.0001061080, weight_decay: 0.0000513120

epoch time: 2197.7192382813s, input_fired_time: 32.0476303101, output_fired_time: 13.2319478989, propagate_forward_time: 854.8820800781, propagate_backward_time: 1277.0554199219, weight_update_time: 0.5401195884, other_time: 20.5021972656
epoch time: 175.1733856201s, input_fired_time: 2.3679430485, output_fired_time: 0.0000000000, propagate_forward_time: 169.0837707520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7216644287
validation[          , genome 10089] predictions:    6626/  10000 (66.26%), best:    6626/10000 (66.26%), error:      9616.32324, best error:      9616.32324 on epoch:    41, epoch:   41/100, mu: 0.9310585856, learning_rate: 0.0000981097, weight_decay: 0.0000483977

epoch time: 2186.5637207031s, input_fired_time: 32.1438560486, output_fired_time: 13.2054328918, propagate_forward_time: 848.0125122070, propagate_backward_time: 1273.1405029297, weight_update_time: 0.5322310925, other_time: 20.0616455078
epoch time: 175.0540771484s, input_fired_time: 2.3693742752, output_fired_time: 0.0000000000, propagate_forward_time: 169.0809326172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6037750244
validation[          , genome 10089] predictions:    6692/  10000 (66.92%), best:    6692/10000 (66.92%), error:      9524.15039, best error:      9524.15039 on epoch:    42, epoch:   42/100, mu: 0.9339551926, learning_rate: 0.0000907144, weight_decay: 0.0000456488

epoch time: 2187.4436035156s, input_fired_time: 32.2000198364, output_fired_time: 13.1423349380, propagate_forward_time: 849.9829101563, propagate_backward_time: 1271.4838867188, weight_update_time: 0.5278993845, other_time: 20.6345214844
epoch time: 179.1711120605s, input_fired_time: 2.3725914955, output_fired_time: 0.0000000000, propagate_forward_time: 172.9551544189, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8433685303
validation[          , genome 10089] predictions:    6741/  10000 (67.41%), best:    6741/10000 (67.41%), error:      9445.65137, best error:      9445.65137 on epoch:    43, epoch:   43/100, mu: 0.9367094636, learning_rate: 0.0000838765, weight_decay: 0.0000430561

epoch time: 2255.3818359375s, input_fired_time: 32.1921424866, output_fired_time: 13.4481878281, propagate_forward_time: 878.4940185547, propagate_backward_time: 1310.0968017578, weight_update_time: 0.5306144357, other_time: 21.1506347656
epoch time: 197.3393096924s, input_fired_time: 2.5645759106, output_fired_time: 0.0000000000, propagate_forward_time: 191.0133209229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7614135742
validation[          , genome 10089] predictions:    6741/  10000 (67.41%), best:    6741/10000 (67.41%), error:      9412.92871, best error:      9412.92871 on epoch:    44, epoch:   44/100, mu: 0.9393283725, learning_rate: 0.0000775540, weight_decay: 0.0000406106

epoch time: 2294.5502929688s, input_fired_time: 33.0329284668, output_fired_time: 13.7752246857, propagate_forward_time: 894.0397338867, propagate_backward_time: 1333.2097167969, weight_update_time: 0.5589225888, other_time: 20.4926757813
epoch time: 188.5829925537s, input_fired_time: 2.4935064316, output_fired_time: 0.0000000000, propagate_forward_time: 182.4574584961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6320343018
validation[          , genome 10089] predictions:    6786/  10000 (67.86%), best:    6786/10000 (67.86%), error:      9348.57324, best error:      9348.57324 on epoch:    45, epoch:   45/100, mu: 0.9418185949, learning_rate: 0.0000717081, weight_decay: 0.0000383041

epoch time: 2297.9472656250s, input_fired_time: 32.8953247070, output_fired_time: 13.6895246506, propagate_forward_time: 892.8017578125, propagate_backward_time: 1337.5495605469, weight_update_time: 0.5355732441, other_time: 21.0112304688
epoch time: 182.5591125488s, input_fired_time: 2.3970005512, output_fired_time: 0.0000000000, propagate_forward_time: 176.3705291748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7915802002
validation[          , genome 10089] predictions:    6753/  10000 (67.53%), best:    6786/10000 (67.86%), error:      9372.54297, best error:      9348.57324 on epoch:    45, epoch:   46/100, mu: 0.9441863894, learning_rate: 0.0000663029, weight_decay: 0.0000361285

epoch time: 2242.7670898438s, input_fired_time: 32.1977577209, output_fired_time: 13.4135951996, propagate_forward_time: 871.4785156250, propagate_backward_time: 1304.9846191406, weight_update_time: 0.5288184881, other_time: 20.6926269531
epoch time: 180.3878173828s, input_fired_time: 2.3827598095, output_fired_time: 0.0000000000, propagate_forward_time: 174.2259826660, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7790679932
validation[          , genome 10089] predictions:    6789/  10000 (67.89%), best:    6789/10000 (67.89%), error:      9256.52832, best error:      9256.52832 on epoch:    47, epoch:   47/100, mu: 0.9464378357, learning_rate: 0.0000613051, weight_decay: 0.0000340765

epoch time: 2283.7128906250s, input_fired_time: 32.2404937744, output_fired_time: 13.5561065674, propagate_forward_time: 887.8095703125, propagate_backward_time: 1328.8878173828, weight_update_time: 0.5363822579, other_time: 21.2188720703
epoch time: 181.6945037842s, input_fired_time: 2.4046494961, output_fired_time: 0.0000000000, propagate_forward_time: 175.6356201172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6542358398
validation[          , genome 10089] predictions:    6613/  10000 (66.13%), best:    6789/10000 (67.89%), error:      9676.29102, best error:      9256.52832 on epoch:    47, epoch:   48/100, mu: 0.9485786557, learning_rate: 0.0000566840, weight_decay: 0.0000321411

epoch time: 2274.0063476563s, input_fired_time: 32.3862953186, output_fired_time: 13.4596824646, propagate_forward_time: 884.6620483398, propagate_backward_time: 1322.1861572266, weight_update_time: 0.5344470143, other_time: 21.3121337891
epoch time: 183.4764556885s, input_fired_time: 2.4109005928, output_fired_time: 0.0000000000, propagate_forward_time: 177.1785583496, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8869934082
validation[          , genome 10089] predictions:    6748/  10000 (67.48%), best:    6789/10000 (67.89%), error:      9425.63770, best error:      9256.52832 on epoch:    47, epoch:   49/100, mu: 0.9506142735, learning_rate: 0.0000524113, weight_decay: 0.0000303156

epoch time: 2291.7080078125s, input_fired_time: 32.5175209045, output_fired_time: 13.6190242767, propagate_forward_time: 891.6659545898, propagate_backward_time: 1332.4885253906, weight_update_time: 0.5355923176, other_time: 21.4167480469
epoch time: 183.2316894531s, input_fired_time: 2.4056789875, output_fired_time: 0.0000000000, propagate_forward_time: 177.2154083252, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6105957031
validation[          , genome 10089] predictions:    6765/  10000 (67.65%), best:    6789/10000 (67.89%), error:      9440.10156, best error:      9256.52832 on epoch:    47, epoch:   50/100, mu: 0.9525498152, learning_rate: 0.0000484606, weight_decay: 0.0000285937

epoch time: 2323.8100585938s, input_fired_time: 32.4852066040, output_fired_time: 13.7561521530, propagate_forward_time: 903.2645263672, propagate_backward_time: 1352.5032958984, weight_update_time: 0.5368318558, other_time: 21.8010253906
epoch time: 188.5819091797s, input_fired_time: 2.4539446831, output_fired_time: 0.0000000000, propagate_forward_time: 182.1109313965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0170288086
validation[          , genome 10089] predictions:    6798/  10000 (67.98%), best:    6798/10000 (67.98%), error:      9234.85352, best error:      9234.85352 on epoch:    51, epoch:   51/100, mu: 0.9543902874, learning_rate: 0.0000448077, weight_decay: 0.0000269697

epoch time: 2368.7495117188s, input_fired_time: 32.5652732849, output_fired_time: 13.8770675659, propagate_forward_time: 920.1530151367, propagate_backward_time: 1379.7197265625, weight_update_time: 0.5331214666, other_time: 22.4345703125
epoch time: 192.1542358398s, input_fired_time: 2.4465816021, output_fired_time: 0.0000000000, propagate_forward_time: 185.6625213623, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0451354980
validation[          , genome 10089] predictions:    6805/  10000 (68.05%), best:    6798/10000 (67.98%), error:      9385.47949, best error:      9234.85352 on epoch:    51, epoch:   52/100, mu: 0.9561402798, learning_rate: 0.0000414302, weight_decay: 0.0000254379

epoch time: 2398.2265625000s, input_fired_time: 32.5826377869, output_fired_time: 14.0715789795, propagate_forward_time: 932.3430786133, propagate_backward_time: 1395.9897460938, weight_update_time: 0.5363730788, other_time: 23.2397460938
epoch time: 194.5882415771s, input_fired_time: 2.4636085033, output_fired_time: 0.0000000000, propagate_forward_time: 187.9691772461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1554565430
validation[          , genome 10089] predictions:    6754/  10000 (67.54%), best:    6798/10000 (67.98%), error:      9370.20020, best error:      9234.85352 on epoch:    51, epoch:   53/100, mu: 0.9578042626, learning_rate: 0.0000383073, weight_decay: 0.0000239931

epoch time: 2394.7128906250s, input_fired_time: 32.6282196045, output_fired_time: 14.0048666000, propagate_forward_time: 930.6883544922, propagate_backward_time: 1394.7153320313, weight_update_time: 0.5391690731, other_time: 22.6761474609
epoch time: 190.2875671387s, input_fired_time: 2.4336619377, output_fired_time: 0.0000000000, propagate_forward_time: 183.8041992188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0497131348
validation[          , genome 10089] predictions:    6806/  10000 (68.06%), best:    6798/10000 (67.98%), error:      9310.33691, best error:      9234.85352 on epoch:    51, epoch:   54/100, mu: 0.9593864679, learning_rate: 0.0000354197, weight_decay: 0.0000226303

epoch time: 2384.0764160156s, input_fired_time: 32.6737022400, output_fired_time: 13.9613924026, propagate_forward_time: 927.7016601563, propagate_backward_time: 1387.2718505859, weight_update_time: 0.5382216573, other_time: 22.4678955078
epoch time: 189.8940429688s, input_fired_time: 2.4457633495, output_fired_time: 0.0000000000, propagate_forward_time: 183.4331512451, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0151214600
validation[          , genome 10089] predictions:    6822/  10000 (68.22%), best:    6798/10000 (67.98%), error:      9255.31641, best error:      9234.85352 on epoch:    51, epoch:   55/100, mu: 0.9608909488, learning_rate: 0.0000327498, weight_decay: 0.0000213450

epoch time: 2402.3117675781s, input_fired_time: 33.0466842651, output_fired_time: 14.1518650055, propagate_forward_time: 933.1026611328, propagate_backward_time: 1399.4067382813, weight_update_time: 0.5448096395, other_time: 22.6038818359
epoch time: 191.6347045898s, input_fired_time: 2.4613046646, output_fired_time: 0.0000000000, propagate_forward_time: 185.2448577881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9285430908
validation[          , genome 10089] predictions:    6831/  10000 (68.31%), best:    6798/10000 (67.98%), error:      9265.57520, best error:      9234.85352 on epoch:    51, epoch:   56/100, mu: 0.9623214602, learning_rate: 0.0000302812, weight_decay: 0.0000201327

epoch time: 2362.2414550781s, input_fired_time: 32.5793418884, output_fired_time: 13.9770412445, propagate_forward_time: 915.5186157227, propagate_backward_time: 1378.1235351563, weight_update_time: 0.5363732576, other_time: 22.0429687500
epoch time: 188.7200164795s, input_fired_time: 2.4405772686, output_fired_time: 0.0000000000, propagate_forward_time: 182.2785186768, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0009155273
validation[          , genome 10089] predictions:    6836/  10000 (68.36%), best:    6836/10000 (68.36%), error:      9132.64746, best error:      9132.64746 on epoch:    57, epoch:   57/100, mu: 0.9636816978, learning_rate: 0.0000279987, weight_decay: 0.0000189892

epoch time: 2368.1416015625s, input_fired_time: 32.6548728943, output_fired_time: 13.8793687820, propagate_forward_time: 918.8096923828, propagate_backward_time: 1380.4223632813, weight_update_time: 0.5323398113, other_time: 22.3753662109
epoch time: 189.1837158203s, input_fired_time: 2.4345176220, output_fired_time: 0.0000000000, propagate_forward_time: 182.6266937256, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1224975586
validation[          , genome 10089] predictions:    6834/  10000 (68.34%), best:    6836/10000 (68.36%), error:      9250.26563, best error:      9132.64746 on epoch:    57, epoch:   58/100, mu: 0.9649750590, learning_rate: 0.0000258882, weight_decay: 0.0000179107

epoch time: 2380.3391113281s, input_fired_time: 32.6682434082, output_fired_time: 13.9390659332, propagate_forward_time: 926.2120361328, propagate_backward_time: 1384.9553222656, weight_update_time: 0.5402927399, other_time: 22.5645751953
epoch time: 197.1048583984s, input_fired_time: 2.5205943584, output_fired_time: 0.0000000000, propagate_forward_time: 190.2077484131, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3765106201
validation[          , genome 10089] predictions:    6840/  10000 (68.40%), best:    6836/10000 (68.36%), error:      9197.33789, best error:      9132.64746 on epoch:    57, epoch:   59/100, mu: 0.9662048817, learning_rate: 0.0000239368, weight_decay: 0.0000168934

epoch time: 2411.1960449219s, input_fired_time: 32.5868682861, output_fired_time: 14.0746364594, propagate_forward_time: 936.8762207031, propagate_backward_time: 1404.4448242188, weight_update_time: 0.5305531621, other_time: 23.2133789063
epoch time: 193.1619567871s, input_fired_time: 2.4390203953, output_fired_time: 0.0000000000, propagate_forward_time: 186.3881378174, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3347930908
validation[          , genome 10089] predictions:    6840/  10000 (68.40%), best:    6836/10000 (68.36%), error:      9152.97852, best error:      9132.64746 on epoch:    57, epoch:   60/100, mu: 0.9673742652, learning_rate: 0.0000221325, weight_decay: 0.0000159339

epoch time: 2409.8247070313s, input_fired_time: 32.5842628479, output_fired_time: 14.0561189651, propagate_forward_time: 935.5402832031, propagate_backward_time: 1404.0957031250, weight_update_time: 0.5319596529, other_time: 23.5483398438
epoch time: 195.2425842285s, input_fired_time: 2.4807558060, output_fired_time: 0.0000000000, propagate_forward_time: 188.5684204102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1934051514
validation[          , genome 10089] predictions:    6871/  10000 (68.71%), best:    6871/10000 (68.71%), error:      9059.86914, best error:      9059.86914 on epoch:    61, epoch:   61/100, mu: 0.9684861898, learning_rate: 0.0000204642, weight_decay: 0.0000150289

epoch time: 2409.8310546875s, input_fired_time: 32.5829315186, output_fired_time: 14.0643091202, propagate_forward_time: 936.1450195313, propagate_backward_time: 1403.9832763672, weight_update_time: 0.5328292847, other_time: 23.0555419922
epoch time: 193.6644592285s, input_fired_time: 2.4550869465, output_fired_time: 0.0000000000, propagate_forward_time: 187.0614166260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1479492188
validation[          , genome 10089] predictions:    6868/  10000 (68.68%), best:    6871/10000 (68.71%), error:      9150.57910, best error:      9059.86914 on epoch:    61, epoch:   62/100, mu: 0.9695434570, learning_rate: 0.0000189216, weight_decay: 0.0000141753

epoch time: 2411.0275878906s, input_fired_time: 32.5358581543, output_fired_time: 14.0521497726, propagate_forward_time: 937.6877441406, propagate_backward_time: 1403.4589843750, weight_update_time: 0.5416007638, other_time: 23.2927246094
epoch time: 193.7685852051s, input_fired_time: 2.4526476860, output_fired_time: 0.0000000000, propagate_forward_time: 186.9563598633, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3595733643
validation[          , genome 10089] predictions:    6867/  10000 (68.67%), best:    6867/10000 (68.67%), error:      9049.61621, best error:      9049.61621 on epoch:    63, epoch:   63/100, mu: 0.9705487490, learning_rate: 0.0000174953, weight_decay: 0.0000133702

epoch time: 2414.1477050781s, input_fired_time: 32.6258773804, output_fired_time: 14.1048336029, propagate_forward_time: 938.1652832031, propagate_backward_time: 1406.1588134766, weight_update_time: 0.5356706381, other_time: 23.0928955078
epoch time: 193.0440368652s, input_fired_time: 2.4547216892, output_fired_time: 0.0000000000, propagate_forward_time: 186.3438415527, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2454681396
validation[          , genome 10089] predictions:    6876/  10000 (68.76%), best:    6876/10000 (68.76%), error:      9010.14258, best error:      9010.14258 on epoch:    64, epoch:   64/100, mu: 0.9715046883, learning_rate: 0.0000161766, weight_decay: 0.0000126108

epoch time: 2412.4038085938s, input_fired_time: 32.5869331360, output_fired_time: 14.0953111649, propagate_forward_time: 937.9665527344, propagate_backward_time: 1405.0637207031, weight_update_time: 0.5361585021, other_time: 22.6914062500
epoch time: 192.9979248047s, input_fired_time: 2.4454367161, output_fired_time: 0.0000000000, propagate_forward_time: 186.3140411377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2384490967
validation[          , genome 10089] predictions:    6899/  10000 (68.99%), best:    6899/10000 (68.99%), error:      8970.45020, best error:      8970.45020 on epoch:    65, epoch:   65/100, mu: 0.9724135995, learning_rate: 0.0000149572, weight_decay: 0.0000118945

epoch time: 2410.1447753906s, input_fired_time: 32.5736618042, output_fired_time: 14.0755796432, propagate_forward_time: 936.2192993164, propagate_backward_time: 1404.1445312500, weight_update_time: 0.5330966115, other_time: 23.1315917969
epoch time: 195.0868225098s, input_fired_time: 2.4685118198, output_fired_time: 0.0000000000, propagate_forward_time: 188.2346801758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3836364746
validation[          , genome 10089] predictions:    6852/  10000 (68.52%), best:    6899/10000 (68.99%), error:      9027.81445, best error:      8970.45020 on epoch:    65, epoch:   66/100, mu: 0.9732778668, learning_rate: 0.0000138297, weight_decay: 0.0000112190

epoch time: 2413.4992675781s, input_fired_time: 32.6179885864, output_fired_time: 14.0919303894, propagate_forward_time: 938.6527709961, propagate_backward_time: 1405.0733642578, weight_update_time: 0.5336135626, other_time: 23.0631103516
epoch time: 193.6177673340s, input_fired_time: 2.4589500427, output_fired_time: 0.0000000000, propagate_forward_time: 187.2393493652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9194641113
validation[          , genome 10089] predictions:    6870/  10000 (68.70%), best:    6899/10000 (68.99%), error:      8993.50488, best error:      8970.45020 on epoch:    65, epoch:   67/100, mu: 0.9740996361, learning_rate: 0.0000127873, weight_decay: 0.0000105818

epoch time: 2413.7685546875s, input_fired_time: 32.6258506775, output_fired_time: 14.0546903610, propagate_forward_time: 937.3220825195, propagate_backward_time: 1406.6699218750, weight_update_time: 0.5325562358, other_time: 23.0961914063
epoch time: 193.3413238525s, input_fired_time: 2.4457798004, output_fired_time: 0.0000000000, propagate_forward_time: 186.6001586914, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2953796387
validation[          , genome 10089] predictions:    6907/  10000 (69.07%), best:    6907/10000 (69.07%), error:      8958.14844, best error:      8958.14844 on epoch:    68, epoch:   68/100, mu: 0.9748810530, learning_rate: 0.0000118234, weight_decay: 0.0000099807

epoch time: 2379.9792480469s, input_fired_time: 32.6459274292, output_fired_time: 13.9731416702, propagate_forward_time: 925.1539916992, propagate_backward_time: 1385.3311767578, weight_update_time: 0.5420925617, other_time: 22.8748779297
epoch time: 192.0627441406s, input_fired_time: 2.4635560513, output_fired_time: 0.0000000000, propagate_forward_time: 185.6141967773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9849853516
validation[          , genome 10089] predictions:    6904/  10000 (69.04%), best:    6904/10000 (69.04%), error:      8926.69824, best error:      8926.69824 on epoch:    69, epoch:   69/100, mu: 0.9756240845, learning_rate: 0.0000109322, weight_decay: 0.0000094139

epoch time: 2319.9729003906s, input_fired_time: 32.5980148315, output_fired_time: 13.7002878189, propagate_forward_time: 900.9289550781, propagate_backward_time: 1350.6098632813, weight_update_time: 0.5363409519, other_time: 22.1359863281
epoch time: 184.1092071533s, input_fired_time: 2.4197950363, output_fired_time: 0.0000000000, propagate_forward_time: 177.6959991455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9934082031
validation[          , genome 10089] predictions:    6888/  10000 (68.88%), best:    6904/10000 (69.04%), error:      9008.92969, best error:      8926.69824 on epoch:    69, epoch:   70/100, mu: 0.9763305783, learning_rate: 0.0000101081, weight_decay: 0.0000088792

epoch time: 2342.7961425781s, input_fired_time: 33.1176109314, output_fired_time: 13.9982671738, propagate_forward_time: 909.7082519531, propagate_backward_time: 1364.0334472656, weight_update_time: 0.5467241406, other_time: 21.9384765625
epoch time: 190.7270050049s, input_fired_time: 2.5657315254, output_fired_time: 0.0000000000, propagate_forward_time: 184.2569274902, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9043426514
validation[          , genome 10089] predictions:    6884/  10000 (68.84%), best:    6904/10000 (69.04%), error:      9001.77441, best error:      8926.69824 on epoch:    69, epoch:   71/100, mu: 0.9770023227, learning_rate: 0.0000093462, weight_decay: 0.0000083749

epoch time: 2360.3569335938s, input_fired_time: 33.5404739380, output_fired_time: 14.1170902252, propagate_forward_time: 919.8541870117, propagate_backward_time: 1371.3720703125, weight_update_time: 0.5502080917, other_time: 21.4729003906
epoch time: 192.7740173340s, input_fired_time: 2.5339288712, output_fired_time: 0.0000000000, propagate_forward_time: 186.3531951904, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8868865967
validation[          , genome 10089] predictions:    6928/  10000 (69.28%), best:    6904/10000 (69.04%), error:      8958.63672, best error:      8926.69824 on epoch:    69, epoch:   72/100, mu: 0.9776411057, learning_rate: 0.0000086417, weight_decay: 0.0000078992

epoch time: 2385.0195312500s, input_fired_time: 33.4533920288, output_fired_time: 14.1597194672, propagate_forward_time: 930.5009155273, propagate_backward_time: 1385.3190917969, weight_update_time: 0.5652192235, other_time: 21.5864257813
epoch time: 190.9207611084s, input_fired_time: 2.4836127758, output_fired_time: 0.0000000000, propagate_forward_time: 184.6022644043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8348846436
validation[          , genome 10089] predictions:    6909/  10000 (69.09%), best:    6909/10000 (69.09%), error:      8926.59668, best error:      8926.59668 on epoch:    73, epoch:   73/100, mu: 0.9782484770, learning_rate: 0.0000079903, weight_decay: 0.0000074506

epoch time: 2507.1232910156s, input_fired_time: 35.0478706360, output_fired_time: 14.8781108856, propagate_forward_time: 977.6635131836, propagate_backward_time: 1458.4755859375, weight_update_time: 0.5683426261, other_time: 21.0581054688
epoch time: 188.5466461182s, input_fired_time: 2.4963202477, output_fired_time: 0.0000000000, propagate_forward_time: 182.2095031738, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8408203125
validation[          , genome 10089] predictions:    6916/  10000 (69.16%), best:    6909/10000 (69.09%), error:      8944.91016, best error:      8926.59668 on epoch:    73, epoch:   74/100, mu: 0.9788259864, learning_rate: 0.0000073880, weight_decay: 0.0000070274

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1533295503_162_10089.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1533295503_162_10089_2_r1006524637_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.524721
read mu: 0.979375
read mu_delta: 0.950856
read initial_learning_rate: 0.00243885
read learning_rate: 6.8311e-06
read learning_rate_delta: 0.924622
read initial_weight_decay: 0.000532125
read weight_decay: 6.62825e-06
read weight_decay_delta: 0.943203
read batch_size: 91
read epsilon: 1e-07
read alpha: 0.0776757
read input_dropout_probability: 0.00120512
read hidden_dropout_probability: 0.108781
read velocity_reset: 1745
read epoch: 75
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 73
read number_validation_images: 10000
read best_validation_predictions: 6909
read best_validation_error: 8926.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 10089
read normal distribution: '1 -0.991247296333313 1.04744446277618'
generator_str: '1331054342'
read generator: 1331054342
read generated_by_map:
1 merge_node 1
reading 146 nodes.
reading 570 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2531.16162109375s, input_fired_time: 36.0479965209961, output_fired_time: 14.989673614502, propagate_forward_time: 986.606323242188, propagate_backward_time: 1470.27392578125, weight_update_time: 0.623606383800507, other_time: 23.2435302734375
epoch time: 199.993865966797s, input_fired_time: 2.6507420539856, output_fired_time: 0, propagate_forward_time: 193.027374267578, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.31575012207031
validation[          , genome 10089] predictions:    6932/  10000 (69.32%), best:    6932/10000 (69.32%), error:      8923.99609, best error:      8923.99609 on epoch:    75, epoch:   75/100, mu: 0.9793751240, learning_rate: 0.0000068311, weight_decay: 0.0000066282

epoch time: 2327.9467773438s, input_fired_time: 32.7658348083, output_fired_time: 13.7777051926, propagate_forward_time: 904.5769042969, propagate_backward_time: 1353.0059814453, weight_update_time: 0.5388361216, other_time: 23.8204345703
epoch time: 187.0675048828s, input_fired_time: 2.4520766735, output_fired_time: 0.0000000000, propagate_forward_time: 180.3887023926, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2267303467
validation[          , genome 10089] predictions:    6917/  10000 (69.17%), best:    6917/10000 (69.17%), error:      8916.16699, best error:      8916.16699 on epoch:    76, epoch:   76/100, mu: 0.9798972607, learning_rate: 0.0000063162, weight_decay: 0.0000062518

epoch time: 2399.5983886719s, input_fired_time: 33.2156333923, output_fired_time: 14.0828418732, propagate_forward_time: 933.0449829102, propagate_backward_time: 1394.7084960938, weight_update_time: 0.5423528552, other_time: 24.5466308594
epoch time: 192.3540649414s, input_fired_time: 2.4384384155, output_fired_time: 0.0000000000, propagate_forward_time: 185.1457672119, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7698516846
validation[          , genome 10089] predictions:    6921/  10000 (69.21%), best:    6917/10000 (69.17%), error:      8936.13574, best error:      8916.16699 on epoch:    76, epoch:   77/100, mu: 0.9803937674, learning_rate: 0.0000058401, weight_decay: 0.0000058967

epoch time: 2410.8247070313s, input_fired_time: 32.9135360718, output_fired_time: 14.1378202438, propagate_forward_time: 935.8835449219, propagate_backward_time: 1402.7144775391, weight_update_time: 0.5386885405, other_time: 25.1751708984
epoch time: 192.7225189209s, input_fired_time: 2.4280381203, output_fired_time: 0.0000000000, propagate_forward_time: 185.5463714600, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7481079102
validation[          , genome 10089] predictions:    6918/  10000 (69.18%), best:    6918/10000 (69.18%), error:      8877.69238, best error:      8877.69238 on epoch:    78, epoch:   78/100, mu: 0.9808658361, learning_rate: 0.0000053999, weight_decay: 0.0000055618

epoch time: 2408.3828125000s, input_fired_time: 33.1393852234, output_fired_time: 14.1127128601, propagate_forward_time: 936.0556640625, propagate_backward_time: 1400.2634277344, weight_update_time: 0.5381556749, other_time: 24.8115234375
epoch time: 193.3489532471s, input_fired_time: 2.4461374283, output_fired_time: 0.0000000000, propagate_forward_time: 186.2880554199, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6147613525
validation[          , genome 10089] predictions:    6930/  10000 (69.30%), best:    6918/10000 (69.18%), error:      8917.09961, best error:      8877.69238 on epoch:    78, epoch:   79/100, mu: 0.9813147187, learning_rate: 0.0000049928, weight_decay: 0.0000052459

epoch time: 2409.6240234375s, input_fired_time: 33.2324447632, output_fired_time: 14.1899309158, propagate_forward_time: 936.9205932617, propagate_backward_time: 1399.8520507813, weight_update_time: 0.5403282046, other_time: 25.4289550781
epoch time: 193.6746826172s, input_fired_time: 2.4382543564, output_fired_time: 0.0000000000, propagate_forward_time: 186.4051818848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8312530518
validation[          , genome 10089] predictions:    6934/  10000 (69.34%), best:    6918/10000 (69.18%), error:      8905.85352, best error:      8877.69238 on epoch:    78, epoch:   80/100, mu: 0.9817415476, learning_rate: 0.0000046165, weight_decay: 0.0000049479

epoch time: 2412.9072265625s, input_fired_time: 33.2735443115, output_fired_time: 14.3069906235, propagate_forward_time: 937.0869750977, propagate_backward_time: 1403.2012939453, weight_update_time: 0.5422052145, other_time: 25.0386962891
epoch time: 200.3795318604s, input_fired_time: 2.6704742908, output_fired_time: 0.0000000000, propagate_forward_time: 192.8121185303, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8969421387
validation[          , genome 10089] predictions:    6922/  10000 (69.22%), best:    6918/10000 (69.18%), error:      8909.64160, best error:      8877.69238 on epoch:    78, epoch:   81/100, mu: 0.9821473956, learning_rate: 0.0000042685, weight_decay: 0.0000046669

epoch time: 2410.5432128906s, input_fired_time: 32.9630775452, output_fired_time: 14.1428775787, propagate_forward_time: 936.8048095703, propagate_backward_time: 1401.7075195313, weight_update_time: 0.5397272706, other_time: 24.9249267578
epoch time: 198.4635620117s, input_fired_time: 2.5066027641, output_fired_time: 0.0000000000, propagate_forward_time: 191.2835388184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6734161377
validation[          , genome 10089] predictions:    6889/  10000 (68.89%), best:    6918/10000 (69.18%), error:      8904.34961, best error:      8877.69238 on epoch:    78, epoch:   82/100, mu: 0.9825332761, learning_rate: 0.0000039467, weight_decay: 0.0000044018

epoch time: 2403.9716796875s, input_fired_time: 32.7321739197, output_fired_time: 14.0191936493, propagate_forward_time: 934.8861083984, propagate_backward_time: 1397.3428955078, weight_update_time: 0.5436037779, other_time: 24.9912109375
epoch time: 192.4722290039s, input_fired_time: 2.4330861568, output_fired_time: 0.0000000000, propagate_forward_time: 185.2571105957, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7820281982
validation[          , genome 10089] predictions:    6929/  10000 (69.29%), best:    6918/10000 (69.18%), error:      8932.29688, best error:      8877.69238 on epoch:    78, epoch:   83/100, mu: 0.9829002023, learning_rate: 0.0000036492, weight_decay: 0.0000041518

epoch time: 2418.8586425781s, input_fired_time: 33.0970039368, output_fired_time: 14.1524085999, propagate_forward_time: 941.1715087891, propagate_backward_time: 1405.4763183594, weight_update_time: 0.5438854098, other_time: 24.9615478516
epoch time: 194.6421813965s, input_fired_time: 2.4585847855, output_fired_time: 0.0000000000, propagate_forward_time: 187.4760589600, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7075347900
validation[          , genome 10089] predictions:    6929/  10000 (69.29%), best:    6918/10000 (69.18%), error:      8888.80957, best error:      8877.69238 on epoch:    78, epoch:   84/100, mu: 0.9832491279, learning_rate: 0.0000033742, weight_decay: 0.0000039160

epoch time: 2413.2431640625s, input_fired_time: 33.0022087097, output_fired_time: 14.1324157715, propagate_forward_time: 940.2593383789, propagate_backward_time: 1401.2812500000, weight_update_time: 0.5419989824, other_time: 24.5681152344
epoch time: 192.6497344971s, input_fired_time: 2.4332771301, output_fired_time: 0.0000000000, propagate_forward_time: 185.5563201904, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6601409912
validation[          , genome 10089] predictions:    6926/  10000 (69.26%), best:    6918/10000 (69.18%), error:      8925.57324, best error:      8877.69238 on epoch:    78, epoch:   85/100, mu: 0.9835808873, learning_rate: 0.0000031198, weight_decay: 0.0000036936

epoch time: 2397.6440429688s, input_fired_time: 33.1708793640, output_fired_time: 14.0282125473, propagate_forward_time: 931.4166870117, propagate_backward_time: 1394.3598632813, weight_update_time: 0.5437411666, other_time: 24.6682128906
epoch time: 188.4575653076s, input_fired_time: 2.4143707752, output_fired_time: 0.0000000000, propagate_forward_time: 181.2708282471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7723693848
validation[          , genome 10089] predictions:    6948/  10000 (69.48%), best:    6948/10000 (69.48%), error:      8875.64746, best error:      8875.64746 on epoch:    86, epoch:   86/100, mu: 0.9838963747, learning_rate: 0.0000028847, weight_decay: 0.0000034838

epoch time: 2358.7917480469s, input_fired_time: 32.8307266235, output_fired_time: 13.9550151825, propagate_forward_time: 914.9888305664, propagate_backward_time: 1372.6740722656, weight_update_time: 0.5383349657, other_time: 24.3430175781
epoch time: 189.8453369141s, input_fired_time: 2.4389092922, output_fired_time: 0.0000000000, propagate_forward_time: 183.0593566895, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3470764160
validation[          , genome 10089] predictions:    6926/  10000 (69.26%), best:    6926/10000 (69.26%), error:      8857.30273, best error:      8857.30273 on epoch:    87, epoch:   87/100, mu: 0.9841963053, learning_rate: 0.0000026672, weight_decay: 0.0000032859

epoch time: 2364.2177734375s, input_fired_time: 32.8400688171, output_fired_time: 13.9713020325, propagate_forward_time: 919.3209228516, propagate_backward_time: 1373.8645019531, weight_update_time: 0.5396896005, other_time: 24.2210693359
epoch time: 188.6314544678s, input_fired_time: 2.4536597729, output_fired_time: 0.0000000000, propagate_forward_time: 181.7101593018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4676361084
validation[          , genome 10089] predictions:    6949/  10000 (69.49%), best:    6926/10000 (69.26%), error:      8890.88770, best error:      8857.30273 on epoch:    87, epoch:   88/100, mu: 0.9844815135, learning_rate: 0.0000024662, weight_decay: 0.0000030993

epoch time: 2364.9389648438s, input_fired_time: 33.0858306885, output_fired_time: 13.9724102020, propagate_forward_time: 920.1441040039, propagate_backward_time: 1373.3139648438, weight_update_time: 0.5359668732, other_time: 24.4226074219
epoch time: 191.4408111572s, input_fired_time: 2.4893898964, output_fired_time: 0.0000000000, propagate_forward_time: 184.4576721191, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4937438965
validation[          , genome 10089] predictions:    6935/  10000 (69.35%), best:    6926/10000 (69.26%), error:      8882.89844, best error:      8857.30273 on epoch:    87, epoch:   89/100, mu: 0.9847527146, learning_rate: 0.0000022803, weight_decay: 0.0000029233

epoch time: 2343.2785644531s, input_fired_time: 32.8362312317, output_fired_time: 13.8462696075, propagate_forward_time: 911.7015991211, propagate_backward_time: 1360.7584228516, weight_update_time: 0.5376272798, other_time: 24.1361083984
epoch time: 190.1929168701s, input_fired_time: 2.4494252205, output_fired_time: 0.0000000000, propagate_forward_time: 183.4086914063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3347930908
validation[          , genome 10089] predictions:    6940/  10000 (69.40%), best:    6940/10000 (69.40%), error:      8844.00293, best error:      8844.00293 on epoch:    90, epoch:   90/100, mu: 0.9850105643, learning_rate: 0.0000021084, weight_decay: 0.0000027572

epoch time: 2354.6625976563s, input_fired_time: 33.1659202576, output_fired_time: 13.9249305725, propagate_forward_time: 913.7847900391, propagate_backward_time: 1370.3321533203, weight_update_time: 0.5392014980, other_time: 23.4545898438
epoch time: 189.5859985352s, input_fired_time: 2.4556050301, output_fired_time: 0.0000000000, propagate_forward_time: 182.5371856689, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5932006836
validation[          , genome 10089] predictions:    6927/  10000 (69.27%), best:    6940/10000 (69.40%), error:      8863.22754, best error:      8844.00293 on epoch:    90, epoch:   91/100, mu: 0.9852557778, learning_rate: 0.0000019495, weight_decay: 0.0000026006

epoch time: 2322.0710449219s, input_fired_time: 32.9712333679, output_fired_time: 13.8059310913, propagate_forward_time: 903.7062988281, propagate_backward_time: 1347.9943847656, weight_update_time: 0.5341756940, other_time: 23.5932617188
epoch time: 184.0512847900s, input_fired_time: 2.4189586639, output_fired_time: 0.0000000000, propagate_forward_time: 177.1711578369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4611663818
validation[          , genome 10089] predictions:    6943/  10000 (69.43%), best:    6940/10000 (69.40%), error:      8844.83398, best error:      8844.00293 on epoch:    90, epoch:   92/100, mu: 0.9854889512, learning_rate: 0.0000018025, weight_decay: 0.0000024529

epoch time: 2317.5424804688s, input_fired_time: 33.1903839111, output_fired_time: 13.8046655655, propagate_forward_time: 902.1354370117, propagate_backward_time: 1345.3421630859, weight_update_time: 0.5347422957, other_time: 23.0697021484
epoch time: 186.0911712646s, input_fired_time: 2.4435048103, output_fired_time: 0.0000000000, propagate_forward_time: 179.2036437988, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4440155029
validation[          , genome 10089] predictions:    6920/  10000 (69.20%), best:    6940/10000 (69.40%), error:      8864.40039, best error:      8844.00293 on epoch:    90, epoch:   93/100, mu: 0.9857106209, learning_rate: 0.0000016666, weight_decay: 0.0000023136

epoch time: 2323.2048339844s, input_fired_time: 33.2002372742, output_fired_time: 13.8242225647, propagate_forward_time: 903.3619384766, propagate_backward_time: 1349.4398193359, weight_update_time: 0.5475087762, other_time: 23.3786621094
epoch time: 186.3742828369s, input_fired_time: 2.4918489456, output_fired_time: 0.0000000000, propagate_forward_time: 179.5041198730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3783111572
validation[          , genome 10089] predictions:    6951/  10000 (69.51%), best:    6951/10000 (69.51%), error:      8831.58789, best error:      8831.58789 on epoch:    94, epoch:   94/100, mu: 0.9859214425, learning_rate: 0.0000015410, weight_decay: 0.0000021822

epoch time: 2303.7097167969s, input_fired_time: 32.9057006836, output_fired_time: 13.6688661575, propagate_forward_time: 893.5379028320, propagate_backward_time: 1340.3085937500, weight_update_time: 0.5721511245, other_time: 23.2885742188
epoch time: 184.4980316162s, input_fired_time: 2.4443838596, output_fired_time: 0.0000000000, propagate_forward_time: 177.9607086182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0929412842
validation[          , genome 10089] predictions:    6928/  10000 (69.28%), best:    6951/10000 (69.51%), error:      8835.22852, best error:      8831.58789 on epoch:    94, epoch:   95/100, mu: 0.9861218929, learning_rate: 0.0000014249, weight_decay: 0.0000020583

epoch time: 2335.9775390625s, input_fired_time: 33.5735015869, output_fired_time: 13.9838294983, propagate_forward_time: 908.5385131836, propagate_backward_time: 1357.1151123047, weight_update_time: 0.5501576066, other_time: 22.7664794922
epoch time: 183.8748626709s, input_fired_time: 2.4328505993, output_fired_time: 0.0000000000, propagate_forward_time: 177.1629638672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2790527344
validation[          , genome 10089] predictions:    6939/  10000 (69.39%), best:    6951/10000 (69.51%), error:      8848.07617, best error:      8831.58789 on epoch:    94, epoch:   96/100, mu: 0.9863124490, learning_rate: 0.0000013175, weight_decay: 0.0000019414

epoch time: 2333.0957031250s, input_fired_time: 33.5947875977, output_fired_time: 13.8838653564, propagate_forward_time: 907.6246337891, propagate_backward_time: 1354.9375000000, weight_update_time: 0.5452994108, other_time: 23.0550537109
epoch time: 184.5573883057s, input_fired_time: 2.4475452900, output_fired_time: 0.0000000000, propagate_forward_time: 177.8432312012, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2666168213
validation[          , genome 10089] predictions:    6937/  10000 (69.37%), best:    6951/10000 (69.51%), error:      8846.86426, best error:      8831.58789 on epoch:    94, epoch:   97/100, mu: 0.9864936471, learning_rate: 0.0000012181, weight_decay: 0.0000018311

epoch time: 2310.6237792969s, input_fired_time: 33.5275268555, output_fired_time: 13.8019781113, propagate_forward_time: 893.7626953125, propagate_backward_time: 1346.6574707031, weight_update_time: 0.5472434759, other_time: 22.8740234375
epoch time: 189.2165374756s, input_fired_time: 2.4743607044, output_fired_time: 0.0000000000, propagate_forward_time: 182.3893890381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3527832031
validation[          , genome 10089] predictions:    6927/  10000 (69.27%), best:    6951/10000 (69.51%), error:      8860.35059, best error:      8831.58789 on epoch:    94, epoch:   98/100, mu: 0.9866659641, learning_rate: 0.0000011263, weight_decay: 0.0000017271

epoch time: 2289.0065917969s, input_fired_time: 33.0639305115, output_fired_time: 13.6729707718, propagate_forward_time: 888.7747802734, propagate_backward_time: 1331.0393066406, weight_update_time: 0.5369036198, other_time: 22.4556884766
epoch time: 210.6981506348s, input_fired_time: 2.8189468384, output_fired_time: 0.0000000000, propagate_forward_time: 203.8439331055, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0352783203
validation[          , genome 10089] predictions:    6940/  10000 (69.40%), best:    6951/10000 (69.51%), error:      8846.95215, best error:      8831.58789 on epoch:    94, epoch:   99/100, mu: 0.9868298173, learning_rate: 0.0000010414, weight_decay: 0.0000016290

epoch time: 2276.2800292969s, input_fired_time: 33.0011940002, output_fired_time: 13.5960159302, propagate_forward_time: 883.6198120117, propagate_backward_time: 1323.6113281250, weight_update_time: 0.5360227227, other_time: 22.4516601563
epoch time: 184.4963989258s, input_fired_time: 2.4345715046, output_fired_time: 0.0000000000, propagate_forward_time: 177.9511718750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1106567383
validation[          , genome 10089] predictions:    6942/  10000 (69.42%), best:    6942/10000 (69.42%), error:      8823.50098, best error:      8823.50098 on epoch:   100, epoch:  100/100, mu: 0.9869856238, learning_rate: 0.0000009629, weight_decay: 0.0000015365

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 924.4058227539s, input_fired_time: 12.4001750946, output_fired_time: 0.0000000000, propagate_forward_time: 891.9716796875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 20.0339965820
best training[          , genome 10089] predictions:   39690/  50000 (79.38%), best:    6942/10000 (69.42%), error:     32825.66016, best error:      8823.50098 on epoch:   100, epoch:  101/100, mu: 0.9871337414, learning_rate: 0.0000008903, weight_decay: 0.0000014492
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 187.7118988037s, input_fired_time: 2.4733767509, output_fired_time: 0.0000000000, propagate_forward_time: 181.0206604004, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2178649902
   testing[          , genome 10089] predictions:    6942/  10000 (69.42%), best:    6942/10000 (69.42%), error:      8823.50098, best error:      8823.50098 on epoch:   100, epoch:  101/100, mu: 0.9871337414, learning_rate: 0.0000008903, weight_decay: 0.0000014492
backpropagation finished successfully!
17:27:12 (9728): called boinc_finish(0)

</stderr_txt>
]]>