Nameexact_genome_1542531604_187_6232_0
Workunit4096872
Created18 Nov 2018, 9:25:59 UTC
Sent18 Nov 2018, 9:26:36 UTC
Report deadline15 Feb 2019, 13:25:10 UTC
Received22 Nov 2018, 11:42:52 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID84153
Run time1 days 15 hours 33 min 1 sec
CPU time1 days 13 hours 9 min 35 sec
Validate stateValid
Credit22,291.42
Device peak FLOPS4.74 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size534.82 MB
Peak swap size538.61 MB
Peak disk usage7.24 MB

Stderr output

<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
635620117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5095291138
validation[          , genome  6232] predictions:    6045/  10000 (60.45%), best:    6076/10000 (60.76%), error:     11545.49707, best error:     11198.87793 on epoch:    20, epoch:   21/100, mu: 0.8359827399, learning_rate: 0.0001365218, weight_decay: 0.0001747231

epoch time: 1372.9080810547s, input_fired_time: 19.8693523407, output_fired_time: 6.9421868324, propagate_forward_time: 519.7705688477, propagate_backward_time: 808.4412231445, weight_update_time: 0.5167666674, other_time: 17.8847656250
epoch time: 105.6344985962s, input_fired_time: 2.0351431370, output_fired_time: 0.0000000000, propagate_forward_time: 101.1093597412, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4899978638
validation[          , genome  6232] predictions:    6022/  10000 (60.22%), best:    6076/10000 (60.76%), error:     11604.64551, best error:     11198.87793 on epoch:    20, epoch:   22/100, mu: 0.8442791700, learning_rate: 0.0001266882, weight_decay: 0.0001650520

epoch time: 1359.9328613281s, input_fired_time: 21.0037326813, output_fired_time: 6.8546037674, propagate_forward_time: 519.7327880859, propagate_backward_time: 796.3414306641, weight_update_time: 0.5260275006, other_time: 16.0002441406
epoch time: 105.5230560303s, input_fired_time: 1.0569736958, output_fired_time: 0.0000000000, propagate_forward_time: 101.9956207275, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4704589844
validation[          , genome  6232] predictions:    6153/  10000 (61.53%), best:    6153/10000 (61.53%), error:     11137.23730, best error:     11137.23730 on epoch:    23, epoch:   23/100, mu: 0.8521286845, learning_rate: 0.0001175629, weight_decay: 0.0001559163

epoch time: 1376.0291748047s, input_fired_time: 21.1734905243, output_fired_time: 6.8846230507, propagate_forward_time: 521.7905883789, propagate_backward_time: 811.3062744141, weight_update_time: 0.5244070292, other_time: 14.8742065430
epoch time: 108.4996643066s, input_fired_time: 1.0733504295, output_fired_time: 0.0000000000, propagate_forward_time: 104.8307495117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5955657959
validation[          , genome  6232] predictions:    6147/  10000 (61.47%), best:    6147/10000 (61.47%), error:     10965.04102, best error:     10965.04102 on epoch:    24, epoch:   24/100, mu: 0.8595553637, learning_rate: 0.0001090949, weight_decay: 0.0001472862

epoch time: 1387.0394287109s, input_fired_time: 20.4002590179, output_fired_time: 6.9197106361, propagate_forward_time: 515.4244995117, propagate_backward_time: 829.3188476562, weight_update_time: 0.5408091545, other_time: 14.9761352539
epoch time: 109.7622222900s, input_fired_time: 1.0872797966, output_fired_time: 0.0000000000, propagate_forward_time: 105.1373138428, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5376281738
validation[          , genome  6232] predictions:    5973/  10000 (59.73%), best:    6147/10000 (61.47%), error:     11414.09961, best error:     10965.04102 on epoch:    24, epoch:   25/100, mu: 0.8665820360, learning_rate: 0.0001012368, weight_decay: 0.0001391338

epoch time: 1388.2264404297s, input_fired_time: 21.2526149750, output_fired_time: 8.9304637909, propagate_forward_time: 518.7320556641, propagate_backward_time: 822.3067626953, weight_update_time: 0.5496615767, other_time: 17.0046386719
epoch time: 111.1111450195s, input_fired_time: 1.1443074942, output_fired_time: 0.0000000000, propagate_forward_time: 106.3047485352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6620864868
validation[          , genome  6232] predictions:    6215/  10000 (62.15%), best:    6147/10000 (61.47%), error:     10988.62305, best error:     10965.04102 on epoch:    24, epoch:   26/100, mu: 0.8732301593, learning_rate: 0.0000939448, weight_decay: 0.0001314326

epoch time: 1383.3332519531s, input_fired_time: 18.2084598541, output_fired_time: 6.8130135536, propagate_forward_time: 527.7687377930, propagate_backward_time: 812.2951660156, weight_update_time: 0.5381389856, other_time: 18.2478637695
epoch time: 107.2868270874s, input_fired_time: 1.0510123968, output_fired_time: 0.0000000000, propagate_forward_time: 103.6526107788, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5832061768
validation[          , genome  6232] predictions:    6210/  10000 (62.10%), best:    6147/10000 (61.47%), error:     11113.26465, best error:     10965.04102 on epoch:    24, epoch:   27/100, mu: 0.8795201778, learning_rate: 0.0000871780, weight_decay: 0.0001241577

epoch time: 1377.4572753906s, input_fired_time: 18.0550670624, output_fired_time: 8.9087848663, propagate_forward_time: 530.4798583984, propagate_backward_time: 804.6239013672, weight_update_time: 0.5458244681, other_time: 15.3896484375
epoch time: 106.4939498901s, input_fired_time: 1.0797773600, output_fired_time: 0.0000000000, propagate_forward_time: 102.8475341797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5666351318
validation[          , genome  6232] predictions:    6124/  10000 (61.24%), best:    6147/10000 (61.47%), error:     11114.04883, best error:     10965.04102 on epoch:    24, epoch:   28/100, mu: 0.8854714036, learning_rate: 0.0000808986, weight_decay: 0.0001172855

epoch time: 1391.8645019531s, input_fired_time: 19.1872806549, output_fired_time: 8.9904909134, propagate_forward_time: 541.1170043945, propagate_backward_time: 807.3555908203, weight_update_time: 0.5609892607, other_time: 15.2141723633
epoch time: 106.2898025513s, input_fired_time: 1.0592645407, output_fired_time: 0.0000000000, propagate_forward_time: 102.7268676758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5036697388
validation[          , genome  6232] predictions:    6183/  10000 (61.83%), best:    6183/10000 (61.83%), error:     10900.62109, best error:     10900.62109 on epoch:    29, epoch:   29/100, mu: 0.8911020160, learning_rate: 0.0000750715, weight_decay: 0.0001107936

epoch time: 1375.5167236328s, input_fired_time: 20.1280155182, output_fired_time: 8.0139923096, propagate_forward_time: 516.5477905273, propagate_backward_time: 815.4002685547, weight_update_time: 0.5522252321, other_time: 15.4265747070
epoch time: 111.1830520630s, input_fired_time: 1.1551038027, output_fired_time: 0.0000000000, propagate_forward_time: 107.2292404175, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7987060547
validation[          , genome  6232] predictions:    6225/  10000 (62.25%), best:    6225/10000 (62.25%), error:     10864.46680, best error:     10864.46680 on epoch:    30, epoch:   30/100, mu: 0.8964293599, learning_rate: 0.0000696641, weight_decay: 0.0001046611

epoch time: 1394.2769775391s, input_fired_time: 21.5138397217, output_fired_time: 9.0990056992, propagate_forward_time: 530.9455566406, propagate_backward_time: 813.8132324219, weight_update_time: 0.5777586699, other_time: 18.9053955078
epoch time: 104.4197311401s, input_fired_time: 1.1204516888, output_fired_time: 0.0000000000, propagate_forward_time: 100.6336593628, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6656188965
validation[          , genome  6232] predictions:    6409/  10000 (64.09%), best:    6409/10000 (64.09%), error:     10460.54883, best error:     10460.54883 on epoch:    31, epoch:   31/100, mu: 0.9014697075, learning_rate: 0.0000646463, weight_decay: 0.0000988681

epoch time: 1391.0198974609s, input_fired_time: 20.4282932281, output_fired_time: 8.1819877625, propagate_forward_time: 513.0347900391, propagate_backward_time: 833.9602050781, weight_update_time: 0.5745506883, other_time: 15.4145507812
epoch time: 111.4784469604s, input_fired_time: 1.1475409269, output_fired_time: 0.0000000000, propagate_forward_time: 107.3961257935, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9347839355
validation[          , genome  6232] predictions:    6281/  10000 (62.81%), best:    6409/10000 (64.09%), error:     10669.50293, best error:     10460.54883 on epoch:    31, epoch:   32/100, mu: 0.9062385559, learning_rate: 0.0000599898, weight_decay: 0.0000933956

epoch time: 1413.2635498047s, input_fired_time: 18.7072963715, output_fired_time: 10.3739614487, propagate_forward_time: 528.8131103516, propagate_backward_time: 836.2937622070, weight_update_time: 0.5805748701, other_time: 19.0755004883
epoch time: 111.4860992432s, input_fired_time: 1.2007969618, output_fired_time: 0.0000000000, propagate_forward_time: 107.5102767944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7750244141
validation[          , genome  6232] predictions:    6310/  10000 (63.10%), best:    6409/10000 (64.09%), error:     10612.39746, best error:     10460.54883 on epoch:    31, epoch:   33/100, mu: 0.9107505083, learning_rate: 0.0000556688, weight_decay: 0.0000882261

epoch time: 1411.9387207031s, input_fired_time: 19.8635711670, output_fired_time: 7.3564028740, propagate_forward_time: 535.0871582031, propagate_backward_time: 831.4244384766, weight_update_time: 0.5712026358, other_time: 18.2071533203
epoch time: 111.7340927124s, input_fired_time: 1.2014528513, output_fired_time: 0.0000000000, propagate_forward_time: 107.6447372437, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8879013062
validation[          , genome  6232] predictions:    6265/  10000 (62.65%), best:    6409/10000 (64.09%), error:     10626.96191, best error:     10460.54883 on epoch:    31, epoch:   34/100, mu: 0.9150194526, learning_rate: 0.0000516590, weight_decay: 0.0000833427

epoch time: 1417.3386230469s, input_fired_time: 21.8634834290, output_fired_time: 8.4552288055, propagate_forward_time: 543.8718872070, propagate_backward_time: 825.6506347656, weight_update_time: 0.6021819711, other_time: 17.4973754883
epoch time: 111.5171585083s, input_fired_time: 1.1894496679, output_fired_time: 0.0000000000, propagate_forward_time: 107.4192504883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9084548950
validation[          , genome  6232] predictions:    6198/  10000 (61.98%), best:    6409/10000 (64.09%), error:     10960.92969, best error:     10460.54883 on epoch:    31, epoch:   35/100, mu: 0.9190584421, learning_rate: 0.0000479380, weight_decay: 0.0000787296

epoch time: 1411.4652099609s, input_fired_time: 20.7867584229, output_fired_time: 7.4042658806, propagate_forward_time: 532.1315917969, propagate_backward_time: 831.5703125000, weight_update_time: 0.5989273787, other_time: 19.5722656250
epoch time: 111.9559478760s, input_fired_time: 1.2371492386, output_fired_time: 0.0000000000, propagate_forward_time: 105.9991073608, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7196884155
validation[          , genome  6232] predictions:    6359/  10000 (63.59%), best:    6359/10000 (63.59%), error:     10391.22461, best error:     10391.22461 on epoch:    36, epoch:   36/100, mu: 0.9228798151, learning_rate: 0.0000444850, weight_decay: 0.0000743719

epoch time: 1419.2573242188s, input_fired_time: 18.8846874237, output_fired_time: 7.4068064690, propagate_forward_time: 533.4266967773, propagate_backward_time: 841.1681518555, weight_update_time: 0.6037498116, other_time: 18.3709716797
epoch time: 108.2547454834s, input_fired_time: 1.1752719879, output_fired_time: 0.0000000000, propagate_forward_time: 104.3674087524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7120666504
validation[          , genome  6232] predictions:    6331/  10000 (63.31%), best:    6359/10000 (63.59%), error:     10605.69238, best error:     10391.22461 on epoch:    36, epoch:   37/100, mu: 0.9264953732, learning_rate: 0.0000412808, weight_decay: 0.0000702554

epoch time: 1417.5965576172s, input_fired_time: 19.8137874603, output_fired_time: 8.4978189468, propagate_forward_time: 526.2117309570, propagate_backward_time: 845.6411743164, weight_update_time: 0.6404579878, other_time: 17.4320068359
epoch time: 112.7466506958s, input_fired_time: 1.2910028696, output_fired_time: 0.0000000000, propagate_forward_time: 108.6162338257, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8394165039
validation[          , genome  6232] predictions:    6444/  10000 (64.44%), best:    6444/10000 (64.44%), error:     10362.32129, best error:     10362.32129 on epoch:    38, epoch:   38/100, mu: 0.9299161434, learning_rate: 0.0000383074, weight_decay: 0.0000663667

epoch time: 1418.5917968750s, input_fired_time: 19.8813781738, output_fired_time: 9.4844722748, propagate_forward_time: 533.3820800781, propagate_backward_time: 839.3527221680, weight_update_time: 0.5890347362, other_time: 16.4911499023
epoch time: 112.6317596436s, input_fired_time: 1.2256791592, output_fired_time: 0.0000000000, propagate_forward_time: 107.4462661743, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9598159790
validation[          , genome  6232] predictions:    6387/  10000 (63.87%), best:    6444/10000 (64.44%), error:     10450.36523, best error:     10362.32129 on epoch:    38, epoch:   39/100, mu: 0.9331526756, learning_rate: 0.0000355481, weight_decay: 0.0000626932

epoch time: 1413.8906250000s, input_fired_time: 18.8313941956, output_fired_time: 7.2592101097, propagate_forward_time: 538.4515380859, propagate_backward_time: 831.9391479492, weight_update_time: 0.5956285596, other_time: 17.4093627930
epoch time: 111.9443435669s, input_fired_time: 1.2015136480, output_fired_time: 0.0000000000, propagate_forward_time: 107.0416641235, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7011642456
validation[          , genome  6232] predictions:    5942/  10000 (59.42%), best:    6444/10000 (64.44%), error:     11678.77148, best error:     10362.32129 on epoch:    38, epoch:   40/100, mu: 0.9362148643, learning_rate: 0.0000329876, weight_decay: 0.0000592231

epoch time: 1409.2416992188s, input_fired_time: 20.9270610809, output_fired_time: 9.4815464020, propagate_forward_time: 517.3515625000, propagate_backward_time: 843.4628906250, weight_update_time: 0.5883559585, other_time: 18.0186767578
epoch time: 111.4833984375s, input_fired_time: 1.1977202892, output_fired_time: 0.0000000000, propagate_forward_time: 107.5330123901, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7526626587
validation[          , genome  6232] predictions:    6383/  10000 (63.83%), best:    6444/10000 (64.44%), error:     10440.74121, best error:     10362.32129 on epoch:    38, epoch:   41/100, mu: 0.9391121268, learning_rate: 0.0000306115, weight_decay: 0.0000559451

epoch time: 1415.7427978516s, input_fired_time: 21.9580497742, output_fired_time: 8.4566564560, propagate_forward_time: 524.4118041992, propagate_backward_time: 843.6334228516, weight_update_time: 0.5999820828, other_time: 17.2828979492
epoch time: 111.7305221558s, input_fired_time: 1.2608033419, output_fired_time: 0.0000000000, propagate_forward_time: 107.6512832642, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8184356689
validation[          , genome  6232] predictions:    6328/  10000 (63.28%), best:    6444/10000 (64.44%), error:     10488.13086, best error:     10362.32129 on epoch:    38, epoch:   42/100, mu: 0.9418532848, learning_rate: 0.0000284066, weight_decay: 0.0000528484

epoch time: 1396.0076904297s, input_fired_time: 22.4033641815, output_fired_time: 9.1944656372, propagate_forward_time: 516.5617675781, propagate_backward_time: 832.0180053711, weight_update_time: 0.5624720454, other_time: 15.8301391602
epoch time: 106.2075881958s, input_fired_time: 1.1517411470, output_fired_time: 0.0000000000, propagate_forward_time: 102.3780212402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6778259277
validation[          , genome  6232] predictions:    6431/  10000 (64.31%), best:    6431/10000 (64.31%), error:     10303.29297, best error:     10303.29297 on epoch:    43, epoch:   43/100, mu: 0.9444468021, learning_rate: 0.0000263605, weight_decay: 0.0000499232

epoch time: 1376.1157226562s, input_fired_time: 21.3619270325, output_fired_time: 9.9375047684, propagate_forward_time: 523.8966064453, propagate_backward_time: 804.9451293945, weight_update_time: 0.5874065757, other_time: 15.9745483398
epoch time: 105.7250900269s, input_fired_time: 1.1376361847, output_fired_time: 0.0000000000, propagate_forward_time: 101.9328308105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6546249390
validation[          , genome  6232] predictions:    6443/  10000 (64.43%), best:    6431/10000 (64.31%), error:     10363.27637, best error:     10303.29297 on epoch:    43, epoch:   44/100, mu: 0.9469006062, learning_rate: 0.0000244617, weight_decay: 0.0000471600

epoch time: 1389.5373535156s, input_fired_time: 19.2707881927, output_fired_time: 9.1561031342, propagate_forward_time: 514.4881591797, propagate_backward_time: 830.8629760742, weight_update_time: 0.5487140417, other_time: 15.7593383789
epoch time: 109.9087066650s, input_fired_time: 1.2026475668, output_fired_time: 0.0000000000, propagate_forward_time: 105.8904266357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8156356812
validation[          , genome  6232] predictions:    6398/  10000 (63.98%), best:    6431/10000 (64.31%), error:     10311.77637, best error:     10303.29297 on epoch:    43, epoch:   45/100, mu: 0.9492222071, learning_rate: 0.0000226998, weight_decay: 0.0000445496

epoch time: 1415.3537597656s, input_fired_time: 18.9449367523, output_fired_time: 7.3600807190, propagate_forward_time: 534.6423339844, propagate_backward_time: 832.7315063477, weight_update_time: 0.5913394094, other_time: 21.6748657227
epoch time: 111.3267440796s, input_fired_time: 1.2191063166, output_fired_time: 0.0000000000, propagate_forward_time: 107.2935180664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8141174316
validation[          , genome  6232] predictions:    6449/  10000 (64.49%), best:    6449/10000 (64.49%), error:     10252.98633, best error:     10252.98633 on epoch:    46, epoch:   46/100, mu: 0.9514187574, learning_rate: 0.0000210647, weight_decay: 0.0000420838

epoch time: 1407.6109619141s, input_fired_time: 22.7007045746, output_fired_time: 8.2660169601, propagate_forward_time: 540.7255249023, propagate_backward_time: 818.2279052734, weight_update_time: 0.6105023026, other_time: 17.6908569336
epoch time: 109.8830642700s, input_fired_time: 1.1776605844, output_fired_time: 0.0000000000, propagate_forward_time: 105.9748764038, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7305297852
validation[          , genome  6232] predictions:    6469/  10000 (64.69%), best:    6449/10000 (64.49%), error:     10266.36719, best error:     10252.98633 on epoch:    46, epoch:   47/100, mu: 0.9534969926, learning_rate: 0.0000195474, weight_decay: 0.0000397544

epoch time: 1397.7360839844s, input_fired_time: 18.5883827209, output_fired_time: 9.2572851181, propagate_forward_time: 526.7163696289, propagate_backward_time: 826.9865112305, weight_update_time: 0.5655840039, other_time: 16.1875000000
epoch time: 110.9379806519s, input_fired_time: 1.2453986406, output_fired_time: 0.0000000000, propagate_forward_time: 106.8891067505, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8034744263
validation[          , genome  6232] predictions:    6392/  10000 (63.92%), best:    6449/10000 (64.49%), error:     10380.44141, best error:     10252.98633 on epoch:    46, epoch:   48/100, mu: 0.9554632902, learning_rate: 0.0000181394, weight_decay: 0.0000375540

epoch time: 1415.1191406250s, input_fired_time: 20.0350990295, output_fired_time: 9.5334835052, propagate_forward_time: 534.9163208008, propagate_backward_time: 833.2227783203, weight_update_time: 0.5868822336, other_time: 17.4114379883
epoch time: 110.3699645996s, input_fired_time: 1.2106556892, output_fired_time: 0.0000000000, propagate_forward_time: 104.3705520630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7887573242
validation[          , genome  6232] predictions:    6465/  10000 (64.65%), best:    6449/10000 (64.49%), error:     10309.83691, best error:     10252.98633 on epoch:    46, epoch:   49/100, mu: 0.9573236704, learning_rate: 0.0000168329, weight_decay: 0.0000354753

epoch time: 1420.6149902344s, input_fired_time: 25.1128349304, output_fired_time: 8.4673023224, propagate_forward_time: 526.5933227539, propagate_backward_time: 842.9590454102, weight_update_time: 0.5926398039, other_time: 17.4825439453
epoch time: 112.6393127441s, input_fired_time: 1.2342066765, output_fired_time: 0.0000000000, propagate_forward_time: 108.5077056885, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8973999023
validation[          , genome  6232] predictions:    6493/  10000 (64.93%), best:    6493/10000 (64.93%), error:     10109.19727, best error:     10109.19727 on epoch:    50, epoch:   50/100, mu: 0.9590838552, learning_rate: 0.0000156204, weight_decay: 0.0000335117

epoch time: 1403.7164306641s, input_fired_time: 23.0120239258, output_fired_time: 9.2745504379, propagate_forward_time: 528.8390502930, propagate_backward_time: 825.2055053711, weight_update_time: 0.6360285282, other_time: 17.3852539062
epoch time: 106.7427749634s, input_fired_time: 1.1308879852, output_fired_time: 0.0000000000, propagate_forward_time: 102.9582290649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6536560059
validation[          , genome  6232] predictions:    6438/  10000 (64.38%), best:    6493/10000 (64.93%), error:     10228.00000, best error:     10109.19727 on epoch:    50, epoch:   51/100, mu: 0.9607492089, learning_rate: 0.0000144953, weight_decay: 0.0000316568

epoch time: 1379.7557373047s, input_fired_time: 22.5251922607, output_fired_time: 7.0199685097, propagate_forward_time: 534.1380004883, propagate_backward_time: 799.2023315430, weight_update_time: 0.5713695288, other_time: 16.8702392578
epoch time: 108.8428955078s, input_fired_time: 1.1683311462, output_fired_time: 0.0000000000, propagate_forward_time: 105.0057907104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6687698364
validation[          , genome  6232] predictions:    6456/  10000 (64.56%), best:    6493/10000 (64.93%), error:     10286.42871, best error:     10109.19727 on epoch:    50, epoch:   52/100, mu: 0.9623248577, learning_rate: 0.0000134512, weight_decay: 0.0000299046

epoch time: 1373.6812744141s, input_fired_time: 20.3873748779, output_fired_time: 7.0170426369, propagate_forward_time: 512.6145019531, propagate_backward_time: 817.0869750977, weight_update_time: 0.6163458228, other_time: 16.5753784180
epoch time: 109.4005279541s, input_fired_time: 1.1703581810, output_fired_time: 0.0000000000, propagate_forward_time: 105.4535827637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7765884399
validation[          , genome  6232] predictions:    6402/  10000 (64.02%), best:    6493/10000 (64.93%), error:     10387.31152, best error:     10109.19727 on epoch:    50, epoch:   53/100, mu: 0.9638156295, learning_rate: 0.0000124823, weight_decay: 0.0000282494

epoch time: 1379.7037353516s, input_fired_time: 19.4479694366, output_fired_time: 8.0930738449, propagate_forward_time: 529.4732666016, propagate_backward_time: 805.7734375000, weight_update_time: 0.5560416579, other_time: 16.9160156250
epoch time: 108.6998825073s, input_fired_time: 1.1885555983, output_fired_time: 0.0000000000, propagate_forward_time: 104.8108673096, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7004623413
validation[          , genome  6232] predictions:    6507/  10000 (65.07%), best:    6493/10000 (64.93%), error:     10197.76855, best error:     10109.19727 on epoch:    50, epoch:   54/100, mu: 0.9652261138, learning_rate: 0.0000115832, weight_decay: 0.0000266857

epoch time: 1379.8565673828s, input_fired_time: 20.4691791534, output_fired_time: 7.1358742714, propagate_forward_time: 519.7723999023, propagate_backward_time: 815.6766967773, weight_update_time: 0.5897066593, other_time: 16.8023681641
epoch time: 111.4980392456s, input_fired_time: 1.2138073444, output_fired_time: 0.0000000000, propagate_forward_time: 106.4917678833, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7924652100
validation[          , genome  6232] predictions:    6470/  10000 (64.70%), best:    6493/10000 (64.93%), error:     10227.48047, best error:     10109.19727 on epoch:    50, epoch:   55/100, mu: 0.9665606022, learning_rate: 0.0000107489, weight_decay: 0.0000252087

epoch time: 1380.3966064453s, input_fired_time: 26.3710136414, output_fired_time: 6.9109234810, propagate_forward_time: 534.0869140625, propagate_backward_time: 797.0413818359, weight_update_time: 0.5864866972, other_time: 15.9864501953
epoch time: 106.5084381104s, input_fired_time: 1.1441414356, output_fired_time: 0.0000000000, propagate_forward_time: 101.6534194946, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7108764648
validation[          , genome  6232] predictions:    6508/  10000 (65.08%), best:    6508/10000 (65.08%), error:     10092.43652, best error:     10092.43652 on epoch:    56, epoch:   56/100, mu: 0.9678232074, learning_rate: 0.0000099746, weight_decay: 0.0000238133

epoch time: 1391.8865966797s, input_fired_time: 20.9429206848, output_fired_time: 8.0607395172, propagate_forward_time: 532.9025268555, propagate_backward_time: 812.9334106445, weight_update_time: 0.5628272295, other_time: 17.0469970703
epoch time: 107.3222198486s, input_fired_time: 1.1542954445, output_fired_time: 0.0000000000, propagate_forward_time: 103.5549163818, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6130065918
validation[          , genome  6232] predictions:    6484/  10000 (64.84%), best:    6508/10000 (65.08%), error:     10301.10547, best error:     10092.43652 on epoch:    56, epoch:   57/100, mu: 0.9690178037, learning_rate: 0.0000092562, weight_decay: 0.0000224953

epoch time: 1379.9317626953s, input_fired_time: 19.5191764832, output_fired_time: 6.9456210136, propagate_forward_time: 526.9379272461, propagate_backward_time: 810.6511840820, weight_update_time: 0.5575801730, other_time: 15.8778076172
epoch time: 107.8576812744s, input_fired_time: 1.1932101250, output_fired_time: 0.0000000000, propagate_forward_time: 103.9653244019, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6991500854
validation[          , genome  6232] predictions:    6404/  10000 (64.04%), best:    6508/10000 (65.08%), error:     10278.15039, best error:     10092.43652 on epoch:    56, epoch:   58/100, mu: 0.9701480269, learning_rate: 0.0000085894, weight_decay: 0.0000212501

epoch time: 1386.2003173828s, input_fired_time: 21.6207695007, output_fired_time: 7.0555086136, propagate_forward_time: 529.7542114258, propagate_backward_time: 810.7993164062, weight_update_time: 0.5862171650, other_time: 16.9705200195
epoch time: 112.5875854492s, input_fired_time: 1.2373900414, output_fired_time: 0.0000000000, propagate_forward_time: 108.6766967773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6735000610
validation[          , genome  6232] predictions:    6581/  10000 (65.81%), best:    6581/10000 (65.81%), error:     10020.91309, best error:     10020.91309 on epoch:    59, epoch:   59/100, mu: 0.9712173939, learning_rate: 0.0000079707, weight_decay: 0.0000200739

epoch time: 1386.0334472656s, input_fired_time: 18.5435810089, output_fired_time: 10.1045980453, propagate_forward_time: 514.9402465820, propagate_backward_time: 826.4722900391, weight_update_time: 0.5736486316, other_time: 15.9727172852
epoch time: 113.6381912231s, input_fired_time: 2.2408876419, output_fired_time: 0.0000000000, propagate_forward_time: 108.6983032227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6989974976
validation[          , genome  6232] predictions:    6496/  10000 (64.96%), best:    6581/10000 (65.81%), error:     10137.01562, best error:     10020.91309 on epoch:    59, epoch:   60/100, mu: 0.9722291827, learning_rate: 0.0000073966, weight_decay: 0.0000189628

epoch time: 1390.4642333984s, input_fired_time: 19.6007022858, output_fired_time: 7.0940022469, propagate_forward_time: 520.6491699219, propagate_backward_time: 825.5077514648, weight_update_time: 0.5704889297, other_time: 17.6126098633
epoch time: 111.2170257568s, input_fired_time: 1.2501912117, output_fired_time: 0.0000000000, propagate_forward_time: 107.2141494751, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7526855469
validation[          , genome  6232] predictions:    6514/  10000 (65.14%), best:    6581/10000 (65.81%), error:     10137.71191, best error:     10020.91309 on epoch:    59, epoch:   61/100, mu: 0.9731864333, learning_rate: 0.0000068638, weight_decay: 0.0000179132

epoch time: 1398.9439697266s, input_fired_time: 19.7130088806, output_fired_time: 8.2574424744, propagate_forward_time: 524.0493774414, propagate_backward_time: 828.2402954102, weight_update_time: 0.5939257741, other_time: 18.6838378906
epoch time: 111.2628784180s, input_fired_time: 1.2058646679, output_fired_time: 0.0000000000, propagate_forward_time: 107.3100585938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7469558716
validation[          , genome  6232] predictions:    6566/  10000 (65.66%), best:    6566/10000 (65.66%), error:     10018.22461, best error:     10018.22461 on epoch:    62, epoch:   62/100, mu: 0.9740921259, learning_rate: 0.0000063694, weight_decay: 0.0000169217

epoch time: 1404.5968017578s, input_fired_time: 18.7253437042, output_fired_time: 7.4558534622, propagate_forward_time: 537.0426025391, propagate_backward_time: 821.7751464844, weight_update_time: 0.5874313712, other_time: 19.5979003906
epoch time: 110.1496734619s, input_fired_time: 1.2346202135, output_fired_time: 0.0000000000, propagate_forward_time: 105.1209259033, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7941284180
validation[          , genome  6232] predictions:    6516/  10000 (65.16%), best:    6566/10000 (65.66%), error:     10049.73145, best error:     10018.22461 on epoch:    62, epoch:   63/100, mu: 0.9749490619, learning_rate: 0.0000059107, weight_decay: 0.0000159851

epoch time: 1408.5628662109s, input_fired_time: 21.8536567688, output_fired_time: 7.4726686478, propagate_forward_time: 535.9007568359, propagate_backward_time: 824.5249633789, weight_update_time: 0.6219079494, other_time: 18.8108520508
epoch time: 105.5293731689s, input_fired_time: 1.2188861370, output_fired_time: 0.0000000000, propagate_forward_time: 100.6548843384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6556015015
validation[          , genome  6232] predictions:    6532/  10000 (65.32%), best:    6566/10000 (65.66%), error:     10036.95312, best error:     10018.22461 on epoch:    62, epoch:   64/100, mu: 0.9757598042, learning_rate: 0.0000054849, weight_decay: 0.0000151003

epoch time: 1415.5090332031s, input_fired_time: 20.1615467072, output_fired_time: 7.6062631607, propagate_forward_time: 531.4072265625, propagate_backward_time: 839.4749145508, weight_update_time: 0.6186204553, other_time: 16.8590698242
epoch time: 110.9805297852s, input_fired_time: 1.2799319029, output_fired_time: 0.0000000000, propagate_forward_time: 106.7188110352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9817886353
validation[          , genome  6232] predictions:    6585/  10000 (65.85%), best:    6566/10000 (65.66%), error:     10047.71191, best error:     10018.22461 on epoch:    62, epoch:   65/100, mu: 0.9765268564, learning_rate: 0.0000050898, weight_decay: 0.0000142645

epoch time: 1411.5194091797s, input_fired_time: 20.0208873749, output_fired_time: 9.6857767105, propagate_forward_time: 516.9421386719, propagate_backward_time: 846.8409423828, weight_update_time: 0.6659931540, other_time: 18.0296630859
epoch time: 109.8380126953s, input_fired_time: 2.2803668976, output_fired_time: 0.0000000000, propagate_forward_time: 104.5234375000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0342102051
validation[          , genome  6232] predictions:    6524/  10000 (65.24%), best:    6566/10000 (65.66%), error:     10055.62598, best error:     10018.22461 on epoch:    62, epoch:   66/100, mu: 0.9772526026, learning_rate: 0.0000047232, weight_decay: 0.0000134749

epoch time: 1409.0214843750s, input_fired_time: 23.1975555420, output_fired_time: 7.6910724640, propagate_forward_time: 523.0540771484, propagate_backward_time: 835.7456054688, weight_update_time: 0.6807940602, other_time: 19.3332519531
epoch time: 112.0823593140s, input_fired_time: 2.3084189892, output_fired_time: 0.0000000000, propagate_forward_time: 105.6590423584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1148986816
validation[          , genome  6232] predictions:    6569/  10000 (65.69%), best:    6569/10000 (65.69%), error:     10011.78906, best error:     10011.78906 on epoch:    67, epoch:   67/100, mu: 0.9779392481, learning_rate: 0.0000043830, weight_decay: 0.0000127291

epoch time: 1415.7261962891s, input_fired_time: 20.3538761139, output_fired_time: 9.8263359070, propagate_forward_time: 530.8753051758, propagate_backward_time: 834.3825073242, weight_update_time: 1.6405802965, other_time: 20.2882080078
epoch time: 111.0314407349s, input_fired_time: 1.3052724600, output_fired_time: 0.0000000000, propagate_forward_time: 106.7584991455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9676666260
validation[          , genome  6232] predictions:    6511/  10000 (65.11%), best:    6569/10000 (65.69%), error:     10102.62793, best error:     10011.78906 on epoch:    67, epoch:   68/100, mu: 0.9785889387, learning_rate: 0.0000040673, weight_decay: 0.0000120245

epoch time: 1406.7321777344s, input_fired_time: 22.1743259430, output_fired_time: 7.6883034706, propagate_forward_time: 530.0141601562, propagate_backward_time: 827.8506469727, weight_update_time: 0.6253623366, other_time: 19.0046997070
epoch time: 112.5447540283s, input_fired_time: 1.3094217777, output_fired_time: 0.0000000000, propagate_forward_time: 107.2379074097, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9974212646
validation[          , genome  6232] predictions:    6553/  10000 (65.53%), best:    6553/10000 (65.53%), error:     10001.00586, best error:     10001.00586 on epoch:    69, epoch:   69/100, mu: 0.9792036414, learning_rate: 0.0000037743, weight_decay: 0.0000113589

epoch time: 1411.3864746094s, input_fired_time: 21.1034984589, output_fired_time: 8.6206026077, propagate_forward_time: 535.2391357422, propagate_backward_time: 828.1434326172, weight_update_time: 0.6250286102, other_time: 18.2797851562
epoch time: 111.2961273193s, input_fired_time: 1.2817254066, output_fired_time: 0.0000000000, propagate_forward_time: 107.0950469971, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9193572998
validation[          , genome  6232] predictions:    6553/  10000 (65.53%), best:    6553/10000 (65.53%), error:      9997.69531, best error:      9997.69531 on epoch:    70, epoch:   70/100, mu: 0.9797852039, learning_rate: 0.0000035025, weight_decay: 0.0000107302

epoch time: 1420.5653076172s, input_fired_time: 21.3736133575, output_fired_time: 7.6948337555, propagate_forward_time: 534.5009765625, propagate_backward_time: 837.4213867188, weight_update_time: 0.6580027938, other_time: 19.5744628906
epoch time: 108.2629776001s, input_fired_time: 1.2659728527, output_fired_time: 0.0000000000, propagate_forward_time: 103.1606674194, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8363342285
validation[          , genome  6232] predictions:    6567/  10000 (65.67%), best:    6567/10000 (65.67%), error:      9953.25098, best error:      9953.25098 on epoch:    71, epoch:   71/100, mu: 0.9803354144, learning_rate: 0.0000032502, weight_decay: 0.0000101363

epoch time: 1415.4689941406s, input_fired_time: 21.2241897583, output_fired_time: 7.7486181259, propagate_forward_time: 524.6538696289, propagate_backward_time: 842.6983642578, weight_update_time: 0.6310898662, other_time: 19.1438598633
epoch time: 109.3403015137s, input_fired_time: 1.3137058020, output_fired_time: 0.0000000000, propagate_forward_time: 105.1023330688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9242630005
validation[          , genome  6232] predictions:    6560/  10000 (65.60%), best:    6567/10000 (65.67%), error:      9979.22363, best error:      9953.25098 on epoch:    71, epoch:   72/100, mu: 0.9808560014, learning_rate: 0.0000030161, weight_decay: 0.0000095752

epoch time: 1415.0249023438s, input_fired_time: 23.5081901550, output_fired_time: 8.6804199219, propagate_forward_time: 527.9487915039, propagate_backward_time: 836.4983520508, weight_update_time: 0.6483986974, other_time: 18.3891601562
epoch time: 111.1500854492s, input_fired_time: 1.3034714460, output_fired_time: 0.0000000000, propagate_forward_time: 106.8969573975, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9496536255
validation[          , genome  6232] predictions:    6527/  10000 (65.27%), best:    6567/10000 (65.67%), error:      9982.32031, best error:      9953.25098 on epoch:    71, epoch:   73/100, mu: 0.9813485742, learning_rate: 0.0000027988, weight_decay: 0.0000090452

epoch time: 1399.7468261719s, input_fired_time: 19.1867065430, output_fired_time: 7.7301397324, propagate_forward_time: 522.3729248047, propagate_backward_time: 831.4523925781, weight_update_time: 0.6201158762, other_time: 19.0046386719
epoch time: 109.5975494385s, input_fired_time: 1.3005344868, output_fired_time: 0.0000000000, propagate_forward_time: 104.4240036011, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8730087280
validation[          , genome  6232] predictions:    6560/  10000 (65.60%), best:    6567/10000 (65.67%), error:     10040.42383, best error:      9953.25098 on epoch:    71, epoch:   74/100, mu: 0.9818146229, learning_rate: 0.0000025972, weight_decay: 0.0000085446

epoch time: 1406.1728515625s, input_fired_time: 19.9647159576, output_fired_time: 8.7597494125, propagate_forward_time: 524.6121215820, propagate_backward_time: 834.4842529297, weight_update_time: 0.6278541684, other_time: 18.3519897461
epoch time: 107.8774795532s, input_fired_time: 1.2847229242, output_fired_time: 0.0000000000, propagate_forward_time: 103.7084045410, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8843536377
validation[          , genome  6232] predictions:    6581/  10000 (65.81%), best:    6567/10000 (65.67%), error:      9959.37305, best error:      9953.25098 on epoch:    71, epoch:   75/100, mu: 0.9822555184, learning_rate: 0.0000024102, weight_decay: 0.0000080716

epoch time: 1421.0371093750s, input_fired_time: 21.2650661469, output_fired_time: 7.7213544846, propagate_forward_time: 536.0292968750, propagate_backward_time: 837.9026489258, weight_update_time: 0.6382545829, other_time: 18.1188354492
epoch time: 112.5734100342s, input_fired_time: 1.3294507265, output_fired_time: 0.0000000000, propagate_forward_time: 107.1995239258, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0444335938
validation[          , genome  6232] predictions:    6540/  10000 (65.40%), best:    6567/10000 (65.67%), error:     10005.47559, best error:      9953.25098 on epoch:    71, epoch:   76/100, mu: 0.9826726913, learning_rate: 0.0000022366, weight_decay: 0.0000076249

epoch time: 1407.3007812500s, input_fired_time: 21.1307811737, output_fired_time: 8.6318311691, propagate_forward_time: 524.1636962891, propagate_backward_time: 835.1228027344, weight_update_time: 0.6329241395, other_time: 18.2517089844
epoch time: 111.9878540039s, input_fired_time: 1.3329654932, output_fired_time: 0.0000000000, propagate_forward_time: 107.6107406616, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0441513062
validation[          , genome  6232] predictions:    6605/  10000 (66.05%), best:    6567/10000 (65.67%), error:      9957.96191, best error:      9953.25098 on epoch:    71, epoch:   77/100, mu: 0.9830673933, learning_rate: 0.0000020755, weight_decay: 0.0000072028

epoch time: 1418.7919921875s, input_fired_time: 21.2783164978, output_fired_time: 8.7465744019, propagate_forward_time: 530.4442138672, propagate_backward_time: 839.9421386719, weight_update_time: 0.6509765387, other_time: 18.3807373047
epoch time: 107.7918548584s, input_fired_time: 1.2833703756, output_fired_time: 0.0000000000, propagate_forward_time: 103.6659774780, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8425064087
validation[          , genome  6232] predictions:    6581/  10000 (65.81%), best:    6581/10000 (65.81%), error:      9945.27051, best error:      9945.27051 on epoch:    78, epoch:   78/100, mu: 0.9834408164, learning_rate: 0.0000019260, weight_decay: 0.0000068041

epoch time: 1423.3001708984s, input_fired_time: 21.3574409485, output_fired_time: 8.8636522293, propagate_forward_time: 526.1115722656, propagate_backward_time: 848.6978149414, weight_update_time: 0.6313022375, other_time: 18.2697143555
epoch time: 110.4730072021s, input_fired_time: 1.2997338772, output_fired_time: 0.0000000000, propagate_forward_time: 106.2483444214, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9249267578
validation[          , genome  6232] predictions:    6586/  10000 (65.86%), best:    6586/10000 (65.86%), error:      9922.36426, best error:      9922.36426 on epoch:    79, epoch:   79/100, mu: 0.9837941527, learning_rate: 0.0000017872, weight_decay: 0.0000064275

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_187_6232.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_187_6232_0_r835434109_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497312
read mu: 0.984128
read mu_delta: 0.946133
read initial_learning_rate: 0.000656123
read learning_rate: 1.6585e-06
read learning_rate_delta: 0.92797
read initial_weight_decay: 0.000577655
read weight_decay: 6.07175e-06
read weight_decay_delta: 0.944649
read batch_size: 90
read epsilon: 1e-07
read alpha: 0.0824234
read input_dropout_probability: 0.00118524
read hidden_dropout_probability: 0.0994064
read velocity_reset: 1834
read epoch: 80
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 79
read number_validation_images: 10000
read best_validation_predictions: 6586
read best_validation_error: 9922.36
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6232
read normal distribution: '1 0 0'
generator_str: '1426963230'
read generator: 1426963230
read generated_by_map:
1 crossover 1
reading 83 nodes.
reading 360 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1416.71643066406s, input_fired_time: 23.0986080169678, output_fired_time: 9.68113231658936, propagate_forward_time: 534.228881835938, propagate_backward_time: 831.491271972656, weight_update_time: 0.642244517803192, other_time: 18.2164916992188
epoch time: 112.46223449707s, input_fired_time: 1.32774770259857, output_fired_time: 0, propagate_forward_time: 108.167495727539, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.96698760986328
validation[          , genome  6232] predictions:    6592/  10000 (65.92%), best:    6586/10000 (65.86%), error:      9931.23633, best error:      9922.36426 on epoch:    79, epoch:   80/100, mu: 0.9841284156, learning_rate: 0.0000016585, weight_decay: 0.0000060718

epoch time: 1409.1527099609s, input_fired_time: 22.9691886902, output_fired_time: 8.7273912430, propagate_forward_time: 525.9461059570, propagate_backward_time: 832.2404785156, weight_update_time: 0.6388159990, other_time: 19.2694702148
epoch time: 109.2620315552s, input_fired_time: 1.3041266203, output_fired_time: 0.0000000000, propagate_forward_time: 104.9722747803, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9856338501
validation[          , genome  6232] predictions:    6597/  10000 (65.97%), best:    6586/10000 (65.86%), error:      9953.76855, best error:      9922.36426 on epoch:    79, epoch:   81/100, mu: 0.9844446778, learning_rate: 0.0000015390, weight_decay: 0.0000057357

epoch time: 1412.7816162109s, input_fired_time: 20.0984954834, output_fired_time: 7.7142386436, propagate_forward_time: 539.7595825195, propagate_backward_time: 825.8229980469, weight_update_time: 0.6549317837, other_time: 19.3862915039
epoch time: 111.3086395264s, input_fired_time: 1.2832335234, output_fired_time: 0.0000000000, propagate_forward_time: 106.1216506958, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9037551880
validation[          , genome  6232] predictions:    6570/  10000 (65.70%), best:    6586/10000 (65.86%), error:      9988.80566, best error:      9922.36426 on epoch:    79, epoch:   82/100, mu: 0.9847439528, learning_rate: 0.0000014282, weight_decay: 0.0000054182

epoch time: 1416.7436523438s, input_fired_time: 20.6329174042, output_fired_time: 8.0646219254, propagate_forward_time: 541.6447143555, propagate_backward_time: 825.4519653320, weight_update_time: 2.6738500595, other_time: 20.9494628906
epoch time: 110.6787719727s, input_fired_time: 2.4573321342, output_fired_time: 0.0000000000, propagate_forward_time: 105.1359405518, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0855026245
validation[          , genome  6232] predictions:    6570/  10000 (65.70%), best:    6586/10000 (65.86%), error:      9959.20410, best error:      9922.36426 on epoch:    79, epoch:   83/100, mu: 0.9850270748, learning_rate: 0.0000013253, weight_decay: 0.0000051183

epoch time: 1700.7122802734s, input_fired_time: 23.6471195221, output_fired_time: 10.6320190430, propagate_forward_time: 645.7670288086, propagate_backward_time: 999.3645019531, weight_update_time: 0.7613569498, other_time: 21.3015747070
epoch time: 204.1599884033s, input_fired_time: 2.2816669941, output_fired_time: 0.0000000000, propagate_forward_time: 195.8664093018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0119171143
validation[          , genome  6232] predictions:    6592/  10000 (65.92%), best:    6592/10000 (65.92%), error:      9903.29785, best error:      9903.29785 on epoch:    84, epoch:   84/100, mu: 0.9852949381, learning_rate: 0.0000012299, weight_decay: 0.0000048350

epoch time: 1965.4118652344s, input_fired_time: 30.1542224884, output_fired_time: 12.3639154434, propagate_forward_time: 754.7874755859, propagate_backward_time: 1142.3190917969, weight_update_time: 0.8805832267, other_time: 25.7872314453
epoch time: 221.0794830322s, input_fired_time: 2.3735427856, output_fired_time: 0.0000000000, propagate_forward_time: 213.6694793701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0364532471
validation[          , genome  6232] predictions:    6595/  10000 (65.95%), best:    6592/10000 (65.92%), error:      9934.45996, best error:      9903.29785 on epoch:    84, epoch:   85/100, mu: 0.9855483770, learning_rate: 0.0000011413, weight_decay: 0.0000045674

epoch time: 1506.3278808594s, input_fired_time: 25.1321353912, output_fired_time: 9.5247020721, propagate_forward_time: 574.0360717773, propagate_backward_time: 877.4210205078, weight_update_time: 0.7496974468, other_time: 20.2140502930
epoch time: 113.2133712769s, input_fired_time: 1.4464859962, output_fired_time: 0.0000000000, propagate_forward_time: 108.5832138062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1836700439
validation[          , genome  6232] predictions:    6597/  10000 (65.97%), best:    6597/10000 (65.97%), error:      9896.82227, best error:      9896.82227 on epoch:    86, epoch:   86/100, mu: 0.9857881665, learning_rate: 0.0000010591, weight_decay: 0.0000043146

epoch time: 1432.5764160156s, input_fired_time: 20.7376289368, output_fired_time: 10.2347631454, propagate_forward_time: 539.9901733398, propagate_backward_time: 841.9845581055, weight_update_time: 0.7044096589, other_time: 19.6292724609
epoch time: 111.6197433472s, input_fired_time: 1.4160844088, output_fired_time: 0.0000000000, propagate_forward_time: 107.1512603760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0523986816
validation[          , genome  6232] predictions:    6581/  10000 (65.81%), best:    6597/10000 (65.97%), error:      9952.32324, best error:      9896.82227 on epoch:    86, epoch:   87/100, mu: 0.9860150218, learning_rate: 0.0000009828, weight_decay: 0.0000040758

epoch time: 1428.9844970703s, input_fired_time: 21.6685237885, output_fired_time: 8.2745523453, propagate_forward_time: 542.3855590820, propagate_backward_time: 838.2517089844, weight_update_time: 0.6928722262, other_time: 18.4041137695
epoch time: 111.8289184570s, input_fired_time: 1.4362646341, output_fired_time: 0.0000000000, propagate_forward_time: 106.2781295776, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1145248413
validation[          , genome  6232] predictions:    6602/  10000 (66.02%), best:    6597/10000 (65.97%), error:      9923.62695, best error:      9896.82227 on epoch:    86, epoch:   88/100, mu: 0.9862296581, learning_rate: 0.0000009120, weight_decay: 0.0000038502

epoch time: 1451.1539306641s, input_fired_time: 21.2114791870, output_fired_time: 10.7417373657, propagate_forward_time: 539.5371093750, propagate_backward_time: 858.6543579102, weight_update_time: 0.7638706565, other_time: 21.0093383789
epoch time: 115.7730102539s, input_fired_time: 1.5342508554, output_fired_time: 0.0000000000, propagate_forward_time: 109.8948745728, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3438873291
validation[          , genome  6232] predictions:    6602/  10000 (66.02%), best:    6597/10000 (65.97%), error:      9911.46094, best error:      9896.82227 on epoch:    86, epoch:   89/100, mu: 0.9864327312, learning_rate: 0.0000008463, weight_decay: 0.0000036370

epoch time: 1456.8427734375s, input_fired_time: 22.1642208099, output_fired_time: 9.8783960342, propagate_forward_time: 538.9833984375, propagate_backward_time: 866.3459472656, weight_update_time: 0.7897481918, other_time: 19.4708251953
epoch time: 113.1071472168s, input_fired_time: 1.5659949780, output_fired_time: 0.0000000000, propagate_forward_time: 108.1639709473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3771820068
validation[          , genome  6232] predictions:    6583/  10000 (65.83%), best:    6597/10000 (65.97%), error:      9931.24316, best error:      9896.82227 on epoch:    86, epoch:   90/100, mu: 0.9866248965, learning_rate: 0.0000007853, weight_decay: 0.0000034357

epoch time: 1469.4223632812s, input_fired_time: 20.4925460815, output_fired_time: 11.0853576660, propagate_forward_time: 547.4172973633, propagate_backward_time: 867.8610229492, weight_update_time: 1.7629184723, other_time: 22.5661621094
epoch time: 116.2856597900s, input_fired_time: 1.5692079067, output_fired_time: 0.0000000000, propagate_forward_time: 111.3002471924, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4162063599
validation[          , genome  6232] predictions:    6566/  10000 (65.66%), best:    6597/10000 (65.97%), error:      9949.57617, best error:      9896.82227 on epoch:    86, epoch:   91/100, mu: 0.9868066907, learning_rate: 0.0000007288, weight_decay: 0.0000032456

epoch time: 1450.0179443359s, input_fired_time: 23.3553009033, output_fired_time: 8.8208265305, propagate_forward_time: 541.7448730469, propagate_backward_time: 856.0040283203, weight_update_time: 0.7330543995, other_time: 20.0928955078
epoch time: 117.0851593018s, input_fired_time: 2.5290191174, output_fired_time: 0.0000000000, propagate_forward_time: 111.2413558960, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3147811890
validation[          , genome  6232] predictions:    6597/  10000 (65.97%), best:    6597/10000 (65.97%), error:      9922.07031, best error:      9896.82227 on epoch:    86, epoch:   92/100, mu: 0.9869787097, learning_rate: 0.0000006763, weight_decay: 0.0000030659

epoch time: 1451.0655517578s, input_fired_time: 22.1789875031, output_fired_time: 10.6984052658, propagate_forward_time: 536.0503540039, propagate_backward_time: 860.4266357422, weight_update_time: 0.7215538025, other_time: 21.7112426758
epoch time: 115.2939605713s, input_fired_time: 1.5209596157, output_fired_time: 0.0000000000, propagate_forward_time: 110.4778900146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2951126099
validation[          , genome  6232] predictions:    6604/  10000 (66.04%), best:    6597/10000 (65.97%), error:      9906.47266, best error:      9896.82227 on epoch:    86, epoch:   93/100, mu: 0.9871414304, learning_rate: 0.0000006276, weight_decay: 0.0000028962

epoch time: 1458.8231201172s, input_fired_time: 20.5893630981, output_fired_time: 10.7405700684, propagate_forward_time: 542.7546386719, propagate_backward_time: 865.8565673828, weight_update_time: 0.7379981279, other_time: 18.8819580078
epoch time: 112.1148071289s, input_fired_time: 1.4487209320, output_fired_time: 0.0000000000, propagate_forward_time: 107.3470001221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3190841675
validation[          , genome  6232] predictions:    6605/  10000 (66.05%), best:    6597/10000 (65.97%), error:      9917.53809, best error:      9896.82227 on epoch:    86, epoch:   94/100, mu: 0.9872953892, learning_rate: 0.0000005824, weight_decay: 0.0000027359

epoch time: 1469.1301269531s, input_fired_time: 24.4429702759, output_fired_time: 9.9118862152, propagate_forward_time: 547.9830322266, propagate_backward_time: 867.4454345703, weight_update_time: 0.7655469775, other_time: 19.3468017578
epoch time: 118.0590515137s, input_fired_time: 2.5760962963, output_fired_time: 0.0000000000, propagate_forward_time: 112.2106246948, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2723312378
validation[          , genome  6232] predictions:    6610/  10000 (66.10%), best:    6597/10000 (65.97%), error:      9915.48145, best error:      9896.82227 on epoch:    86, epoch:   95/100, mu: 0.9874410629, learning_rate: 0.0000005404, weight_decay: 0.0000025845

epoch time: 1463.0661621094s, input_fired_time: 21.3975944519, output_fired_time: 10.7481956482, propagate_forward_time: 547.4255371094, propagate_backward_time: 863.0865478516, weight_update_time: 0.7496184707, other_time: 20.4083251953
epoch time: 113.7244796753s, input_fired_time: 1.5306720734, output_fired_time: 0.0000000000, propagate_forward_time: 108.9110183716, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2827911377
validation[          , genome  6232] predictions:    6575/  10000 (65.75%), best:    6597/10000 (65.97%), error:      9905.70410, best error:      9896.82227 on epoch:    86, epoch:   96/100, mu: 0.9875789285, learning_rate: 0.0000005015, weight_decay: 0.0000024414

epoch time: 1460.5541992188s, input_fired_time: 26.3910026550, output_fired_time: 9.6646080017, propagate_forward_time: 542.0839233398, propagate_backward_time: 863.3432006836, weight_update_time: 0.7477353215, other_time: 19.0715332031
epoch time: 115.8765640259s, input_fired_time: 2.5704317093, output_fired_time: 0.0000000000, propagate_forward_time: 108.9184570312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3876724243
validation[          , genome  6232] predictions:    6601/  10000 (66.01%), best:    6601/10000 (66.01%), error:      9894.24316, best error:      9894.24316 on epoch:    97, epoch:   97/100, mu: 0.9877093434, learning_rate: 0.0000004654, weight_decay: 0.0000023063

epoch time: 1456.7802734375s, input_fired_time: 23.4601249695, output_fired_time: 10.6344413757, propagate_forward_time: 537.9074096680, propagate_backward_time: 862.8385620117, weight_update_time: 0.7374365926, other_time: 21.9398193359
epoch time: 114.6104583740s, input_fired_time: 1.4805492163, output_fired_time: 0.0000000000, propagate_forward_time: 108.8830032349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2469024658
validation[          , genome  6232] predictions:    6609/  10000 (66.09%), best:    6601/10000 (66.01%), error:      9905.83691, best error:      9894.24316 on epoch:    97, epoch:   98/100, mu: 0.9878327250, learning_rate: 0.0000004318, weight_decay: 0.0000021786

epoch time: 1464.2955322266s, input_fired_time: 22.1382198334, output_fired_time: 8.6445951462, propagate_forward_time: 547.7968750000, propagate_backward_time: 865.5048828125, weight_update_time: 0.7603395581, other_time: 20.2109375000
epoch time: 114.3699340820s, input_fired_time: 1.5157082081, output_fired_time: 0.0000000000, propagate_forward_time: 109.5446701050, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3095550537
validation[          , genome  6232] predictions:    6596/  10000 (65.96%), best:    6601/10000 (66.01%), error:      9916.66895, best error:      9894.24316 on epoch:    97, epoch:   99/100, mu: 0.9879494905, learning_rate: 0.0000004007, weight_decay: 0.0000020580

epoch time: 1460.0808105469s, input_fired_time: 25.4047107697, output_fired_time: 9.7322826385, propagate_forward_time: 548.5673217773, propagate_backward_time: 857.2313232422, weight_update_time: 0.7420505881, other_time: 19.1452026367
epoch time: 113.8709030151s, input_fired_time: 1.4687361717, output_fired_time: 0.0000000000, propagate_forward_time: 109.1829071045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2192611694
validation[          , genome  6232] predictions:    6613/  10000 (66.13%), best:    6601/10000 (66.01%), error:      9900.20703, best error:      9894.24316 on epoch:    97, epoch:  100/100, mu: 0.9880599380, learning_rate: 0.0000003719, weight_decay: 0.0000019441

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 583.2689208984s, input_fired_time: 7.7554874420, output_fired_time: 0.0000000000, propagate_forward_time: 554.9146728516, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 20.5987548828
best training[          , genome  6232] predictions:   35157/  50000 (70.31%), best:    6601/10000 (66.01%), error:     44988.70312, best error:      9894.24316 on epoch:    97, epoch:  101/100, mu: 0.9881644249, learning_rate: 0.0000003451, weight_decay: 0.0000018365
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 110.5347900391s, input_fired_time: 1.4435155392, output_fired_time: 0.0000000000, propagate_forward_time: 105.9329528809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1583251953
   testing[          , genome  6232] predictions:    6601/  10000 (66.01%), best:    6601/10000 (66.01%), error:      9894.24316, best error:      9894.24316 on epoch:    97, epoch:  101/100, mu: 0.9881644249, learning_rate: 0.0000003451, weight_decay: 0.0000018365
backpropagation finished successfully!
03:42:22 (27751): called boinc_finish(0)

</stderr_txt>
]]>