Nameexact_genome_1518178202_49_18349_1
Workunit2992121
Created9 Feb 2018, 15:58:24 UTC
Sent9 Feb 2018, 16:27:23 UTC
Report deadline9 Apr 2018, 1:20:41 UTC
Received14 Feb 2018, 18:57:30 UTC
Server stateOver
OutcomeComputation error
Client stateCompute error
Exit status1 (0x1) Unknown error number
Computer ID68550
Run time1 days 11 hours 53 min 28 sec
CPU time1 days 11 hours 8 min 5 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS2.59 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size187.20 MB
Peak swap size190.95 MB
Peak disk usage8.97 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
Incorrect function.
 (0x1) - exit code 1 (0x1)</message>
<stderr_txt>
ed_time: 3.1480925083, output_fired_time: 0.0000000000, propagate_forward_time: 137.3882293701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8156280518
validation[          , genome 18349] predictions:    9900/  10000 (99.00%), best:    9900/10000 (99.00%), error:       334.54590, best error:       334.54590 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1827.5310058594s, input_fired_time: 20.6452178955, output_fired_time: 22.6766033173, propagate_forward_time: 693.4051513672, propagate_backward_time: 1069.8093261719, weight_update_time: 2.4190053940, other_time: 20.9946289063
epoch time: 144.5529174805s, input_fired_time: 3.2762734890, output_fired_time: 0.0000000000, propagate_forward_time: 138.4225311279, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8541107178
validation[          , genome 18349] predictions:    9882/  10000 (98.82%), best:    9900/10000 (99.00%), error:       402.69864, best error:       334.54590 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1827.8875732422s, input_fired_time: 25.6996555328, output_fired_time: 22.4726409912, propagate_forward_time: 690.6659545898, propagate_backward_time: 1071.1763916016, weight_update_time: 1.4244327545, other_time: 17.8729248047
epoch time: 143.9370727539s, input_fired_time: 3.1593973637, output_fired_time: 0.0000000000, propagate_forward_time: 136.9528198242, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8248596191
validation[          , genome 18349] predictions:    9882/  10000 (98.82%), best:    9900/10000 (99.00%), error:       395.64752, best error:       334.54590 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1836.1313476563s, input_fired_time: 25.8602657318, output_fired_time: 23.6325073242, propagate_forward_time: 688.8184204102, propagate_backward_time: 1074.7618408203, weight_update_time: 1.4299923182, other_time: 23.0584716797
epoch time: 145.4897003174s, input_fired_time: 2.1803371906, output_fired_time: 0.0000000000, propagate_forward_time: 140.4678802490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8414764404
validation[          , genome 18349] predictions:    9860/  10000 (98.60%), best:    9900/10000 (99.00%), error:       414.79080, best error:       334.54590 on epoch:    34, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1834.5133056641s, input_fired_time: 25.8313274384, output_fired_time: 21.4401187897, propagate_forward_time: 688.4791870117, propagate_backward_time: 1077.7044677734, weight_update_time: 1.4271804094, other_time: 21.0582275391
epoch time: 143.4660186768s, input_fired_time: 2.1734058857, output_fired_time: 0.0000000000, propagate_forward_time: 137.4627685547, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8298492432
validation[          , genome 18349] predictions:    9845/  10000 (98.45%), best:    9900/10000 (99.00%), error:       463.95630, best error:       334.54590 on epoch:    34, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 1837.2655029297s, input_fired_time: 23.7530326843, output_fired_time: 20.4860668182, propagate_forward_time: 681.8195800781, propagate_backward_time: 1088.5577392578, weight_update_time: 1.4214371443, other_time: 22.6490478516
epoch time: 145.3529205322s, input_fired_time: 3.1770391464, output_fired_time: 0.0000000000, propagate_forward_time: 138.3305969238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8452911377
validation[          , genome 18349] predictions:    9892/  10000 (98.92%), best:    9900/10000 (99.00%), error:       338.55124, best error:       334.54590 on epoch:    34, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1830.3498535156s, input_fired_time: 22.9101486206, output_fired_time: 22.6349372864, propagate_forward_time: 692.5416870117, propagate_backward_time: 1072.3403320313, weight_update_time: 1.4271910191, other_time: 19.9228515625
epoch time: 143.3766479492s, input_fired_time: 3.1791670322, output_fired_time: 0.0000000000, propagate_forward_time: 137.3292846680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8681945801
validation[          , genome 18349] predictions:    9861/  10000 (98.61%), best:    9900/10000 (99.00%), error:       425.71698, best error:       334.54590 on epoch:    34, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1825.1562500000s, input_fired_time: 23.5985584259, output_fired_time: 24.4317684174, propagate_forward_time: 684.0025634766, propagate_backward_time: 1075.1228027344, weight_update_time: 1.4228559732, other_time: 18.0006103516
epoch time: 141.5438079834s, input_fired_time: 2.1941964626, output_fired_time: 0.0000000000, propagate_forward_time: 135.4985504150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8510589600
validation[          , genome 18349] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       325.38086, best error:       325.38086 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1823.9176025391s, input_fired_time: 21.7581901550, output_fired_time: 20.6376533508, propagate_forward_time: 697.6563720703, propagate_backward_time: 1059.0668945313, weight_update_time: 1.4239362478, other_time: 24.7984619141
epoch time: 144.0748901367s, input_fired_time: 2.1844539642, output_fired_time: 0.0000000000, propagate_forward_time: 136.9342041016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9562377930
validation[          , genome 18349] predictions:    9887/  10000 (98.87%), best:    9892/10000 (98.92%), error:       334.60800, best error:       325.38086 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1837.1495361328s, input_fired_time: 23.9094791412, output_fired_time: 22.5091495514, propagate_forward_time: 686.7907714844, propagate_backward_time: 1084.9307861328, weight_update_time: 1.4274191856, other_time: 19.0093994141
epoch time: 144.4447479248s, input_fired_time: 3.1595396996, output_fired_time: 0.0000000000, propagate_forward_time: 136.3554992676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9297027588
validation[          , genome 18349] predictions:    9898/  10000 (98.98%), best:    9892/10000 (98.92%), error:       328.63239, best error:       325.38086 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1839.0965576172s, input_fired_time: 20.7484874725, output_fired_time: 21.6711845398, propagate_forward_time: 690.4995727539, propagate_backward_time: 1083.6148681641, weight_update_time: 3.5293271542, other_time: 22.5623779297
epoch time: 144.9878692627s, input_fired_time: 3.1589636803, output_fired_time: 0.0000000000, propagate_forward_time: 138.9534912109, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8754119873
validation[          , genome 18349] predictions:    9907/  10000 (99.07%), best:    9892/10000 (98.92%), error:       345.18649, best error:       325.38086 on epoch:    41, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1836.1063232422s, input_fired_time: 24.5987014771, output_fired_time: 19.6192474365, propagate_forward_time: 691.1047973633, propagate_backward_time: 1074.7330322266, weight_update_time: 1.4267622232, other_time: 26.0504150391
epoch time: 145.1464233398s, input_fired_time: 4.2551164627, output_fired_time: 0.0000000000, propagate_forward_time: 136.0859375000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8053741455
validation[          , genome 18349] predictions:    9891/  10000 (98.91%), best:    9891/10000 (98.91%), error:       325.26257, best error:       325.26257 on epoch:    45, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1828.3310546875s, input_fired_time: 21.6478347778, output_fired_time: 22.5735263824, propagate_forward_time: 684.7686767578, propagate_backward_time: 1074.2901611328, weight_update_time: 1.4187219143, other_time: 25.0509033203
epoch time: 144.0218963623s, input_fired_time: 6.2418661118, output_fired_time: 0.0000000000, propagate_forward_time: 133.9356842041, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8443450928
validation[          , genome 18349] predictions:    9882/  10000 (98.82%), best:    9891/10000 (98.91%), error:       335.91971, best error:       325.26257 on epoch:    45, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1826.4957275391s, input_fired_time: 19.6212348938, output_fired_time: 22.5869159698, propagate_forward_time: 694.2072143555, propagate_backward_time: 1068.8691406250, weight_update_time: 1.4272930622, other_time: 21.2111816406
epoch time: 143.6912536621s, input_fired_time: 2.1759195328, output_fired_time: 0.0000000000, propagate_forward_time: 138.6630096436, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8523254395
validation[          , genome 18349] predictions:    9862/  10000 (98.62%), best:    9891/10000 (98.91%), error:       422.66464, best error:       325.26257 on epoch:    45, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1823.6512451172s, input_fired_time: 22.6566638947, output_fired_time: 21.5287628174, propagate_forward_time: 681.7645874023, propagate_backward_time: 1073.6126708984, weight_update_time: 2.5242421627, other_time: 24.0885009766
epoch time: 144.3479461670s, input_fired_time: 2.1984245777, output_fired_time: 0.0000000000, propagate_forward_time: 139.2863006592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8632202148
validation[          , genome 18349] predictions:    9850/  10000 (98.50%), best:    9891/10000 (98.91%), error:       499.71982, best error:       325.26257 on epoch:    45, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 1825.3509521484s, input_fired_time: 21.6167755127, output_fired_time: 20.4719562531, propagate_forward_time: 682.4603881836, propagate_backward_time: 1081.7238769531, weight_update_time: 2.4191424847, other_time: 19.0778808594
epoch time: 142.8762664795s, input_fired_time: 2.1698658466, output_fired_time: 0.0000000000, propagate_forward_time: 137.8628997803, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8435058594
validation[          , genome 18349] predictions:    9900/  10000 (99.00%), best:    9900/10000 (99.00%), error:       322.76151, best error:       322.76151 on epoch:    49, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1822.9228515625s, input_fired_time: 24.7567176819, output_fired_time: 20.6717262268, propagate_forward_time: 689.1754150391, propagate_backward_time: 1070.2727050781, weight_update_time: 1.4245954752, other_time: 18.0462646484
epoch time: 142.2870178223s, input_fired_time: 2.1773116589, output_fired_time: 0.0000000000, propagate_forward_time: 137.2729492188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8367614746
validation[          , genome 18349] predictions:    9907/  10000 (99.07%), best:    9907/10000 (99.07%), error:       293.49686, best error:       293.49686 on epoch:    50, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 1824.5952148438s, input_fired_time: 23.9735240936, output_fired_time: 23.6405696869, propagate_forward_time: 675.9441528320, propagate_backward_time: 1080.9683837891, weight_update_time: 1.4355283976, other_time: 20.0684814453
epoch time: 143.5334320068s, input_fired_time: 2.1778194904, output_fired_time: 0.0000000000, propagate_forward_time: 138.5173645020, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8382415771
validation[          , genome 18349] predictions:    9894/  10000 (98.94%), best:    9907/10000 (99.07%), error:       350.13373, best error:       293.49686 on epoch:    50, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 1829.2309570313s, input_fired_time: 21.6549167633, output_fired_time: 21.4761028290, propagate_forward_time: 688.7670288086, propagate_backward_time: 1078.1691894531, weight_update_time: 1.4342663288, other_time: 19.1638183594
epoch time: 144.8527221680s, input_fired_time: 3.1694869995, output_fired_time: 0.0000000000, propagate_forward_time: 137.8435363770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8396911621
validation[          , genome 18349] predictions:    9907/  10000 (99.07%), best:    9907/10000 (99.07%), error:       303.75217, best error:       293.49686 on epoch:    50, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 1839.4598388672s, input_fired_time: 22.6006183624, output_fired_time: 25.8116970062, propagate_forward_time: 695.7521362305, propagate_backward_time: 1075.3000488281, weight_update_time: 3.5234584808, other_time: 19.9953613281
epoch time: 145.1985778809s, input_fired_time: 2.1949162483, output_fired_time: 0.0000000000, propagate_forward_time: 139.1747283936, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8289337158
validation[          , genome 18349] predictions:    9885/  10000 (98.85%), best:    9907/10000 (99.07%), error:       392.44473, best error:       293.49686 on epoch:    50, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 1833.2723388672s, input_fired_time: 21.7044849396, output_fired_time: 25.6841354370, propagate_forward_time: 689.4560546875, propagate_backward_time: 1076.4593505859, weight_update_time: 1.4266608953, other_time: 19.9683837891
epoch time: 143.8127593994s, input_fired_time: 2.1809716225, output_fired_time: 0.0000000000, propagate_forward_time: 137.7155151367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9162750244
validation[          , genome 18349] predictions:    9912/  10000 (99.12%), best:    9912/10000 (99.12%), error:       272.38223, best error:       272.38223 on epoch:    54, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 1835.8585205078s, input_fired_time: 26.9167461395, output_fired_time: 23.5138149261, propagate_forward_time: 682.0790405273, propagate_backward_time: 1080.3671875000, weight_update_time: 1.4269295931, other_time: 22.9816894531
epoch time: 145.0224304199s, input_fired_time: 2.1912631989, output_fired_time: 0.0000000000, propagate_forward_time: 139.9870147705, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8441467285
validation[          , genome 18349] predictions:    9901/  10000 (99.01%), best:    9912/10000 (99.12%), error:       304.87006, best error:       272.38223 on epoch:    54, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 1831.3425292969s, input_fired_time: 21.8882179260, output_fired_time: 23.8232440948, propagate_forward_time: 694.8316040039, propagate_backward_time: 1069.7294921875, weight_update_time: 1.4260091782, other_time: 21.0700683594
epoch time: 139.6100006104s, input_fired_time: 2.1666553020, output_fired_time: 0.0000000000, propagate_forward_time: 134.6132659912, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8300781250
validation[          , genome 18349] predictions:    9898/  10000 (98.98%), best:    9912/10000 (99.12%), error:       312.44818, best error:       272.38223 on epoch:    54, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 1836.9617919922s, input_fired_time: 23.6833858490, output_fired_time: 22.5604305267, propagate_forward_time: 691.5753784180, propagate_backward_time: 1075.9587402344, weight_update_time: 2.4211628437, other_time: 23.1838378906
epoch time: 144.9188995361s, input_fired_time: 4.2607364655, output_fired_time: 0.0000000000, propagate_forward_time: 136.8074188232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8507385254
validation[          , genome 18349] predictions:    9911/  10000 (99.11%), best:    9912/10000 (99.12%), error:       287.58206, best error:       272.38223 on epoch:    54, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1840.2651367188s, input_fired_time: 23.7758541107, output_fired_time: 23.9307136536, propagate_forward_time: 687.9192504883, propagate_backward_time: 1080.6475830078, weight_update_time: 1.4221023321, other_time: 23.9918212891
epoch time: 145.7316741943s, input_fired_time: 3.1803655624, output_fired_time: 0.0000000000, propagate_forward_time: 139.6940460205, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8572692871
validation[          , genome 18349] predictions:    9888/  10000 (98.88%), best:    9912/10000 (99.12%), error:       359.13455, best error:       272.38223 on epoch:    54, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 1841.6162109375s, input_fired_time: 17.6776008606, output_fired_time: 21.6328983307, propagate_forward_time: 695.4090576172, propagate_backward_time: 1085.8994140625, weight_update_time: 2.4107472897, other_time: 20.9971923828
epoch time: 143.9350585938s, input_fired_time: 2.1666057110, output_fired_time: 0.0000000000, propagate_forward_time: 138.9038085938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8646392822
validation[          , genome 18349] predictions:    9908/  10000 (99.08%), best:    9912/10000 (99.12%), error:       281.55280, best error:       272.38223 on epoch:    54, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 1835.1480712891s, input_fired_time: 23.7416019440, output_fired_time: 21.6675853729, propagate_forward_time: 690.3962402344, propagate_backward_time: 1076.4764404297, weight_update_time: 2.4108469486, other_time: 22.8662109375
epoch time: 143.6851043701s, input_fired_time: 4.3591089249, output_fired_time: 0.0000000000, propagate_forward_time: 136.5037841797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8222045898
validation[          , genome 18349] predictions:    9892/  10000 (98.92%), best:    9912/10000 (99.12%), error:       360.90579, best error:       272.38223 on epoch:    54, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 1832.1696777344s, input_fired_time: 28.7350368500, output_fired_time: 24.4201641083, propagate_forward_time: 683.0106811523, propagate_backward_time: 1073.9815673828, weight_update_time: 1.4234670401, other_time: 22.0223388672
epoch time: 145.4750061035s, input_fired_time: 3.1670579910, output_fired_time: 0.0000000000, propagate_forward_time: 138.4856872559, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8222656250
validation[          , genome 18349] predictions:    9911/  10000 (99.11%), best:    9912/10000 (99.12%), error:       288.71741, best error:       272.38223 on epoch:    54, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 1836.6840820313s, input_fired_time: 18.6408309937, output_fired_time: 25.6846370697, propagate_forward_time: 691.7167358398, propagate_backward_time: 1080.5839843750, weight_update_time: 1.4307904243, other_time: 20.0578613281
epoch time: 143.4367980957s, input_fired_time: 4.1504764557, output_fired_time: 0.0000000000, propagate_forward_time: 136.4572143555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8291015625
validation[          , genome 18349] predictions:    9909/  10000 (99.09%), best:    9912/10000 (99.12%), error:       306.63116, best error:       272.38223 on epoch:    54, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 1835.1528320313s, input_fired_time: 18.5946807861, output_fired_time: 22.4758110046, propagate_forward_time: 690.8875122070, propagate_backward_time: 1085.1954345703, weight_update_time: 2.4135735035, other_time: 17.9993896484
epoch time: 144.6900482178s, input_fired_time: 2.1955823898, output_fired_time: 0.0000000000, propagate_forward_time: 139.6158752441, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8785858154
validation[          , genome 18349] predictions:    9920/  10000 (99.20%), best:    9920/10000 (99.20%), error:       270.41864, best error:       270.41864 on epoch:    63, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 1833.2849121094s, input_fired_time: 22.6844882965, output_fired_time: 23.9214115143, propagate_forward_time: 692.3227539063, propagate_backward_time: 1071.1632080078, weight_update_time: 1.4302241802, other_time: 23.1931152344
epoch time: 142.8556213379s, input_fired_time: 5.2410106659, output_fired_time: 0.0000000000, propagate_forward_time: 132.7976226807, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8169860840
validation[          , genome 18349] predictions:    9909/  10000 (99.09%), best:    9920/10000 (99.20%), error:       286.74814, best error:       270.41864 on epoch:    63, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 1835.2072753906s, input_fired_time: 23.8773365021, output_fired_time: 24.7463836670, propagate_forward_time: 688.3829956055, propagate_backward_time: 1081.2957763672, weight_update_time: 1.4321942329, other_time: 16.9049072266
epoch time: 145.0941467285s, input_fired_time: 2.1899776459, output_fired_time: 0.0000000000, propagate_forward_time: 139.0877685547, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8164062500
validation[          , genome 18349] predictions:    9917/  10000 (99.17%), best:    9917/10000 (99.17%), error:       269.68045, best error:       269.68045 on epoch:    65, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 1834.3100585938s, input_fired_time: 21.5771675110, output_fired_time: 20.5972843170, propagate_forward_time: 685.3460083008, propagate_backward_time: 1082.6352539063, weight_update_time: 1.4196947813, other_time: 24.1542968750
epoch time: 144.6105194092s, input_fired_time: 2.1833610535, output_fired_time: 0.0000000000, propagate_forward_time: 139.5844116211, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8427429199
validation[          , genome 18349] predictions:    9914/  10000 (99.14%), best:    9917/10000 (99.17%), error:       287.74655, best error:       269.68045 on epoch:    65, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 1831.3941650391s, input_fired_time: 20.6985130310, output_fired_time: 20.4694976807, propagate_forward_time: 678.9398803711, propagate_backward_time: 1090.4400634766, weight_update_time: 1.4237180948, other_time: 20.8463134766
epoch time: 142.1867828369s, input_fired_time: 2.1747522354, output_fired_time: 0.0000000000, propagate_forward_time: 135.2202911377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7917327881
validation[          , genome 18349] predictions:    9908/  10000 (99.08%), best:    9917/10000 (99.17%), error:       308.43948, best error:       269.68045 on epoch:    65, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 1830.4750976563s, input_fired_time: 22.5844669342, output_fired_time: 22.6663112640, propagate_forward_time: 691.3732910156, propagate_backward_time: 1073.8869628906, weight_update_time: 1.4211580753, other_time: 19.9641113281
epoch time: 144.7983093262s, input_fired_time: 5.1508955956, output_fired_time: 0.0000000000, propagate_forward_time: 133.7073974609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.9400177002
validation[          , genome 18349] predictions:    9900/  10000 (99.00%), best:    9917/10000 (99.17%), error:       314.68311, best error:       269.68045 on epoch:    65, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 1826.7191162109s, input_fired_time: 21.9316272736, output_fired_time: 22.5617771149, propagate_forward_time: 688.0742187500, propagate_backward_time: 1077.2343750000, weight_update_time: 1.4295786619, other_time: 16.9171142578
epoch time: 144.0736083984s, input_fired_time: 3.1635510921, output_fired_time: 0.0000000000, propagate_forward_time: 136.0874176025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8226470947
validation[          , genome 18349] predictions:    9916/  10000 (99.16%), best:    9917/10000 (99.17%), error:       272.43900, best error:       269.68045 on epoch:    65, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 1834.8958740234s, input_fired_time: 20.7039833069, output_fired_time: 26.7226676941, propagate_forward_time: 701.2070922852, propagate_backward_time: 1066.3325195313, weight_update_time: 1.4285699129, other_time: 19.9296875000
epoch time: 144.9288177490s, input_fired_time: 2.1931746006, output_fired_time: 0.0000000000, propagate_forward_time: 139.8660583496, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8695831299
validation[          , genome 18349] predictions:    9913/  10000 (99.13%), best:    9917/10000 (99.17%), error:       273.21402, best error:       269.68045 on epoch:    65, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.97716
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000327556
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.31022e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 65
read number_validation_images: 10000
read best_validation_predictions: 9917
read best_validation_error: 269.68
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '1385575231'
read generator: 1385575231
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 2137.0654296875s, input_fired_time: 22.6819343566895, output_fired_time: 27.2313957214355, propagate_forward_time: 811.0654296875, propagate_backward_time: 1250.61657714844, weight_update_time: 2.66377973556519, other_time: 25.4700927734375
epoch time: 165.245864868164s, input_fired_time: 2.58569192886353, output_fired_time: 0, propagate_forward_time: 157.702621459961, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.95755004882813
validation[          , genome 18349] predictions:    9892/  10000 (98.92%), best:    9917/10000 (99.17%), error:       313.56061, best error:       269.68045 on epoch:    65, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 2148.8417968750s, input_fired_time: 29.7991027832, output_fired_time: 26.9758987427, propagate_forward_time: 811.1557006836, propagate_backward_time: 1256.2421875000, weight_update_time: 1.6822893620, other_time: 24.6689453125
epoch time: 164.8288269043s, input_fired_time: 3.4354853630, output_fired_time: 0.0000000000, propagate_forward_time: 157.4565887451, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9367523193
validation[          , genome 18349] predictions:    9919/  10000 (99.19%), best:    9919/10000 (99.19%), error:       258.15112, best error:       258.15112 on epoch:    72, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 2137.5563964844s, input_fired_time: 24.0945262909, output_fired_time: 25.8949489594, propagate_forward_time: 809.7448730469, propagate_backward_time: 1253.0023193359, weight_update_time: 1.6124192476, other_time: 24.8197021484
epoch time: 169.8419647217s, input_fired_time: 2.5586695671, output_fired_time: 0.0000000000, propagate_forward_time: 162.5079803467, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7753143311
validation[          , genome 18349] predictions:    9918/  10000 (99.18%), best:    9919/10000 (99.19%), error:       262.16095, best error:       258.15112 on epoch:    72, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 2136.0998535156s, input_fired_time: 30.4628562927, output_fired_time: 30.9694042206, propagate_forward_time: 785.6168212891, propagate_backward_time: 1264.1250000000, weight_update_time: 1.6560235023, other_time: 24.9256591797
epoch time: 166.6800537109s, input_fired_time: 3.5159525871, output_fired_time: 0.0000000000, propagate_forward_time: 159.5803680420, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5837402344
validation[          , genome 18349] predictions:    9917/  10000 (99.17%), best:    9919/10000 (99.19%), error:       264.07095, best error:       258.15112 on epoch:    72, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 2143.4099121094s, input_fired_time: 25.0448875427, output_fired_time: 27.1411838531, propagate_forward_time: 803.1680908203, propagate_backward_time: 1263.3348388672, weight_update_time: 1.6602865458, other_time: 24.7209472656
epoch time: 166.6669006348s, input_fired_time: 3.5213944912, output_fired_time: 0.0000000000, propagate_forward_time: 158.5971527100, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5483551025
validation[          , genome 18349] predictions:    9910/  10000 (99.10%), best:    9919/10000 (99.19%), error:       277.14792, best error:       258.15112 on epoch:    72, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 2103.6967773438s, input_fired_time: 25.0552864075, output_fired_time: 25.4099788666, propagate_forward_time: 787.4528808594, propagate_backward_time: 1240.2910156250, weight_update_time: 1.5864770412, other_time: 25.4877929688
epoch time: 174.5244293213s, input_fired_time: 3.7185380459, output_fired_time: 0.0000000000, propagate_forward_time: 165.2235107422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5823822021
validation[          , genome 18349] predictions:    9914/  10000 (99.14%), best:    9919/10000 (99.19%), error:       274.80270, best error:       258.15112 on epoch:    72, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 2128.1604003906s, input_fired_time: 29.9670467377, output_fired_time: 25.8946304321, propagate_forward_time: 792.6111450195, propagate_backward_time: 1255.5277099609, weight_update_time: 2.7484080791, other_time: 24.1600341797
epoch time: 161.9616394043s, input_fired_time: 3.3356184959, output_fired_time: 0.0000000000, propagate_forward_time: 155.0711975098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5548248291
validation[          , genome 18349] predictions:    9911/  10000 (99.11%), best:    9919/10000 (99.19%), error:       271.23053, best error:       258.15112 on epoch:    72, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 2063.7885742188s, input_fired_time: 21.1981067657, output_fired_time: 26.3345508575, propagate_forward_time: 780.5858764648, propagate_backward_time: 1214.6577148438, weight_update_time: 2.6628615856, other_time: 21.0122070313
epoch time: 163.6150817871s, input_fired_time: 2.8100948334, output_fired_time: 0.0000000000, propagate_forward_time: 157.2970733643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5079193115
validation[          , genome 18349] predictions:    9912/  10000 (99.12%), best:    9912/10000 (99.12%), error:       256.53320, best error:       256.53320 on epoch:    78, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 2097.4514160156s, input_fired_time: 28.0593070984, output_fired_time: 26.8704204559, propagate_forward_time: 793.2480468750, propagate_backward_time: 1227.0454101563, weight_update_time: 1.6020376682, other_time: 22.2282714844
epoch time: 170.8797760010s, input_fired_time: 3.5579447746, output_fired_time: 0.0000000000, propagate_forward_time: 161.6769409180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.6448974609
validation[          , genome 18349] predictions:    9924/  10000 (99.24%), best:    9912/10000 (99.12%), error:       265.64679, best error:       256.53320 on epoch:    78, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 2118.4582519531s, input_fired_time: 24.4773826599, output_fired_time: 29.5115261078, propagate_forward_time: 789.4563598633, propagate_backward_time: 1252.4617919922, weight_update_time: 1.5942505598, other_time: 22.5513916016
epoch time: 165.5048217773s, input_fired_time: 3.5131723881, output_fired_time: 0.0000000000, propagate_forward_time: 158.2868652344, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7047882080
validation[          , genome 18349] predictions:    9911/  10000 (99.11%), best:    9912/10000 (99.12%), error:       297.30048, best error:       256.53320 on epoch:    78, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 2055.3227539063s, input_fired_time: 24.9140739441, output_fired_time: 28.3733463287, propagate_forward_time: 748.6116943359, propagate_backward_time: 1226.9611816406, weight_update_time: 2.6683273315, other_time: 26.4625244141
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1518178202_49_18349_1_r2079945528_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.982312
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00019612
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.84479e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 81
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 78
read number_validation_images: 10000
read best_validation_predictions: 9912
read best_validation_error: 256.533
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18349
read normal distribution: '1 -1.66620361804962 0.0640846490859985'
generator_str: '454959729'
read generator: 454959729
read generated_by_map:
1 split_edge 1
reading 128 nodes.
reading 485 edges.
number input nodes: 1
number softmax nodes: 10
ERROR: invalid input file, expected line to be 'BACKPROP_ORDER' but line was 'INNOVATION_NUMBERS'
parsed input file
SANITY CHECK FAILED! nodes[1] had total inputs: 0 but 1 inputs were counted. 
node innovation number: 1028
	edge with innovation number 3866 had node as output, edge disabled? 0
ERROR! genome failed sanity check! This should never happen!
19:50:18 (3508): called boinc_finish(1)

</stderr_txt>
]]>