Nameexact_genome_1535714404_164_12086_0
Workunit3943847
Created6 Sep 2018, 12:13:06 UTC
Sent6 Sep 2018, 12:16:34 UTC
Report deadline10 Feb 2019, 11:56:32 UTC
Received21 Nov 2018, 22:29:29 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID86145
Run time2 days 0 hours 34 min 27 sec
CPU time2 days 0 hours 28 min 22 sec
Validate stateInitial
Credit0.00
Device peak FLOPS4.23 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size551.07 MB
Peak swap size553.60 MB
Peak disk usage9.54 MB

Stderr output

<core_client_version>7.6.31</core_client_version>
<![CDATA[
<stderr_txt>
9521179, learning_rate: 0.0002101796, weight_decay: 0.0000823780

epoch time: 1530.7307128906s, input_fired_time: 24.5385456085, output_fired_time: 6.2731533051, propagate_forward_time: 570.1422729492, propagate_backward_time: 916.6732788086, weight_update_time: 0.5294715762, other_time: 13.1033935547
epoch time: 116.5108261108s, input_fired_time: 1.2440268993, output_fired_time: 0.0000000000, propagate_forward_time: 113.2408828735, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0259170532
validation[          , genome 12086] predictions:    6604/  10000 (66.04%), best:    6604/10000 (66.04%), error:      9858.55469, best error:      9858.55469 on epoch:    28, epoch:   28/100, mu: 0.8647741079, learning_rate: 0.0001933767, weight_decay: 0.0000769676

epoch time: 1615.6766357422s, input_fired_time: 24.7935600281, output_fired_time: 6.9572114944, propagate_forward_time: 609.0396728516, propagate_backward_time: 960.1317749023, weight_update_time: 0.5581832528, other_time: 14.7544555664
epoch time: 101.1665115356s, input_fired_time: 1.1928212643, output_fired_time: 0.0000000000, propagate_forward_time: 98.4044570923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5692367554
validation[          , genome 12086] predictions:    6574/  10000 (65.74%), best:    6574/10000 (65.74%), error:      9845.58691, best error:      9845.58691 on epoch:    29, epoch:   29/100, mu: 0.8712436557, learning_rate: 0.0001779171, weight_decay: 0.0000719125

epoch time: 1680.7695312500s, input_fired_time: 24.9326286316, output_fired_time: 7.2184157372, propagate_forward_time: 638.1785888672, propagate_backward_time: 995.5637207031, weight_update_time: 0.5742226243, other_time: 14.8762207031
epoch time: 158.3213043213s, input_fired_time: 1.3521925211, output_fired_time: 0.0000000000, propagate_forward_time: 153.9783782959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9907379150
validation[          , genome 12086] predictions:    6574/  10000 (65.74%), best:    6574/10000 (65.74%), error:      9898.68262, best error:      9845.58691 on epoch:    29, epoch:   30/100, mu: 0.8773790002, learning_rate: 0.0001636934, weight_decay: 0.0000671895

epoch time: 1631.3973388672s, input_fired_time: 24.8114986420, output_fired_time: 6.9413228035, propagate_forward_time: 615.4381713867, propagate_backward_time: 969.8881835938, weight_update_time: 0.5579677820, other_time: 14.3181762695
epoch time: 123.3905258179s, input_fired_time: 1.2653346062, output_fired_time: 0.0000000000, propagate_forward_time: 119.9890975952, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1360931396
validation[          , genome 12086] predictions:    6589/  10000 (65.89%), best:    6574/10000 (65.74%), error:      9896.68066, best error:      9845.58691 on epoch:    29, epoch:   31/100, mu: 0.8831973672, learning_rate: 0.0001506069, weight_decay: 0.0000627766

epoch time: 1546.0832519531s, input_fired_time: 24.7107677460, output_fired_time: 6.4518990517, propagate_forward_time: 576.3876953125, propagate_backward_time: 925.1159667969, weight_update_time: 0.5395098925, other_time: 13.4168701172
epoch time: 134.6913452148s, input_fired_time: 1.2998769283, output_fired_time: 0.0000000000, propagate_forward_time: 130.9340057373, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4574584961
validation[          , genome 12086] predictions:    6620/  10000 (66.20%), best:    6620/10000 (66.20%), error:      9806.70508, best error:      9806.70508 on epoch:    32, epoch:   32/100, mu: 0.8887151480, learning_rate: 0.0001385666, weight_decay: 0.0000586536

epoch time: 1671.3176269531s, input_fired_time: 24.9564399719, output_fired_time: 7.1493782997, propagate_forward_time: 632.0250854492, propagate_backward_time: 991.9244995117, weight_update_time: 0.5779300928, other_time: 15.2622070312
epoch time: 139.9247894287s, input_fired_time: 1.3613899946, output_fired_time: 0.0000000000, propagate_forward_time: 135.9176483154, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6457519531
validation[          , genome 12086] predictions:    6668/  10000 (66.68%), best:    6668/10000 (66.68%), error:      9688.89648, best error:      9688.89648 on epoch:    33, epoch:   33/100, mu: 0.8939478397, learning_rate: 0.0001274888, weight_decay: 0.0000548014

epoch time: 1600.2779541016s, input_fired_time: 24.7841339111, output_fired_time: 6.7258577347, propagate_forward_time: 603.5468139648, propagate_backward_time: 951.3049926758, weight_update_time: 0.5529631972, other_time: 13.9161376953
epoch time: 131.4207763672s, input_fired_time: 1.2895576954, output_fired_time: 0.0000000000, propagate_forward_time: 127.8292160034, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3020095825
validation[          , genome 12086] predictions:    6648/  10000 (66.48%), best:    6668/10000 (66.68%), error:      9704.33691, best error:      9688.89648 on epoch:    33, epoch:   34/100, mu: 0.8989102244, learning_rate: 0.0001172966, weight_decay: 0.0000512022

epoch time: 1616.8404541016s, input_fired_time: 24.7945308685, output_fired_time: 6.7898998260, propagate_forward_time: 610.4387817383, propagate_backward_time: 961.0264282227, weight_update_time: 0.5454750657, other_time: 13.7907714844
epoch time: 155.1148071289s, input_fired_time: 1.3615804911, output_fired_time: 0.0000000000, propagate_forward_time: 150.8319702148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9212493896
validation[          , genome 12086] predictions:    6694/  10000 (66.94%), best:    6694/10000 (66.94%), error:      9638.81738, best error:      9638.81738 on epoch:    35, epoch:   35/100, mu: 0.9036161900, learning_rate: 0.0001079193, weight_decay: 0.0000478393

epoch time: 1744.6442871094s, input_fired_time: 25.2208766937, output_fired_time: 7.6465086937, propagate_forward_time: 667.8517456055, propagate_backward_time: 1028.4498291016, weight_update_time: 0.5803787112, other_time: 15.4754638672
epoch time: 159.0430450439s, input_fired_time: 1.3643547297, output_fired_time: 0.0000000000, propagate_forward_time: 154.6636505127, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0150451660
validation[          , genome 12086] predictions:    6661/  10000 (66.61%), best:    6661/10000 (66.61%), error:      9638.58789, best error:      9638.58789 on epoch:    36, epoch:   36/100, mu: 0.9080790281, learning_rate: 0.0000992917, weight_decay: 0.0000446973

epoch time: 1686.9575195312s, input_fired_time: 24.8769569397, output_fired_time: 7.2426748276, propagate_forward_time: 640.1208496094, propagate_backward_time: 1000.4004516602, weight_update_time: 0.5564845204, other_time: 14.3165893555
epoch time: 125.5933685303s, input_fired_time: 1.2754546404, output_fired_time: 0.0000000000, propagate_forward_time: 122.2024230957, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1154937744
validation[          , genome 12086] predictions:    6656/  10000 (66.56%), best:    6661/10000 (66.61%), error:      9736.42383, best error:      9638.58789 on epoch:    36, epoch:   37/100, mu: 0.9123113155, learning_rate: 0.0000913537, weight_decay: 0.0000417617

epoch time: 1735.2612304688s, input_fired_time: 25.2197074890, output_fired_time: 7.5562529564, propagate_forward_time: 662.9123535156, propagate_backward_time: 1024.7471923828, weight_update_time: 0.5707035065, other_time: 14.8256835938
epoch time: 157.6526336670s, input_fired_time: 1.3506990671, output_fired_time: 0.0000000000, propagate_forward_time: 153.3246002197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9773406982
validation[          , genome 12086] predictions:    6628/  10000 (66.28%), best:    6661/10000 (66.61%), error:      9761.10547, best error:      9638.58789 on epoch:    36, epoch:   38/100, mu: 0.9163249731, learning_rate: 0.0000840504, weight_decay: 0.0000390189

epoch time: 1750.6785888672s, input_fired_time: 25.2397174835, output_fired_time: 7.6652793884, propagate_forward_time: 669.3161621094, propagate_backward_time: 1032.7736816406, weight_update_time: 0.5864502192, other_time: 15.6837158203
epoch time: 148.8560638428s, input_fired_time: 1.3505487442, output_fired_time: 0.0000000000, propagate_forward_time: 144.7554321289, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7500762939
validation[          , genome 12086] predictions:    6629/  10000 (66.29%), best:    6661/10000 (66.61%), error:      9676.71777, best error:      9638.58789 on epoch:    36, epoch:   39/100, mu: 0.9201312661, learning_rate: 0.0000773310, weight_decay: 0.0000364562

epoch time: 1857.2413330078s, input_fired_time: 25.4422569275, output_fired_time: 8.3260326385, propagate_forward_time: 716.6971435547, propagate_backward_time: 1090.4832763672, weight_update_time: 0.5971066952, other_time: 16.2926025391
epoch time: 159.3373565674s, input_fired_time: 1.3848420382, output_fired_time: 0.0000000000, propagate_forward_time: 154.9317016602, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0208129883
validation[          , genome 12086] predictions:    6673/  10000 (66.73%), best:    6661/10000 (66.61%), error:      9641.90137, best error:      9638.58789 on epoch:    36, epoch:   40/100, mu: 0.9237409234, learning_rate: 0.0000711487, weight_decay: 0.0000340619

epoch time: 1864.2171630859s, input_fired_time: 25.5832939148, output_fired_time: 8.4028711319, propagate_forward_time: 719.6883544922, propagate_backward_time: 1093.8572998047, weight_update_time: 0.6082279682, other_time: 16.6854248047
epoch time: 124.7554168701s, input_fired_time: 1.3633527756, output_fired_time: 0.0000000000, propagate_forward_time: 120.9075698853, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4844970703
validation[          , genome 12086] predictions:    6685/  10000 (66.85%), best:    6685/10000 (66.85%), error:      9628.25586, best error:      9628.25586 on epoch:    41, epoch:   41/100, mu: 0.9271640778, learning_rate: 0.0000654607, weight_decay: 0.0000318248

epoch time: 1574.3753662109s, input_fired_time: 24.8978137970, output_fired_time: 6.6696844101, propagate_forward_time: 588.3645629883, propagate_backward_time: 940.7247314453, weight_update_time: 0.5444613695, other_time: 13.7185668945
epoch time: 127.5943984985s, input_fired_time: 1.3038605452, output_fired_time: 0.0000000000, propagate_forward_time: 124.0184020996, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2721328735
validation[          , genome 12086] predictions:    6643/  10000 (66.43%), best:    6685/10000 (66.85%), error:      9710.33594, best error:      9628.25586 on epoch:    41, epoch:   42/100, mu: 0.9304103851, learning_rate: 0.0000602274, weight_decay: 0.0000297346

epoch time: 1921.4023437500s, input_fired_time: 25.7505798340, output_fired_time: 8.7280950546, propagate_forward_time: 745.9172363281, propagate_backward_time: 1124.0069580078, weight_update_time: 0.6276445389, other_time: 16.9993896484
epoch time: 117.1631622314s, input_fired_time: 1.2768082619, output_fired_time: 0.0000000000, propagate_forward_time: 113.9726028442, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9137496948
validation[          , genome 12086] predictions:    6692/  10000 (66.92%), best:    6692/10000 (66.92%), error:      9527.04688, best error:      9527.04688 on epoch:    43, epoch:   43/100, mu: 0.9334889650, learning_rate: 0.0000554125, weight_decay: 0.0000277817

epoch time: 1876.7529296875s, input_fired_time: 25.6612453461, output_fired_time: 8.4230995178, propagate_forward_time: 724.0076904297, propagate_backward_time: 1102.2877197266, weight_update_time: 0.6003420949, other_time: 16.3731689453
epoch time: 145.2534027100s, input_fired_time: 1.3630036116, output_fired_time: 0.0000000000, propagate_forward_time: 141.1773223877, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7130737305
validation[          , genome 12086] predictions:    6722/  10000 (67.22%), best:    6692/10000 (66.92%), error:      9580.04590, best error:      9527.04688 on epoch:    43, epoch:   44/100, mu: 0.9364085197, learning_rate: 0.0000509825, weight_decay: 0.0000259571

epoch time: 1832.1992187500s, input_fired_time: 25.7118644714, output_fired_time: 8.1974239349, propagate_forward_time: 702.9897460938, propagate_backward_time: 1078.3575439453, weight_update_time: 0.6079849601, other_time: 16.9426269531
epoch time: 159.3869323730s, input_fired_time: 1.3844598532, output_fired_time: 0.0000000000, propagate_forward_time: 155.1202697754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8822021484
validation[          , genome 12086] predictions:    6726/  10000 (67.26%), best:    6726/10000 (67.26%), error:      9441.39355, best error:      9441.39355 on epoch:    45, epoch:   45/100, mu: 0.9391772151, learning_rate: 0.0000469067, weight_decay: 0.0000242523

epoch time: 1811.2622070312s, input_fired_time: 25.6170558929, output_fired_time: 8.0382862091, propagate_forward_time: 697.6271972656, propagate_backward_time: 1063.8525390625, weight_update_time: 0.5992735624, other_time: 16.1270751953
epoch time: 132.9125671387s, input_fired_time: 1.3708225489, output_fired_time: 0.0000000000, propagate_forward_time: 129.0812683105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4604797363
validation[          , genome 12086] predictions:    6766/  10000 (67.66%), best:    6766/10000 (67.66%), error:      9435.72754, best error:      9435.72754 on epoch:    46, epoch:   46/100, mu: 0.9418028593, learning_rate: 0.0000431567, weight_decay: 0.0000226594

epoch time: 1579.1699218750s, input_fired_time: 25.0519161224, output_fired_time: 6.5498366356, propagate_forward_time: 591.3131713867, propagate_backward_time: 942.6071166992, weight_update_time: 0.5473979712, other_time: 13.6479492188
epoch time: 121.3982315063s, input_fired_time: 1.4095796347, output_fired_time: 0.0000000000, propagate_forward_time: 117.3862228394, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6024322510
validation[          , genome 12086] predictions:    6726/  10000 (67.26%), best:    6766/10000 (67.66%), error:      9486.95215, best error:      9435.72754 on epoch:    46, epoch:   47/100, mu: 0.9442929029, learning_rate: 0.0000397065, weight_decay: 0.0000211712

epoch time: 1514.3122558594s, input_fired_time: 24.9698276520, output_fired_time: 6.4074711800, propagate_forward_time: 561.4038696289, propagate_backward_time: 907.3742675781, weight_update_time: 0.5601006746, other_time: 14.1567993164
epoch time: 115.3201675415s, input_fired_time: 1.3169969320, output_fired_time: 0.0000000000, propagate_forward_time: 111.9990539551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0041198730
validation[          , genome 12086] predictions:    6676/  10000 (66.76%), best:    6766/10000 (67.66%), error:      9620.78125, best error:      9435.72754 on epoch:    46, epoch:   48/100, mu: 0.9466542602, learning_rate: 0.0000365322, weight_decay: 0.0000197807

epoch time: 1542.9747314453s, input_fired_time: 25.0473937988, output_fired_time: 6.3898234367, propagate_forward_time: 575.0370483398, propagate_backward_time: 922.7910156250, weight_update_time: 0.5569466352, other_time: 13.7095336914
epoch time: 108.9630279541s, input_fired_time: 1.3158935308, output_fired_time: 0.0000000000, propagate_forward_time: 105.7298049927, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9173278809
validation[          , genome 12086] predictions:    6672/  10000 (66.72%), best:    6766/10000 (67.66%), error:      9629.29395, best error:      9435.72754 on epoch:    46, epoch:   49/100, mu: 0.9488936663, learning_rate: 0.0000336116, weight_decay: 0.0000184816

epoch time: 1539.7677001953s, input_fired_time: 25.2127418518, output_fired_time: 6.5486159325, propagate_forward_time: 572.1212768555, propagate_backward_time: 921.0676269531, weight_update_time: 0.5799183249, other_time: 14.8174438477
epoch time: 128.0717315674s, input_fired_time: 1.3508738279, output_fired_time: 0.0000000000, propagate_forward_time: 124.3738403320, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3470153809
validation[          , genome 12086] predictions:    6808/  10000 (68.08%), best:    6808/10000 (68.08%), error:      9319.96387, best error:      9319.96387 on epoch:    50, epoch:   50/100, mu: 0.9510173202, learning_rate: 0.0000309245, weight_decay: 0.0000172678

epoch time: 1497.2098388672s, input_fired_time: 24.9929027557, output_fired_time: 6.2307238579, propagate_forward_time: 554.8729858398, propagate_backward_time: 897.4880371094, weight_update_time: 0.5512991548, other_time: 13.6251831055
epoch time: 127.9821319580s, input_fired_time: 1.3381515741, output_fired_time: 0.0000000000, propagate_forward_time: 124.3178558350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3261260986
validation[          , genome 12086] predictions:    6732/  10000 (67.32%), best:    6808/10000 (68.08%), error:      9518.80273, best error:      9319.96387 on epoch:    50, epoch:   51/100, mu: 0.9530313015, learning_rate: 0.0000284522, weight_decay: 0.0000161337

epoch time: 1498.4727783203s, input_fired_time: 24.8510112762, output_fired_time: 6.1684780121, propagate_forward_time: 554.8175659180, propagate_backward_time: 899.2127075195, weight_update_time: 0.5406606793, other_time: 13.4230957031
epoch time: 114.6590957642s, input_fired_time: 1.3338631392, output_fired_time: 0.0000000000, propagate_forward_time: 111.3124694824, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0127639771
validation[          , genome 12086] predictions:    6813/  10000 (68.13%), best:    6813/10000 (68.13%), error:      9319.15430, best error:      9319.15430 on epoch:    52, epoch:   52/100, mu: 0.9549412131, learning_rate: 0.0000261776, weight_decay: 0.0000150740

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_164_12086.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_164_12086_0_r621545404_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.43698
read mu: 0.956752
read mu_delta: 0.948337
read initial_learning_rate: 0.00199354
read learning_rate: 2.40848e-05
read learning_rate_delta: 0.920055
read initial_weight_decay: 0.000515702
read weight_decay: 1.4084e-05
read weight_decay_delta: 0.934322
read batch_size: 79
read epsilon: 1e-07
read alpha: 0.057557
read input_dropout_probability: 0.00147438
read hidden_dropout_probability: 0.0806304
read velocity_reset: 2315
read epoch: 53
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 52
read number_validation_images: 10000
read best_validation_predictions: 6813
read best_validation_error: 9319.15
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 12086
read normal distribution: '1 0 0'
generator_str: '2142298914'
read generator: 2142298914
read generated_by_map:
1 enable_node 1
reading 132 nodes.
reading 530 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1510.41455078125s, input_fired_time: 24.8224925994873, output_fired_time: 6.27116060256958, propagate_forward_time: 560.190734863281, propagate_backward_time: 904.834106445312, weight_update_time: 0.565879344940186, other_time: 14.2960815429688
epoch time: 111.152725219727s, input_fired_time: 1.33031976222992, output_fired_time: 0, propagate_forward_time: 107.819297790527, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.00310516357422
validation[          , genome 12086] predictions:    6730/  10000 (67.30%), best:    6813/10000 (68.13%), error:      9426.75195, best error:      9319.15430 on epoch:    52, epoch:   53/100, mu: 0.9567524791, learning_rate: 0.0000240848, weight_decay: 0.0000140840

epoch time: 1525.5146484375s, input_fired_time: 25.0041923523, output_fired_time: 6.4949150085, propagate_forward_time: 566.7017211914, propagate_backward_time: 912.6496582031, weight_update_time: 0.5770069361, other_time: 14.6642456055
epoch time: 117.3637924194s, input_fired_time: 1.4086437225, output_fired_time: 0.0000000000, propagate_forward_time: 113.6509170532, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3042297363
validation[          , genome 12086] predictions:    6839/  10000 (68.39%), best:    6839/10000 (68.39%), error:      9228.94434, best error:      9228.94434 on epoch:    54, epoch:   54/100, mu: 0.9584701657, learning_rate: 0.0000221594, weight_decay: 0.0000131590

epoch time: 1521.1259765625s, input_fired_time: 24.9768562317, output_fired_time: 6.4928851128, propagate_forward_time: 564.9541015625, propagate_backward_time: 910.3925170898, weight_update_time: 0.5686314702, other_time: 14.3096313477
epoch time: 120.6876373291s, input_fired_time: 1.4019484520, output_fired_time: 0.0000000000, propagate_forward_time: 116.9359130859, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3497772217
validation[          , genome 12086] predictions:    6790/  10000 (67.90%), best:    6839/10000 (68.39%), error:      9343.09570, best error:      9228.94434 on epoch:    54, epoch:   55/100, mu: 0.9600991011, learning_rate: 0.0000203878, weight_decay: 0.0000122948

epoch time: 1571.0987548828s, input_fired_time: 25.2274131775, output_fired_time: 6.8611135483, propagate_forward_time: 589.2740478516, propagate_backward_time: 934.1263427734, weight_update_time: 0.5987403989, other_time: 15.6098632812
epoch time: 124.1797180176s, input_fired_time: 1.4151748419, output_fired_time: 0.0000000000, propagate_forward_time: 120.2689132690, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4956283569
validation[          , genome 12086] predictions:    6829/  10000 (68.29%), best:    6839/10000 (68.39%), error:      9244.55957, best error:      9228.94434 on epoch:    54, epoch:   56/100, mu: 0.9616438746, learning_rate: 0.0000187579, weight_decay: 0.0000114873

epoch time: 1511.7077636719s, input_fired_time: 24.9942836761, output_fired_time: 6.4154028893, propagate_forward_time: 561.9166259766, propagate_backward_time: 903.8203735352, weight_update_time: 0.5728712082, other_time: 14.5610961914
epoch time: 115.7986907959s, input_fired_time: 1.3556250334, output_fired_time: 0.0000000000, propagate_forward_time: 112.2295989990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2134704590
validation[          , genome 12086] predictions:    6828/  10000 (68.28%), best:    6828/10000 (68.28%), error:      9220.74609, best error:      9220.74609 on epoch:    57, epoch:   57/100, mu: 0.9631088376, learning_rate: 0.0000172583, weight_decay: 0.0000107328

epoch time: 1535.9490966797s, input_fired_time: 25.1029014587, output_fired_time: 6.5778408051, propagate_forward_time: 571.5776367188, propagate_backward_time: 917.3495483398, weight_update_time: 0.5871474743, other_time: 15.3411254883
epoch time: 126.7681961060s, input_fired_time: 1.4134781361, output_fired_time: 0.0000000000, propagate_forward_time: 122.8950195312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4597015381
validation[          , genome 12086] predictions:    6808/  10000 (68.08%), best:    6828/10000 (68.28%), error:      9226.31152, best error:      9220.74609 on epoch:    57, epoch:   58/100, mu: 0.9644981027, learning_rate: 0.0000158786, weight_decay: 0.0000100279

epoch time: 1498.8889160156s, input_fired_time: 24.9250049591, output_fired_time: 6.2541308403, propagate_forward_time: 557.3999633789, propagate_backward_time: 896.1314697266, weight_update_time: 0.5620103478, other_time: 14.1782836914
epoch time: 124.7332916260s, input_fired_time: 1.3825035095, output_fired_time: 0.0000000000, propagate_forward_time: 120.9961624146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3546218872
validation[          , genome 12086] predictions:    6781/  10000 (67.81%), best:    6828/10000 (68.28%), error:      9406.46484, best error:      9220.74609 on epoch:    57, epoch:   59/100, mu: 0.9658156037, learning_rate: 0.0000146092, weight_decay: 0.0000093693

epoch time: 1553.2921142578s, input_fired_time: 25.1724872589, output_fired_time: 6.6947941780, propagate_forward_time: 581.3017578125, propagate_backward_time: 924.7548828125, weight_update_time: 0.5868277550, other_time: 15.3681640625
epoch time: 121.6734619141s, input_fired_time: 1.3470407724, output_fired_time: 0.0000000000, propagate_forward_time: 118.0693740845, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2570495605
validation[          , genome 12086] predictions:    6851/  10000 (68.51%), best:    6851/10000 (68.51%), error:      9157.32129, best error:      9157.32129 on epoch:    60, epoch:   60/100, mu: 0.9670650363, learning_rate: 0.0000134412, weight_decay: 0.0000087539

epoch time: 1509.1333007812s, input_fired_time: 24.9588184357, output_fired_time: 6.3406872749, propagate_forward_time: 560.4794311523, propagate_backward_time: 903.5107421875, weight_update_time: 0.5575730205, other_time: 13.8435668945
epoch time: 118.8780822754s, input_fired_time: 1.3708448410, output_fired_time: 0.0000000000, propagate_forward_time: 115.2285156250, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2787246704
validation[          , genome 12086] predictions:    6859/  10000 (68.59%), best:    6851/10000 (68.51%), error:      9176.69922, best error:      9157.32129 on epoch:    60, epoch:   61/100, mu: 0.9682499170, learning_rate: 0.0000123667, weight_decay: 0.0000081790

epoch time: 1542.1784667969s, input_fired_time: 25.1054363251, output_fired_time: 6.5505537987, propagate_forward_time: 575.4425048828, propagate_backward_time: 920.3674316406, weight_update_time: 0.5722174644, other_time: 14.7125244141
epoch time: 115.2300262451s, input_fired_time: 1.3378120661, output_fired_time: 0.0000000000, propagate_forward_time: 111.8573379517, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0348739624
validation[          , genome 12086] predictions:    6867/  10000 (68.67%), best:    6867/10000 (68.67%), error:      9085.42285, best error:      9085.42285 on epoch:    62, epoch:   62/100, mu: 0.9693735838, learning_rate: 0.0000113780, weight_decay: 0.0000076418

epoch time: 1559.5141601562s, input_fired_time: 25.1266670227, output_fired_time: 6.6000776291, propagate_forward_time: 582.1092529297, propagate_backward_time: 930.8621826172, weight_update_time: 0.5802112222, other_time: 14.8159179688
epoch time: 119.7641372681s, input_fired_time: 1.3841501474, output_fired_time: 0.0000000000, propagate_forward_time: 116.0281982422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3517913818
validation[          , genome 12086] predictions:    6839/  10000 (68.39%), best:    6867/10000 (68.67%), error:      9142.53516, best error:      9085.42285 on epoch:    62, epoch:   63/100, mu: 0.9704391956, learning_rate: 0.0000104684, weight_decay: 0.0000071399

epoch time: 1532.8898925781s, input_fired_time: 25.1296119690, output_fired_time: 6.5391883850, propagate_forward_time: 569.3312377930, propagate_backward_time: 917.2667236328, weight_update_time: 0.5712731481, other_time: 14.6231079102
epoch time: 121.0078430176s, input_fired_time: 1.3839869499, output_fired_time: 0.0000000000, propagate_forward_time: 117.2457427979, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3781127930
validation[          , genome 12086] predictions:    6873/  10000 (68.73%), best:    6867/10000 (68.67%), error:      9106.61426, best error:      9085.42285 on epoch:    62, epoch:   64/100, mu: 0.9714497924, learning_rate: 0.0000096315, weight_decay: 0.0000066710

epoch time: 1534.2806396484s, input_fired_time: 25.1237335205, output_fired_time: 6.4919915199, propagate_forward_time: 568.9915161133, propagate_backward_time: 919.2604980469, weight_update_time: 0.5677523017, other_time: 14.4129028320
epoch time: 118.6322708130s, input_fired_time: 1.3685420752, output_fired_time: 0.0000000000, propagate_forward_time: 115.0649566650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1987686157
validation[          , genome 12086] predictions:    6877/  10000 (68.77%), best:    6877/10000 (68.77%), error:      9075.17188, best error:      9075.17188 on epoch:    65, epoch:   65/100, mu: 0.9724081755, learning_rate: 0.0000088615, weight_decay: 0.0000062329

epoch time: 1494.4138183594s, input_fired_time: 24.9633750916, output_fired_time: 6.2209939957, propagate_forward_time: 555.2880859375, propagate_backward_time: 894.1720581055, weight_update_time: 0.5559586287, other_time: 13.7693481445
epoch time: 119.0175399780s, input_fired_time: 1.3710482121, output_fired_time: 0.0000000000, propagate_forward_time: 115.3623504639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2841415405
validation[          , genome 12086] predictions:    6895/  10000 (68.95%), best:    6895/10000 (68.95%), error:      9028.40234, best error:      9028.40234 on epoch:    66, epoch:   66/100, mu: 0.9733170271, learning_rate: 0.0000081531, weight_decay: 0.0000058235

epoch time: 1559.5327148438s, input_fired_time: 25.2071189880, output_fired_time: 6.6396427155, propagate_forward_time: 582.1495361328, propagate_backward_time: 930.2865600586, weight_update_time: 0.5872055888, other_time: 15.2498168945
epoch time: 115.1814193726s, input_fired_time: 1.3521579504, output_fired_time: 0.0000000000, propagate_forward_time: 111.6888656616, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1403961182
validation[          , genome 12086] predictions:    6828/  10000 (68.28%), best:    6895/10000 (68.95%), error:      9203.24707, best error:      9028.40234 on epoch:    66, epoch:   67/100, mu: 0.9741789103, learning_rate: 0.0000075013, weight_decay: 0.0000054410

epoch time: 1473.7008056641s, input_fired_time: 24.8349742889, output_fired_time: 6.0430068970, propagate_forward_time: 547.4879760742, propagate_backward_time: 882.0083618164, weight_update_time: 0.5458953977, other_time: 13.3265380859
epoch time: 124.6893768311s, input_fired_time: 1.3990321159, output_fired_time: 0.0000000000, propagate_forward_time: 120.9300537109, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3602905273
validation[          , genome 12086] predictions:    6879/  10000 (68.79%), best:    6895/10000 (68.95%), error:      9047.86523, best error:      9028.40234 on epoch:    66, epoch:   68/100, mu: 0.9749962687, learning_rate: 0.0000069016, weight_decay: 0.0000050837

epoch time: 1460.4758300781s, input_fired_time: 24.8057041168, output_fired_time: 5.9499440193, propagate_forward_time: 540.3944091797, propagate_backward_time: 876.9251708984, weight_update_time: 0.5251169205, other_time: 12.4006347656
epoch time: 109.1418991089s, input_fired_time: 1.3339596987, output_fired_time: 0.0000000000, propagate_forward_time: 105.8462753296, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9616622925
validation[          , genome 12086] predictions:    6851/  10000 (68.51%), best:    6895/10000 (68.95%), error:      9222.82520, best error:      9028.40234 on epoch:    66, epoch:   69/100, mu: 0.9757714272, learning_rate: 0.0000063498, weight_decay: 0.0000047498

epoch time: 1489.5452880859s, input_fired_time: 24.8754463196, output_fired_time: 6.1342349052, propagate_forward_time: 550.9862670898, propagate_backward_time: 894.1531982422, weight_update_time: 0.5460981131, other_time: 13.3960571289
epoch time: 117.1727981567s, input_fired_time: 1.3656795025, output_fired_time: 0.0000000000, propagate_forward_time: 113.5956878662, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2114334106
validation[          , genome 12086] predictions:    6886/  10000 (68.86%), best:    6895/10000 (68.95%), error:      9077.13281, best error:      9028.40234 on epoch:    66, epoch:   70/100, mu: 0.9765065312, learning_rate: 0.0000058422, weight_decay: 0.0000044378

epoch time: 1478.7485351562s, input_fired_time: 24.8731536865, output_fired_time: 6.0614509583, propagate_forward_time: 546.5992431641, propagate_backward_time: 887.7819824219, weight_update_time: 0.5540664792, other_time: 13.4327392578
epoch time: 117.8350830078s, input_fired_time: 1.3301606178, output_fired_time: 0.0000000000, propagate_forward_time: 114.4170684814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0878524780
validation[          , genome 12086] predictions:    6883/  10000 (68.83%), best:    6895/10000 (68.95%), error:      9054.32324, best error:      9028.40234 on epoch:    66, epoch:   71/100, mu: 0.9772036672, learning_rate: 0.0000053751, weight_decay: 0.0000041464

epoch time: 1495.8167724609s, input_fired_time: 24.9024944305, output_fired_time: 6.2255487442, propagate_forward_time: 553.2696533203, propagate_backward_time: 897.9102783203, weight_update_time: 0.5533700585, other_time: 13.5087890625
epoch time: 119.2038497925s, input_fired_time: 1.3776649237, output_fired_time: 0.0000000000, propagate_forward_time: 115.5013961792, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3247909546
validation[          , genome 12086] predictions:    6903/  10000 (69.03%), best:    6895/10000 (68.95%), error:      9031.62598, best error:      9028.40234 on epoch:    66, epoch:   72/100, mu: 0.9778647423, learning_rate: 0.0000049454, weight_decay: 0.0000038740

epoch time: 1541.7409667969s, input_fired_time: 25.0996665955, output_fired_time: 6.4966416359, propagate_forward_time: 570.3803100586, propagate_backward_time: 925.3148803711, weight_update_time: 0.5652744770, other_time: 14.4495849609
epoch time: 123.5453186035s, input_fired_time: 1.3793277740, output_fired_time: 0.0000000000, propagate_forward_time: 119.8286285400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3373641968
validation[          , genome 12086] predictions:    6884/  10000 (68.84%), best:    6895/10000 (68.95%), error:      9057.38965, best error:      9028.40234 on epoch:    66, epoch:   73/100, mu: 0.9784916639, learning_rate: 0.0000045500, weight_decay: 0.0000036196

epoch time: 1543.4132080078s, input_fired_time: 25.0356864929, output_fired_time: 6.4927868843, propagate_forward_time: 576.8651733398, propagate_backward_time: 920.4739379883, weight_update_time: 0.5695917010, other_time: 14.5456542969
epoch time: 114.2612075806s, input_fired_time: 1.3564006090, output_fired_time: 0.0000000000, propagate_forward_time: 110.8020401001, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1027679443
validation[          , genome 12086] predictions:    6870/  10000 (68.70%), best:    6895/10000 (68.95%), error:      9043.21973, best error:      9028.40234 on epoch:    66, epoch:   74/100, mu: 0.9790862203, learning_rate: 0.0000041863, weight_decay: 0.0000033819

epoch time: 1539.8103027344s, input_fired_time: 25.0980129242, output_fired_time: 6.4681077003, propagate_forward_time: 572.8742675781, propagate_backward_time: 920.6158447266, weight_update_time: 0.5754770637, other_time: 14.7540283203
epoch time: 113.6799774170s, input_fired_time: 1.3219300508, output_fired_time: 0.0000000000, propagate_forward_time: 110.2980117798, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0600357056
validation[          , genome 12086] predictions:    6883/  10000 (68.83%), best:    6883/10000 (68.83%), error:      8991.38184, best error:      8991.38184 on epoch:    75, epoch:   75/100, mu: 0.9796500802, learning_rate: 0.0000038516, weight_decay: 0.0000031598

epoch time: 1502.7554931641s, input_fired_time: 24.8986320496, output_fired_time: 6.2176384926, propagate_forward_time: 553.6026611328, propagate_backward_time: 903.9127197266, weight_update_time: 0.5566454530, other_time: 14.1237792969
epoch time: 119.5511169434s, input_fired_time: 1.4026870728, output_fired_time: 0.0000000000, propagate_forward_time: 115.8274078369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3210220337
validation[          , genome 12086] predictions:    6917/  10000 (69.17%), best:    6917/10000 (69.17%), error:      8950.75977, best error:      8950.75977 on epoch:    76, epoch:   76/100, mu: 0.9801847935, learning_rate: 0.0000035437, weight_decay: 0.0000029522

epoch time: 1524.1142578125s, input_fired_time: 24.9668483734, output_fired_time: 6.3284478188, propagate_forward_time: 566.9204711914, propagate_backward_time: 911.4364624023, weight_update_time: 0.5716535449, other_time: 14.4620361328
epoch time: 114.1722793579s, input_fired_time: 1.3265690804, output_fired_time: 0.0000000000, propagate_forward_time: 110.8493118286, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9963989258
validation[          , genome 12086] predictions:    6872/  10000 (68.72%), best:    6917/10000 (69.17%), error:      9062.04883, best error:      8950.75977 on epoch:    76, epoch:   77/100, mu: 0.9806918502, learning_rate: 0.0000032604, weight_decay: 0.0000027583

epoch time: 1533.9659423828s, input_fired_time: 25.0107364655, output_fired_time: 6.4610118866, propagate_forward_time: 571.0596923828, propagate_backward_time: 917.3546142578, weight_update_time: 0.5594816804, other_time: 14.0798339844
epoch time: 124.1014175415s, input_fired_time: 1.3966258764, output_fired_time: 0.0000000000, propagate_forward_time: 120.2400970459, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4646911621
validation[          , genome 12086] predictions:    6910/  10000 (69.10%), best:    6917/10000 (69.17%), error:      8951.91504, best error:      8950.75977 on epoch:    76, epoch:   78/100, mu: 0.9811727405, learning_rate: 0.0000029997, weight_decay: 0.0000025772

epoch time: 1523.2386474609s, input_fired_time: 24.9720478058, output_fired_time: 6.3903622627, propagate_forward_time: 565.7929687500, propagate_backward_time: 911.8358764648, weight_update_time: 0.5643513203, other_time: 14.2473754883
epoch time: 110.6177520752s, input_fired_time: 1.3309332132, output_fired_time: 0.0000000000, propagate_forward_time: 107.3385620117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9482574463
validation[          , genome 12086] predictions:    6912/  10000 (69.12%), best:    6917/10000 (69.17%), error:      8987.35938, best error:      8950.75977 on epoch:    76, epoch:   79/100, mu: 0.9816287756, learning_rate: 0.0000027599, weight_decay: 0.0000024079

epoch time: 1515.1368408203s, input_fired_time: 24.9978981018, output_fired_time: 6.3943443298, propagate_forward_time: 561.6881713867, propagate_backward_time: 908.3049926758, weight_update_time: 0.5500191450, other_time: 13.7514648438
epoch time: 115.0540008545s, input_fired_time: 1.3252022266, output_fired_time: 0.0000000000, propagate_forward_time: 111.6351776123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0936203003
validation[          , genome 12086] predictions:    6911/  10000 (69.11%), best:    6911/10000 (69.11%), error:      8937.38672, best error:      8937.38672 on epoch:    80, epoch:   80/100, mu: 0.9820612669, learning_rate: 0.0000025393, weight_decay: 0.0000022498

epoch time: 1531.2916259766s, input_fired_time: 25.0644817352, output_fired_time: 6.4410853386, propagate_forward_time: 570.0147094727, propagate_backward_time: 915.7706298828, weight_update_time: 0.5634756684, other_time: 14.0007934570
epoch time: 126.9820022583s, input_fired_time: 1.4120755196, output_fired_time: 0.0000000000, propagate_forward_time: 123.0265884399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5433349609
validation[          , genome 12086] predictions:    6925/  10000 (69.25%), best:    6911/10000 (69.11%), error:      8940.86426, best error:      8937.38672 on epoch:    80, epoch:   81/100, mu: 0.9824714065, learning_rate: 0.0000023363, weight_decay: 0.0000021020

epoch time: 1519.6807861328s, input_fired_time: 25.0430641174, output_fired_time: 6.3745470047, propagate_forward_time: 565.8338623047, propagate_backward_time: 908.3990478516, weight_update_time: 0.5597116947, other_time: 14.0302734375
epoch time: 123.9862213135s, input_fired_time: 1.3730609417, output_fired_time: 0.0000000000, propagate_forward_time: 120.3157043457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2974548340
validation[          , genome 12086] predictions:    6914/  10000 (69.14%), best:    6911/10000 (69.11%), error:      8947.03320, best error:      8937.38672 on epoch:    80, epoch:   82/100, mu: 0.9828603864, learning_rate: 0.0000021495, weight_decay: 0.0000019640

epoch time: 1538.5758056641s, input_fired_time: 25.1249904633, output_fired_time: 6.4148449898, propagate_forward_time: 573.4843139648, propagate_backward_time: 919.5076904297, weight_update_time: 0.5585327148, other_time: 14.0440063477
epoch time: 116.6929092407s, input_fired_time: 1.3526389599, output_fired_time: 0.0000000000, propagate_forward_time: 113.2349243164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1053466797
validation[          , genome 12086] predictions:    6928/  10000 (69.28%), best:    6928/10000 (69.28%), error:      8936.65039, best error:      8936.65039 on epoch:    83, epoch:   83/100, mu: 0.9832292199, learning_rate: 0.0000019777, weight_decay: 0.0000018350

epoch time: 1557.6293945312s, input_fired_time: 25.3295936584, output_fired_time: 6.6018934250, propagate_forward_time: 584.6469726562, propagate_backward_time: 926.3336791992, weight_update_time: 0.5763919353, other_time: 14.7172241211
epoch time: 116.5244293213s, input_fired_time: 1.3666177988, output_fired_time: 0.0000000000, propagate_forward_time: 113.0243682861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1334457397
validation[          , genome 12086] predictions:    6916/  10000 (69.16%), best:    6928/10000 (69.28%), error:      8951.04492, best error:      8936.65039 on epoch:    83, epoch:   84/100, mu: 0.9835790396, learning_rate: 0.0000018196, weight_decay: 0.0000017145

epoch time: 1553.4304199219s, input_fired_time: 25.3095436096, output_fired_time: 6.5454831123, propagate_forward_time: 580.2992553711, propagate_backward_time: 926.8477172852, weight_update_time: 0.5665566325, other_time: 14.4283447266
epoch time: 120.7834396362s, input_fired_time: 1.3518592119, output_fired_time: 0.0000000000, propagate_forward_time: 117.3798370361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0517425537
validation[          , genome 12086] predictions:    6923/  10000 (69.23%), best:    6928/10000 (69.28%), error:      8945.63770, best error:      8936.65039 on epoch:    83, epoch:   85/100, mu: 0.9839107394, learning_rate: 0.0000016741, weight_decay: 0.0000016019

epoch time: 1517.8354492188s, input_fired_time: 25.1669940948, output_fired_time: 6.2049021721, propagate_forward_time: 565.9682006836, propagate_backward_time: 906.8562622070, weight_update_time: 0.5500822663, other_time: 13.6390380859
epoch time: 113.6951293945s, input_fired_time: 1.3473272324, output_fired_time: 0.0000000000, propagate_forward_time: 110.3956375122, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9521636963
validation[          , genome 12086] predictions:    6915/  10000 (69.15%), best:    6928/10000 (69.28%), error:      8944.77441, best error:      8936.65039 on epoch:    83, epoch:   86/100, mu: 0.9842253327, learning_rate: 0.0000015403, weight_decay: 0.0000014966

epoch time: 1579.5172119141s, input_fired_time: 25.4332237244, output_fired_time: 6.7152948380, propagate_forward_time: 590.8234863281, propagate_backward_time: 941.9784545898, weight_update_time: 0.5726265907, other_time: 14.5667114258
epoch time: 118.2171020508s, input_fired_time: 1.3880566359, output_fired_time: 0.0000000000, propagate_forward_time: 114.7171401978, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1119079590
validation[          , genome 12086] predictions:    6932/  10000 (69.32%), best:    6932/10000 (69.32%), error:      8924.09863, best error:      8924.09863 on epoch:    87, epoch:   87/100, mu: 0.9845236540, learning_rate: 0.0000014171, weight_decay: 0.0000013983

epoch time: 1558.7862548828s, input_fired_time: 25.3835144043, output_fired_time: 6.6836485863, propagate_forward_time: 580.6722412109, propagate_backward_time: 931.6962890625, weight_update_time: 0.5611643791, other_time: 14.3505859375
epoch time: 114.2124862671s, input_fired_time: 1.3240104914, output_fired_time: 0.0000000000, propagate_forward_time: 111.0124511719, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8760223389
validation[          , genome 12086] predictions:    6924/  10000 (69.24%), best:    6932/10000 (69.32%), error:      8947.87207, best error:      8924.09863 on epoch:    87, epoch:   88/100, mu: 0.9848065972, learning_rate: 0.0000013038, weight_decay: 0.0000013065

epoch time: 1530.6343994141s, input_fired_time: 25.2250671387, output_fired_time: 6.2926707268, propagate_forward_time: 571.4948120117, propagate_backward_time: 914.2918090820, weight_update_time: 0.5440781116, other_time: 13.3299560547
epoch time: 112.6586074829s, input_fired_time: 1.3218210936, output_fired_time: 0.0000000000, propagate_forward_time: 109.4857559204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8510284424
validation[          , genome 12086] predictions:    6938/  10000 (69.38%), best:    6932/10000 (69.32%), error:      8946.99609, best error:      8924.09863 on epoch:    87, epoch:   89/100, mu: 0.9850748777, learning_rate: 0.0000011996, weight_decay: 0.0000012207

epoch time: 1542.6457519531s, input_fired_time: 25.2881965637, output_fired_time: 6.4507756233, propagate_forward_time: 574.8269042969, propagate_backward_time: 922.2390747070, weight_update_time: 0.5565310717, other_time: 13.8407592773
epoch time: 131.0082550049s, input_fired_time: 1.4266322851, output_fired_time: 0.0000000000, propagate_forward_time: 127.0730361938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5085830688
validation[          , genome 12086] predictions:    6930/  10000 (69.30%), best:    6930/10000 (69.30%), error:      8923.96973, best error:      8923.96973 on epoch:    90, epoch:   90/100, mu: 0.9853293300, learning_rate: 0.0000011037, weight_decay: 0.0000011405

epoch time: 1545.9562988281s, input_fired_time: 25.1856250763, output_fired_time: 6.4533519745, propagate_forward_time: 576.1972045898, propagate_backward_time: 924.5918579102, weight_update_time: 0.5494728684, other_time: 13.5281982422
epoch time: 124.8175506592s, input_fired_time: 1.3950977325, output_fired_time: 0.0000000000, propagate_forward_time: 121.1478195190, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2746353149
validation[          , genome 12086] predictions:    6909/  10000 (69.09%), best:    6930/10000 (69.30%), error:      8951.48633, best error:      8923.96973 on epoch:    90, epoch:   91/100, mu: 0.9855706096, learning_rate: 0.0000010155, weight_decay: 0.0000010656

epoch time: 1602.7144775391s, input_fired_time: 25.3908424377, output_fired_time: 6.7854247093, propagate_forward_time: 602.2735595703, propagate_backward_time: 953.5730590820, weight_update_time: 0.5712178349, other_time: 14.6915893555
epoch time: 128.1526489258s, input_fired_time: 1.3718206882, output_fired_time: 0.0000000000, propagate_forward_time: 124.4961166382, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2847137451
validation[          , genome 12086] predictions:    6912/  10000 (69.12%), best:    6930/10000 (69.30%), error:      8956.31836, best error:      8923.96973 on epoch:    90, epoch:   92/100, mu: 0.9857994318, learning_rate: 0.0000009343, weight_decay: 0.0000009956

epoch time: 1590.2526855469s, input_fired_time: 25.3626880646, output_fired_time: 6.7658538818, propagate_forward_time: 598.3646240234, propagate_backward_time: 945.6720581055, weight_update_time: 0.5547270775, other_time: 14.0874633789
epoch time: 121.6116180420s, input_fired_time: 1.3562300205, output_fired_time: 0.0000000000, propagate_forward_time: 118.0950088501, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1603775024
validation[          , genome 12086] predictions:    6932/  10000 (69.32%), best:    6932/10000 (69.32%), error:      8918.97363, best error:      8918.97363 on epoch:    93, epoch:   93/100, mu: 0.9860164523, learning_rate: 0.0000008596, weight_decay: 0.0000009302

epoch time: 1582.9498291016s, input_fired_time: 25.3378219604, output_fired_time: 6.6800813675, propagate_forward_time: 592.5441894531, propagate_backward_time: 944.2911376953, weight_update_time: 0.5635221601, other_time: 14.0966796875
epoch time: 125.7301559448s, input_fired_time: 1.3558527231, output_fired_time: 0.0000000000, propagate_forward_time: 122.1438446045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2304611206
validation[          , genome 12086] predictions:    6919/  10000 (69.19%), best:    6919/10000 (69.19%), error:      8915.59277, best error:      8915.59277 on epoch:    94, epoch:   94/100, mu: 0.9862222672, learning_rate: 0.0000007909, weight_decay: 0.0000008691

epoch time: 1583.5277099609s, input_fired_time: 25.3592700958, output_fired_time: 6.6485819817, propagate_forward_time: 595.9260864258, propagate_backward_time: 941.7167358398, weight_update_time: 0.5616505742, other_time: 13.8770751953
epoch time: 123.0017166138s, input_fired_time: 1.3590320349, output_fired_time: 0.0000000000, propagate_forward_time: 119.4901123047, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1525726318
validation[          , genome 12086] predictions:    6937/  10000 (69.37%), best:    6919/10000 (69.19%), error:      8922.84766, best error:      8915.59277 on epoch:    94, epoch:   95/100, mu: 0.9864174128, learning_rate: 0.0000007276, weight_decay: 0.0000008121

epoch time: 1563.4915771484s, input_fired_time: 25.2569637299, output_fired_time: 6.5379867554, propagate_forward_time: 583.5526123047, propagate_backward_time: 934.3649291992, weight_update_time: 0.5509865880, other_time: 13.7791137695
epoch time: 120.8788986206s, input_fired_time: 1.3519436121, output_fired_time: 0.0000000000, propagate_forward_time: 117.3781127930, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1488418579
validation[          , genome 12086] predictions:    6938/  10000 (69.38%), best:    6938/10000 (69.38%), error:      8914.98047, best error:      8914.98047 on epoch:    96, epoch:   96/100, mu: 0.9866024852, learning_rate: 0.0000006695, weight_decay: 0.0000007587

epoch time: 1586.4163818359s, input_fired_time: 25.2834739685, output_fired_time: 6.6647601128, propagate_forward_time: 594.3013916016, propagate_backward_time: 946.1764526367, weight_update_time: 0.5565854907, other_time: 13.9902954102
epoch time: 123.3458557129s, input_fired_time: 1.3895635605, output_fired_time: 0.0000000000, propagate_forward_time: 119.6887054443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2675857544
validation[          , genome 12086] predictions:    6942/  10000 (69.42%), best:    6942/10000 (69.42%), error:      8908.80957, best error:      8908.80957 on epoch:    97, epoch:   97/100, mu: 0.9867780209, learning_rate: 0.0000006159, weight_decay: 0.0000007089

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_164_12086.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_164_12086_0_r621545404_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.43698
read mu: 0.986944
read mu_delta: 0.948337
read initial_learning_rate: 0.00199354
read learning_rate: 5.66702e-07
read learning_rate_delta: 0.920055
read initial_weight_decay: 0.000515702
read weight_decay: 6.62338e-07
read weight_decay_delta: 0.934322
read batch_size: 79
read epsilon: 1e-07
read alpha: 0.057557
read input_dropout_probability: 0.00147438
read hidden_dropout_probability: 0.0806304
read velocity_reset: 2315
read epoch: 98
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6942
read best_validation_error: 8908.81
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 12086
read normal distribution: '1 0 0'
generator_str: '2048591747'
read generator: 2048591747
read generated_by_map:
1 enable_node 1
reading 132 nodes.
reading 530 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2042.49096679688s, input_fired_time: 30.2073593139648, output_fired_time: 10.5878620147705, propagate_forward_time: 783.76513671875, propagate_backward_time: 1186.16821289062, weight_update_time: 0.915151298046112, other_time: 31.7623291015625
epoch time: 145.987228393555s, input_fired_time: 1.66679561138153, output_fired_time: 0, propagate_forward_time: 140.531311035156, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.78912353515625
validation[          , genome 12086] predictions:    6930/  10000 (69.30%), best:    6930/10000 (69.30%), error:      8903.39453, best error:      8903.39453 on epoch:    98, epoch:   98/100, mu: 0.9869444966, learning_rate: 0.0000005667, weight_decay: 0.0000006623

epoch time: 2112.7871093750s, input_fired_time: 28.9743442535, output_fired_time: 10.4025077820, propagate_forward_time: 823.7573852539, propagate_backward_time: 1224.5036621094, weight_update_time: 0.7677509785, other_time: 25.1491699219
epoch time: 176.0034942627s, input_fired_time: 1.8170392513, output_fired_time: 0.0000000000, propagate_forward_time: 169.8887939453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2976684570
validation[          , genome 12086] predictions:    6938/  10000 (69.38%), best:    6930/10000 (69.30%), error:      8905.03906, best error:      8903.39453 on epoch:    98, epoch:   99/100, mu: 0.9871023297, learning_rate: 0.0000005214, weight_decay: 0.0000006188

epoch time: 2100.5659179688s, input_fired_time: 28.1565437317, output_fired_time: 9.8346614838, propagate_forward_time: 817.5370483398, propagate_backward_time: 1223.4140625000, weight_update_time: 0.7137435079, other_time: 21.6235351562
epoch time: 158.9284210205s, input_fired_time: 1.5453852415, output_fired_time: 0.0000000000, propagate_forward_time: 154.0723266602, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3107147217
validation[          , genome 12086] predictions:    6916/  10000 (69.16%), best:    6930/10000 (69.30%), error:      8911.35645, best error:      8903.39453 on epoch:    98, epoch:  100/100, mu: 0.9872520566, learning_rate: 0.0000004797, weight_decay: 0.0000005782

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 810.8862304688s, input_fired_time: 7.4443788528, output_fired_time: 0.0000000000, propagate_forward_time: 787.3006591797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 16.1411743164
best training[          , genome 12086] predictions:   39163/  50000 (78.33%), best:    6930/10000 (69.30%), error:     33292.24219, best error:      8903.39453 on epoch:    98, epoch:  101/100, mu: 0.9873940349, learning_rate: 0.0000004414, weight_decay: 0.0000005402
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 168.2831268311s, input_fired_time: 1.5147922039, output_fired_time: 0.0000000000, propagate_forward_time: 163.3404693604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4278717041
   testing[          , genome 12086] predictions:    6930/  10000 (69.30%), best:    6930/10000 (69.30%), error:      8903.39453, best error:      8903.39453 on epoch:    98, epoch:  101/100, mu: 0.9873940349, learning_rate: 0.0000004414, weight_decay: 0.0000005402
backpropagation finished successfully!
20:34:33 (4962): called boinc_finish(0)

</stderr_txt>
]]>