Nameexact_genome_1542006908_187_5502_2
Workunit4079382
Created1 Mar 2019, 17:55:10 UTC
Sent1 Mar 2019, 18:01:53 UTC
Report deadline8 Jun 2019, 22:59:33 UTC
Received6 Mar 2019, 5:25:42 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID31505
Run time1 days 17 hours 57 min 5 sec
CPU time1 days 16 hours 29 min 10 sec
Validate stateValid
Credit24,801.68
Device peak FLOPS4.23 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size549.00 MB
Peak swap size546.72 MB
Peak disk usage11.94 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
313843s, input_fired_time: 1.3197981119, output_fired_time: 0.0000000000, propagate_forward_time: 103.5744552612, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9069747925
validation[          , genome  5502] predictions:    6411/  10000 (64.11%), best:    6365/10000 (63.65%), error:     10512.03906, best error:     10501.50293 on epoch:    53, epoch:   55/100, mu: 0.9681369662, learning_rate: 0.0000085252, weight_decay: 0.0000284888

epoch time: 1485.5670166016s, input_fired_time: 18.5566482544, output_fired_time: 9.1973772049, propagate_forward_time: 572.7380981445, propagate_backward_time: 872.4197387695, weight_update_time: 0.6598650217, other_time: 12.6551513672
epoch time: 109.9175033569s, input_fired_time: 1.3670740128, output_fired_time: 0.0000000000, propagate_forward_time: 106.6284408569, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9219894409
validation[          , genome  5502] predictions:    6432/  10000 (64.32%), best:    6432/10000 (64.32%), error:     10427.81348, best error:     10427.81348 on epoch:    56, epoch:   56/100, mu: 0.9693425298, learning_rate: 0.0000079266, weight_decay: 0.0000269671

epoch time: 1374.7419433594s, input_fired_time: 17.1544990540, output_fired_time: 8.7104406357, propagate_forward_time: 522.4860839844, propagate_backward_time: 814.8021850586, weight_update_time: 0.6637750864, other_time: 11.5886840820
epoch time: 113.7374191284s, input_fired_time: 1.3874059916, output_fired_time: 0.0000000000, propagate_forward_time: 110.3192977905, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0307159424
validation[          , genome  5502] predictions:    6384/  10000 (63.84%), best:    6432/10000 (64.32%), error:     10468.66211, best error:     10427.81348 on epoch:    56, epoch:   57/100, mu: 0.9704816341, learning_rate: 0.0000073700, weight_decay: 0.0000255268

epoch time: 1382.1944580078s, input_fired_time: 17.2686901093, output_fired_time: 8.7605428696, propagate_forward_time: 524.7689208984, propagate_backward_time: 819.4719848633, weight_update_time: 0.6455275416, other_time: 11.9243774414
epoch time: 116.9987106323s, input_fired_time: 1.4613329172, output_fired_time: 0.0000000000, propagate_forward_time: 113.4174804688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1198959351
validation[          , genome  5502] predictions:    6372/  10000 (63.72%), best:    6432/10000 (64.32%), error:     10555.12695, best error:     10427.81348 on epoch:    56, epoch:   58/100, mu: 0.9715579152, learning_rate: 0.0000068525, weight_decay: 0.0000241634

epoch time: 1353.8273925781s, input_fired_time: 16.9379310608, output_fired_time: 8.4940996170, propagate_forward_time: 517.4934082031, propagate_backward_time: 799.6715698242, weight_update_time: 0.6404700875, other_time: 11.2302856445
epoch time: 118.3063278198s, input_fired_time: 1.3344866037, output_fired_time: 0.0000000000, propagate_forward_time: 115.0481109619, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9237289429
validation[          , genome  5502] predictions:    6403/  10000 (64.03%), best:    6432/10000 (64.32%), error:     10455.57422, best error:     10427.81348 on epoch:    56, epoch:   59/100, mu: 0.9725748301, learning_rate: 0.0000063714, weight_decay: 0.0000228728

epoch time: 1348.4012451172s, input_fired_time: 16.8216609955, output_fired_time: 8.5761394501, propagate_forward_time: 511.9172973633, propagate_backward_time: 799.8464965820, weight_update_time: 0.6324527860, other_time: 11.2396240234
epoch time: 106.4480285645s, input_fired_time: 1.3105357885, output_fired_time: 0.0000000000, propagate_forward_time: 103.2574234009, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8800659180
validation[          , genome  5502] predictions:    6415/  10000 (64.15%), best:    6432/10000 (64.32%), error:     10520.35156, best error:     10427.81348 on epoch:    56, epoch:   60/100, mu: 0.9735357165, learning_rate: 0.0000059240, weight_decay: 0.0000216511

epoch time: 1427.0255126953s, input_fired_time: 17.6494693756, output_fired_time: 8.9431114197, propagate_forward_time: 550.6749877930, propagate_backward_time: 837.5219116211, weight_update_time: 0.8040590882, other_time: 12.2360839844
epoch time: 114.3818435669s, input_fired_time: 1.4250749350, output_fired_time: 0.0000000000, propagate_forward_time: 110.8678131104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0889587402
validation[          , genome  5502] predictions:    6430/  10000 (64.30%), best:    6430/10000 (64.30%), error:     10371.63086, best error:     10371.63086 on epoch:    61, epoch:   61/100, mu: 0.9744436145, learning_rate: 0.0000055081, weight_decay: 0.0000204947

epoch time: 1341.2612304688s, input_fired_time: 16.7204761505, output_fired_time: 8.5223159790, propagate_forward_time: 510.0679321289, propagate_backward_time: 794.7828369141, weight_update_time: 0.6262716055, other_time: 11.1676635742
epoch time: 112.0067291260s, input_fired_time: 1.4027723074, output_fired_time: 0.0000000000, propagate_forward_time: 108.6048660278, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9990921021
validation[          , genome  5502] predictions:    6436/  10000 (64.36%), best:    6430/10000 (64.30%), error:     10379.71777, best error:     10371.63086 on epoch:    61, epoch:   62/100, mu: 0.9753014445, learning_rate: 0.0000051213, weight_decay: 0.0000194001

epoch time: 1371.4295654297s, input_fired_time: 17.1048374176, output_fired_time: 8.5907239914, propagate_forward_time: 521.4799804688, propagate_backward_time: 812.5507202148, weight_update_time: 0.6542302370, other_time: 11.7033081055
epoch time: 107.4039077759s, input_fired_time: 1.3347822428, output_fired_time: 0.0000000000, propagate_forward_time: 104.1501693726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9189529419
validation[          , genome  5502] predictions:    6402/  10000 (64.02%), best:    6430/10000 (64.30%), error:     10485.09082, best error:     10371.63086 on epoch:    61, epoch:   63/100, mu: 0.9761119485, learning_rate: 0.0000047617, weight_decay: 0.0000183639

epoch time: 1357.4006347656s, input_fired_time: 16.9523830414, output_fired_time: 8.6189861298, propagate_forward_time: 515.2233886719, propagate_backward_time: 805.1764526367, weight_update_time: 0.6351752281, other_time: 11.4293823242
epoch time: 106.1245269775s, input_fired_time: 1.3112270832, output_fired_time: 0.0000000000, propagate_forward_time: 102.9197616577, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8935394287
validation[          , genome  5502] predictions:    6408/  10000 (64.08%), best:    6430/10000 (64.30%), error:     10521.67188, best error:     10371.63086 on epoch:    61, epoch:   64/100, mu: 0.9768777490, learning_rate: 0.0000044274, weight_decay: 0.0000173830

epoch time: 1354.0507812500s, input_fired_time: 16.9596881866, output_fired_time: 8.6403837204, propagate_forward_time: 514.3151245117, propagate_backward_time: 802.7575073242, weight_update_time: 0.6264402866, other_time: 11.3780517578
epoch time: 112.2641906738s, input_fired_time: 1.4056800604, output_fired_time: 0.0000000000, propagate_forward_time: 108.8307952881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0277175903
validation[          , genome  5502] predictions:    6407/  10000 (64.07%), best:    6430/10000 (64.30%), error:     10534.51367, best error:     10371.63086 on epoch:    61, epoch:   65/100, mu: 0.9776013494, learning_rate: 0.0000041165, weight_decay: 0.0000164546

epoch time: 1360.5626220703s, input_fired_time: 17.0571575165, output_fired_time: 8.6294002533, propagate_forward_time: 516.9140625000, propagate_backward_time: 806.4838867188, weight_update_time: 0.6341913939, other_time: 11.4781494141
epoch time: 113.8316040039s, input_fired_time: 1.3933140039, output_fired_time: 0.0000000000, propagate_forward_time: 110.3723983765, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0658950806
validation[          , genome  5502] predictions:    6360/  10000 (63.60%), best:    6430/10000 (64.30%), error:     10566.60938, best error:     10371.63086 on epoch:    61, epoch:   66/100, mu: 0.9782850146, learning_rate: 0.0000038275, weight_decay: 0.0000155757

epoch time: 1346.4758300781s, input_fired_time: 16.8641414642, output_fired_time: 8.5544996262, propagate_forward_time: 511.3902587891, propagate_backward_time: 798.2354736328, weight_update_time: 0.6344515681, other_time: 11.4315185547
epoch time: 112.5000152588s, input_fired_time: 1.4050189257, output_fired_time: 0.0000000000, propagate_forward_time: 109.0679702759, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0270233154
validation[          , genome  5502] predictions:    6366/  10000 (63.66%), best:    6430/10000 (64.30%), error:     10570.34082, best error:     10371.63086 on epoch:    61, epoch:   67/100, mu: 0.9789310098, learning_rate: 0.0000035587, weight_decay: 0.0000147438

epoch time: 1344.1848144531s, input_fired_time: 16.7566070557, output_fired_time: 8.4623193741, propagate_forward_time: 516.3031616211, propagate_backward_time: 791.4781494141, weight_update_time: 0.6231369376, other_time: 11.1846313477
epoch time: 105.7460327148s, input_fired_time: 1.2977565527, output_fired_time: 0.0000000000, propagate_forward_time: 102.5562133789, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8920593262
validation[          , genome  5502] predictions:    6442/  10000 (64.42%), best:    6430/10000 (64.30%), error:     10421.56250, best error:     10371.63086 on epoch:    61, epoch:   68/100, mu: 0.9795413613, learning_rate: 0.0000033089, weight_decay: 0.0000139563

epoch time: 1359.3571777344s, input_fired_time: 17.0663051605, output_fired_time: 8.6597270966, propagate_forward_time: 517.3973999023, propagate_backward_time: 804.7896118164, weight_update_time: 0.6470183134, other_time: 11.4442138672
epoch time: 109.7348098755s, input_fired_time: 1.3640964031, output_fired_time: 0.0000000000, propagate_forward_time: 106.3905792236, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9801330566
validation[          , genome  5502] predictions:    6449/  10000 (64.49%), best:    6449/10000 (64.49%), error:     10346.15137, best error:     10346.15137 on epoch:    69, epoch:   69/100, mu: 0.9801180959, learning_rate: 0.0000030765, weight_decay: 0.0000132109

epoch time: 1364.2208251953s, input_fired_time: 17.0634574890, output_fired_time: 8.6410007477, propagate_forward_time: 518.3786010742, propagate_backward_time: 808.5177001953, weight_update_time: 0.6663038731, other_time: 11.6200561523
epoch time: 105.7534255981s, input_fired_time: 1.3232628107, output_fired_time: 0.0000000000, propagate_forward_time: 102.5488204956, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8813400269
validation[          , genome  5502] predictions:    6433/  10000 (64.33%), best:    6449/10000 (64.49%), error:     10428.49609, best error:     10346.15137 on epoch:    69, epoch:   70/100, mu: 0.9806630015, learning_rate: 0.0000028605, weight_decay: 0.0000125053

epoch time: 1354.4968261719s, input_fired_time: 17.1305179596, output_fired_time: 8.5214405060, propagate_forward_time: 515.6428222656, propagate_backward_time: 801.9562988281, weight_update_time: 0.6263351440, other_time: 11.2457275391
epoch time: 104.6725769043s, input_fired_time: 1.3021656275, output_fired_time: 0.0000000000, propagate_forward_time: 101.4842376709, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8861770630
validation[          , genome  5502] predictions:    6440/  10000 (64.40%), best:    6449/10000 (64.49%), error:     10356.44043, best error:     10346.15137 on epoch:    69, epoch:   71/100, mu: 0.9811778665, learning_rate: 0.0000026597, weight_decay: 0.0000118374

epoch time: 1350.4365234375s, input_fired_time: 16.9214401245, output_fired_time: 8.5438833237, propagate_forward_time: 513.4815673828, propagate_backward_time: 799.9318847656, weight_update_time: 0.6529719830, other_time: 11.5578613281
epoch time: 106.2900390625s, input_fired_time: 1.3238410950, output_fired_time: 0.0000000000, propagate_forward_time: 103.0921554565, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8740463257
validation[          , genome  5502] predictions:    6414/  10000 (64.14%), best:    6449/10000 (64.49%), error:     10485.13086, best error:     10346.15137 on epoch:    69, epoch:   72/100, mu: 0.9816643596, learning_rate: 0.0000024729, weight_decay: 0.0000112051

epoch time: 1359.8135986328s, input_fired_time: 16.9810485840, output_fired_time: 8.5799636841, propagate_forward_time: 516.0172119141, propagate_backward_time: 806.8493652344, weight_update_time: 0.6431055069, other_time: 11.3859863281
epoch time: 105.7809295654s, input_fired_time: 1.3080946207, output_fired_time: 0.0000000000, propagate_forward_time: 102.5627899170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9100418091
validation[          , genome  5502] predictions:    6438/  10000 (64.38%), best:    6449/10000 (64.49%), error:     10401.56055, best error:     10346.15137 on epoch:    69, epoch:   73/100, mu: 0.9821240306, learning_rate: 0.0000022993, weight_decay: 0.0000106066

epoch time: 1358.0041503906s, input_fired_time: 16.9529056549, output_fired_time: 8.6580219269, propagate_forward_time: 513.9897460938, propagate_backward_time: 806.9744873047, weight_update_time: 0.6350203156, other_time: 11.4289550781
epoch time: 107.8009262085s, input_fired_time: 1.3490260839, output_fired_time: 0.0000000000, propagate_forward_time: 104.5539932251, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8979034424
validation[          , genome  5502] predictions:    6441/  10000 (64.41%), best:    6449/10000 (64.49%), error:     10397.73340, best error:     10346.15137 on epoch:    69, epoch:   74/100, mu: 0.9825583100, learning_rate: 0.0000021378, weight_decay: 0.0000100401

epoch time: 1354.9251708984s, input_fired_time: 16.9908237457, output_fired_time: 8.5886402130, propagate_forward_time: 515.4965209961, propagate_backward_time: 802.5622558594, weight_update_time: 0.6374043226, other_time: 11.2869262695
epoch time: 106.4608001709s, input_fired_time: 1.3291305304, output_fired_time: 0.0000000000, propagate_forward_time: 103.2375335693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8941345215
validation[          , genome  5502] predictions:    6457/  10000 (64.57%), best:    6449/10000 (64.49%), error:     10363.99609, best error:     10346.15137 on epoch:    69, epoch:   75/100, mu: 0.9829686880, learning_rate: 0.0000019877, weight_decay: 0.0000095039

epoch time: 1328.0789794922s, input_fired_time: 16.7094306946, output_fired_time: 8.4556846619, propagate_forward_time: 505.3020019531, propagate_backward_time: 786.4871215820, weight_update_time: 0.6238192916, other_time: 11.1246948242
epoch time: 106.5683364868s, input_fired_time: 1.3265054226, output_fired_time: 0.0000000000, propagate_forward_time: 103.3403320313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9014968872
validation[          , genome  5502] predictions:    6411/  10000 (64.11%), best:    6449/10000 (64.49%), error:     10455.19238, best error:     10346.15137 on epoch:    69, epoch:   76/100, mu: 0.9833564162, learning_rate: 0.0000018482, weight_decay: 0.0000089962

epoch time: 1338.9201660156s, input_fired_time: 16.8020076752, output_fired_time: 8.4075260162, propagate_forward_time: 509.1302490234, propagate_backward_time: 793.4638061523, weight_update_time: 0.6193132997, other_time: 11.1166381836
epoch time: 107.9014053345s, input_fired_time: 1.3343740702, output_fired_time: 0.0000000000, propagate_forward_time: 104.7643661499, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8026657104
validation[          , genome  5502] predictions:    6489/  10000 (64.89%), best:    6449/10000 (64.49%), error:     10380.30469, best error:     10346.15137 on epoch:    69, epoch:   77/100, mu: 0.9837227464, learning_rate: 0.0000017184, weight_decay: 0.0000085157

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.984069
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 1.59775e-06
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 8.06091e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 78
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6449
read best_validation_error: 10346.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '225424116'
read generator: 225424116
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1354.9033203125s, input_fired_time: 16.7773342132568, output_fired_time: 8.41814136505127, propagate_forward_time: 517.150939941406, propagate_backward_time: 802.008544921875, weight_update_time: 0.620193123817444, other_time: 10.5484008789063
epoch time: 110.254486083984s, input_fired_time: 1.34096598625183, output_fired_time: 0, propagate_forward_time: 107.083206176758, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.83031463623047
validation[          , genome  5502] predictions:    6477/  10000 (64.77%), best:    6449/10000 (64.49%), error:     10354.91211, best error:     10346.15137 on epoch:    69, epoch:   78/100, mu: 0.9840688705, learning_rate: 0.0000015977, weight_decay: 0.0000080609

epoch time: 1367.5559082031s, input_fired_time: 16.8147354126, output_fired_time: 8.4524898529, propagate_forward_time: 522.3041381836, propagate_backward_time: 809.0397949219, weight_update_time: 0.6248465776, other_time: 10.9447631836
epoch time: 106.8273162842s, input_fired_time: 1.3057560921, output_fired_time: 0.0000000000, propagate_forward_time: 103.7815475464, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7400131226
validation[          , genome  5502] predictions:    6437/  10000 (64.37%), best:    6449/10000 (64.49%), error:     10382.28906, best error:     10346.15137 on epoch:    69, epoch:   79/100, mu: 0.9843959212, learning_rate: 0.0000014856, weight_decay: 0.0000076304

epoch time: 1356.5245361328s, input_fired_time: 16.7451992035, output_fired_time: 8.4546880722, propagate_forward_time: 516.1172485352, propagate_backward_time: 804.2110595703, weight_update_time: 0.6257776022, other_time: 10.9962768555
epoch time: 105.7328109741s, input_fired_time: 1.3423960209, output_fired_time: 0.0000000000, propagate_forward_time: 102.5124053955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8780059814
validation[          , genome  5502] predictions:    6481/  10000 (64.81%), best:    6449/10000 (64.49%), error:     10361.41895, best error:     10346.15137 on epoch:    69, epoch:   80/100, mu: 0.9847049713, learning_rate: 0.0000013813, weight_decay: 0.0000072228

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.984997
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 1.28427e-06
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 6.83704e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 81
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6449
read best_validation_error: 10346.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '837913275'
read generator: 837913275
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1361.72192382813s, input_fired_time: 17.1537857055664, output_fired_time: 8.66928005218506, propagate_forward_time: 516.618774414063, propagate_backward_time: 807.782165527344, weight_update_time: 0.648035526275635, other_time: 11.4978637695313
epoch time: 103.761108398438s, input_fired_time: 1.29301917552948, output_fired_time: 0, propagate_forward_time: 100.634155273438, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.83393096923828
validation[          , genome  5502] predictions:    6485/  10000 (64.85%), best:    6449/10000 (64.49%), error:     10394.99023, best error:     10346.15137 on epoch:    69, epoch:   81/100, mu: 0.9849969745, learning_rate: 0.0000012843, weight_decay: 0.0000068370

epoch time: 1351.5242919922s, input_fired_time: 16.8065166473, output_fired_time: 8.5341262817, propagate_forward_time: 514.7815551758, propagate_backward_time: 800.3190307617, weight_update_time: 0.6276304722, other_time: 11.0830078125
epoch time: 107.1755371094s, input_fired_time: 1.3285292387, output_fired_time: 0.0000000000, propagate_forward_time: 103.9669113159, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8800964355
validation[          , genome  5502] predictions:    6438/  10000 (64.38%), best:    6449/10000 (64.49%), error:     10397.12207, best error:     10346.15137 on epoch:    69, epoch:   82/100, mu: 0.9852728248, learning_rate: 0.0000011941, weight_decay: 0.0000064719

epoch time: 1336.4396972656s, input_fired_time: 16.7330131531, output_fired_time: 8.4138908386, propagate_forward_time: 508.1840209961, propagate_backward_time: 792.0201416016, weight_update_time: 0.6269502640, other_time: 11.0885620117
epoch time: 105.6737365723s, input_fired_time: 1.3308854103, output_fired_time: 0.0000000000, propagate_forward_time: 102.4816513062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8611984253
validation[          , genome  5502] predictions:    6487/  10000 (64.87%), best:    6449/10000 (64.49%), error:     10357.96094, best error:     10346.15137 on epoch:    69, epoch:   83/100, mu: 0.9855334759, learning_rate: 0.0000011103, weight_decay: 0.0000061262

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.98578
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 1.0323e-06
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 5.79898e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 84
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6449
read best_validation_error: 10346.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '834135374'
read generator: 834135374
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1368.80725097656s, input_fired_time: 16.8210601806641, output_fired_time: 8.43808746337891, propagate_forward_time: 522.298889160156, propagate_backward_time: 810.605773925781, weight_update_time: 0.624171912670135, other_time: 10.6434326171875
epoch time: 108.132202148438s, input_fired_time: 1.34289181232452, output_fired_time: 0, propagate_forward_time: 105.021934509277, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.76737213134766
validation[          , genome  5502] predictions:    6489/  10000 (64.89%), best:    6489/10000 (64.89%), error:     10305.67676, best error:     10305.67676 on epoch:    84, epoch:   84/100, mu: 0.9857797623, learning_rate: 0.0000010323, weight_decay: 0.0000057990

epoch time: 1355.7264404297s, input_fired_time: 16.5851268768, output_fired_time: 8.4205713272, propagate_forward_time: 516.1920166016, propagate_backward_time: 803.7240600586, weight_update_time: 0.6265048981, other_time: 10.8047485352
epoch time: 104.9837951660s, input_fired_time: 1.3087654114, output_fired_time: 0.0000000000, propagate_forward_time: 101.9394454956, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7355880737
validation[          , genome  5502] predictions:    6488/  10000 (64.88%), best:    6488/10000 (64.88%), error:     10301.06348, best error:     10301.06348 on epoch:    85, epoch:   85/100, mu: 0.9860124588, learning_rate: 0.0000009598, weight_decay: 0.0000054893

epoch time: 1361.7366943359s, input_fired_time: 16.8766002655, output_fired_time: 8.5443944931, propagate_forward_time: 518.3676147461, propagate_backward_time: 807.0758666992, weight_update_time: 0.6338147521, other_time: 10.8721923828
epoch time: 106.3679428101s, input_fired_time: 1.3107506037, output_fired_time: 0.0000000000, propagate_forward_time: 103.2577667236, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7994232178
validation[          , genome  5502] predictions:    6504/  10000 (65.04%), best:    6504/10000 (65.04%), error:     10280.82910, best error:     10280.82910 on epoch:    86, epoch:   86/100, mu: 0.9862323403, learning_rate: 0.0000008924, weight_decay: 0.0000051961

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.98644
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 8.29763e-07
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 4.91853e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 87
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 6504
read best_validation_error: 10280.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '1129256212'
read generator: 1129256212
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1371.12426757813s, input_fired_time: 16.851037979126, output_fired_time: 8.53313064575195, propagate_forward_time: 520.977783203125, propagate_backward_time: 813.961181640625, weight_update_time: 0.624032557010651, other_time: 10.8011474609375
epoch time: 109.254547119141s, input_fired_time: 1.34894275665283, output_fired_time: 0, propagate_forward_time: 106.11669921875, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.78890228271484
validation[          , genome  5502] predictions:    6487/  10000 (64.87%), best:    6504/10000 (65.04%), error:     10327.55469, best error:     10280.82910 on epoch:    86, epoch:   87/100, mu: 0.9864401221, learning_rate: 0.0000008298, weight_decay: 0.0000049185

epoch time: 1347.6427001953s, input_fired_time: 16.7703170776, output_fired_time: 8.5511798859, propagate_forward_time: 512.4083862305, propagate_backward_time: 798.8850097656, weight_update_time: 0.6498306394, other_time: 11.0278930664
epoch time: 107.0434417725s, input_fired_time: 1.3498409986, output_fired_time: 0.0000000000, propagate_forward_time: 103.7390975952, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9545059204
validation[          , genome  5502] predictions:    6458/  10000 (64.58%), best:    6504/10000 (65.04%), error:     10323.28809, best error:     10280.82910 on epoch:    86, epoch:   88/100, mu: 0.9866364002, learning_rate: 0.0000007715, weight_decay: 0.0000046558

epoch time: 1388.0302734375s, input_fired_time: 17.1182518005, output_fired_time: 8.6676054001, propagate_forward_time: 521.3385620117, propagate_backward_time: 829.6954345703, weight_update_time: 0.6383442283, other_time: 11.2103881836
epoch time: 110.8282775879s, input_fired_time: 1.3409590721, output_fired_time: 0.0000000000, propagate_forward_time: 107.7008819580, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7864379883
validation[          , genome  5502] predictions:    6505/  10000 (65.05%), best:    6504/10000 (65.04%), error:     10297.38086, best error:     10280.82910 on epoch:    86, epoch:   89/100, mu: 0.9868218899, learning_rate: 0.0000007173, weight_decay: 0.0000044072

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.986997
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 6.66964e-07
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 4.17176e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 90
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 6504
read best_validation_error: 10280.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '1888337452'
read generator: 1888337452
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1354.49011230469s, input_fired_time: 16.7898998260498, output_fired_time: 8.39380741119385, propagate_forward_time: 515.692199707031, propagate_backward_time: 802.9619140625, weight_update_time: 0.624939262866974, other_time: 10.6522827148438
epoch time: 109.855010986328s, input_fired_time: 1.37135112285614, output_fired_time: 0, propagate_forward_time: 106.557563781738, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.92609405517578
validation[          , genome  5502] predictions:    6482/  10000 (64.82%), best:    6504/10000 (65.04%), error:     10313.55273, best error:     10280.82910 on epoch:    86, epoch:   90/100, mu: 0.9869971275, learning_rate: 0.0000006670, weight_decay: 0.0000041718

epoch time: 1346.7514648438s, input_fired_time: 16.7395057678, output_fired_time: 8.4992532730, propagate_forward_time: 512.5772705078, propagate_backward_time: 798.1778564453, weight_update_time: 0.6147767305, other_time: 10.7575683594
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_187_5502_2_r506726020_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.495048
read mu: 0.987163
read mu_delta: 0.944857
read initial_learning_rate: 0.000467354
read learning_rate: 6.20134e-07
read learning_rate_delta: 0.929786
read initial_weight_decay: 0.000583179
read weight_decay: 3.94894e-06
read weight_decay_delta: 0.946589
read batch_size: 94
read epsilon: 1e-07
read alpha: 0.0791366
read input_dropout_probability: 0.00119598
read hidden_dropout_probability: 0.0974674
read velocity_reset: 1749
read epoch: 91
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 6504
read best_validation_error: 10280.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5502
read normal distribution: '1 0 0'
generator_str: '1015099714'
read generator: 1015099714
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 394 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1462.05236816406s, input_fired_time: 18.0993957519531, output_fired_time: 9.15128040313721, propagate_forward_time: 555.628295898438, propagate_backward_time: 867.557312011719, weight_update_time: 0.683004021644592, other_time: 11.6161499023438
epoch time: 113.087593078613s, input_fired_time: 1.40558457374573, output_fired_time: 0, propagate_forward_time: 109.77717590332, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.90483093261719
validation[          , genome  5502] predictions:    6493/  10000 (64.93%), best:    6504/10000 (65.04%), error:     10310.23047, best error:     10280.82910 on epoch:    86, epoch:   91/100, mu: 0.9871627092, learning_rate: 0.0000006201, weight_decay: 0.0000039489

epoch time: 1453.2695312500s, input_fired_time: 18.0910530090, output_fired_time: 9.0960493088, propagate_forward_time: 553.5490722656, propagate_backward_time: 860.9370727539, weight_update_time: 0.6579565406, other_time: 11.5962524414
epoch time: 114.2472076416s, input_fired_time: 1.4026688337, output_fired_time: 0.0000000000, propagate_forward_time: 110.9307632446, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9137725830
validation[          , genome  5502] predictions:    6448/  10000 (64.48%), best:    6504/10000 (65.04%), error:     10314.37598, best error:     10280.82910 on epoch:    86, epoch:   92/100, mu: 0.9873191714, learning_rate: 0.0000005766, weight_decay: 0.0000037380

epoch time: 1455.4044189453s, input_fired_time: 18.3461017609, output_fired_time: 9.0841732025, propagate_forward_time: 551.1485595703, propagate_backward_time: 865.2122802734, weight_update_time: 0.6652358174, other_time: 11.6132812500
epoch time: 119.0937423706s, input_fired_time: 1.5911222696, output_fired_time: 0.0000000000, propagate_forward_time: 115.4686050415, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0340118408
validation[          , genome  5502] predictions:    6493/  10000 (64.93%), best:    6504/10000 (65.04%), error:     10305.60742, best error:     10280.82910 on epoch:    86, epoch:   93/100, mu: 0.9874669909, learning_rate: 0.0000005361, weight_decay: 0.0000035384

epoch time: 1480.8310546875s, input_fired_time: 18.7116317749, output_fired_time: 9.3342056274, propagate_forward_time: 565.0076293945, propagate_backward_time: 875.6462402344, weight_update_time: 0.8180584908, other_time: 12.1312866211
epoch time: 119.3154678345s, input_fired_time: 1.5101908445, output_fired_time: 0.0000000000, propagate_forward_time: 115.7620849609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0431900024
validation[          , genome  5502] predictions:    6501/  10000 (65.01%), best:    6504/10000 (65.04%), error:     10316.90234, best error:     10280.82910 on epoch:    86, epoch:   94/100, mu: 0.9876066446, learning_rate: 0.0000004985, weight_decay: 0.0000033494

epoch time: 1552.0468750000s, input_fired_time: 19.1653537750, output_fired_time: 9.3861846924, propagate_forward_time: 587.7448120117, propagate_backward_time: 923.2035522461, weight_update_time: 0.7270720005, other_time: 12.5468750000
epoch time: 126.8518142700s, input_fired_time: 1.6511543989, output_fired_time: 0.0000000000, propagate_forward_time: 122.8803405762, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3203201294
validation[          , genome  5502] predictions:    6506/  10000 (65.06%), best:    6504/10000 (65.04%), error:     10291.14258, best error:     10280.82910 on epoch:    86, epoch:   95/100, mu: 0.9877386093, learning_rate: 0.0000004635, weight_decay: 0.0000031705

epoch time: 1750.8867187500s, input_fired_time: 22.5972690582, output_fired_time: 11.5846366882, propagate_forward_time: 657.3212890625, propagate_backward_time: 1044.4179687500, weight_update_time: 0.9224051833, other_time: 14.9655761719
epoch time: 113.3646469116s, input_fired_time: 1.4015315771, output_fired_time: 0.0000000000, propagate_forward_time: 110.0069732666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9561386108
validation[          , genome  5502] predictions:    6499/  10000 (64.99%), best:    6504/10000 (65.04%), error:     10313.33496, best error:     10280.82910 on epoch:    86, epoch:   96/100, mu: 0.9878633022, learning_rate: 0.0000004309, weight_decay: 0.0000030011

epoch time: 1549.2332763672s, input_fired_time: 19.5091094971, output_fired_time: 9.7723999023, propagate_forward_time: 587.2003784180, propagate_backward_time: 920.1005859375, weight_update_time: 0.6858349442, other_time: 12.6506958008
epoch time: 110.3226623535s, input_fired_time: 1.3812124729, output_fired_time: 0.0000000000, propagate_forward_time: 107.0151138306, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9263381958
validation[          , genome  5502] predictions:    6499/  10000 (64.99%), best:    6499/10000 (64.99%), error:     10279.49219, best error:     10279.49219 on epoch:    97, epoch:   97/100, mu: 0.9879811406, learning_rate: 0.0000004007, weight_decay: 0.0000028409

epoch time: 1528.6120605469s, input_fired_time: 18.8556270599, output_fired_time: 9.6609792709, propagate_forward_time: 580.2174072266, propagate_backward_time: 907.3453979492, weight_update_time: 0.6792134047, other_time: 12.5326538086
epoch time: 115.8143844604s, input_fired_time: 1.3916867971, output_fired_time: 0.0000000000, propagate_forward_time: 112.4112930298, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0114059448
validation[          , genome  5502] predictions:    6495/  10000 (64.95%), best:    6499/10000 (64.99%), error:     10290.53906, best error:     10279.49219 on epoch:    97, epoch:   98/100, mu: 0.9880924821, learning_rate: 0.0000003725, weight_decay: 0.0000026891

epoch time: 1547.2325439453s, input_fired_time: 19.4092655182, output_fired_time: 9.7502670288, propagate_forward_time: 588.7003784180, propagate_backward_time: 916.5065917969, weight_update_time: 0.6951266527, other_time: 12.8660278320
epoch time: 131.9829711914s, input_fired_time: 1.6525259018, output_fired_time: 0.0000000000, propagate_forward_time: 127.9216232300, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4088211060
validation[          , genome  5502] predictions:    6494/  10000 (64.94%), best:    6499/10000 (64.99%), error:     10319.49023, best error:     10279.49219 on epoch:    97, epoch:   99/100, mu: 0.9881976843, learning_rate: 0.0000003464, weight_decay: 0.0000025455

epoch time: 1455.0882568359s, input_fired_time: 18.4668121338, output_fired_time: 9.3406724930, propagate_forward_time: 554.1871337891, propagate_backward_time: 860.4877319336, weight_update_time: 0.6795508862, other_time: 12.6058959961
epoch time: 118.2294769287s, input_fired_time: 1.4918090105, output_fired_time: 0.0000000000, propagate_forward_time: 114.5804595947, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1572113037
validation[          , genome  5502] predictions:    6506/  10000 (65.06%), best:    6499/10000 (64.99%), error:     10283.19141, best error:     10279.49219 on epoch:    97, epoch:  100/100, mu: 0.9882970452, learning_rate: 0.0000003221, weight_decay: 0.0000024095

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 571.1000976563s, input_fired_time: 7.1567678452, output_fired_time: 0.0000000000, propagate_forward_time: 553.4175415039, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.5258178711
best training[          , genome  5502] predictions:   34248/  50000 (68.50%), best:    6499/10000 (64.99%), error:     47850.04297, best error:     10279.49219 on epoch:    97, epoch:  101/100, mu: 0.9883909225, learning_rate: 0.0000002994, weight_decay: 0.0000022808
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 124.4496459961s, input_fired_time: 1.5518784523, output_fired_time: 0.0000000000, propagate_forward_time: 120.5991287231, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2986373901
   testing[          , genome  5502] predictions:    6499/  10000 (64.99%), best:    6499/10000 (64.99%), error:     10279.49219, best error:     10279.49219 on epoch:    97, epoch:  101/100, mu: 0.9883909225, learning_rate: 0.0000002994, weight_decay: 0.0000022808
backpropagation finished successfully!
14:24:24 (11528): called boinc_finish(0)

</stderr_txt>
]]>