Nameexact_genome_1542605141_188_6288_1
Workunit4098213
Created19 Nov 2018, 5:58:00 UTC
Sent19 Nov 2018, 5:58:32 UTC
Report deadline8 Feb 2019, 14:14:27 UTC
Received25 Nov 2018, 21:27:34 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID30770
Run time1 days 2 hours 27 min 4 sec
CPU time1 days 2 hours 18 min 8 sec
Validate stateValid
Credit20,336.10
Device peak FLOPS4.85 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size536.07 MB
Peak swap size557.34 MB
Peak disk usage10.60 MB

Stderr output

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<stderr_txt>
time: 335.8226013184, propagate_backward_time: 527.3823852539, weight_update_time: 0.3360462487, other_time: 10.1714477539
epoch time: 69.0145263672s, input_fired_time: 0.8825406432, output_fired_time: 0.0000000000, propagate_forward_time: 66.4334411621, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6985473633
validation[          , genome  6288] predictions:    6224/  10000 (62.24%), best:    6281/10000 (62.81%), error:     10727.13867, best error:     10637.85449 on epoch:    38, epoch:   40/100, mu: 0.9432417750, learning_rate: 0.0000612973, weight_decay: 0.0000453898

epoch time: 889.4017333984s, input_fired_time: 12.7709350586, output_fired_time: 5.3054885864, propagate_forward_time: 334.9023437500, propagate_backward_time: 526.2287597656, weight_update_time: 0.3399922550, other_time: 10.1942138672
epoch time: 69.4544143677s, input_fired_time: 0.8898664117, output_fired_time: 0.0000000000, propagate_forward_time: 66.8858032227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6787414551
validation[          , genome  6288] predictions:    6291/  10000 (62.91%), best:    6281/10000 (62.81%), error:     10681.12793, best error:     10637.85449 on epoch:    38, epoch:   41/100, mu: 0.9459148049, learning_rate: 0.0000570915, weight_decay: 0.0000427678

epoch time: 890.8731689453s, input_fired_time: 12.7844839096, output_fired_time: 5.3064842224, propagate_forward_time: 335.2532958984, propagate_backward_time: 527.3437500000, weight_update_time: 0.3333440721, other_time: 10.1851806641
epoch time: 69.8387298584s, input_fired_time: 0.8897098303, output_fired_time: 0.0000000000, propagate_forward_time: 67.2040405273, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7449798584
validation[          , genome  6288] predictions:    6324/  10000 (63.24%), best:    6324/10000 (63.24%), error:     10609.83887, best error:     10609.83887 on epoch:    42, epoch:   42/100, mu: 0.9484350681, learning_rate: 0.0000531743, weight_decay: 0.0000402974

epoch time: 890.8526000977s, input_fired_time: 12.7822303772, output_fired_time: 5.3328137398, propagate_forward_time: 335.7308654785, propagate_backward_time: 526.7850341797, weight_update_time: 0.3334188759, other_time: 10.2216796875
epoch time: 69.4398574829s, input_fired_time: 0.8924462199, output_fired_time: 0.0000000000, propagate_forward_time: 66.8343734741, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7130355835
validation[          , genome  6288] predictions:    6252/  10000 (62.52%), best:    6324/10000 (63.24%), error:     10747.56934, best error:     10609.83887 on epoch:    42, epoch:   43/100, mu: 0.9508112073, learning_rate: 0.0000495259, weight_decay: 0.0000379696

epoch time: 890.0892944336s, input_fired_time: 12.7690706253, output_fired_time: 5.3016467094, propagate_forward_time: 334.3655395508, propagate_backward_time: 527.4827880859, weight_update_time: 0.3352186382, other_time: 10.1702880859
epoch time: 69.9160537720s, input_fired_time: 0.8996730447, output_fired_time: 0.0000000000, propagate_forward_time: 67.3116149902, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7047653198
validation[          , genome  6288] predictions:    6259/  10000 (62.59%), best:    6324/10000 (63.24%), error:     10715.75977, best error:     10609.83887 on epoch:    42, epoch:   44/100, mu: 0.9530515075, learning_rate: 0.0000461278, weight_decay: 0.0000357763

epoch time: 889.4844360352s, input_fired_time: 12.7744827271, output_fired_time: 5.3119592667, propagate_forward_time: 334.8009338379, propagate_backward_time: 526.4653930664, weight_update_time: 0.3344761729, other_time: 10.1316528320
epoch time: 69.8909301758s, input_fired_time: 0.8967658877, output_fired_time: 0.0000000000, propagate_forward_time: 67.2765960693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7175674438
validation[          , genome  6288] predictions:    6115/  10000 (61.15%), best:    6324/10000 (63.24%), error:     11054.95898, best error:     10609.83887 on epoch:    42, epoch:   45/100, mu: 0.9551637769, learning_rate: 0.0000429629, weight_decay: 0.0000337097

epoch time: 890.3740844727s, input_fired_time: 12.7961559296, output_fired_time: 5.3458690643, propagate_forward_time: 335.1363830566, propagate_backward_time: 526.9323730469, weight_update_time: 0.3321339786, other_time: 10.1633300781
epoch time: 69.3349151611s, input_fired_time: 0.8926357031, output_fired_time: 0.0000000000, propagate_forward_time: 66.7235107422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7187652588
validation[          , genome  6288] predictions:    6288/  10000 (62.88%), best:    6324/10000 (63.24%), error:     10830.65234, best error:     10609.83887 on epoch:    42, epoch:   46/100, mu: 0.9571552873, learning_rate: 0.0000400151, weight_decay: 0.0000317624

epoch time: 887.8146972656s, input_fired_time: 12.7742176056, output_fired_time: 5.3241539001, propagate_forward_time: 334.1467895508, propagate_backward_time: 525.3933105469, weight_update_time: 0.3340908885, other_time: 10.1762084961
epoch time: 69.9691543579s, input_fired_time: 0.8998347521, output_fired_time: 0.0000000000, propagate_forward_time: 67.3378295898, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7314910889
validation[          , genome  6288] predictions:    6286/  10000 (62.86%), best:    6324/10000 (63.24%), error:     10704.00684, best error:     10609.83887 on epoch:    42, epoch:   47/100, mu: 0.9590329528, learning_rate: 0.0000372695, weight_decay: 0.0000299277

epoch time: 890.3128051758s, input_fired_time: 12.8058090210, output_fired_time: 5.3640203476, propagate_forward_time: 335.6385192871, propagate_backward_time: 526.3277587891, weight_update_time: 0.3354626298, other_time: 10.1767578125
epoch time: 69.4628982544s, input_fired_time: 0.8982300758, output_fired_time: 0.0000000000, propagate_forward_time: 66.8379287720, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7267379761
validation[          , genome  6288] predictions:    6319/  10000 (63.19%), best:    6324/10000 (63.24%), error:     10659.26465, best error:     10609.83887 on epoch:    42, epoch:   48/100, mu: 0.9608032703, learning_rate: 0.0000347124, weight_decay: 0.0000281989

epoch time: 890.1866455078s, input_fired_time: 12.7777023315, output_fired_time: 5.3404483795, propagate_forward_time: 334.8275451660, propagate_backward_time: 527.0553588867, weight_update_time: 0.3331256509, other_time: 10.1856079102
epoch time: 69.8308410645s, input_fired_time: 0.8986144662, output_fired_time: 0.0000000000, propagate_forward_time: 67.2386398315, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6935882568
validation[          , genome  6288] predictions:    6240/  10000 (62.40%), best:    6324/10000 (63.24%), error:     10731.52734, best error:     10609.83887 on epoch:    42, epoch:   49/100, mu: 0.9624723792, learning_rate: 0.0000323307, weight_decay: 0.0000265700

epoch time: 888.6780395508s, input_fired_time: 12.7705936432, output_fired_time: 5.3218626976, propagate_forward_time: 334.2761840820, propagate_backward_time: 526.1441040039, weight_update_time: 0.3386615515, other_time: 10.1653442383
epoch time: 69.4753723145s, input_fired_time: 0.9008362293, output_fired_time: 0.0000000000, propagate_forward_time: 66.8955764771, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6789627075
validation[          , genome  6288] predictions:    6388/  10000 (63.88%), best:    6388/10000 (63.88%), error:     10444.39355, best error:     10444.39355 on epoch:    50, epoch:   50/100, mu: 0.9640460610, learning_rate: 0.0000301124, weight_decay: 0.0000250352

epoch time: 890.3517456055s, input_fired_time: 12.7967681885, output_fired_time: 5.3348069191, propagate_forward_time: 335.5325927734, propagate_backward_time: 526.4403686523, weight_update_time: 0.3380182385, other_time: 10.2472534180
epoch time: 69.9597930908s, input_fired_time: 0.9002696872, output_fired_time: 0.0000000000, propagate_forward_time: 67.3333587646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7261657715
validation[          , genome  6288] predictions:    6296/  10000 (62.96%), best:    6388/10000 (63.88%), error:     10662.26172, best error:     10444.39355 on epoch:    50, epoch:   51/100, mu: 0.9655297995, learning_rate: 0.0000280463, weight_decay: 0.0000235890

epoch time: 891.4234619141s, input_fired_time: 12.8081626892, output_fired_time: 5.3149509430, propagate_forward_time: 335.8008422852, propagate_backward_time: 527.2812500000, weight_update_time: 0.3367323577, other_time: 10.2182617188
epoch time: 69.4851684570s, input_fired_time: 0.9007425308, output_fired_time: 0.0000000000, propagate_forward_time: 66.8906784058, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6937484741
validation[          , genome  6288] predictions:    6358/  10000 (63.58%), best:    6358/10000 (63.58%), error:     10436.27930, best error:     10436.27930 on epoch:    52, epoch:   52/100, mu: 0.9669287205, learning_rate: 0.0000261220, weight_decay: 0.0000222264

epoch time: 889.5984497070s, input_fired_time: 12.7994413376, output_fired_time: 5.3420128822, propagate_forward_time: 334.5382080078, propagate_backward_time: 526.6577148438, weight_update_time: 0.3413116336, other_time: 10.2610473633
epoch time: 69.8042602539s, input_fired_time: 0.9014708996, output_fired_time: 0.0000000000, propagate_forward_time: 67.1979751587, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7048110962
validation[          , genome  6288] predictions:    6369/  10000 (63.69%), best:    6358/10000 (63.58%), error:     10494.89746, best error:     10436.27930 on epoch:    52, epoch:   53/100, mu: 0.9682476521, learning_rate: 0.0000243297, weight_decay: 0.0000209425

epoch time: 886.4603271484s, input_fired_time: 12.7786417007, output_fired_time: 5.2916140556, propagate_forward_time: 333.2967224121, propagate_backward_time: 524.9349975586, weight_update_time: 0.3377459347, other_time: 10.1583862305
epoch time: 69.2638244629s, input_fired_time: 0.8989923596, output_fired_time: 0.0000000000, propagate_forward_time: 66.6552505493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7095794678
validation[          , genome  6288] predictions:    6435/  10000 (64.35%), best:    6435/10000 (64.35%), error:     10418.56738, best error:     10418.56738 on epoch:    54, epoch:   54/100, mu: 0.9694911838, learning_rate: 0.0000226604, weight_decay: 0.0000197328

epoch time: 884.5751342773s, input_fired_time: 12.7520532608, output_fired_time: 5.3108983040, propagate_forward_time: 332.8807067871, propagate_backward_time: 523.4197387695, weight_update_time: 0.3433524668, other_time: 10.2117309570
epoch time: 69.0918655396s, input_fired_time: 0.8954890370, output_fired_time: 0.0000000000, propagate_forward_time: 66.4772720337, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7191009521
validation[          , genome  6288] predictions:    6272/  10000 (62.72%), best:    6435/10000 (64.35%), error:     10627.36523, best error:     10418.56738 on epoch:    54, epoch:   55/100, mu: 0.9706636071, learning_rate: 0.0000211056, weight_decay: 0.0000185929

epoch time: 886.7323608398s, input_fired_time: 12.7933378220, output_fired_time: 5.2897992134, propagate_forward_time: 335.2289428711, propagate_backward_time: 523.3132324219, weight_update_time: 0.3309762776, other_time: 10.1070556641
epoch time: 69.4609985352s, input_fired_time: 0.8952488899, output_fired_time: 0.0000000000, propagate_forward_time: 66.8402481079, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7255020142
validation[          , genome  6288] predictions:    6357/  10000 (63.57%), best:    6435/10000 (64.35%), error:     10484.94727, best error:     10418.56738 on epoch:    54, epoch:   56/100, mu: 0.9717690349, learning_rate: 0.0000196575, weight_decay: 0.0000175189

epoch time: 889.9484863281s, input_fired_time: 12.8292016983, output_fired_time: 5.3440527916, propagate_forward_time: 335.2958374023, propagate_backward_time: 526.2493896484, weight_update_time: 0.3364506960, other_time: 10.2299804688
epoch time: 69.7579879761s, input_fired_time: 0.9045583010, output_fired_time: 0.0000000000, propagate_forward_time: 67.1407394409, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7126922607
validation[          , genome  6288] predictions:    6381/  10000 (63.81%), best:    6381/10000 (63.81%), error:     10391.59766, best error:     10391.59766 on epoch:    57, epoch:   57/100, mu: 0.9728112817, learning_rate: 0.0000183087, weight_decay: 0.0000165069

epoch time: 892.9768676758s, input_fired_time: 12.8477926254, output_fired_time: 5.3456335068, propagate_forward_time: 335.8757934570, propagate_backward_time: 528.6476440430, weight_update_time: 0.3410353363, other_time: 10.2600097656
epoch time: 69.9546585083s, input_fired_time: 0.9117389321, output_fired_time: 0.0000000000, propagate_forward_time: 67.3554534912, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6874694824
validation[          , genome  6288] predictions:    6367/  10000 (63.67%), best:    6367/10000 (63.67%), error:     10369.92676, best error:     10369.92676 on epoch:    58, epoch:   58/100, mu: 0.9737939239, learning_rate: 0.0000170525, weight_decay: 0.0000155534

epoch time: 893.2081298828s, input_fired_time: 12.8491020203, output_fired_time: 5.3527827263, propagate_forward_time: 336.1069030762, propagate_backward_time: 528.6602172852, weight_update_time: 0.3393453956, other_time: 10.2390747070
epoch time: 70.8027725220s, input_fired_time: 0.9148109555, output_fired_time: 0.0000000000, propagate_forward_time: 68.1457061768, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7422561646
validation[          , genome  6288] predictions:    6267/  10000 (62.67%), best:    6367/10000 (63.67%), error:     10670.47070, best error:     10369.92676 on epoch:    58, epoch:   59/100, mu: 0.9747203588, learning_rate: 0.0000158825, weight_decay: 0.0000146550

epoch time: 893.6725463867s, input_fired_time: 12.8564882278, output_fired_time: 5.3463702202, propagate_forward_time: 335.9783325195, propagate_backward_time: 529.2623291016, weight_update_time: 0.3394750655, other_time: 10.2290039063
epoch time: 70.1215515137s, input_fired_time: 0.9118210673, output_fired_time: 0.0000000000, propagate_forward_time: 67.4768676758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7328643799
validation[          , genome  6288] predictions:    6263/  10000 (62.63%), best:    6367/10000 (63.67%), error:     10622.97363, best error:     10369.92676 on epoch:    58, epoch:   60/100, mu: 0.9755938649, learning_rate: 0.0000147928, weight_decay: 0.0000138084

epoch time: 893.8356933594s, input_fired_time: 12.8806352615, output_fired_time: 5.3259272575, propagate_forward_time: 336.8404846191, propagate_backward_time: 528.5225830078, weight_update_time: 0.3457847536, other_time: 10.2661132813
epoch time: 69.4470748901s, input_fired_time: 0.9055804610, output_fired_time: 0.0000000000, propagate_forward_time: 66.8220977783, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7193984985
validation[          , genome  6288] predictions:    6438/  10000 (64.38%), best:    6438/10000 (64.38%), error:     10314.54492, best error:     10314.54492 on epoch:    61, epoch:   61/100, mu: 0.9764174223, learning_rate: 0.0000137778, weight_decay: 0.0000130108

epoch time: 892.6776123047s, input_fired_time: 12.8761997223, output_fired_time: 5.3710737228, propagate_forward_time: 336.3225402832, propagate_backward_time: 527.8862915039, weight_update_time: 0.3463364244, other_time: 10.2214965820
epoch time: 69.6813354492s, input_fired_time: 0.9099823236, output_fired_time: 0.0000000000, propagate_forward_time: 67.0937042236, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6776504517
validation[          , genome  6288] predictions:    6422/  10000 (64.22%), best:    6438/10000 (64.38%), error:     10352.17285, best error:     10314.54492 on epoch:    61, epoch:   62/100, mu: 0.9771938920, learning_rate: 0.0000128325, weight_decay: 0.0000122592

epoch time: 900.0432739258s, input_fired_time: 12.9637098312, output_fired_time: 5.3636617661, propagate_forward_time: 338.9055175781, propagate_backward_time: 532.5414428711, weight_update_time: 0.3417114317, other_time: 10.2689819336
epoch time: 70.4762649536s, input_fired_time: 0.9125339985, output_fired_time: 0.0000000000, propagate_forward_time: 67.8306579590, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7330703735
validation[          , genome  6288] predictions:    6409/  10000 (64.09%), best:    6409/10000 (64.09%), error:     10300.95801, best error:     10300.95801 on epoch:    63, epoch:   63/100, mu: 0.9779260159, learning_rate: 0.0000119520, weight_decay: 0.0000115511

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288_1_r1706264427_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497418
read mu: 0.978616
read mu_delta: 0.942832
read initial_learning_rate: 0.00105251
read learning_rate: 1.11319e-05
read learning_rate_delta: 0.931387
read initial_weight_decay: 0.000490441
read weight_decay: 1.08838e-05
read weight_decay_delta: 0.942235
read batch_size: 85
read epsilon: 1e-07
read alpha: 0.0831412
read input_dropout_probability: 0.00121737
read hidden_dropout_probability: 0.0984132
read velocity_reset: 1592
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 63
read number_validation_images: 10000
read best_validation_predictions: 6409
read best_validation_error: 10301
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6288
read normal distribution: '1 0 0'
generator_str: '1006984341'
read generator: 1006984341
read generated_by_map:
1 disable_edge 1
reading 70 nodes.
reading 293 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1094.80236816406s, input_fired_time: 14.6700983047485, output_fired_time: 6.54079818725586, propagate_forward_time: 417.450531005859, propagate_backward_time: 644.535705566406, weight_update_time: 0.401984661817551, other_time: 11.6052856445313
epoch time: 80.2034454345703s, input_fired_time: 1.01294851303101, output_fired_time: 0, propagate_forward_time: 77.3158798217773, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.87461853027344
validation[          , genome  6288] predictions:    6447/  10000 (64.47%), best:    6447/10000 (64.47%), error:     10254.24023, best error:     10254.24023 on epoch:    64, epoch:   64/100, mu: 0.9786162376, learning_rate: 0.0000111319, weight_decay: 0.0000108838

epoch time: 940.8860473633s, input_fired_time: 13.2944707870, output_fired_time: 5.6774344444, propagate_forward_time: 355.1859741211, propagate_backward_time: 556.1434936523, weight_update_time: 0.3558515906, other_time: 10.5846557617
epoch time: 70.1942443848s, input_fired_time: 0.9136856198, output_fired_time: 0.0000000000, propagate_forward_time: 67.5723114014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7082443237
validation[          , genome  6288] predictions:    6390/  10000 (63.90%), best:    6447/10000 (64.47%), error:     10372.24219, best error:     10254.24023 on epoch:    64, epoch:   65/100, mu: 0.9792670012, learning_rate: 0.0000103681, weight_decay: 0.0000102551

epoch time: 899.8909301758s, input_fired_time: 12.9617509842, output_fired_time: 5.3956151009, propagate_forward_time: 338.5950622559, propagate_backward_time: 532.6102905273, weight_update_time: 0.3413008153, other_time: 10.3281860352
epoch time: 76.5317001343s, input_fired_time: 0.9839122891, output_fired_time: 0.0000000000, propagate_forward_time: 73.6954421997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8523483276
validation[          , genome  6288] predictions:    6429/  10000 (64.29%), best:    6447/10000 (64.47%), error:     10292.00977, best error:     10254.24023 on epoch:    64, epoch:   66/100, mu: 0.9798805714, learning_rate: 0.0000096568, weight_decay: 0.0000096627

epoch time: 1006.9009399414s, input_fired_time: 13.9911508560, output_fired_time: 6.0004878044, propagate_forward_time: 381.6308288574, propagate_backward_time: 593.7969360352, weight_update_time: 0.3709474802, other_time: 11.4815063477
epoch time: 86.1098251343s, input_fired_time: 1.1140944958, output_fired_time: 0.0000000000, propagate_forward_time: 82.8424682617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1532592773
validation[          , genome  6288] predictions:    6416/  10000 (64.16%), best:    6447/10000 (64.47%), error:     10319.01660, best error:     10254.24023 on epoch:    64, epoch:   67/100, mu: 0.9804590940, learning_rate: 0.0000089942, weight_decay: 0.0000091046

epoch time: 1020.1470336914s, input_fired_time: 14.1415090561, output_fired_time: 6.1290388107, propagate_forward_time: 387.1813964844, propagate_backward_time: 601.0725097656, weight_update_time: 0.3751318753, other_time: 11.6226196289
epoch time: 86.9275283813s, input_fired_time: 1.1085915565, output_fired_time: 0.0000000000, propagate_forward_time: 83.7394943237, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0794448853
validation[          , genome  6288] predictions:    6399/  10000 (63.99%), best:    6447/10000 (64.47%), error:     10315.66406, best error:     10254.24023 on epoch:    64, epoch:   68/100, mu: 0.9810045362, learning_rate: 0.0000083771, weight_decay: 0.0000085786

epoch time: 923.5740966797s, input_fired_time: 13.2131233215, output_fired_time: 5.5609850883, propagate_forward_time: 348.5712280273, propagate_backward_time: 545.6873779297, weight_update_time: 0.3505567610, other_time: 10.5413818359
epoch time: 70.5421066284s, input_fired_time: 0.9189304709, output_fired_time: 0.0000000000, propagate_forward_time: 67.9186325073, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7045440674
validation[          , genome  6288] predictions:    6417/  10000 (64.17%), best:    6447/10000 (64.47%), error:     10292.36035, best error:     10254.24023 on epoch:    64, epoch:   69/100, mu: 0.9815188050, learning_rate: 0.0000078023, weight_decay: 0.0000080831

epoch time: 904.3453369141s, input_fired_time: 13.0286788940, output_fired_time: 5.4089646339, propagate_forward_time: 340.3746643066, propagate_backward_time: 535.2257690430, weight_update_time: 0.3382577896, other_time: 10.3073120117
epoch time: 70.8947448730s, input_fired_time: 0.9183328152, output_fired_time: 0.0000000000, propagate_forward_time: 68.2679443359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7084655762
validation[          , genome  6288] predictions:    6411/  10000 (64.11%), best:    6411/10000 (64.11%), error:     10242.62988, best error:     10242.62988 on epoch:    70, epoch:   70/100, mu: 0.9820036292, learning_rate: 0.0000072670, weight_decay: 0.0000076162

epoch time: 899.7534790039s, input_fired_time: 13.0074882507, output_fired_time: 5.3603105545, propagate_forward_time: 340.1300659180, propagate_backward_time: 530.9559936523, weight_update_time: 0.3449253738, other_time: 10.2996215820
epoch time: 70.2269210815s, input_fired_time: 0.9147148132, output_fired_time: 0.0000000000, propagate_forward_time: 67.6329879761, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6792221069
validation[          , genome  6288] predictions:    6390/  10000 (63.90%), best:    6411/10000 (64.11%), error:     10254.63672, best error:     10242.62988 on epoch:    70, epoch:   71/100, mu: 0.9824607372, learning_rate: 0.0000067684, weight_decay: 0.0000071762

epoch time: 900.1463623047s, input_fired_time: 12.9833297729, output_fired_time: 5.4386634827, propagate_forward_time: 338.5470275879, propagate_backward_time: 532.9216918945, weight_update_time: 0.3402206600, other_time: 10.2556762695
epoch time: 70.8566436768s, input_fired_time: 0.9184694290, output_fired_time: 0.0000000000, propagate_forward_time: 68.1873855591, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7507858276
validation[          , genome  6288] predictions:    6431/  10000 (64.31%), best:    6411/10000 (64.11%), error:     10356.91602, best error:     10242.62988 on epoch:    70, epoch:   72/100, mu: 0.9828917384, learning_rate: 0.0000063040, weight_decay: 0.0000067617

epoch time: 898.0106201172s, input_fired_time: 12.9343204498, output_fired_time: 5.4073734283, propagate_forward_time: 337.9436645508, propagate_backward_time: 531.4871826172, weight_update_time: 0.3376714289, other_time: 10.2380981445
epoch time: 69.8349456787s, input_fired_time: 0.9126009345, output_fired_time: 0.0000000000, propagate_forward_time: 67.2187347412, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7036132813
validation[          , genome  6288] predictions:    6446/  10000 (64.46%), best:    6446/10000 (64.46%), error:     10218.77051, best error:     10218.77051 on epoch:    73, epoch:   73/100, mu: 0.9832981229, learning_rate: 0.0000058714, weight_decay: 0.0000063711

epoch time: 902.2268066406s, input_fired_time: 12.9941158295, output_fired_time: 5.3838362694, propagate_forward_time: 339.8373718262, propagate_backward_time: 533.7487182617, weight_update_time: 0.3411644101, other_time: 10.2627563477
epoch time: 70.9440307617s, input_fired_time: 0.9265415668, output_fired_time: 0.0000000000, propagate_forward_time: 68.2889938354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7284927368
validation[          , genome  6288] predictions:    6458/  10000 (64.58%), best:    6458/10000 (64.58%), error:     10192.55859, best error:     10192.55859 on epoch:    74, epoch:   74/100, mu: 0.9836812615, learning_rate: 0.0000054686, weight_decay: 0.0000060031

epoch time: 912.2651977539s, input_fired_time: 13.1913824081, output_fired_time: 5.4705662727, propagate_forward_time: 343.6031188965, propagate_backward_time: 539.6383666992, weight_update_time: 0.3498288393, other_time: 10.3617553711
epoch time: 71.3226318359s, input_fired_time: 0.9260843396, output_fired_time: 0.0000000000, propagate_forward_time: 68.7086181641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6879272461
validation[          , genome  6288] predictions:    6448/  10000 (64.48%), best:    6458/10000 (64.58%), error:     10215.54688, best error:     10192.55859 on epoch:    74, epoch:   75/100, mu: 0.9840424657, learning_rate: 0.0000050934, weight_decay: 0.0000056563

epoch time: 905.8315429688s, input_fired_time: 13.0889863968, output_fired_time: 5.4075894356, propagate_forward_time: 340.8304443359, propagate_backward_time: 536.1655273438, weight_update_time: 0.3494306207, other_time: 10.3389892578
epoch time: 70.3952865601s, input_fired_time: 0.9175658226, output_fired_time: 0.0000000000, propagate_forward_time: 67.7426300049, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7350921631
validation[          , genome  6288] predictions:    6468/  10000 (64.68%), best:    6458/10000 (64.58%), error:     10198.22461, best error:     10192.55859 on epoch:    74, epoch:   76/100, mu: 0.9843830466, learning_rate: 0.0000047439, weight_decay: 0.0000053296

epoch time: 903.8347778320s, input_fired_time: 13.0278043747, output_fired_time: 5.3939299583, propagate_forward_time: 340.1994628906, propagate_backward_time: 534.8875122070, weight_update_time: 0.3527624607, other_time: 10.3260498047
epoch time: 70.6732940674s, input_fired_time: 0.9243270755, output_fired_time: 0.0000000000, propagate_forward_time: 68.0340576172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7149124146
validation[          , genome  6288] predictions:    6439/  10000 (64.39%), best:    6458/10000 (64.58%), error:     10228.70801, best error:     10192.55859 on epoch:    74, epoch:   77/100, mu: 0.9847041368, learning_rate: 0.0000044184, weight_decay: 0.0000050217

epoch time: 900.0863037109s, input_fired_time: 12.9892606735, output_fired_time: 5.4137420654, propagate_forward_time: 339.4366455078, propagate_backward_time: 531.9232177734, weight_update_time: 0.3440013528, other_time: 10.3234252930
epoch time: 70.3761672974s, input_fired_time: 0.9218731523, output_fired_time: 0.0000000000, propagate_forward_time: 67.7309188843, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7233734131
validation[          , genome  6288] predictions:    6452/  10000 (64.52%), best:    6452/10000 (64.52%), error:     10166.57715, best error:     10166.57715 on epoch:    78, epoch:   78/100, mu: 0.9850068688, learning_rate: 0.0000041153, weight_decay: 0.0000047316

epoch time: 899.8317260742s, input_fired_time: 12.9884386063, output_fired_time: 5.3606410027, propagate_forward_time: 338.9275817871, propagate_backward_time: 532.2640991211, weight_update_time: 0.3451038599, other_time: 10.2909545898
epoch time: 70.9112014771s, input_fired_time: 0.9246004224, output_fired_time: 0.0000000000, propagate_forward_time: 68.2310943604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7555084229
validation[          , genome  6288] predictions:    6420/  10000 (64.20%), best:    6452/10000 (64.52%), error:     10178.13867, best error:     10166.57715 on epoch:    78, epoch:   79/100, mu: 0.9852923155, learning_rate: 0.0000038329, weight_decay: 0.0000044583

epoch time: 903.4774780273s, input_fired_time: 13.0212764740, output_fired_time: 5.4367017746, propagate_forward_time: 340.1890869141, propagate_backward_time: 534.5801391602, weight_update_time: 0.3474077582, other_time: 10.2502441406
epoch time: 70.0689468384s, input_fired_time: 0.9239532351, output_fired_time: 0.0000000000, propagate_forward_time: 67.4479522705, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6970443726
validation[          , genome  6288] predictions:    6458/  10000 (64.58%), best:    6452/10000 (64.52%), error:     10182.23438, best error:     10166.57715 on epoch:    78, epoch:   80/100, mu: 0.9855614305, learning_rate: 0.0000035699, weight_decay: 0.0000042008

epoch time: 901.5042114258s, input_fired_time: 13.0792446136, output_fired_time: 5.3339090347, propagate_forward_time: 338.9015197754, propagate_backward_time: 534.0864257813, weight_update_time: 0.3416611254, other_time: 10.1031494141
epoch time: 70.5083618164s, input_fired_time: 0.9251028895, output_fired_time: 0.0000000000, propagate_forward_time: 67.8835754395, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6996841431
validation[          , genome  6288] predictions:    6438/  10000 (64.38%), best:    6452/10000 (64.52%), error:     10182.64648, best error:     10166.57715 on epoch:    78, epoch:   81/100, mu: 0.9858151674, learning_rate: 0.0000033250, weight_decay: 0.0000039581

epoch time: 903.1380004883s, input_fired_time: 13.0781984329, output_fired_time: 5.4335002899, propagate_forward_time: 340.3689270020, propagate_backward_time: 534.0554199219, weight_update_time: 0.3425273299, other_time: 10.2020263672
epoch time: 70.3458175659s, input_fired_time: 0.9252216220, output_fired_time: 0.0000000000, propagate_forward_time: 67.7214584351, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6991348267
validation[          , genome  6288] predictions:    6477/  10000 (64.77%), best:    6452/10000 (64.52%), error:     10198.84668, best error:     10166.57715 on epoch:    78, epoch:   82/100, mu: 0.9860544205, learning_rate: 0.0000030968, weight_decay: 0.0000037295

epoch time: 902.4710083008s, input_fired_time: 13.0284709930, output_fired_time: 5.3570060730, propagate_forward_time: 340.4642333984, propagate_backward_time: 533.3980712891, weight_update_time: 0.3390786052, other_time: 10.2232666016
epoch time: 70.7269592285s, input_fired_time: 0.9217302203, output_fired_time: 0.0000000000, propagate_forward_time: 68.0764617920, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7287673950
validation[          , genome  6288] predictions:    6446/  10000 (64.46%), best:    6446/10000 (64.46%), error:     10164.49512, best error:     10164.49512 on epoch:    83, epoch:   83/100, mu: 0.9862799644, learning_rate: 0.0000028844, weight_decay: 0.0000035140

epoch time: 901.9863891602s, input_fired_time: 13.0161476135, output_fired_time: 5.4068269730, propagate_forward_time: 339.4329528809, propagate_backward_time: 533.9824218750, weight_update_time: 0.3379441500, other_time: 10.1480712891
epoch time: 70.2696228027s, input_fired_time: 0.9190362692, output_fired_time: 0.0000000000, propagate_forward_time: 67.6348190308, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7157669067
validation[          , genome  6288] predictions:    6456/  10000 (64.56%), best:    6446/10000 (64.46%), error:     10187.43457, best error:     10164.49512 on epoch:    83, epoch:   84/100, mu: 0.9864926338, learning_rate: 0.0000026864, weight_decay: 0.0000033111

epoch time: 898.1029663086s, input_fired_time: 12.9818935394, output_fired_time: 5.3891091347, propagate_forward_time: 338.1699523926, propagate_backward_time: 531.3298339844, weight_update_time: 0.3472365439, other_time: 10.2321777344
epoch time: 71.3056030273s, input_fired_time: 0.9288785458, output_fired_time: 0.0000000000, propagate_forward_time: 68.6294479370, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7472763062
validation[          , genome  6288] predictions:    6462/  10000 (64.62%), best:    6462/10000 (64.62%), error:     10150.79688, best error:     10150.79688 on epoch:    85, epoch:   85/100, mu: 0.9866931438, learning_rate: 0.0000025021, weight_decay: 0.0000031198

epoch time: 904.9004516602s, input_fired_time: 13.0688276291, output_fired_time: 5.3575496674, propagate_forward_time: 341.6056823730, propagate_backward_time: 534.5975341797, weight_update_time: 0.3456906676, other_time: 10.2708740234
epoch time: 71.1656112671s, input_fired_time: 0.9307411313, output_fired_time: 0.0000000000, propagate_forward_time: 68.5577621460, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6771087646
validation[          , genome  6288] predictions:    6492/  10000 (64.92%), best:    6462/10000 (64.62%), error:     10171.72070, best error:     10150.79688 on epoch:    85, epoch:   86/100, mu: 0.9868822098, learning_rate: 0.0000023304, weight_decay: 0.0000029396

epoch time: 910.9106445313s, input_fired_time: 13.1480760574, output_fired_time: 5.4638872147, propagate_forward_time: 343.3193054199, propagate_backward_time: 538.7144775391, weight_update_time: 0.3449541032, other_time: 10.2648925781
epoch time: 71.3397903442s, input_fired_time: 0.9330473542, output_fired_time: 0.0000000000, propagate_forward_time: 68.6935043335, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7132415771
validation[          , genome  6288] predictions:    6443/  10000 (64.43%), best:    6443/10000 (64.43%), error:     10119.36719, best error:     10119.36719 on epoch:    87, epoch:   87/100, mu: 0.9870604277, learning_rate: 0.0000021706, weight_decay: 0.0000027698

epoch time: 907.3400268555s, input_fired_time: 13.1290197372, output_fired_time: 5.3853964806, propagate_forward_time: 341.8282470703, propagate_backward_time: 536.7297363281, weight_update_time: 0.3484624624, other_time: 10.2676391602
epoch time: 70.8515090942s, input_fired_time: 0.9252499938, output_fired_time: 0.0000000000, propagate_forward_time: 68.2035751343, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7226867676
validation[          , genome  6288] predictions:    6477/  10000 (64.77%), best:    6443/10000 (64.43%), error:     10141.14551, best error:     10119.36719 on epoch:    87, epoch:   88/100, mu: 0.9872284532, learning_rate: 0.0000020216, weight_decay: 0.0000026098

epoch time: 905.4260864258s, input_fired_time: 13.0635204315, output_fired_time: 5.3736872673, propagate_forward_time: 341.1072387695, propagate_backward_time: 535.5960083008, weight_update_time: 0.3526856899, other_time: 10.2856445313
epoch time: 70.0088806152s, input_fired_time: 0.9214258790, output_fired_time: 0.0000000000, propagate_forward_time: 67.3906173706, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6968383789
validation[          , genome  6288] predictions:    6437/  10000 (64.37%), best:    6443/10000 (64.43%), error:     10162.60059, best error:     10119.36719 on epoch:    87, epoch:   89/100, mu: 0.9873868823, learning_rate: 0.0000018829, weight_decay: 0.0000024590

epoch time: 900.6970214844s, input_fired_time: 13.0055150986, output_fired_time: 5.3877682686, propagate_forward_time: 339.5039672852, propagate_backward_time: 532.6035766602, weight_update_time: 0.3408519030, other_time: 10.1962280273
epoch time: 70.6337432861s, input_fired_time: 0.9247482419, output_fired_time: 0.0000000000, propagate_forward_time: 67.9989624023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7100296021
validation[          , genome  6288] predictions:    6467/  10000 (64.67%), best:    6443/10000 (64.43%), error:     10173.17871, best error:     10119.36719 on epoch:    87, epoch:   90/100, mu: 0.9875362515, learning_rate: 0.0000017537, weight_decay: 0.0000023170

epoch time: 897.9390869141s, input_fired_time: 12.9862489700, output_fired_time: 5.3425230980, propagate_forward_time: 338.2518310547, propagate_backward_time: 531.1694946289, weight_update_time: 0.3485298753, other_time: 10.1889648438
epoch time: 70.3891296387s, input_fired_time: 0.9227763414, output_fired_time: 0.0000000000, propagate_forward_time: 67.7713012695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6950531006
validation[          , genome  6288] predictions:    6484/  10000 (64.84%), best:    6443/10000 (64.43%), error:     10128.66699, best error:     10119.36719 on epoch:    87, epoch:   91/100, mu: 0.9876770973, learning_rate: 0.0000016334, weight_decay: 0.0000021831

epoch time: 904.0935058594s, input_fired_time: 13.0505475998, output_fired_time: 5.3460745811, propagate_forward_time: 341.7648010254, propagate_backward_time: 533.6914672852, weight_update_time: 0.3458375931, other_time: 10.2406616211
epoch time: 70.7502670288s, input_fired_time: 0.9313919544, output_fired_time: 0.0000000000, propagate_forward_time: 68.1223907471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6964874268
validation[          , genome  6288] predictions:    6452/  10000 (64.52%), best:    6443/10000 (64.43%), error:     10135.62695, best error:     10119.36719 on epoch:    87, epoch:   92/100, mu: 0.9878098965, learning_rate: 0.0000015213, weight_decay: 0.0000020570

epoch time: 908.9895629883s, input_fired_time: 13.1593036652, output_fired_time: 5.3980431557, propagate_forward_time: 343.0234985352, propagate_backward_time: 537.1482543945, weight_update_time: 0.3528413177, other_time: 10.2604370117
epoch time: 71.2108078003s, input_fired_time: 0.9336324334, output_fired_time: 0.0000000000, propagate_forward_time: 68.5752639771, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7019119263
validation[          , genome  6288] predictions:    6472/  10000 (64.72%), best:    6443/10000 (64.43%), error:     10127.24902, best error:     10119.36719 on epoch:    87, epoch:   93/100, mu: 0.9879351258, learning_rate: 0.0000014169, weight_decay: 0.0000019382

epoch time: 908.4501953125s, input_fired_time: 13.1084079742, output_fired_time: 5.4495191574, propagate_forward_time: 342.4908447266, propagate_backward_time: 537.0840454102, weight_update_time: 0.3526611328, other_time: 10.3173828125
epoch time: 71.1185760498s, input_fired_time: 0.9272138476, output_fired_time: 0.0000000000, propagate_forward_time: 68.4810791016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7102813721
validation[          , genome  6288] predictions:    6477/  10000 (64.77%), best:    6443/10000 (64.43%), error:     10132.85059, best error:     10119.36719 on epoch:    87, epoch:   94/100, mu: 0.9880531430, learning_rate: 0.0000013197, weight_decay: 0.0000018262

epoch time: 903.9696655273s, input_fired_time: 13.0583429337, output_fired_time: 5.3899846077, propagate_forward_time: 340.1307373047, propagate_backward_time: 535.1179199219, weight_update_time: 0.3477025330, other_time: 10.2726440430
epoch time: 70.2377090454s, input_fired_time: 0.9191372395, output_fired_time: 0.0000000000, propagate_forward_time: 67.6465377808, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6720352173
validation[          , genome  6288] predictions:    6467/  10000 (64.67%), best:    6443/10000 (64.43%), error:     10139.66211, best error:     10119.36719 on epoch:    87, epoch:   95/100, mu: 0.9881644249, learning_rate: 0.0000012292, weight_decay: 0.0000017208

epoch time: 903.0393066406s, input_fired_time: 13.0239410400, output_fired_time: 5.4110159874, propagate_forward_time: 340.5826110840, propagate_backward_time: 533.7359619141, weight_update_time: 0.3547834456, other_time: 10.2857666016
epoch time: 70.8901977539s, input_fired_time: 0.9298107624, output_fired_time: 0.0000000000, propagate_forward_time: 68.2387084961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7216796875
validation[          , genome  6288] predictions:    6448/  10000 (64.48%), best:    6443/10000 (64.43%), error:     10153.91797, best error:     10119.36719 on epoch:    87, epoch:   96/100, mu: 0.9882693887, learning_rate: 0.0000011448, weight_decay: 0.0000016214

epoch time: 897.0617675781s, input_fired_time: 12.9982347488, output_fired_time: 5.3236141205, propagate_forward_time: 337.2073364258, propagate_backward_time: 531.3959350586, weight_update_time: 0.3433930278, other_time: 10.1366577148
epoch time: 69.5295639038s, input_fired_time: 0.9135217667, output_fired_time: 0.0000000000, propagate_forward_time: 66.9095840454, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7064590454
validation[          , genome  6288] predictions:    6477/  10000 (64.77%), best:    6477/10000 (64.77%), error:     10117.01953, best error:     10117.01953 on epoch:    97, epoch:   97/100, mu: 0.9883683324, learning_rate: 0.0000010663, weight_decay: 0.0000015277

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288_1_r1706264427_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497418
read mu: 0.988462
read mu_delta: 0.942832
read initial_learning_rate: 0.00105251
read learning_rate: 9.93126e-07
read learning_rate_delta: 0.931387
read initial_weight_decay: 0.000490441
read weight_decay: 1.43945e-06
read weight_decay_delta: 0.942235
read batch_size: 85
read epsilon: 1e-07
read alpha: 0.0831412
read input_dropout_probability: 0.00121737
read hidden_dropout_probability: 0.0984132
read velocity_reset: 1592
read epoch: 98
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6477
read best_validation_error: 10117
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6288
read normal distribution: '1 0 0'
generator_str: '1398929753'
read generator: 1398929753
read generated_by_map:
1 disable_edge 1
reading 70 nodes.
reading 293 edges.
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288_1_r1706264427_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497418
read mu: 0.988462
read mu_delta: 0.942832
read initial_learning_rate: 0.00105251
read learning_rate: 9.93126e-07
read learning_rate_delta: 0.931387
read initial_weight_decay: 0.000490441
read weight_decay: 1.43945e-06
read weight_decay_delta: 0.942235
read batch_size: 85
read epsilon: 1e-07
read alpha: 0.0831412
read input_dropout_probability: 0.00121737
read hidden_dropout_probability: 0.0984132
read velocity_reset: 1592
read epoch: 98
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6477
read best_validation_error: 10117
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6288
read normal distribution: '1 0 0'
generator_str: '1398929753'
read generator: 1398929753
read generated_by_map:
1 disable_edge 1
reading 70 nodes.
reading 293 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288_1_r1706264427_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497418
read mu: 0.988462
read mu_delta: 0.942832
read initial_learning_rate: 0.00105251
read learning_rate: 9.93126e-07
read learning_rate_delta: 0.931387
read initial_weight_decay: 0.000490441
read weight_decay: 1.43945e-06
read weight_decay_delta: 0.942235
read batch_size: 85
read epsilon: 1e-07
read alpha: 0.0831412
read input_dropout_probability: 0.00121737
read hidden_dropout_probability: 0.0984132
read velocity_reset: 1592
read epoch: 98
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6477
read best_validation_error: 10117
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6288
read normal distribution: '1 0 0'
generator_str: '1398929753'
read generator: 1398929753
read generated_by_map:
1 disable_edge 1
reading 70 nodes.
reading 293 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_188_6288_1_r1706264427_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.497418
read mu: 0.988462
read mu_delta: 0.942832
read initial_learning_rate: 0.00105251
read learning_rate: 9.93126e-07
read learning_rate_delta: 0.931387
read initial_weight_decay: 0.000490441
read weight_decay: 1.43945e-06
read weight_decay_delta: 0.942235
read batch_size: 85
read epsilon: 1e-07
read alpha: 0.0831412
read input_dropout_probability: 0.00121737
read hidden_dropout_probability: 0.0984132
read velocity_reset: 1592
read epoch: 98
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6477
read best_validation_error: 10117
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6288
read normal distribution: '1 0 0'
generator_str: '1398929753'
read generator: 1398929753
read generated_by_map:
1 disable_edge 1
reading 70 nodes.
reading 293 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 900.6220703125s, input_fired_time: 12.7810440063477, output_fired_time: 5.47359895706177, propagate_forward_time: 341.070770263672, propagate_backward_time: 531.284729003906, weight_update_time: 0.351956844329834, other_time: 10.0119018554688
epoch time: 70.6741561889648s, input_fired_time: 0.901869356632233, output_fired_time: 0, propagate_forward_time: 68.0865859985352, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.68569946289063
validation[          , genome  6288] predictions:    6470/  10000 (64.70%), best:    6477/10000 (64.77%), error:     10121.93066, best error:     10117.01953 on epoch:    97, epoch:   98/100, mu: 0.9884616137, learning_rate: 0.0000009931, weight_decay: 0.0000014395

epoch time: 903.8664550781s, input_fired_time: 12.8197212219, output_fired_time: 5.4820165634, propagate_forward_time: 341.3661499023, propagate_backward_time: 534.1422729492, weight_update_time: 0.3596919775, other_time: 10.0563354492
epoch time: 69.7328033447s, input_fired_time: 0.8946128488, output_fired_time: 0.0000000000, propagate_forward_time: 67.2309722900, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6072158813
validation[          , genome  6288] predictions:    6471/  10000 (64.71%), best:    6477/10000 (64.77%), error:     10126.81250, best error:     10117.01953 on epoch:    97, epoch:   99/100, mu: 0.9885495305, learning_rate: 0.0000009250, weight_decay: 0.0000013563

epoch time: 893.6853027344s, input_fired_time: 12.7524814606, output_fired_time: 5.4175105095, propagate_forward_time: 338.1909790039, propagate_backward_time: 527.3533325195, weight_update_time: 0.3503373563, other_time: 9.9710083008
epoch time: 70.3585739136s, input_fired_time: 0.8953251243, output_fired_time: 0.0000000000, propagate_forward_time: 67.7939605713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6692886353
validation[          , genome  6288] predictions:    6464/  10000 (64.64%), best:    6464/10000 (64.64%), error:     10116.12500, best error:     10116.12500 on epoch:   100, epoch:  100/100, mu: 0.9886324406, learning_rate: 0.0000008615, weight_decay: 0.0000012780

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 351.7650146484s, input_fired_time: 4.4933886528, output_fired_time: 0.0000000000, propagate_forward_time: 338.9364624023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.3351745605
best training[          , genome  6288] predictions:   35465/  50000 (70.93%), best:    6464/10000 (64.64%), error:     43865.54297, best error:     10116.12500 on epoch:   100, epoch:  101/100, mu: 0.9887106419, learning_rate: 0.0000008024, weight_decay: 0.0000012041
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 70.2638626099s, input_fired_time: 0.8966711164, output_fired_time: 0.0000000000, propagate_forward_time: 67.7081146240, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6590805054
   testing[          , genome  6288] predictions:    6464/  10000 (64.64%), best:    6464/10000 (64.64%), error:     10116.12500, best error:     10116.12500 on epoch:   100, epoch:  101/100, mu: 0.9887106419, learning_rate: 0.0000008024, weight_decay: 0.0000012041
backpropagation finished successfully!
15:26:41 (3528): called boinc_finish(0)

</stderr_txt>
]]>