Nameexact_genome_1542031203_181_5606_1
Workunit4081696
Created12 Nov 2018, 19:22:47 UTC
Sent12 Nov 2018, 19:27:30 UTC
Report deadline1 Jan 2019, 17:31:31 UTC
Received10 Dec 2018, 12:47:35 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64195
Run time1 days 1 hours 5 min 38 sec
CPU time1 days 0 hours 57 min 24 sec
Validate stateValid
Credit12,479.87
Device peak FLOPS3.83 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size545.94 MB
Peak swap size565.50 MB
Peak disk usage10.91 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606_1_r771452320_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477976
read mu: 0.477976
read mu_delta: 0.945824
read initial_learning_rate: 0.00183238
read learning_rate: 0.00183238
read learning_rate_delta: 0.927727
read initial_weight_decay: 0.000581555
read weight_decay: 0.000581555
read weight_decay_delta: 0.935276
read batch_size: 96
read epsilon: 1e-07
read alpha: 0.0998313
read input_dropout_probability: 0.00117304
read hidden_dropout_probability: 0.0962418
read velocity_reset: 1621
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5606
read normal distribution: '1 -0.194099321961403 1.22166979312897'
generator_str: '590723349'
read generator: 590723349
read generated_by_map:
1 change_size_x 1
reading 87 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 470426562
post shuffle 1: 578250556
epoch time: 2301.05615234375s, input_fired_time: 21.0771102905273, output_fired_time: 6.56526041030884, propagate_forward_time: 650.352600097656, propagate_backward_time: 1601.31774902344, weight_update_time: 0.340249836444855, other_time: 21.7435302734375
epoch time: 93.3408737182617s, input_fired_time: 1.0497590303421, output_fired_time: 0, propagate_forward_time: 90.1901473999023, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.10096740722656
validation[          , genome  5606] predictions:    5434/  10000 (54.34%), best:    5434/10000 (54.34%), error:     13380.39746, best error:     13380.39746 on epoch:     0, epoch:    0/50, mu: 0.4779757559, learning_rate: 0.0018323843, weight_decay: 0.0005815554

epoch time: 1221.1505126953s, input_fired_time: 16.3317031860, output_fired_time: 6.5187077522, propagate_forward_time: 454.0229797363, propagate_backward_time: 732.0596313477, weight_update_time: 0.3355985284, other_time: 12.2175903320
epoch time: 90.7190628052s, input_fired_time: 1.0493794680, output_fired_time: 0.0000000000, propagate_forward_time: 87.5255050659, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1441802979
validation[          , genome  5606] predictions:    5536/  10000 (55.36%), best:    5536/10000 (55.36%), error:     12939.36426, best error:     12939.36426 on epoch:     1, epoch:    1/50, mu: 0.5057151914, learning_rate: 0.0016999528, weight_decay: 0.0005439149

epoch time: 1233.2019042969s, input_fired_time: 16.4289970398, output_fired_time: 6.5443053246, propagate_forward_time: 463.1257324219, propagate_backward_time: 734.9242553711, weight_update_time: 0.3347846866, other_time: 12.1786499023
epoch time: 87.1785125732s, input_fired_time: 0.9888668060, output_fired_time: 0.0000000000, propagate_forward_time: 84.2268981934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9627456665
validation[          , genome  5606] predictions:    5359/  10000 (53.59%), best:    5536/10000 (55.36%), error:     13158.61621, best error:     12939.36426 on epoch:     1, epoch:    2/50, mu: 0.5319518447, learning_rate: 0.0015770925, weight_decay: 0.0005087106

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606_1_r771452320_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477976
read mu: 0.556767
read mu_delta: 0.945824
read initial_learning_rate: 0.00183238
read learning_rate: 0.00146311
read learning_rate_delta: 0.927727
read initial_weight_decay: 0.000581555
read weight_decay: 0.000475785
read weight_decay_delta: 0.935276
read batch_size: 96
read epsilon: 1e-07
read alpha: 0.0998313
read input_dropout_probability: 0.00117304
read hidden_dropout_probability: 0.0962418
read velocity_reset: 1621
read epoch: 3
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 1
read number_validation_images: 10000
read best_validation_predictions: 5536
read best_validation_error: 12939.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5606
read normal distribution: '1 -0.194099321961403 1.22166979312897'
generator_str: '254435076'
read generator: 254435076
read generated_by_map:
1 change_size_x 1
reading 87 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1709.716796875s, input_fired_time: 17.4889183044434, output_fired_time: 8.44825649261475, propagate_forward_time: 658.494079589844, propagate_backward_time: 1010.72131347656, weight_update_time: 0.433922111988068, other_time: 14.5643310546875
epoch time: 134.449508666992s, input_fired_time: 1.28514134883881, output_fired_time: 0, propagate_forward_time: 130.704803466797, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.45956420898438
validation[          , genome  5606] predictions:    5490/  10000 (54.90%), best:    5490/10000 (54.90%), error:     12929.40137, best error:     12929.40137 on epoch:     3, epoch:    3/50, mu: 0.5567671061, learning_rate: 0.0014631116, weight_decay: 0.0004757849

epoch time: 1717.5753173828s, input_fired_time: 17.4377269745, output_fired_time: 8.4502105713, propagate_forward_time: 663.3369140625, propagate_backward_time: 1014.0689086914, weight_update_time: 0.4296812117, other_time: 14.2815551758
epoch time: 136.3585052490s, input_fired_time: 1.2942395210, output_fired_time: 0.0000000000, propagate_forward_time: 132.5820617676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4822082520
validation[          , genome  5606] predictions:    5383/  10000 (53.83%), best:    5490/10000 (54.90%), error:     13110.48730, best error:     12929.40137 on epoch:     3, epoch:    4/50, mu: 0.5802379251, learning_rate: 0.0013573684, weight_decay: 0.0004449902

epoch time: 1671.8752441406s, input_fired_time: 17.2667102814, output_fired_time: 8.1947803497, propagate_forward_time: 642.4080200195, propagate_backward_time: 989.9365844727, weight_update_time: 0.4237381816, other_time: 14.0690917969
epoch time: 130.2967224121s, input_fired_time: 1.2566200495, output_fired_time: 0.0000000000, propagate_forward_time: 126.7012786865, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3388214111
validation[          , genome  5606] predictions:    5522/  10000 (55.22%), best:    5522/10000 (55.22%), error:     12783.11328, best error:     12783.11328 on epoch:     5, epoch:    5/50, mu: 0.6024371982, learning_rate: 0.0012592677, weight_decay: 0.0004161888

epoch time: 1607.2832031250s, input_fired_time: 16.9570579529, output_fired_time: 8.0479297638, propagate_forward_time: 613.5932617188, propagate_backward_time: 955.1738281250, weight_update_time: 0.4055660069, other_time: 13.5111083984
epoch time: 128.0721282959s, input_fired_time: 1.2444356680, output_fired_time: 0.0000000000, propagate_forward_time: 124.4695510864, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3581390381
validation[          , genome  5606] predictions:    5487/  10000 (54.87%), best:    5522/10000 (55.22%), error:     12819.09082, best error:     12783.11328 on epoch:     5, epoch:    6/50, mu: 0.6234338284, learning_rate: 0.0011682569, weight_decay: 0.0003892514

epoch time: 1540.5585937500s, input_fired_time: 16.6751651764, output_fired_time: 7.7104668617, propagate_forward_time: 589.5538940430, propagate_backward_time: 913.5228881836, weight_update_time: 0.3917646706, other_time: 13.0961914063
epoch time: 119.9957427979s, input_fired_time: 1.1802788973, output_fired_time: 0.0000000000, propagate_forward_time: 116.6533432007, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1621170044
validation[          , genome  5606] predictions:    5560/  10000 (55.60%), best:    5560/10000 (55.60%), error:     12718.24512, best error:     12718.24512 on epoch:     7, epoch:    7/50, mu: 0.6432929039, learning_rate: 0.0010838237, weight_decay: 0.0003640576

epoch time: 1490.9855957031s, input_fired_time: 16.4860000610, output_fired_time: 7.4609003067, propagate_forward_time: 564.3632202148, propagate_backward_time: 890.0256958008, weight_update_time: 0.3775932789, other_time: 12.6497802734
epoch time: 115.4421539307s, input_fired_time: 1.1425989866, output_fired_time: 0.0000000000, propagate_forward_time: 112.1473693848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1521835327
validation[          , genome  5606] predictions:    5645/  10000 (56.45%), best:    5645/10000 (56.45%), error:     12470.06836, best error:     12470.06836 on epoch:     8, epoch:    8/50, mu: 0.6620761156, learning_rate: 0.0010054928, weight_decay: 0.0003404944

epoch time: 1497.8616943359s, input_fired_time: 16.5480556488, output_fired_time: 7.5284576416, propagate_forward_time: 572.3994140625, propagate_backward_time: 888.4609375000, weight_update_time: 0.3839237094, other_time: 12.9248046875
epoch time: 114.9349899292s, input_fired_time: 1.1387063265, output_fired_time: 0.0000000000, propagate_forward_time: 111.6586151123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1376647949
validation[          , genome  5606] predictions:    5707/  10000 (57.07%), best:    5707/10000 (57.07%), error:     12439.19238, best error:     12439.19238 on epoch:     9, epoch:    9/50, mu: 0.6798417568, learning_rate: 0.0009328230, weight_decay: 0.0003184562

epoch time: 1503.6025390625s, input_fired_time: 16.5666999817, output_fired_time: 7.5312790871, propagate_forward_time: 568.6713256836, propagate_backward_time: 897.9789428711, weight_update_time: 0.3820771277, other_time: 12.8543701172
epoch time: 120.0958709717s, input_fired_time: 1.1799145937, output_fired_time: 0.0000000000, propagate_forward_time: 116.7313919067, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1845626831
validation[          , genome  5606] predictions:    5630/  10000 (56.30%), best:    5707/10000 (57.07%), error:     12527.04980, best error:     12439.19238 on epoch:     9, epoch:   10/50, mu: 0.6966449022, learning_rate: 0.0008654053, weight_decay: 0.0002978445

epoch time: 1436.6402587891s, input_fired_time: 16.3372650146, output_fired_time: 7.1320123672, propagate_forward_time: 547.1912841797, propagate_backward_time: 853.3678588867, weight_update_time: 0.3701523840, other_time: 12.6118774414
epoch time: 109.1918716431s, input_fired_time: 1.1001729965, output_fired_time: 0.0000000000, propagate_forward_time: 105.9876403809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1040573120
validation[          , genome  5606] predictions:    5462/  10000 (54.62%), best:    5707/10000 (57.07%), error:     13257.01563, best error:     12439.19238 on epoch:     9, epoch:   11/50, mu: 0.7125377059, learning_rate: 0.0008028601, weight_decay: 0.0002785669

epoch time: 1439.6403808594s, input_fired_time: 16.3652267456, output_fired_time: 7.2915449142, propagate_forward_time: 541.4099731445, propagate_backward_time: 862.1973876953, weight_update_time: 0.3610005975, other_time: 12.3762817383
epoch time: 115.7886734009s, input_fired_time: 1.1655055285, output_fired_time: 0.0000000000, propagate_forward_time: 112.4263381958, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1968307495
validation[          , genome  5606] predictions:    5665/  10000 (56.65%), best:    5665/10000 (56.65%), error:     12256.90918, best error:     12256.90918 on epoch:    12, epoch:   12/50, mu: 0.7275695205, learning_rate: 0.0007448351, weight_decay: 0.0002605370

epoch time: 1459.6276855469s, input_fired_time: 16.4579944611, output_fired_time: 7.4144353867, propagate_forward_time: 553.0103759766, propagate_backward_time: 869.9224853516, weight_update_time: 0.3778791726, other_time: 12.8223876953
epoch time: 117.8246917725s, input_fired_time: 1.1772217751, output_fired_time: 0.0000000000, propagate_forward_time: 114.4341506958, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2133178711
validation[          , genome  5606] predictions:    5709/  10000 (57.09%), best:    5665/10000 (56.65%), error:     12397.39746, best error:     12256.90918 on epoch:    12, epoch:   13/50, mu: 0.7417869568, learning_rate: 0.0006910038, weight_decay: 0.0002436740

epoch time: 1496.8601074219s, input_fired_time: 16.5954704285, output_fired_time: 7.4842753410, propagate_forward_time: 567.6569213867, propagate_backward_time: 892.2320556641, weight_update_time: 0.3804390132, other_time: 12.8914184570
epoch time: 121.7179489136s, input_fired_time: 1.2107366323, output_fired_time: 0.0000000000, propagate_forward_time: 118.1694717407, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3377380371
validation[          , genome  5606] predictions:    5766/  10000 (57.66%), best:    5766/10000 (57.66%), error:     12161.35547, best error:     12161.35547 on epoch:    14, epoch:   14/50, mu: 0.7552341819, learning_rate: 0.0006410630, weight_decay: 0.0002279025

epoch time: 1524.2580566406s, input_fired_time: 16.7074947357, output_fired_time: 7.6299409866, propagate_forward_time: 578.1246337891, propagate_backward_time: 908.7583618164, weight_update_time: 0.3871471882, other_time: 13.0376586914
epoch time: 120.5212554932s, input_fired_time: 1.2081267834, output_fired_time: 0.0000000000, propagate_forward_time: 117.0783157349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2348098755
validation[          , genome  5606] predictions:    5698/  10000 (56.98%), best:    5698/10000 (56.98%), error:     12104.33594, best error:     12104.33594 on epoch:    15, epoch:   15/50, mu: 0.7679528594, learning_rate: 0.0005947317, weight_decay: 0.0002131518

epoch time: 1488.5169677734s, input_fired_time: 16.5393295288, output_fired_time: 7.4522290230, propagate_forward_time: 565.6710205078, propagate_backward_time: 886.1582031250, weight_update_time: 0.3794448972, other_time: 12.6961669922
epoch time: 113.9486694336s, input_fired_time: 1.1521247625, output_fired_time: 0.0000000000, propagate_forward_time: 110.6730422974, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1235046387
validation[          , genome  5606] predictions:    5846/  10000 (58.46%), best:    5846/10000 (58.46%), error:     12032.70410, best error:     12032.70410 on epoch:    16, epoch:   16/50, mu: 0.7799825072, learning_rate: 0.0005517487, weight_decay: 0.0001993558

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606_1_r771452320_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477976
read mu: 0.79136
read mu_delta: 0.945824
read initial_learning_rate: 0.00183238
read learning_rate: 0.000511872
read learning_rate_delta: 0.927727
read initial_weight_decay: 0.000581555
read weight_decay: 0.000186453
read weight_decay_delta: 0.935276
read batch_size: 96
read epsilon: 1e-07
read alpha: 0.0998313
read input_dropout_probability: 0.00117304
read hidden_dropout_probability: 0.0962418
read velocity_reset: 1621
read epoch: 17
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 16
read number_validation_images: 10000
read best_validation_predictions: 5846
read best_validation_error: 12032.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5606
read normal distribution: '1 -0.194099321961403 1.22166979312897'
generator_str: '1110570413'
read generator: 1110570413
read generated_by_map:
1 change_size_x 1
reading 87 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1843.5087890625s, input_fired_time: 43.5860862731934, output_fired_time: 29.1091842651367, propagate_forward_time: 646.922424316406, propagate_backward_time: 1049.56323242188, weight_update_time: 2.23745322227478, other_time: 74.327880859375
epoch time: 141.039031982422s, input_fired_time: 5.46732473373413, output_fired_time: 0, propagate_forward_time: 121.596176147461, propagate_backward_time: 0, weight_update_time: 0, other_time: 13.9755249023438
validation[          , genome  5606] predictions:    5799/  10000 (57.99%), best:    5799/10000 (57.99%), error:     12012.41602, best error:     12012.41602 on epoch:    17, epoch:   17/50, mu: 0.7913604379, learning_rate: 0.0005118723, weight_decay: 0.0001864527

epoch time: 1666.5610351563s, input_fired_time: 41.5097656250, output_fired_time: 27.5462341309, propagate_forward_time: 576.6187133789, propagate_backward_time: 947.6528320313, weight_update_time: 2.1586260796, other_time: 73.2334594727
epoch time: 132.3477478027s, input_fired_time: 5.2588090897, output_fired_time: 0.0000000000, propagate_forward_time: 113.5630950928, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.5258407593
validation[          , genome  5606] predictions:    5911/  10000 (59.11%), best:    5911/10000 (59.11%), error:     11814.59961, best error:     11814.59961 on epoch:    18, epoch:   18/50, mu: 0.8021219373, learning_rate: 0.0004748779, weight_decay: 0.0001743848

epoch time: 1701.0672607422s, input_fired_time: 41.6199188232, output_fired_time: 27.7332401276, propagate_forward_time: 589.8996582031, propagate_backward_time: 966.8053588867, weight_update_time: 2.1768624783, other_time: 75.0090942383
epoch time: 130.3373870850s, input_fired_time: 5.1250495911, output_fired_time: 0.0000000000, propagate_forward_time: 112.3958969116, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.8164443970
validation[          , genome  5606] predictions:    5891/  10000 (58.91%), best:    5891/10000 (58.91%), error:     11799.07422, best error:     11799.07422 on epoch:    19, epoch:   19/50, mu: 0.8123004436, learning_rate: 0.0004405571, weight_decay: 0.0001630979

epoch time: 1637.7808837891s, input_fired_time: 41.0403480530, output_fired_time: 27.3866844177, propagate_forward_time: 572.3452758789, propagate_backward_time: 925.3248901367, weight_update_time: 2.1333734989, other_time: 71.6835937500
epoch time: 136.7595062256s, input_fired_time: 5.3202443123, output_fired_time: 0.0000000000, propagate_forward_time: 117.4458541870, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.9934005737
validation[          , genome  5606] predictions:    5774/  10000 (57.74%), best:    5891/10000 (58.91%), error:     12114.99609, best error:     11799.07422 on epoch:    19, epoch:   20/50, mu: 0.8219274879, learning_rate: 0.0004087168, weight_decay: 0.0001525416

epoch time: 1679.8601074219s, input_fired_time: 42.1334342957, output_fired_time: 28.0237274170, propagate_forward_time: 587.7316894531, propagate_backward_time: 948.1898193359, weight_update_time: 2.1967155933, other_time: 73.7814941406
epoch time: 141.3876647949s, input_fired_time: 5.5211486816, output_fired_time: 0.0000000000, propagate_forward_time: 121.5962142944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 14.2703018188
validation[          , genome  5606] predictions:    5872/  10000 (58.72%), best:    5872/10000 (58.72%), error:     11751.70801, best error:     11751.70801 on epoch:    21, epoch:   21/50, mu: 0.8310329914, learning_rate: 0.0003791777, weight_decay: 0.0001426685

epoch time: 1676.1978759766s, input_fired_time: 42.1681900024, output_fired_time: 28.1787319183, propagate_forward_time: 585.5718383789, propagate_backward_time: 947.6837768555, weight_update_time: 2.1203615665, other_time: 72.5953369141
epoch time: 125.2844390869s, input_fired_time: 4.9861130714, output_fired_time: 0.0000000000, propagate_forward_time: 108.9496002197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.3487243652
validation[          , genome  5606] predictions:    5826/  10000 (58.26%), best:    5872/10000 (58.72%), error:     12096.88672, best error:     11751.70801 on epoch:    21, epoch:   22/50, mu: 0.8396452069, learning_rate: 0.0003517735, weight_decay: 0.0001334345

epoch time: 1690.2828369141s, input_fired_time: 42.5241165161, output_fired_time: 28.2738475800, propagate_forward_time: 588.2572021484, propagate_backward_time: 956.6762084961, weight_update_time: 2.1616091728, other_time: 74.5514526367
epoch time: 137.2316589355s, input_fired_time: 5.2677574158, output_fired_time: 0.0000000000, propagate_forward_time: 118.1127243042, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.8511734009
validation[          , genome  5606] predictions:    5899/  10000 (58.99%), best:    5872/10000 (58.72%), error:     11798.62109, best error:     11751.70801 on epoch:    21, epoch:   23/50, mu: 0.8477908373, learning_rate: 0.0003263498, weight_decay: 0.0001247981

epoch time: 1780.9047851563s, input_fired_time: 43.8329963684, output_fired_time: 28.4598064423, propagate_forward_time: 628.3512573242, propagate_backward_time: 1003.3132934570, weight_update_time: 2.2507882118, other_time: 76.9474487305
epoch time: 146.9025268555s, input_fired_time: 5.7334828377, output_fired_time: 0.0000000000, propagate_forward_time: 127.5905914307, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.5784454346
validation[          , genome  5606] predictions:    5896/  10000 (58.96%), best:    5896/10000 (58.96%), error:     11702.98926, best error:     11702.98926 on epoch:    24, epoch:   24/50, mu: 0.8554951549, learning_rate: 0.0003027636, weight_decay: 0.0001167207

epoch time: 1816.2304687500s, input_fired_time: 44.2869987488, output_fired_time: 28.8313407898, propagate_forward_time: 639.8449707031, propagate_backward_time: 1025.8542480469, weight_update_time: 2.2210247517, other_time: 77.4129638672
epoch time: 148.4236755371s, input_fired_time: 5.9254260063, output_fired_time: 0.0000000000, propagate_forward_time: 128.0376281738, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 14.4606170654
validation[          , genome  5606] predictions:    5891/  10000 (58.91%), best:    5896/10000 (58.96%), error:     11767.53223, best error:     11702.98926 on epoch:    24, epoch:   25/50, mu: 0.8627821207, learning_rate: 0.0002808821, weight_decay: 0.0001091660

epoch time: 1967.6049804688s, input_fired_time: 45.9717407227, output_fired_time: 29.3507518768, propagate_forward_time: 697.8802490234, propagate_backward_time: 1116.0744628906, weight_update_time: 2.2430369854, other_time: 78.3278808594
epoch time: 182.4610595703s, input_fired_time: 7.1818037033, output_fired_time: 0.0000000000, propagate_forward_time: 156.6245422363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 18.6547088623
validation[          , genome  5606] predictions:    5939/  10000 (59.39%), best:    5939/10000 (59.39%), error:     11663.04688, best error:     11663.04688 on epoch:    26, epoch:   26/50, mu: 0.8696742654, learning_rate: 0.0002605819, weight_decay: 0.0001021004

epoch time: 2149.7365722656s, input_fired_time: 49.5982017517, output_fired_time: 31.4939384460, propagate_forward_time: 765.8593139648, propagate_backward_time: 1214.1384277344, weight_update_time: 2.3480272293, other_time: 88.6467285156
epoch time: 181.9429473877s, input_fired_time: 6.7682781219, output_fired_time: 0.0000000000, propagate_forward_time: 155.7025299072, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.4721374512
validation[          , genome  5606] predictions:    6029/  10000 (60.29%), best:    6029/10000 (60.29%), error:     11473.71289, best error:     11473.71289 on epoch:    27, epoch:   27/50, mu: 0.8761930466, learning_rate: 0.0002417489, weight_decay: 0.0000954921

epoch time: 2166.3498535156s, input_fired_time: 49.7861289978, output_fired_time: 31.7730846405, propagate_forward_time: 768.3190917969, propagate_backward_time: 1223.8725585938, weight_update_time: 2.4521572590, other_time: 92.5988769531
epoch time: 186.9872283936s, input_fired_time: 7.3424844742, output_fired_time: 0.0000000000, propagate_forward_time: 158.1945953369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 21.4501495361
validation[          , genome  5606] predictions:    5948/  10000 (59.48%), best:    6029/10000 (60.29%), error:     11570.71680, best error:     11473.71289 on epoch:    27, epoch:   28/50, mu: 0.8823586702, learning_rate: 0.0002242771, weight_decay: 0.0000893115

epoch time: 2116.8264160156s, input_fired_time: 47.1928977966, output_fired_time: 29.8089256287, propagate_forward_time: 756.4672241211, propagate_backward_time: 1198.9400634766, weight_update_time: 2.2851445675, other_time: 84.4173583984
epoch time: 180.8948669434s, input_fired_time: 6.3124952316, output_fired_time: 0.0000000000, propagate_forward_time: 156.3740386963, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 18.2083282471
validation[          , genome  5606] predictions:    6091/  10000 (60.91%), best:    6091/10000 (60.91%), error:     11357.08496, best error:     11357.08496 on epoch:    29, epoch:   29/50, mu: 0.8881902695, learning_rate: 0.0002080679, weight_decay: 0.0000835309

epoch time: 2101.0354003906s, input_fired_time: 44.0195770264, output_fired_time: 27.5728950500, propagate_forward_time: 759.5267944336, propagate_backward_time: 1194.2958984375, weight_update_time: 2.1115286350, other_time: 75.6203613281
epoch time: 172.7615509033s, input_fired_time: 5.5515079498, output_fired_time: 0.0000000000, propagate_forward_time: 151.6919250488, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 15.5181121826
validation[          , genome  5606] predictions:    6000/  10000 (60.00%), best:    6091/10000 (60.91%), error:     11568.55273, best error:     11357.08496 on epoch:    29, epoch:   30/50, mu: 0.8937059045, learning_rate: 0.0001930303, weight_decay: 0.0000781244

epoch time: 1963.6424560547s, input_fired_time: 39.0179748535, output_fired_time: 24.3677387238, propagate_forward_time: 716.7784423828, propagate_backward_time: 1121.1170654297, weight_update_time: 1.8299094439, other_time: 62.3612060547
epoch time: 158.0815582275s, input_fired_time: 4.6280550957, output_fired_time: 0.0000000000, propagate_forward_time: 142.6534881592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.8000183105
validation[          , genome  5606] predictions:    5988/  10000 (59.88%), best:    6091/10000 (60.91%), error:     11535.24023, best error:     11357.08496 on epoch:    29, epoch:   31/50, mu: 0.8989227414, learning_rate: 0.0001790795, weight_decay: 0.0000730679

epoch time: 1974.1483154297s, input_fired_time: 37.9287223816, output_fired_time: 23.2640419006, propagate_forward_time: 723.4241943359, propagate_backward_time: 1129.6301269531, weight_update_time: 1.7748644352, other_time: 59.9012451172
epoch time: 165.2575073242s, input_fired_time: 4.8238477707, output_fired_time: 0.0000000000, propagate_forward_time: 147.8398742676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.5937805176
validation[          , genome  5606] predictions:    6029/  10000 (60.29%), best:    6091/10000 (60.91%), error:     11370.29199, best error:     11357.08496 on epoch:    29, epoch:   32/50, mu: 0.9038569331, learning_rate: 0.0001661369, weight_decay: 0.0000683387

epoch time: 1993.0456542969s, input_fired_time: 36.2885780334, output_fired_time: 22.2169075012, propagate_forward_time: 733.1268920898, propagate_backward_time: 1141.4510498047, weight_update_time: 1.6719473600, other_time: 59.9622802734
epoch time: 158.1741180420s, input_fired_time: 4.1730213165, output_fired_time: 0.0000000000, propagate_forward_time: 144.8177490234, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.1833496094
validation[          , genome  5606] predictions:    6046/  10000 (60.46%), best:    6046/10000 (60.46%), error:     11305.41992, best error:     11305.41992 on epoch:    33, epoch:   33/50, mu: 0.9085237980, learning_rate: 0.0001541297, weight_decay: 0.0000639155

epoch time: 1816.3571777344s, input_fired_time: 25.6507568359, output_fired_time: 14.1987581253, propagate_forward_time: 683.4896850586, propagate_backward_time: 1058.5905761719, weight_update_time: 0.9562860727, other_time: 34.4274902344
epoch time: 139.0238494873s, input_fired_time: 1.4153970480, output_fired_time: 0.0000000000, propagate_forward_time: 134.2794036865, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3290557861
validation[          , genome  5606] predictions:    6077/  10000 (60.77%), best:    6046/10000 (60.46%), error:     11311.67773, best error:     11305.41992 on epoch:    33, epoch:   34/50, mu: 0.9129378796, learning_rate: 0.0001429903, weight_decay: 0.0000597787

epoch time: 1747.8736572266s, input_fired_time: 18.3286457062, output_fired_time: 8.9116096497, propagate_forward_time: 670.1160888672, propagate_backward_time: 1031.6341552734, weight_update_time: 0.4897831976, other_time: 18.8831787109
epoch time: 140.1853942871s, input_fired_time: 1.4444251060, output_fired_time: 0.0000000000, propagate_forward_time: 135.1959228516, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5450439453
validation[          , genome  5606] predictions:    6049/  10000 (60.49%), best:    6049/10000 (60.49%), error:     11254.60645, best error:     11254.60645 on epoch:    35, epoch:   35/50, mu: 0.9171128273, learning_rate: 0.0001326560, weight_decay: 0.0000559096

epoch time: 1718.8656005859s, input_fired_time: 18.2115688324, output_fired_time: 8.7072143555, propagate_forward_time: 656.9343872070, propagate_backward_time: 1016.1525878906, weight_update_time: 0.4913649559, other_time: 18.8598632813
epoch time: 138.5127410889s, input_fired_time: 1.4307507277, output_fired_time: 0.0000000000, propagate_forward_time: 133.7764129639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3055725098
validation[          , genome  5606] predictions:    6076/  10000 (60.76%), best:    6076/10000 (60.76%), error:     11247.10840, best error:     11247.10840 on epoch:    36, epoch:   36/50, mu: 0.9210615754, learning_rate: 0.0001230686, weight_decay: 0.0000522909

epoch time: 1677.1856689453s, input_fired_time: 17.6334629059, output_fired_time: 8.2837114334, propagate_forward_time: 646.3593750000, propagate_backward_time: 989.9349365234, weight_update_time: 0.4323834777, other_time: 14.9742431641
epoch time: 125.8275985718s, input_fired_time: 1.2613867521, output_fired_time: 0.0000000000, propagate_forward_time: 122.4224319458, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1437835693
validation[          , genome  5606] predictions:    5989/  10000 (59.89%), best:    6076/10000 (60.76%), error:     11390.64648, best error:     11247.10840 on epoch:    36, epoch:   37/50, mu: 0.9247964025, learning_rate: 0.0001141741, weight_decay: 0.0000489064

epoch time: 1568.7049560547s, input_fired_time: 16.8926124573, output_fired_time: 7.5849084854, propagate_forward_time: 598.4736938477, propagate_backward_time: 933.1504516602, weight_update_time: 0.3968068957, other_time: 12.6032714844
epoch time: 120.4703750610s, input_fired_time: 1.2309086323, output_fired_time: 0.0000000000, propagate_forward_time: 117.1113815308, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1280822754
validation[          , genome  5606] predictions:    6173/  10000 (61.73%), best:    6173/10000 (61.73%), error:     11104.91895, best error:     11104.91895 on epoch:    38, epoch:   38/50, mu: 0.9283288717, learning_rate: 0.0001059224, weight_decay: 0.0000457410

epoch time: 1575.8614501953s, input_fired_time: 16.9627075195, output_fired_time: 7.6379632950, propagate_forward_time: 603.6033325195, propagate_backward_time: 935.2415161133, weight_update_time: 0.3858473599, other_time: 12.4158935547
epoch time: 127.8744735718s, input_fired_time: 1.2636471987, output_fired_time: 0.0000000000, propagate_forward_time: 124.4811325073, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1296920776
validation[          , genome  5606] predictions:    6122/  10000 (61.22%), best:    6173/10000 (61.73%), error:     11164.09375, best error:     11104.91895 on epoch:    38, epoch:   39/50, mu: 0.9316699505, learning_rate: 0.0000982671, weight_decay: 0.0000427805

epoch time: 1612.3680419922s, input_fired_time: 17.0853309631, output_fired_time: 7.7342357635, propagate_forward_time: 618.4275512695, propagate_backward_time: 956.4124145508, weight_update_time: 0.4004462957, other_time: 12.7084960938
epoch time: 128.2357940674s, input_fired_time: 1.2819774151, output_fired_time: 0.0000000000, propagate_forward_time: 124.8194732666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1343460083
validation[          , genome  5606] predictions:    6139/  10000 (61.39%), best:    6139/10000 (61.39%), error:     11102.71191, best error:     11102.71191 on epoch:    40, epoch:   40/50, mu: 0.9348300695, learning_rate: 0.0000911651, weight_decay: 0.0000400116

epoch time: 1600.0622558594s, input_fired_time: 17.0631637573, output_fired_time: 7.6645426750, propagate_forward_time: 613.5783691406, propagate_backward_time: 949.1555175781, weight_update_time: 0.3983497322, other_time: 12.6007080078
epoch time: 127.3355712891s, input_fired_time: 1.2794123888, output_fired_time: 0.0000000000, propagate_forward_time: 123.9073410034, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1488189697
validation[          , genome  5606] predictions:    6168/  10000 (61.68%), best:    6139/10000 (61.39%), error:     11122.86816, best error:     11102.71191 on epoch:    40, epoch:   41/50, mu: 0.9378189445, learning_rate: 0.0000845763, weight_decay: 0.0000374219

epoch time: 1591.9361572266s, input_fired_time: 17.0329265594, output_fired_time: 7.5992155075, propagate_forward_time: 612.1014404297, propagate_backward_time: 942.6265258789, weight_update_time: 0.3968549073, other_time: 12.5759887695
epoch time: 124.5714645386s, input_fired_time: 1.2610783577, output_fired_time: 0.0000000000, propagate_forward_time: 121.2018966675, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1084899902
validation[          , genome  5606] predictions:    6207/  10000 (62.07%), best:    6207/10000 (62.07%), error:     11045.45313, best error:     11045.45313 on epoch:    42, epoch:   42/50, mu: 0.9406459332, learning_rate: 0.0000784638, weight_decay: 0.0000349998

epoch time: 1526.6174316406s, input_fired_time: 16.8484554291, output_fired_time: 7.4769072533, propagate_forward_time: 583.6781616211, propagate_backward_time: 906.3440551758, weight_update_time: 0.3798328042, other_time: 12.2697753906
epoch time: 118.9230880737s, input_fired_time: 1.2162208557, output_fired_time: 0.0000000000, propagate_forward_time: 115.6356353760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0712280273
validation[          , genome  5606] predictions:    6188/  10000 (61.88%), best:    6188/10000 (61.88%), error:     11009.85547, best error:     11009.85547 on epoch:    43, epoch:   43/50, mu: 0.9433197379, learning_rate: 0.0000727930, weight_decay: 0.0000327345

epoch time: 1524.6994628906s, input_fired_time: 16.8680248260, output_fired_time: 7.4046535492, propagate_forward_time: 590.4429931641, propagate_backward_time: 897.5965576172, weight_update_time: 0.3869578242, other_time: 12.3872070313
epoch time: 112.1388549805s, input_fired_time: 1.1696904898, output_fired_time: 0.0000000000, propagate_forward_time: 108.9545288086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0146331787
validation[          , genome  5606] predictions:    6165/  10000 (61.65%), best:    6188/10000 (61.88%), error:     11056.41113, best error:     11009.85547 on epoch:    43, epoch:   44/50, mu: 0.9458487034, learning_rate: 0.0000675320, weight_decay: 0.0000306158

epoch time: 1524.0073242188s, input_fired_time: 16.8572978973, output_fired_time: 7.3060340881, propagate_forward_time: 582.6083984375, propagate_backward_time: 904.8225708008, weight_update_time: 0.3852506578, other_time: 12.4130249023
epoch time: 116.3637619019s, input_fired_time: 1.2042863369, output_fired_time: 0.0000000000, propagate_forward_time: 113.1098403931, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0496368408
validation[          , genome  5606] predictions:    6113/  10000 (61.13%), best:    6188/10000 (61.88%), error:     11109.70898, best error:     11009.85547 on epoch:    43, epoch:   45/50, mu: 0.9482406378, learning_rate: 0.0000626513, weight_decay: 0.0000286342

epoch time: 1525.4664306641s, input_fired_time: 16.8247375488, output_fired_time: 7.4253764153, propagate_forward_time: 581.3086547852, propagate_backward_time: 907.5374755859, weight_update_time: 0.3843747675, other_time: 12.3701782227
epoch time: 120.3474502563s, input_fired_time: 1.2353703976, output_fired_time: 0.0000000000, propagate_forward_time: 117.0412139893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0708694458
validation[          , genome  5606] predictions:    6165/  10000 (61.65%), best:    6188/10000 (61.88%), error:     11039.40820, best error:     11009.85547 on epoch:    43, epoch:   46/50, mu: 0.9505029917, learning_rate: 0.0000581233, weight_decay: 0.0000267809

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5606_1_r771452320_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477976
read mu: 0.952643
read mu_delta: 0.945824
read initial_learning_rate: 0.00183238
read learning_rate: 5.39226e-05
read learning_rate_delta: 0.927727
read initial_weight_decay: 0.000581555
read weight_decay: 2.50475e-05
read weight_decay_delta: 0.935276
read batch_size: 96
read epsilon: 1e-07
read alpha: 0.0998313
read input_dropout_probability: 0.00117304
read hidden_dropout_probability: 0.0962418
read velocity_reset: 1621
read epoch: 47
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 43
read number_validation_images: 10000
read best_validation_predictions: 6188
read best_validation_error: 11009.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5606
read normal distribution: '1 -0.194099321961403 1.22166979312897'
generator_str: '2064154919'
read generator: 2064154919
read generated_by_map:
1 change_size_x 1
reading 87 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1092.84777832031s, input_fired_time: 15.0792560577393, output_fired_time: 5.74612998962402, propagate_forward_time: 404.391052246094, propagate_backward_time: 656.365844726563, weight_update_time: 0.308522164821625, other_time: 11.2655639648438
epoch time: 107.849510192871s, input_fired_time: 1.19664561748505, output_fired_time: 0, propagate_forward_time: 104.446868896484, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.20599365234375
validation[          , genome  5606] predictions:    6112/  10000 (61.12%), best:    6188/10000 (61.88%), error:     11187.30566, best error:     11009.85547 on epoch:    43, epoch:   47/50, mu: 0.9526427984, learning_rate: 0.0000539226, weight_decay: 0.0000250475

epoch time: 1154.3922119141s, input_fired_time: 15.4223260880, output_fired_time: 6.0386142731, propagate_forward_time: 432.5273742676, propagate_backward_time: 688.8192749023, weight_update_time: 0.3218165636, other_time: 11.5846557617
epoch time: 105.8022613525s, input_fired_time: 1.1811128855, output_fired_time: 0.0000000000, propagate_forward_time: 102.4311294556, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1900177002
validation[          , genome  5606] predictions:    6243/  10000 (62.43%), best:    6243/10000 (62.43%), error:     10934.93750, best error:     10934.93750 on epoch:    48, epoch:   48/50, mu: 0.9546666741, learning_rate: 0.0000500254, weight_decay: 0.0000234263

epoch time: 1160.0134277344s, input_fired_time: 15.3415727615, output_fired_time: 6.0731868744, propagate_forward_time: 435.8439025879, propagate_backward_time: 691.2100219727, weight_update_time: 0.3221940100, other_time: 11.5447387695
epoch time: 86.7835006714s, input_fired_time: 1.0303350687, output_fired_time: 0.0000000000, propagate_forward_time: 83.7592391968, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9939270020
validation[          , genome  5606] predictions:    6197/  10000 (61.97%), best:    6243/10000 (62.43%), error:     10954.72266, best error:     10934.93750 on epoch:    48, epoch:   49/50, mu: 0.9565808773, learning_rate: 0.0000464100, weight_decay: 0.0000219101

epoch time: 1144.0209960938s, input_fired_time: 15.2132205963, output_fired_time: 5.9884185791, propagate_forward_time: 427.6304016113, propagate_backward_time: 683.7732543945, weight_update_time: 0.3167305291, other_time: 11.4157104492
epoch time: 78.8676605225s, input_fired_time: 0.9616867900, output_fired_time: 0.0000000000, propagate_forward_time: 76.0259780884, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8799972534
validation[          , genome  5606] predictions:    6174/  10000 (61.74%), best:    6243/10000 (62.43%), error:     11031.57129, best error:     10934.93750 on epoch:    48, epoch:   50/50, mu: 0.9583913684, learning_rate: 0.0000430558, weight_decay: 0.0000204920

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 445.8182983398s, input_fired_time: 5.1167469025, output_fired_time: 0.0000000000, propagate_forward_time: 431.1064758301, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.5950622559
best training[          , genome  5606] predictions:   34103/  50000 (68.21%), best:    6243/10000 (62.43%), error:     47564.18359, best error:     10934.93750 on epoch:    48, epoch:   51/50, mu: 0.9601038098, learning_rate: 0.0000399440, weight_decay: 0.0000191657
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 94.4873733521s, input_fired_time: 1.0672967434, output_fired_time: 0.0000000000, propagate_forward_time: 91.4600982666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9599761963
   testing[          , genome  5606] predictions:    6243/  10000 (62.43%), best:    6243/10000 (62.43%), error:     10934.93750, best error:     10934.93750 on epoch:    48, epoch:   51/50, mu: 0.9601038098, learning_rate: 0.0000399440, weight_decay: 0.0000191657
backpropagation finished successfully!
13:21:02 (15668): called boinc_finish(0)

</stderr_txt>
]]>