Nameexact_genome_1542649503_188_6333_0
Workunit4099354
Created20 Nov 2018, 1:51:44 UTC
Sent20 Nov 2018, 1:56:53 UTC
Report deadline19 Feb 2019, 2:07:49 UTC
Received24 Nov 2018, 9:13:35 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID78968
Run time2 days 5 hours 12 min 44 sec
CPU time2 days 4 hours 49 min 14 sec
Validate stateValid
Credit22,751.90
Device peak FLOPS2.20 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size550.47 MB
Peak swap size570.27 MB
Peak disk usage11.78 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
42 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.952388
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 9.22803e-06
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 5.9626e-06
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 60
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 58
read number_validation_images: 10000
read best_validation_predictions: 6523
read best_validation_error: 10144.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '1651536674'
read generator: 1651536674
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.952388
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 9.22803e-06
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 5.9626e-06
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 60
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 58
read number_validation_images: 10000
read best_validation_predictions: 6523
read best_validation_error: 10144.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '1651536674'
read generator: 1651536674
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1835.83581542969s, input_fired_time: 28.9538249969482, output_fired_time: 11.9520483016968, propagate_forward_time: 679.335693359375, propagate_backward_time: 1091.017578125, weight_update_time: 1.04170286655426, other_time: 24.57666015625
epoch time: 138.041687011719s, input_fired_time: 2.04590582847595, output_fired_time: 0, propagate_forward_time: 132.104415893555, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.89137268066406
validation[          , genome  6333] predictions:    6452/  10000 (64.52%), best:    6523/10000 (65.23%), error:     10278.62988, best error:     10144.44727 on epoch:    58, epoch:   60/100, mu: 0.9523883462, learning_rate: 0.0000092280, weight_decay: 0.0000059626

epoch time: 1844.6528320313s, input_fired_time: 29.6082248688, output_fired_time: 12.0633411407, propagate_forward_time: 676.0565185547, propagate_backward_time: 1102.9926757813, weight_update_time: 1.0599758625, other_time: 23.9320068359
epoch time: 143.5701141357s, input_fired_time: 2.1219460964, output_fired_time: 0.0000000000, propagate_forward_time: 137.5690307617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8791351318
validation[          , genome  6333] predictions:    6563/  10000 (65.63%), best:    6563/10000 (65.63%), error:     10111.55176, best error:     10111.55176 on epoch:    61, epoch:   61/100, mu: 0.9539574981, learning_rate: 0.0000085345, weight_decay: 0.0000055536

epoch time: 1854.0986328125s, input_fired_time: 29.2079563141, output_fired_time: 12.2038831711, propagate_forward_time: 682.7863159180, propagate_backward_time: 1106.1259765625, weight_update_time: 1.0397825241, other_time: 23.7744140625
epoch time: 145.3220825195s, input_fired_time: 2.1518411636, output_fired_time: 0.0000000000, propagate_forward_time: 139.0155029297, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1547393799
validation[          , genome  6333] predictions:    6505/  10000 (65.05%), best:    6563/10000 (65.63%), error:     10196.56934, best error:     10111.55176 on epoch:    61, epoch:   62/100, mu: 0.9554612041, learning_rate: 0.0000078930, weight_decay: 0.0000051726

epoch time: 1857.7015380859s, input_fired_time: 29.0396919250, output_fired_time: 12.0564899445, propagate_forward_time: 685.4081420898, propagate_backward_time: 1107.1080322266, weight_update_time: 1.0333986282, other_time: 24.0892333984
epoch time: 141.4138641357s, input_fired_time: 2.1172888279, output_fired_time: 0.0000000000, propagate_forward_time: 135.3015594482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9950103760
validation[          , genome  6333] predictions:    6522/  10000 (65.22%), best:    6563/10000 (65.63%), error:     10167.76563, best error:     10111.55176 on epoch:    61, epoch:   63/100, mu: 0.9569021463, learning_rate: 0.0000072998, weight_decay: 0.0000048178

epoch time: 1838.3024902344s, input_fired_time: 28.9934062958, output_fired_time: 11.9193944931, propagate_forward_time: 678.9102172852, propagate_backward_time: 1094.8488769531, weight_update_time: 1.0403282642, other_time: 23.6306152344
epoch time: 136.2080535889s, input_fired_time: 2.0775845051, output_fired_time: 0.0000000000, propagate_forward_time: 130.3116302490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8188323975
validation[          , genome  6333] predictions:    6531/  10000 (65.31%), best:    6531/10000 (65.31%), error:     10104.50391, best error:     10104.50391 on epoch:    64, epoch:   64/100, mu: 0.9582830071, learning_rate: 0.0000067511, weight_decay: 0.0000044873

epoch time: 1812.7225341797s, input_fired_time: 28.8076267242, output_fired_time: 11.8212537766, propagate_forward_time: 664.9520263672, propagate_backward_time: 1083.3325195313, weight_update_time: 1.0330071449, other_time: 23.8090820313
epoch time: 138.3239288330s, input_fired_time: 2.0836811066, output_fired_time: 0.0000000000, propagate_forward_time: 132.3969879150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8432617188
validation[          , genome  6333] predictions:    6557/  10000 (65.57%), best:    6557/10000 (65.57%), error:     10074.40234, best error:     10074.40234 on epoch:    65, epoch:   65/100, mu: 0.9596062303, learning_rate: 0.0000062437, weight_decay: 0.0000041794

epoch time: 1822.9050292969s, input_fired_time: 29.6164512634, output_fired_time: 12.0163135529, propagate_forward_time: 670.2357177734, propagate_backward_time: 1087.8298339844, weight_update_time: 1.0366275311, other_time: 23.2066650391
epoch time: 139.2244415283s, input_fired_time: 2.1072490215, output_fired_time: 0.0000000000, propagate_forward_time: 133.1773681641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9398193359
validation[          , genome  6333] predictions:    6526/  10000 (65.26%), best:    6557/10000 (65.57%), error:     10113.86816, best error:     10074.40234 on epoch:    65, epoch:   66/100, mu: 0.9608742595, learning_rate: 0.0000057744, weight_decay: 0.0000038927

epoch time: 1828.3110351563s, input_fired_time: 30.0945072174, output_fired_time: 11.9671926498, propagate_forward_time: 667.9658203125, propagate_backward_time: 1095.0938720703, weight_update_time: 1.0212695599, other_time: 23.1896972656
epoch time: 138.7903594971s, input_fired_time: 2.1049354076, output_fired_time: 0.0000000000, propagate_forward_time: 132.8501129150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8353118896
validation[          , genome  6333] predictions:    6532/  10000 (65.32%), best:    6557/10000 (65.57%), error:     10107.83398, best error:     10074.40234 on epoch:    65, epoch:   67/100, mu: 0.9620894194, learning_rate: 0.0000053404, weight_decay: 0.0000036257

epoch time: 1822.7570800781s, input_fired_time: 29.8155670166, output_fired_time: 11.8660478592, propagate_forward_time: 669.7669677734, propagate_backward_time: 1088.0656738281, weight_update_time: 1.0326486826, other_time: 23.2427978516
epoch time: 136.0572662354s, input_fired_time: 2.0518887043, output_fired_time: 0.0000000000, propagate_forward_time: 130.1993713379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8059997559
validation[          , genome  6333] predictions:    6570/  10000 (65.70%), best:    6557/10000 (65.57%), error:     10080.59863, best error:     10074.40234 on epoch:    65, epoch:   68/100, mu: 0.9632538557, learning_rate: 0.0000049390, weight_decay: 0.0000033770

epoch time: 1879.8983154297s, input_fired_time: 30.4032974243, output_fired_time: 12.1817703247, propagate_forward_time: 691.7193603516, propagate_backward_time: 1121.4947509766, weight_update_time: 1.1041171551, other_time: 24.0991210938
epoch time: 142.3505554199s, input_fired_time: 2.1299681664, output_fired_time: 0.0000000000, propagate_forward_time: 136.2495574951, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9710235596
validation[          , genome  6333] predictions:    6585/  10000 (65.85%), best:    6585/10000 (65.85%), error:     10072.27441, best error:     10072.27441 on epoch:    69, epoch:   69/100, mu: 0.9643697143, learning_rate: 0.0000045678, weight_decay: 0.0000031453

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.965439
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 4.22452e-06
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 2.92955e-06
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 70
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6585
read best_validation_error: 10072.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '969130383'
read generator: 969130383
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1783.13134765625s, input_fired_time: 29.7074546813965, output_fired_time: 11.7182397842407, propagate_forward_time: 652.892517089844, propagate_backward_time: 1066.04577636719, weight_update_time: 1.00729036331177, other_time: 22.7674560546875
epoch time: 138.77685546875s, input_fired_time: 2.09649896621704, output_fired_time: 0, propagate_forward_time: 132.668579101563, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.01177978515625
validation[          , genome  6333] predictions:    6512/  10000 (65.12%), best:    6585/10000 (65.85%), error:     10123.60254, best error:     10072.27441 on epoch:    69, epoch:   70/100, mu: 0.9654390216, learning_rate: 0.0000042245, weight_decay: 0.0000029296

epoch time: 1789.5905761719s, input_fired_time: 29.8083477020, output_fired_time: 11.6082086563, propagate_forward_time: 656.4993286133, propagate_backward_time: 1068.8331298828, weight_update_time: 1.0158034563, other_time: 22.8416748047
epoch time: 135.7645263672s, input_fired_time: 2.1092007160, output_fired_time: 0.0000000000, propagate_forward_time: 129.8268890381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8284301758
validation[          , genome  6333] predictions:    6510/  10000 (65.10%), best:    6585/10000 (65.85%), error:     10140.17969, best error:     10072.27441 on epoch:    69, epoch:   71/100, mu: 0.9664636850, learning_rate: 0.0000039070, weight_decay: 0.0000027286

epoch time: 1848.1528320313s, input_fired_time: 30.2318496704, output_fired_time: 11.9786682129, propagate_forward_time: 680.2653808594, propagate_backward_time: 1102.2727050781, weight_update_time: 1.0333150625, other_time: 23.4042968750
epoch time: 136.8626251221s, input_fired_time: 2.0725009441, output_fired_time: 0.0000000000, propagate_forward_time: 130.8635559082, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9265747070
validation[          , genome  6333] predictions:    6547/  10000 (65.47%), best:    6547/10000 (65.47%), error:     10062.36523, best error:     10062.36523 on epoch:    72, epoch:   72/100, mu: 0.9674456120, learning_rate: 0.0000036134, weight_decay: 0.0000025414

epoch time: 1802.0921630859s, input_fired_time: 30.0123596191, output_fired_time: 11.8375272751, propagate_forward_time: 660.5180053711, propagate_backward_time: 1076.6199951172, weight_update_time: 1.0471013784, other_time: 23.1043701172
epoch time: 131.0532684326s, input_fired_time: 2.0394704342, output_fired_time: 0.0000000000, propagate_forward_time: 125.3593139648, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6544799805
validation[          , genome  6333] predictions:    6554/  10000 (65.54%), best:    6547/10000 (65.47%), error:     10065.44922, best error:     10062.36523 on epoch:    72, epoch:   73/100, mu: 0.9683865905, learning_rate: 0.0000033418, weight_decay: 0.0000023671

epoch time: 1808.6826171875s, input_fired_time: 29.9549293518, output_fired_time: 11.6331052780, propagate_forward_time: 661.2494506836, propagate_backward_time: 1082.7680664063, weight_update_time: 1.0130350590, other_time: 23.0771484375
epoch time: 140.8283843994s, input_fired_time: 2.1384549141, output_fired_time: 0.0000000000, propagate_forward_time: 134.6751098633, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0148162842
validation[          , genome  6333] predictions:    6511/  10000 (65.11%), best:    6547/10000 (65.47%), error:     10148.68066, best error:     10062.36523 on epoch:    72, epoch:   74/100, mu: 0.9692882895, learning_rate: 0.0000030906, weight_decay: 0.0000022047

epoch time: 1839.4472656250s, input_fired_time: 29.6405353546, output_fired_time: 12.0284566879, propagate_forward_time: 679.3862304688, propagate_backward_time: 1094.7114257813, weight_update_time: 1.0572161674, other_time: 23.6806640625
epoch time: 133.3303070068s, input_fired_time: 2.0170221329, output_fired_time: 0.0000000000, propagate_forward_time: 127.7565536499, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5567245483
validation[          , genome  6333] predictions:    6563/  10000 (65.63%), best:    6563/10000 (65.63%), error:     10038.95215, best error:     10038.95215 on epoch:    75, epoch:   75/100, mu: 0.9701523781, learning_rate: 0.0000028583, weight_decay: 0.0000020534

epoch time: 1851.2626953125s, input_fired_time: 29.9108238220, output_fired_time: 12.1817893982, propagate_forward_time: 675.0803222656, propagate_backward_time: 1110.0561523438, weight_update_time: 1.0513570309, other_time: 24.0336914063
epoch time: 139.0197143555s, input_fired_time: 2.1396083832, output_fired_time: 0.0000000000, propagate_forward_time: 132.7686004639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1115112305
validation[          , genome  6333] predictions:    6570/  10000 (65.70%), best:    6570/10000 (65.70%), error:     10005.06738, best error:     10005.06738 on epoch:    76, epoch:   76/100, mu: 0.9709804058, learning_rate: 0.0000026435, weight_decay: 0.0000019126

epoch time: 1842.9810791016s, input_fired_time: 29.6485500336, output_fired_time: 12.1259307861, propagate_forward_time: 675.2669677734, propagate_backward_time: 1102.8061523438, weight_update_time: 1.0135594606, other_time: 23.1334228516
epoch time: 143.3605194092s, input_fired_time: 2.1491596699, output_fired_time: 0.0000000000, propagate_forward_time: 137.2185363770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9928283691
validation[          , genome  6333] predictions:    6587/  10000 (65.87%), best:    6570/10000 (65.70%), error:     10011.57324, best error:     10005.06738 on epoch:    76, epoch:   77/100, mu: 0.9717739224, learning_rate: 0.0000024448, weight_decay: 0.0000017814

epoch time: 1858.0035400391s, input_fired_time: 29.7100944519, output_fired_time: 12.1844005585, propagate_forward_time: 683.3228149414, propagate_backward_time: 1109.0419921875, weight_update_time: 1.0518236160, other_time: 23.7441406250
epoch time: 146.3008422852s, input_fired_time: 2.2031724453, output_fired_time: 0.0000000000, propagate_forward_time: 140.2243041992, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8733673096
validation[          , genome  6333] predictions:    6555/  10000 (65.55%), best:    6570/10000 (65.70%), error:     10049.15039, best error:     10005.06738 on epoch:    76, epoch:   78/100, mu: 0.9725342989, learning_rate: 0.0000022611, weight_decay: 0.0000016592

epoch time: 1870.0144042969s, input_fired_time: 30.4043350220, output_fired_time: 12.3657398224, propagate_forward_time: 686.8547363281, propagate_backward_time: 1116.6760253906, weight_update_time: 1.0693955421, other_time: 23.7136230469
epoch time: 142.8189392090s, input_fired_time: 2.1988794804, output_fired_time: 0.0000000000, propagate_forward_time: 136.7364349365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8836212158
validation[          , genome  6333] predictions:    6531/  10000 (65.31%), best:    6570/10000 (65.70%), error:     10064.83789, best error:     10005.06738 on epoch:    76, epoch:   79/100, mu: 0.9732629657, learning_rate: 0.0000020911, weight_decay: 0.0000015454

epoch time: 1848.6392822266s, input_fired_time: 30.8462600708, output_fired_time: 12.6626224518, propagate_forward_time: 677.6287841797, propagate_backward_time: 1102.8098144531, weight_update_time: 1.0673050880, other_time: 24.6917724609
epoch time: 150.3013763428s, input_fired_time: 2.2182290554, output_fired_time: 0.0000000000, propagate_forward_time: 143.9179840088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1651611328
validation[          , genome  6333] predictions:    6565/  10000 (65.65%), best:    6570/10000 (65.70%), error:     10029.38379, best error:     10005.06738 on epoch:    76, epoch:   80/100, mu: 0.9739612341, learning_rate: 0.0000019339, weight_decay: 0.0000014393

epoch time: 1835.0185546875s, input_fired_time: 30.4161109924, output_fired_time: 12.1020650864, propagate_forward_time: 668.7340087891, propagate_backward_time: 1100.4288330078, weight_update_time: 1.0492807627, other_time: 23.3375244141
epoch time: 144.1753997803s, input_fired_time: 2.1981847286, output_fired_time: 0.0000000000, propagate_forward_time: 138.1045532227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8726654053
validation[          , genome  6333] predictions:    6561/  10000 (65.61%), best:    6570/10000 (65.70%), error:     10013.34473, best error:     10005.06738 on epoch:    76, epoch:   81/100, mu: 0.9746303558, learning_rate: 0.0000017886, weight_decay: 0.0000013406

epoch time: 1807.9042968750s, input_fired_time: 29.7610530853, output_fired_time: 11.7661371231, propagate_forward_time: 657.3614501953, propagate_backward_time: 1085.7296142578, weight_update_time: 1.0364745855, other_time: 23.2860107422
epoch time: 141.3512878418s, input_fired_time: 2.1174838543, output_fired_time: 0.0000000000, propagate_forward_time: 135.5945434570, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6392669678
validation[          , genome  6333] predictions:    6580/  10000 (65.80%), best:    6580/10000 (65.80%), error:     10003.83008, best error:     10003.83008 on epoch:    82, epoch:   82/100, mu: 0.9752715826, learning_rate: 0.0000016542, weight_decay: 0.0000012486

epoch time: 1818.8363037109s, input_fired_time: 29.9613876343, output_fired_time: 12.0226087570, propagate_forward_time: 665.8392333984, propagate_backward_time: 1088.0500488281, weight_update_time: 1.0236095190, other_time: 22.9630126953
epoch time: 143.1058959961s, input_fired_time: 2.1512227058, output_fired_time: 0.0000000000, propagate_forward_time: 136.9453582764, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0093078613
validation[          , genome  6333] predictions:    6585/  10000 (65.85%), best:    6580/10000 (65.80%), error:     10003.86426, best error:     10003.83008 on epoch:    82, epoch:   83/100, mu: 0.9758860469, learning_rate: 0.0000015298, weight_decay: 0.0000011630

epoch time: 1854.2048339844s, input_fired_time: 30.4135875702, output_fired_time: 12.0484037399, propagate_forward_time: 681.6083984375, propagate_backward_time: 1106.4515380859, weight_update_time: 1.0450342894, other_time: 23.6828613281
epoch time: 142.7095336914s, input_fired_time: 2.1395683289, output_fired_time: 0.0000000000, propagate_forward_time: 136.6534576416, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9165039063
validation[          , genome  6333] predictions:    6583/  10000 (65.83%), best:    6583/10000 (65.83%), error:      9991.22949, best error:      9991.22949 on epoch:    84, epoch:   84/100, mu: 0.9764748812, learning_rate: 0.0000014149, weight_decay: 0.0000010832

epoch time: 1896.1269531250s, input_fired_time: 30.7792320251, output_fired_time: 12.3613157272, propagate_forward_time: 699.1256713867, propagate_backward_time: 1129.6182861328, weight_update_time: 1.0409388542, other_time: 24.2425537109
epoch time: 149.1050109863s, input_fired_time: 2.2568113804, output_fired_time: 0.0000000000, propagate_forward_time: 142.7785644531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0696411133
validation[          , genome  6333] predictions:    6578/  10000 (65.78%), best:    6578/10000 (65.78%), error:      9969.63574, best error:      9969.63574 on epoch:    85, epoch:   85/100, mu: 0.9770391583, learning_rate: 0.0000013085, weight_decay: 0.0000010089

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.97758
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 1.21017e-06
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 9.39692e-07
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 86
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 85
read number_validation_images: 10000
read best_validation_predictions: 6578
read best_validation_error: 9969.64
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '1093666706'
read generator: 1093666706
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1661.25427246094s, input_fired_time: 26.7579555511475, output_fired_time: 11.3326272964478, propagate_forward_time: 615.800354003906, propagate_backward_time: 985.932006835938, weight_update_time: 0.980867922306061, other_time: 21.4313354492188
epoch time: 139.119018554688s, input_fired_time: 2.11457252502441, output_fired_time: 0, propagate_forward_time: 132.950866699219, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.05357360839844
validation[          , genome  6333] predictions:    6579/  10000 (65.79%), best:    6578/10000 (65.78%), error:     10002.87207, best error:      9969.63574 on epoch:    85, epoch:   86/100, mu: 0.9775798917, learning_rate: 0.0000012102, weight_decay: 0.0000009397

epoch time: 1661.8270263672s, input_fired_time: 27.0942668915, output_fired_time: 10.9908895493, propagate_forward_time: 611.0142822266, propagate_backward_time: 991.5186767578, weight_update_time: 0.9493803382, other_time: 21.2089843750
epoch time: 127.1786422729s, input_fired_time: 1.9661142826, output_fired_time: 0.0000000000, propagate_forward_time: 121.5675735474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6449508667
validation[          , genome  6333] predictions:    6581/  10000 (65.81%), best:    6578/10000 (65.78%), error:      9984.97461, best error:      9969.63574 on epoch:    85, epoch:   87/100, mu: 0.9780980349, learning_rate: 0.0000011192, weight_decay: 0.0000008752

epoch time: 1673.7713623047s, input_fired_time: 26.8945178986, output_fired_time: 11.6048517227, propagate_forward_time: 620.1315917969, propagate_backward_time: 993.9711914063, weight_update_time: 0.9697021246, other_time: 21.1691894531
epoch time: 132.5566101074s, input_fired_time: 2.0815157890, output_fired_time: 0.0000000000, propagate_forward_time: 126.6010131836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8740844727
validation[          , genome  6333] predictions:    6535/  10000 (65.35%), best:    6578/10000 (65.78%), error:     10036.55566, best error:      9969.63574 on epoch:    85, epoch:   88/100, mu: 0.9785946012, learning_rate: 0.0000010351, weight_decay: 0.0000008152

epoch time: 1685.8657226563s, input_fired_time: 26.8893394470, output_fired_time: 11.3708248138, propagate_forward_time: 624.5566406250, propagate_backward_time: 1001.5981445313, weight_update_time: 0.9657294750, other_time: 21.4508056641
epoch time: 124.7971420288s, input_fired_time: 1.9078972340, output_fired_time: 0.0000000000, propagate_forward_time: 119.3688583374, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5203857422
validation[          , genome  6333] predictions:    6540/  10000 (65.40%), best:    6578/10000 (65.78%), error:     10010.43164, best error:      9969.63574 on epoch:    85, epoch:   89/100, mu: 0.9790704250, learning_rate: 0.0000009573, weight_decay: 0.0000007593

epoch time: 1599.5917968750s, input_fired_time: 25.8335895538, output_fired_time: 10.6651277542, propagate_forward_time: 588.4952392578, propagate_backward_time: 954.2617187500, weight_update_time: 0.9164456129, other_time: 20.3360595703
epoch time: 123.9058151245s, input_fired_time: 1.8587986231, output_fired_time: 0.0000000000, propagate_forward_time: 118.5014724731, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5455474854
validation[          , genome  6333] predictions:    6580/  10000 (65.80%), best:    6578/10000 (65.78%), error:      9980.78809, best error:      9969.63574 on epoch:    85, epoch:   90/100, mu: 0.9795264006, learning_rate: 0.0000008853, weight_decay: 0.0000007072

epoch time: 1632.6456298828s, input_fired_time: 26.1576423645, output_fired_time: 10.6240253448, propagate_forward_time: 599.9181518555, propagate_backward_time: 975.2412719727, weight_update_time: 0.9519255757, other_time: 20.7045898438
epoch time: 130.4950408936s, input_fired_time: 1.9219222069, output_fired_time: 0.0000000000, propagate_forward_time: 125.0240783691, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5490417480
validation[          , genome  6333] predictions:    6600/  10000 (66.00%), best:    6578/10000 (65.78%), error:      9970.21289, best error:      9969.63574 on epoch:    85, epoch:   91/100, mu: 0.9799633622, learning_rate: 0.0000008188, weight_decay: 0.0000006587

epoch time: 1660.8410644531s, input_fired_time: 26.4888362885, output_fired_time: 10.9293594360, propagate_forward_time: 612.4902343750, propagate_backward_time: 990.1112060547, weight_update_time: 0.9517831802, other_time: 20.8214111328
epoch time: 138.7102966309s, input_fired_time: 2.0818519592, output_fired_time: 0.0000000000, propagate_forward_time: 132.7632598877, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8651885986
validation[          , genome  6333] predictions:    6572/  10000 (65.72%), best:    6578/10000 (65.78%), error:      9991.43945, best error:      9969.63574 on epoch:    85, epoch:   92/100, mu: 0.9803820848, learning_rate: 0.0000007573, weight_decay: 0.0000006135

epoch time: 1706.6906738281s, input_fired_time: 27.1779956818, output_fired_time: 11.6815338135, propagate_forward_time: 631.5753173828, propagate_backward_time: 1013.8527221680, weight_update_time: 1.0000642538, other_time: 22.4031372070
epoch time: 121.7712707520s, input_fired_time: 1.8693873882, output_fired_time: 0.0000000000, propagate_forward_time: 116.6424179077, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2594680786
validation[          , genome  6333] predictions:    6563/  10000 (65.63%), best:    6578/10000 (65.78%), error:     10007.23242, best error:      9969.63574 on epoch:    85, epoch:   93/100, mu: 0.9807833433, learning_rate: 0.0000007003, weight_decay: 0.0000005714

epoch time: 1638.8002929688s, input_fired_time: 26.3647022247, output_fired_time: 11.0622682571, propagate_forward_time: 607.9077148438, propagate_backward_time: 972.5240478516, weight_update_time: 0.9375600219, other_time: 20.9415283203
epoch time: 126.5237197876s, input_fired_time: 1.9671616554, output_fired_time: 0.0000000000, propagate_forward_time: 121.1094512939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4471054077
validation[          , genome  6333] predictions:    6583/  10000 (65.83%), best:    6578/10000 (65.78%), error:      9985.64063, best error:      9969.63574 on epoch:    85, epoch:   94/100, mu: 0.9811678529, learning_rate: 0.0000006477, weight_decay: 0.0000005322

epoch time: 1592.3116455078s, input_fired_time: 26.1866588593, output_fired_time: 10.5577421188, propagate_forward_time: 588.9922485352, propagate_backward_time: 946.3233642578, weight_update_time: 0.9294324517, other_time: 20.2516479492
epoch time: 126.4164505005s, input_fired_time: 1.9138842821, output_fired_time: 0.0000000000, propagate_forward_time: 121.0208816528, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4816818237
validation[          , genome  6333] predictions:    6591/  10000 (65.91%), best:    6578/10000 (65.78%), error:      9971.32617, best error:      9969.63574 on epoch:    85, epoch:   95/100, mu: 0.9815363288, learning_rate: 0.0000005990, weight_decay: 0.0000004957

epoch time: 1604.4024658203s, input_fired_time: 25.9738445282, output_fired_time: 10.8240518570, propagate_forward_time: 589.9739379883, propagate_backward_time: 957.5618286133, weight_update_time: 0.9452162385, other_time: 20.0687255859
epoch time: 124.6091995239s, input_fired_time: 1.9581518173, output_fired_time: 0.0000000000, propagate_forward_time: 118.9997558594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6512908936
validation[          , genome  6333] predictions:    6579/  10000 (65.79%), best:    6578/10000 (65.78%), error:      9987.07715, best error:      9969.63574 on epoch:    85, epoch:   96/100, mu: 0.9818894267, learning_rate: 0.0000005540, weight_decay: 0.0000004617

epoch time: 1656.4497070313s, input_fired_time: 26.4952125549, output_fired_time: 10.7958478928, propagate_forward_time: 614.4025268555, propagate_backward_time: 984.3226318359, weight_update_time: 0.9217376709, other_time: 20.4334106445
epoch time: 129.3302307129s, input_fired_time: 1.9427468777, output_fired_time: 0.0000000000, propagate_forward_time: 123.8282089233, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5592727661
validation[          , genome  6333] predictions:    6576/  10000 (65.76%), best:    6578/10000 (65.78%), error:      9976.53613, best error:      9969.63574 on epoch:    85, epoch:   97/100, mu: 0.9822278023, learning_rate: 0.0000005124, weight_decay: 0.0000004300

epoch time: 1751.8864746094s, input_fired_time: 27.7403831482, output_fired_time: 11.7818689346, propagate_forward_time: 649.0601196289, propagate_backward_time: 1040.7653808594, weight_update_time: 1.0354913473, other_time: 22.5388183594
epoch time: 140.0494079590s, input_fired_time: 2.1475281715, output_fired_time: 0.0000000000, propagate_forward_time: 133.7643127441, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1375732422
validation[          , genome  6333] predictions:    6568/  10000 (65.68%), best:    6578/10000 (65.78%), error:      9990.71973, best error:      9969.63574 on epoch:    85, epoch:   98/100, mu: 0.9825520515, learning_rate: 0.0000004739, weight_decay: 0.0000004005

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.982863
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 4.38243e-07
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 3.73045e-07
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 99
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 85
read number_validation_images: 10000
read best_validation_predictions: 6578
read best_validation_error: 9969.64
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '1529004354'
read generator: 1529004354
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1712.94384765625s, input_fired_time: 27.5186672210693, output_fired_time: 11.7713623046875, propagate_forward_time: 633.151306152344, propagate_backward_time: 1018.65264892578, weight_update_time: 0.983801662921906, other_time: 21.8497924804688
epoch time: 130.587890625s, input_fired_time: 2.05825567245483, output_fired_time: 0, propagate_forward_time: 124.64134979248, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.88828277587891
validation[          , genome  6333] predictions:    6565/  10000 (65.65%), best:    6578/10000 (65.78%), error:      9977.02246, best error:      9969.63574 on epoch:    85, epoch:   99/100, mu: 0.9828627706, learning_rate: 0.0000004382, weight_decay: 0.0000003730

epoch time: 1678.4643554688s, input_fired_time: 27.6635494232, output_fired_time: 11.7088022232, propagate_forward_time: 623.0603637695, propagate_backward_time: 994.1483154297, weight_update_time: 1.0024492741, other_time: 21.8832397461
epoch time: 135.9717864990s, input_fired_time: 2.0798978806, output_fired_time: 0.0000000000, propagate_forward_time: 130.2218475342, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6700439453
validation[          , genome  6333] predictions:    6586/  10000 (65.86%), best:    6578/10000 (65.78%), error:      9972.20801, best error:      9969.63574 on epoch:    85, epoch:  100/100, mu: 0.9831605554, learning_rate: 0.0000004053, weight_decay: 0.0000003475

evaluating best weights on full training data.
evaluting training set with running mean/variance:
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_188_6333_0_r402851491_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.504957
read mu: 0.983446
read mu_delta: 0.95828
read initial_learning_rate: 0.00100254
read learning_rate: 3.74842e-07
read learning_rate_delta: 0.92484
read initial_weight_decay: 0.00042388
read weight_decay: 3.23619e-07
read weight_decay_delta: 0.931401
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.122282
read input_dropout_probability: 0.00123694
read hidden_dropout_probability: 0.0873652
read velocity_reset: 1932
read epoch: 101
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 85
read number_validation_images: 10000
read best_validation_predictions: 6578
read best_validation_error: 9969.64
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6333
read normal distribution: '1 0 0'
generator_str: '1851364752'
read generator: 1851364752
read generated_by_map:
1 enable_edge 1
reading 79 nodes.
reading 342 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1712.29650878906s, input_fired_time: 28.9588737487793, output_fired_time: 12.675482749939, propagate_forward_time: 639.224731445313, propagate_backward_time: 1007.92407226563, weight_update_time: 1.02661442756653, other_time: 23.5133056640625
epoch time: 134.68505859375s, input_fired_time: 2.06796288490295, output_fired_time: 0, propagate_forward_time: 128.795684814453, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.8214111328125
validation[          , genome  6333] predictions:    6586/  10000 (65.86%), best:    6586/10000 (65.86%), error:      9966.21777, best error:      9966.21777 on epoch:   101, epoch:  101/100, mu: 0.9834458828, learning_rate: 0.0000003748, weight_decay: 0.0000003236

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 669.5356445313s, input_fired_time: 10.1634769440, output_fired_time: 0.0000000000, propagate_forward_time: 640.0889282227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.2832641602
best training[          , genome  6333] predictions:   35821/  50000 (71.64%), best:    6586/10000 (65.86%), error:     42998.69922, best error:      9966.21777 on epoch:   101, epoch:  102/100, mu: 0.9837193489, learning_rate: 0.0000003467, weight_decay: 0.0000003014
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 116.2336883545s, input_fired_time: 1.7893840075, output_fired_time: 0.0000000000, propagate_forward_time: 111.0876464844, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3566589355
   testing[          , genome  6333] predictions:    6586/  10000 (65.86%), best:    6586/10000 (65.86%), error:      9966.21777, best error:      9966.21777 on epoch:   101, epoch:  102/100, mu: 0.9837193489, learning_rate: 0.0000003467, weight_decay: 0.0000003014
backpropagation finished successfully!
01:13:16 (10080): called boinc_finish(0)

</stderr_txt>
]]>