Nameexact_genome_1542649503_187_6332_0
Workunit4099348
Created20 Nov 2018, 1:51:19 UTC
Sent20 Nov 2018, 1:56:53 UTC
Report deadline12 Feb 2019, 20:49:57 UTC
Received22 Nov 2018, 23:39:21 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID78968
Run time2 days 2 hours 12 min 51 sec
CPU time2 days 1 hours 47 min 28 sec
Validate stateValid
Credit21,196.72
Device peak FLOPS2.20 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size547.37 MB
Peak swap size566.96 MB
Peak disk usage10.52 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
2] predictions:    5867/  10000 (58.67%), best:    6176/10000 (61.76%), error:     12485.78320, best error:     11466.99121 on epoch:     0, epoch:   61/100, mu: 0.9726220965, learning_rate: 0.0000007838, weight_decay: 0.0000183148

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.973547
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 7.26571e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 1.73163e-05
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 62
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '1957439655'
read generator: 1957439655
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1716.78466796875s, input_fired_time: 27.9219932556152, output_fired_time: 11.6515035629272, propagate_forward_time: 620.699462890625, propagate_backward_time: 1032.94458007813, weight_update_time: 0.975846827030182, other_time: 23.567138671875
epoch time: 128.538635253906s, input_fired_time: 1.83334183692932, output_fired_time: 0, propagate_forward_time: 123.019500732422, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.685791015625
validation[          , genome  6332] predictions:    5821/  10000 (58.21%), best:    6176/10000 (61.76%), error:     12529.34375, best error:     11466.99121 on epoch:     0, epoch:   62/100, mu: 0.9735468626, learning_rate: 0.0000007266, weight_decay: 0.0000173163

epoch time: 1731.1968994141s, input_fired_time: 27.9975490570, output_fired_time: 11.4286746979, propagate_forward_time: 625.5341796875, propagate_backward_time: 1042.7719726563, weight_update_time: 0.9949370623, other_time: 23.4644775391
epoch time: 137.3787536621s, input_fired_time: 1.9663735628, output_fired_time: 0.0000000000, propagate_forward_time: 131.4392700195, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9731140137
validation[          , genome  6332] predictions:    5898/  10000 (58.98%), best:    6176/10000 (61.76%), error:     12445.46387, best error:     11466.99121 on epoch:     0, epoch:   63/100, mu: 0.9744223952, learning_rate: 0.0000006735, weight_decay: 0.0000163722

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.975251
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 6.24272e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 1.54796e-05
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '2066980959'
read generator: 2066980959
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.975251
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 6.24272e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 1.54796e-05
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '2066980959'
read generator: 2066980959
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.975251
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 6.24272e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 1.54796e-05
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '2066980959'
read generator: 2066980959
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.975251
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 6.24272e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 1.54796e-05
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '2066980959'
read generator: 2066980959
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1740.79077148438s, input_fired_time: 27.7541885375977, output_fired_time: 11.6759204864502, propagate_forward_time: 631.398864746094, propagate_backward_time: 1046.08142089844, weight_update_time: 0.98479151725769, other_time: 23.8804931640625
epoch time: 129.264053344727s, input_fired_time: 1.86599349975586, output_fired_time: 0, propagate_forward_time: 123.66869354248, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.72936248779297
validation[          , genome  6332] predictions:    5902/  10000 (59.02%), best:    6176/10000 (61.76%), error:     12433.92090, best error:     11466.99121 on epoch:     0, epoch:   64/100, mu: 0.9752513170, learning_rate: 0.0000006243, weight_decay: 0.0000154796

epoch time: 1740.7443847656s, input_fired_time: 28.0959129333, output_fired_time: 11.4550151825, propagate_forward_time: 630.6168823242, propagate_backward_time: 1047.1339111328, weight_update_time: 0.9940243959, other_time: 23.4427490234
epoch time: 136.9479522705s, input_fired_time: 1.9786155224, output_fired_time: 0.0000000000, propagate_forward_time: 130.6938171387, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2755126953
validation[          , genome  6332] predictions:    5837/  10000 (58.37%), best:    6176/10000 (61.76%), error:     12539.65430, best error:     11466.99121 on epoch:     0, epoch:   65/100, mu: 0.9760361314, learning_rate: 0.0000005787, weight_decay: 0.0000146356

epoch time: 1745.5490722656s, input_fired_time: 27.4941463470, output_fired_time: 11.5485706329, propagate_forward_time: 633.6915283203, propagate_backward_time: 1049.1929931641, weight_update_time: 0.9873878360, other_time: 23.6218261719
epoch time: 138.6346130371s, input_fired_time: 1.9826153517, output_fired_time: 0.0000000000, propagate_forward_time: 132.6239166260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0280761719
validation[          , genome  6332] predictions:    5737/  10000 (57.37%), best:    6176/10000 (61.76%), error:     12608.06152, best error:     11466.99121 on epoch:     0, epoch:   66/100, mu: 0.9767792225, learning_rate: 0.0000005364, weight_decay: 0.0000138377

epoch time: 1772.6499023438s, input_fired_time: 27.9225292206, output_fired_time: 11.7465133667, propagate_forward_time: 647.8578491211, propagate_backward_time: 1061.4031982422, weight_update_time: 0.9912662506, other_time: 23.7198486328
epoch time: 129.7055969238s, input_fired_time: 1.8880542517, output_fired_time: 0.0000000000, propagate_forward_time: 123.9468154907, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8707275391
validation[          , genome  6332] predictions:    5872/  10000 (58.72%), best:    6176/10000 (61.76%), error:     12428.31348, best error:     11466.99121 on epoch:     0, epoch:   67/100, mu: 0.9774827361, learning_rate: 0.0000004972, weight_decay: 0.0000130833

epoch time: 1746.5189208984s, input_fired_time: 27.6863956451, output_fired_time: 11.7228221893, propagate_forward_time: 631.7835693359, propagate_backward_time: 1051.6741943359, weight_update_time: 0.9743045568, other_time: 23.6519775391
epoch time: 129.8409881592s, input_fired_time: 1.8892145157, output_fired_time: 0.0000000000, propagate_forward_time: 124.0131683350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9386062622
validation[          , genome  6332] predictions:    5893/  10000 (58.93%), best:    6176/10000 (61.76%), error:     12436.18262, best error:     11466.99121 on epoch:     0, epoch:   68/100, mu: 0.9781488180, learning_rate: 0.0000004609, weight_decay: 0.0000123700

epoch time: 1718.0416259766s, input_fired_time: 27.4457149506, output_fired_time: 11.5034694672, propagate_forward_time: 623.4049072266, propagate_backward_time: 1032.6717529297, weight_update_time: 0.9795120955, other_time: 23.0158691406
epoch time: 127.2972030640s, input_fired_time: 1.8669373989, output_fired_time: 0.0000000000, propagate_forward_time: 121.5649642944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8653030396
validation[          , genome  6332] predictions:    5868/  10000 (58.68%), best:    6176/10000 (61.76%), error:     12480.79492, best error:     11466.99121 on epoch:     0, epoch:   69/100, mu: 0.9787794948, learning_rate: 0.0000004272, weight_decay: 0.0000116956

epoch time: 1742.9359130859s, input_fired_time: 28.0057048798, output_fired_time: 11.8275499344, propagate_forward_time: 625.3554687500, propagate_backward_time: 1053.9040527344, weight_update_time: 1.0109007359, other_time: 23.8431396484
epoch time: 126.6515045166s, input_fired_time: 1.8672761917, output_fired_time: 0.0000000000, propagate_forward_time: 121.0779495239, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7062759399
validation[          , genome  6332] predictions:    5764/  10000 (57.64%), best:    6176/10000 (61.76%), error:     12564.05273, best error:     11466.99121 on epoch:     0, epoch:   70/100, mu: 0.9793766141, learning_rate: 0.0000003960, weight_decay: 0.0000110579

epoch time: 1701.5561523438s, input_fired_time: 28.7829837799, output_fired_time: 11.3879461288, propagate_forward_time: 610.7595825195, propagate_backward_time: 1027.9804687500, weight_update_time: 0.9903661609, other_time: 22.6452636719
epoch time: 136.7063903809s, input_fired_time: 1.9884134531, output_fired_time: 0.0000000000, propagate_forward_time: 130.7448883057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9730834961
validation[          , genome  6332] predictions:    5905/  10000 (59.05%), best:    6176/10000 (61.76%), error:     12492.37793, best error:     11466.99121 on epoch:     0, epoch:   71/100, mu: 0.9799419045, learning_rate: 0.0000003670, weight_decay: 0.0000104551

epoch time: 1720.7485351563s, input_fired_time: 28.5229988098, output_fired_time: 11.4735021591, propagate_forward_time: 623.2731933594, propagate_backward_time: 1034.7629394531, weight_update_time: 0.9598913789, other_time: 22.7159423828
epoch time: 135.3775177002s, input_fired_time: 1.9986128807, output_fired_time: 0.0000000000, propagate_forward_time: 129.4374847412, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9414215088
validation[          , genome  6332] predictions:    5874/  10000 (58.74%), best:    6176/10000 (61.76%), error:     12442.54199, best error:     11466.99121 on epoch:     0, epoch:   72/100, mu: 0.9804771543, learning_rate: 0.0000003402, weight_decay: 0.0000098850

epoch time: 1763.8251953125s, input_fired_time: 28.8523197174, output_fired_time: 11.5819463730, propagate_forward_time: 637.6094970703, propagate_backward_time: 1062.5925292969, weight_update_time: 1.0102615356, other_time: 23.1889648438
epoch time: 137.6630249023s, input_fired_time: 1.9758130312, output_fired_time: 0.0000000000, propagate_forward_time: 131.6990509033, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9881591797
validation[          , genome  6332] predictions:    5907/  10000 (59.07%), best:    6176/10000 (61.76%), error:     12450.28516, best error:     11466.99121 on epoch:     0, epoch:   73/100, mu: 0.9809839129, learning_rate: 0.0000003154, weight_decay: 0.0000093461

epoch time: 1759.5717773438s, input_fired_time: 28.5838871002, output_fired_time: 11.8522968292, propagate_forward_time: 637.5709228516, propagate_backward_time: 1057.9368896484, weight_update_time: 0.9979014993, other_time: 23.6278076172
epoch time: 129.5163879395s, input_fired_time: 1.8933708668, output_fired_time: 0.0000000000, propagate_forward_time: 123.9384994507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6845169067
validation[          , genome  6332] predictions:    5886/  10000 (58.86%), best:    6176/10000 (61.76%), error:     12431.18164, best error:     11466.99121 on epoch:     0, epoch:   74/100, mu: 0.9814636707, learning_rate: 0.0000002923, weight_decay: 0.0000088366

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.981918
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 2.70958e-07
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 8.3548e-06
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 75
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '1834500290'
read generator: 1834500290
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1686.48059082031s, input_fired_time: 28.3240947723389, output_fired_time: 11.2921762466431, propagate_forward_time: 605.84130859375, propagate_backward_time: 1019.00939941406, weight_update_time: 0.955494463443756, other_time: 22.0135498046875
epoch time: 123.077247619629s, input_fired_time: 1.81320106983185, output_fired_time: 0, propagate_forward_time: 117.693237304688, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.57080841064453
validation[          , genome  6332] predictions:    5864/  10000 (58.64%), best:    6176/10000 (61.76%), error:     12471.31055, best error:     11466.99121 on epoch:     0, epoch:   75/100, mu: 0.9819179177, learning_rate: 0.0000002710, weight_decay: 0.0000083548

epoch time: 1693.1077880859s, input_fired_time: 28.1886482239, output_fired_time: 11.2886419296, propagate_forward_time: 609.0873413086, propagate_backward_time: 1021.7392578125, weight_update_time: 0.9893431067, other_time: 22.8039550781
epoch time: 127.2229537964s, input_fired_time: 1.8709454536, output_fired_time: 0.0000000000, propagate_forward_time: 121.6066894531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7453155518
validation[          , genome  6332] predictions:    5902/  10000 (59.02%), best:    6176/10000 (61.76%), error:     12438.18164, best error:     11466.99121 on epoch:     0, epoch:   76/100, mu: 0.9823480248, learning_rate: 0.0000002512, weight_decay: 0.0000078993

epoch time: 1762.4418945313s, input_fired_time: 28.9397830963, output_fired_time: 11.8864183426, propagate_forward_time: 636.4827880859, propagate_backward_time: 1061.3256835938, weight_update_time: 1.0070573092, other_time: 23.8071289063
epoch time: 123.2720260620s, input_fired_time: 1.8859028816, output_fired_time: 0.0000000000, propagate_forward_time: 117.7314605713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6546630859
validation[          , genome  6332] predictions:    5908/  10000 (59.08%), best:    6176/10000 (61.76%), error:     12454.69043, best error:     11466.99121 on epoch:     0, epoch:   77/100, mu: 0.9827552438, learning_rate: 0.0000002328, weight_decay: 0.0000074686

epoch time: 1710.9309082031s, input_fired_time: 28.8425006866, output_fired_time: 11.3600530624, propagate_forward_time: 612.6012573242, propagate_backward_time: 1035.9860839844, weight_update_time: 0.9740440249, other_time: 22.1408691406
epoch time: 132.3005523682s, input_fired_time: 1.9071484804, output_fired_time: 0.0000000000, propagate_forward_time: 126.5427017212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8507003784
validation[          , genome  6332] predictions:    5902/  10000 (59.02%), best:    6176/10000 (61.76%), error:     12462.79590, best error:     11466.99121 on epoch:     0, epoch:   78/100, mu: 0.9831407666, learning_rate: 0.0000002158, weight_decay: 0.0000070614

epoch time: 1691.9548339844s, input_fired_time: 28.6151485443, output_fired_time: 11.4373741150, propagate_forward_time: 612.0848388672, propagate_backward_time: 1017.4255981445, weight_update_time: 0.9699257612, other_time: 22.3919067383
epoch time: 134.6689147949s, input_fired_time: 1.9623647928, output_fired_time: 0.0000000000, propagate_forward_time: 128.8023529053, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9041900635
validation[          , genome  6332] predictions:    5897/  10000 (58.97%), best:    6176/10000 (61.76%), error:     12428.08301, best error:     11466.99121 on epoch:     0, epoch:   79/100, mu: 0.9835057855, learning_rate: 0.0000002000, weight_decay: 0.0000066765

epoch time: 1738.3496093750s, input_fired_time: 28.3237686157, output_fired_time: 11.6483860016, propagate_forward_time: 627.1770629883, propagate_backward_time: 1047.8839111328, weight_update_time: 0.9854422212, other_time: 23.3165283203
epoch time: 133.4245758057s, input_fired_time: 1.9179178476, output_fired_time: 0.0000000000, propagate_forward_time: 127.5191268921, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9875259399
validation[          , genome  6332] predictions:    5914/  10000 (59.14%), best:    6176/10000 (61.76%), error:     12421.39258, best error:     11466.99121 on epoch:     0, epoch:   80/100, mu: 0.9838513732, learning_rate: 0.0000001854, weight_decay: 0.0000063125

epoch time: 1728.8798828125s, input_fired_time: 28.2807216644, output_fired_time: 11.5512123108, propagate_forward_time: 626.4811401367, propagate_backward_time: 1039.8121337891, weight_update_time: 0.9764593840, other_time: 22.7545166016
epoch time: 135.9137725830s, input_fired_time: 1.9550101757, output_fired_time: 0.0000000000, propagate_forward_time: 129.8452911377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1134643555
validation[          , genome  6332] predictions:    5896/  10000 (58.96%), best:    6176/10000 (61.76%), error:     12425.37012, best error:     11466.99121 on epoch:     0, epoch:   81/100, mu: 0.9841785431, learning_rate: 0.0000001719, weight_decay: 0.0000059683

epoch time: 1768.4138183594s, input_fired_time: 28.6765308380, output_fired_time: 11.8785467148, propagate_forward_time: 640.0137939453, propagate_backward_time: 1063.9593505859, weight_update_time: 1.0116467476, other_time: 23.8856201172
epoch time: 130.3914642334s, input_fired_time: 1.8982663155, output_fired_time: 0.0000000000, propagate_forward_time: 124.5689086914, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9242858887
validation[          , genome  6332] predictions:    5923/  10000 (59.23%), best:    6176/10000 (61.76%), error:     12436.19434, best error:     11466.99121 on epoch:     0, epoch:   82/100, mu: 0.9844883084, learning_rate: 0.0000001593, weight_decay: 0.0000056429

epoch time: 1753.9760742188s, input_fired_time: 28.7311744690, output_fired_time: 11.7982425690, propagate_forward_time: 637.7068481445, propagate_backward_time: 1052.2707519531, weight_update_time: 0.9946887493, other_time: 23.4689941406
epoch time: 137.6686096191s, input_fired_time: 1.9894351959, output_fired_time: 0.0000000000, propagate_forward_time: 131.5095672607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1696014404
validation[          , genome  6332] predictions:    5912/  10000 (59.12%), best:    6176/10000 (61.76%), error:     12442.52930, best error:     11466.99121 on epoch:     0, epoch:   83/100, mu: 0.9847816229, learning_rate: 0.0000001477, weight_decay: 0.0000053353

epoch time: 1770.2385253906s, input_fired_time: 28.9319820404, output_fired_time: 11.6330652237, propagate_forward_time: 645.7504882813, propagate_backward_time: 1060.7608642578, weight_update_time: 0.9812143445, other_time: 23.1621093750
epoch time: 131.7080230713s, input_fired_time: 1.9353722334, output_fired_time: 0.0000000000, propagate_forward_time: 125.8201217651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9525222778
validation[          , genome  6332] predictions:    5923/  10000 (59.23%), best:    6176/10000 (61.76%), error:     12459.35840, best error:     11466.99121 on epoch:     0, epoch:   84/100, mu: 0.9850593209, learning_rate: 0.0000001369, weight_decay: 0.0000050444

epoch time: 1752.3852539063s, input_fired_time: 29.3610973358, output_fired_time: 11.7071409225, propagate_forward_time: 632.6187744141, propagate_backward_time: 1055.2386474609, weight_update_time: 0.9984844327, other_time: 23.4595947266
epoch time: 136.9563446045s, input_fired_time: 1.9912043810, output_fired_time: 0.0000000000, propagate_forward_time: 131.0879364014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8771972656
validation[          , genome  6332] predictions:    5910/  10000 (59.10%), best:    6176/10000 (61.76%), error:     12438.32910, best error:     11466.99121 on epoch:     0, epoch:   85/100, mu: 0.9853222370, learning_rate: 0.0000001269, weight_decay: 0.0000047694

epoch time: 1778.1585693359s, input_fired_time: 29.4892024994, output_fired_time: 12.3765106201, propagate_forward_time: 643.7965698242, propagate_backward_time: 1067.6070556641, weight_update_time: 1.0193626881, other_time: 24.8892822266
epoch time: 126.3797912598s, input_fired_time: 1.8674943447, output_fired_time: 0.0000000000, propagate_forward_time: 120.7978439331, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7144546509
validation[          , genome  6332] predictions:    5851/  10000 (58.51%), best:    6176/10000 (61.76%), error:     12466.61719, best error:     11466.99121 on epoch:     0, epoch:   86/100, mu: 0.9855711460, learning_rate: 0.0000001176, weight_decay: 0.0000045093

epoch time: 1737.7430419922s, input_fired_time: 29.3236427307, output_fired_time: 11.5900993347, propagate_forward_time: 626.8095703125, propagate_backward_time: 1046.6394042969, weight_update_time: 1.0033909082, other_time: 23.3803710938
epoch time: 133.5476989746s, input_fired_time: 1.9506797791, output_fired_time: 0.0000000000, propagate_forward_time: 127.6496047974, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9474105835
validation[          , genome  6332] predictions:    5900/  10000 (59.00%), best:    6176/10000 (61.76%), error:     12448.26758, best error:     11466.99121 on epoch:     0, epoch:   87/100, mu: 0.9858068228, learning_rate: 0.0000001090, weight_decay: 0.0000042635

epoch time: 1718.8167724609s, input_fired_time: 28.2822113037, output_fired_time: 11.4189386368, propagate_forward_time: 619.2518920898, propagate_backward_time: 1036.9022216797, weight_update_time: 0.9797093868, other_time: 22.9615478516
epoch time: 131.2334747314s, input_fired_time: 1.9481248856, output_fired_time: 0.0000000000, propagate_forward_time: 125.5044555664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7808990479
validation[          , genome  6332] predictions:    5932/  10000 (59.32%), best:    6176/10000 (61.76%), error:     12448.45996, best error:     11466.99121 on epoch:     0, epoch:   88/100, mu: 0.9860299826, learning_rate: 0.0000001010, weight_decay: 0.0000040311

epoch time: 1727.5487060547s, input_fired_time: 28.5484333038, output_fired_time: 11.6637573242, propagate_forward_time: 619.1902465820, propagate_backward_time: 1045.1169433594, weight_update_time: 1.0055694580, other_time: 23.0292968750
epoch time: 129.5457916260s, input_fired_time: 1.8968394995, output_fired_time: 0.0000000000, propagate_forward_time: 123.7308349609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9181137085
validation[          , genome  6332] predictions:    5896/  10000 (58.96%), best:    6176/10000 (61.76%), error:     12478.77441, best error:     11466.99121 on epoch:     0, epoch:   89/100, mu: 0.9862412214, learning_rate: 0.0000000937, weight_decay: 0.0000038113

epoch time: 1763.8979492188s, input_fired_time: 28.8871669769, output_fired_time: 11.8516178131, propagate_forward_time: 636.8536987305, propagate_backward_time: 1062.5170898438, weight_update_time: 0.9979401231, other_time: 23.7883300781
epoch time: 140.4657745361s, input_fired_time: 2.0608298779, output_fired_time: 0.0000000000, propagate_forward_time: 134.2423706055, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1625671387
validation[          , genome  6332] predictions:    5923/  10000 (59.23%), best:    6176/10000 (61.76%), error:     12435.95605, best error:     11466.99121 on epoch:     0, epoch:   90/100, mu: 0.9864412546, learning_rate: 0.0000000868, weight_decay: 0.0000036035

epoch time: 1795.2965087891s, input_fired_time: 29.4734878540, output_fired_time: 11.9422540665, propagate_forward_time: 648.7897338867, propagate_backward_time: 1081.5297851563, weight_update_time: 1.0204434395, other_time: 23.5612792969
epoch time: 131.9895782471s, input_fired_time: 1.9751594067, output_fired_time: 0.0000000000, propagate_forward_time: 126.0877532959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9266662598
validation[          , genome  6332] predictions:    5919/  10000 (59.19%), best:    6176/10000 (61.76%), error:     12416.48438, best error:     11466.99121 on epoch:     0, epoch:   91/100, mu: 0.9866306186, learning_rate: 0.0000000805, weight_decay: 0.0000034070

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6332_0_r497733984_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.501746
read mu: 0.98681
read mu_delta: 0.946786
read initial_learning_rate: 8.02232e-05
read learning_rate: 7.45966e-08
read learning_rate_delta: 0.926932
read initial_weight_decay: 0.000559758
read weight_decay: 3.22128e-06
read weight_decay_delta: 0.94548
read batch_size: 93
read epsilon: 1e-07
read alpha: 0.0748496
read input_dropout_probability: 0.00121485
read hidden_dropout_probability: 0.100745
read velocity_reset: 1870
read epoch: 92
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6176
read best_validation_error: 11467
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6332
read normal distribution: '1 0 0'
generator_str: '1315278887'
read generator: 1315278887
read generated_by_map:
1 disable_edge 1
reading 77 nodes.
reading 316 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1595.81872558594s, input_fired_time: 25.6804313659668, output_fired_time: 10.9097957611084, propagate_forward_time: 583.31298828125, propagate_backward_time: 954.453918457031, weight_update_time: 0.945079147815704, other_time: 21.4616088867188
epoch time: 124.78791809082s, input_fired_time: 1.86393237113953, output_fired_time: 0, propagate_forward_time: 119.037315368652, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.88667297363281
validation[          , genome  6332] predictions:    5922/  10000 (59.22%), best:    6176/10000 (61.76%), error:     12486.16113, best error:     11466.99121 on epoch:     0, epoch:   92/100, mu: 0.9868099093, learning_rate: 0.0000000746, weight_decay: 0.0000032213

epoch time: 1602.3410644531s, input_fired_time: 26.5052890778, output_fired_time: 11.2464628220, propagate_forward_time: 583.0800781250, propagate_backward_time: 959.6902465820, weight_update_time: 0.9352148771, other_time: 21.8190307617
epoch time: 119.8879699707s, input_fired_time: 1.7007808685, output_fired_time: 0.0000000000, propagate_forward_time: 114.4841918945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7029953003
validation[          , genome  6332] predictions:    5893/  10000 (58.93%), best:    6176/10000 (61.76%), error:     12463.24512, best error:     11466.99121 on epoch:     0, epoch:   93/100, mu: 0.9869796634, learning_rate: 0.0000000691, weight_decay: 0.0000030457

epoch time: 1591.0739746094s, input_fired_time: 26.0964450836, output_fired_time: 10.9555330276, propagate_forward_time: 581.4351806641, propagate_backward_time: 951.3841552734, weight_update_time: 0.9165194631, other_time: 21.2026367188
epoch time: 126.8237152100s, input_fired_time: 1.9938966036, output_fired_time: 0.0000000000, propagate_forward_time: 120.9334640503, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8963546753
validation[          , genome  6332] predictions:    5916/  10000 (59.16%), best:    6176/10000 (61.76%), error:     12455.01074, best error:     11466.99121 on epoch:     0, epoch:   94/100, mu: 0.9871404171, learning_rate: 0.0000000641, weight_decay: 0.0000028796

epoch time: 1620.7136230469s, input_fired_time: 26.1847229004, output_fired_time: 11.1748991013, propagate_forward_time: 594.6069946289, propagate_backward_time: 967.0604248047, weight_update_time: 0.9255919456, other_time: 21.6865844727
epoch time: 122.4336166382s, input_fired_time: 1.8014440536, output_fired_time: 0.0000000000, propagate_forward_time: 117.0285491943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6036224365
validation[          , genome  6332] predictions:    5923/  10000 (59.23%), best:    6176/10000 (61.76%), error:     12447.85938, best error:     11466.99121 on epoch:     0, epoch:   95/100, mu: 0.9872925878, learning_rate: 0.0000000594, weight_decay: 0.0000027226

epoch time: 1516.4970703125s, input_fired_time: 25.0533943176, output_fired_time: 10.2757749557, propagate_forward_time: 552.3585815430, propagate_backward_time: 908.6425170898, weight_update_time: 0.8518810868, other_time: 20.1668701172
epoch time: 121.3455276489s, input_fired_time: 1.7648593187, output_fired_time: 0.0000000000, propagate_forward_time: 116.0802230835, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5004425049
validation[          , genome  6332] predictions:    5932/  10000 (59.32%), best:    6176/10000 (61.76%), error:     12437.12500, best error:     11466.99121 on epoch:     0, epoch:   96/100, mu: 0.9874366522, learning_rate: 0.0000000551, weight_decay: 0.0000025742

epoch time: 1569.6905517578s, input_fired_time: 25.3391017914, output_fired_time: 10.5798616409, propagate_forward_time: 575.0190429688, propagate_backward_time: 938.1734008789, weight_update_time: 0.8648491502, other_time: 20.5791625977
epoch time: 124.2678604126s, input_fired_time: 1.7858543396, output_fired_time: 0.0000000000, propagate_forward_time: 118.8085021973, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6735076904
validation[          , genome  6332] predictions:    5891/  10000 (58.91%), best:    6176/10000 (61.76%), error:     12480.04590, best error:     11466.99121 on epoch:     0, epoch:   97/100, mu: 0.9875730872, learning_rate: 0.0000000510, weight_decay: 0.0000024338

epoch time: 1568.8223876953s, input_fired_time: 25.5388774872, output_fired_time: 10.7618303299, propagate_forward_time: 569.3440551758, propagate_backward_time: 942.4305419922, weight_update_time: 0.8811491728, other_time: 20.7471313477
epoch time: 117.8223495483s, input_fired_time: 1.7129771709, output_fired_time: 0.0000000000, propagate_forward_time: 112.5046844482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6046905518
validation[          , genome  6332] predictions:    5935/  10000 (59.35%), best:    6176/10000 (61.76%), error:     12448.48242, best error:     11466.99121 on epoch:     0, epoch:   98/100, mu: 0.9877022505, learning_rate: 0.0000000473, weight_decay: 0.0000023011

epoch time: 1646.9777832031s, input_fired_time: 26.5112171173, output_fired_time: 11.1903028488, propagate_forward_time: 602.7628784180, propagate_backward_time: 984.1566162109, weight_update_time: 0.9585992098, other_time: 22.3567504883
epoch time: 118.3847885132s, input_fired_time: 1.6808254719, output_fired_time: 0.0000000000, propagate_forward_time: 113.2668533325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4371109009
validation[          , genome  6332] predictions:    5899/  10000 (58.99%), best:    6176/10000 (61.76%), error:     12461.17969, best error:     11466.99121 on epoch:     0, epoch:   99/100, mu: 0.9878244996, learning_rate: 0.0000000439, weight_decay: 0.0000021757

epoch time: 1609.4473876953s, input_fired_time: 25.5547657013, output_fired_time: 10.8687496185, propagate_forward_time: 588.7797851563, propagate_backward_time: 962.7873535156, weight_update_time: 0.9593576193, other_time: 21.4566650391
epoch time: 125.7862243652s, input_fired_time: 1.8708986044, output_fired_time: 0.0000000000, propagate_forward_time: 120.3053741455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6099548340
validation[          , genome  6332] predictions:    5900/  10000 (59.00%), best:    6176/10000 (61.76%), error:     12476.26270, best error:     11466.99121 on epoch:     0, epoch:  100/100, mu: 0.9879402518, learning_rate: 0.0000000407, weight_decay: 0.0000020571

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 568.2543945313s, input_fired_time: 8.8873071671, output_fired_time: 0.0000000000, propagate_forward_time: 542.1047363281, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 17.2623291016
best training[          , genome  6332] predictions:   33758/  50000 (67.52%), best:    6176/10000 (61.76%), error:     51884.46094, best error:     11466.99121 on epoch:     0, epoch:  101/100, mu: 0.9880498648, learning_rate: 0.0000000377, weight_decay: 0.0000019449
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 110.2594833374s, input_fired_time: 1.7937581539, output_fired_time: 0.0000000000, propagate_forward_time: 105.2199172974, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2458114624
   testing[          , genome  6332] predictions:    6176/  10000 (61.76%), best:    6176/10000 (61.76%), error:     11466.99121, best error:     11466.99121 on epoch:     0, epoch:  101/100, mu: 0.9880498648, learning_rate: 0.0000000377, weight_decay: 0.0000019449
backpropagation finished successfully!
15:39:03 (7060): called boinc_finish(0)

</stderr_txt>
]]>