Nameexact_genome_1542649503_183_6334_2
Workunit4099334
Created20 Nov 2018, 1:52:09 UTC
Sent20 Nov 2018, 1:56:53 UTC
Report deadline28 Jan 2019, 15:14:55 UTC
Received22 Nov 2018, 1:19:44 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID78968
Run time1 days 16 hours 40 min 44 sec
CPU time1 days 16 hours 26 min 25 sec
Validate stateValid
Credit17,388.55
Device peak FLOPS2.20 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size522.05 MB
Peak swap size541.88 MB
Peak disk usage9.24 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
eading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.981033
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000228745
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 9.14978e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 78
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 70
read number_validation_images: 10000
read best_validation_predictions: 6156
read best_validation_error: 11568.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '1972491360'
read generator: 1972491360
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.981033
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000228745
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 9.14978e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 78
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 70
read number_validation_images: 10000
read best_validation_predictions: 6156
read best_validation_error: 11568.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '1972491360'
read generator: 1972491360
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.981033
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000228745
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 9.14978e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 78
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 70
read number_validation_images: 10000
read best_validation_predictions: 6156
read best_validation_error: 11568.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '1972491360'
read generator: 1972491360
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1437.33630371094s, input_fired_time: 13.2293186187744, output_fired_time: 10.7025260925293, propagate_forward_time: 520.976257324219, propagate_backward_time: 870.589782714844, weight_update_time: 1.69377815723419, other_time: 21.83837890625
epoch time: 97.5694351196289s, input_fired_time: 1.51766312122345, output_fired_time: 0, propagate_forward_time: 92.4853210449219, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.56645202636719
validation[          , genome  6334] predictions:    6177/  10000 (61.77%), best:    6156/10000 (61.56%), error:     11685.64844, best error:     11568.10254 on epoch:    70, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1393.6347656250s, input_fired_time: 13.2003221512, output_fired_time: 10.4951400757, propagate_forward_time: 503.2878112793, propagate_backward_time: 846.0803222656, weight_update_time: 1.6802494526, other_time: 20.5711669922
epoch time: 107.9589004517s, input_fired_time: 1.6642711163, output_fired_time: 0.0000000000, propagate_forward_time: 102.4796295166, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8150024414
validation[          , genome  6334] predictions:    6218/  10000 (62.18%), best:    6218/10000 (62.18%), error:     11513.51563, best error:     11513.51563 on epoch:    79, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1354.3576660156s, input_fired_time: 12.9254293442, output_fired_time: 10.2831907272, propagate_forward_time: 483.4773864746, propagate_backward_time: 828.1177368164, weight_update_time: 1.6255913973, other_time: 19.5538940430
epoch time: 108.3339920044s, input_fired_time: 1.6663260460, output_fired_time: 0.0000000000, propagate_forward_time: 102.9816513062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6860122681
validation[          , genome  6334] predictions:    6180/  10000 (61.80%), best:    6218/10000 (62.18%), error:     11550.46191, best error:     11513.51563 on epoch:    79, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.982312
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00019612
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.84479e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 81
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 79
read number_validation_images: 10000
read best_validation_predictions: 6218
read best_validation_error: 11513.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '1576213159'
read generator: 1576213159
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1396.92309570313s, input_fired_time: 13.3032217025757, output_fired_time: 10.6536741256714, propagate_forward_time: 505.230316162109, propagate_backward_time: 846.323120117188, weight_update_time: 1.66295993328094, other_time: 21.4127197265625
epoch time: 111.611793518066s, input_fired_time: 1.67427182197571, output_fired_time: 0, propagate_forward_time: 105.931022644043, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.00650024414063
validation[          , genome  6334] predictions:    6217/  10000 (62.17%), best:    6218/10000 (62.18%), error:     11606.35449, best error:     11513.51563 on epoch:    79, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1398.5588378906s, input_fired_time: 13.1625423431, output_fired_time: 10.4289388657, propagate_forward_time: 506.2355346680, propagate_backward_time: 848.6347656250, weight_update_time: 1.6553688049, other_time: 20.0969848633
epoch time: 106.7484512329s, input_fired_time: 1.6033804417, output_fired_time: 0.0000000000, propagate_forward_time: 101.4636764526, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6813964844
validation[          , genome  6334] predictions:    6232/  10000 (62.32%), best:    6232/10000 (62.32%), error:     11506.00879, best error:     11506.00879 on epoch:    82, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.983062
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000176998
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.07993e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 83
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 82
read number_validation_images: 10000
read best_validation_predictions: 6232
read best_validation_error: 11506
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '294834672'
read generator: 294834672
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.983062
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000176998
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.07993e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 83
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 82
read number_validation_images: 10000
read best_validation_predictions: 6232
read best_validation_error: 11506
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '294834672'
read generator: 294834672
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.983062
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000176998
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.07993e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 83
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 82
read number_validation_images: 10000
read best_validation_predictions: 6232
read best_validation_error: 11506
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '294834672'
read generator: 294834672
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.983062
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000176998
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.07993e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 83
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 82
read number_validation_images: 10000
read best_validation_predictions: 6232
read best_validation_error: 11506
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '294834672'
read generator: 294834672
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1424.92297363281s, input_fired_time: 13.2179861068726, output_fired_time: 10.775032043457, propagate_forward_time: 518.215637207031, propagate_backward_time: 860.906005859375, weight_update_time: 1.6715726852417, other_time: 21.8082885742188
epoch time: 105.535934448242s, input_fired_time: 1.62983667850494, output_fired_time: 0, propagate_forward_time: 100.16463470459, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.74146270751953
validation[          , genome  6334] predictions:    6167/  10000 (61.67%), best:    6232/10000 (62.32%), error:     11736.61523, best error:     11506.00879 on epoch:    82, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1404.6309814453s, input_fired_time: 13.2606811523, output_fired_time: 10.7197484970, propagate_forward_time: 506.8314208984, propagate_backward_time: 852.9414672852, weight_update_time: 1.6801657677, other_time: 20.8777465820
epoch time: 107.3626022339s, input_fired_time: 1.6509718895, output_fired_time: 0.0000000000, propagate_forward_time: 101.9886627197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7229690552
validation[          , genome  6334] predictions:    6209/  10000 (62.09%), best:    6232/10000 (62.32%), error:     11564.87109, best error:     11506.00879 on epoch:    82, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 1427.6060791016s, input_fired_time: 13.3047771454, output_fired_time: 10.7045936584, propagate_forward_time: 517.8967895508, propagate_backward_time: 864.3823852539, weight_update_time: 1.7026612759, other_time: 21.3175048828
epoch time: 110.2153701782s, input_fired_time: 1.6684982777, output_fired_time: 0.0000000000, propagate_forward_time: 104.7562789917, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7905960083
validation[          , genome  6334] predictions:    6246/  10000 (62.46%), best:    6246/10000 (62.46%), error:     11504.26563, best error:     11504.26563 on epoch:    85, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1435.0537109375s, input_fired_time: 13.2753448486, output_fired_time: 10.7829771042, propagate_forward_time: 517.9602661133, propagate_backward_time: 871.7454833984, weight_update_time: 1.6919807196, other_time: 21.2896118164
epoch time: 108.2333450317s, input_fired_time: 1.6155560017, output_fired_time: 0.0000000000, propagate_forward_time: 102.9462432861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6715469360
validation[          , genome  6334] predictions:    6259/  10000 (62.59%), best:    6259/10000 (62.59%), error:     11417.75195, best error:     11417.75195 on epoch:    86, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1434.1499023438s, input_fired_time: 13.3789300919, output_fired_time: 10.8734922409, propagate_forward_time: 521.5687866211, propagate_backward_time: 866.9844360352, weight_update_time: 1.6812139750, other_time: 21.3442382813
epoch time: 100.8465576172s, input_fired_time: 1.5930287838, output_fired_time: 0.0000000000, propagate_forward_time: 95.6569595337, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5965728760
validation[          , genome  6334] predictions:    6220/  10000 (62.20%), best:    6259/10000 (62.59%), error:     11519.47949, best error:     11417.75195 on epoch:    86, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1418.5577392578s, input_fired_time: 13.2396936417, output_fired_time: 10.8385009766, propagate_forward_time: 512.6230468750, propagate_backward_time: 860.7262573242, weight_update_time: 1.6740043163, other_time: 21.1301879883
epoch time: 111.4471511841s, input_fired_time: 1.6649243832, output_fired_time: 0.0000000000, propagate_forward_time: 106.0130615234, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7691650391
validation[          , genome  6334] predictions:    6252/  10000 (62.52%), best:    6259/10000 (62.59%), error:     11483.74414, best error:     11417.75195 on epoch:    86, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1394.4675292969s, input_fired_time: 13.1153602600, output_fired_time: 10.5723981857, propagate_forward_time: 505.2584228516, propagate_backward_time: 844.5619506836, weight_update_time: 1.6402407885, other_time: 20.9594116211
epoch time: 107.3465881348s, input_fired_time: 1.6451085806, output_fired_time: 0.0000000000, propagate_forward_time: 102.0113906860, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6900863647
validation[          , genome  6334] predictions:    6192/  10000 (61.92%), best:    6259/10000 (62.59%), error:     11612.40430, best error:     11417.75195 on epoch:    86, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1405.2170410156s, input_fired_time: 13.2148523331, output_fired_time: 10.8305931091, propagate_forward_time: 505.6956176758, propagate_backward_time: 854.5522460938, weight_update_time: 1.6677970886, other_time: 20.9237670898
epoch time: 108.3386383057s, input_fired_time: 1.6374261379, output_fired_time: 0.0000000000, propagate_forward_time: 102.9447021484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7565078735
validation[          , genome  6334] predictions:    6255/  10000 (62.55%), best:    6259/10000 (62.59%), error:     11482.10352, best error:     11417.75195 on epoch:    86, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1402.3555908203s, input_fired_time: 13.4449748993, output_fired_time: 10.5948219299, propagate_forward_time: 506.2572326660, propagate_backward_time: 851.7011108398, weight_update_time: 1.6582622528, other_time: 20.3574829102
epoch time: 105.8267593384s, input_fired_time: 1.6779069901, output_fired_time: 0.0000000000, propagate_forward_time: 100.3308792114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8179702759
validation[          , genome  6334] predictions:    6253/  10000 (62.53%), best:    6259/10000 (62.59%), error:     11595.87207, best error:     11417.75195 on epoch:    86, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1410.9671630859s, input_fired_time: 13.5432949066, output_fired_time: 10.5780649185, propagate_forward_time: 504.8654479980, propagate_backward_time: 861.2756347656, weight_update_time: 1.7083803415, other_time: 20.7045898438
epoch time: 103.7187423706s, input_fired_time: 1.6381540298, output_fired_time: 0.0000000000, propagate_forward_time: 98.5874328613, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4931564331
validation[          , genome  6334] predictions:    6236/  10000 (62.36%), best:    6259/10000 (62.59%), error:     11528.91992, best error:     11417.75195 on epoch:    86, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1395.7744140625s, input_fired_time: 13.2848138809, output_fired_time: 10.5510644913, propagate_forward_time: 498.6725158691, propagate_backward_time: 852.6622924805, weight_update_time: 1.6670845747, other_time: 20.6038208008
epoch time: 116.2217864990s, input_fired_time: 1.7959218025, output_fired_time: 0.0000000000, propagate_forward_time: 110.3616333008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0642318726
validation[          , genome  6334] predictions:    6230/  10000 (62.30%), best:    6259/10000 (62.59%), error:     11562.39063, best error:     11417.75195 on epoch:    86, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1428.2163085938s, input_fired_time: 13.5164413452, output_fired_time: 10.7010850906, propagate_forward_time: 515.9564819336, propagate_backward_time: 867.7266845703, weight_update_time: 1.6739844084, other_time: 20.3156127930
epoch time: 105.5181503296s, input_fired_time: 1.6407427788, output_fired_time: 0.0000000000, propagate_forward_time: 100.2062683105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6711425781
validation[          , genome  6334] predictions:    6289/  10000 (62.89%), best:    6259/10000 (62.59%), error:     11558.78320, best error:     11417.75195 on epoch:    86, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1447.5222167969s, input_fired_time: 13.6978435516, output_fired_time: 10.9100828171, propagate_forward_time: 521.5783691406, propagate_backward_time: 880.4790649414, weight_update_time: 1.7303081751, other_time: 20.8568725586
epoch time: 107.5739746094s, input_fired_time: 1.6305911541, output_fired_time: 0.0000000000, propagate_forward_time: 102.1088867188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8344955444
validation[          , genome  6334] predictions:    6216/  10000 (62.16%), best:    6259/10000 (62.59%), error:     11633.56250, best error:     11417.75195 on epoch:    86, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_183_6334_2_r2064773809_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.986438
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 9.08606e-05
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 3.63442e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 96
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 6259
read best_validation_error: 11417.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6334
read normal distribution: '1 0 0'
generator_str: '938371199'
read generator: 938371199
read generated_by_map:
1 enable_node 1
reading 67 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1383.72875976563s, input_fired_time: 13.291651725769, output_fired_time: 10.4077234268188, propagate_forward_time: 494.234771728516, propagate_backward_time: 846.107543945313, weight_update_time: 1.64069497585297, other_time: 19.6871337890625
epoch time: 100.456726074219s, input_fired_time: 1.57908654212952, output_fired_time: 0, propagate_forward_time: 95.5414657592773, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.33617401123047
validation[          , genome  6334] predictions:    6262/  10000 (62.62%), best:    6259/10000 (62.59%), error:     11510.76758, best error:     11417.75195 on epoch:    86, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1373.9873046875s, input_fired_time: 13.1459474564, output_fired_time: 10.4953641891, propagate_forward_time: 494.0155334473, propagate_backward_time: 836.7557373047, weight_update_time: 1.6251244545, other_time: 19.5747070313
epoch time: 103.0714797974s, input_fired_time: 1.5963635445, output_fired_time: 0.0000000000, propagate_forward_time: 97.9166183472, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5584945679
validation[          , genome  6334] predictions:    6249/  10000 (62.49%), best:    6259/10000 (62.59%), error:     11460.51270, best error:     11417.75195 on epoch:    86, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1410.2489013672s, input_fired_time: 13.5263519287, output_fired_time: 10.6776599884, propagate_forward_time: 502.5435180664, propagate_backward_time: 863.2035522461, weight_update_time: 1.6984634399, other_time: 20.2978515625
epoch time: 111.5254745483s, input_fired_time: 1.6999330521, output_fired_time: 0.0000000000, propagate_forward_time: 106.0727920532, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7527465820
validation[          , genome  6334] predictions:    6224/  10000 (62.24%), best:    6259/10000 (62.59%), error:     11589.42383, best error:     11417.75195 on epoch:    86, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1407.1286621094s, input_fired_time: 13.4746894836, output_fired_time: 10.5374631882, propagate_forward_time: 504.0652160645, propagate_backward_time: 858.7780761719, weight_update_time: 1.7147700787, other_time: 20.2731933594
epoch time: 98.8739852905s, input_fired_time: 1.5852365494, output_fired_time: 0.0000000000, propagate_forward_time: 93.9111557007, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3775939941
validation[          , genome  6334] predictions:    6094/  10000 (60.94%), best:    6259/10000 (62.59%), error:     12093.36621, best error:     11417.75195 on epoch:    86, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1403.5401611328s, input_fired_time: 13.4566955566, output_fired_time: 10.4638404846, propagate_forward_time: 504.7348327637, propagate_backward_time: 854.8684082031, weight_update_time: 1.6804360151, other_time: 20.0163574219
epoch time: 104.8124084473s, input_fired_time: 1.6456341743, output_fired_time: 0.0000000000, propagate_forward_time: 99.6497421265, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5170288086
validation[          , genome  6334] predictions:    6219/  10000 (62.19%), best:    6259/10000 (62.59%), error:     11547.85938, best error:     11417.75195 on epoch:    86, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 511.2907714844s, input_fired_time: 8.0034704208, output_fired_time: 0.0000000000, propagate_forward_time: 485.9877624512, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 17.2995300293
best training[          , genome  6334] predictions:   39658/  50000 (79.32%), best:    6259/10000 (62.59%), error:     31003.89063, best error:     11417.75195 on epoch:    86, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 106.0253829956s, input_fired_time: 1.6598119736, output_fired_time: 0.0000000000, propagate_forward_time: 100.8312911987, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5342788696
   testing[          , genome  6334] predictions:    6259/  10000 (62.59%), best:    6259/10000 (62.59%), error:     11417.75195, best error:     11417.75195 on epoch:    86, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
17:19:18 (6588): called boinc_finish(0)

</stderr_txt>
]]>