Nameexact_genome_1542031203_181_5605_0
Workunit4081695
Created12 Nov 2018, 19:22:47 UTC
Sent12 Nov 2018, 19:28:18 UTC
Report deadline1 Jan 2019, 16:18:44 UTC
Received25 Nov 2018, 5:16:23 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID87539
Run time1 days 18 hours 36 min 50 sec
CPU time1 days 8 hours 35 min 48 sec
Validate stateValid
Credit12,467.09
Device peak FLOPS2.54 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34

Stderr output

<core_client_version>7.0.80</core_client_version>
<![CDATA[
<stderr_txt>

boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.887688
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000721318
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 6.46315e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 24
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 21
read number_validation_images: 10000
read best_validation_predictions: 5996
read best_validation_error: 11388
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1895482414'
read generator: 1895482414
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 5946.04736328125s, input_fired_time: 59.599967956543, output_fired_time: 37.326229095459, propagate_forward_time: 2198.240234375, propagate_backward_time: 3339.84423828125, weight_update_time: 0.549349427223206, other_time: 311.03662109375
epoch time: 416.031005859375s, input_fired_time: 6.55445432662964, output_fired_time: 0, propagate_forward_time: 392.660888671875, propagate_backward_time: 0, weight_update_time: 0, other_time: 16.815673828125
validation[          , genome  5605] predictions:    6067/  10000 (60.67%), best:    6067/10000 (60.67%), error:     11381.92578, best error:     11381.92578 on epoch:    24, epoch:   24/50, mu: 0.8876883388, learning_rate: 0.0007213180, weight_decay: 0.0000646315

epoch time: 6248.7519531250s, input_fired_time: 69.8233871460, output_fired_time: 30.9865245819, propagate_forward_time: 2311.6708984375, propagate_backward_time: 3575.2731933594, weight_update_time: 2.5160226822, other_time: 260.9982910156
epoch time: 591.3076782227s, input_fired_time: 3.5513808727, output_fired_time: 0.0000000000, propagate_forward_time: 561.2966918945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 26.4595947266
validation[          , genome  5605] predictions:    6099/  10000 (60.99%), best:    6099/10000 (60.99%), error:     11259.61328, best error:     11259.61328 on epoch:    25, epoch:   25/50, mu: 0.8942435384, learning_rate: 0.0006621272, weight_decay: 0.0000601098

epoch time: 7675.0629882813s, input_fired_time: 70.2190856934, output_fired_time: 31.9012336731, propagate_forward_time: 2515.5402832031, propagate_backward_time: 3804.3967285156, weight_update_time: 0.6459608674, other_time: 1253.0056152344
epoch time: 460.5301513672s, input_fired_time: 2.8475704193, output_fired_time: 0.0000000000, propagate_forward_time: 426.1775207520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 31.5050659180
validation[          , genome  5605] predictions:    6019/  10000 (60.19%), best:    6099/10000 (60.99%), error:     11383.51465, best error:     11259.61328 on epoch:    25, epoch:   26/50, mu: 0.9003787637, learning_rate: 0.0006077936, weight_decay: 0.0000559045

epoch time: 8642.9013671875s, input_fired_time: 67.0478897095, output_fired_time: 32.5848884583, propagate_forward_time: 2361.3757324219, propagate_backward_time: 3574.7250976563, weight_update_time: 0.6526572704, other_time: 2607.1674804688
epoch time: 423.6106567383s, input_fired_time: 1.5974394083, output_fired_time: 0.0000000000, propagate_forward_time: 386.2016296387, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 35.8115844727
validation[          , genome  5605] predictions:    6071/  10000 (60.71%), best:    6099/10000 (60.99%), error:     11360.15820, best error:     11259.61328 on epoch:    25, epoch:   27/50, mu: 0.9061208963, learning_rate: 0.0005579185, weight_decay: 0.0000519933

epoch time: 5723.7114257813s, input_fired_time: 79.2547836304, output_fired_time: 23.1160755157, propagate_forward_time: 2076.0683593750, propagate_backward_time: 3169.3032226563, weight_update_time: 0.5282089114, other_time: 375.9687500000
epoch time: 583.0930175781s, input_fired_time: 6.4150404930, output_fired_time: 0.0000000000, propagate_forward_time: 550.3756713867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 26.3023071289
validation[          , genome  5605] predictions:    6095/  10000 (60.95%), best:    6099/10000 (60.99%), error:     11267.19336, best error:     11259.61328 on epoch:    25, epoch:   28/50, mu: 0.9114951491, learning_rate: 0.0005121361, weight_decay: 0.0000483558

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.916525
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000470111
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 4.49728e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 29
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 25
read number_validation_images: 10000
read best_validation_predictions: 6099
read best_validation_error: 11259.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1209456442'
read generator: 1209456442
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.916525
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000470111
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 4.49728e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 29
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 25
read number_validation_images: 10000
read best_validation_predictions: 6099
read best_validation_error: 11259.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1209456442'
read generator: 1209456442
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 5222.45703125s, input_fired_time: 71.4461822509766, output_fired_time: 32.9514083862305, propagate_forward_time: 1986.32177734375, propagate_backward_time: 3080.20678710938, weight_update_time: 1.51935589313507, other_time: 51.531005859375
epoch time: 433.083648681641s, input_fired_time: 1.79500603675842, output_fired_time: 0, propagate_forward_time: 419.763977050781, propagate_backward_time: 0, weight_update_time: 0, other_time: 11.524658203125
validation[          , genome  5605] predictions:    6100/  10000 (61.00%), best:    6099/10000 (60.99%), error:     11260.12598, best error:     11259.61328 on epoch:    25, epoch:   29/50, mu: 0.9165250659, learning_rate: 0.0004701107, weight_decay: 0.0000449728

epoch time: 7807.9399414063s, input_fired_time: 57.7582397461, output_fired_time: 27.6132259369, propagate_forward_time: 2075.2661132813, propagate_backward_time: 3203.8076171875, weight_update_time: 0.5706821084, other_time: 2443.4946289063
epoch time: 379.6031188965s, input_fired_time: 4.4584989548, output_fired_time: 0.0000000000, propagate_forward_time: 324.6874084473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 50.4572143555
validation[          , genome  5605] predictions:    6070/  10000 (60.70%), best:    6099/10000 (60.99%), error:     11298.35156, best error:     11259.61328 on epoch:    25, epoch:   30/50, mu: 0.9212327003, learning_rate: 0.0004315338, weight_decay: 0.0000418264

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.925639
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000396122
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 3.89002e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 31
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 25
read number_validation_images: 10000
read best_validation_predictions: 6099
read best_validation_error: 11259.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '93158933'
read generator: 93158933
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 9614.3466796875s, input_fired_time: 99.3347854614258, output_fired_time: 31.6295795440674, propagate_forward_time: 3492.71215820313, propagate_backward_time: 5301.578125, weight_update_time: 4.85110664367676, other_time: 689.091796875
epoch time: 572.649475097656s, input_fired_time: 11.4890518188477, output_fired_time: 0, propagate_forward_time: 522.843811035156, propagate_backward_time: 0, weight_update_time: 0, other_time: 38.3165893554688
validation[          , genome  5605] predictions:    6087/  10000 (60.87%), best:    6099/10000 (60.99%), error:     11287.83301, best error:     11259.61328 on epoch:    25, epoch:   31/50, mu: 0.9256386757, learning_rate: 0.0003961224, weight_decay: 0.0000389002

epoch time: 7975.3808593750s, input_fired_time: 80.7122497559, output_fired_time: 39.3371353149, propagate_forward_time: 2640.9226074219, propagate_backward_time: 3932.9436035156, weight_update_time: 0.6119717360, other_time: 1281.4655761719
epoch time: 769.2255249023s, input_fired_time: 7.9494752884, output_fired_time: 0.0000000000, propagate_forward_time: 736.4564208984, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 24.8196411133
validation[          , genome  5605] predictions:    6160/  10000 (61.60%), best:    6160/10000 (61.60%), error:     11107.30566, best error:     11107.30566 on epoch:    32, epoch:   32/50, mu: 0.9297623634, learning_rate: 0.0003636169, weight_decay: 0.0000361787

epoch time: 9112.8320312500s, input_fired_time: 108.9736404419, output_fired_time: 50.0459175110, propagate_forward_time: 3145.8098144531, propagate_backward_time: 4796.7866210938, weight_update_time: 2.8997960091, other_time: 1011.2163085938
epoch time: 444.2592468262s, input_fired_time: 3.6139266491, output_fired_time: 0.0000000000, propagate_forward_time: 392.0468750000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 48.5984497070
validation[          , genome  5605] predictions:    6120/  10000 (61.20%), best:    6160/10000 (61.60%), error:     11179.01660, best error:     11107.30566 on epoch:    32, epoch:   33/50, mu: 0.9336218238, learning_rate: 0.0003337788, weight_decay: 0.0000336476

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.937234
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000306389
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 3.12936e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 34
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 32
read number_validation_images: 10000
read best_validation_predictions: 6160
read best_validation_error: 11107.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '790916650'
read generator: 790916650
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 5972.34326171875s, input_fired_time: 79.2924652099609, output_fired_time: 34.6082954406738, propagate_forward_time: 2258.17822265625, propagate_backward_time: 3482.94970703125, weight_update_time: 5.52377653121948, other_time: 117.314453125
epoch time: 569.838073730469s, input_fired_time: 4.25295066833496, output_fired_time: 0, propagate_forward_time: 549.857238769531, propagate_backward_time: 0, weight_update_time: 0, other_time: 15.7279052734375
validation[          , genome  5605] predictions:    6167/  10000 (61.67%), best:    6167/10000 (61.67%), error:     11090.78613, best error:     11090.78613 on epoch:    34, epoch:   34/50, mu: 0.9372340441, learning_rate: 0.0003063892, weight_decay: 0.0000312936

epoch time: 7283.6811523438s, input_fired_time: 71.0215835571, output_fired_time: 35.3101348877, propagate_forward_time: 2589.2353515625, propagate_backward_time: 3908.5502929688, weight_update_time: 1.6600415707, other_time: 679.5639648438
epoch time: 455.2496643066s, input_fired_time: 3.8142399788, output_fired_time: 0.0000000000, propagate_forward_time: 425.0504455566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 26.3849792480
validation[          , genome  5605] predictions:    6127/  10000 (61.27%), best:    6167/10000 (61.67%), error:     11233.70801, best error:     11090.78613 on epoch:    34, epoch:   35/50, mu: 0.9406148195, learning_rate: 0.0002812472, weight_decay: 0.0000291042

epoch time: 5677.7421875000s, input_fired_time: 70.3445663452, output_fired_time: 28.4405689240, propagate_forward_time: 2177.5397949219, propagate_backward_time: 3336.9733886719, weight_update_time: 1.5926337242, other_time: 64.4438476563
epoch time: 523.0934448242s, input_fired_time: 3.1270139217, output_fired_time: 0.0000000000, propagate_forward_time: 503.3507385254, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 16.6156921387
validation[          , genome  5605] predictions:    6147/  10000 (61.47%), best:    6167/10000 (61.67%), error:     11132.34961, best error:     11090.78613 on epoch:    34, epoch:   36/50, mu: 0.9437789917, learning_rate: 0.0002581682, weight_decay: 0.0000270681

epoch time: 10192.5888671875s, input_fired_time: 73.1198501587, output_fired_time: 32.9469223022, propagate_forward_time: 2913.3356933594, propagate_backward_time: 4370.3925781250, weight_update_time: 0.7614646554, other_time: 2802.7929687500
epoch time: 418.4084167480s, input_fired_time: 3.5249116421, output_fired_time: 0.0000000000, propagate_forward_time: 307.1059570313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 107.7775573730
validation[          , genome  5605] predictions:    6104/  10000 (61.04%), best:    6167/10000 (61.67%), error:     11233.72559, best error:     11090.78613 on epoch:    34, epoch:   37/50, mu: 0.9467404485, learning_rate: 0.0002369831, weight_decay: 0.0000251744

epoch time: 13130.7998046875s, input_fired_time: 73.7002029419, output_fired_time: 20.6671409607, propagate_forward_time: 1671.5078125000, propagate_backward_time: 2577.4470214844, weight_update_time: 5.4846606255, other_time: 8787.4775390625
epoch time: 306.0791625977s, input_fired_time: 3.0863204002, output_fired_time: 0.0000000000, propagate_forward_time: 222.7593994141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 80.2334289551
validation[          , genome  5605] predictions:    6277/  10000 (62.77%), best:    6277/10000 (62.77%), error:     10880.24023, best error:     10880.24023 on epoch:    38, epoch:   38/50, mu: 0.9495121241, learning_rate: 0.0002175365, weight_decay: 0.0000234131

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.952106
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000199686
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 2.17751e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 39
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 38
read number_validation_images: 10000
read best_validation_predictions: 6277
read best_validation_error: 10880.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1862797979'
read generator: 1862797979
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 4878.75341796875s, input_fired_time: 76.4689407348633, output_fired_time: 29.0722007751465, propagate_forward_time: 1808.94714355469, propagate_backward_time: 2802.95947265625, weight_update_time: 0.478964030742645, other_time: 161.3056640625
epoch time: 448.4775390625s, input_fired_time: 2.81804037094116, output_fired_time: 0, propagate_forward_time: 428.403686523438, propagate_backward_time: 0, weight_update_time: 0, other_time: 17.2557983398438
validation[          , genome  5605] predictions:    6257/  10000 (62.57%), best:    6257/10000 (62.57%), error:     10816.17578, best error:     10816.17578 on epoch:    39, epoch:   39/50, mu: 0.9521062374, learning_rate: 0.0001996856, weight_decay: 0.0000217751

epoch time: 6211.6953125000s, input_fired_time: 78.3754653931, output_fired_time: 35.3983192444, propagate_forward_time: 2311.5266113281, propagate_backward_time: 3490.9191894531, weight_update_time: 0.6248071790, other_time: 295.4755859375
epoch time: 483.2182617188s, input_fired_time: 1.8768583536, output_fired_time: 0.0000000000, propagate_forward_time: 464.4101867676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 16.9312133789
validation[          , genome  5605] predictions:    6268/  10000 (62.68%), best:    6268/10000 (62.68%), error:     10789.51660, best error:     10789.51660 on epoch:    40, epoch:   40/50, mu: 0.9545341134, learning_rate: 0.0001832996, weight_decay: 0.0000202517

epoch time: 7281.6591796875s, input_fired_time: 89.3087921143, output_fired_time: 21.8668155670, propagate_forward_time: 2318.0737304688, propagate_backward_time: 3506.9777832031, weight_update_time: 2.5220563412, other_time: 1345.4323730469
epoch time: 464.0840454102s, input_fired_time: 5.7903423309, output_fired_time: 0.0000000000, propagate_forward_time: 417.9414978027, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 40.3522033691
validation[          , genome  5605] predictions:    6219/  10000 (62.19%), best:    6268/10000 (62.68%), error:     10873.00293, best error:     10789.51660 on epoch:    40, epoch:   41/50, mu: 0.9568064213, learning_rate: 0.0001682582, weight_decay: 0.0000188349

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.958933
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000154451
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 1.75172e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 42
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 40
read number_validation_images: 10000
read best_validation_predictions: 6268
read best_validation_error: 10789.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '361592338'
read generator: 361592338
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 7901.73779296875s, input_fired_time: 80.9863739013672, output_fired_time: 34.0686683654785, propagate_forward_time: 2812.96069335938, propagate_backward_time: 4193.6044921875, weight_update_time: 2.86669111251831, other_time: 780.1171875
epoch time: 666.132873535156s, input_fired_time: 8.73294258117676, output_fired_time: 0, propagate_forward_time: 638.170043945313, propagate_backward_time: 0, weight_update_time: 0, other_time: 19.2298583984375
validation[          , genome  5605] predictions:    6240/  10000 (62.40%), best:    6268/10000 (62.68%), error:     10874.02246, best error:     10789.51660 on epoch:    40, epoch:   42/50, mu: 0.9589331746, learning_rate: 0.0001544510, weight_decay: 0.0000175172

epoch time: 6752.8920898438s, input_fired_time: 79.0924530029, output_fired_time: 26.7272472382, propagate_forward_time: 2583.2636718750, propagate_backward_time: 3890.3242187500, weight_update_time: 2.6449227333, other_time: 173.4848632813
epoch time: 448.7022399902s, input_fired_time: 3.8615550995, output_fired_time: 0.0000000000, propagate_forward_time: 416.8335571289, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 28.0071411133
validation[          , genome  5605] predictions:    6229/  10000 (62.29%), best:    6268/10000 (62.68%), error:     10892.34375, best error:     10789.51660 on epoch:    40, epoch:   43/50, mu: 0.9609236717, learning_rate: 0.0001417769, weight_decay: 0.0000162917

epoch time: 5826.5527343750s, input_fired_time: 61.6954345703, output_fired_time: 23.9980850220, propagate_forward_time: 2052.3066406250, propagate_backward_time: 3134.0114746094, weight_update_time: 0.5380759239, other_time: 554.5412597656
epoch time: 470.7665405273s, input_fired_time: 10.8830013275, output_fired_time: 0.0000000000, propagate_forward_time: 431.8778381348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 28.0057067871
validation[          , genome  5605] predictions:    6255/  10000 (62.55%), best:    6268/10000 (62.68%), error:     10807.62500, best error:     10789.51660 on epoch:    40, epoch:   44/50, mu: 0.9627866149, learning_rate: 0.0001301428, weight_decay: 0.0000151519

epoch time: 5860.0615234375s, input_fired_time: 73.0758438110, output_fired_time: 29.2733116150, propagate_forward_time: 2039.4611816406, propagate_backward_time: 3130.8664550781, weight_update_time: 0.5367062092, other_time: 587.3847656250
epoch time: 422.9778747559s, input_fired_time: 2.6998410225, output_fired_time: 0.0000000000, propagate_forward_time: 392.3783569336, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 27.8996887207
validation[          , genome  5605] predictions:    6222/  10000 (62.22%), best:    6268/10000 (62.68%), error:     10891.68945, best error:     10789.51660 on epoch:    40, epoch:   45/50, mu: 0.9645302296, learning_rate: 0.0001194634, weight_decay: 0.0000140918

epoch time: 5945.0161132813s, input_fired_time: 62.1727409363, output_fired_time: 38.9563674927, propagate_forward_time: 2005.9362792969, propagate_backward_time: 3020.4978027344, weight_update_time: 0.5726214051, other_time: 817.4526367188
epoch time: 404.8652954102s, input_fired_time: 3.6625084877, output_fired_time: 0.0000000000, propagate_forward_time: 358.2151184082, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 42.9876708984
validation[          , genome  5605] predictions:    6238/  10000 (62.38%), best:    6268/10000 (62.68%), error:     10803.82324, best error:     10789.51660 on epoch:    40, epoch:   46/50, mu: 0.9661620855, learning_rate: 0.0001096603, weight_decay: 0.0000131059

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.967689
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000100662
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 1.2189e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 47
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 40
read number_validation_images: 10000
read best_validation_predictions: 6268
read best_validation_error: 10789.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1778115754'
read generator: 1778115754
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 6124.8271484375s, input_fired_time: 88.1279830932617, output_fired_time: 30.3530464172363, propagate_forward_time: 2268.78100585938, propagate_backward_time: 3475.7490234375, weight_update_time: 0.604273974895477, other_time: 261.816162109375
epoch time: 525.895751953125s, input_fired_time: 5.18912124633789, output_fired_time: 0, propagate_forward_time: 505.908569335938, propagate_backward_time: 0, weight_update_time: 0, other_time: 14.7980346679688
validation[          , genome  5605] predictions:    6286/  10000 (62.86%), best:    6286/10000 (62.86%), error:     10734.12402, best error:     10734.12402 on epoch:    47, epoch:   47/50, mu: 0.9676893950, learning_rate: 0.0001006617, weight_decay: 0.0000121890

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_0_r1500952523_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.969119
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 9.24015e-05
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 1.13363e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 48
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 47
read number_validation_images: 10000
read best_validation_predictions: 6286
read best_validation_error: 10734.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '249646630'
read generator: 249646630
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 5743.85302734375s, input_fired_time: 86.0247039794922, output_fired_time: 22.9161834716797, propagate_forward_time: 2156.63696289063, propagate_backward_time: 3353.56396484375, weight_update_time: 2.57600617408752, other_time: 124.711181640625
epoch time: 480.469543457031s, input_fired_time: 6.1327977180481, output_fired_time: 0, propagate_forward_time: 462.153198242188, propagate_backward_time: 0, weight_update_time: 0, other_time: 12.1835327148438
validation[          , genome  5605] predictions:    6281/  10000 (62.81%), best:    6286/10000 (62.86%), error:     10766.98438, best error:     10734.12402 on epoch:    47, epoch:   48/50, mu: 0.9691188335, learning_rate: 0.0000924015, weight_decay: 0.0000113363

epoch time: 6125.2866210938s, input_fired_time: 62.3876342773, output_fired_time: 29.9537601471, propagate_forward_time: 2223.6555175781, propagate_backward_time: 3348.4099121094, weight_update_time: 0.5970700383, other_time: 460.8798828125
epoch time: 442.8215332031s, input_fired_time: 3.7871935368, output_fired_time: 0.0000000000, propagate_forward_time: 413.5158691406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 25.5184631348
validation[          , genome  5605] predictions:    6302/  10000 (63.02%), best:    6286/10000 (62.86%), error:     10772.02441, best error:     10734.12402 on epoch:    47, epoch:   49/50, mu: 0.9704567194, learning_rate: 0.0000848191, weight_decay: 0.0000105432

epoch time: 5931.3818359375s, input_fired_time: 81.0589447021, output_fired_time: 20.3900814056, propagate_forward_time: 1974.2185058594, propagate_backward_time: 2993.9599609375, weight_update_time: 2.5262620449, other_time: 861.7541503906
epoch time: 453.6474914551s, input_fired_time: 6.9705791473, output_fired_time: 0.0000000000, propagate_forward_time: 428.8654174805, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 17.8114929199
validation[          , genome  5605] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10693.95410, best error:     10693.95410 on epoch:    50, epoch:   50/50, mu: 0.9717088938, learning_rate: 0.0000778589, weight_decay: 0.0000098056

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 2425.3376464844s, input_fired_time: 18.0552654266, output_fired_time: 0.0000000000, propagate_forward_time: 2064.7038574219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 342.5786132813
best training[          , genome  5605] predictions:   36478/  50000 (72.96%), best:    6336/10000 (63.36%), error:     41169.80469, best error:     10693.95410 on epoch:    50, epoch:   51/50, mu: 0.9728808403, learning_rate: 0.0000714698, weight_decay: 0.0000091196
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 426.4200134277s, input_fired_time: 5.7715072632, output_fired_time: 0.0000000000, propagate_forward_time: 401.0694580078, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.5790405273
   testing[          , genome  5605] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10693.95410, best error:     10693.95410 on epoch:    50, epoch:   51/50, mu: 0.9728808403, learning_rate: 0.0000714698, weight_decay: 0.0000091196
backpropagation finished successfully!
08:14:27 (9224): called boinc_finish(0)

</stderr_txt>
]]>