Nameexact_genome_1529063102_163_6778_2
Workunit3816711
Created26 Jun 2018, 16:24:29 UTC
Sent26 Jun 2018, 16:44:07 UTC
Report deadline23 Oct 2018, 1:42:06 UTC
Received29 Jun 2018, 6:50:10 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64061
Run time2 days 7 hours 16 min 37 sec
CPU time2 days 6 hours 51 min 33 sec
Validate stateInitial
Credit0.00
Device peak FLOPS2.67 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size544.93 MB
Peak swap size546.30 MB
Peak disk usage7.78 MB

Stderr output

<core_client_version>7.8.4</core_client_version>
<![CDATA[
<stderr_txt>
rd_time: 156.6503295898, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9335937500
validation[          , genome  6778] predictions:    6315/  10000 (63.15%), best:    6348/10000 (63.48%), error:     10630.05078, best error:     10609.54492 on epoch:    63, epoch:   67/100, mu: 0.9766745567, learning_rate: 0.0000034115, weight_decay: 0.0000077229

epoch time: 1776.7159423828s, input_fired_time: 25.1750335693, output_fired_time: 7.6062183380, propagate_forward_time: 783.9660644531, propagate_backward_time: 937.1172485352, weight_update_time: 0.5858319998, other_time: 22.8513793945
epoch time: 163.0323333740s, input_fired_time: 1.2485898733, output_fired_time: 0.0000000000, propagate_forward_time: 157.5966186523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1871185303
validation[          , genome  6778] predictions:    6361/  10000 (63.61%), best:    6361/10000 (63.61%), error:     10586.73828, best error:     10586.73828 on epoch:    68, epoch:   68/100, mu: 0.9773727059, learning_rate: 0.0000031885, weight_decay: 0.0000072686

epoch time: 1781.3190917969s, input_fired_time: 25.0533523560, output_fired_time: 7.5841856003, propagate_forward_time: 788.6465454102, propagate_backward_time: 935.8716430664, weight_update_time: 0.5820747614, other_time: 24.1633300781
epoch time: 163.3214569092s, input_fired_time: 1.2444225550, output_fired_time: 0.0000000000, propagate_forward_time: 157.8195953369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2574462891
validation[          , genome  6778] predictions:    6315/  10000 (63.15%), best:    6361/10000 (63.61%), error:     10661.08105, best error:     10586.73828 on epoch:    68, epoch:   69/100, mu: 0.9780342579, learning_rate: 0.0000029800, weight_decay: 0.0000068411

epoch time: 1778.4079589844s, input_fired_time: 24.9667873383, output_fired_time: 7.5859813690, propagate_forward_time: 787.3166503906, propagate_backward_time: 934.4350585938, weight_update_time: 0.5815711617, other_time: 24.1035156250
epoch time: 163.2241210938s, input_fired_time: 1.2467322350, output_fired_time: 0.0000000000, propagate_forward_time: 157.7443542480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2330322266
validation[          , genome  6778] predictions:    6397/  10000 (63.97%), best:    6397/10000 (63.97%), error:     10567.85156, best error:     10567.85156 on epoch:    70, epoch:   70/100, mu: 0.9786611795, learning_rate: 0.0000027852, weight_decay: 0.0000064388

epoch time: 1782.8699951172s, input_fired_time: 25.0545310974, output_fired_time: 7.6021590233, propagate_forward_time: 789.1254272461, propagate_backward_time: 936.8533325195, weight_update_time: 0.5818001032, other_time: 24.2344970703
epoch time: 164.2815551758s, input_fired_time: 1.2567216158, output_fired_time: 0.0000000000, propagate_forward_time: 158.7779235840, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2469024658
validation[          , genome  6778] predictions:    6374/  10000 (63.74%), best:    6397/10000 (63.97%), error:     10595.80469, best error:     10567.85156 on epoch:    70, epoch:   71/100, mu: 0.9792552590, learning_rate: 0.0000026032, weight_decay: 0.0000060601

epoch time: 1788.2106933594s, input_fired_time: 25.2004699707, output_fired_time: 7.6338644028, propagate_forward_time: 791.3894653320, propagate_backward_time: 939.7112426758, weight_update_time: 0.5827439427, other_time: 24.2756347656
epoch time: 164.2276000977s, input_fired_time: 1.2528672218, output_fired_time: 0.0000000000, propagate_forward_time: 158.7139434814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2607879639
validation[          , genome  6778] predictions:    6364/  10000 (63.64%), best:    6397/10000 (63.97%), error:     10578.12305, best error:     10567.85156 on epoch:    70, epoch:   72/100, mu: 0.9798182249, learning_rate: 0.0000024330, weight_decay: 0.0000057036

epoch time: 1785.6014404297s, input_fired_time: 25.1409454346, output_fired_time: 7.6321601868, propagate_forward_time: 790.1492309570, propagate_backward_time: 938.4799804688, weight_update_time: 0.5818330646, other_time: 24.1990356445
epoch time: 163.2366943359s, input_fired_time: 1.2451913357, output_fired_time: 0.0000000000, propagate_forward_time: 157.7925567627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1989440918
validation[          , genome  6778] predictions:    6352/  10000 (63.52%), best:    6397/10000 (63.97%), error:     10596.89746, best error:     10567.85156 on epoch:    70, epoch:   73/100, mu: 0.9803516865, learning_rate: 0.0000022739, weight_decay: 0.0000053682

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.980857
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 2.1253e-06
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 5.05244e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 74
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 70
read number_validation_images: 10000
read best_validation_predictions: 6397
read best_validation_error: 10567.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1462308343'
read generator: 1462308343
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1799.40295410156s, input_fired_time: 25.6030101776123, output_fired_time: 8.18210983276367, propagate_forward_time: 794.808471679688, propagate_backward_time: 945.5107421875, weight_update_time: 0.612111806869507, other_time: 25.298583984375
epoch time: 165.483413696289s, input_fired_time: 1.38343107700348, output_fired_time: 0, propagate_forward_time: 159.525772094727, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.57420349121094
validation[          , genome  6778] predictions:    6401/  10000 (64.01%), best:    6401/10000 (64.01%), error:     10560.64844, best error:     10560.64844 on epoch:    74, epoch:   74/100, mu: 0.9808571935, learning_rate: 0.0000021253, weight_decay: 0.0000050524

epoch time: 1805.9887695312s, input_fired_time: 25.8261356354, output_fired_time: 8.1231355667, propagate_forward_time: 798.9139404297, propagate_backward_time: 947.8605346680, weight_update_time: 0.6184422970, other_time: 25.2649536133
epoch time: 162.6297912598s, input_fired_time: 1.2881952524, output_fired_time: 0.0000000000, propagate_forward_time: 157.3041381836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0374603271
validation[          , genome  6778] predictions:    6329/  10000 (63.29%), best:    6401/10000 (64.01%), error:     10599.04883, best error:     10560.64844 on epoch:    74, epoch:   75/100, mu: 0.9813362360, learning_rate: 0.0000019864, weight_decay: 0.0000047553

epoch time: 1784.0295410156s, input_fired_time: 25.1934051514, output_fired_time: 7.7109127045, propagate_forward_time: 788.3660888672, propagate_backward_time: 939.4909057617, weight_update_time: 0.5954528451, other_time: 23.2682495117
epoch time: 163.4016571045s, input_fired_time: 1.2752478123, output_fired_time: 0.0000000000, propagate_forward_time: 158.0820007324, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0444030762
validation[          , genome  6778] predictions:    6357/  10000 (63.57%), best:    6401/10000 (64.01%), error:     10595.37402, best error:     10560.64844 on epoch:    74, epoch:   76/100, mu: 0.9817901850, learning_rate: 0.0000018565, weight_decay: 0.0000044756

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.98222
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 1.73516e-06
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 4.21236e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 77
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 74
read number_validation_images: 10000
read best_validation_predictions: 6401
read best_validation_error: 10560.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '675368248'
read generator: 675368248
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1826.79650878906s, input_fired_time: 25.8239669799805, output_fired_time: 7.77266931533813, propagate_forward_time: 806.209777832031, propagate_backward_time: 963.765625, weight_update_time: 0.599049210548401, other_time: 23.2244262695312
epoch time: 161.83659362793s, input_fired_time: 1.26259124279022, output_fired_time: 0, propagate_forward_time: 156.701766967773, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.87223815917969
validation[          , genome  6778] predictions:    6393/  10000 (63.93%), best:    6393/10000 (63.93%), error:     10524.15430, best error:     10524.15430 on epoch:    77, epoch:   77/100, mu: 0.9822202921, learning_rate: 0.0000017352, weight_decay: 0.0000042124

epoch time: 1771.9896240234s, input_fired_time: 25.0307712555, output_fired_time: 7.5426344872, propagate_forward_time: 782.8835449219, propagate_backward_time: 934.1701049805, weight_update_time: 0.5861614943, other_time: 22.3626098633
epoch time: 162.4613189697s, input_fired_time: 1.2622646093, output_fired_time: 0.0000000000, propagate_forward_time: 157.2333984375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9656524658
validation[          , genome  6778] predictions:    6395/  10000 (63.95%), best:    6393/10000 (63.93%), error:     10548.99219, best error:     10524.15430 on epoch:    77, epoch:   78/100, mu: 0.9826278687, learning_rate: 0.0000016217, weight_decay: 0.0000039646

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.983014
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 1.51572e-06
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 3.73143e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 79
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 77
read number_validation_images: 10000
read best_validation_predictions: 6393
read best_validation_error: 10524.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1994171472'
read generator: 1994171472
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1794.7275390625s, input_fired_time: 25.5009670257568, output_fired_time: 8.02274608612061, propagate_forward_time: 792.845458984375, propagate_backward_time: 943.508605957031, weight_update_time: 0.606811702251434, other_time: 24.8497924804688
epoch time: 163.99333190918s, input_fired_time: 1.36338210105896, output_fired_time: 0, propagate_forward_time: 158.174575805664, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.45536804199219
validation[          , genome  6778] predictions:    6356/  10000 (63.56%), best:    6356/10000 (63.56%), error:     10509.37305, best error:     10509.37305 on epoch:    79, epoch:   79/100, mu: 0.9830141068, learning_rate: 0.0000015157, weight_decay: 0.0000037314

epoch time: 1795.5384521484s, input_fired_time: 25.5590362549, output_fired_time: 8.1122446060, propagate_forward_time: 793.0306396484, propagate_backward_time: 943.4829711914, weight_update_time: 0.6234470010, other_time: 25.3534545898
epoch time: 164.7907104492s, input_fired_time: 1.3610913754, output_fired_time: 0.0000000000, propagate_forward_time: 159.0423889160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3872375488
validation[          , genome  6778] predictions:    6411/  10000 (64.11%), best:    6411/10000 (64.11%), error:     10506.85059, best error:     10506.85059 on epoch:    80, epoch:   80/100, mu: 0.9833801389, learning_rate: 0.0000014166, weight_decay: 0.0000035120

epoch time: 1776.9301757812s, input_fired_time: 25.1352653503, output_fired_time: 7.6756381989, propagate_forward_time: 785.1044921875, propagate_backward_time: 935.5650024414, weight_update_time: 0.5907822847, other_time: 23.4497680664
epoch time: 162.6939544678s, input_fired_time: 1.2646781206, output_fired_time: 0.0000000000, propagate_forward_time: 157.3966522217, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0326232910
validation[          , genome  6778] predictions:    6423/  10000 (64.23%), best:    6411/10000 (64.11%), error:     10508.42480, best error:     10506.85059 on epoch:    80, epoch:   81/100, mu: 0.9837269783, learning_rate: 0.0000013240, weight_decay: 0.0000033054

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.984056
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 1.23748e-06
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 3.111e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 82
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 80
read number_validation_images: 10000
read best_validation_predictions: 6411
read best_validation_error: 10506.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1984239516'
read generator: 1984239516
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1770.28332519531s, input_fired_time: 25.1110324859619, output_fired_time: 7.51675605773926, propagate_forward_time: 781.539794921875, propagate_backward_time: 933.703674316406, weight_update_time: 0.588329553604126, other_time: 22.4120483398438
epoch time: 162.143188476562s, input_fired_time: 1.25304186344147, output_fired_time: 0, propagate_forward_time: 157.024154663086, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.86599731445312
validation[          , genome  6778] predictions:    6396/  10000 (63.96%), best:    6396/10000 (63.96%), error:     10494.85547, best error:     10494.85547 on epoch:    82, epoch:   82/100, mu: 0.9840556383, learning_rate: 0.0000012375, weight_decay: 0.0000031110

epoch time: 1773.6392822266s, input_fired_time: 25.0458240509, output_fired_time: 7.5570979118, propagate_forward_time: 783.1940917969, propagate_backward_time: 935.1262207031, weight_update_time: 0.5866667628, other_time: 22.7160644531
epoch time: 164.0792541504s, input_fired_time: 1.3696409464, output_fired_time: 0.0000000000, propagate_forward_time: 158.2754821777, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4341278076
validation[          , genome  6778] predictions:    6400/  10000 (64.00%), best:    6400/10000 (64.00%), error:     10485.32031, best error:     10485.32031 on epoch:    83, epoch:   83/100, mu: 0.9843670726, learning_rate: 0.0000011566, weight_decay: 0.0000029280

epoch time: 1793.5686035156s, input_fired_time: 25.5129528046, output_fired_time: 8.0850753784, propagate_forward_time: 792.1378784180, propagate_backward_time: 942.5675659180, weight_update_time: 0.6136223078, other_time: 25.2651367188
epoch time: 164.4037170410s, input_fired_time: 1.3707889318, output_fired_time: 0.0000000000, propagate_forward_time: 158.6440429688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3888854980
validation[          , genome  6778] predictions:    6400/  10000 (64.00%), best:    6400/10000 (64.00%), error:     10498.53418, best error:     10485.32031 on epoch:    83, epoch:   84/100, mu: 0.9846621752, learning_rate: 0.0000010810, weight_decay: 0.0000027558

epoch time: 1791.9094238281s, input_fired_time: 25.4895744324, output_fired_time: 8.0493392944, propagate_forward_time: 791.7636718750, propagate_backward_time: 941.6913452148, weight_update_time: 0.6122200489, other_time: 24.9154663086
epoch time: 163.3737335205s, input_fired_time: 1.2718738317, output_fired_time: 0.0000000000, propagate_forward_time: 158.0917510986, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0101013184
validation[          , genome  6778] predictions:    6384/  10000 (63.84%), best:    6400/10000 (64.00%), error:     10570.65820, best error:     10485.32031 on epoch:    83, epoch:   85/100, mu: 0.9849418402, learning_rate: 0.0000010103, weight_decay: 0.0000025937

epoch time: 1783.7045898438s, input_fired_time: 25.1516799927, output_fired_time: 7.6750540733, propagate_forward_time: 788.3535766602, propagate_backward_time: 939.2934570312, weight_update_time: 0.5913484097, other_time: 23.2307739258
epoch time: 162.8415832520s, input_fired_time: 1.2657076120, output_fired_time: 0.0000000000, propagate_forward_time: 157.5883941650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9874877930
validation[          , genome  6778] predictions:    6401/  10000 (64.01%), best:    6401/10000 (64.01%), error:     10483.15820, best error:     10483.15820 on epoch:    86, epoch:   86/100, mu: 0.9852068424, learning_rate: 0.0000009443, weight_decay: 0.0000024412

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.985458
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 8.82544e-07
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 2.29759e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 87
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 6401
read best_validation_error: 10483.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '432632681'
read generator: 432632681
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1779.90905761719s, input_fired_time: 25.1782684326172, output_fired_time: 7.66136169433594, propagate_forward_time: 787.272338867188, propagate_backward_time: 935.976501464844, weight_update_time: 0.586607158184052, other_time: 23.8206176757812
epoch time: 162.752136230469s, input_fired_time: 1.26293730735779, output_fired_time: 0, propagate_forward_time: 157.365020751953, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.12417602539062
validation[          , genome  6778] predictions:    6398/  10000 (63.98%), best:    6401/10000 (64.01%), error:     10485.70996, best error:     10483.15820 on epoch:    86, epoch:   87/100, mu: 0.9854579568, learning_rate: 0.0000008825, weight_decay: 0.0000022976

epoch time: 1779.2810058594s, input_fired_time: 25.0679035187, output_fired_time: 7.5601735115, propagate_forward_time: 786.9976196289, propagate_backward_time: 935.8905639648, weight_update_time: 0.5834738016, other_time: 23.7647705078
epoch time: 163.5946044922s, input_fired_time: 1.2824473381, output_fired_time: 0.0000000000, propagate_forward_time: 158.1529846191, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1591796875
validation[          , genome  6778] predictions:    6400/  10000 (64.00%), best:    6401/10000 (64.01%), error:     10489.23242, best error:     10483.15820 on epoch:    86, epoch:   88/100, mu: 0.9856958985, learning_rate: 0.0000008249, weight_decay: 0.0000021625

epoch time: 1786.7497558594s, input_fired_time: 25.2013263702, output_fired_time: 7.5912981033, propagate_forward_time: 790.4418945312, propagate_backward_time: 939.7116699219, weight_update_time: 0.5852283239, other_time: 23.8035888672
epoch time: 163.3571929932s, input_fired_time: 1.2829269171, output_fired_time: 0.0000000000, propagate_forward_time: 157.9165344238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1577301025
validation[          , genome  6778] predictions:    6432/  10000 (64.32%), best:    6401/10000 (64.01%), error:     10490.26953, best error:     10483.15820 on epoch:    86, epoch:   89/100, mu: 0.9859213829, learning_rate: 0.0000007709, weight_decay: 0.0000020353

epoch time: 1785.9323730469s, input_fired_time: 25.1350154877, output_fired_time: 7.6122350693, propagate_forward_time: 789.8412475586, propagate_backward_time: 939.5715942383, weight_update_time: 0.5823959708, other_time: 23.7723388672
epoch time: 163.8006744385s, input_fired_time: 1.2884962559, output_fired_time: 0.0000000000, propagate_forward_time: 158.3475494385, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1646270752
validation[          , genome  6778] predictions:    6393/  10000 (63.93%), best:    6401/10000 (64.01%), error:     10510.31836, best error:     10483.15820 on epoch:    86, epoch:   90/100, mu: 0.9861350656, learning_rate: 0.0000007205, weight_decay: 0.0000019156

epoch time: 1782.3006591797s, input_fired_time: 25.1486206055, output_fired_time: 7.6220302582, propagate_forward_time: 788.2984008789, propagate_backward_time: 937.4581298828, weight_update_time: 0.5854558945, other_time: 23.7734985352
epoch time: 162.9999694824s, input_fired_time: 1.2641602755, output_fired_time: 0.0000000000, propagate_forward_time: 157.5976104736, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1381988525
validation[          , genome  6778] predictions:    6421/  10000 (64.21%), best:    6421/10000 (64.21%), error:     10474.66016, best error:     10474.66016 on epoch:    91, epoch:   91/100, mu: 0.9863375425, learning_rate: 0.0000006734, weight_decay: 0.0000018029

epoch time: 1780.0976562500s, input_fired_time: 25.1460704803, output_fired_time: 7.6606688499, propagate_forward_time: 787.1445922852, propagate_backward_time: 936.3081665039, weight_update_time: 0.5836475492, other_time: 23.8381347656
epoch time: 162.7919006348s, input_fired_time: 1.2653902769, output_fired_time: 0.0000000000, propagate_forward_time: 157.3943023682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1322021484
validation[          , genome  6778] predictions:    6425/  10000 (64.25%), best:    6425/10000 (64.25%), error:     10460.27246, best error:     10460.27246 on epoch:    92, epoch:   92/100, mu: 0.9865294099, learning_rate: 0.0000006294, weight_decay: 0.0000016969

epoch time: 1782.2822265625s, input_fired_time: 25.0882263184, output_fired_time: 7.6542606354, propagate_forward_time: 788.2070922852, propagate_backward_time: 937.5317382812, weight_update_time: 0.5832406878, other_time: 23.8008422852
epoch time: 162.7271728516s, input_fired_time: 1.2673727274, output_fired_time: 0.0000000000, propagate_forward_time: 157.3146820068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1451110840
validation[          , genome  6778] predictions:    6418/  10000 (64.18%), best:    6425/10000 (64.25%), error:     10464.07031, best error:     10460.27246 on epoch:    92, epoch:   93/100, mu: 0.9867112637, learning_rate: 0.0000005883, weight_decay: 0.0000015971

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.986884
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 5.49812e-07
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 1.50313e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 94
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 92
read number_validation_images: 10000
read best_validation_predictions: 6425
read best_validation_error: 10460.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1566862012'
read generator: 1566862012
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1804.13024902344s, input_fired_time: 25.6998157501221, output_fired_time: 8.22418594360352, propagate_forward_time: 796.541564941406, propagate_backward_time: 948.279907226562, weight_update_time: 0.613552391529083, other_time: 25.3847045898438
epoch time: 165.198684692383s, input_fired_time: 1.37243318557739, output_fired_time: 0, propagate_forward_time: 159.429214477539, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.39703369140625
validation[          , genome  6778] predictions:    6390/  10000 (63.90%), best:    6425/10000 (64.25%), error:     10525.58887, best error:     10460.27246 on epoch:    92, epoch:   94/100, mu: 0.9868835807, learning_rate: 0.0000005498, weight_decay: 0.0000015031

epoch time: 1793.4444580078s, input_fired_time: 25.5102958679, output_fired_time: 8.0525789261, propagate_forward_time: 792.4096679688, propagate_backward_time: 942.5534667969, weight_update_time: 0.6044489741, other_time: 24.9184570312
epoch time: 162.5330810547s, input_fired_time: 1.2769029140, output_fired_time: 0.0000000000, propagate_forward_time: 157.2348480225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0213317871
validation[          , genome  6778] predictions:    6408/  10000 (64.08%), best:    6425/10000 (64.25%), error:     10483.79004, best error:     10460.27246 on epoch:    92, epoch:   95/100, mu: 0.9870468378, learning_rate: 0.0000005139, weight_decay: 0.0000014147

epoch time: 1779.4492187500s, input_fired_time: 25.1997756958, output_fired_time: 7.6318478584, propagate_forward_time: 786.1994018555, propagate_backward_time: 937.3277587891, weight_update_time: 0.5849455595, other_time: 23.0903930664
epoch time: 162.1846923828s, input_fired_time: 1.2701315880, output_fired_time: 0.0000000000, propagate_forward_time: 156.9246215820, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9899444580
validation[          , genome  6778] predictions:    6411/  10000 (64.11%), best:    6425/10000 (64.25%), error:     10472.62988, best error:     10460.27246 on epoch:    92, epoch:   96/100, mu: 0.9872015715, learning_rate: 0.0000004803, weight_decay: 0.0000013315

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.987348
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 4.48884e-07
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 1.2532e-06
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 97
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 92
read number_validation_images: 10000
read best_validation_predictions: 6425
read best_validation_error: 10460.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1569589535'
read generator: 1569589535
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1787.80590820312s, input_fired_time: 25.2050189971924, output_fired_time: 7.68880653381348, propagate_forward_time: 791.125244140625, propagate_backward_time: 939.876220703125, weight_update_time: 0.592474758625031, other_time: 23.9105224609375
epoch time: 163.890838623047s, input_fired_time: 1.28047096729279, output_fired_time: 0, propagate_forward_time: 158.445922851562, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.16444396972656
validation[          , genome  6778] predictions:    6453/  10000 (64.53%), best:    6425/10000 (64.25%), error:     10467.66113, best error:     10460.27246 on epoch:    92, epoch:   97/100, mu: 0.9873481989, learning_rate: 0.0000004489, weight_decay: 0.0000012532

epoch time: 1784.6416015625s, input_fired_time: 25.1484088898, output_fired_time: 7.6127877235, propagate_forward_time: 789.5558471680, propagate_backward_time: 938.4677124023, weight_update_time: 0.5866886377, other_time: 23.8568115234
epoch time: 164.0229644775s, input_fired_time: 1.2736049891, output_fired_time: 0.0000000000, propagate_forward_time: 158.5881500244, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1612091064
validation[          , genome  6778] predictions:    6419/  10000 (64.19%), best:    6419/10000 (64.19%), error:     10444.57324, best error:     10444.57324 on epoch:    98, epoch:   98/100, mu: 0.9874871373, learning_rate: 0.0000004195, weight_decay: 0.0000011795

epoch time: 1779.5784912109s, input_fired_time: 25.0780181885, output_fired_time: 7.5969605446, propagate_forward_time: 787.1226196289, propagate_backward_time: 935.9534301758, weight_update_time: 0.5841397643, other_time: 23.8275146484
epoch time: 162.9267883301s, input_fired_time: 1.2671128511, output_fired_time: 0.0000000000, propagate_forward_time: 157.5063171387, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1533508301
validation[          , genome  6778] predictions:    6434/  10000 (64.34%), best:    6419/10000 (64.19%), error:     10449.73340, best error:     10444.57324 on epoch:    98, epoch:   99/100, mu: 0.9876188040, learning_rate: 0.0000003921, weight_decay: 0.0000011101

epoch time: 1786.4674072266s, input_fired_time: 25.2024593353, output_fired_time: 7.5976958275, propagate_forward_time: 790.3779296875, propagate_backward_time: 939.4071655273, weight_update_time: 0.5849955678, other_time: 23.8821411133
epoch time: 163.7933197021s, input_fired_time: 1.2750049829, output_fired_time: 0.0000000000, propagate_forward_time: 158.3408660889, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1774444580
validation[          , genome  6778] predictions:    6424/  10000 (64.24%), best:    6419/10000 (64.19%), error:     10464.46289, best error:     10444.57324 on epoch:    98, epoch:  100/100, mu: 0.9877435565, learning_rate: 0.0000003665, weight_decay: 0.0000010448

evaluating best weights on full training data.
evaluting training set with running mean/variance:
arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1529063102_163_6778_2_r677339110_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.499562
read mu: 0.987862
read mu_delta: 0.947607
read initial_learning_rate: 0.000316286
read learning_rate: 3.42525e-07
read learning_rate_delta: 0.93463
read initial_weight_decay: 0.000448302
read weight_decay: 9.83376e-07
read weight_decay_delta: 0.941184
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.10278
read input_dropout_probability: 0.00120508
read hidden_dropout_probability: 0.0971343
read velocity_reset: 1502
read epoch: 101
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 98
read number_validation_images: 10000
read best_validation_predictions: 6419
read best_validation_error: 10444.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6778
read normal distribution: '1 0 0'
generator_str: '1507759514'
read generator: 1507759514
read generated_by_map:
1 disable_edge 1
reading 89 nodes.
reading 344 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1767.32434082031s, input_fired_time: 24.9547882080078, output_fired_time: 7.48751068115234, propagate_forward_time: 780.006164550781, propagate_backward_time: 932.419921875, weight_update_time: 0.580260396003723, other_time: 22.4558715820312
epoch time: 161.947402954102s, input_fired_time: 1.26805651187897, output_fired_time: 0, propagate_forward_time: 156.803024291992, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.87632751464844
validation[          , genome  6778] predictions:    6423/  10000 (64.23%), best:    6419/10000 (64.19%), error:     10447.91602, best error:     10444.57324 on epoch:    98, epoch:  101/100, mu: 0.9878617525, learning_rate: 0.0000003425, weight_decay: 0.0000009834

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 813.4683837891s, input_fired_time: 6.4021887779, output_fired_time: 0.0000000000, propagate_forward_time: 786.4108276367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 20.6553955078
best training[          , genome  6778] predictions:   34095/  50000 (68.19%), best:    6419/10000 (64.19%), error:     47947.92969, best error:     10444.57324 on epoch:    98, epoch:  102/100, mu: 0.9879738092, learning_rate: 0.0000003201, weight_decay: 0.0000009255
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 163.1288452148s, input_fired_time: 1.2689384222, output_fired_time: 0.0000000000, propagate_forward_time: 157.7032012939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1567077637
   testing[          , genome  6778] predictions:    6419/  10000 (64.19%), best:    6419/10000 (64.19%), error:     10444.57324, best error:     10444.57324 on epoch:    98, epoch:  102/100, mu: 0.9879738092, learning_rate: 0.0000003201, weight_decay: 0.0000009255
backpropagation finished successfully!
02:49:46 (18336): called boinc_finish(0)

</stderr_txt>
]]>