Nameexact_genome_1542031203_170_5592_0
Workunit4081751
Created12 Nov 2018, 19:28:01 UTC
Sent12 Nov 2018, 19:33:57 UTC
Report deadline9 Dec 2018, 2:58:54 UTC
Received23 Nov 2018, 1:41:19 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID23692
Run time11 hours 15 min 56 sec
CPU time10 hours 59 min 45 sec
Validate stateValid
Credit6,577.25
Device peak FLOPS4.15 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size573.21 MB
Peak swap size593.04 MB
Peak disk usage11.09 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592_0_r1726017654_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.461515
read mu: 0.461515
read mu_delta: 0.942066
read initial_learning_rate: 0.000944818
read learning_rate: 0.000944818
read learning_rate_delta: 0.908391
read initial_weight_decay: 0.000221004
read weight_decay: 0.000221004
read weight_decay_delta: 0.928208
read batch_size: 106
read epsilon: 1e-07
read alpha: 0.0564001
read input_dropout_probability: 0.00151447
read hidden_dropout_probability: 0.0830253
read velocity_reset: 1478
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5592
read normal distribution: '1 0 0'
generator_str: '1021643290'
read generator: 1021643290
read generated_by_map:
1 crossover 1
reading 109 nodes.
reading 392 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1627017265
post shuffle 1: 480158743
epoch time: 1446.03186035156s, input_fired_time: 22.3485488891602, output_fired_time: 11.2819528579712, propagate_forward_time: 554.246276855469, propagate_backward_time: 843.130737304688, weight_update_time: 0.413925141096115, other_time: 15.0243530273438
epoch time: 140.295776367188s, input_fired_time: 2.1772575378418, output_fired_time: 0, propagate_forward_time: 134.955032348633, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.16348266601563
validation[          , genome  5592] predictions:    5885/  10000 (58.85%), best:    5885/10000 (58.85%), error:     11752.34277, best error:     11752.34277 on epoch:     0, epoch:    0/25, mu: 0.4615150690, learning_rate: 0.0009448181, weight_decay: 0.0002210039

epoch time: 1436.1824951172s, input_fired_time: 21.9976348877, output_fired_time: 11.2758769989, propagate_forward_time: 552.4075927734, propagate_backward_time: 835.3742065430, weight_update_time: 0.3621382713, other_time: 15.1271362305
epoch time: 132.5983581543s, input_fired_time: 1.9835443497, output_fired_time: 0.0000000000, propagate_forward_time: 127.6696701050, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9451370239
validation[          , genome  5592] predictions:    6147/  10000 (61.47%), best:    6147/10000 (61.47%), error:     11145.26758, best error:     11145.26758 on epoch:     1, epoch:    1/25, mu: 0.4921321571, learning_rate: 0.0008582646, weight_decay: 0.0002051377

epoch time: 1388.0036621094s, input_fired_time: 21.3429317474, output_fired_time: 10.9037790298, propagate_forward_time: 533.7658081055, propagate_backward_time: 807.5742797852, weight_update_time: 0.3577325940, other_time: 14.4168701172
epoch time: 110.1495056152s, input_fired_time: 1.6733452082, output_fired_time: 0.0000000000, propagate_forward_time: 105.9189300537, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5572280884
validation[          , genome  5592] predictions:    6162/  10000 (61.62%), best:    6162/10000 (61.62%), error:     11034.75098, best error:     11034.75098 on epoch:     2, epoch:    2/25, mu: 0.5209754705, learning_rate: 0.0007796402, weight_decay: 0.0001904105

epoch time: 1423.8288574219s, input_fired_time: 21.6860065460, output_fired_time: 10.9337196350, propagate_forward_time: 548.5988159180, propagate_backward_time: 827.7313232422, weight_update_time: 0.3546430171, other_time: 14.8789672852
epoch time: 115.0651321411s, input_fired_time: 1.7911615372, output_fired_time: 0.0000000000, propagate_forward_time: 110.6119918823, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6619796753
validation[          , genome  5592] predictions:    6095/  10000 (60.95%), best:    6162/10000 (61.62%), error:     11217.89648, best error:     11034.75098 on epoch:     2, epoch:    3/25, mu: 0.5481477976, learning_rate: 0.0007082185, weight_decay: 0.0001767406

epoch time: 1395.7401123047s, input_fired_time: 21.4877700806, output_fired_time: 10.6668081284, propagate_forward_time: 536.7577514648, propagate_backward_time: 812.0954589844, weight_update_time: 0.3524557650, other_time: 14.7323608398
epoch time: 117.5117263794s, input_fired_time: 1.7891308069, output_fired_time: 0.0000000000, propagate_forward_time: 113.0199966431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7025985718
validation[          , genome  5592] predictions:    6113/  10000 (61.13%), best:    6162/10000 (61.62%), error:     11088.78906, best error:     11034.75098 on epoch:     2, epoch:    4/25, mu: 0.5737459064, learning_rate: 0.0006433396, weight_decay: 0.0001640522

epoch time: 1386.3586425781s, input_fired_time: 21.2914085388, output_fired_time: 10.7206001282, propagate_forward_time: 533.4326171875, propagate_backward_time: 806.4413452148, weight_update_time: 0.3553611636, other_time: 14.4727172852
epoch time: 110.2739410400s, input_fired_time: 1.6529290676, output_fired_time: 0.0000000000, propagate_forward_time: 106.0809173584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5400924683
validation[          , genome  5592] predictions:    5992/  10000 (59.92%), best:    6162/10000 (61.62%), error:     11419.72461, best error:     11034.75098 on epoch:     2, epoch:    5/25, mu: 0.5978610516, learning_rate: 0.0005844042, weight_decay: 0.0001522746

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592_0_r1726017654_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.461515
read mu: 0.620579
read mu_delta: 0.942066
read initial_learning_rate: 0.000944818
read learning_rate: 0.000530868
read learning_rate_delta: 0.908391
read initial_weight_decay: 0.000221004
read weight_decay: 0.000141343
read weight_decay_delta: 0.928208
read batch_size: 106
read epsilon: 1e-07
read alpha: 0.0564001
read input_dropout_probability: 0.00151447
read hidden_dropout_probability: 0.0830253
read velocity_reset: 1478
read epoch: 6
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 2
read number_validation_images: 10000
read best_validation_predictions: 6162
read best_validation_error: 11034.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5592
read normal distribution: '1 0 0'
generator_str: '1556660537'
read generator: 1556660537
read generated_by_map:
1 crossover 1
reading 109 nodes.
reading 392 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1409.89001464844s, input_fired_time: 20.8869304656982, output_fired_time: 10.7527198791504, propagate_forward_time: 542.583251953125, propagate_backward_time: 821.565795898438, weight_update_time: 0.355739206075668, other_time: 14.101318359375
epoch time: 112.512969970703s, input_fired_time: 1.66464924812317, output_fired_time: 0, propagate_forward_time: 108.375061035156, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.47325897216797
validation[          , genome  5592] predictions:    6159/  10000 (61.59%), best:    6162/10000 (61.62%), error:     11081.86328, best error:     11034.75098 on epoch:     2, epoch:    6/25, mu: 0.6205791235, learning_rate: 0.0005308677, weight_decay: 0.0001413425

epoch time: 1407.8105468750s, input_fired_time: 20.9453601837, output_fired_time: 10.8355274200, propagate_forward_time: 542.8612060547, propagate_backward_time: 819.1022949219, weight_update_time: 0.3559575975, other_time: 14.0661621094
epoch time: 114.5440979004s, input_fired_time: 1.6945283413, output_fired_time: 0.0000000000, propagate_forward_time: 110.3284912109, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5210800171
validation[          , genome  5592] predictions:    6076/  10000 (60.76%), best:    6162/10000 (61.62%), error:     11207.72363, best error:     11034.75098 on epoch:     2, epoch:    7/25, mu: 0.6419810653, learning_rate: 0.0004822357, weight_decay: 0.0001311953

epoch time: 1416.8358154297s, input_fired_time: 21.1869583130, output_fired_time: 11.0113945007, propagate_forward_time: 548.1803588867, propagate_backward_time: 822.2252197266, weight_update_time: 0.3658982813, other_time: 14.2318725586
epoch time: 114.1914672852s, input_fired_time: 1.6845238209, output_fired_time: 0.0000000000, propagate_forward_time: 110.0238418579, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4831008911
validation[          , genome  5592] predictions:    6165/  10000 (61.65%), best:    6165/10000 (61.65%), error:     11026.94336, best error:     11026.94336 on epoch:     8, epoch:    8/25, mu: 0.6621431112, learning_rate: 0.0004380587, weight_decay: 0.0001217766

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592_0_r1726017654_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.461515
read mu: 0.681137
read mu_delta: 0.942066
read initial_learning_rate: 0.000944818
read learning_rate: 0.000397929
read learning_rate_delta: 0.908391
read initial_weight_decay: 0.000221004
read weight_decay: 0.000113034
read weight_decay_delta: 0.928208
read batch_size: 106
read epsilon: 1e-07
read alpha: 0.0564001
read input_dropout_probability: 0.00151447
read hidden_dropout_probability: 0.0830253
read velocity_reset: 1478
read epoch: 9
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 8
read number_validation_images: 10000
read best_validation_predictions: 6165
read best_validation_error: 11026.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5592
read normal distribution: '1 0 0'
generator_str: '1680141435'
read generator: 1680141435
read generated_by_map:
1 crossover 1
reading 109 nodes.
reading 392 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1398.65258789063s, input_fired_time: 21.0671119689941, output_fired_time: 11.1197729110718, propagate_forward_time: 539.213439941406, propagate_backward_time: 812.120422363281, weight_update_time: 0.368045568466187, other_time: 15.1318359375
epoch time: 110.605522155762s, input_fired_time: 1.66800963878632, output_fired_time: 0, propagate_forward_time: 106.175926208496, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.76158905029297
validation[          , genome  5592] predictions:    6236/  10000 (62.36%), best:    6236/10000 (62.36%), error:     11008.05664, best error:     11008.05664 on epoch:     9, epoch:    9/25, mu: 0.6811370850, learning_rate: 0.0003979288, weight_decay: 0.0001130341

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_170_5592_0_r1726017654_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.461515
read mu: 0.699031
read mu_delta: 0.942066
read initial_learning_rate: 0.000944818
read learning_rate: 0.000361475
read learning_rate_delta: 0.908391
read initial_weight_decay: 0.000221004
read weight_decay: 0.000104919
read weight_decay_delta: 0.928208
read batch_size: 106
read epsilon: 1e-07
read alpha: 0.0564001
read input_dropout_probability: 0.00151447
read hidden_dropout_probability: 0.0830253
read velocity_reset: 1478
read epoch: 10
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 9
read number_validation_images: 10000
read best_validation_predictions: 6236
read best_validation_error: 11008.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5592
read normal distribution: '1 0 0'
generator_str: '1769100629'
read generator: 1769100629
read generated_by_map:
1 crossover 1
reading 109 nodes.
reading 392 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1409.82312011719s, input_fired_time: 20.8005142211914, output_fired_time: 10.5970220565796, propagate_forward_time: 541.587280273438, propagate_backward_time: 823.729370117188, weight_update_time: 0.3399278819561, other_time: 13.10888671875
epoch time: 113.259048461914s, input_fired_time: 1.65252411365509, output_fired_time: 0, propagate_forward_time: 109.287666320801, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.31885528564453
validation[          , genome  5592] predictions:    6060/  10000 (60.60%), best:    6236/10000 (62.36%), error:     11273.84863, best error:     11008.05664 on epoch:     9, epoch:   10/25, mu: 0.6990306377, learning_rate: 0.0003614751, weight_decay: 0.0001049192

epoch time: 1405.4986572266s, input_fired_time: 20.6412963867, output_fired_time: 10.4643678665, propagate_forward_time: 538.7503662109, propagate_backward_time: 822.5632324219, weight_update_time: 0.3403362036, other_time: 13.0794677734
epoch time: 111.5494155884s, input_fired_time: 1.6389266253, output_fired_time: 0.0000000000, propagate_forward_time: 107.6201934814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2902984619
validation[          , genome  5592] predictions:    6203/  10000 (62.03%), best:    6203/10000 (62.03%), error:     10953.60645, best error:     10953.60645 on epoch:    11, epoch:   11/25, mu: 0.7158875465, learning_rate: 0.0003283609, weight_decay: 0.0000973869

epoch time: 1448.6478271484s, input_fired_time: 21.3313522339, output_fired_time: 10.8713274002, propagate_forward_time: 554.3206176758, propagate_backward_time: 848.6319580078, weight_update_time: 0.3428523839, other_time: 13.4926147461
epoch time: 113.0354309082s, input_fired_time: 1.6825416088, output_fired_time: 0.0000000000, propagate_forward_time: 109.0231399536, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3297500610
validation[          , genome  5592] predictions:    6188/  10000 (61.88%), best:    6203/10000 (62.03%), error:     10987.51465, best error:     10953.60645 on epoch:    11, epoch:   12/25, mu: 0.7317678928, learning_rate: 0.0002982802, weight_decay: 0.0000903953

epoch time: 2771.7712402344s, input_fired_time: 20.8648796082, output_fired_time: 10.7057571411, propagate_forward_time: 1901.8912353516, propagate_backward_time: 824.6992187500, weight_update_time: 0.3504582047, other_time: 13.6099853516
epoch time: 113.0343856812s, input_fired_time: 1.6660972834, output_fired_time: 0.0000000000, propagate_forward_time: 108.9460601807, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4222259521
validation[          , genome  5592] predictions:    6235/  10000 (62.35%), best:    6235/10000 (62.35%), error:     10833.18359, best error:     10833.18359 on epoch:    13, epoch:   13/25, mu: 0.7467282414, learning_rate: 0.0002709552, weight_decay: 0.0000839057

epoch time: 1434.4824218750s, input_fired_time: 21.2468509674, output_fired_time: 10.8151245117, propagate_forward_time: 550.9393920898, propagate_backward_time: 837.4608764648, weight_update_time: 0.3635647595, other_time: 14.0201416016
epoch time: 119.6494979858s, input_fired_time: 1.7986887693, output_fired_time: 0.0000000000, propagate_forward_time: 115.2042541504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6465530396
validation[          , genome  5592] predictions:    6311/  10000 (63.11%), best:    6311/10000 (63.11%), error:     10653.95898, best error:     10653.95898 on epoch:    14, epoch:   14/25, mu: 0.7608218789, learning_rate: 0.0002461334, weight_decay: 0.0000778820

epoch time: 1409.7192382813s, input_fired_time: 20.9220638275, output_fired_time: 10.7170372009, propagate_forward_time: 543.1055297852, propagate_backward_time: 821.3062744141, weight_update_time: 0.3491770923, other_time: 13.6682739258
epoch time: 113.8174362183s, input_fired_time: 1.6764707565, output_fired_time: 0.0000000000, propagate_forward_time: 109.7224731445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4184951782
validation[          , genome  5592] predictions:    6178/  10000 (61.78%), best:    6311/10000 (63.11%), error:     10947.52539, best error:     10653.95898 on epoch:    14, epoch:   15/25, mu: 0.7740989923, learning_rate: 0.0002235855, weight_decay: 0.0000722907

epoch time: 1420.6512451172s, input_fired_time: 21.0972385406, output_fired_time: 10.8267850876, propagate_forward_time: 546.6602783203, propagate_backward_time: 828.2587280273, weight_update_time: 0.3565314710, other_time: 13.8081665039
epoch time: 114.0250015259s, input_fired_time: 1.6903275251, output_fired_time: 0.0000000000, propagate_forward_time: 109.9022827148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4323883057
validation[          , genome  5592] predictions:    6227/  10000 (62.27%), best:    6311/10000 (63.11%), error:     10861.09570, best error:     10653.95898 on epoch:    14, epoch:   16/25, mu: 0.7866069078, learning_rate: 0.0002031031, weight_decay: 0.0000671008

epoch time: 1419.4724121094s, input_fired_time: 21.0083560944, output_fired_time: 10.7242450714, propagate_forward_time: 545.1752929688, propagate_backward_time: 828.9334716797, weight_update_time: 0.3499235213, other_time: 13.6311035156
epoch time: 113.4693984985s, input_fired_time: 1.6748349667, output_fired_time: 0.0000000000, propagate_forward_time: 109.3851852417, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4093780518
validation[          , genome  5592] predictions:    6329/  10000 (63.29%), best:    6329/10000 (63.29%), error:     10582.03906, best error:     10582.03906 on epoch:    17, epoch:   17/25, mu: 0.7983902097, learning_rate: 0.0001844971, weight_decay: 0.0000622836

epoch time: 1425.5046386719s, input_fired_time: 21.0420036316, output_fired_time: 10.7950372696, propagate_forward_time: 547.7055664063, propagate_backward_time: 832.2048950195, weight_update_time: 0.3524938524, other_time: 13.7571411133
epoch time: 113.1583709717s, input_fired_time: 1.6807765961, output_fired_time: 0.0000000000, propagate_forward_time: 109.0936660767, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3839263916
validation[          , genome  5592] predictions:    6290/  10000 (62.90%), best:    6329/10000 (63.29%), error:     10672.75000, best error:     10582.03906 on epoch:    17, epoch:   18/25, mu: 0.8094908595, learning_rate: 0.0001675956, weight_decay: 0.0000578121

epoch time: 1413.6220703125s, input_fired_time: 20.8925361633, output_fired_time: 10.7544460297, propagate_forward_time: 543.9810791016, propagate_backward_time: 824.2169799805, weight_update_time: 0.3596905768, other_time: 13.7770385742
epoch time: 113.7637557983s, input_fired_time: 1.6846724749, output_fired_time: 0.0000000000, propagate_forward_time: 109.6416854858, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4374008179
validation[          , genome  5592] predictions:    6316/  10000 (63.16%), best:    6329/10000 (63.29%), error:     10599.29004, best error:     10582.03906 on epoch:    17, epoch:   19/25, mu: 0.8199484348, learning_rate: 0.0001522424, weight_decay: 0.0000536617

epoch time: 1419.8889160156s, input_fired_time: 21.0511531830, output_fired_time: 10.7148485184, propagate_forward_time: 546.2913208008, propagate_backward_time: 828.0160522461, weight_update_time: 0.3497891426, other_time: 13.8155517578
epoch time: 116.8508834839s, input_fired_time: 1.7326691151, output_fired_time: 0.0000000000, propagate_forward_time: 112.6418914795, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4763259888
validation[          , genome  5592] predictions:    6325/  10000 (63.25%), best:    6329/10000 (63.29%), error:     10595.97266, best error:     10582.03906 on epoch:    17, epoch:   20/25, mu: 0.8298001289, learning_rate: 0.0001382957, weight_decay: 0.0000498092

epoch time: 1415.9511718750s, input_fired_time: 20.8851470947, output_fired_time: 10.6917695999, propagate_forward_time: 543.5317993164, propagate_backward_time: 827.1071166992, weight_update_time: 0.3480485678, other_time: 13.7353515625
epoch time: 113.2009429932s, input_fired_time: 1.6811758280, output_fired_time: 0.0000000000, propagate_forward_time: 109.0941238403, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4256439209
validation[          , genome  5592] predictions:    6360/  10000 (63.60%), best:    6360/10000 (63.60%), error:     10524.76855, best error:     10524.76855 on epoch:    21, epoch:   21/25, mu: 0.8390811086, learning_rate: 0.0001256266, weight_decay: 0.0000462334

epoch time: 1400.5230712891s, input_fired_time: 20.8146228790, output_fired_time: 10.5301122665, propagate_forward_time: 539.1154785156, propagate_backward_time: 816.5514526367, weight_update_time: 0.3461335897, other_time: 13.5114135742
epoch time: 112.6416931152s, input_fired_time: 1.6766614914, output_fired_time: 0.0000000000, propagate_forward_time: 108.5133438110, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4516906738
validation[          , genome  5592] predictions:    6302/  10000 (63.02%), best:    6360/10000 (63.60%), error:     10580.57520, best error:     10524.76855 on epoch:    21, epoch:   22/25, mu: 0.8478243947, learning_rate: 0.0001141181, weight_decay: 0.0000429142

epoch time: 1402.1506347656s, input_fired_time: 20.7816238403, output_fired_time: 10.5799217224, propagate_forward_time: 538.6191406250, propagate_backward_time: 818.6631469727, weight_update_time: 0.3480064273, other_time: 13.5067749023
epoch time: 112.3893051147s, input_fired_time: 1.6785477400, output_fired_time: 0.0000000000, propagate_forward_time: 108.2868270874, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4239273071
validation[          , genome  5592] predictions:    6358/  10000 (63.58%), best:    6358/10000 (63.58%), error:     10522.83203, best error:     10522.83203 on epoch:    23, epoch:   23/25, mu: 0.8560611606, learning_rate: 0.0001036639, weight_decay: 0.0000398333

epoch time: 1395.1005859375s, input_fired_time: 20.7473754883, output_fired_time: 10.5700130463, propagate_forward_time: 537.6531372070, propagate_backward_time: 812.7623901367, weight_update_time: 0.3455107212, other_time: 13.3676757813
epoch time: 115.2853469849s, input_fired_time: 1.7118511200, output_fired_time: 0.0000000000, propagate_forward_time: 111.2040939331, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3694000244
validation[          , genome  5592] predictions:    6367/  10000 (63.67%), best:    6367/10000 (63.67%), error:     10465.76758, best error:     10465.76758 on epoch:    24, epoch:   24/25, mu: 0.8638207316, learning_rate: 0.0000941674, weight_decay: 0.0000369736

epoch time: 1401.1207275391s, input_fired_time: 20.6518669128, output_fired_time: 10.3881349564, propagate_forward_time: 537.2926635742, propagate_backward_time: 819.9542846680, weight_update_time: 0.3351311684, other_time: 12.8337402344
epoch time: 111.7727127075s, input_fired_time: 1.6511030197, output_fired_time: 0.0000000000, propagate_forward_time: 107.8434295654, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2781829834
validation[          , genome  5592] predictions:    6437/  10000 (64.37%), best:    6437/10000 (64.37%), error:     10373.00684, best error:     10373.00684 on epoch:    25, epoch:   25/25, mu: 0.8711307645, learning_rate: 0.0000855409, weight_decay: 0.0000343192

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 556.1978759766s, input_fired_time: 8.2294807434, output_fired_time: 0.0000000000, propagate_forward_time: 536.6211547852, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.3472290039
best training[          , genome  5592] predictions:   34837/  50000 (69.67%), best:    6437/10000 (64.37%), error:     45148.98438, best error:     10373.00684 on epoch:    25, epoch:   26/25, mu: 0.8780173063, learning_rate: 0.0000777046, weight_decay: 0.0000318554
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 111.3404083252s, input_fired_time: 1.6537846327, output_fired_time: 0.0000000000, propagate_forward_time: 107.4029617310, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2836608887
   testing[          , genome  5592] predictions:    6437/  10000 (64.37%), best:    6437/10000 (64.37%), error:     10373.00684, best error:     10373.00684 on epoch:    25, epoch:   26/25, mu: 0.8780173063, learning_rate: 0.0000777046, weight_decay: 0.0000318554
backpropagation finished successfully!
20:40:36 (3320): called boinc_finish(0)

</stderr_txt>
]]>