Nameexact_genome_1542031203_182_5611_1
Workunit4081760
Created12 Nov 2018, 19:29:12 UTC
Sent12 Nov 2018, 19:31:17 UTC
Report deadline29 Jan 2019, 0:56:56 UTC
Received24 Nov 2018, 10:20:02 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID87944
Run time2 days 2 hours 2 min 9 sec
CPU time1 days 21 hours 12 min 6 sec
Validate stateValid
Credit19,306.54
Device peak FLOPS2.49 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size524.38 MB
Peak swap size544.10 MB
Peak disk usage9.24 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
g filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.968555
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000547078
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 2.18831e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 61
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 45
read number_validation_images: 10000
read best_validation_predictions: 6041
read best_validation_error: 12011.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '739337873'
read generator: 739337873
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2697.71826171875s, input_fired_time: 22.4850902557373, output_fired_time: 23.997859954834, propagate_forward_time: 1042.12902832031, propagate_backward_time: 1571.48315429688, weight_update_time: 2.70075941085815, other_time: 37.6231689453125
epoch time: 201.896911621094s, input_fired_time: 2.11337780952454, output_fired_time: 0, propagate_forward_time: 195.747695922852, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.03584289550781
validation[          , genome  5611] predictions:    5990/  10000 (59.90%), best:    6041/10000 (60.41%), error:     12340.74121, best error:     12011.34082 on epoch:    45, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 2669.2226562500s, input_fired_time: 30.3690204620, output_fired_time: 29.1207408905, propagate_forward_time: 939.3054199219, propagate_backward_time: 1631.5430908203, weight_update_time: 3.8305234909, other_time: 38.8841552734
epoch time: 198.4855194092s, input_fired_time: 5.1112918854, output_fired_time: 0.0000000000, propagate_forward_time: 184.3283843994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.0458374023
validation[          , genome  5611] predictions:    6113/  10000 (61.13%), best:    6113/10000 (61.13%), error:     11820.00781, best error:     11820.00781 on epoch:    62, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 2685.8020019531s, input_fired_time: 30.2154312134, output_fired_time: 26.9432296753, propagate_forward_time: 975.5818481445, propagate_backward_time: 1615.6201171875, weight_update_time: 1.9309585094, other_time: 37.4416503906
epoch time: 204.8851928711s, input_fired_time: 4.1703386307, output_fired_time: 0.0000000000, propagate_forward_time: 194.2931976318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.4216613770
validation[          , genome  5611] predictions:    6098/  10000 (60.98%), best:    6113/10000 (61.13%), error:     11918.36035, best error:     11820.00781 on epoch:    62, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 2673.5236816406s, input_fired_time: 33.0777435303, output_fired_time: 23.5362567902, propagate_forward_time: 940.7169189453, propagate_backward_time: 1637.7775878906, weight_update_time: 3.8800277710, other_time: 38.4151611328
epoch time: 197.9532318115s, input_fired_time: 6.2619228363, output_fired_time: 0.0000000000, propagate_forward_time: 183.8070526123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.8842620850
validation[          , genome  5611] predictions:    6015/  10000 (60.15%), best:    6113/10000 (61.13%), error:     12311.33789, best error:     11820.00781 on epoch:    62, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 2631.1411132813s, input_fired_time: 28.4225749969, output_fired_time: 27.1499309540, propagate_forward_time: 937.0757446289, propagate_backward_time: 1602.5299072266, weight_update_time: 4.9236288071, other_time: 35.9630126953
epoch time: 192.9726715088s, input_fired_time: 2.1980865002, output_fired_time: 0.0000000000, propagate_forward_time: 184.0308074951, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.7437744141
validation[          , genome  5611] predictions:    6057/  10000 (60.57%), best:    6113/10000 (61.13%), error:     11962.36719, best error:     11820.00781 on epoch:    62, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 2693.3447265625s, input_fired_time: 30.7886390686, output_fired_time: 26.2826995850, propagate_forward_time: 966.3115844727, propagate_backward_time: 1637.9206542969, weight_update_time: 1.8333041668, other_time: 32.0412597656
epoch time: 201.0167541504s, input_fired_time: 2.1692447662, output_fired_time: 0.0000000000, propagate_forward_time: 192.0735626221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.7739410400
validation[          , genome  5611] predictions:    6122/  10000 (61.22%), best:    6113/10000 (61.13%), error:     11865.46484, best error:     11820.00781 on epoch:    62, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 2728.1174316406s, input_fired_time: 25.7231140137, output_fired_time: 28.5165271759, propagate_forward_time: 1001.0229492188, propagate_backward_time: 1637.8181152344, weight_update_time: 2.7110183239, other_time: 35.0366210938
epoch time: 201.2413940430s, input_fired_time: 1.8401151896, output_fired_time: 0.0000000000, propagate_forward_time: 195.1509246826, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2503509521
validation[          , genome  5611] predictions:    6075/  10000 (60.75%), best:    6113/10000 (61.13%), error:     12001.02637, best error:     11820.00781 on epoch:    62, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.975024
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000382045
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.52818e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 68
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 62
read number_validation_images: 10000
read best_validation_predictions: 6113
read best_validation_error: 11820
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '513645869'
read generator: 513645869
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2926.63745117188s, input_fired_time: 27.013219833374, output_fired_time: 26.0996551513672, propagate_forward_time: 986.178466796875, propagate_backward_time: 1853.17468261719, weight_update_time: 1.86548686027527, other_time: 34.1715087890625
epoch time: 198.017562866211s, input_fired_time: 1.80484068393707, output_fired_time: 0, propagate_forward_time: 188.379333496094, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.83338928222656
validation[          , genome  5611] predictions:    6039/  10000 (60.39%), best:    6113/10000 (61.13%), error:     12287.05566, best error:     11820.00781 on epoch:    62, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 2551.5439453125s, input_fired_time: 27.8891944885, output_fired_time: 28.8366794586, propagate_forward_time: 978.8308105469, propagate_backward_time: 1483.2026367188, weight_update_time: 1.5954116583, other_time: 32.7846679688
epoch time: 194.2441864014s, input_fired_time: 1.7670867443, output_fired_time: 0.0000000000, propagate_forward_time: 187.4694061279, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0076904297
validation[          , genome  5611] predictions:    5990/  10000 (59.90%), best:    6113/10000 (61.13%), error:     12202.77734, best error:     11820.00781 on epoch:    62, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 2613.1591796875s, input_fired_time: 27.3908557892, output_fired_time: 22.7722187042, propagate_forward_time: 987.4899291992, propagate_backward_time: 1545.8760986328, weight_update_time: 3.6343657970, other_time: 29.6300048828
epoch time: 200.4902648926s, input_fired_time: 2.7940425873, output_fired_time: 0.0000000000, propagate_forward_time: 190.8117523193, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.8844757080
validation[          , genome  5611] predictions:    6106/  10000 (61.06%), best:    6113/10000 (61.13%), error:     11871.96582, best error:     11820.00781 on epoch:    62, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 2692.1455078125s, input_fired_time: 32.9250259399, output_fired_time: 26.6051807404, propagate_forward_time: 1029.4709472656, propagate_backward_time: 1567.6065673828, weight_update_time: 3.7490394115, other_time: 35.5377197266
epoch time: 184.8300781250s, input_fired_time: 1.7498431206, output_fired_time: 0.0000000000, propagate_forward_time: 177.5113983154, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5688323975
validation[          , genome  5611] predictions:    6076/  10000 (60.76%), best:    6113/10000 (61.13%), error:     11979.30664, best error:     11820.00781 on epoch:    62, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.977802
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000311178
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.24471e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 72
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 62
read number_validation_images: 10000
read best_validation_predictions: 6113
read best_validation_error: 11820
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '1132642374'
read generator: 1132642374
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2623.67651367188s, input_fired_time: 28.6048889160156, output_fired_time: 20.2841567993164, propagate_forward_time: 907.957397460938, propagate_backward_time: 1629.03015136719, weight_update_time: 1.67158389091492, other_time: 37.7998046875
epoch time: 193.298858642578s, input_fired_time: 2.75604677200317, output_fired_time: 0, propagate_forward_time: 182.928863525391, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.61395263671875
validation[          , genome  5611] predictions:    6082/  10000 (60.82%), best:    6113/10000 (61.13%), error:     11896.01563, best error:     11820.00781 on epoch:    62, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.978412
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000295619
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.18248e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 73
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 62
read number_validation_images: 10000
read best_validation_predictions: 6113
read best_validation_error: 11820
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '23531653'
read generator: 23531653
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2774.51000976563s, input_fired_time: 32.4729957580566, output_fired_time: 27.3175716400146, propagate_forward_time: 1078.58325195313, propagate_backward_time: 1576.55114746094, weight_update_time: 2.62757921218872, other_time: 59.5850830078125
epoch time: 197.480895996094s, input_fired_time: 2.72163701057434, output_fired_time: 0, propagate_forward_time: 190.741653442383, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.01760864257813
validation[          , genome  5611] predictions:    6085/  10000 (60.85%), best:    6113/10000 (61.13%), error:     12005.76660, best error:     11820.00781 on epoch:    62, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 2597.8674316406s, input_fired_time: 23.4549388885, output_fired_time: 19.3086090088, propagate_forward_time: 972.0859985352, propagate_backward_time: 1553.6066894531, weight_update_time: 1.6954171658, other_time: 29.4113769531
epoch time: 190.0779876709s, input_fired_time: 2.8448204994, output_fired_time: 0.0000000000, propagate_forward_time: 182.2622222900, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9709472656
validation[          , genome  5611] predictions:    6012/  10000 (60.12%), best:    6113/10000 (61.13%), error:     12019.16797, best error:     11820.00781 on epoch:    62, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 2555.6726074219s, input_fired_time: 25.0144233704, output_fired_time: 24.8235645294, propagate_forward_time: 941.6270141602, propagate_backward_time: 1532.7690429688, weight_update_time: 2.7117257118, other_time: 31.4387207031
epoch time: 202.9801940918s, input_fired_time: 4.2504215240, output_fired_time: 0.0000000000, propagate_forward_time: 189.3901062012, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.3396606445
validation[          , genome  5611] predictions:    5952/  10000 (59.52%), best:    6113/10000 (61.13%), error:     12348.00781, best error:     11820.00781 on epoch:    62, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 2663.4931640625s, input_fired_time: 29.2083854675, output_fired_time: 27.0935630798, propagate_forward_time: 926.9252319336, propagate_backward_time: 1636.6864013672, weight_update_time: 3.0135145187, other_time: 43.5794677734
epoch time: 196.4658813477s, input_fired_time: 3.3332428932, output_fired_time: 0.0000000000, propagate_forward_time: 189.1342926025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9983520508
validation[          , genome  5611] predictions:    6040/  10000 (60.40%), best:    6113/10000 (61.13%), error:     12206.49609, best error:     11820.00781 on epoch:    62, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 2673.4799804688s, input_fired_time: 30.8274097443, output_fired_time: 26.5477809906, propagate_forward_time: 909.9951171875, propagate_backward_time: 1662.7000732422, weight_update_time: 1.9534361362, other_time: 43.4095458984
epoch time: 202.2117767334s, input_fired_time: 3.2199466228, output_fired_time: 0.0000000000, propagate_forward_time: 194.9540405273, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0377960205
validation[          , genome  5611] predictions:    6114/  10000 (61.14%), best:    6114/10000 (61.14%), error:     11762.16406, best error:     11762.16406 on epoch:    77, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 2720.7404785156s, input_fired_time: 30.3741188049, output_fired_time: 23.3299827576, propagate_forward_time: 960.0822143555, propagate_backward_time: 1672.2983398438, weight_update_time: 3.0034811497, other_time: 34.6557617188
epoch time: 192.6131896973s, input_fired_time: 2.3214585781, output_fired_time: 0.0000000000, propagate_forward_time: 178.8299102783, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.4618225098
validation[          , genome  5611] predictions:    6121/  10000 (61.21%), best:    6114/10000 (61.14%), error:     11783.91992, best error:     11762.16406 on epoch:    77, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 2689.6474609375s, input_fired_time: 31.5405807495, output_fired_time: 23.3640136719, propagate_forward_time: 945.0170288086, propagate_backward_time: 1640.4650878906, weight_update_time: 5.9295692444, other_time: 49.2607421875
epoch time: 196.0855865479s, input_fired_time: 4.3485646248, output_fired_time: 0.0000000000, propagate_forward_time: 185.5945587158, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1424560547
validation[          , genome  5611] predictions:    6094/  10000 (60.94%), best:    6114/10000 (61.14%), error:     11969.84473, best error:     11762.16406 on epoch:    77, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 2707.6887207031s, input_fired_time: 39.0045738220, output_fired_time: 24.9591903687, propagate_forward_time: 945.2708740234, propagate_backward_time: 1661.0061035156, weight_update_time: 2.9033296108, other_time: 37.4478759766
epoch time: 206.0151519775s, input_fired_time: 5.4045801163, output_fired_time: 0.0000000000, propagate_forward_time: 193.4189453125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1916198730
validation[          , genome  5611] predictions:    6054/  10000 (60.54%), best:    6114/10000 (61.14%), error:     11994.35059, best error:     11762.16406 on epoch:    77, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 2688.3732910156s, input_fired_time: 30.4512767792, output_fired_time: 28.2781105042, propagate_forward_time: 922.5812988281, propagate_backward_time: 1673.7015380859, weight_update_time: 1.9506779909, other_time: 33.3612060547
epoch time: 194.8512878418s, input_fired_time: 4.0978894234, output_fired_time: 0.0000000000, propagate_forward_time: 183.6154327393, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1379699707
validation[          , genome  5611] predictions:    6062/  10000 (60.62%), best:    6114/10000 (61.14%), error:     12091.36914, best error:     11762.16406 on epoch:    77, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 2745.0742187500s, input_fired_time: 42.5534286499, output_fired_time: 28.9859790802, propagate_forward_time: 952.0016479492, propagate_backward_time: 1681.2205810547, weight_update_time: 4.1141834259, other_time: 40.3123779297
epoch time: 204.5482788086s, input_fired_time: 5.1284842491, output_fired_time: 0.0000000000, propagate_forward_time: 192.3962707520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.0235290527
validation[          , genome  5611] predictions:    5999/  10000 (59.99%), best:    6114/10000 (61.14%), error:     12116.19531, best error:     11762.16406 on epoch:    77, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 2710.7390136719s, input_fired_time: 26.7186374664, output_fired_time: 25.2766017914, propagate_forward_time: 934.0352783203, propagate_backward_time: 1684.4829101563, weight_update_time: 3.0296130180, other_time: 40.2254638672
epoch time: 195.8522186279s, input_fired_time: 6.2641005516, output_fired_time: 0.0000000000, propagate_forward_time: 183.5227661133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0653533936
validation[          , genome  5611] predictions:    6070/  10000 (60.70%), best:    6114/10000 (61.14%), error:     12003.25195, best error:     11762.16406 on epoch:    77, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 2794.2932128906s, input_fired_time: 36.3729095459, output_fired_time: 20.4851818085, propagate_forward_time: 975.2731323242, propagate_backward_time: 1716.8647460938, weight_update_time: 2.9620013237, other_time: 45.2973632813
epoch time: 197.4658050537s, input_fired_time: 2.2446608543, output_fired_time: 0.0000000000, propagate_forward_time: 187.8546752930, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.3664703369
validation[          , genome  5611] predictions:    6113/  10000 (61.13%), best:    6114/10000 (61.14%), error:     11942.36328, best error:     11762.16406 on epoch:    77, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 2652.8874511719s, input_fired_time: 31.8348770142, output_fired_time: 31.3525924683, propagate_forward_time: 939.7710571289, propagate_backward_time: 1608.6829833984, weight_update_time: 2.9206089973, other_time: 41.2459716797
epoch time: 191.3706054688s, input_fired_time: 3.1868116856, output_fired_time: 0.0000000000, propagate_forward_time: 183.9958648682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1879272461
validation[          , genome  5611] predictions:    6063/  10000 (60.63%), best:    6114/10000 (61.14%), error:     11879.38867, best error:     11762.16406 on epoch:    77, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 2646.3376464844s, input_fired_time: 32.5216178894, output_fired_time: 23.0213890076, propagate_forward_time: 952.3850708008, propagate_backward_time: 1599.9140625000, weight_update_time: 1.8661578894, other_time: 38.4953613281
epoch time: 195.0112609863s, input_fired_time: 3.2659657001, output_fired_time: 0.0000000000, propagate_forward_time: 183.8155364990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.9297637939
validation[          , genome  5611] predictions:    6089/  10000 (60.89%), best:    6114/10000 (61.14%), error:     11894.42676, best error:     11762.16406 on epoch:    77, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 2693.5905761719s, input_fired_time: 32.5937004089, output_fired_time: 24.3662681580, propagate_forward_time: 952.5570068359, propagate_backward_time: 1645.1007080078, weight_update_time: 1.9679673910, other_time: 38.9729003906
epoch time: 198.7056121826s, input_fired_time: 3.1970069408, output_fired_time: 0.0000000000, propagate_forward_time: 190.7491455078, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7594604492
validation[          , genome  5611] predictions:    6041/  10000 (60.41%), best:    6114/10000 (61.14%), error:     12089.48242, best error:     11762.16406 on epoch:    77, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 2933.8872070313s, input_fired_time: 31.9772319794, output_fired_time: 22.2368602753, propagate_forward_time: 1075.6811523438, propagate_backward_time: 1754.9888916016, weight_update_time: 3.1863913536, other_time: 49.0030517578
epoch time: 206.8668823242s, input_fired_time: 4.0608644485, output_fired_time: 0.0000000000, propagate_forward_time: 188.9512939453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.8547210693
validation[          , genome  5611] predictions:    6064/  10000 (60.64%), best:    6114/10000 (61.14%), error:     11925.73242, best error:     11762.16406 on epoch:    77, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.9849
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00013011
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 5.2044e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 89
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 77
read number_validation_images: 10000
read best_validation_predictions: 6114
read best_validation_error: 11762.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '1217680201'
read generator: 1217680201
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.9849
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00013011
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 5.2044e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 89
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 77
read number_validation_images: 10000
read best_validation_predictions: 6114
read best_validation_error: 11762.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '1217680201'
read generator: 1217680201
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2879.73120117188s, input_fired_time: 28.1417350769043, output_fired_time: 23.9248123168945, propagate_forward_time: 1042.28234863281, propagate_backward_time: 1746.35620117188, weight_update_time: 1.76701951026917, other_time: 39.0260009765625
epoch time: 181.391204833984s, input_fired_time: 2.70852375030518, output_fired_time: 0, propagate_forward_time: 170.886322021484, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.79635620117188
validation[          , genome  5611] predictions:    6108/  10000 (61.08%), best:    6114/10000 (61.14%), error:     11907.72656, best error:     11762.16406 on epoch:    77, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 2753.0417480469s, input_fired_time: 26.0787963867, output_fired_time: 17.6519260406, propagate_forward_time: 965.6207885742, propagate_backward_time: 1711.7602539063, weight_update_time: 2.8128516674, other_time: 31.9299316406
epoch time: 178.8923797607s, input_fired_time: 1.7186297178, output_fired_time: 0.0000000000, propagate_forward_time: 173.1143493652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0594024658
validation[          , genome  5611] predictions:    6100/  10000 (61.00%), best:    6114/10000 (61.14%), error:     11923.79102, best error:     11762.16406 on epoch:    77, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 2860.5783691406s, input_fired_time: 31.2480640411, output_fired_time: 21.0658092499, propagate_forward_time: 1057.6123046875, propagate_backward_time: 1710.4335937500, weight_update_time: 1.7947663069, other_time: 40.2185058594
epoch time: 213.5755767822s, input_fired_time: 3.0883052349, output_fired_time: 0.0000000000, propagate_forward_time: 203.2541351318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2331390381
validation[          , genome  5611] predictions:    6035/  10000 (60.35%), best:    6114/10000 (61.14%), error:     12039.29590, best error:     11762.16406 on epoch:    77, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 2975.9919433594s, input_fired_time: 27.2262001038, output_fired_time: 27.6253547668, propagate_forward_time: 1078.4268798828, propagate_backward_time: 1800.5953369141, weight_update_time: 2.0664598942, other_time: 42.1181640625
epoch time: 207.8199005127s, input_fired_time: 3.0220334530, output_fired_time: 0.0000000000, propagate_forward_time: 199.6119079590, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1859588623
validation[          , genome  5611] predictions:    6045/  10000 (60.45%), best:    6114/10000 (61.14%), error:     12181.73926, best error:     11762.16406 on epoch:    77, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 3024.8920898438s, input_fired_time: 36.2354316711, output_fired_time: 29.0167865753, propagate_forward_time: 1066.6970214844, propagate_backward_time: 1854.4250488281, weight_update_time: 2.0101742744, other_time: 38.5178222656
epoch time: 200.0141754150s, input_fired_time: 2.7698872089, output_fired_time: 0.0000000000, propagate_forward_time: 190.0763549805, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1679382324
validation[          , genome  5611] predictions:    6071/  10000 (60.71%), best:    6114/10000 (61.14%), error:     11972.86035, best error:     11762.16406 on epoch:    77, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_182_5611_1_r1502932211_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.986053
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000100677
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 4.02706e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 94
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 77
read number_validation_images: 10000
read best_validation_predictions: 6114
read best_validation_error: 11762.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5611
read normal distribution: '0 -0.185655727982521 1.07132852077484'
generator_str: '1410289254'
read generator: 1410289254
read generated_by_map:
1 add_node 1
reading 75 nodes.
reading 259 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 4523.61376953125s, input_fired_time: 43.6850471496582, output_fired_time: 48.6692237854004, propagate_forward_time: 1578.04467773438, propagate_backward_time: 2741.88500976563, weight_update_time: 5.29506206512451, other_time: 111.32958984375
epoch time: 205.666320800781s, input_fired_time: 3.20232081413269, output_fired_time: 0, propagate_forward_time: 195.082611083984, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.38139343261719
validation[          , genome  5611] predictions:    6122/  10000 (61.22%), best:    6114/10000 (61.14%), error:     11890.68750, best error:     11762.16406 on epoch:    77, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 2671.8127441406s, input_fired_time: 29.2780323029, output_fired_time: 27.6589546204, propagate_forward_time: 860.3657836914, propagate_backward_time: 1719.7698974609, weight_update_time: 1.9348683357, other_time: 34.7401123047
epoch time: 198.4593505859s, input_fired_time: 3.3291900158, output_fired_time: 0.0000000000, propagate_forward_time: 189.7330017090, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3971557617
validation[          , genome  5611] predictions:    6180/  10000 (61.80%), best:    6114/10000 (61.14%), error:     11844.80371, best error:     11762.16406 on epoch:    77, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 2776.1877441406s, input_fired_time: 36.4172821045, output_fired_time: 26.8315944672, propagate_forward_time: 957.2035522461, propagate_backward_time: 1711.7460937500, weight_update_time: 3.0409646034, other_time: 43.9892578125
epoch time: 200.3849487305s, input_fired_time: 4.3230485916, output_fired_time: 0.0000000000, propagate_forward_time: 188.7772216797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2846832275
validation[          , genome  5611] predictions:    6144/  10000 (61.44%), best:    6114/10000 (61.14%), error:     11783.34668, best error:     11762.16406 on epoch:    77, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 2700.1166992188s, input_fired_time: 28.2884235382, output_fired_time: 25.8257446289, propagate_forward_time: 921.2833862305, propagate_backward_time: 1683.4173583984, weight_update_time: 1.9336290359, other_time: 41.3018798828
epoch time: 167.6775970459s, input_fired_time: 2.1188406944, output_fired_time: 0.0000000000, propagate_forward_time: 154.8246002197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.7341613770
validation[          , genome  5611] predictions:    6113/  10000 (61.13%), best:    6114/10000 (61.14%), error:     11908.45117, best error:     11762.16406 on epoch:    77, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 2667.7578125000s, input_fired_time: 27.3796596527, output_fired_time: 21.2390289307, propagate_forward_time: 864.7346801758, propagate_backward_time: 1708.7568359375, weight_update_time: 2.9712963104, other_time: 45.6477050781
epoch time: 184.3968811035s, input_fired_time: 3.7352561951, output_fired_time: 0.0000000000, propagate_forward_time: 176.6572875977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0043334961
validation[          , genome  5611] predictions:    6106/  10000 (61.06%), best:    6114/10000 (61.14%), error:     11930.30273, best error:     11762.16406 on epoch:    77, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 2642.8815917969s, input_fired_time: 26.3615188599, output_fired_time: 19.5620975494, propagate_forward_time: 849.0918579102, propagate_backward_time: 1714.7885742188, weight_update_time: 3.6715145111, other_time: 33.0776367188
epoch time: 184.3567962646s, input_fired_time: 1.9505459070, output_fired_time: 0.0000000000, propagate_forward_time: 176.3473815918, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0588684082
validation[          , genome  5611] predictions:    6129/  10000 (61.29%), best:    6114/10000 (61.14%), error:     11810.42383, best error:     11762.16406 on epoch:    77, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 2541.1149902344s, input_fired_time: 29.1819343567, output_fired_time: 19.4791755676, propagate_forward_time: 830.1763305664, propagate_backward_time: 1627.4654541016, weight_update_time: 2.6914732456, other_time: 34.8121337891
epoch time: 229.5789031982s, input_fired_time: 3.6996099949, output_fired_time: 0.0000000000, propagate_forward_time: 216.4387207031, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.4405670166
validation[          , genome  5611] predictions:    6129/  10000 (61.29%), best:    6114/10000 (61.14%), error:     11845.76172, best error:     11762.16406 on epoch:    77, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 944.9108276367s, input_fired_time: 13.0565462112, output_fired_time: 0.0000000000, propagate_forward_time: 898.8889160156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 32.9653930664
best training[          , genome  5611] predictions:   38963/  50000 (77.93%), best:    6114/10000 (61.14%), error:     33221.66406, best error:     11762.16406 on epoch:    77, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 175.7330780029s, input_fired_time: 1.7397483587, output_fired_time: 0.0000000000, propagate_forward_time: 168.0261993408, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.9671325684
   testing[          , genome  5611] predictions:    6114/  10000 (61.14%), best:    6114/10000 (61.14%), error:     11762.16406, best error:     11762.16406 on epoch:    77, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
10:42:23 (10644): called boinc_finish(0)

</stderr_txt>
]]>