Nameexact_genome_1542303603_187_5990_4
Workunit4091018
Created22 Jun 2019, 21:27:26 UTC
Sent22 Jun 2019, 21:27:49 UTC
Report deadline23 Sep 2019, 13:50:04 UTC
Received30 Jun 2019, 15:03:14 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID6323
Run time1 days 20 hours 42 min 59 sec
CPU time1 days 19 hours 26 min 51 sec
Validate stateValid
Credit23,170.53
Device peak FLOPS3.36 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size543.52 MB
Peak swap size563.03 MB
Peak disk usage11.04 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
258
epoch time: 113.8984069824s, input_fired_time: 1.5235921144, output_fired_time: 0.0000000000, propagate_forward_time: 109.9848022461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3900146484
validation[          , genome  5990] predictions:    6511/  10000 (65.11%), best:    6510/10000 (65.10%), error:     10086.22266, best error:     10017.02344 on epoch:    50, epoch:   55/100, mu: 0.9677840471, learning_rate: 0.0000238593, weight_decay: 0.0000337573

epoch time: 1434.4415283203s, input_fired_time: 19.6904850006, output_fired_time: 10.0220785141, propagate_forward_time: 544.2368774414, propagate_backward_time: 846.2017211914, weight_update_time: 0.5209017396, other_time: 14.2904052734
epoch time: 114.9170684814s, input_fired_time: 1.5353606939, output_fired_time: 0.0000000000, propagate_forward_time: 110.9743957520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4073104858
validation[          , genome  5990] predictions:    6478/  10000 (64.78%), best:    6510/10000 (65.10%), error:     10050.06250, best error:     10017.02344 on epoch:    50, epoch:   56/100, mu: 0.9689860940, learning_rate: 0.0000221358, weight_decay: 0.0000320450

epoch time: 1435.8236083984s, input_fired_time: 20.0701732635, output_fired_time: 10.1775903702, propagate_forward_time: 541.8612060547, propagate_backward_time: 848.6782836914, weight_update_time: 0.5207704306, other_time: 15.0363159180
epoch time: 116.3134841919s, input_fired_time: 1.6397893429, output_fired_time: 0.0000000000, propagate_forward_time: 111.5707855225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1029129028
validation[          , genome  5990] predictions:    6474/  10000 (64.74%), best:    6510/10000 (65.10%), error:     10100.10352, best error:     10017.02344 on epoch:    50, epoch:   57/100, mu: 0.9701230526, learning_rate: 0.0000205369, weight_decay: 0.0000304196

epoch time: 1423.1196289063s, input_fired_time: 20.0852565765, output_fired_time: 10.3026113510, propagate_forward_time: 535.8234863281, propagate_backward_time: 841.5336303711, weight_update_time: 0.5324239731, other_time: 15.3746948242
epoch time: 113.9242858887s, input_fired_time: 1.5131400824, output_fired_time: 0.0000000000, propagate_forward_time: 110.0172042847, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3939437866
validation[          , genome  5990] predictions:    6476/  10000 (64.76%), best:    6510/10000 (65.10%), error:     10037.61719, best error:     10017.02344 on epoch:    50, epoch:   58/100, mu: 0.9711984992, learning_rate: 0.0000190534, weight_decay: 0.0000288767

epoch time: 1414.2451171875s, input_fired_time: 20.9467525482, output_fired_time: 11.0570564270, propagate_forward_time: 525.7613525391, propagate_backward_time: 839.7438354492, weight_update_time: 0.5735583901, other_time: 16.7361450195
epoch time: 120.2225799561s, input_fired_time: 2.2781727314, output_fired_time: 0.0000000000, propagate_forward_time: 113.4439468384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5004577637
validation[          , genome  5990] predictions:    6564/  10000 (65.64%), best:    6564/10000 (65.64%), error:      9992.31348, best error:      9992.31348 on epoch:    59, epoch:   59/100, mu: 0.9722157717, learning_rate: 0.0000176770, weight_decay: 0.0000274120

epoch time: 1523.2274169922s, input_fired_time: 28.6091060638, output_fired_time: 17.2802276611, propagate_forward_time: 558.3112182617, propagate_backward_time: 890.3786621094, weight_update_time: 0.9533863068, other_time: 28.6481323242
epoch time: 120.1345825195s, input_fired_time: 2.7444541454, output_fired_time: 0.0000000000, propagate_forward_time: 111.9229278564, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4672012329
validation[          , genome  5990] predictions:    6516/  10000 (65.16%), best:    6564/10000 (65.64%), error:     10040.61523, best error:      9992.31348 on epoch:    59, epoch:   60/100, mu: 0.9731780291, learning_rate: 0.0000164001, weight_decay: 0.0000260216

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.974088
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 1.52155e-05
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 2.47017e-05
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 61
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 59
read number_validation_images: 10000
read best_validation_predictions: 6564
read best_validation_error: 9992.31
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '412261806'
read generator: 412261806
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1456.51672363281s, input_fired_time: 21.8429565429688, output_fired_time: 11.738374710083, propagate_forward_time: 547.583801269531, propagate_backward_time: 855.882934570313, weight_update_time: 0.700311064720154, other_time: 19.4686889648438
epoch time: 113.656036376953s, input_fired_time: 1.68171310424805, output_fired_time: 0, propagate_forward_time: 108.878829956055, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.09548950195313
validation[          , genome  5990] predictions:    6479/  10000 (64.79%), best:    6564/10000 (65.64%), error:     10090.04883, best error:      9992.31348 on epoch:    59, epoch:   61/100, mu: 0.9740881920, learning_rate: 0.0000152155, weight_decay: 0.0000247017

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.974949
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 1.41164e-05
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 2.34487e-05
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 62
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 59
read number_validation_images: 10000
read best_validation_predictions: 6564
read best_validation_error: 9992.31
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '699494667'
read generator: 699494667
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1439.39514160156s, input_fired_time: 20.1667213439941, output_fired_time: 10.4534273147583, propagate_forward_time: 546.214599609375, propagate_backward_time: 846.901184082031, weight_update_time: 0.54279237985611, other_time: 15.6592407226563
epoch time: 115.83667755127s, input_fired_time: 1.57098233699799, output_fired_time: 0, propagate_forward_time: 111.532867431641, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.73282623291016
validation[          , genome  5990] predictions:    6533/  10000 (65.33%), best:    6533/10000 (65.33%), error:      9891.52441, best error:      9891.52441 on epoch:    62, epoch:   62/100, mu: 0.9749491215, learning_rate: 0.0000141164, weight_decay: 0.0000234487

epoch time: 1432.0643310547s, input_fired_time: 19.9966831207, output_fired_time: 10.3102607727, propagate_forward_time: 541.8662109375, propagate_backward_time: 844.8374023438, weight_update_time: 0.5312573910, other_time: 15.0537109375
epoch time: 108.9710083008s, input_fired_time: 1.6517506838, output_fired_time: 0.0000000000, propagate_forward_time: 104.4093246460, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9099349976
validation[          , genome  5990] predictions:    6534/  10000 (65.34%), best:    6534/10000 (65.34%), error:      9850.05469, best error:      9850.05469 on epoch:    63, epoch:   63/100, mu: 0.9757634997, learning_rate: 0.0000130967, weight_decay: 0.0000222594

epoch time: 1500.8248291016s, input_fired_time: 25.3770198822, output_fired_time: 14.8361005783, propagate_forward_time: 556.3631591797, propagate_backward_time: 879.9083251953, weight_update_time: 0.8077791333, other_time: 24.3402099609
epoch time: 122.7843551636s, input_fired_time: 2.7319943905, output_fired_time: 0.0000000000, propagate_forward_time: 114.8287658691, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2235946655
validation[          , genome  5990] predictions:    6548/  10000 (65.48%), best:    6534/10000 (65.34%), error:      9893.19531, best error:      9850.05469 on epoch:    63, epoch:   64/100, mu: 0.9765337706, learning_rate: 0.0000121506, weight_decay: 0.0000211303

epoch time: 1542.2739257813s, input_fired_time: 30.5847473145, output_fired_time: 18.2684192657, propagate_forward_time: 562.3646240234, propagate_backward_time: 900.1442871094, weight_update_time: 1.0371657610, other_time: 30.9118652344
epoch time: 122.5280075073s, input_fired_time: 2.5885002613, output_fired_time: 0.0000000000, propagate_forward_time: 114.5301513672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4093551636
validation[          , genome  5990] predictions:    6533/  10000 (65.33%), best:    6534/10000 (65.34%), error:      9912.49609, best error:      9850.05469 on epoch:    63, epoch:   65/100, mu: 0.9772623777, learning_rate: 0.0000112729, weight_decay: 0.0000200585

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.977952
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 1.04586e-05
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 1.90411e-05
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 66
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 63
read number_validation_images: 10000
read best_validation_predictions: 6534
read best_validation_error: 9850.05
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '2106221631'
read generator: 2106221631
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1510.52429199219s, input_fired_time: 26.9966564178467, output_fired_time: 15.7614889144897, propagate_forward_time: 557.940612792969, propagate_backward_time: 883.307556152344, weight_update_time: 0.868953764438629, other_time: 26.5179443359375
epoch time: 120.300170898438s, input_fired_time: 2.34683132171631, output_fired_time: 0, propagate_forward_time: 113.088569641113, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.86476898193359
validation[          , genome  5990] predictions:    6562/  10000 (65.62%), best:    6534/10000 (65.34%), error:      9926.92578, best error:      9850.05469 on epoch:    63, epoch:   66/100, mu: 0.9779515862, learning_rate: 0.0000104586, weight_decay: 0.0000190411

epoch time: 1525.5819091797s, input_fired_time: 27.7671451569, output_fired_time: 16.3536491394, propagate_forward_time: 562.2953491211, propagate_backward_time: 892.1403198242, weight_update_time: 0.9172000289, other_time: 27.0255126953
epoch time: 123.2837677002s, input_fired_time: 2.5224573612, output_fired_time: 0.0000000000, propagate_forward_time: 115.7252807617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0360260010
validation[          , genome  5990] predictions:    6526/  10000 (65.26%), best:    6534/10000 (65.34%), error:      9979.73438, best error:      9850.05469 on epoch:    63, epoch:   67/100, mu: 0.9786034822, learning_rate: 0.0000097031, weight_decay: 0.0000180753

epoch time: 1605.2436523438s, input_fired_time: 31.6290359497, output_fired_time: 18.1703224182, propagate_forward_time: 587.0670166016, propagate_backward_time: 939.0793457031, weight_update_time: 1.0088891983, other_time: 29.2979736328
epoch time: 124.8639907837s, input_fired_time: 2.9401988983, output_fired_time: 0.0000000000, propagate_forward_time: 116.4516677856, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4721221924
validation[          , genome  5990] predictions:    6558/  10000 (65.58%), best:    6534/10000 (65.34%), error:      9917.13867, best error:      9850.05469 on epoch:    63, epoch:   68/100, mu: 0.9792200923, learning_rate: 0.0000090022, weight_decay: 0.0000171585

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.979803
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 8.35195e-06
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 1.62882e-05
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 69
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 63
read number_validation_images: 10000
read best_validation_predictions: 6534
read best_validation_error: 9850.05
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '1987041766'
read generator: 1987041766
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1478.04443359375s, input_fired_time: 26.9005870819092, output_fired_time: 15.3768968582153, propagate_forward_time: 540.657470703125, propagate_backward_time: 869.167907714844, weight_update_time: 0.911624789237976, other_time: 25.9414672851563
epoch time: 113.90535736084s, input_fired_time: 1.8422919511795, output_fired_time: 0, propagate_forward_time: 108.433685302734, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.62937927246094
validation[          , genome  5990] predictions:    6547/  10000 (65.47%), best:    6534/10000 (65.34%), error:      9875.91895, best error:      9850.05469 on epoch:    63, epoch:   69/100, mu: 0.9798033834, learning_rate: 0.0000083520, weight_decay: 0.0000162882

epoch time: 1464.7028808594s, input_fired_time: 21.9774856567, output_fired_time: 11.4516706467, propagate_forward_time: 552.1253662109, propagate_backward_time: 861.2217407227, weight_update_time: 0.6598712802, other_time: 17.9265747070
epoch time: 114.2103271484s, input_fired_time: 1.6944330931, output_fired_time: 0.0000000000, propagate_forward_time: 109.5017471313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0141448975
validation[          , genome  5990] predictions:    6622/  10000 (66.22%), best:    6622/10000 (66.22%), error:      9726.60059, best error:      9726.60059 on epoch:    70, epoch:   70/100, mu: 0.9803550839, learning_rate: 0.0000077486, weight_decay: 0.0000154620

epoch time: 1427.9489746094s, input_fired_time: 20.5154838562, output_fired_time: 10.6014709473, propagate_forward_time: 538.3309936523, propagate_backward_time: 842.4686279297, weight_update_time: 0.5781683326, other_time: 16.0324096680
epoch time: 117.9767150879s, input_fired_time: 1.7496179342, output_fired_time: 0.0000000000, propagate_forward_time: 113.4607467651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7663497925
validation[          , genome  5990] predictions:    6562/  10000 (65.62%), best:    6622/10000 (66.22%), error:      9925.07520, best error:      9726.60059 on epoch:    70, epoch:   71/100, mu: 0.9808769226, learning_rate: 0.0000071889, weight_decay: 0.0000146777

epoch time: 1424.5750732422s, input_fired_time: 20.0982379913, output_fired_time: 10.1438455582, propagate_forward_time: 535.5273437500, propagate_backward_time: 844.3707275391, weight_update_time: 0.5497660041, other_time: 14.4349365234
epoch time: 115.6625442505s, input_fired_time: 1.6380097866, output_fired_time: 0.0000000000, propagate_forward_time: 111.5787582397, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4457778931
validation[          , genome  5990] predictions:    6626/  10000 (66.26%), best:    6622/10000 (66.22%), error:      9756.39258, best error:      9726.60059 on epoch:    70, epoch:   72/100, mu: 0.9813705683, learning_rate: 0.0000066696, weight_decay: 0.0000139332

epoch time: 1442.8496093750s, input_fired_time: 20.5168190002, output_fired_time: 10.3694868088, propagate_forward_time: 546.4531250000, propagate_backward_time: 850.9049682617, weight_update_time: 0.5793638229, other_time: 14.6051635742
epoch time: 113.1558227539s, input_fired_time: 1.6084184647, output_fired_time: 0.0000000000, propagate_forward_time: 109.1025238037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4448776245
validation[          , genome  5990] predictions:    6592/  10000 (65.92%), best:    6622/10000 (66.22%), error:      9841.98340, best error:      9726.60059 on epoch:    70, epoch:   73/100, mu: 0.9818374515, learning_rate: 0.0000061878, weight_decay: 0.0000132265

epoch time: 1435.2003173828s, input_fired_time: 20.1903533936, output_fired_time: 10.0641126633, propagate_forward_time: 542.6805419922, propagate_backward_time: 848.3189697266, weight_update_time: 0.5448296070, other_time: 13.9464111328
epoch time: 114.6917648315s, input_fired_time: 1.5955389738, output_fired_time: 0.0000000000, propagate_forward_time: 110.5936813354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5025482178
validation[          , genome  5990] predictions:    6617/  10000 (66.17%), best:    6622/10000 (66.22%), error:      9739.26465, best error:      9726.60059 on epoch:    70, epoch:   74/100, mu: 0.9822791219, learning_rate: 0.0000057409, weight_decay: 0.0000125556

epoch time: 1416.8000488281s, input_fired_time: 20.2156200409, output_fired_time: 10.0714273453, propagate_forward_time: 531.0034179688, propagate_backward_time: 841.3072509766, weight_update_time: 0.5576832891, other_time: 14.2023925781
epoch time: 110.3191146851s, input_fired_time: 1.6339030266, output_fired_time: 0.0000000000, propagate_forward_time: 105.9034729004, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7817382813
validation[          , genome  5990] predictions:    6658/  10000 (66.58%), best:    6658/10000 (66.58%), error:      9698.79883, best error:      9698.79883 on epoch:    75, epoch:   75/100, mu: 0.9826968908, learning_rate: 0.0000053262, weight_decay: 0.0000119188

epoch time: 1409.0083007813s, input_fired_time: 20.1991558075, output_fired_time: 9.9889812469, propagate_forward_time: 531.2401733398, propagate_backward_time: 833.7301025391, weight_update_time: 0.5448094606, other_time: 13.8499145508
epoch time: 115.7821426392s, input_fired_time: 1.5707147121, output_fired_time: 0.0000000000, propagate_forward_time: 111.8795166016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3319091797
validation[          , genome  5990] predictions:    6662/  10000 (66.62%), best:    6658/10000 (66.58%), error:      9780.66992, best error:      9698.79883 on epoch:    75, epoch:   76/100, mu: 0.9830920100, learning_rate: 0.0000049414, weight_decay: 0.0000113142

epoch time: 1400.4910888672s, input_fired_time: 19.8796100616, output_fired_time: 9.9204902649, propagate_forward_time: 527.6635742188, propagate_backward_time: 829.1880493164, weight_update_time: 0.5576853752, other_time: 13.8392944336
epoch time: 117.0601501465s, input_fired_time: 1.5888184309, output_fired_time: 0.0000000000, propagate_forward_time: 112.9311676025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5401611328
validation[          , genome  5990] predictions:    6645/  10000 (66.45%), best:    6658/10000 (66.58%), error:      9709.39453, best error:      9698.79883 on epoch:    75, epoch:   77/100, mu: 0.9834657907, learning_rate: 0.0000045845, weight_decay: 0.0000107403

epoch time: 1460.1009521484s, input_fired_time: 20.5579299927, output_fired_time: 10.5784292221, propagate_forward_time: 551.9046630859, propagate_backward_time: 862.3280639648, weight_update_time: 0.5956006646, other_time: 14.7318725586
epoch time: 121.4058532715s, input_fired_time: 1.6764165163, output_fired_time: 0.0000000000, propagate_forward_time: 116.9770355225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7524032593
validation[          , genome  5990] predictions:    6654/  10000 (66.54%), best:    6654/10000 (66.54%), error:      9689.65332, best error:      9689.65332 on epoch:    78, epoch:   78/100, mu: 0.9838193059, learning_rate: 0.0000042533, weight_decay: 0.0000101956

epoch time: 1491.3210449219s, input_fired_time: 21.5625457764, output_fired_time: 11.1098909378, propagate_forward_time: 563.5126953125, propagate_backward_time: 878.9824218750, weight_update_time: 0.6668018103, other_time: 16.1535644531
epoch time: 121.8783569336s, input_fired_time: 1.8209791183, output_fired_time: 0.0000000000, propagate_forward_time: 117.0822906494, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9750900269
validation[          , genome  5990] predictions:    6570/  10000 (65.70%), best:    6654/10000 (66.54%), error:      9825.21777, best error:      9689.65332 on epoch:    78, epoch:   79/100, mu: 0.9841537476, learning_rate: 0.0000039461, weight_decay: 0.0000096784

epoch time: 1511.2968750000s, input_fired_time: 22.3609542847, output_fired_time: 11.5857095718, propagate_forward_time: 566.0484008789, propagate_backward_time: 893.6124267578, weight_update_time: 0.6758778691, other_time: 17.6893920898
epoch time: 118.9446182251s, input_fired_time: 1.7341446877, output_fired_time: 0.0000000000, propagate_forward_time: 114.1966400146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0138320923
validation[          , genome  5990] predictions:    6583/  10000 (65.83%), best:    6654/10000 (66.54%), error:      9740.26660, best error:      9689.65332 on epoch:    78, epoch:   80/100, mu: 0.9844700694, learning_rate: 0.0000036610, weight_decay: 0.0000091875

epoch time: 1444.9669189453s, input_fired_time: 20.8430404663, output_fired_time: 10.6542015076, propagate_forward_time: 544.1273193359, propagate_backward_time: 853.3645019531, weight_update_time: 0.6369946003, other_time: 15.9779052734
epoch time: 119.4825515747s, input_fired_time: 1.7020649910, output_fired_time: 0.0000000000, propagate_forward_time: 114.9552154541, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8252716064
validation[          , genome  5990] predictions:    6659/  10000 (66.59%), best:    6654/10000 (66.54%), error:      9737.52734, best error:      9689.65332 on epoch:    78, epoch:   81/100, mu: 0.9847692847, learning_rate: 0.0000033966, weight_decay: 0.0000087215

epoch time: 1444.8352050781s, input_fired_time: 20.4660778046, output_fired_time: 10.4369001389, propagate_forward_time: 547.2846069336, propagate_backward_time: 851.7838134766, weight_update_time: 0.6312404275, other_time: 14.8638305664
epoch time: 114.4194488525s, input_fired_time: 1.6375547647, output_fired_time: 0.0000000000, propagate_forward_time: 109.9464645386, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8354263306
validation[          , genome  5990] predictions:    6638/  10000 (66.38%), best:    6654/10000 (66.54%), error:      9716.87109, best error:      9689.65332 on epoch:    78, epoch:   82/100, mu: 0.9850522876, learning_rate: 0.0000031512, weight_decay: 0.0000082791

epoch time: 1432.6313476563s, input_fired_time: 20.7055301666, output_fired_time: 10.5534162521, propagate_forward_time: 537.7437133789, propagate_backward_time: 847.9269409180, weight_update_time: 0.6306890845, other_time: 15.7016601563
epoch time: 107.2432937622s, input_fired_time: 1.5568788052, output_fired_time: 0.0000000000, propagate_forward_time: 102.8889007568, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7975158691
validation[          , genome  5990] predictions:    6662/  10000 (66.62%), best:    6654/10000 (66.54%), error:      9736.57422, best error:      9689.65332 on epoch:    78, epoch:   83/100, mu: 0.9853199720, learning_rate: 0.0000029236, weight_decay: 0.0000078592

epoch time: 1551.7229003906s, input_fired_time: 22.5698547363, output_fired_time: 11.4745359421, propagate_forward_time: 584.7810668945, propagate_backward_time: 916.0032958984, weight_update_time: 0.7228211164, other_time: 16.8942260742
epoch time: 127.1753540039s, input_fired_time: 1.7499617338, output_fired_time: 0.0000000000, propagate_forward_time: 122.8440017700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5813903809
validation[          , genome  5990] predictions:    6668/  10000 (66.68%), best:    6668/10000 (66.68%), error:      9662.23242, best error:      9662.23242 on epoch:    84, epoch:   84/100, mu: 0.9855731726, learning_rate: 0.0000027124, weight_decay: 0.0000074605

epoch time: 1540.5767822266s, input_fired_time: 21.7555370331, output_fired_time: 11.4772911072, propagate_forward_time: 588.0613403320, propagate_backward_time: 903.6484985352, weight_update_time: 0.7241228819, other_time: 15.6341552734
epoch time: 102.2795410156s, input_fired_time: 1.4781315327, output_fired_time: 0.0000000000, propagate_forward_time: 98.3696365356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4317703247
validation[          , genome  5990] predictions:    6665/  10000 (66.65%), best:    6665/10000 (66.65%), error:      9634.64648, best error:      9634.64648 on epoch:    85, epoch:   85/100, mu: 0.9858126640, learning_rate: 0.0000025165, weight_decay: 0.0000070821

epoch time: 1425.9952392578s, input_fired_time: 20.2602462769, output_fired_time: 10.0740442276, propagate_forward_time: 538.2826538086, propagate_backward_time: 842.8598632813, weight_update_time: 0.5974230170, other_time: 14.5183715820
epoch time: 114.6118927002s, input_fired_time: 1.5676033497, output_fired_time: 0.0000000000, propagate_forward_time: 110.6092834473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4350051880
validation[          , genome  5990] predictions:    6637/  10000 (66.37%), best:    6665/10000 (66.65%), error:      9677.20313, best error:      9634.64648 on epoch:    85, epoch:   86/100, mu: 0.9860392213, learning_rate: 0.0000023347, weight_decay: 0.0000067229

epoch time: 1407.0170898438s, input_fired_time: 20.1836299896, output_fired_time: 10.0881004333, propagate_forward_time: 527.9652099609, propagate_backward_time: 834.7001342773, weight_update_time: 0.6447416544, other_time: 14.0800170898
epoch time: 106.9914932251s, input_fired_time: 1.5459170341, output_fired_time: 0.0000000000, propagate_forward_time: 102.8892974854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5562820435
validation[          , genome  5990] predictions:    6681/  10000 (66.81%), best:    6681/10000 (66.81%), error:      9631.60352, best error:      9631.60352 on epoch:    87, epoch:   87/100, mu: 0.9862535000, learning_rate: 0.0000021661, weight_decay: 0.0000063819

epoch time: 1475.3487548828s, input_fired_time: 21.9041061401, output_fired_time: 11.6830368042, propagate_forward_time: 552.4618530273, propagate_backward_time: 871.6903076172, weight_update_time: 0.7141209841, other_time: 17.6095581055
epoch time: 118.1274032593s, input_fired_time: 2.0024871826, output_fired_time: 0.0000000000, propagate_forward_time: 112.3779449463, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7469711304
validation[          , genome  5990] predictions:    6650/  10000 (66.50%), best:    6681/10000 (66.81%), error:      9704.22656, best error:      9631.60352 on epoch:    87, epoch:   88/100, mu: 0.9864562154, learning_rate: 0.0000020096, weight_decay: 0.0000060582

epoch time: 1488.6025390625s, input_fired_time: 24.7995872498, output_fired_time: 13.9891843796, propagate_forward_time: 550.8613281250, propagate_backward_time: 876.0160522461, weight_update_time: 0.8881443143, other_time: 22.9364624023
epoch time: 116.4504013062s, input_fired_time: 2.2456784248, output_fired_time: 0.0000000000, propagate_forward_time: 109.8359985352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3687210083
validation[          , genome  5990] predictions:    6668/  10000 (66.68%), best:    6681/10000 (66.81%), error:      9644.75000, best error:      9631.60352 on epoch:    87, epoch:   89/100, mu: 0.9866479635, learning_rate: 0.0000018644, weight_decay: 0.0000057509

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.986829
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 1.72975e-06
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 5.4592e-06
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 90
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 87
read number_validation_images: 10000
read best_validation_predictions: 6681
read best_validation_error: 9631.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '278103492'
read generator: 278103492
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_187_5990_4_r533157395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.516478
read mu: 0.986829
read mu_delta: 0.945894
read initial_learning_rate: 0.00147427
read learning_rate: 1.72975e-06
read learning_rate_delta: 0.927765
read initial_weight_decay: 0.00059121
read weight_decay: 5.4592e-06
read weight_decay_delta: 0.949278
read batch_size: 84
read epsilon: 1e-07
read alpha: 0.0653186
read input_dropout_probability: 0.00131188
read hidden_dropout_probability: 0.102149
read velocity_reset: 1926
read epoch: 90
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 87
read number_validation_images: 10000
read best_validation_predictions: 6681
read best_validation_error: 9631.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5990
read normal distribution: '0 -0.566831350326538 -0.967898428440094'
generator_str: '278103492'
read generator: 278103492
read generated_by_map:
1 split_edge 1
reading 86 nodes.
reading 359 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1462.220703125s, input_fired_time: 20.3485813140869, output_fired_time: 9.98700714111328, propagate_forward_time: 557.123046875, propagate_backward_time: 861.363708496094, weight_update_time: 0.643992602825165, other_time: 13.3982543945313
epoch time: 115.93921661377s, input_fired_time: 1.55649614334106, output_fired_time: 0, propagate_forward_time: 112.082344055176, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.30037689208984
validation[          , genome  5990] predictions:    6661/  10000 (66.61%), best:    6681/10000 (66.81%), error:      9653.29785, best error:      9631.60352 on epoch:    87, epoch:   90/100, mu: 0.9868293405, learning_rate: 0.0000017297, weight_decay: 0.0000054592

epoch time: 1484.1768798828s, input_fired_time: 20.4976367950, output_fired_time: 10.1566057205, propagate_forward_time: 568.2470703125, propagate_backward_time: 871.7691650391, weight_update_time: 0.6306354403, other_time: 13.5063476563
epoch time: 119.1537780762s, input_fired_time: 1.5857267380, output_fired_time: 0.0000000000, propagate_forward_time: 115.1016387939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4664154053
validation[          , genome  5990] predictions:    6648/  10000 (66.48%), best:    6681/10000 (66.81%), error:      9674.45313, best error:      9631.60352 on epoch:    87, epoch:   91/100, mu: 0.9870008826, learning_rate: 0.0000016048, weight_decay: 0.0000051823

epoch time: 1510.8712158203s, input_fired_time: 20.8995285034, output_fired_time: 10.3537197113, propagate_forward_time: 575.3854370117, propagate_backward_time: 890.3068847656, weight_update_time: 0.6538262367, other_time: 13.9255981445
epoch time: 125.4140243530s, input_fired_time: 1.7019245625, output_fired_time: 0.0000000000, propagate_forward_time: 121.2970657349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4150314331
validation[          , genome  5990] predictions:    6680/  10000 (66.80%), best:    6680/10000 (66.80%), error:      9627.68848, best error:      9627.68848 on epoch:    92, epoch:   92/100, mu: 0.9871631265, learning_rate: 0.0000014889, weight_decay: 0.0000049194

epoch time: 1471.7269287109s, input_fired_time: 20.6752777100, output_fired_time: 9.9399499893, propagate_forward_time: 556.7315673828, propagate_backward_time: 870.8994750977, weight_update_time: 0.7031413913, other_time: 13.4806518555
epoch time: 125.5691452026s, input_fired_time: 1.6972985268, output_fired_time: 0.0000000000, propagate_forward_time: 121.3289947510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5428543091
validation[          , genome  5990] predictions:    6652/  10000 (66.52%), best:    6680/10000 (66.80%), error:      9639.02734, best error:      9627.68848 on epoch:    92, epoch:   93/100, mu: 0.9873166084, learning_rate: 0.0000013813, weight_decay: 0.0000046699

epoch time: 1508.3039550781s, input_fired_time: 21.1372013092, output_fired_time: 10.4184494019, propagate_forward_time: 573.0648803711, propagate_backward_time: 889.5525512695, weight_update_time: 0.6845148206, other_time: 14.1308593750
epoch time: 119.7985305786s, input_fired_time: 1.6162183285, output_fired_time: 0.0000000000, propagate_forward_time: 115.8509979248, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3313140869
validation[          , genome  5990] predictions:    6655/  10000 (66.55%), best:    6655/10000 (66.55%), error:      9621.87012, best error:      9621.87012 on epoch:    94, epoch:   94/100, mu: 0.9874618053, learning_rate: 0.0000012815, weight_decay: 0.0000044330

epoch time: 1478.1826171875s, input_fired_time: 20.7693786621, output_fired_time: 10.1253805161, propagate_forward_time: 561.7554931641, propagate_backward_time: 871.8309326172, weight_update_time: 0.6974673867, other_time: 13.7014160156
epoch time: 115.0110855103s, input_fired_time: 1.5465775728, output_fired_time: 0.0000000000, propagate_forward_time: 111.2210388184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2434692383
validation[          , genome  5990] predictions:    6673/  10000 (66.73%), best:    6673/10000 (66.73%), error:      9616.65625, best error:      9616.65625 on epoch:    95, epoch:   95/100, mu: 0.9875991344, learning_rate: 0.0000011890, weight_decay: 0.0000042082

epoch time: 1441.3798828125s, input_fired_time: 20.0256500244, output_fired_time: 9.7958259583, propagate_forward_time: 547.6649169922, propagate_backward_time: 850.6709594727, weight_update_time: 0.6553765535, other_time: 13.2225952148
epoch time: 116.7023544312s, input_fired_time: 1.5756896734, output_fired_time: 0.0000000000, propagate_forward_time: 112.7619476318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3647155762
validation[          , genome  5990] predictions:    6683/  10000 (66.83%), best:    6673/10000 (66.73%), error:      9626.13379, best error:      9616.65625 on epoch:    95, epoch:   96/100, mu: 0.9877290130, learning_rate: 0.0000011031, weight_decay: 0.0000039947

epoch time: 1447.7152099609s, input_fired_time: 20.2486190796, output_fired_time: 10.0255250931, propagate_forward_time: 549.0642089844, propagate_backward_time: 855.1663208008, weight_update_time: 0.6635605097, other_time: 13.2105102539
epoch time: 111.8356323242s, input_fired_time: 1.5107820034, output_fired_time: 0.0000000000, propagate_forward_time: 108.1394805908, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1853713989
validation[          , genome  5990] predictions:    6657/  10000 (66.57%), best:    6673/10000 (66.73%), error:      9624.55273, best error:      9616.65625 on epoch:    95, epoch:   97/100, mu: 0.9878519177, learning_rate: 0.0000010234, weight_decay: 0.0000037921

epoch time: 1420.8161621094s, input_fired_time: 19.9885444641, output_fired_time: 9.6374139786, propagate_forward_time: 539.3483886719, propagate_backward_time: 838.8759765625, weight_update_time: 0.6824759841, other_time: 12.9658203125
epoch time: 117.4282989502s, input_fired_time: 1.5800536871, output_fired_time: 0.0000000000, propagate_forward_time: 113.6127777100, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2354660034
validation[          , genome  5990] predictions:    6671/  10000 (66.71%), best:    6673/10000 (66.73%), error:      9617.22656, best error:      9616.65625 on epoch:    95, epoch:   98/100, mu: 0.9879681468, learning_rate: 0.0000009495, weight_decay: 0.0000035998

epoch time: 1382.6402587891s, input_fired_time: 19.4743213654, output_fired_time: 9.4701938629, propagate_forward_time: 519.2653808594, propagate_backward_time: 821.5578002930, weight_update_time: 0.6564284563, other_time: 12.8724975586
epoch time: 104.1139678955s, input_fired_time: 1.4490667582, output_fired_time: 0.0000000000, propagate_forward_time: 100.5286254883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1362762451
validation[          , genome  5990] predictions:    6700/  10000 (67.00%), best:    6700/10000 (67.00%), error:      9611.17188, best error:      9611.17188 on epoch:    99, epoch:   99/100, mu: 0.9880780578, learning_rate: 0.0000008809, weight_decay: 0.0000034172

epoch time: 1428.1185302734s, input_fired_time: 19.8438911438, output_fired_time: 9.8227643967, propagate_forward_time: 544.0440063477, propagate_backward_time: 841.3009643555, weight_update_time: 0.7075903416, other_time: 13.1069335938
epoch time: 115.1436767578s, input_fired_time: 1.5530537367, output_fired_time: 0.0000000000, propagate_forward_time: 111.2968978882, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2937240601
validation[          , genome  5990] predictions:    6675/  10000 (66.75%), best:    6675/10000 (66.75%), error:      9601.17969, best error:      9601.17969 on epoch:   100, epoch:  100/100, mu: 0.9881820679, learning_rate: 0.0000008173, weight_decay: 0.0000032439

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 573.2712402344s, input_fired_time: 7.9066452980, output_fired_time: 0.0000000000, propagate_forward_time: 553.9197387695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.4448852539
best training[          , genome  5990] predictions:   36225/  50000 (72.45%), best:    6675/10000 (66.75%), error:     41938.92188, best error:      9601.17969 on epoch:   100, epoch:  101/100, mu: 0.9882804155, learning_rate: 0.0000007582, weight_decay: 0.0000030793
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 115.0144195557s, input_fired_time: 1.5766283274, output_fired_time: 0.0000000000, propagate_forward_time: 111.1605682373, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2772216797
   testing[          , genome  5990] predictions:    6675/  10000 (66.75%), best:    6675/10000 (66.75%), error:      9601.17969, best error:      9601.17969 on epoch:   100, epoch:  101/100, mu: 0.9882804155, learning_rate: 0.0000007582, weight_decay: 0.0000030793
backpropagation finished successfully!
17:02:35 (116): called boinc_finish(0)

</stderr_txt>
]]>