Nameexact_genome_1540823703_186_5453_0
Workunit4077937
Created29 Oct 2018, 18:02:31 UTC
Sent29 Oct 2018, 18:56:29 UTC
Report deadline17 Jan 2019, 6:37:59 UTC
Received22 Dec 2018, 8:50:10 UTC
Server stateOver
OutcomeComputation error
Client stateCompute error
Exit status194 (0xc2) EXIT_ABORTED_BY_CLIENT
Computer ID64195
Run time1 days 9 hours 3 min 12 sec
CPU time1 days 8 hours 6 min 3 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS3.80 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size539.03 MB
Peak swap size558.41 MB
Peak disk usage9.39 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<message>
finish file present too long</message>
<stderr_txt>
514160156s, input_fired_time: 33.2665748596, output_fired_time: 21.1965045929, propagate_forward_time: 605.3495483398, propagate_backward_time: 964.5227050781, weight_update_time: 1.8655073643, other_time: 49.6160278320
epoch time: 126.1184997559s, input_fired_time: 4.0556755066, output_fired_time: 0.0000000000, propagate_forward_time: 114.2193450928, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.8434753418
validation[          , genome  5453] predictions:    6185/  10000 (61.85%), best:    6163/10000 (61.63%), error:     11018.15234, best error:     10964.51270 on epoch:    49, epoch:   50/100, mu: 0.9451671839, learning_rate: 0.0000895237, weight_decay: 0.0000427437

epoch time: 1573.2895507813s, input_fired_time: 32.9993858337, output_fired_time: 20.6749382019, propagate_forward_time: 563.5199584961, propagate_backward_time: 904.1125488281, weight_update_time: 1.7214536667, other_time: 51.9827270508
epoch time: 125.8044891357s, input_fired_time: 3.9840118885, output_fired_time: 0.0000000000, propagate_forward_time: 112.9088287354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.9116516113
validation[          , genome  5453] predictions:    6161/  10000 (61.61%), best:    6163/10000 (61.63%), error:     11067.62695, best error:     10964.51270 on epoch:    49, epoch:   51/100, mu: 0.9472198486, learning_rate: 0.0000837075, weight_decay: 0.0000405169

epoch time: 1557.9436035156s, input_fired_time: 30.8740901947, output_fired_time: 18.9083080292, propagate_forward_time: 561.8375244141, propagate_backward_time: 899.5281372070, weight_update_time: 1.5854727030, other_time: 46.7954711914
epoch time: 131.6580047607s, input_fired_time: 4.1144485474, output_fired_time: 0.0000000000, propagate_forward_time: 117.6414337158, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.9021224976
validation[          , genome  5453] predictions:    6127/  10000 (61.27%), best:    6163/10000 (61.63%), error:     11066.22168, best error:     10964.51270 on epoch:    49, epoch:   52/100, mu: 0.9491785169, learning_rate: 0.0000782691, weight_decay: 0.0000384062

epoch time: 1645.5367431641s, input_fired_time: 32.2851371765, output_fired_time: 19.6477947235, propagate_forward_time: 593.7929687500, propagate_backward_time: 950.4710693359, weight_update_time: 1.6412130594, other_time: 49.3397216797
epoch time: 130.4121246338s, input_fired_time: 3.8251183033, output_fired_time: 0.0000000000, propagate_forward_time: 117.9513320923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.6356735229
validation[          , genome  5453] predictions:    6159/  10000 (61.59%), best:    6163/10000 (61.63%), error:     11011.83398, best error:     10964.51270 on epoch:    49, epoch:   53/100, mu: 0.9510475397, learning_rate: 0.0000731840, weight_decay: 0.0000364055

epoch time: 1563.7276611328s, input_fired_time: 29.7270431519, output_fired_time: 18.0608444214, propagate_forward_time: 567.1240234375, propagate_backward_time: 904.5305175781, weight_update_time: 1.5863865614, other_time: 44.2852783203
epoch time: 117.8282241821s, input_fired_time: 2.4046986103, output_fired_time: 0.0000000000, propagate_forward_time: 109.2199401855, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.2035827637
validation[          , genome  5453] predictions:    6178/  10000 (61.78%), best:    6163/10000 (61.63%), error:     11069.66309, best error:     10964.51270 on epoch:    49, epoch:   54/100, mu: 0.9528309703, learning_rate: 0.0000684293, weight_decay: 0.0000345090

epoch time: 1397.7314453125s, input_fired_time: 16.0260524750, output_fired_time: 7.5825033188, propagate_forward_time: 524.6769409180, propagate_backward_time: 832.2526245117, weight_update_time: 0.6926352978, other_time: 17.1933593750
epoch time: 110.2719650269s, input_fired_time: 1.2418549061, output_fired_time: 0.0000000000, propagate_forward_time: 106.1901702881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8399429321
validation[          , genome  5453] predictions:    6214/  10000 (62.14%), best:    6214/10000 (62.14%), error:     10962.79199, best error:     10962.79199 on epoch:    55, epoch:   55/100, mu: 0.9545327425, learning_rate: 0.0000639835, weight_decay: 0.0000327112

epoch time: 1404.2095947266s, input_fired_time: 15.8187675476, output_fired_time: 7.4978518486, propagate_forward_time: 526.5911254883, propagate_backward_time: 837.6270141602, weight_update_time: 0.6834087372, other_time: 16.6749267578
epoch time: 109.8338928223s, input_fired_time: 1.2338349819, output_fired_time: 0.0000000000, propagate_forward_time: 105.6266479492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9734115601
validation[          , genome  5453] predictions:    6287/  10000 (62.87%), best:    6287/10000 (62.87%), error:     10784.36426, best error:     10784.36426 on epoch:    56, epoch:   56/100, mu: 0.9561566114, learning_rate: 0.0000598266, weight_decay: 0.0000310072

epoch time: 1383.7510986328s, input_fired_time: 15.7154827118, output_fired_time: 7.3161039352, propagate_forward_time: 518.5632324219, propagate_backward_time: 825.7094116211, weight_update_time: 0.6777442694, other_time: 16.4468383789
epoch time: 105.2603530884s, input_fired_time: 1.1997625828, output_fired_time: 0.0000000000, propagate_forward_time: 101.1872940063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8732986450
validation[          , genome  5453] predictions:    6158/  10000 (61.58%), best:    6287/10000 (62.87%), error:     11025.98438, best error:     10784.36426 on epoch:    56, epoch:   57/100, mu: 0.9577061534, learning_rate: 0.0000559397, weight_decay: 0.0000293919

epoch time: 1387.0910644531s, input_fired_time: 15.5829095840, output_fired_time: 7.3123636246, propagate_forward_time: 521.3572998047, propagate_backward_time: 828.2719116211, weight_update_time: 0.6553023458, other_time: 14.5665893555
epoch time: 98.0946197510s, input_fired_time: 1.0837323666, output_fired_time: 0.0000000000, propagate_forward_time: 95.2287597656, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7821273804
validation[          , genome  5453] predictions:    6215/  10000 (62.15%), best:    6287/10000 (62.87%), error:     10933.37891, best error:     10784.36426 on epoch:    56, epoch:   58/100, mu: 0.9591847062, learning_rate: 0.0000523054, weight_decay: 0.0000278607

epoch time: 1266.4919433594s, input_fired_time: 14.6563873291, output_fired_time: 6.4246473312, propagate_forward_time: 475.5608825684, propagate_backward_time: 758.7312622070, weight_update_time: 0.5909842849, other_time: 11.1187133789
epoch time: 98.5917129517s, input_fired_time: 1.0870029926, output_fired_time: 0.0000000000, propagate_forward_time: 95.7464752197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7582321167
validation[          , genome  5453] predictions:    6229/  10000 (62.29%), best:    6287/10000 (62.87%), error:     10838.83301, best error:     10784.36426 on epoch:    56, epoch:   59/100, mu: 0.9605956078, learning_rate: 0.0000489071, weight_decay: 0.0000264093

epoch time: 1238.5230712891s, input_fired_time: 14.5652322769, output_fired_time: 6.2612733841, propagate_forward_time: 466.4533691406, propagate_backward_time: 740.2880249023, weight_update_time: 0.5823577642, other_time: 10.9552612305
epoch time: 95.8062133789s, input_fired_time: 1.0724481344, output_fired_time: 0.0000000000, propagate_forward_time: 92.9729003906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7608642578
validation[          , genome  5453] predictions:    6329/  10000 (63.29%), best:    6329/10000 (63.29%), error:     10704.87109, best error:     10704.87109 on epoch:    60, epoch:   60/100, mu: 0.9619418979, learning_rate: 0.0000457297, weight_decay: 0.0000250336

epoch time: 1261.3118896484s, input_fired_time: 14.6508846283, output_fired_time: 6.4304714203, propagate_forward_time: 473.8822631836, propagate_backward_time: 755.2682495117, weight_update_time: 0.5882554054, other_time: 11.0800781250
epoch time: 98.4836502075s, input_fired_time: 1.0966609716, output_fired_time: 0.0000000000, propagate_forward_time: 95.5670394897, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8199462891
validation[          , genome  5453] predictions:    6330/  10000 (63.30%), best:    6329/10000 (63.29%), error:     10723.50879, best error:     10704.87109 on epoch:    60, epoch:   61/100, mu: 0.9632265568, learning_rate: 0.0000427587, weight_decay: 0.0000237294

epoch time: 1300.2366943359s, input_fired_time: 14.8125629425, output_fired_time: 6.6059350967, propagate_forward_time: 489.6905822754, propagate_backward_time: 777.8968505859, weight_update_time: 0.5979415774, other_time: 11.2307128906
epoch time: 99.4988861084s, input_fired_time: 1.0988399982, output_fired_time: 0.0000000000, propagate_forward_time: 96.5625610352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8374862671
validation[          , genome  5453] predictions:    6187/  10000 (61.87%), best:    6329/10000 (63.29%), error:     10969.21973, best error:     10704.87109 on epoch:    60, epoch:   62/100, mu: 0.9644523859, learning_rate: 0.0000399807, weight_decay: 0.0000224933

epoch time: 1289.3391113281s, input_fired_time: 14.7582340240, output_fired_time: 6.5372629166, propagate_forward_time: 483.4913635254, propagate_backward_time: 773.2593994141, weight_update_time: 0.6031649113, other_time: 11.2929687500
epoch time: 102.3674011230s, input_fired_time: 1.1232795715, output_fired_time: 0.0000000000, propagate_forward_time: 99.3907928467, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8533325195
validation[          , genome  5453] predictions:    6259/  10000 (62.59%), best:    6329/10000 (63.29%), error:     10771.89648, best error:     10704.87109 on epoch:    60, epoch:   63/100, mu: 0.9656220675, learning_rate: 0.0000373832, weight_decay: 0.0000213215

epoch time: 1280.8986816406s, input_fired_time: 14.7576866150, output_fired_time: 6.5146555901, propagate_forward_time: 483.1251525879, propagate_backward_time: 765.3288574219, weight_update_time: 0.5959259868, other_time: 11.1723632813
epoch time: 101.7555541992s, input_fired_time: 1.1206723452, output_fired_time: 0.0000000000, propagate_forward_time: 98.8123092651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8225708008
validation[          , genome  5453] predictions:    6295/  10000 (62.95%), best:    6329/10000 (63.29%), error:     10748.98145, best error:     10704.87109 on epoch:    60, epoch:   64/100, mu: 0.9667382240, learning_rate: 0.0000349544, weight_decay: 0.0000202108

epoch time: 1254.7637939453s, input_fired_time: 14.6295042038, output_fired_time: 6.3771786690, propagate_forward_time: 470.6708679199, propagate_backward_time: 752.0608520508, weight_update_time: 0.5856469870, other_time: 11.0253295898
epoch time: 94.4921569824s, input_fired_time: 1.0616555214, output_fired_time: 0.0000000000, propagate_forward_time: 91.6951446533, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7353591919
validation[          , genome  5453] predictions:    6284/  10000 (62.84%), best:    6329/10000 (63.29%), error:     10760.26465, best error:     10704.87109 on epoch:    60, epoch:   65/100, mu: 0.9678032398, learning_rate: 0.0000326835, weight_decay: 0.0000191579

epoch time: 1213.5562744141s, input_fired_time: 14.5004148483, output_fired_time: 6.2025909424, propagate_forward_time: 452.9219055176, propagate_backward_time: 729.1141357422, weight_update_time: 0.5739227533, other_time: 10.8172607422
epoch time: 85.9310684204s, input_fired_time: 0.9967960715, output_fired_time: 0.0000000000, propagate_forward_time: 83.2911148071, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6431579590
validation[          , genome  5453] predictions:    6295/  10000 (62.95%), best:    6329/10000 (63.29%), error:     10722.84766, best error:     10704.87109 on epoch:    60, epoch:   66/100, mu: 0.9688194990, learning_rate: 0.0000305601, weight_decay: 0.0000181599

epoch time: 1203.8552246094s, input_fired_time: 14.4501695633, output_fired_time: 6.1316151619, propagate_forward_time: 452.0148925781, propagate_backward_time: 720.4343872070, weight_update_time: 0.5761095285, other_time: 10.8241577148
epoch time: 96.8808212280s, input_fired_time: 1.0808619261, output_fired_time: 0.0000000000, propagate_forward_time: 94.0326461792, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7673110962
validation[          , genome  5453] predictions:    6288/  10000 (62.88%), best:    6329/10000 (63.29%), error:     10757.72070, best error:     10704.87109 on epoch:    60, epoch:   67/100, mu: 0.9697892666, learning_rate: 0.0000285746, weight_decay: 0.0000172139

epoch time: 1211.9163818359s, input_fired_time: 14.5026502609, output_fired_time: 6.1932711601, propagate_forward_time: 450.9456787109, propagate_backward_time: 729.3287353516, weight_update_time: 0.5787292123, other_time: 10.9460449219
epoch time: 95.7616424561s, input_fired_time: 1.0784724951, output_fired_time: 0.0000000000, propagate_forward_time: 92.9198760986, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7632904053
validation[          , genome  5453] predictions:    6294/  10000 (62.94%), best:    6294/10000 (62.94%), error:     10685.69824, best error:     10685.69824 on epoch:    68, epoch:   68/100, mu: 0.9707146287, learning_rate: 0.0000267181, weight_decay: 0.0000163171

epoch time: 1205.4506835938s, input_fired_time: 14.4865360260, output_fired_time: 6.1557054520, propagate_forward_time: 448.1933288574, propagate_backward_time: 725.8614501953, weight_update_time: 0.5661882758, other_time: 10.7535400391
epoch time: 90.6686706543s, input_fired_time: 1.0349086523, output_fired_time: 0.0000000000, propagate_forward_time: 87.9559173584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6778411865
validation[          , genome  5453] predictions:    6331/  10000 (63.31%), best:    6331/10000 (63.31%), error:     10631.73242, best error:     10631.73242 on epoch:    69, epoch:   69/100, mu: 0.9715976119, learning_rate: 0.0000249823, weight_decay: 0.0000154671

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_186_5453.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_186_5453_0_r1692075883_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523036
read mu: 0.97244
read mu_delta: 0.954215
read initial_learning_rate: 0.00257417
read learning_rate: 2.33592e-05
read learning_rate_delta: 0.935031
read initial_weight_decay: 0.000620322
read weight_decay: 1.46613e-05
read weight_decay_delta: 0.947906
read batch_size: 99
read epsilon: 1e-07
read alpha: 0.0827559
read input_dropout_probability: 0.0011609
read hidden_dropout_probability: 0.114033
read velocity_reset: 1891
read epoch: 70
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 69
read number_validation_images: 10000
read best_validation_predictions: 6331
read best_validation_error: 10631.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5453
read normal distribution: '1 0.85310035943985 -0.00741988653317094'
generator_str: '1560956695'
read generator: 1560956695
read generated_by_map:
1 crossover 1
reading 66 nodes.
reading 245 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 974.4775390625s, input_fired_time: 13.6502418518066, output_fired_time: 5.31760120391846, propagate_forward_time: 356.845153808594, propagate_backward_time: 588.444946289063, weight_update_time: 0.512628734111786, other_time: 10.2195434570313
epoch time: 95.0168228149414s, input_fired_time: 1.11010420322418, output_fired_time: 0, propagate_forward_time: 91.9800262451172, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.92668914794922
validation[          , genome  5453] predictions:    6288/  10000 (62.88%), best:    6331/10000 (63.31%), error:     10764.62891, best error:     10631.73242 on epoch:    69, epoch:   70/100, mu: 0.9724401832, learning_rate: 0.0000233592, weight_decay: 0.0000146613

epoch time: 909.5560302734s, input_fired_time: 13.3180017471, output_fired_time: 5.0075106621, propagate_forward_time: 330.3183898926, propagate_backward_time: 551.0457153320, weight_update_time: 0.4933340847, other_time: 9.8663940430
epoch time: 98.0663146973s, input_fired_time: 1.1388480663, output_fired_time: 0.0000000000, propagate_forward_time: 94.9791488647, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9483184814
validation[          , genome  5453] predictions:    6304/  10000 (63.04%), best:    6331/10000 (63.31%), error:     10771.24023, best error:     10631.73242 on epoch:    69, epoch:   71/100, mu: 0.9732441306, learning_rate: 0.0000218416, weight_decay: 0.0000138976

epoch time: 963.4486694336s, input_fired_time: 13.5742082596, output_fired_time: 5.2808279991, propagate_forward_time: 352.8641662598, propagate_backward_time: 581.5853881836, weight_update_time: 0.5094152093, other_time: 10.1441040039
epoch time: 53.4513740540s, input_fired_time: 0.7358474731, output_fired_time: 0.0000000000, propagate_forward_time: 51.3742752075, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3412513733
validation[          , genome  5453] predictions:    6294/  10000 (62.94%), best:    6331/10000 (63.31%), error:     10654.19238, best error:     10631.73242 on epoch:    69, epoch:   72/100, mu: 0.9740113020, learning_rate: 0.0000204226, weight_decay: 0.0000131736

epoch time: 1000.2797241211s, input_fired_time: 13.7196779251, output_fired_time: 5.4475293159, propagate_forward_time: 368.1058349609, propagate_backward_time: 602.6649169922, weight_update_time: 0.5207087398, other_time: 10.3417968750
epoch time: 69.0606689453s, input_fired_time: 0.8881505132, output_fired_time: 0.0000000000, propagate_forward_time: 66.5511322021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6213836670
validation[          , genome  5453] predictions:    6340/  10000 (63.40%), best:    6340/10000 (63.40%), error:     10621.78320, best error:     10621.78320 on epoch:    73, epoch:   73/100, mu: 0.9747433662, learning_rate: 0.0000190957, weight_decay: 0.0000124873

epoch time: 946.1448974609s, input_fired_time: 13.4229164124, output_fired_time: 5.1920385361, propagate_forward_time: 345.3470458984, propagate_backward_time: 572.1897583008, weight_update_time: 0.5013364553, other_time: 9.9931640625
epoch time: 107.2193679810s, input_fired_time: 1.2184060812, output_fired_time: 0.0000000000, propagate_forward_time: 103.9324798584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0684814453
validation[          , genome  5453] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10570.07715, best error:     10570.07715 on epoch:    74, epoch:   74/100, mu: 0.9754418731, learning_rate: 0.0000178551, weight_decay: 0.0000118368

epoch time: 960.8192749023s, input_fired_time: 13.4896259308, output_fired_time: 5.2451004982, propagate_forward_time: 351.9889526367, propagate_backward_time: 579.9625854492, weight_update_time: 0.5074238777, other_time: 10.1329956055
epoch time: 67.4101486206s, input_fired_time: 0.8742582202, output_fired_time: 0.0000000000, propagate_forward_time: 64.9668807983, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5690078735
validation[          , genome  5453] predictions:    6313/  10000 (63.13%), best:    6336/10000 (63.36%), error:     10662.22852, best error:     10570.07715 on epoch:    74, epoch:   75/100, mu: 0.9761084318, learning_rate: 0.0000166951, weight_decay: 0.0000112202

epoch time: 1085.7165527344s, input_fired_time: 14.7342557907, output_fired_time: 6.0721540451, propagate_forward_time: 399.1839294434, propagate_backward_time: 654.7530517578, weight_update_time: 0.5436068773, other_time: 10.9731445313
epoch time: 191.8674316406s, input_fired_time: 2.2282028198, output_fired_time: 0.0000000000, propagate_forward_time: 186.0730133057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5662078857
validation[          , genome  5453] predictions:    6331/  10000 (63.31%), best:    6336/10000 (63.36%), error:     10684.96289, best error:     10570.07715 on epoch:    74, epoch:   76/100, mu: 0.9767444730, learning_rate: 0.0000156104, weight_decay: 0.0000106356

epoch time: 2275.7580566406s, input_fired_time: 25.1772212982, output_fired_time: 11.1335382462, propagate_forward_time: 863.7496948242, propagate_backward_time: 1356.2166748047, weight_update_time: 1.2200086117, other_time: 19.4808349609
epoch time: 192.5208892822s, input_fired_time: 2.0814943314, output_fired_time: 0.0000000000, propagate_forward_time: 187.5519714355, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8874206543
validation[          , genome  5453] predictions:    6369/  10000 (63.69%), best:    6369/10000 (63.69%), error:     10532.74902, best error:     10532.74902 on epoch:    77, epoch:   77/100, mu: 0.9773513675, learning_rate: 0.0000145962, weight_decay: 0.0000100816

epoch time: 2398.9892578125s, input_fired_time: 26.5931491852, output_fired_time: 12.1736869812, propagate_forward_time: 908.9436645508, propagate_backward_time: 1431.2028808594, weight_update_time: 0.9991600513, other_time: 20.0759277344
epoch time: 191.0516510010s, input_fired_time: 2.1299543381, output_fired_time: 0.0000000000, propagate_forward_time: 185.7080078125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2136840820
validation[          , genome  5453] predictions:    6369/  10000 (63.69%), best:    6369/10000 (63.69%), error:     10514.33887, best error:     10514.33887 on epoch:    78, epoch:   78/100, mu: 0.9779304862, learning_rate: 0.0000136479, weight_decay: 0.0000095564

epoch time: 1918.3897705078s, input_fired_time: 21.7960357666, output_fired_time: 10.0031280518, propagate_forward_time: 723.0062255859, propagate_backward_time: 1146.6627197266, weight_update_time: 0.8698409200, other_time: 16.9216308594
epoch time: 82.8524932861s, input_fired_time: 0.9873273373, output_fired_time: 0.0000000000, propagate_forward_time: 80.1783981323, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6867675781
validation[          , genome  5453] predictions:    6340/  10000 (63.40%), best:    6369/10000 (63.69%), error:     10590.67383, best error:     10514.33887 on epoch:    78, epoch:   79/100, mu: 0.9784830809, learning_rate: 0.0000127612, weight_decay: 0.0000090586

epoch time: 1027.6348876953s, input_fired_time: 13.9364080429, output_fired_time: 5.5840730667, propagate_forward_time: 379.4470520020, propagate_backward_time: 618.1130371094, weight_update_time: 0.5305541158, other_time: 10.5543212891
epoch time: 53.2806167603s, input_fired_time: 0.7382467389, output_fired_time: 0.0000000000, propagate_forward_time: 51.2009124756, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3414573669
validation[          , genome  5453] predictions:    6377/  10000 (63.77%), best:    6369/10000 (63.69%), error:     10578.20117, best error:     10514.33887 on epoch:    78, epoch:   80/100, mu: 0.9790104032, learning_rate: 0.0000119321, weight_decay: 0.0000085867

epoch time: 745.9480590820s, input_fired_time: 11.8226518631, output_fired_time: 4.1433176994, propagate_forward_time: 266.5288085938, propagate_backward_time: 454.9070739746, weight_update_time: 0.4259435534, other_time: 8.5462341309
epoch time: 46.5942115784s, input_fired_time: 0.6452886462, output_fired_time: 0.0000000000, propagate_forward_time: 44.7981147766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1508064270
validation[          , genome  5453] predictions:    6370/  10000 (63.70%), best:    6369/10000 (63.69%), error:     10548.90039, best error:     10514.33887 on epoch:    78, epoch:   81/100, mu: 0.9795135856, learning_rate: 0.0000111569, weight_decay: 0.0000081393

epoch time: 632.4276123047s, input_fired_time: 10.6787757874, output_fired_time: 3.5672585964, propagate_forward_time: 222.1908569336, propagate_backward_time: 388.3999328613, weight_update_time: 0.3746142387, other_time: 7.5907897949
epoch time: 46.4938201904s, input_fired_time: 0.6421636939, output_fired_time: 0.0000000000, propagate_forward_time: 44.6999359131, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1517219543
validation[          , genome  5453] predictions:    6344/  10000 (63.44%), best:    6369/10000 (63.69%), error:     10521.80469, best error:     10514.33887 on epoch:    78, epoch:   82/100, mu: 0.9799937010, learning_rate: 0.0000104321, weight_decay: 0.0000077153

epoch time: 631.9947509766s, input_fired_time: 10.6808719635, output_fired_time: 3.5645203590, propagate_forward_time: 222.0819091797, propagate_backward_time: 388.0436401367, weight_update_time: 0.3747994304, other_time: 7.6238403320
epoch time: 46.6256675720s, input_fired_time: 0.6436603665, output_fired_time: 0.0000000000, propagate_forward_time: 44.8316116333, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1503944397
validation[          , genome  5453] predictions:    6322/  10000 (63.22%), best:    6369/10000 (63.69%), error:     10588.90332, best error:     10514.33887 on epoch:    78, epoch:   83/100, mu: 0.9804518223, learning_rate: 0.0000097543, weight_decay: 0.0000073134

epoch time: 632.6713867188s, input_fired_time: 10.6906528473, output_fired_time: 3.5677192211, propagate_forward_time: 222.2534179688, propagate_backward_time: 388.5708923340, weight_update_time: 0.3754893243, other_time: 7.5886535645
epoch time: 46.6245956421s, input_fired_time: 0.6442002654, output_fired_time: 0.0000000000, propagate_forward_time: 44.8204383850, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1599578857
validation[          , genome  5453] predictions:    6344/  10000 (63.44%), best:    6369/10000 (63.69%), error:     10632.24805, best error:     10514.33887 on epoch:    78, epoch:   84/100, mu: 0.9808889627, learning_rate: 0.0000091206, weight_decay: 0.0000069324

epoch time: 632.0627441406s, input_fired_time: 10.6865854263, output_fired_time: 3.5662522316, propagate_forward_time: 222.1364746094, propagate_backward_time: 388.0682373047, weight_update_time: 0.3751839697, other_time: 7.6052246094
epoch time: 46.5333900452s, input_fired_time: 0.6423512101, output_fired_time: 0.0000000000, propagate_forward_time: 44.7373580933, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1536788940
validation[          , genome  5453] predictions:    6360/  10000 (63.60%), best:    6369/10000 (63.69%), error:     10546.30762, best error:     10514.33887 on epoch:    78, epoch:   85/100, mu: 0.9813061357, learning_rate: 0.0000085280, weight_decay: 0.0000065713

epoch time: 631.9461669922s, input_fired_time: 10.6790761948, output_fired_time: 3.5615396500, propagate_forward_time: 222.1257019043, propagate_backward_time: 387.9807128906, weight_update_time: 0.3759054840, other_time: 7.5991516113
epoch time: 46.5806846619s, input_fired_time: 0.6439875364, output_fired_time: 0.0000000000, propagate_forward_time: 44.7862129211, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1504859924
validation[          , genome  5453] predictions:    6387/  10000 (63.87%), best:    6387/10000 (63.87%), error:     10480.86230, best error:     10480.86230 on epoch:    86, epoch:   86/100, mu: 0.9817041755, learning_rate: 0.0000079740, weight_decay: 0.0000062290

epoch time: 631.9536132813s, input_fired_time: 10.6818122864, output_fired_time: 3.5642948151, propagate_forward_time: 222.0707550049, propagate_backward_time: 388.0414123535, weight_update_time: 0.3762257397, other_time: 7.5953674316
epoch time: 46.4293594360s, input_fired_time: 0.6417498589, output_fired_time: 0.0000000000, propagate_forward_time: 44.6445045471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1431045532
validation[          , genome  5453] predictions:    6365/  10000 (63.65%), best:    6387/10000 (63.87%), error:     10504.48633, best error:     10480.86230 on epoch:    86, epoch:   87/100, mu: 0.9820839763, learning_rate: 0.0000074559, weight_decay: 0.0000059045

epoch time: 632.4280395508s, input_fired_time: 10.6873331070, output_fired_time: 3.5674808025, propagate_forward_time: 222.1548767090, propagate_backward_time: 388.4060363770, weight_update_time: 0.3763086498, other_time: 7.6123046875
epoch time: 46.5577163696s, input_fired_time: 0.6459908485, output_fired_time: 0.0000000000, propagate_forward_time: 44.7615280151, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1501960754
validation[          , genome  5453] predictions:    6383/  10000 (63.83%), best:    6387/10000 (63.87%), error:     10482.09668, best error:     10480.86230 on epoch:    86, epoch:   88/100, mu: 0.9824464321, learning_rate: 0.0000069715, weight_decay: 0.0000055969

epoch time: 608.2277221680s, input_fired_time: 10.2644033432, output_fired_time: 3.4298489094, propagate_forward_time: 213.7631683350, propagate_backward_time: 373.6152954102, weight_update_time: 0.3586330712, other_time: 7.1549682617
epoch time: 42.7591743469s, input_fired_time: 0.5867737532, output_fired_time: 0.0000000000, propagate_forward_time: 41.1849403381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9874610901
validation[          , genome  5453] predictions:    6378/  10000 (63.78%), best:    6378/10000 (63.78%), error:     10473.92773, best error:     10473.92773 on epoch:    89, epoch:   89/100, mu: 0.9827922583, learning_rate: 0.0000065186, weight_decay: 0.0000053053

epoch time: 581.3769531250s, input_fired_time: 9.7776727676, output_fired_time: 3.2810308933, propagate_forward_time: 204.4730834961, propagate_backward_time: 357.4242248535, weight_update_time: 0.3359414637, other_time: 6.4209899902
epoch time: 42.7679290771s, input_fired_time: 0.5876656175, output_fired_time: 0.0000000000, propagate_forward_time: 41.1930007935, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9872627258
validation[          , genome  5453] predictions:    6389/  10000 (63.89%), best:    6389/10000 (63.89%), error:     10454.48145, best error:     10454.48145 on epoch:    90, epoch:   90/100, mu: 0.9831222892, learning_rate: 0.0000060951, weight_decay: 0.0000050289

epoch time: 581.4690551758s, input_fired_time: 9.7808046341, output_fired_time: 3.2786488533, propagate_forward_time: 204.5262603760, propagate_backward_time: 357.4676513672, weight_update_time: 0.3375227749, other_time: 6.4157104492
epoch time: 42.7677154541s, input_fired_time: 0.5864353180, output_fired_time: 0.0000000000, propagate_forward_time: 41.1942710876, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9870109558
validation[          , genome  5453] predictions:    6376/  10000 (63.76%), best:    6376/10000 (63.76%), error:     10444.28223, best error:     10444.28223 on epoch:    91, epoch:   91/100, mu: 0.9834371805, learning_rate: 0.0000056991, weight_decay: 0.0000047669

epoch time: 581.3798217773s, input_fired_time: 9.7819719315, output_fired_time: 3.2787601948, propagate_forward_time: 204.4416046143, propagate_backward_time: 357.4641418457, weight_update_time: 0.3364899457, other_time: 6.4133605957
epoch time: 42.7576980591s, input_fired_time: 0.5872602463, output_fired_time: 0.0000000000, propagate_forward_time: 41.1827812195, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9876556396
validation[          , genome  5453] predictions:    6404/  10000 (64.04%), best:    6376/10000 (63.76%), error:     10468.62891, best error:     10444.28223 on epoch:    91, epoch:   92/100, mu: 0.9837376475, learning_rate: 0.0000053288, weight_decay: 0.0000045186

epoch time: 576.3504028320s, input_fired_time: 9.6929960251, output_fired_time: 3.2519938946, propagate_forward_time: 202.8824768066, propagate_backward_time: 354.1634521484, weight_update_time: 0.3339136541, other_time: 6.3594665527
epoch time: 42.3787002563s, input_fired_time: 0.5822845697, output_fired_time: 0.0000000000, propagate_forward_time: 40.8209190369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9754981995
validation[          , genome  5453] predictions:    6388/  10000 (63.88%), best:    6376/10000 (63.76%), error:     10463.56543, best error:     10444.28223 on epoch:    91, epoch:   93/100, mu: 0.9840243459, learning_rate: 0.0000049826, weight_decay: 0.0000042832

epoch time: 575.5343627930s, input_fired_time: 9.6833925247, output_fired_time: 3.2495481968, propagate_forward_time: 202.6221923828, propagate_backward_time: 353.6179199219, weight_update_time: 0.3346209526, other_time: 6.3612670898
epoch time: 42.3693962097s, input_fired_time: 0.5818206072, output_fired_time: 0.0000000000, propagate_forward_time: 40.8130149841, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9745597839
validation[          , genome  5453] predictions:    6412/  10000 (64.12%), best:    6376/10000 (63.76%), error:     10476.62207, best error:     10444.28223 on epoch:    91, epoch:   94/100, mu: 0.9842979312, learning_rate: 0.0000046589, weight_decay: 0.0000040601

epoch time: 575.2359008789s, input_fired_time: 9.6771030426, output_fired_time: 3.2461230755, propagate_forward_time: 202.5276947021, propagate_backward_time: 353.4517211914, weight_update_time: 0.3331983387, other_time: 6.3332519531
epoch time: 42.3637962341s, input_fired_time: 0.5824616551, output_fired_time: 0.0000000000, propagate_forward_time: 40.8067550659, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9745788574
validation[          , genome  5453] predictions:    6386/  10000 (63.86%), best:    6376/10000 (63.76%), error:     10460.32227, best error:     10444.28223 on epoch:    91, epoch:   95/100, mu: 0.9845589995, learning_rate: 0.0000043562, weight_decay: 0.0000038486

epoch time: 575.3646850586s, input_fired_time: 9.6833209991, output_fired_time: 3.2470116615, propagate_forward_time: 202.5671997070, propagate_backward_time: 353.5244140625, weight_update_time: 0.3330717087, other_time: 6.3427124023
epoch time: 42.3695220947s, input_fired_time: 0.5824823380, output_fired_time: 0.0000000000, propagate_forward_time: 40.8104896545, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9765510559
validation[          , genome  5453] predictions:    6387/  10000 (63.87%), best:    6376/10000 (63.76%), error:     10454.70410, best error:     10444.28223 on epoch:    91, epoch:   96/100, mu: 0.9848080873, learning_rate: 0.0000040732, weight_decay: 0.0000036481

epoch time: 575.4588623047s, input_fired_time: 9.6828269958, output_fired_time: 3.2453024387, propagate_forward_time: 202.6241455078, propagate_backward_time: 353.5700988770, weight_update_time: 0.3334825933, other_time: 6.3365173340
epoch time: 42.3840026855s, input_fired_time: 0.5834712386, output_fired_time: 0.0000000000, propagate_forward_time: 40.8239059448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9766273499
validation[          , genome  5453] predictions:    6421/  10000 (64.21%), best:    6421/10000 (64.21%), error:     10434.46289, best error:     10434.46289 on epoch:    97, epoch:   97/100, mu: 0.9850457907, learning_rate: 0.0000038086, weight_decay: 0.0000034580

epoch time: 575.2998657227s, input_fired_time: 9.6800527573, output_fired_time: 3.2451121807, propagate_forward_time: 202.5585327148, propagate_backward_time: 353.4790344238, weight_update_time: 0.3332092166, other_time: 6.3371276855
epoch time: 42.3496437073s, input_fired_time: 0.5820616484, output_fired_time: 0.0000000000, propagate_forward_time: 40.7918701172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9757118225
validation[          , genome  5453] predictions:    6402/  10000 (64.02%), best:    6421/10000 (64.21%), error:     10437.90430, best error:     10434.46289 on epoch:    97, epoch:   98/100, mu: 0.9852726460, learning_rate: 0.0000035611, weight_decay: 0.0000032779

epoch time: 575.2481079102s, input_fired_time: 9.6798448563, output_fired_time: 3.2451624870, propagate_forward_time: 202.5474090576, propagate_backward_time: 353.4375610352, weight_update_time: 0.3333958089, other_time: 6.3380737305
epoch time: 42.3553428650s, input_fired_time: 0.5826528072, output_fired_time: 0.0000000000, propagate_forward_time: 40.7977027893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9749870300
validation[          , genome  5453] predictions:    6395/  10000 (63.95%), best:    6421/10000 (64.21%), error:     10468.06152, best error:     10434.46289 on epoch:    97, epoch:   99/100, mu: 0.9854890704, learning_rate: 0.0000033298, weight_decay: 0.0000031071

epoch time: 575.2289428711s, input_fired_time: 9.6781387329, output_fired_time: 3.2448925972, propagate_forward_time: 202.5241394043, propagate_backward_time: 353.4483337402, weight_update_time: 0.3326038420, other_time: 6.3334350586
epoch time: 42.3591079712s, input_fired_time: 0.5823505521, output_fired_time: 0.0000000000, propagate_forward_time: 40.8031196594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9736366272
validation[          , genome  5453] predictions:    6402/  10000 (64.02%), best:    6402/10000 (64.02%), error:     10430.77832, best error:     10430.77832 on epoch:   100, epoch:  100/100, mu: 0.9856956005, learning_rate: 0.0000031134, weight_decay: 0.0000029453

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 210.2522430420s, input_fired_time: 2.9033212662, output_fired_time: 0.0000000000, propagate_forward_time: 202.4459991455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9029235840
best training[          , genome  5453] predictions:   35050/  50000 (70.10%), best:    6402/10000 (64.02%), error:     45084.79688, best error:     10430.77832 on epoch:   100, epoch:  101/100, mu: 0.9858926535, learning_rate: 0.0000029111, weight_decay: 0.0000027918
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 42.3682823181s, input_fired_time: 0.5834740996, output_fired_time: 0.0000000000, propagate_forward_time: 40.8083992004, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9764099121
   testing[          , genome  5453] predictions:    6402/  10000 (64.02%), best:    6402/10000 (64.02%), error:     10430.77832, best error:     10430.77832 on epoch:   100, epoch:  101/100, mu: 0.9858926535, learning_rate: 0.0000029111, weight_decay: 0.0000027918
backpropagation finished successfully!
19:50:32 (43468): called boinc_finish(0)
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_186_5453.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_186_5453_0_r1692075883_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523036
read mu: 0.985893
read mu_delta: 0.954215
read initial_learning_rate: 0.00257417
read learning_rate: 2.91115e-06
read learning_rate_delta: 0.935031
read initial_weight_decay: 0.000620322
read weight_decay: 2.79184e-06
read weight_decay_delta: 0.947906
read batch_size: 99
read epsilon: 1e-07
read alpha: 0.0827559
read input_dropout_probability: 0.0011609
read hidden_dropout_probability: 0.114033
read velocity_reset: 1891
read epoch: 101
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 100
read number_validation_images: 10000
read best_validation_predictions: 6402
read best_validation_error: 10430.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5453
read normal distribution: '1 0.85310035943985 -0.00741988653317094'
generator_str: '2071534966'
read generator: 2071534966
read generated_by_map:
1 crossover 1
reading 66 nodes.
reading 245 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FF880C0C5D2

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 12/22/18 09:11:25
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\Program Files\BOINC\\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files\BOINC\\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\33;C:\ProgramData\BOINC\projects\csgrid.org_csg


ModLoad: 0000000006350000 00000000001da000 C:\ProgramData\BOINC\projects\csgrid.org_csg\exact_client_0.34_windows_x86_64.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : C:\Users\travi\Code\boinc_2015\win_build\Build\x64\Release\exact.pdb

ModLoad: 00000000841b0000 00000000001e1000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000081c20000 00000000000b2000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 10.0.17134.376 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.376

ModLoad: 0000000080b60000 0000000000273000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.17134.441) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 000000007d4c0000 000000000008b000 C:\WINDOWS\SYSTEM32\apphelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000083cb0000 0000000000190000 C:\WINDOWS\System32\USER32.dll (6.2.17134.376) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 00000000815f0000 0000000000020000 C:\WINDOWS\System32\win32u.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : win32u.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000083c70000 0000000000028000 C:\WINDOWS\System32\GDI32.dll (6.2.17134.285) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 10.0.17134.285 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.285

ModLoad: 0000000080770000 0000000000192000 C:\WINDOWS\System32\gdi32full.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32full.pdb
    File Version          : 10.0.17134.471 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.471

ModLoad: 00000000805d0000 000000000009f000 C:\WINDOWS\System32\msvcp_win.dll (6.2.17134.137) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcp_win.pdb
    File Version          : 10.0.17134.137 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.137

ModLoad: 0000000080670000 00000000000fa000 C:\WINDOWS\System32\ucrtbase.dll (6.2.17134.319) (-exported- Symbols Loaded)
    Linked PDB Filename   : ucrtbase.pdb
    File Version          : 10.0.17134.319 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.319

ModLoad: 0000000081ce0000 00000000000a1000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000083700000 000000000009e000 C:\WINDOWS\System32\msvcrt.dll (7.0.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.17134.1

ModLoad: 0000000081610000 000000000005b000 C:\WINDOWS\System32\sechost.dll (6.2.17134.319) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000083b30000 0000000000124000 C:\WINDOWS\System32\RPCRT4.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 00000000822c0000 0000000001440000 C:\WINDOWS\System32\SHELL32.dll (6.2.17134.441) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000080e90000 0000000000049000 C:\WINDOWS\System32\cfgmgr32.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000083fb0000 00000000000a9000 C:\WINDOWS\System32\shcore.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : shcore.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 00000000837a0000 0000000000322000 C:\WINDOWS\System32\combase.dll (6.2.17134.407) (-exported- Symbols Loaded)
    Linked PDB Filename   : combase.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000080550000 000000000007a000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000080ee0000 000000000070d000 C:\WINDOWS\System32\windows.storage.dll (6.2.17134.471) (-exported- Symbols Loaded)
    Linked PDB Filename   : Windows.Storage.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 00000000816a0000 0000000000051000 C:\WINDOWS\System32\shlwapi.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 00000000804e0000 0000000000011000 C:\WINDOWS\System32\kernel.appcore.dll (6.2.17134.112) (-exported- Symbols Loaded)
    Linked PDB Filename   : Kernel.Appcore.pdb
    File Version          : 10.0.17134.112 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.112

ModLoad: 0000000080510000 000000000001f000 C:\WINDOWS\System32\profapi.dll (6.2.17134.1) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : profapi.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000080490000 000000000004c000 C:\WINDOWS\System32\powrprof.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : powrprof.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000080500000 000000000000a000 C:\WINDOWS\System32\FLTLIB.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : fltLib.pdb
    File Version          : 10.0.17134.466 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Sistema operativo Microsoft&#174; Windows&#174;
    Product Version       : 10.0.17134.466

ModLoad: 0000000081670000 000000000002d000 C:\WINDOWS\System32\IMM32.DLL (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 0000000079530000 00000000001c9000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1

ModLoad: 000000007aa80000 000000000000a000 C:\WINDOWS\SYSTEM32\version.dll (6.2.17134.1) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.17134.1 (WinBuild.160101.0800)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.17134.1



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 54898, Write: 4949, Other 1145

- I/O Transfers Counters -
Read: 224840366, Write: 5118, Other 44810

- Paged Pool Usage -
QuotaPagedPoolUsage: 158936, QuotaPeakPagedPoolUsage: 159096
QuotaNonPagedPoolUsage: 13320, QuotaPeakNonPagedPoolUsage: 13320

- Virtual Memory Usage -
VirtualSize: 586166272, PeakVirtualSize: 724684800

- Pagefile Usage -
PagefileUsage: 586166272, PeakPagefileUsage: 586166272

- Working Set Size -
WorkingSetSize: 566226944, PeakWorkingSetSize: 566226944, PageFaultCount: 163973

*** Dump of thread ID 3236 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x00007FF880C0C5D2

- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=00000000064a9440 rdx=000000005dbff740 rsi=0000000000000000 rdi=0000000000000000
r8=000000005dbff740 r9=00000000064a9430 r10=0000000000000001 r11=0000000000000fff r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000080c0c5d2 rsp=000000005dbff718 rbp=0000000000000000
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
5dbff710 063de421 00000001 5dbff740 5dbff740 064a9430 KERNELBASE!DebugBreak+0x0 
5dbffb50 063df820 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
5dbffdb0 81c33034 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
5dbffde0 84223691 00000000 00000000 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
5dbffe30 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 

*** Dump of thread ID 32760 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Unknown, , Kernel Time: 31.000000, User Time: 0.000000, Wait Time: 3809055744.000000

- Registers -
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000000000000 rsp=0000000000000000 rbp=0000000000000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !+0x0 

*** Dump of thread ID 30710221 (state: Unknown): ***

- Information -
Status: Base Priority: Normal, Priority: Unknown, , Kernel Time: 17179869184.000000, User Time: 21474836480.000000, Wait Time: 0.000000

- Registers -
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=0000000000000000 rsp=0000000000000000 rbp=0000000000000000
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000

- Callstack -
ChildEBP RetAddr  Args to Child
(-nosymbols- PC == 0)
00000000 00000000 00000000 00000000 00000000 00000000 !+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>