Nameexact_genome_1542649503_169_6308_0
Workunit4098824
Created19 Nov 2018, 21:07:42 UTC
Sent19 Nov 2018, 21:07:45 UTC
Report deadline14 Dec 2018, 15:16:06 UTC
Received20 Nov 2018, 9:04:44 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID10687
Run time10 hours 24 min 50 sec
CPU time10 hours 2 min 26 sec
Validate stateValid
Credit6,188.95
Device peak FLOPS4.69 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size517.63 MB
Peak swap size516.16 MB
Peak disk usage11.74 MB

Stderr output

<core_client_version>7.4.42</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308_0_r385948415_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6308
read normal distribution: '1 0.108104564249516 0.197248354554176'
generator_str: '365516459'
read generator: 365516459
read generated_by_map:
1 add_edge 1
reading 98 nodes.
reading 381 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1431895993
post shuffle 1: 2130697795
epoch time: 1287.12829589844s, input_fired_time: 10.2189664840698, output_fired_time: 9.3926887512207, propagate_forward_time: 472.114135742188, propagate_backward_time: 782.451782226563, weight_update_time: 1.37239003181458, other_time: 12.95068359375
epoch time: 98.0278549194336s, input_fired_time: 1.14851605892181, output_fired_time: 0, propagate_forward_time: 94.6086654663086, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.27067565917969
validation[          , genome  6308] predictions:    4820/  10000 (48.20%), best:    4820/10000 (48.20%), error:     18754.27148, best error:     18754.27148 on epoch:     0, epoch:    0/25, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 1287.9833984375s, input_fired_time: 10.1103105545, output_fired_time: 8.5044651031, propagate_forward_time: 473.6124877930, propagate_backward_time: 783.3582153320, weight_update_time: 1.4153188467, other_time: 12.3978271484
epoch time: 98.5743865967s, input_fired_time: 1.1359149218, output_fired_time: 0.0000000000, propagate_forward_time: 95.1776657104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2608032227
validation[          , genome  6308] predictions:    5116/  10000 (51.16%), best:    5116/10000 (51.16%), error:     16458.94922, best error:     16458.94922 on epoch:     1, epoch:    1/25, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 1344.6655273438s, input_fired_time: 10.1100463867, output_fired_time: 8.3734569550, propagate_forward_time: 476.2755737305, propagate_backward_time: 836.7698974609, weight_update_time: 1.5189598799, other_time: 13.1365356445
epoch time: 98.4350814819s, input_fired_time: 1.1287560463, output_fired_time: 0.0000000000, propagate_forward_time: 95.0431518555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2631759644
validation[          , genome  6308] predictions:    4742/  10000 (47.42%), best:    5116/10000 (51.16%), error:     17811.59961, best error:     16458.94922 on epoch:     1, epoch:    2/25, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 1286.5844726563s, input_fired_time: 10.1429529190, output_fired_time: 8.3402271271, propagate_forward_time: 475.4578552246, propagate_backward_time: 779.5391845703, weight_update_time: 1.4499057531, other_time: 13.1042480469
epoch time: 98.7728118896s, input_fired_time: 1.1332025528, output_fired_time: 0.0000000000, propagate_forward_time: 95.3277740479, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3118362427
validation[          , genome  6308] predictions:    4957/  10000 (49.57%), best:    5116/10000 (51.16%), error:     16826.24609, best error:     16458.94922 on epoch:     1, epoch:    3/25, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 1283.7839355469s, input_fired_time: 9.9169654846, output_fired_time: 8.3364238739, propagate_forward_time: 468.9613037109, propagate_backward_time: 783.8151245117, weight_update_time: 1.4383625984, other_time: 12.7540893555
epoch time: 98.7265090942s, input_fired_time: 1.1830452681, output_fired_time: 0.0000000000, propagate_forward_time: 95.2440261841, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2994384766
validation[          , genome  6308] predictions:    5225/  10000 (52.25%), best:    5225/10000 (52.25%), error:     15259.49805, best error:     15259.49805 on epoch:     4, epoch:    4/25, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 1296.4267578125s, input_fired_time: 10.0264625549, output_fired_time: 8.4276504517, propagate_forward_time: 475.1939392090, propagate_backward_time: 790.1004638672, weight_update_time: 1.4108642340, other_time: 12.6782226563
epoch time: 97.5033111572s, input_fired_time: 1.1194219589, output_fired_time: 0.0000000000, propagate_forward_time: 94.0833663940, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3005218506
validation[          , genome  6308] predictions:    5169/  10000 (51.69%), best:    5225/10000 (52.25%), error:     16238.85547, best error:     15259.49805 on epoch:     4, epoch:    5/25, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 1277.0084228516s, input_fired_time: 9.9642887115, output_fired_time: 8.3175783157, propagate_forward_time: 469.6601257324, propagate_backward_time: 776.3673706055, weight_update_time: 1.4024920464, other_time: 12.6990356445
epoch time: 96.7965316772s, input_fired_time: 1.1154800653, output_fired_time: 0.0000000000, propagate_forward_time: 93.3874206543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2936325073
validation[          , genome  6308] predictions:    5469/  10000 (54.69%), best:    5469/10000 (54.69%), error:     14161.33203, best error:     14161.33203 on epoch:     6, epoch:    6/25, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 1265.9051513672s, input_fired_time: 9.8945131302, output_fired_time: 8.2253170013, propagate_forward_time: 466.5921630859, propagate_backward_time: 767.9641113281, weight_update_time: 1.4692618847, other_time: 13.2290039063
epoch time: 97.0308761597s, input_fired_time: 1.1074366570, output_fired_time: 0.0000000000, propagate_forward_time: 93.6421508789, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2812881470
validation[          , genome  6308] predictions:    5416/  10000 (54.16%), best:    5469/10000 (54.69%), error:     14241.91602, best error:     14161.33203 on epoch:     6, epoch:    7/25, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 1275.2618408203s, input_fired_time: 9.8842697144, output_fired_time: 8.3099966049, propagate_forward_time: 466.8093261719, propagate_backward_time: 777.8225097656, weight_update_time: 1.3730639219, other_time: 12.4357910156
epoch time: 96.7397155762s, input_fired_time: 1.1211520433, output_fired_time: 0.0000000000, propagate_forward_time: 93.3387527466, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2798080444
validation[          , genome  6308] predictions:    5369/  10000 (53.69%), best:    5469/10000 (54.69%), error:     14801.07422, best error:     14161.33203 on epoch:     6, epoch:    8/25, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 1288.9532470703s, input_fired_time: 10.0440387726, output_fired_time: 8.3951025009, propagate_forward_time: 471.8287658691, propagate_backward_time: 786.1337890625, weight_update_time: 1.4139537811, other_time: 12.5515136719
epoch time: 96.1451950073s, input_fired_time: 1.1042712927, output_fired_time: 0.0000000000, propagate_forward_time: 92.7661132813, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2748107910
validation[          , genome  6308] predictions:    5097/  10000 (50.97%), best:    5469/10000 (54.69%), error:     15885.43750, best error:     14161.33203 on epoch:     6, epoch:    9/25, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 1261.2324218750s, input_fired_time: 9.8417062759, output_fired_time: 8.1705951691, propagate_forward_time: 464.0147399902, propagate_backward_time: 766.0947265625, weight_update_time: 1.4574950933, other_time: 13.1105957031
epoch time: 95.4202194214s, input_fired_time: 1.1026486158, output_fired_time: 0.0000000000, propagate_forward_time: 92.0568923950, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2606811523
validation[          , genome  6308] predictions:    5181/  10000 (51.81%), best:    5469/10000 (54.69%), error:     15603.22168, best error:     14161.33203 on epoch:     6, epoch:   10/25, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 1275.7454833984s, input_fired_time: 9.9449081421, output_fired_time: 8.4075975418, propagate_forward_time: 466.8670349121, propagate_backward_time: 778.6729125977, weight_update_time: 1.3546686172, other_time: 11.8529663086
epoch time: 97.5151443481s, input_fired_time: 1.0719420910, output_fired_time: 0.0000000000, propagate_forward_time: 94.0937118530, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3494873047
validation[          , genome  6308] predictions:    5300/  10000 (53.00%), best:    5469/10000 (54.69%), error:     14587.20313, best error:     14161.33203 on epoch:     6, epoch:   11/25, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 1274.4251708984s, input_fired_time: 9.8123779297, output_fired_time: 8.2456016541, propagate_forward_time: 463.1910400391, propagate_backward_time: 776.3283691406, weight_update_time: 1.3462364674, other_time: 16.8477783203
epoch time: 95.8915710449s, input_fired_time: 1.1136529446, output_fired_time: 0.0000000000, propagate_forward_time: 92.5071640015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2707519531
validation[          , genome  6308] predictions:    5646/  10000 (56.46%), best:    5646/10000 (56.46%), error:     13674.54883, best error:     13674.54883 on epoch:    12, epoch:   12/25, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 1249.5533447266s, input_fired_time: 9.7568006516, output_fired_time: 8.0643301010, propagate_forward_time: 458.2958679199, propagate_backward_time: 760.5993041992, weight_update_time: 1.4412630796, other_time: 12.8369750977
epoch time: 94.6724777222s, input_fired_time: 1.0879831314, output_fired_time: 0.0000000000, propagate_forward_time: 91.3261032104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2583923340
validation[          , genome  6308] predictions:    5451/  10000 (54.51%), best:    5646/10000 (56.46%), error:     14490.35645, best error:     13674.54883 on epoch:    12, epoch:   13/25, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 1255.8077392578s, input_fired_time: 9.7559432983, output_fired_time: 8.0994529724, propagate_forward_time: 458.6936950684, propagate_backward_time: 767.0130004883, weight_update_time: 1.3613373041, other_time: 12.2455444336
epoch time: 95.7000656128s, input_fired_time: 1.1111485958, output_fired_time: 0.0000000000, propagate_forward_time: 92.3293380737, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2595825195
validation[          , genome  6308] predictions:    5499/  10000 (54.99%), best:    5646/10000 (56.46%), error:     13679.59766, best error:     13674.54883 on epoch:    12, epoch:   14/25, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 1264.8505859375s, input_fired_time: 9.8478279114, output_fired_time: 8.1250238419, propagate_forward_time: 461.4497680664, propagate_backward_time: 772.3990478516, weight_update_time: 1.4659270048, other_time: 13.0289916992
epoch time: 95.1874084473s, input_fired_time: 1.1046012640, output_fired_time: 0.0000000000, propagate_forward_time: 91.8488540649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2339553833
validation[          , genome  6308] predictions:    5350/  10000 (53.50%), best:    5646/10000 (56.46%), error:     14316.09375, best error:     13674.54883 on epoch:    12, epoch:   15/25, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 1260.6236572266s, input_fired_time: 9.8750944138, output_fired_time: 8.1083812714, propagate_forward_time: 460.4373474121, propagate_backward_time: 769.2866210938, weight_update_time: 1.4477754831, other_time: 12.9161376953
epoch time: 95.4699935913s, input_fired_time: 1.1090322733, output_fired_time: 0.0000000000, propagate_forward_time: 92.1186599731, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2423019409
validation[          , genome  6308] predictions:    5373/  10000 (53.73%), best:    5646/10000 (56.46%), error:     14717.02344, best error:     13674.54883 on epoch:    12, epoch:   16/25, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 1278.1795654297s, input_fired_time: 10.0015840530, output_fired_time: 8.3571090698, propagate_forward_time: 469.8632202148, propagate_backward_time: 777.5073242188, weight_update_time: 1.3640537262, other_time: 12.4503784180
epoch time: 96.2136154175s, input_fired_time: 1.1120289564, output_fired_time: 0.0000000000, propagate_forward_time: 92.6376037598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4639816284
validation[          , genome  6308] predictions:    5338/  10000 (53.38%), best:    5646/10000 (56.46%), error:     14008.25195, best error:     13674.54883 on epoch:    12, epoch:   17/25, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 1293.8500976563s, input_fired_time: 10.0902318954, output_fired_time: 8.3666582108, propagate_forward_time: 470.9148559570, propagate_backward_time: 792.2955932617, weight_update_time: 1.4036958218, other_time: 12.1827392578
epoch time: 98.3354415894s, input_fired_time: 1.1281454563, output_fired_time: 0.0000000000, propagate_forward_time: 94.9143142700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2929840088
validation[          , genome  6308] predictions:    5638/  10000 (56.38%), best:    5638/10000 (56.38%), error:     13069.08398, best error:     13069.08398 on epoch:    18, epoch:   18/25, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308_0_r385948415_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.805097
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00471692
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000188677
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 19
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 18
read number_validation_images: 10000
read best_validation_predictions: 5638
read best_validation_error: 13069.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6308
read normal distribution: '1 0.108104564249516 0.197248354554176'
generator_str: '847969076'
read generator: 847969076
read generated_by_map:
1 add_edge 1
reading 98 nodes.
reading 381 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308_0_r385948415_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.805097
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00471692
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000188677
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 19
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 18
read number_validation_images: 10000
read best_validation_predictions: 5638
read best_validation_error: 13069.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6308
read normal distribution: '1 0.108104564249516 0.197248354554176'
generator_str: '847969076'
read generator: 847969076
read generated_by_map:
1 add_edge 1
reading 98 nodes.
reading 381 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1519.44128417969s, input_fired_time: 11.7710943222046, output_fired_time: 9.7169075012207, propagate_forward_time: 563.6552734375, propagate_backward_time: 919.296325683594, weight_update_time: 1.62254297733307, other_time: 15.0016479492188
epoch time: 112.111404418945s, input_fired_time: 1.29446911811829, output_fired_time: 0, propagate_forward_time: 108.083992004395, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.73294067382813
validation[          , genome  6308] predictions:    5558/  10000 (55.58%), best:    5638/10000 (56.38%), error:     13725.06738, best error:     13069.08398 on epoch:    18, epoch:   19/25, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308_0_r385948415_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.814342
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00448107
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000179243
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 20
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 18
read number_validation_images: 10000
read best_validation_predictions: 5638
read best_validation_error: 13069.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6308
read normal distribution: '1 0.108104564249516 0.197248354554176'
generator_str: '151115369'
read generator: 151115369
read generated_by_map:
1 add_edge 1
reading 98 nodes.
reading 381 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1537.68286132813s, input_fired_time: 11.9223804473877, output_fired_time: 9.7302713394165, propagate_forward_time: 555.996337890625, propagate_backward_time: 945.068725585938, weight_update_time: 1.57310724258423, other_time: 14.9652099609375
epoch time: 123.484741210938s, input_fired_time: 1.42248344421387, output_fired_time: 0, propagate_forward_time: 119.141647338867, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.92060852050781
validation[          , genome  6308] predictions:    5697/  10000 (56.97%), best:    5638/10000 (56.38%), error:     14269.84570, best error:     13069.08398 on epoch:    18, epoch:   20/25, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 1536.9655761719s, input_fired_time: 11.9298782349, output_fired_time: 9.6690855026, propagate_forward_time: 551.0223388672, propagate_backward_time: 949.6158447266, weight_update_time: 1.5556266308, other_time: 14.7283935547
epoch time: 117.0233688354s, input_fired_time: 1.3990993500, output_fired_time: 0.0000000000, propagate_forward_time: 112.8880233765, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7362442017
validation[          , genome  6308] predictions:    5535/  10000 (55.35%), best:    5638/10000 (56.38%), error:     13395.13281, best error:     13069.08398 on epoch:    18, epoch:   21/25, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 1511.3094482422s, input_fired_time: 11.8662586212, output_fired_time: 9.6091451645, propagate_forward_time: 559.7355346680, propagate_backward_time: 914.6449584961, weight_update_time: 1.6375428438, other_time: 15.4536132813
epoch time: 124.2680358887s, input_fired_time: 1.4912412167, output_fired_time: 0.0000000000, propagate_forward_time: 119.8129425049, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9638519287
validation[          , genome  6308] predictions:    5838/  10000 (58.38%), best:    5838/10000 (58.38%), error:     12702.14551, best error:     12702.14551 on epoch:    22, epoch:   22/25, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1389.8723144531s, input_fired_time: 10.9638862610, output_fired_time: 8.9498929977, propagate_forward_time: 508.9533691406, propagate_backward_time: 846.1797485352, weight_update_time: 1.6203103065, other_time: 14.8253784180
epoch time: 119.6483688354s, input_fired_time: 1.4505497217, output_fired_time: 0.0000000000, propagate_forward_time: 115.2623443604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9354782104
validation[          , genome  6308] predictions:    5480/  10000 (54.80%), best:    5838/10000 (58.38%), error:     13978.62500, best error:     12702.14551 on epoch:    22, epoch:   23/25, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_169_6308_0_r385948415_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.846925
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00364986
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000145994
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 24
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 22
read number_validation_images: 10000
read best_validation_predictions: 5838
read best_validation_error: 12702.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6308
read normal distribution: '1 0.108104564249516 0.197248354554176'
generator_str: '1077654715'
read generator: 1077654715
read generated_by_map:
1 add_edge 1
reading 98 nodes.
reading 381 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1270.92431640625s, input_fired_time: 10.0328855514526, output_fired_time: 8.16316223144531, propagate_forward_time: 464.181823730469, propagate_backward_time: 775.514831542969, weight_update_time: 1.44806015491486, other_time: 13.0316162109375
epoch time: 98.2692642211914s, input_fired_time: 1.16539871692657, output_fired_time: 0, propagate_forward_time: 94.7869567871094, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.31690979003906
validation[          , genome  6308] predictions:    5540/  10000 (55.40%), best:    5838/10000 (58.38%), error:     13982.47266, best error:     12702.14551 on epoch:    22, epoch:   24/25, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 1281.6409912109s, input_fired_time: 10.1785678864, output_fired_time: 8.2192268372, propagate_forward_time: 467.5570678711, propagate_backward_time: 782.5717773438, weight_update_time: 1.4607045650, other_time: 13.1143188477
epoch time: 95.7761001587s, input_fired_time: 1.1548086405, output_fired_time: 0.0000000000, propagate_forward_time: 92.3816909790, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2396011353
validation[          , genome  6308] predictions:    5792/  10000 (57.92%), best:    5792/10000 (57.92%), error:     12589.49707, best error:     12589.49707 on epoch:    25, epoch:   25/25, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 481.9628295898s, input_fired_time: 5.8025469780, output_fired_time: 0.0000000000, propagate_forward_time: 464.8036193848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.3566589355
best training[          , genome  6308] predictions:   33310/  50000 (66.62%), best:    5792/10000 (57.92%), error:     48357.46094, best error:     12589.49707 on epoch:    25, epoch:   26/25, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 95.9101333618s, input_fired_time: 1.1575368643, output_fired_time: 0.0000000000, propagate_forward_time: 92.4836044312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2689895630
   testing[          , genome  6308] predictions:    5792/  10000 (57.92%), best:    5792/10000 (57.92%), error:     12589.49707, best error:     12589.49707 on epoch:    25, epoch:   26/25, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600
backpropagation finished successfully!
19:33:35 (6628): called boinc_finish(0)

</stderr_txt>
]]>