Nameexact_genome_1542531604_175_6272_1
Workunit4097850
Created18 Nov 2018, 17:46:14 UTC
Sent18 Nov 2018, 18:19:14 UTC
Report deadline10 Jan 2019, 10:19:29 UTC
Received21 Nov 2018, 14:11:29 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID58206
Run time23 hours 12 min 6 sec
CPU time22 hours 32 min 44 sec
Validate stateValid
Credit13,166.71
Device peak FLOPS2.94 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33

Stderr output

<core_client_version>6.10.58</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272_1_r208679257_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6272
read normal distribution: '1 1.21127879619598 -0.519413530826569'
generator_str: '2009028439'
read generator: 2009028439
read generated_by_map:
1 add_edge 1
reading 90 nodes.
reading 324 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 855592492
post shuffle 1: 2061188556
epoch time: 1853.14636230469s, input_fired_time: 12.5771350860596, output_fired_time: 8.99769687652588, propagate_forward_time: 803.117553710938, propagate_backward_time: 997.036437988281, weight_update_time: 5.07459735870361, other_time: 31.4175415039062
epoch time: 166.984893798828s, input_fired_time: 1.34268450737, output_fired_time: 0, propagate_forward_time: 158.529098510742, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.11311340332031
validation[          , genome  6272] predictions:    5161/  10000 (51.61%), best:    5161/10000 (51.61%), error:     16604.99414, best error:     16604.99414 on epoch:     0, epoch:    0/50, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 1859.2670898438s, input_fired_time: 11.5495281219, output_fired_time: 8.0662212372, propagate_forward_time: 814.3814697266, propagate_backward_time: 997.1023559570, weight_update_time: 3.9826509953, other_time: 28.1675415039
epoch time: 168.2332153320s, input_fired_time: 1.2534494400, output_fired_time: 0.0000000000, propagate_forward_time: 162.8167419434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1630249023
validation[          , genome  6272] predictions:    5037/  10000 (50.37%), best:    5161/10000 (51.61%), error:     16855.66016, best error:     16604.99414 on epoch:     0, epoch:    1/50, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 1865.1159667969s, input_fired_time: 13.5451860428, output_fired_time: 9.0422143936, propagate_forward_time: 818.8535766602, propagate_backward_time: 994.5260620117, weight_update_time: 4.9658064842, other_time: 29.1489257812
epoch time: 171.1503448486s, input_fired_time: 1.4621366262, output_fired_time: 0.0000000000, propagate_forward_time: 164.4939575195, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1942443848
validation[          , genome  6272] predictions:    5072/  10000 (50.72%), best:    5072/10000 (50.72%), error:     16445.55078, best error:     16445.55078 on epoch:     2, epoch:    2/50, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 1852.3765869141s, input_fired_time: 15.5088109970, output_fired_time: 7.9811029434, propagate_forward_time: 802.4595336914, propagate_backward_time: 990.1672973633, weight_update_time: 3.9886255264, other_time: 36.2599487305
epoch time: 170.0098266602s, input_fired_time: 1.2837008238, output_fired_time: 0.0000000000, propagate_forward_time: 162.5586853027, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1674346924
validation[          , genome  6272] predictions:    5352/  10000 (53.52%), best:    5352/10000 (53.52%), error:     15198.37695, best error:     15198.37695 on epoch:     3, epoch:    3/50, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 1865.0863037109s, input_fired_time: 13.5125722885, output_fired_time: 10.9533786774, propagate_forward_time: 817.7927246094, propagate_backward_time: 991.4349975586, weight_update_time: 3.0114316940, other_time: 31.3926391602
epoch time: 169.7385711670s, input_fired_time: 1.3202177286, output_fired_time: 0.0000000000, propagate_forward_time: 164.2522583008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1660919189
validation[          , genome  6272] predictions:    5373/  10000 (53.73%), best:    5352/10000 (53.52%), error:     15519.72461, best error:     15198.37695 on epoch:     3, epoch:    4/50, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 1862.5986328125s, input_fired_time: 11.4833917618, output_fired_time: 8.0228443146, propagate_forward_time: 804.3344116211, propagate_backward_time: 1009.0418090820, weight_update_time: 4.0148992538, other_time: 29.7162475586
epoch time: 165.3864288330s, input_fired_time: 1.2535604239, output_fired_time: 0.0000000000, propagate_forward_time: 157.9750366211, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1578369141
validation[          , genome  6272] predictions:    5171/  10000 (51.71%), best:    5352/10000 (53.52%), error:     16112.93066, best error:     15198.37695 on epoch:     3, epoch:    5/50, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 1866.1600341797s, input_fired_time: 13.4320602417, output_fired_time: 12.0136718750, propagate_forward_time: 810.9573974609, propagate_backward_time: 997.1169433594, weight_update_time: 4.9986681938, other_time: 32.6400146484
epoch time: 169.6625213623s, input_fired_time: 2.2894132137, output_fired_time: 0.0000000000, propagate_forward_time: 162.1635131836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2095947266
validation[          , genome  6272] predictions:    5431/  10000 (54.31%), best:    5431/10000 (54.31%), error:     14288.53711, best error:     14288.53711 on epoch:     6, epoch:    6/50, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 1857.5554199219s, input_fired_time: 16.5043067932, output_fired_time: 9.0177984238, propagate_forward_time: 810.0499267578, propagate_backward_time: 993.9055175781, weight_update_time: 3.9859347343, other_time: 28.0778808594
epoch time: 169.0812072754s, input_fired_time: 3.2925920486, output_fired_time: 0.0000000000, propagate_forward_time: 160.5699157715, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2187042236
validation[          , genome  6272] predictions:    5581/  10000 (55.81%), best:    5581/10000 (55.81%), error:     13535.06445, best error:     13535.06445 on epoch:     7, epoch:    7/50, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 1855.1778564453s, input_fired_time: 11.4957923889, output_fired_time: 10.3742761612, propagate_forward_time: 814.5167236328, propagate_backward_time: 991.6030883789, weight_update_time: 3.9768297672, other_time: 27.1879272461
epoch time: 165.8425292969s, input_fired_time: 1.2835006714, output_fired_time: 0.0000000000, propagate_forward_time: 160.3621368408, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1968841553
validation[          , genome  6272] predictions:    5442/  10000 (54.42%), best:    5581/10000 (55.81%), error:     14141.19727, best error:     13535.06445 on epoch:     7, epoch:    8/50, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 1856.9647216797s, input_fired_time: 13.9724960327, output_fired_time: 11.0242204666, propagate_forward_time: 814.1239013672, propagate_backward_time: 989.3440551758, weight_update_time: 3.0034937859, other_time: 28.5000610352
epoch time: 167.6893310547s, input_fired_time: 1.2765324116, output_fired_time: 0.0000000000, propagate_forward_time: 159.2744750977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1383209229
validation[          , genome  6272] predictions:    5369/  10000 (53.69%), best:    5581/10000 (55.81%), error:     15014.81836, best error:     13535.06445 on epoch:     7, epoch:    9/50, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 1858.1427001953s, input_fired_time: 14.9816579819, output_fired_time: 11.9533634186, propagate_forward_time: 810.6414794922, propagate_backward_time: 987.2501220703, weight_update_time: 5.0376367569, other_time: 33.3160400391
epoch time: 169.3478393555s, input_fired_time: 1.2982439995, output_fired_time: 0.0000000000, propagate_forward_time: 163.8215789795, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2280120850
validation[          , genome  6272] predictions:    5557/  10000 (55.57%), best:    5581/10000 (55.81%), error:     13930.28320, best error:     13535.06445 on epoch:     7, epoch:   10/50, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 1851.4793701172s, input_fired_time: 14.5369682312, output_fired_time: 11.9870014191, propagate_forward_time: 805.0101928711, propagate_backward_time: 989.6188964844, weight_update_time: 4.9818792343, other_time: 30.3262329102
epoch time: 170.8966674805s, input_fired_time: 1.3257085085, output_fired_time: 0.0000000000, propagate_forward_time: 164.3536224365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2173309326
validation[          , genome  6272] predictions:    5479/  10000 (54.79%), best:    5581/10000 (55.81%), error:     14539.17090, best error:     13535.06445 on epoch:     7, epoch:   11/50, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 1854.5159912109s, input_fired_time: 14.4943552017, output_fired_time: 10.9897356033, propagate_forward_time: 802.4914550781, propagate_backward_time: 989.1182250977, weight_update_time: 3.9890093803, other_time: 37.4221801758
epoch time: 165.8448181152s, input_fired_time: 1.2840141058, output_fired_time: 0.0000000000, propagate_forward_time: 155.4503173828, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.1104888916
validation[          , genome  6272] predictions:    5489/  10000 (54.89%), best:    5581/10000 (55.81%), error:     14272.39648, best error:     13535.06445 on epoch:     7, epoch:   12/50, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 1856.5172119141s, input_fired_time: 12.6289453506, output_fired_time: 10.0759773254, propagate_forward_time: 816.7073974609, propagate_backward_time: 989.5685424805, weight_update_time: 2.9985110760, other_time: 27.5364379883
epoch time: 182.3539733887s, input_fired_time: 1.3406044245, output_fired_time: 0.0000000000, propagate_forward_time: 175.6916809082, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3216857910
validation[          , genome  6272] predictions:    5753/  10000 (57.53%), best:    5753/10000 (57.53%), error:     12955.90625, best error:     12955.90625 on epoch:    13, epoch:   13/50, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 1852.1785888672s, input_fired_time: 14.6086091995, output_fired_time: 9.1080808640, propagate_forward_time: 815.0704345703, propagate_backward_time: 982.6628417969, weight_update_time: 3.9914448261, other_time: 30.7286376953
epoch time: 170.6875152588s, input_fired_time: 1.3158918619, output_fired_time: 0.0000000000, propagate_forward_time: 163.1518859863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.2197418213
validation[          , genome  6272] predictions:    5571/  10000 (55.71%), best:    5753/10000 (57.53%), error:     13900.36621, best error:     12955.90625 on epoch:    13, epoch:   14/50, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 1865.9877929688s, input_fired_time: 13.4736166000, output_fired_time: 8.9663238525, propagate_forward_time: 813.9102172852, propagate_backward_time: 1002.6931152344, weight_update_time: 2.9946999550, other_time: 26.9445800781
epoch time: 170.1895141602s, input_fired_time: 1.2749427557, output_fired_time: 0.0000000000, propagate_forward_time: 163.7268524170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1877136230
validation[          , genome  6272] predictions:    5397/  10000 (53.97%), best:    5753/10000 (57.53%), error:     14892.40039, best error:     12955.90625 on epoch:    13, epoch:   15/50, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 1861.9245605469s, input_fired_time: 12.5802249908, output_fired_time: 9.9651126862, propagate_forward_time: 825.2218627930, propagate_backward_time: 983.1835327148, weight_update_time: 3.9975519180, other_time: 30.9738769531
epoch time: 172.6971435547s, input_fired_time: 1.3196642399, output_fired_time: 0.0000000000, propagate_forward_time: 165.1692962646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.2081756592
validation[          , genome  6272] predictions:    5579/  10000 (55.79%), best:    5753/10000 (57.53%), error:     13469.60840, best error:     12955.90625 on epoch:    13, epoch:   16/50, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272_1_r208679257_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.785121
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0052265
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.00020906
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 17
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 13
read number_validation_images: 10000
read best_validation_predictions: 5753
read best_validation_error: 12955.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6272
read normal distribution: '1 1.21127879619598 -0.519413530826569'
generator_str: '1377764339'
read generator: 1377764339
read generated_by_map:
1 add_edge 1
reading 90 nodes.
reading 324 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1863.24877929688s, input_fired_time: 11.4886035919189, output_fired_time: 9.97056293487549, propagate_forward_time: 810.854919433594, propagate_backward_time: 997.729553222656, weight_update_time: 4.00234317779541, other_time: 33.2050170898438
epoch time: 168.678665161133s, input_fired_time: 1.2781400680542, output_fired_time: 0, propagate_forward_time: 160.182052612305, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.21847534179688
validation[          , genome  6272] predictions:    5559/  10000 (55.59%), best:    5753/10000 (57.53%), error:     13842.22070, best error:     12955.90625 on epoch:    13, epoch:   17/50, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 1857.8916015625s, input_fired_time: 12.5346288681, output_fired_time: 11.0077600479, propagate_forward_time: 807.9501953125, propagate_backward_time: 993.6083374023, weight_update_time: 3.0120239258, other_time: 32.7905883789
epoch time: 171.5214080811s, input_fired_time: 2.2774064541, output_fired_time: 0.0000000000, propagate_forward_time: 161.0754241943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.1685791016
validation[          , genome  6272] predictions:    5517/  10000 (55.17%), best:    5753/10000 (57.53%), error:     14274.64453, best error:     12955.90625 on epoch:    13, epoch:   18/50, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 1854.9207763672s, input_fired_time: 12.5990276337, output_fired_time: 9.0614967346, propagate_forward_time: 801.8345336914, propagate_backward_time: 998.6253051758, weight_update_time: 4.9836926460, other_time: 32.8004760742
epoch time: 166.3613128662s, input_fired_time: 2.2741310596, output_fired_time: 0.0000000000, propagate_forward_time: 159.8392639160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2479248047
validation[          , genome  6272] predictions:    5724/  10000 (57.24%), best:    5753/10000 (57.53%), error:     13236.58594, best error:     12955.90625 on epoch:    13, epoch:   19/50, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 1857.3602294922s, input_fired_time: 14.5662937164, output_fired_time: 8.9957981110, propagate_forward_time: 808.2696533203, propagate_backward_time: 995.4085083008, weight_update_time: 2.9741606712, other_time: 30.1199340820
epoch time: 168.5168609619s, input_fired_time: 3.2935879230, output_fired_time: 0.0000000000, propagate_forward_time: 160.9905548096, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2327117920
validation[          , genome  6272] predictions:    5167/  10000 (51.67%), best:    5753/10000 (57.53%), error:     15390.97363, best error:     12955.90625 on epoch:    13, epoch:   20/50, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 1864.8854980469s, input_fired_time: 14.5559244156, output_fired_time: 13.0813245773, propagate_forward_time: 811.9627075195, propagate_backward_time: 994.9076538086, weight_update_time: 4.0166368484, other_time: 30.3779907227
epoch time: 172.4320068359s, input_fired_time: 1.3080976009, output_fired_time: 0.0000000000, propagate_forward_time: 165.9031982422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2207183838
validation[          , genome  6272] predictions:    5761/  10000 (57.61%), best:    5753/10000 (57.53%), error:     13205.59668, best error:     12955.90625 on epoch:    13, epoch:   21/50, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 1869.9722900391s, input_fired_time: 13.6508073807, output_fired_time: 10.1668853760, propagate_forward_time: 819.1874389648, propagate_backward_time: 995.2772216797, weight_update_time: 3.0051686764, other_time: 31.6900634766
epoch time: 169.9637298584s, input_fired_time: 1.2981141806, output_fired_time: 0.0000000000, propagate_forward_time: 163.4010925293, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2645263672
validation[          , genome  6272] predictions:    5874/  10000 (58.74%), best:    5874/10000 (58.74%), error:     12615.67090, best error:     12615.67090 on epoch:    22, epoch:   22/50, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1848.2485351562s, input_fired_time: 12.7172775269, output_fired_time: 8.9783906937, propagate_forward_time: 815.3921508789, propagate_backward_time: 982.7359619141, weight_update_time: 3.9971060753, other_time: 28.4247436523
epoch time: 171.2149658203s, input_fired_time: 1.3022435904, output_fired_time: 0.0000000000, propagate_forward_time: 165.7560729980, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1566467285
validation[          , genome  6272] predictions:    5715/  10000 (57.15%), best:    5874/10000 (58.74%), error:     13094.31738, best error:     12615.67090 on epoch:    22, epoch:   23/50, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 1859.6666259766s, input_fired_time: 12.6748456955, output_fired_time: 10.3842830658, propagate_forward_time: 812.9110107422, propagate_backward_time: 995.3641967773, weight_update_time: 4.0132598877, other_time: 28.3323364258
epoch time: 169.8360748291s, input_fired_time: 1.2636860609, output_fired_time: 0.0000000000, propagate_forward_time: 164.4057769775, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1666107178
validation[          , genome  6272] predictions:    5500/  10000 (55.00%), best:    5874/10000 (58.74%), error:     14041.48633, best error:     12615.67090 on epoch:    22, epoch:   24/50, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 1860.9592285156s, input_fired_time: 10.7135457993, output_fired_time: 9.9486684799, propagate_forward_time: 809.3152465820, propagate_backward_time: 1000.6951293945, weight_update_time: 5.0399284363, other_time: 30.2868041992
arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_175_6272_1_r208679257_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.854079
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00346737
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.000138695
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 25
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 22
read number_validation_images: 10000
read best_validation_predictions: 5874
read best_validation_error: 12615.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6272
read normal distribution: '1 1.21127879619598 -0.519413530826569'
generator_str: '34110538'
read generator: 34110538
read generated_by_map:
1 add_edge 1
reading 90 nodes.
reading 324 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1857.55432128906s, input_fired_time: 14.7110567092896, output_fired_time: 10.9735584259033, propagate_forward_time: 818.256958007812, propagate_backward_time: 983.387390136719, weight_update_time: 3.97414565086365, other_time: 30.2254028320312
epoch time: 170.409439086914s, input_fired_time: 1.30841290950775, output_fired_time: 0, propagate_forward_time: 163.961959838867, propagate_backward_time: 0, weight_update_time: 0, other_time: 5.13906860351562
validation[          , genome  6272] predictions:    5683/  10000 (56.83%), best:    5874/10000 (58.74%), error:     13368.08496, best error:     12615.67090 on epoch:    22, epoch:   25/50, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 1867.8105468750s, input_fired_time: 13.6763315201, output_fired_time: 8.9912319183, propagate_forward_time: 821.4118041992, propagate_backward_time: 990.5415649414, weight_update_time: 3.0139067173, other_time: 33.1895751953
epoch time: 165.4037933350s, input_fired_time: 1.3080635071, output_fired_time: 0.0000000000, propagate_forward_time: 158.9508056641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1449279785
validation[          , genome  6272] predictions:    5931/  10000 (59.31%), best:    5931/10000 (59.31%), error:     12483.93848, best error:     12483.93848 on epoch:    26, epoch:   26/50, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 1854.5074462891s, input_fired_time: 12.7678775787, output_fired_time: 11.9955615997, propagate_forward_time: 809.0122070312, propagate_backward_time: 989.6284179688, weight_update_time: 2.9905097485, other_time: 31.1033935547
epoch time: 170.4531097412s, input_fired_time: 1.3304017782, output_fired_time: 0.0000000000, propagate_forward_time: 163.9205932617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2021179199
validation[          , genome  6272] predictions:    5872/  10000 (58.72%), best:    5931/10000 (59.31%), error:     13002.25098, best error:     12483.93848 on epoch:    26, epoch:   27/50, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 1863.9114990234s, input_fired_time: 11.7154388428, output_fired_time: 11.9632616043, propagate_forward_time: 817.4114990234, propagate_backward_time: 994.6265258789, weight_update_time: 2.9883220196, other_time: 28.1947631836
epoch time: 168.8117523193s, input_fired_time: 1.3132327795, output_fired_time: 0.0000000000, propagate_forward_time: 162.2473602295, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2511596680
validation[          , genome  6272] predictions:    5946/  10000 (59.46%), best:    5931/10000 (59.31%), error:     12656.87598, best error:     12483.93848 on epoch:    26, epoch:   28/50, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 1855.6983642578s, input_fired_time: 13.6800136566, output_fired_time: 8.9626359940, propagate_forward_time: 813.5224609375, propagate_backward_time: 993.4123535156, weight_update_time: 2.9849176407, other_time: 26.1208496094
epoch time: 166.2187652588s, input_fired_time: 1.2878354788, output_fired_time: 0.0000000000, propagate_forward_time: 160.8104400635, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1204833984
validation[          , genome  6272] predictions:    5779/  10000 (57.79%), best:    5931/10000 (59.31%), error:     12980.98730, best error:     12483.93848 on epoch:    26, epoch:   29/50, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1864.3625488281s, input_fired_time: 12.8530092239, output_fired_time: 8.9833049774, propagate_forward_time: 816.1218261719, propagate_backward_time: 996.1638793945, weight_update_time: 4.0074887276, other_time: 30.2405395508
epoch time: 167.1107025146s, input_fired_time: 2.3094308376, output_fired_time: 0.0000000000, propagate_forward_time: 158.6893463135, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1119232178
validation[          , genome  6272] predictions:    5927/  10000 (59.27%), best:    5931/10000 (59.31%), error:     12582.82910, best error:     12483.93848 on epoch:    26, epoch:   30/50, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 1852.1573486328s, input_fired_time: 13.7125129700, output_fired_time: 11.0095014572, propagate_forward_time: 802.7142944336, propagate_backward_time: 996.6030883789, weight_update_time: 3.9849648476, other_time: 28.1178588867
epoch time: 169.4290313721s, input_fired_time: 2.3366608620, output_fired_time: 0.0000000000, propagate_forward_time: 160.8885192871, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.2038574219
validation[          , genome  6272] predictions:    5808/  10000 (58.08%), best:    5931/10000 (59.31%), error:     12878.24023, best error:     12483.93848 on epoch:    26, epoch:   31/50, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 1860.6114501953s, input_fired_time: 14.7148332596, output_fired_time: 11.9608793259, propagate_forward_time: 829.8331909180, propagate_backward_time: 974.8522949219, weight_update_time: 3.0518310070, other_time: 29.2501831055
epoch time: 170.3513336182s, input_fired_time: 1.3475705385, output_fired_time: 0.0000000000, propagate_forward_time: 161.8545989990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1491699219
validation[          , genome  6272] predictions:    5963/  10000 (59.63%), best:    5963/10000 (59.63%), error:     12306.18555, best error:     12306.18555 on epoch:    32, epoch:   32/50, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 1859.6873779297s, input_fired_time: 12.7262964249, output_fired_time: 9.9142370224, propagate_forward_time: 807.9766845703, propagate_backward_time: 999.7636108398, weight_update_time: 4.9863891602, other_time: 29.3065795898
epoch time: 169.3092346191s, input_fired_time: 1.3260051012, output_fired_time: 0.0000000000, propagate_forward_time: 161.8408050537, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1424255371
validation[          , genome  6272] predictions:    6009/  10000 (60.09%), best:    5963/10000 (59.63%), error:     12370.53320, best error:     12306.18555 on epoch:    32, epoch:   33/50, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 1862.9356689453s, input_fired_time: 15.7927846909, output_fired_time: 10.0964994431, propagate_forward_time: 822.5367431641, propagate_backward_time: 985.3967895508, weight_update_time: 3.0078582764, other_time: 29.1128540039
epoch time: 165.4467468262s, input_fired_time: 1.3277114630, output_fired_time: 0.0000000000, propagate_forward_time: 157.9992218018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1198120117
validation[          , genome  6272] predictions:    6082/  10000 (60.82%), best:    6082/10000 (60.82%), error:     12273.09277, best error:     12273.09277 on epoch:    34, epoch:   34/50, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1848.6450195312s, input_fired_time: 11.7754602432, output_fired_time: 9.9470081329, propagate_forward_time: 806.7662963867, propagate_backward_time: 988.0408325195, weight_update_time: 4.9798784256, other_time: 32.1153564453
epoch time: 168.9160461426s, input_fired_time: 1.4013960361, output_fired_time: 0.0000000000, propagate_forward_time: 162.3170318604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1976165771
validation[          , genome  6272] predictions:    5750/  10000 (57.50%), best:    6082/10000 (60.82%), error:     13440.90137, best error:     12273.09277 on epoch:    34, epoch:   35/50, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1853.5610351562s, input_fired_time: 14.8096590042, output_fired_time: 10.0346097946, propagate_forward_time: 810.0510864258, propagate_backward_time: 982.4526977539, weight_update_time: 4.9947504997, other_time: 36.2128906250
epoch time: 173.7665252686s, input_fired_time: 1.3556973934, output_fired_time: 0.0000000000, propagate_forward_time: 168.2218627930, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1889648438
validation[          , genome  6272] predictions:    6049/  10000 (60.49%), best:    6082/10000 (60.82%), error:     12303.91699, best error:     12273.09277 on epoch:    34, epoch:   36/50, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1849.3142089844s, input_fired_time: 12.7411594391, output_fired_time: 10.9875755310, propagate_forward_time: 813.8462524414, propagate_backward_time: 979.5041503906, weight_update_time: 2.9925444126, other_time: 32.2350463867
epoch time: 164.9878540039s, input_fired_time: 1.3133307695, output_fired_time: 0.0000000000, propagate_forward_time: 158.5596466064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1148834229
validation[          , genome  6272] predictions:    5894/  10000 (58.94%), best:    6082/10000 (60.82%), error:     12899.97559, best error:     12273.09277 on epoch:    34, epoch:   37/50, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1861.0146484375s, input_fired_time: 12.8648328781, output_fired_time: 12.0380163193, propagate_forward_time: 816.7834472656, propagate_backward_time: 989.1119995117, weight_update_time: 4.9959383011, other_time: 30.2163696289
epoch time: 165.1728820801s, input_fired_time: 1.3300473690, output_fired_time: 0.0000000000, propagate_forward_time: 159.7214508057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1213836670
validation[          , genome  6272] predictions:    6016/  10000 (60.16%), best:    6016/10000 (60.16%), error:     12250.64355, best error:     12250.64355 on epoch:    38, epoch:   38/50, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 1853.1143798828s, input_fired_time: 12.7600288391, output_fired_time: 10.9760112762, propagate_forward_time: 808.1838989258, propagate_backward_time: 994.0700683594, weight_update_time: 2.9956686497, other_time: 27.1244506836
epoch time: 169.9870147705s, input_fired_time: 1.3736866713, output_fired_time: 0.0000000000, propagate_forward_time: 162.4303588867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1829681396
validation[          , genome  6272] predictions:    6053/  10000 (60.53%), best:    6053/10000 (60.53%), error:     12211.12500, best error:     12211.12500 on epoch:    39, epoch:   39/50, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1853.3967285156s, input_fired_time: 11.8530006409, output_fired_time: 13.2075033188, propagate_forward_time: 823.6807250977, propagate_backward_time: 976.3521118164, weight_update_time: 3.0119185448, other_time: 28.3033447266
epoch time: 168.9069519043s, input_fired_time: 1.3601788282, output_fired_time: 0.0000000000, propagate_forward_time: 162.2980346680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2487335205
validation[          , genome  6272] predictions:    6083/  10000 (60.83%), best:    6083/10000 (60.83%), error:     12199.79883, best error:     12199.79883 on epoch:    40, epoch:   40/50, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1850.5262451172s, input_fired_time: 12.8574991226, output_fired_time: 9.9484519958, propagate_forward_time: 808.4458007812, propagate_backward_time: 989.9385375977, weight_update_time: 2.9866530895, other_time: 29.3358764648
epoch time: 166.5321960449s, input_fired_time: 1.3948044777, output_fired_time: 0.0000000000, propagate_forward_time: 160.9820251465, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1553649902
validation[          , genome  6272] predictions:    5965/  10000 (59.65%), best:    6083/10000 (60.83%), error:     12629.19824, best error:     12199.79883 on epoch:    40, epoch:   41/50, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1856.1968994141s, input_fired_time: 16.1920890808, output_fired_time: 10.9499359131, propagate_forward_time: 813.3157958984, propagate_backward_time: 984.5497436523, weight_update_time: 3.9880769253, other_time: 31.1892700195
epoch time: 165.0987091064s, input_fired_time: 1.3244134188, output_fired_time: 0.0000000000, propagate_forward_time: 155.6812286377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.0930633545
validation[          , genome  6272] predictions:    6044/  10000 (60.44%), best:    6083/10000 (60.83%), error:     12255.32422, best error:     12199.79883 on epoch:    40, epoch:   42/50, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1860.9957275391s, input_fired_time: 10.8847236633, output_fired_time: 9.9358062744, propagate_forward_time: 800.2367553711, propagate_backward_time: 1008.7338867188, weight_update_time: 3.9773759842, other_time: 31.2045898438
epoch time: 171.1515502930s, input_fired_time: 1.3291753531, output_fired_time: 0.0000000000, propagate_forward_time: 164.5505981445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2717742920
validation[          , genome  6272] predictions:    5920/  10000 (59.20%), best:    6083/10000 (60.83%), error:     12572.75586, best error:     12199.79883 on epoch:    40, epoch:   43/50, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1854.1843261719s, input_fired_time: 10.8303098679, output_fired_time: 11.0188770294, propagate_forward_time: 816.7352905273, propagate_backward_time: 987.2890014648, weight_update_time: 3.9897682667, other_time: 28.3107910156
epoch time: 172.3466644287s, input_fired_time: 1.3725310564, output_fired_time: 0.0000000000, propagate_forward_time: 165.7748870850, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1992492676
validation[          , genome  6272] predictions:    6128/  10000 (61.28%), best:    6128/10000 (61.28%), error:     11927.92090, best error:     11927.92090 on epoch:    44, epoch:   44/50, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1849.0988769531s, input_fired_time: 14.2637672424, output_fired_time: 8.9145936966, propagate_forward_time: 800.8472290039, propagate_backward_time: 998.8619995117, weight_update_time: 4.0028862953, other_time: 26.2112426758
epoch time: 164.8686523438s, input_fired_time: 2.3152673244, output_fired_time: 0.0000000000, propagate_forward_time: 158.4476318359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1057586670
validation[          , genome  6272] predictions:    6042/  10000 (60.42%), best:    6128/10000 (61.28%), error:     12470.20898, best error:     11927.92090 on epoch:    44, epoch:   45/50, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1855.7717285156s, input_fired_time: 13.9007511139, output_fired_time: 10.9259338379, propagate_forward_time: 812.2195434570, propagate_backward_time: 986.4561767578, weight_update_time: 2.9885039330, other_time: 32.2693481445
epoch time: 166.8781433105s, input_fired_time: 1.3347768784, output_fired_time: 0.0000000000, propagate_forward_time: 158.4208068848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1225585938
validation[          , genome  6272] predictions:    6036/  10000 (60.36%), best:    6128/10000 (61.28%), error:     12155.72461, best error:     11927.92090 on epoch:    44, epoch:   46/50, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1850.5574951172s, input_fired_time: 11.8719587326, output_fired_time: 9.0423803329, propagate_forward_time: 817.1112670898, propagate_backward_time: 983.5277709961, weight_update_time: 4.0061779022, other_time: 29.0041503906
epoch time: 169.8868560791s, input_fired_time: 1.3531907797, output_fired_time: 0.0000000000, propagate_forward_time: 163.2718353271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2618255615
validation[          , genome  6272] predictions:    6079/  10000 (60.79%), best:    6128/10000 (61.28%), error:     12306.47266, best error:     11927.92090 on epoch:    44, epoch:   47/50, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1856.6243896484s, input_fired_time: 11.8766784668, output_fired_time: 8.0035371780, propagate_forward_time: 816.5679931641, propagate_backward_time: 991.1021728516, weight_update_time: 3.0680668354, other_time: 29.0739746094
epoch time: 168.5155181885s, input_fired_time: 1.4093717337, output_fired_time: 0.0000000000, propagate_forward_time: 162.9147949219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1913452148
validation[          , genome  6272] predictions:    6112/  10000 (61.12%), best:    6128/10000 (61.28%), error:     12283.98633, best error:     11927.92090 on epoch:    44, epoch:   48/50, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 1857.5115966797s, input_fired_time: 14.8723363876, output_fired_time: 10.3071117401, propagate_forward_time: 820.1797485352, propagate_backward_time: 981.8858642578, weight_update_time: 2.9864363670, other_time: 30.2665405273
epoch time: 165.6385040283s, input_fired_time: 1.3327368498, output_fired_time: 0.0000000000, propagate_forward_time: 159.1766662598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1291046143
validation[          , genome  6272] predictions:    6082/  10000 (60.82%), best:    6128/10000 (61.28%), error:     12162.89551, best error:     11927.92090 on epoch:    44, epoch:   49/50, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1866.9804687500s, input_fired_time: 16.8653354645, output_fired_time: 8.9831590652, propagate_forward_time: 802.9313964844, propagate_backward_time: 1009.2162475586, weight_update_time: 2.9837453365, other_time: 28.9843139648
epoch time: 166.1851654053s, input_fired_time: 1.3268963099, output_fired_time: 0.0000000000, propagate_forward_time: 160.7223205566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1359558105
validation[          , genome  6272] predictions:    6180/  10000 (61.80%), best:    6180/10000 (61.80%), error:     11821.51465, best error:     11821.51465 on epoch:    50, epoch:   50/50, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 843.6306762695s, input_fired_time: 10.7594509125, output_fired_time: 0.0000000000, propagate_forward_time: 806.0466918945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 26.8245239258
best training[          , genome  6272] predictions:   37931/  50000 (75.86%), best:    6180/10000 (61.80%), error:     35253.81641, best error:     11821.51465 on epoch:    50, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 165.8256835938s, input_fired_time: 1.3419777155, output_fired_time: 0.0000000000, propagate_forward_time: 160.3416595459, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1420440674
   testing[          , genome  6272] predictions:    6180/  10000 (61.80%), best:    6180/10000 (61.80%), error:     11821.51465, best error:     11821.51465 on epoch:    50, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
backpropagation finished successfully!
00:40:34 (2769): called boinc_finish(0)

</stderr_txt>
]]>