Nameexact_genome_1542414603_174_6125_3
Workunit4094338
Created26 Dec 2018, 14:58:49 UTC
Sent26 Dec 2018, 14:59:25 UTC
Report deadline4 Feb 2019, 0:02:42 UTC
Received27 Dec 2018, 0:56:04 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID80138
Run time9 hours 56 min 17 sec
CPU time9 hours 54 min 16 sec
Validate stateValid
Credit9,844.32
Device peak FLOPS4.38 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size522.86 MB
Peak swap size542.54 MB
Peak disk usage9.50 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_174_6125.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_174_6125_3_r1405532660_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542414603_174_6125.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6125
read normal distribution: '1 0.586401402950287 -0.895910322666168'
generator_str: '1808694270'
read generator: 1808694270
read generated_by_map:
1 change_size_y 1
reading 88 nodes.
reading 302 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1093572605
post shuffle 1: 1134831251
epoch time: 668.392700195313s, input_fired_time: 6.19554090499878, output_fired_time: 4.24680185317993, propagate_forward_time: 189.637359619141, propagate_backward_time: 462.236389160156, weight_update_time: 0.724522650241852, other_time: 6.07656860351563
epoch time: 39.0148620605469s, input_fired_time: 0.60837310552597, output_fired_time: 0, propagate_forward_time: 37.2924652099609, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.1140251159668
validation[          , genome  6125] predictions:    4544/  10000 (45.44%), best:    4544/10000 (45.44%), error:     18480.95508, best error:     18480.95508 on epoch:     0, epoch:    0/50, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 658.8250732422s, input_fired_time: 5.9907474518, output_fired_time: 4.1779713631, propagate_forward_time: 187.0483703613, propagate_backward_time: 455.5857849121, weight_update_time: 0.7087174654, other_time: 6.0222167969
epoch time: 38.9202537537s, input_fired_time: 0.6004678607, output_fired_time: 0.0000000000, propagate_forward_time: 37.2517204285, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0680656433
validation[          , genome  6125] predictions:    4766/  10000 (47.66%), best:    4766/10000 (47.66%), error:     17138.69727, best error:     17138.69727 on epoch:     1, epoch:    1/50, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 658.1108398438s, input_fired_time: 5.9376688004, output_fired_time: 4.1758155823, propagate_forward_time: 186.8208007813, propagate_backward_time: 455.0841674805, weight_update_time: 0.7081431150, other_time: 6.0923461914
epoch time: 40.1244697571s, input_fired_time: 0.6260030866, output_fired_time: 0.0000000000, propagate_forward_time: 38.3736648560, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1248016357
validation[          , genome  6125] predictions:    4779/  10000 (47.79%), best:    4766/10000 (47.66%), error:     18021.30664, best error:     17138.69727 on epoch:     1, epoch:    2/50, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 657.2192993164s, input_fired_time: 5.9122233391, output_fired_time: 4.1600766182, propagate_forward_time: 186.5559082031, propagate_backward_time: 454.5863952637, weight_update_time: 0.7084814310, other_time: 6.0046691895
epoch time: 38.9352607727s, input_fired_time: 0.6008327603, output_fired_time: 0.0000000000, propagate_forward_time: 37.3232040405, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0112228394
validation[          , genome  6125] predictions:    4825/  10000 (48.25%), best:    4825/10000 (48.25%), error:     16796.98828, best error:     16796.98828 on epoch:     3, epoch:    3/50, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 659.2686767578s, input_fired_time: 5.9161396027, output_fired_time: 4.1763162613, propagate_forward_time: 187.2311401367, propagate_backward_time: 455.8465270996, weight_update_time: 0.7127615213, other_time: 6.0985412598
epoch time: 39.0643577576s, input_fired_time: 0.5855364799, output_fired_time: 0.0000000000, propagate_forward_time: 37.3893852234, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0894355774
validation[          , genome  6125] predictions:    4699/  10000 (46.99%), best:    4825/10000 (48.25%), error:     18071.31641, best error:     16796.98828 on epoch:     3, epoch:    4/50, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 660.0360107422s, input_fired_time: 5.9199442863, output_fired_time: 4.1871333122, propagate_forward_time: 187.4125213623, propagate_backward_time: 456.3902587891, weight_update_time: 0.7142131925, other_time: 6.1261596680
epoch time: 38.7992286682s, input_fired_time: 0.5851523280, output_fired_time: 0.0000000000, propagate_forward_time: 37.1247177124, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0893592834
validation[          , genome  6125] predictions:    5014/  10000 (50.14%), best:    5014/10000 (50.14%), error:     16142.80762, best error:     16142.80762 on epoch:     5, epoch:    5/50, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 658.2874755859s, input_fired_time: 5.9011116028, output_fired_time: 4.1762123108, propagate_forward_time: 186.8594207764, propagate_backward_time: 455.3056640625, weight_update_time: 0.7133889198, other_time: 6.0450439453
epoch time: 38.9187507629s, input_fired_time: 0.5880252123, output_fired_time: 0.0000000000, propagate_forward_time: 37.3124160767, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0183105469
validation[          , genome  6125] predictions:    5000/  10000 (50.00%), best:    5000/10000 (50.00%), error:     15310.28516, best error:     15310.28516 on epoch:     6, epoch:    6/50, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 655.9881591797s, input_fired_time: 5.8777093887, output_fired_time: 4.1501326561, propagate_forward_time: 186.2766265869, propagate_backward_time: 453.6500854492, weight_update_time: 0.7063212395, other_time: 6.0336303711
epoch time: 39.0780029297s, input_fired_time: 0.5992694497, output_fired_time: 0.0000000000, propagate_forward_time: 37.3933258057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0854072571
validation[          , genome  6125] predictions:    4934/  10000 (49.34%), best:    5000/10000 (50.00%), error:     15820.42969, best error:     15310.28516 on epoch:     6, epoch:    7/50, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 658.0582275391s, input_fired_time: 5.9013824463, output_fired_time: 4.1723008156, propagate_forward_time: 186.8160705566, propagate_backward_time: 455.1297607422, weight_update_time: 0.7109103799, other_time: 6.0387268066
epoch time: 38.8710403442s, input_fired_time: 0.5842362046, output_fired_time: 0.0000000000, propagate_forward_time: 37.2103424072, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0764617920
validation[          , genome  6125] predictions:    4788/  10000 (47.88%), best:    5000/10000 (50.00%), error:     17588.48047, best error:     15310.28516 on epoch:     6, epoch:    8/50, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 655.7946166992s, input_fired_time: 5.8798561096, output_fired_time: 4.1529207230, propagate_forward_time: 186.1819152832, propagate_backward_time: 453.5564575195, weight_update_time: 0.7090911865, other_time: 6.0234680176
epoch time: 38.9944953918s, input_fired_time: 0.5915332437, output_fired_time: 0.0000000000, propagate_forward_time: 37.3858337402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0171279907
validation[          , genome  6125] predictions:    5133/  10000 (51.33%), best:    5133/10000 (51.33%), error:     14976.20508, best error:     14976.20508 on epoch:     9, epoch:    9/50, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 659.1992797852s, input_fired_time: 5.9272923470, output_fired_time: 4.1733517647, propagate_forward_time: 187.1605682373, propagate_backward_time: 455.8724670410, weight_update_time: 0.7090234160, other_time: 6.0655822754
epoch time: 38.7810249329s, input_fired_time: 0.5905428529, output_fired_time: 0.0000000000, propagate_forward_time: 37.1108932495, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0795898438
validation[          , genome  6125] predictions:    5134/  10000 (51.34%), best:    5133/10000 (51.33%), error:     15428.24512, best error:     14976.20508 on epoch:     9, epoch:   10/50, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 660.6688842773s, input_fired_time: 5.9682445526, output_fired_time: 4.1940064430, propagate_forward_time: 187.6068267822, propagate_backward_time: 456.7898254395, weight_update_time: 0.7166638374, other_time: 6.1099548340
epoch time: 40.7385711670s, input_fired_time: 0.6312077641, output_fired_time: 0.0000000000, propagate_forward_time: 38.9123649597, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1949996948
validation[          , genome  6125] predictions:    5189/  10000 (51.89%), best:    5189/10000 (51.89%), error:     14804.54297, best error:     14804.54297 on epoch:    11, epoch:   11/50, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 662.5817871094s, input_fired_time: 5.9698181152, output_fired_time: 4.1980152130, propagate_forward_time: 188.0863952637, propagate_backward_time: 458.2644042969, weight_update_time: 0.7148968577, other_time: 6.0631408691
epoch time: 38.9807014465s, input_fired_time: 0.5965597630, output_fired_time: 0.0000000000, propagate_forward_time: 37.3706474304, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0134925842
validation[          , genome  6125] predictions:    5200/  10000 (52.00%), best:    5189/10000 (51.89%), error:     15107.05176, best error:     14804.54297 on epoch:    11, epoch:   12/50, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 657.7920532227s, input_fired_time: 5.9228100777, output_fired_time: 4.1654834747, propagate_forward_time: 186.7632598877, propagate_backward_time: 454.9558105469, weight_update_time: 0.7102901340, other_time: 5.9847412109
epoch time: 38.8625144958s, input_fired_time: 0.6008100510, output_fired_time: 0.0000000000, propagate_forward_time: 37.2100715637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0516319275
validation[          , genome  6125] predictions:    5446/  10000 (54.46%), best:    5446/10000 (54.46%), error:     14144.25098, best error:     14144.25098 on epoch:    13, epoch:   13/50, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 658.8640747070s, input_fired_time: 5.9403238297, output_fired_time: 4.1806335449, propagate_forward_time: 186.9624481201, propagate_backward_time: 455.7348632813, weight_update_time: 0.7105370164, other_time: 6.0458374023
epoch time: 39.0336265564s, input_fired_time: 0.5980451107, output_fired_time: 0.0000000000, propagate_forward_time: 37.3507728577, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0848083496
validation[          , genome  6125] predictions:    5273/  10000 (52.73%), best:    5446/10000 (54.46%), error:     14173.71094, best error:     14144.25098 on epoch:    13, epoch:   14/50, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 656.7528686523s, input_fired_time: 5.9321513176, output_fired_time: 4.1658172607, propagate_forward_time: 186.3798217773, propagate_backward_time: 454.2241210938, weight_update_time: 0.7079291344, other_time: 6.0509643555
epoch time: 39.0008010864s, input_fired_time: 0.5970656872, output_fired_time: 0.0000000000, propagate_forward_time: 37.3109054565, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0928306580
validation[          , genome  6125] predictions:    5275/  10000 (52.75%), best:    5446/10000 (54.46%), error:     14961.02148, best error:     14144.25098 on epoch:    13, epoch:   15/50, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 657.6142578125s, input_fired_time: 5.9427242279, output_fired_time: 4.1681299210, propagate_forward_time: 186.7085266113, propagate_backward_time: 454.7495422363, weight_update_time: 0.7104402184, other_time: 6.0452880859
epoch time: 38.8056678772s, input_fired_time: 0.6029918790, output_fired_time: 0.0000000000, propagate_forward_time: 37.1819839478, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0206909180
validation[          , genome  6125] predictions:    5383/  10000 (53.83%), best:    5446/10000 (54.46%), error:     14271.42383, best error:     14144.25098 on epoch:    13, epoch:   16/50, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 655.8421020508s, input_fired_time: 5.9346055984, output_fired_time: 4.1503152847, propagate_forward_time: 186.2485504150, propagate_backward_time: 453.5009765625, weight_update_time: 0.7067417502, other_time: 6.0076293945
epoch time: 38.5730361938s, input_fired_time: 0.5931824446, output_fired_time: 0.0000000000, propagate_forward_time: 36.9801559448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9996986389
validation[          , genome  6125] predictions:    5171/  10000 (51.71%), best:    5446/10000 (54.46%), error:     15012.73145, best error:     14144.25098 on epoch:    13, epoch:   17/50, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 654.1878662109s, input_fired_time: 5.9189381599, output_fired_time: 4.1328239441, propagate_forward_time: 185.5968475342, propagate_backward_time: 452.5697326660, weight_update_time: 0.7044233680, other_time: 5.9695129395
epoch time: 38.8624801636s, input_fired_time: 0.5990963578, output_fired_time: 0.0000000000, propagate_forward_time: 37.2411155701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0222663879
validation[          , genome  6125] predictions:    5341/  10000 (53.41%), best:    5446/10000 (54.46%), error:     14400.67090, best error:     14144.25098 on epoch:    13, epoch:   18/50, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 655.6221923828s, input_fired_time: 5.9320917130, output_fired_time: 4.1497626305, propagate_forward_time: 186.0728454590, propagate_backward_time: 453.4930419922, weight_update_time: 0.7068463564, other_time: 5.9744567871
epoch time: 38.7838401794s, input_fired_time: 0.5961285830, output_fired_time: 0.0000000000, propagate_forward_time: 37.1646270752, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0230827332
validation[          , genome  6125] predictions:    5370/  10000 (53.70%), best:    5370/10000 (53.70%), error:     14067.01367, best error:     14067.01367 on epoch:    19, epoch:   19/50, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 655.9357910156s, input_fired_time: 5.9468588829, output_fired_time: 4.1530704498, propagate_forward_time: 186.2574768066, propagate_backward_time: 453.6435546875, weight_update_time: 0.7069582343, other_time: 5.9348449707
epoch time: 38.9361076355s, input_fired_time: 0.6000565290, output_fired_time: 0.0000000000, propagate_forward_time: 37.3310737610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0049781799
validation[          , genome  6125] predictions:    5352/  10000 (53.52%), best:    5370/10000 (53.70%), error:     14589.45410, best error:     14067.01367 on epoch:    19, epoch:   20/50, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 653.5291137695s, input_fired_time: 5.9283895493, output_fired_time: 4.1269559860, propagate_forward_time: 185.4733886719, propagate_backward_time: 451.9140625000, weight_update_time: 0.7033460736, other_time: 6.0863037109
epoch time: 38.9193725586s, input_fired_time: 0.6006933451, output_fired_time: 0.0000000000, propagate_forward_time: 37.2080802917, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1105995178
validation[          , genome  6125] predictions:    5328/  10000 (53.28%), best:    5328/10000 (53.28%), error:     14060.77441, best error:     14060.77441 on epoch:    21, epoch:   21/50, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 653.6465454102s, input_fired_time: 5.9329571724, output_fired_time: 4.1328911781, propagate_forward_time: 185.4988861084, propagate_backward_time: 451.9688110352, weight_update_time: 0.7031940818, other_time: 6.1129760742
epoch time: 38.5975341797s, input_fired_time: 0.6026665568, output_fired_time: 0.0000000000, propagate_forward_time: 36.9498100281, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0450592041
validation[          , genome  6125] predictions:    5194/  10000 (51.94%), best:    5328/10000 (53.28%), error:     15109.16699, best error:     14060.77441 on epoch:    21, epoch:   22/50, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 655.9346923828s, input_fired_time: 5.9748573303, output_fired_time: 4.1553192139, propagate_forward_time: 186.2726745605, propagate_backward_time: 453.5569152832, weight_update_time: 0.7076454759, other_time: 5.9749145508
epoch time: 39.0558242798s, input_fired_time: 0.6122828722, output_fired_time: 0.0000000000, propagate_forward_time: 37.3592071533, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0843353271
validation[          , genome  6125] predictions:    5522/  10000 (55.22%), best:    5522/10000 (55.22%), error:     13614.20020, best error:     13614.20020 on epoch:    23, epoch:   23/50, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 659.0280761719s, input_fired_time: 6.0077290535, output_fired_time: 4.1798295975, propagate_forward_time: 187.1138916016, propagate_backward_time: 455.6932678223, weight_update_time: 0.7101202011, other_time: 6.0333557129
epoch time: 38.8190460205s, input_fired_time: 0.6056226492, output_fired_time: 0.0000000000, propagate_forward_time: 37.1295661926, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0838584900
validation[          , genome  6125] predictions:    5443/  10000 (54.43%), best:    5522/10000 (55.22%), error:     13744.04590, best error:     13614.20020 on epoch:    23, epoch:   24/50, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 656.7987670898s, input_fired_time: 5.9865808487, output_fired_time: 4.1573195457, propagate_forward_time: 186.4210662842, propagate_backward_time: 454.2334594727, weight_update_time: 0.7105032802, other_time: 6.0003051758
epoch time: 38.7535476685s, input_fired_time: 0.6067153215, output_fired_time: 0.0000000000, propagate_forward_time: 37.1306915283, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0161399841
validation[          , genome  6125] predictions:    5409/  10000 (54.09%), best:    5522/10000 (55.22%), error:     14179.77930, best error:     13614.20020 on epoch:    23, epoch:   25/50, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 655.6577148438s, input_fired_time: 5.9737205505, output_fired_time: 4.1505694389, propagate_forward_time: 186.0784454346, propagate_backward_time: 453.4013977051, weight_update_time: 0.7077103257, other_time: 6.0536193848
epoch time: 38.8484001160s, input_fired_time: 0.6109952331, output_fired_time: 0.0000000000, propagate_forward_time: 37.2196044922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0177993774
validation[          , genome  6125] predictions:    5441/  10000 (54.41%), best:    5441/10000 (54.41%), error:     13601.46289, best error:     13601.46289 on epoch:    26, epoch:   26/50, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 662.2291259766s, input_fired_time: 6.0426855087, output_fired_time: 4.1954097748, propagate_forward_time: 187.8346862793, propagate_backward_time: 457.9151000977, weight_update_time: 0.7135523558, other_time: 6.2412414551
epoch time: 39.3311882019s, input_fired_time: 0.6235568523, output_fired_time: 0.0000000000, propagate_forward_time: 37.5905799866, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1170501709
validation[          , genome  6125] predictions:    5407/  10000 (54.07%), best:    5441/10000 (54.41%), error:     14258.48145, best error:     13601.46289 on epoch:    26, epoch:   27/50, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 660.3109130859s, input_fired_time: 6.0447998047, output_fired_time: 4.1885151863, propagate_forward_time: 187.4829711914, propagate_backward_time: 456.5153808594, weight_update_time: 0.7135684490, other_time: 6.0792236328
epoch time: 39.0351638794s, input_fired_time: 0.6210258603, output_fired_time: 0.0000000000, propagate_forward_time: 37.3579902649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0561485291
validation[          , genome  6125] predictions:    5638/  10000 (56.38%), best:    5638/10000 (56.38%), error:     13122.24805, best error:     13122.24805 on epoch:    28, epoch:   28/50, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 657.3824462891s, input_fired_time: 6.0289506912, output_fired_time: 4.1618995667, propagate_forward_time: 186.6203765869, propagate_backward_time: 454.6225585938, weight_update_time: 0.7078420520, other_time: 5.9486694336
epoch time: 39.0239524841s, input_fired_time: 0.6223261356, output_fired_time: 0.0000000000, propagate_forward_time: 37.3385276794, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0630989075
validation[          , genome  6125] predictions:    5655/  10000 (56.55%), best:    5655/10000 (56.55%), error:     13012.20313, best error:     13012.20313 on epoch:    29, epoch:   29/50, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 655.5250244141s, input_fired_time: 6.0096278191, output_fired_time: 4.1476683617, propagate_forward_time: 186.1577911377, propagate_backward_time: 453.2078857422, weight_update_time: 0.7066559196, other_time: 6.0020751953
epoch time: 38.8150367737s, input_fired_time: 0.6132796407, output_fired_time: 0.0000000000, propagate_forward_time: 37.1107749939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0909805298
validation[          , genome  6125] predictions:    5631/  10000 (56.31%), best:    5655/10000 (56.55%), error:     13144.31934, best error:     13012.20313 on epoch:    29, epoch:   30/50, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 658.6897583008s, input_fired_time: 6.0464816093, output_fired_time: 4.1715774536, propagate_forward_time: 186.9838714600, propagate_backward_time: 455.4102478027, weight_update_time: 0.7104361653, other_time: 6.0775451660
epoch time: 38.9027748108s, input_fired_time: 0.6190309525, output_fired_time: 0.0000000000, propagate_forward_time: 37.2469367981, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0368080139
validation[          , genome  6125] predictions:    5566/  10000 (55.66%), best:    5655/10000 (56.55%), error:     13403.00977, best error:     13012.20313 on epoch:    29, epoch:   31/50, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 656.8038940430s, input_fired_time: 6.0319890976, output_fired_time: 4.1552581787, propagate_forward_time: 186.4887847900, propagate_backward_time: 454.1433410645, weight_update_time: 0.7084244490, other_time: 5.9845275879
epoch time: 39.0055160522s, input_fired_time: 0.6214628816, output_fired_time: 0.0000000000, propagate_forward_time: 37.3618164063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0222358704
validation[          , genome  6125] predictions:    5527/  10000 (55.27%), best:    5655/10000 (56.55%), error:     13569.47656, best error:     13012.20313 on epoch:    29, epoch:   32/50, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 655.7667846680s, input_fired_time: 6.0183367729, output_fired_time: 4.1466555595, propagate_forward_time: 186.0930175781, propagate_backward_time: 453.1843566895, weight_update_time: 0.7061659098, other_time: 6.3244323730
epoch time: 38.8752174377s, input_fired_time: 0.6203811169, output_fired_time: 0.0000000000, propagate_forward_time: 37.1547164917, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1001205444
validation[          , genome  6125] predictions:    5549/  10000 (55.49%), best:    5655/10000 (56.55%), error:     13618.53418, best error:     13012.20313 on epoch:    29, epoch:   33/50, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 654.7234497070s, input_fired_time: 6.0105528831, output_fired_time: 4.1219701767, propagate_forward_time: 185.8110656738, propagate_backward_time: 452.3760375977, weight_update_time: 0.7029532194, other_time: 6.4038391113
epoch time: 38.7924346924s, input_fired_time: 0.6190585494, output_fired_time: 0.0000000000, propagate_forward_time: 37.0707206726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1026573181
validation[          , genome  6125] predictions:    5748/  10000 (57.48%), best:    5748/10000 (57.48%), error:     12624.66797, best error:     12624.66797 on epoch:    34, epoch:   34/50, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 653.5902099609s, input_fired_time: 6.0102066994, output_fired_time: 4.1168336868, propagate_forward_time: 185.4118957520, propagate_backward_time: 451.6622314453, weight_update_time: 0.7006679773, other_time: 6.3890686035
epoch time: 38.8574142456s, input_fired_time: 0.6170272231, output_fired_time: 0.0000000000, propagate_forward_time: 37.1377487183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1026382446
validation[          , genome  6125] predictions:    5489/  10000 (54.89%), best:    5748/10000 (57.48%), error:     13965.74023, best error:     12624.66797 on epoch:    34, epoch:   35/50, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 654.1592407227s, input_fired_time: 6.0136156082, output_fired_time: 4.1211299896, propagate_forward_time: 185.6366119385, propagate_backward_time: 451.9830627441, weight_update_time: 0.7011733055, other_time: 6.4048156738
epoch time: 38.8181610107s, input_fired_time: 0.6253161430, output_fired_time: 0.0000000000, propagate_forward_time: 37.0915145874, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1013298035
validation[          , genome  6125] predictions:    5653/  10000 (56.53%), best:    5748/10000 (57.48%), error:     13276.41602, best error:     12624.66797 on epoch:    34, epoch:   36/50, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 653.5924072266s, input_fired_time: 6.0282325745, output_fired_time: 4.1149172783, propagate_forward_time: 185.4189147949, propagate_backward_time: 451.6406860352, weight_update_time: 0.6999837160, other_time: 6.3896179199
epoch time: 38.7624015808s, input_fired_time: 0.6202586293, output_fired_time: 0.0000000000, propagate_forward_time: 37.0412368774, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1009063721
validation[          , genome  6125] predictions:    5670/  10000 (56.70%), best:    5748/10000 (57.48%), error:     13003.56055, best error:     12624.66797 on epoch:    34, epoch:   37/50, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 653.5394287109s, input_fired_time: 6.0277991295, output_fired_time: 4.1129364967, propagate_forward_time: 185.4084472656, propagate_backward_time: 451.5722656250, weight_update_time: 0.7013881207, other_time: 6.4180297852
epoch time: 38.9602317810s, input_fired_time: 0.6252065897, output_fired_time: 0.0000000000, propagate_forward_time: 37.1903343201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1446914673
validation[          , genome  6125] predictions:    5656/  10000 (56.56%), best:    5748/10000 (57.48%), error:     13169.77734, best error:     12624.66797 on epoch:    34, epoch:   38/50, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 653.7266235352s, input_fired_time: 6.0379233360, output_fired_time: 4.1152558327, propagate_forward_time: 185.4221496582, propagate_backward_time: 451.7183532715, weight_update_time: 0.7009108663, other_time: 6.4329833984
epoch time: 38.8296127319s, input_fired_time: 0.6318753362, output_fired_time: 0.0000000000, propagate_forward_time: 37.0892868042, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1084518433
validation[          , genome  6125] predictions:    5673/  10000 (56.73%), best:    5748/10000 (57.48%), error:     13152.97363, best error:     12624.66797 on epoch:    34, epoch:   39/50, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 653.4479370117s, input_fired_time: 6.0449380875, output_fired_time: 4.1119637489, propagate_forward_time: 185.3566894531, propagate_backward_time: 451.4810180664, weight_update_time: 0.7007150650, other_time: 6.4533691406
epoch time: 38.7799835205s, input_fired_time: 0.6252831817, output_fired_time: 0.0000000000, propagate_forward_time: 37.0497741699, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1049270630
validation[          , genome  6125] predictions:    5734/  10000 (57.34%), best:    5748/10000 (57.48%), error:     13136.13086, best error:     12624.66797 on epoch:    34, epoch:   40/50, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 653.6007690430s, input_fired_time: 6.0523023605, output_fired_time: 4.1164941788, propagate_forward_time: 185.3967895508, propagate_backward_time: 451.6076354980, weight_update_time: 0.6995946169, other_time: 6.4275207520
epoch time: 38.7688522339s, input_fired_time: 0.6261840463, output_fired_time: 0.0000000000, propagate_forward_time: 37.0428009033, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0998687744
validation[          , genome  6125] predictions:    5796/  10000 (57.96%), best:    5748/10000 (57.48%), error:     12763.95020, best error:     12624.66797 on epoch:    34, epoch:   41/50, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 653.6167602539s, input_fired_time: 6.0448579788, output_fired_time: 4.1118011475, propagate_forward_time: 185.3857116699, propagate_backward_time: 451.6460571289, weight_update_time: 0.7026703954, other_time: 6.4283142090
epoch time: 38.7897605896s, input_fired_time: 0.6303147674, output_fired_time: 0.0000000000, propagate_forward_time: 37.0585365295, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1009101868
validation[          , genome  6125] predictions:    5785/  10000 (57.85%), best:    5785/10000 (57.85%), error:     12621.38379, best error:     12621.38379 on epoch:    42, epoch:   42/50, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 653.5186157227s, input_fired_time: 6.0668115616, output_fired_time: 4.1076917648, propagate_forward_time: 185.3759307861, propagate_backward_time: 451.5891113281, weight_update_time: 0.7010145783, other_time: 6.3790893555
epoch time: 38.8036384583s, input_fired_time: 0.6324442625, output_fired_time: 0.0000000000, propagate_forward_time: 37.0686950684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1025009155
validation[          , genome  6125] predictions:    5732/  10000 (57.32%), best:    5785/10000 (57.85%), error:     13236.38965, best error:     12621.38379 on epoch:    42, epoch:   43/50, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 653.9096069336s, input_fired_time: 6.0692481995, output_fired_time: 4.1154980659, propagate_forward_time: 185.5049285889, propagate_backward_time: 451.8490905762, weight_update_time: 0.7007208467, other_time: 6.3708190918
epoch time: 38.8299293518s, input_fired_time: 0.6324520707, output_fired_time: 0.0000000000, propagate_forward_time: 37.0997238159, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0977516174
validation[          , genome  6125] predictions:    5828/  10000 (58.28%), best:    5785/10000 (57.85%), error:     12834.81641, best error:     12621.38379 on epoch:    42, epoch:   44/50, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 653.6524047852s, input_fired_time: 6.0789632797, output_fired_time: 4.1127433777, propagate_forward_time: 185.4454650879, propagate_backward_time: 451.5810852051, weight_update_time: 0.7001435757, other_time: 6.4341430664
epoch time: 38.8144950867s, input_fired_time: 0.6368014812, output_fired_time: 0.0000000000, propagate_forward_time: 37.0768241882, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1008682251
validation[          , genome  6125] predictions:    5780/  10000 (57.80%), best:    5785/10000 (57.85%), error:     12712.26660, best error:     12621.38379 on epoch:    42, epoch:   45/50, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 653.6498413086s, input_fired_time: 6.0803713799, output_fired_time: 4.1171517372, propagate_forward_time: 185.3600158691, propagate_backward_time: 451.6337280273, weight_update_time: 0.7010402679, other_time: 6.4585876465
epoch time: 38.8940277100s, input_fired_time: 0.6368637681, output_fired_time: 0.0000000000, propagate_forward_time: 37.1557807922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1013832092
validation[          , genome  6125] predictions:    5877/  10000 (58.77%), best:    5877/10000 (58.77%), error:     12398.14844, best error:     12398.14844 on epoch:    46, epoch:   46/50, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 653.7279663086s, input_fired_time: 6.0913681984, output_fired_time: 4.1138701439, propagate_forward_time: 185.4483032227, propagate_backward_time: 451.6628417969, weight_update_time: 0.7011653781, other_time: 6.4115600586
epoch time: 38.8080024719s, input_fired_time: 0.6368970871, output_fired_time: 0.0000000000, propagate_forward_time: 37.0730514526, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0980529785
validation[          , genome  6125] predictions:    5938/  10000 (59.38%), best:    5938/10000 (59.38%), error:     12310.06055, best error:     12310.06055 on epoch:    47, epoch:   47/50, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 653.8397827148s, input_fired_time: 6.0922431946, output_fired_time: 4.1133694649, propagate_forward_time: 185.4831695557, propagate_backward_time: 451.7068481445, weight_update_time: 0.7012355328, other_time: 6.4442138672
epoch time: 38.8271408081s, input_fired_time: 0.6391318440, output_fired_time: 0.0000000000, propagate_forward_time: 37.0906791687, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0973281860
validation[          , genome  6125] predictions:    5816/  10000 (58.16%), best:    5938/10000 (59.38%), error:     12577.91406, best error:     12310.06055 on epoch:    47, epoch:   48/50, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 654.1230468750s, input_fired_time: 6.1041374207, output_fired_time: 4.1186532974, propagate_forward_time: 185.5306243896, propagate_backward_time: 451.9379577637, weight_update_time: 0.7014046311, other_time: 6.4316711426
epoch time: 38.8469848633s, input_fired_time: 0.6390161514, output_fired_time: 0.0000000000, propagate_forward_time: 37.1089591980, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0990104675
validation[          , genome  6125] predictions:    5931/  10000 (59.31%), best:    5938/10000 (59.38%), error:     12504.71289, best error:     12310.06055 on epoch:    47, epoch:   49/50, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 653.7437133789s, input_fired_time: 6.0967764854, output_fired_time: 4.1178536415, propagate_forward_time: 185.4366149902, propagate_backward_time: 451.6654663086, weight_update_time: 0.7010368705, other_time: 6.4269714355
epoch time: 38.8295211792s, input_fired_time: 0.6370492578, output_fired_time: 0.0000000000, propagate_forward_time: 37.0941886902, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0982818604
validation[          , genome  6125] predictions:    5810/  10000 (58.10%), best:    5938/10000 (59.38%), error:     12463.54199, best error:     12310.06055 on epoch:    47, epoch:   50/50, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 193.7979431152s, input_fired_time: 3.1799516678, output_fired_time: 0.0000000000, propagate_forward_time: 185.4864044189, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1315917969
best training[          , genome  6125] predictions:   34929/  50000 (69.86%), best:    5938/10000 (59.38%), error:     43274.04297, best error:     12310.06055 on epoch:    47, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 40.5918807983s, input_fired_time: 0.6382486820, output_fired_time: 0.0000000000, propagate_forward_time: 37.2255287170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7281036377
   testing[          , genome  6125] predictions:    5938/  10000 (59.38%), best:    5938/10000 (59.38%), error:     12310.06055, best error:     12310.06055 on epoch:    47, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
backpropagation finished successfully!
19:55:45 (7840): called boinc_finish(0)

</stderr_txt>
]]>