Nameexact_genome_1534504202_162_10889_1
Workunit3915227
Created18 Aug 2018, 12:58:37 UTC
Sent18 Aug 2018, 13:59:02 UTC
Report deadline20 Feb 2019, 19:32:18 UTC
Received27 Aug 2018, 8:56:41 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID56670
Run time3 days 10 hours 42 min 52 sec
CPU time3 days 5 hours 35 min 30 sec
Validate stateInitial
Credit0.00
Device peak FLOPS3.26 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size611.13 MB
Peak swap size632.52 MB
Peak disk usage17.05 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
 1.6460969448, other_time: 27.7879638672
epoch time: 200.2016754150s, input_fired_time: 3.0056586266, output_fired_time: 0.0000000000, propagate_forward_time: 192.2200012207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9760131836
validation[          , genome 10889] predictions:    6297/  10000 (62.97%), best:    6504/10000 (65.04%), error:     10979.44727, best error:     10150.84473 on epoch:    28, epoch:   29/100, mu: 0.8924949765, learning_rate: 0.0005323299, weight_decay: 0.0001459589

epoch time: 2560.4763183594s, input_fired_time: 44.0587882996, output_fired_time: 16.2923545837, propagate_forward_time: 954.5429687500, propagate_backward_time: 1517.8049316406, weight_update_time: 1.6523330212, other_time: 27.7773437500
epoch time: 200.3786621094s, input_fired_time: 3.0073413849, output_fired_time: 0.0000000000, propagate_forward_time: 192.4493865967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9219360352
validation[          , genome 10889] predictions:    6416/  10000 (64.16%), best:    6504/10000 (65.04%), error:     10345.52734, best error:     10150.84473 on epoch:    28, epoch:   30/100, mu: 0.8977255821, learning_rate: 0.0004987142, weight_decay: 0.0001397901

epoch time: 2587.0666503906s, input_fired_time: 44.5950469971, output_fired_time: 16.4418544769, propagate_forward_time: 962.2233886719, propagate_backward_time: 1535.6483154297, weight_update_time: 1.6861988306, other_time: 28.1580810547
epoch time: 202.8049468994s, input_fired_time: 3.0569434166, output_fired_time: 0.0000000000, propagate_forward_time: 194.7850799561, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9629211426
validation[          , genome 10889] predictions:    6421/  10000 (64.21%), best:    6504/10000 (65.04%), error:     10405.16602, best error:     10150.84473 on epoch:    28, epoch:   31/100, mu: 0.9026755691, learning_rate: 0.0004672212, weight_decay: 0.0001338821

epoch time: 2650.9848632813s, input_fired_time: 46.0378799438, output_fired_time: 17.2477550507, propagate_forward_time: 985.2055053711, propagate_backward_time: 1573.1492919922, weight_update_time: 1.6808873415, other_time: 29.3443603516
epoch time: 206.6729888916s, input_fired_time: 3.1216266155, output_fired_time: 0.0000000000, propagate_forward_time: 198.2560882568, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2952728271
validation[          , genome 10889] predictions:    6565/  10000 (65.65%), best:    6565/10000 (65.65%), error:     10077.64453, best error:     10077.64453 on epoch:    32, epoch:   32/100, mu: 0.9073600173, learning_rate: 0.0004377170, weight_decay: 0.0001282237

epoch time: 2963.0231933594s, input_fired_time: 51.5060729980, output_fired_time: 18.7871723175, propagate_forward_time: 1099.6035156250, propagate_backward_time: 1758.9426269531, weight_update_time: 1.8942114115, other_time: 34.1838378906
epoch time: 221.3378753662s, input_fired_time: 3.3966631889, output_fired_time: 0.0000000000, propagate_forward_time: 211.9851684570, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.9560394287
validation[          , genome 10889] predictions:    6681/  10000 (66.81%), best:    6681/10000 (66.81%), error:      9730.75293, best error:      9730.75293 on epoch:    33, epoch:   33/100, mu: 0.9117931724, learning_rate: 0.0004100759, weight_decay: 0.0001228045

epoch time: 2585.3527832031s, input_fired_time: 44.8522834778, output_fired_time: 16.5004291534, propagate_forward_time: 960.7388305664, propagate_backward_time: 1533.8317871094, weight_update_time: 1.6831389666, other_time: 29.4294433594
epoch time: 202.4337005615s, input_fired_time: 3.0959639549, output_fired_time: 0.0000000000, propagate_forward_time: 194.1161956787, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2215423584
validation[          , genome 10889] predictions:    6523/  10000 (65.23%), best:    6681/10000 (66.81%), error:     10267.26367, best error:      9730.75293 on epoch:    33, epoch:   34/100, mu: 0.9159885049, learning_rate: 0.0003841803, weight_decay: 0.0001176143

epoch time: 2580.6018066406s, input_fired_time: 44.5847167969, output_fired_time: 16.5225296021, propagate_forward_time: 959.3618774414, propagate_backward_time: 1530.7691650391, weight_update_time: 1.6756649017, other_time: 29.3636474609
epoch time: 201.8174133301s, input_fired_time: 3.0969851017, output_fired_time: 0.0000000000, propagate_forward_time: 193.4973297119, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2230987549
validation[          , genome 10889] predictions:    6644/  10000 (66.44%), best:    6681/10000 (66.81%), error:      9881.78711, best error:      9730.75293 on epoch:    33, epoch:   35/100, mu: 0.9199588299, learning_rate: 0.0003599199, weight_decay: 0.0001126435

epoch time: 2581.8542480469s, input_fired_time: 44.7786865234, output_fired_time: 16.4726505280, propagate_forward_time: 960.5416259766, propagate_backward_time: 1530.8597412109, weight_update_time: 1.6726490259, other_time: 29.2016601563
epoch time: 202.2892913818s, input_fired_time: 3.0688793659, output_fired_time: 0.0000000000, propagate_forward_time: 194.0355834961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1848297119
validation[          , genome 10889] predictions:    6593/  10000 (65.93%), best:    6681/10000 (66.81%), error:      9844.76465, best error:      9730.75293 on epoch:    33, epoch:   36/100, mu: 0.9237161279, learning_rate: 0.0003371916, weight_decay: 0.0001078828

epoch time: 2571.6701660156s, input_fired_time: 44.4305000305, output_fired_time: 16.3449192047, propagate_forward_time: 957.3194580078, propagate_backward_time: 1524.8745117188, weight_update_time: 1.6459971666, other_time: 28.7008056641
epoch time: 201.3513031006s, input_fired_time: 3.0750908852, output_fired_time: 0.0000000000, propagate_forward_time: 193.1817474365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0944671631
validation[          , genome 10889] predictions:    6479/  10000 (64.79%), best:    6681/10000 (66.81%), error:     10384.39258, best error:      9730.75293 on epoch:    33, epoch:   37/100, mu: 0.9272719026, learning_rate: 0.0003158985, weight_decay: 0.0001033233

epoch time: 2579.9765625000s, input_fired_time: 44.7153930664, output_fired_time: 16.4316482544, propagate_forward_time: 960.1259765625, propagate_backward_time: 1530.0107421875, weight_update_time: 1.6583791971, other_time: 28.6928710938
epoch time: 202.1086425781s, input_fired_time: 3.1004614830, output_fired_time: 0.0000000000, propagate_forward_time: 193.9351806641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0729980469
validation[          , genome 10889] predictions:    6638/  10000 (66.38%), best:    6681/10000 (66.81%), error:      9815.26270, best error:      9730.75293 on epoch:    33, epoch:   38/100, mu: 0.9306369424, learning_rate: 0.0002959500, weight_decay: 0.0000989564

epoch time: 2571.4587402344s, input_fired_time: 44.5183982849, output_fired_time: 16.3662052155, propagate_forward_time: 957.4191284180, propagate_backward_time: 1524.9058837891, weight_update_time: 1.6443953514, other_time: 28.2491455078
epoch time: 200.9668426514s, input_fired_time: 3.0697381496, output_fired_time: 0.0000000000, propagate_forward_time: 192.8664093018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0307006836
validation[          , genome 10889] predictions:    6608/  10000 (66.08%), best:    6681/10000 (66.81%), error:      9773.95703, best error:      9730.75293 on epoch:    33, epoch:   39/100, mu: 0.9338214397, learning_rate: 0.0002772613, weight_decay: 0.0000947742

epoch time: 2578.3979492188s, input_fired_time: 44.6090583801, output_fired_time: 16.4038753510, propagate_forward_time: 959.8696289063, propagate_backward_time: 1529.1826171875, weight_update_time: 1.6517847776, other_time: 28.3327636719
epoch time: 200.8904266357s, input_fired_time: 3.1402907372, output_fired_time: 0.0000000000, propagate_forward_time: 192.7548980713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9952392578
validation[          , genome 10889] predictions:    6572/  10000 (65.72%), best:    6681/10000 (66.81%), error:      9973.22461, best error:      9730.75293 on epoch:    33, epoch:   40/100, mu: 0.9368351102, learning_rate: 0.0002597527, weight_decay: 0.0000907687

epoch time: 2574.7995605469s, input_fired_time: 44.6552963257, output_fired_time: 16.4041690826, propagate_forward_time: 959.3152465820, propagate_backward_time: 1526.2901611328, weight_update_time: 1.6591024399, other_time: 28.1348876953
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1534504202_162_10889.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1534504202_162_10889_1_r1040257009_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.507551
read mu: 0.939687
read mu_delta: 0.946356
read initial_learning_rate: 0.00352961
read learning_rate: 0.00024335
read learning_rate_delta: 0.936852
read initial_weight_decay: 0.00051062
read weight_decay: 8.69324e-05
read weight_decay_delta: 0.957736
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.0874817
read input_dropout_probability: 0.00127161
read hidden_dropout_probability: 0.100476
read velocity_reset: 2028
read epoch: 41
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 33
read number_validation_images: 10000
read best_validation_predictions: 6681
read best_validation_error: 9730.75
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 10889
read normal distribution: '1 0 0'
generator_str: '1540472950'
read generator: 1540472950
read generated_by_map:
1 enable_node 1
reading 154 nodes.
reading 597 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1534504202_162_10889.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1534504202_162_10889_1_r1040257009_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.507551
read mu: 0.939687
read mu_delta: 0.946356
read initial_learning_rate: 0.00352961
read learning_rate: 0.00024335
read learning_rate_delta: 0.936852
read initial_weight_decay: 0.00051062
read weight_decay: 8.69324e-05
read weight_decay_delta: 0.957736
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.0874817
read input_dropout_probability: 0.00127161
read hidden_dropout_probability: 0.100476
read velocity_reset: 2028
read epoch: 41
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 33
read number_validation_images: 10000
read best_validation_predictions: 6681
read best_validation_error: 9730.75
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 10889
read normal distribution: '1 0 0'
generator_str: '1540472950'
read generator: 1540472950
read generated_by_map:
1 enable_node 1
reading 154 nodes.
reading 597 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2562.48681640625s, input_fired_time: 44.2631187438965, output_fired_time: 16.4005336761475, propagate_forward_time: 953.686645507813, propagate_backward_time: 1519.94165039063, weight_update_time: 1.65951538085938, other_time: 28.1947021484375
epoch time: 200.260391235352s, input_fired_time: 3.06048727035522, output_fired_time: 0, propagate_forward_time: 192.189849853516, propagate_backward_time: 0, weight_update_time: 0, other_time: 5.01005554199219
validation[          , genome 10889] predictions:    6581/  10000 (65.81%), best:    6681/10000 (66.81%), error:      9958.94141, best error:      9730.75293 on epoch:    33, epoch:   41/100, mu: 0.9396871328, learning_rate: 0.0002433498, weight_decay: 0.0000869324

epoch time: 2550.4724121094s, input_fired_time: 44.1539840698, output_fired_time: 16.2635707855, propagate_forward_time: 948.6206665039, propagate_backward_time: 1513.3874511719, weight_update_time: 1.6557170153, other_time: 28.0466308594
epoch time: 201.5195465088s, input_fired_time: 3.0938544273, output_fired_time: 0.0000000000, propagate_forward_time: 193.3916778564, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0340118408
validation[          , genome 10889] predictions:    6650/  10000 (66.50%), best:    6681/10000 (66.81%), error:      9757.47852, best error:      9730.75293 on epoch:    33, epoch:   42/100, mu: 0.9423861504, learning_rate: 0.0002279826, weight_decay: 0.0000832584

epoch time: 2553.6032714844s, input_fired_time: 44.1311073303, output_fired_time: 16.2759952545, propagate_forward_time: 950.8128051758, propagate_backward_time: 1514.3342285156, weight_update_time: 1.6449658871, other_time: 28.0493164063
epoch time: 199.9622497559s, input_fired_time: 3.0717253685, output_fired_time: 0.0000000000, propagate_forward_time: 191.8795013428, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0110168457
validation[          , genome 10889] predictions:    6634/  10000 (66.34%), best:    6681/10000 (66.81%), error:      9814.79004, best error:      9730.75293 on epoch:    33, epoch:   43/100, mu: 0.9449403882, learning_rate: 0.0002135859, weight_decay: 0.0000797396

epoch time: 2552.8505859375s, input_fired_time: 44.1792297363, output_fired_time: 16.2576313019, propagate_forward_time: 950.7589721680, propagate_backward_time: 1513.7027587891, weight_update_time: 1.6495457888, other_time: 27.9520263672
epoch time: 199.9996490479s, input_fired_time: 3.0837225914, output_fired_time: 0.0000000000, propagate_forward_time: 191.9114074707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0045166016
validation[          , genome 10889] predictions:    6732/  10000 (67.32%), best:    6732/10000 (67.32%), error:      9548.91309, best error:      9548.91309 on epoch:    44, epoch:   44/100, mu: 0.9473575950, learning_rate: 0.0002000983, weight_decay: 0.0000763695

epoch time: 2544.3059082031s, input_fired_time: 44.0554351807, output_fired_time: 16.2138061523, propagate_forward_time: 946.0615234375, propagate_backward_time: 1510.0189208984, weight_update_time: 1.6461005211, other_time: 27.9561767578
epoch time: 199.9167022705s, input_fired_time: 3.0667715073, output_fired_time: 0.0000000000, propagate_forward_time: 191.8401947021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0097351074
validation[          , genome 10889] predictions:    6733/  10000 (67.33%), best:    6732/10000 (67.32%), error:      9632.32129, best error:      9548.91309 on epoch:    44, epoch:   45/100, mu: 0.9496451020, learning_rate: 0.0001874624, weight_decay: 0.0000731418

epoch time: 2553.2119140625s, input_fired_time: 44.2131118774, output_fired_time: 16.2881851196, propagate_forward_time: 951.0638427734, propagate_backward_time: 1513.6030273438, weight_update_time: 1.6482156515, other_time: 28.0438232422
epoch time: 200.1291198730s, input_fired_time: 3.0734608173, output_fired_time: 0.0000000000, propagate_forward_time: 192.0736236572, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9820404053
validation[          , genome 10889] predictions:    6769/  10000 (67.69%), best:    6769/10000 (67.69%), error:      9530.31543, best error:      9530.31543 on epoch:    46, epoch:   46/100, mu: 0.9518099427, learning_rate: 0.0001756245, weight_decay: 0.0000700506

epoch time: 2568.8867187500s, input_fired_time: 44.4892807007, output_fired_time: 16.3422527313, propagate_forward_time: 958.1849975586, propagate_backward_time: 1521.7890625000, weight_update_time: 1.6456359625, other_time: 28.0810546875
epoch time: 198.7682189941s, input_fired_time: 3.0651164055, output_fired_time: 0.0000000000, propagate_forward_time: 190.7208251953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9822845459
validation[          , genome 10889] predictions:    6732/  10000 (67.32%), best:    6769/10000 (67.69%), error:      9610.91504, best error:      9530.31543 on epoch:    46, epoch:   47/100, mu: 0.9538586140, learning_rate: 0.0001645341, weight_decay: 0.0000670900

epoch time: 2580.6586914063s, input_fired_time: 44.7895545959, output_fired_time: 16.5695476532, propagate_forward_time: 959.3440551758, propagate_backward_time: 1531.7640380859, weight_update_time: 1.6455142498, other_time: 28.1915283203
epoch time: 213.5377960205s, input_fired_time: 3.2496697903, output_fired_time: 0.0000000000, propagate_forward_time: 204.8787994385, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4093322754
validation[          , genome 10889] predictions:    6734/  10000 (67.34%), best:    6769/10000 (67.69%), error:      9536.93750, best error:      9530.31543 on epoch:    46, epoch:   48/100, mu: 0.9557973742, learning_rate: 0.0001541441, weight_decay: 0.0000642545

epoch time: 2723.9096679688s, input_fired_time: 47.8069267273, output_fired_time: 17.2790184021, propagate_forward_time: 1009.9643554688, propagate_backward_time: 1618.7960205078, weight_update_time: 1.7625974417, other_time: 30.0633544922
epoch time: 218.7683410645s, input_fired_time: 3.4271278381, output_fired_time: 0.0000000000, propagate_forward_time: 209.7564697266, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5847473145
validation[          , genome 10889] predictions:    6769/  10000 (67.69%), best:    6769/10000 (67.69%), error:      9329.00781, best error:      9329.00781 on epoch:    49, epoch:   49/100, mu: 0.9576321244, learning_rate: 0.0001444101, weight_decay: 0.0000615389

epoch time: 2752.3122558594s, input_fired_time: 48.1526489258, output_fired_time: 17.6771488190, propagate_forward_time: 1017.9175415039, propagate_backward_time: 1636.5703125000, weight_update_time: 1.7686586380, other_time: 31.9946289063
epoch time: 209.9519958496s, input_fired_time: 3.3131518364, output_fired_time: 0.0000000000, propagate_forward_time: 200.9483184814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.6905212402
validation[          , genome 10889] predictions:    6789/  10000 (67.89%), best:    6769/10000 (67.69%), error:      9377.02832, best error:      9329.00781 on epoch:    49, epoch:   50/100, mu: 0.9593684673, learning_rate: 0.0001352909, weight_decay: 0.0000589380

epoch time: 2821.5344238281s, input_fired_time: 49.8277359009, output_fired_time: 18.7534885406, propagate_forward_time: 1037.6616210938, propagate_backward_time: 1681.4786376953, weight_update_time: 1.8271481991, other_time: 33.8131103516
epoch time: 253.6706237793s, input_fired_time: 4.0227446556, output_fired_time: 0.0000000000, propagate_forward_time: 242.3465423584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.3013305664
validation[          , genome 10889] predictions:    6570/  10000 (65.70%), best:    6769/10000 (67.69%), error:      9927.93848, best error:      9329.00781 on epoch:    49, epoch:   51/100, mu: 0.9610117078, learning_rate: 0.0001267475, weight_decay: 0.0000564471

epoch time: 3840.3369140625s, input_fired_time: 68.2518005371, output_fired_time: 25.6090011597, propagate_forward_time: 1410.0124511719, propagate_backward_time: 2291.4714355469, weight_update_time: 2.3121674061, other_time: 44.9924316406
epoch time: 290.2432556152s, input_fired_time: 4.6696147919, output_fired_time: 0.0000000000, propagate_forward_time: 277.7145385742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.8591003418
validation[          , genome 10889] predictions:    6660/  10000 (66.60%), best:    6769/10000 (67.69%), error:      9741.62891, best error:      9329.00781 on epoch:    49, epoch:   52/100, mu: 0.9625667930, learning_rate: 0.0001187436, weight_decay: 0.0000540614

epoch time: 3981.7561035156s, input_fired_time: 69.6910171509, output_fired_time: 25.8778972626, propagate_forward_time: 1453.4306640625, propagate_backward_time: 2384.6308593750, weight_update_time: 2.6330962181, other_time: 48.1257324219
epoch time: 316.4131469727s, input_fired_time: 5.3783946037, output_fired_time: 0.0000000000, propagate_forward_time: 302.2728881836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.7618713379
validation[          , genome 10889] predictions:    6706/  10000 (67.06%), best:    6769/10000 (67.69%), error:      9515.97266, best error:      9329.00781 on epoch:    49, epoch:   53/100, mu: 0.9640384316, learning_rate: 0.0001112451, weight_decay: 0.0000517766

epoch time: 3965.1418457031s, input_fired_time: 71.8218307495, output_fired_time: 26.8763599396, propagate_forward_time: 1449.0770263672, propagate_backward_time: 2369.3796386719, weight_update_time: 2.5787756443, other_time: 47.9870605469
epoch time: 298.1554260254s, input_fired_time: 5.0664601326, output_fired_time: 0.0000000000, propagate_forward_time: 285.0227661133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.0661926270
validation[          , genome 10889] predictions:    6678/  10000 (66.78%), best:    6769/10000 (67.69%), error:      9655.83984, best error:      9329.00781 on epoch:    49, epoch:   54/100, mu: 0.9654311538, learning_rate: 0.0001042202, weight_decay: 0.0000495883

epoch time: 3599.9975585938s, input_fired_time: 64.1187591553, output_fired_time: 23.0497303009, propagate_forward_time: 1318.6287841797, propagate_backward_time: 2152.5817871094, weight_update_time: 2.1430439949, other_time: 41.6184082031
epoch time: 201.6555023193s, input_fired_time: 3.1398429871, output_fired_time: 0.0000000000, propagate_forward_time: 193.4196929932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0959625244
validation[          , genome 10889] predictions:    6791/  10000 (67.91%), best:    6769/10000 (67.69%), error:      9463.25488, best error:      9329.00781 on epoch:    49, epoch:   55/100, mu: 0.9667491317, learning_rate: 0.0000976389, weight_decay: 0.0000474925

epoch time: 2675.8515625000s, input_fired_time: 47.2510032654, output_fired_time: 17.0262813568, propagate_forward_time: 994.2901611328, propagate_backward_time: 1586.9130859375, weight_update_time: 1.7363178730, other_time: 30.3709716797
epoch time: 213.4600982666s, input_fired_time: 3.3962612152, output_fired_time: 0.0000000000, propagate_forward_time: 204.6434173584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4204254150
validation[          , genome 10889] predictions:    6750/  10000 (67.50%), best:    6769/10000 (67.69%), error:      9593.81055, best error:      9329.00781 on epoch:    49, epoch:   56/100, mu: 0.9679964185, learning_rate: 0.0000914731, weight_decay: 0.0000454853

epoch time: 2574.3894042969s, input_fired_time: 44.8725738525, output_fired_time: 16.4591636658, propagate_forward_time: 957.1911621094, propagate_backward_time: 1526.8779296875, weight_update_time: 1.6669669151, other_time: 28.9885253906
epoch time: 201.2699584961s, input_fired_time: 3.1819911003, output_fired_time: 0.0000000000, propagate_forward_time: 192.8785705566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2093963623
validation[          , genome 10889] predictions:    6668/  10000 (66.68%), best:    6769/10000 (67.69%), error:      9763.78516, best error:      9329.00781 on epoch:    49, epoch:   57/100, mu: 0.9691767693, learning_rate: 0.0000856968, weight_decay: 0.0000435629

epoch time: 2562.7746582031s, input_fired_time: 44.7270584106, output_fired_time: 16.4013309479, propagate_forward_time: 954.0733032227, propagate_backward_time: 1519.0391845703, weight_update_time: 1.6542842388, other_time: 28.5338134766
epoch time: 201.0674896240s, input_fired_time: 3.1595201492, output_fired_time: 0.0000000000, propagate_forward_time: 192.8544769287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0534973145
validation[          , genome 10889] predictions:    6807/  10000 (68.07%), best:    6769/10000 (67.69%), error:      9335.18945, best error:      9329.00781 on epoch:    49, epoch:   58/100, mu: 0.9702938199, learning_rate: 0.0000802852, weight_decay: 0.0000417218

epoch time: 2571.7189941406s, input_fired_time: 44.8996582031, output_fired_time: 16.3984565735, propagate_forward_time: 958.1419067383, propagate_backward_time: 1523.7766113281, weight_update_time: 1.6579484940, other_time: 28.5024414063
epoch time: 200.6408691406s, input_fired_time: 3.1508157253, output_fired_time: 0.0000000000, propagate_forward_time: 192.4329071045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0571441650
validation[          , genome 10889] predictions:    6717/  10000 (67.17%), best:    6769/10000 (67.69%), error:      9537.70801, best error:      9329.00781 on epoch:    49, epoch:   59/100, mu: 0.9713509679, learning_rate: 0.0000752153, weight_decay: 0.0000399585

epoch time: 2627.1352539063s, input_fired_time: 46.0578804016, output_fired_time: 16.9025382996, propagate_forward_time: 977.0048828125, propagate_backward_time: 1557.8717041016, weight_update_time: 1.6907773018, other_time: 29.2982177734
epoch time: 201.0664825439s, input_fired_time: 3.1587240696, output_fired_time: 0.0000000000, propagate_forward_time: 192.7827758789, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1249847412
validation[          , genome 10889] predictions:    6851/  10000 (68.51%), best:    6851/10000 (68.51%), error:      9168.04395, best error:      9168.04395 on epoch:    60, epoch:   60/100, mu: 0.9723513722, learning_rate: 0.0000704656, weight_decay: 0.0000382697

epoch time: 3425.8767089844s, input_fired_time: 60.5173034668, output_fired_time: 22.8130455017, propagate_forward_time: 1257.9305419922, propagate_backward_time: 2045.3886718750, weight_update_time: 2.1583080292, other_time: 39.2270507813
epoch time: 649.0286865234s, input_fired_time: 11.7684593201, output_fired_time: 0.0000000000, propagate_forward_time: 619.9887084961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 17.2715454102
validation[          , genome 10889] predictions:    6649/  10000 (66.49%), best:    6851/10000 (68.51%), error:      9674.82422, best error:      9168.04395 on epoch:    60, epoch:   61/100, mu: 0.9732981324, learning_rate: 0.0000660158, weight_decay: 0.0000366523

epoch time: 3010.2722167969s, input_fired_time: 51.1679191589, output_fired_time: 18.8714313507, propagate_forward_time: 1111.9378662109, propagate_backward_time: 1793.3826904297, weight_update_time: 1.8278446198, other_time: 34.9123535156
epoch time: 202.6503753662s, input_fired_time: 3.2078201771, output_fired_time: 0.0000000000, propagate_forward_time: 194.2048034668, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2377471924
validation[          , genome 10889] predictions:    6797/  10000 (67.97%), best:    6851/10000 (68.51%), error:      9348.21289, best error:      9168.04395 on epoch:    60, epoch:   62/100, mu: 0.9741941094, learning_rate: 0.0000618470, weight_decay: 0.0000351032

epoch time: 2595.0942382813s, input_fired_time: 45.4423828125, output_fired_time: 16.5384826660, propagate_forward_time: 965.6442871094, propagate_backward_time: 1538.7669677734, weight_update_time: 1.6783782244, other_time: 28.7020263672
epoch time: 210.6758422852s, input_fired_time: 3.2734413147, output_fired_time: 0.0000000000, propagate_forward_time: 202.1060180664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2963867188
validation[          , genome 10889] predictions:    6902/  10000 (69.02%), best:    6902/10000 (69.02%), error:      9132.77344, best error:      9132.77344 on epoch:    63, epoch:   63/100, mu: 0.9750419855, learning_rate: 0.0000579415, weight_decay: 0.0000336196

epoch time: 2608.3220214844s, input_fired_time: 45.8506202698, output_fired_time: 16.7122039795, propagate_forward_time: 969.1055908203, propagate_backward_time: 1547.3692626953, weight_update_time: 1.6772489548, other_time: 29.2844238281
epoch time: 202.8135833740s, input_fired_time: 3.1983585358, output_fired_time: 0.0000000000, propagate_forward_time: 194.3568725586, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2583465576
validation[          , genome 10889] predictions:    6903/  10000 (69.03%), best:    6902/10000 (69.02%), error:      9140.79883, best error:      9132.77344 on epoch:    63, epoch:   64/100, mu: 0.9758443832, learning_rate: 0.0000542826, weight_decay: 0.0000321987

epoch time: 2588.7951660156s, input_fired_time: 45.3699951172, output_fired_time: 16.5904140472, propagate_forward_time: 962.1049804688, propagate_backward_time: 1535.4713134766, weight_update_time: 1.6743805408, other_time: 29.2586669922
epoch time: 203.2237243652s, input_fired_time: 3.2305467129, output_fired_time: 0.0000000000, propagate_forward_time: 194.7049560547, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2882232666
validation[          , genome 10889] predictions:    6788/  10000 (67.88%), best:    6902/10000 (69.02%), error:      9329.74219, best error:      9132.77344 on epoch:    63, epoch:   65/100, mu: 0.9766037464, learning_rate: 0.0000508547, weight_decay: 0.0000308379

epoch time: 2584.0749511719s, input_fired_time: 45.2496414185, output_fired_time: 16.5263099670, propagate_forward_time: 960.5875854492, propagate_backward_time: 1532.6074218750, weight_update_time: 1.6681193113, other_time: 29.1037597656
epoch time: 202.7847137451s, input_fired_time: 3.2118129730, output_fired_time: 0.0000000000, propagate_forward_time: 194.2715606689, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3013458252
validation[          , genome 10889] predictions:    6816/  10000 (68.16%), best:    6902/10000 (69.02%), error:      9270.00781, best error:      9132.77344 on epoch:    63, epoch:   66/100, mu: 0.9773223996, learning_rate: 0.0000476433, weight_decay: 0.0000295346

epoch time: 2585.0141601563s, input_fired_time: 45.3491210938, output_fired_time: 16.5560817719, propagate_forward_time: 961.5145874023, propagate_backward_time: 1532.6170654297, weight_update_time: 1.6725360155, other_time: 28.9771728516
epoch time: 201.5977325439s, input_fired_time: 3.1682071686, output_fired_time: 0.0000000000, propagate_forward_time: 193.3250885010, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1044311523
validation[          , genome 10889] predictions:    6870/  10000 (68.70%), best:    6902/10000 (69.02%), error:      9209.32715, best error:      9132.77344 on epoch:    63, epoch:   67/100, mu: 0.9780024886, learning_rate: 0.0000446347, weight_decay: 0.0000282863

epoch time: 2583.0698242188s, input_fired_time: 45.1839485168, output_fired_time: 16.4783554077, propagate_forward_time: 960.9321289063, propagate_backward_time: 1531.8524169922, weight_update_time: 1.6646522284, other_time: 28.6231689453
epoch time: 202.0291442871s, input_fired_time: 3.2158536911, output_fired_time: 0.0000000000, propagate_forward_time: 193.6669158936, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1463775635
validation[          , genome 10889] predictions:    6909/  10000 (69.09%), best:    6909/10000 (69.09%), error:      9044.00977, best error:      9044.00977 on epoch:    68, epoch:   68/100, mu: 0.9786460996, learning_rate: 0.0000418161, weight_decay: 0.0000270908

epoch time: 2588.6447753906s, input_fired_time: 45.3552055359, output_fired_time: 16.4942855835, propagate_forward_time: 963.0650634766, propagate_backward_time: 1535.1468505859, weight_update_time: 1.6675331593, other_time: 28.5832519531
epoch time: 202.1637115479s, input_fired_time: 3.1743321419, output_fired_time: 0.0000000000, propagate_forward_time: 193.8713989258, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1179809570
validation[          , genome 10889] predictions:    6838/  10000 (68.38%), best:    6909/10000 (69.09%), error:      9199.33594, best error:      9044.00977 on epoch:    68, epoch:   69/100, mu: 0.9792551994, learning_rate: 0.0000391755, weight_decay: 0.0000259459

epoch time: 2587.8254394531s, input_fired_time: 45.3088302612, output_fired_time: 16.5206947327, propagate_forward_time: 961.8779907227, propagate_backward_time: 1535.5189208984, weight_update_time: 1.6637692451, other_time: 28.5989990234
epoch time: 202.2428436279s, input_fired_time: 3.2156450748, output_fired_time: 0.0000000000, propagate_forward_time: 193.8774261475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1497650146
validation[          , genome 10889] predictions:    6798/  10000 (67.98%), best:    6909/10000 (69.09%), error:      9277.37012, best error:      9044.00977 on epoch:    68, epoch:   70/100, mu: 0.9798315763, learning_rate: 0.0000367016, weight_decay: 0.0000248493

epoch time: 2588.6994628906s, input_fired_time: 45.4286193848, output_fired_time: 16.5285739899, propagate_forward_time: 963.9126586914, propagate_backward_time: 1534.0447998047, weight_update_time: 1.6917183399, other_time: 28.7847900391
epoch time: 202.0012817383s, input_fired_time: 3.1944980621, output_fired_time: 0.0000000000, propagate_forward_time: 193.6920166016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1147613525
validation[          , genome 10889] predictions:    6917/  10000 (69.17%), best:    6917/10000 (69.17%), error:      8980.94824, best error:      8980.94824 on epoch:    71, epoch:   71/100, mu: 0.9803770781, learning_rate: 0.0000343840, weight_decay: 0.0000237991

epoch time: 2587.7275390625s, input_fired_time: 45.3996276855, output_fired_time: 16.5266513824, propagate_forward_time: 964.4990234375, propagate_backward_time: 1532.5684814453, weight_update_time: 1.6627106667, other_time: 28.7337646484
epoch time: 202.3662261963s, input_fired_time: 3.2303328514, output_fired_time: 0.0000000000, propagate_forward_time: 193.9340362549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2018585205
validation[          , genome 10889] predictions:    6860/  10000 (68.60%), best:    6917/10000 (69.17%), error:      9115.19434, best error:      8980.94824 on epoch:    71, epoch:   72/100, mu: 0.9808933139, learning_rate: 0.0000322127, weight_decay: 0.0000227933

epoch time: 2589.7077636719s, input_fired_time: 45.4148368835, output_fired_time: 16.6220092773, propagate_forward_time: 963.8385009766, propagate_backward_time: 1534.6616210938, weight_update_time: 1.6681976318, other_time: 29.1707763672
epoch time: 202.7898101807s, input_fired_time: 3.2233483791, output_fired_time: 0.0000000000, propagate_forward_time: 194.3161621094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2503051758
validation[          , genome 10889] predictions:    6814/  10000 (68.14%), best:    6917/10000 (69.17%), error:      9273.89355, best error:      8980.94824 on epoch:    71, epoch:   73/100, mu: 0.9813818336, learning_rate: 0.0000301785, weight_decay: 0.0000218299

epoch time: 2586.4589843750s, input_fired_time: 45.4422874451, output_fired_time: 16.5406818390, propagate_forward_time: 962.2108154297, propagate_backward_time: 1533.1219482422, weight_update_time: 1.6726543903, other_time: 29.1430664063
epoch time: 201.9031524658s, input_fired_time: 3.1903223991, output_fired_time: 0.0000000000, propagate_forward_time: 193.5401763916, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1726531982
validation[          , genome 10889] predictions:    6894/  10000 (68.94%), best:    6917/10000 (69.17%), error:      9095.24609, best error:      8980.94824 on epoch:    71, epoch:   74/100, mu: 0.9818441272, learning_rate: 0.0000282728, weight_decay: 0.0000209073

epoch time: 2597.5734863281s, input_fired_time: 45.4736022949, output_fired_time: 16.6062107086, propagate_forward_time: 966.0515136719, propagate_backward_time: 1540.2795410156, weight_update_time: 1.6786801815, other_time: 29.1625976563
epoch time: 203.2507781982s, input_fired_time: 3.2614006996, output_fired_time: 0.0000000000, propagate_forward_time: 194.7781524658, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2112274170
validation[          , genome 10889] predictions:    6890/  10000 (68.90%), best:    6917/10000 (69.17%), error:      9090.62793, best error:      8980.94824 on epoch:    71, epoch:   75/100, mu: 0.9822816253, learning_rate: 0.0000264874, weight_decay: 0.0000200237

epoch time: 2628.2680664063s, input_fired_time: 46.0477256775, output_fired_time: 16.8234500885, propagate_forward_time: 975.9116821289, propagate_backward_time: 1559.5266113281, weight_update_time: 1.7147327662, other_time: 29.9587402344
epoch time: 202.9306945801s, input_fired_time: 3.2506723404, output_fired_time: 0.0000000000, propagate_forward_time: 194.4603881836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2196350098
validation[          , genome 10889] predictions:    6906/  10000 (69.06%), best:    6917/10000 (69.17%), error:      8992.03809, best error:      8980.94824 on epoch:    71, epoch:   76/100, mu: 0.9826956987, learning_rate: 0.0000248148, weight_decay: 0.0000191774

epoch time: 2695.1989746094s, input_fired_time: 47.4577941895, output_fired_time: 16.9856281281, propagate_forward_time: 1002.1853027344, propagate_backward_time: 1597.7241210938, weight_update_time: 1.7509704828, other_time: 30.8461914063
epoch time: 203.4831390381s, input_fired_time: 3.2876906395, output_fired_time: 0.0000000000, propagate_forward_time: 194.9367523193, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2586975098
validation[          , genome 10889] predictions:    6915/  10000 (69.15%), best:    6917/10000 (69.17%), error:      9004.62207, best error:      8980.94824 on epoch:    71, epoch:   77/100, mu: 0.9830875397, learning_rate: 0.0000232478, weight_decay: 0.0000183669

epoch time: 2672.1469726563s, input_fired_time: 47.1252784729, output_fired_time: 17.1106586456, propagate_forward_time: 994.1285400391, propagate_backward_time: 1583.6627197266, weight_update_time: 1.7261739969, other_time: 30.1198730469
epoch time: 282.5165405273s, input_fired_time: 4.6577281952, output_fired_time: 0.0000000000, propagate_forward_time: 270.3434448242, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.5153808594
validation[          , genome 10889] predictions:    6965/  10000 (69.65%), best:    6965/10000 (69.65%), error:      8924.58496, best error:      8924.58496 on epoch:    78, epoch:   78/100, mu: 0.9834583402, learning_rate: 0.0000217797, weight_decay: 0.0000175907

epoch time: 2837.3098144531s, input_fired_time: 49.9401092529, output_fired_time: 18.0891742706, propagate_forward_time: 1051.3156738281, propagate_backward_time: 1685.4575195313, weight_update_time: 1.8522439003, other_time: 32.5073242188
epoch time: 204.2247009277s, input_fired_time: 3.2836680412, output_fired_time: 0.0000000000, propagate_forward_time: 195.7287445068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2122955322
validation[          , genome 10889] predictions:    6941/  10000 (69.41%), best:    6965/10000 (69.65%), error:      8949.71582, best error:      8924.58496 on epoch:    78, epoch:   79/100, mu: 0.9838092923, learning_rate: 0.0000204044, weight_decay: 0.0000168472

epoch time: 2726.6279296875s, input_fired_time: 48.1880416870, output_fired_time: 17.2899932861, propagate_forward_time: 1013.8430175781, propagate_backward_time: 1616.2736816406, weight_update_time: 1.7479726076, other_time: 31.0332031250
epoch time: 199.3426361084s, input_fired_time: 3.1875064373, output_fired_time: 0.0000000000, propagate_forward_time: 191.0221710205, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1329650879
validation[          , genome 10889] predictions:    6818/  10000 (68.18%), best:    6965/10000 (69.65%), error:      9178.24121, best error:      8924.58496 on epoch:    78, epoch:   80/100, mu: 0.9841414094, learning_rate: 0.0000191159, weight_decay: 0.0000161352

epoch time: 3315.6486816406s, input_fired_time: 58.3364868164, output_fired_time: 22.0049953461, propagate_forward_time: 1225.2854003906, propagate_backward_time: 1971.9732666016, weight_update_time: 2.1756970882, other_time: 38.0487060547
epoch time: 393.7326965332s, input_fired_time: 6.3053030968, output_fired_time: 0.0000000000, propagate_forward_time: 376.1509094238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.2764892578
validation[          , genome 10889] predictions:    6979/  10000 (69.79%), best:    6979/10000 (69.79%), error:      8857.84473, best error:      8857.84473 on epoch:    81, epoch:   81/100, mu: 0.9844557047, learning_rate: 0.0000179087, weight_decay: 0.0000154533

epoch time: 2999.4216308594s, input_fired_time: 53.3538055420, output_fired_time: 19.1318130493, propagate_forward_time: 1111.0452880859, propagate_backward_time: 1781.4616699219, weight_update_time: 1.9739365578, other_time: 34.4290771484
epoch time: 201.7441711426s, input_fired_time: 3.2221326828, output_fired_time: 0.0000000000, propagate_forward_time: 193.5041961670, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0178375244
validation[          , genome 10889] predictions:    6918/  10000 (69.18%), best:    6979/10000 (69.79%), error:      8971.32422, best error:      8857.84473 on epoch:    81, epoch:   82/100, mu: 0.9847531319, learning_rate: 0.0000167778, weight_decay: 0.0000148001

epoch time: 2585.8007812500s, input_fired_time: 45.3904914856, output_fired_time: 16.4455337524, propagate_forward_time: 963.2506713867, propagate_backward_time: 1532.5594482422, weight_update_time: 1.6530729532, other_time: 28.1546630859
epoch time: 202.4327392578s, input_fired_time: 3.2256920338, output_fired_time: 0.0000000000, propagate_forward_time: 194.1517791748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0552673340
validation[          , genome 10889] predictions:    6985/  10000 (69.85%), best:    6985/10000 (69.85%), error:      8846.12598, best error:      8846.12598 on epoch:    83, epoch:   83/100, mu: 0.9850345850, learning_rate: 0.0000157183, weight_decay: 0.0000141746

epoch time: 2562.3464355469s, input_fired_time: 44.7507553101, output_fired_time: 16.2020988464, propagate_forward_time: 955.3754272461, propagate_backward_time: 1518.1910400391, weight_update_time: 1.6420830488, other_time: 27.8270263672
epoch time: 199.4308624268s, input_fired_time: 3.1300587654, output_fired_time: 0.0000000000, propagate_forward_time: 191.3975524902, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9032440186
validation[          , genome 10889] predictions:    6938/  10000 (69.38%), best:    6985/10000 (69.85%), error:      9062.95703, best error:      8846.12598 on epoch:    83, epoch:   84/100, mu: 0.9853009582, learning_rate: 0.0000147257, weight_decay: 0.0000135756

epoch time: 2610.8461914063s, input_fired_time: 45.9330215454, output_fired_time: 16.8170547485, propagate_forward_time: 972.7463989258, propagate_backward_time: 1546.5583496094, weight_update_time: 1.6472197771, other_time: 28.7912597656
epoch time: 266.4463195801s, input_fired_time: 4.5057597160, output_fired_time: 0.0000000000, propagate_forward_time: 254.0946960449, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.8458557129
validation[          , genome 10889] predictions:    6967/  10000 (69.67%), best:    6985/10000 (69.85%), error:      8892.75293, best error:      8846.12598 on epoch:    83, epoch:   85/100, mu: 0.9855530262, learning_rate: 0.0000137958, weight_decay: 0.0000130018

epoch time: 3544.0891113281s, input_fired_time: 63.6284027100, output_fired_time: 23.8015251160, propagate_forward_time: 1292.3472900391, propagate_backward_time: 2120.5830078125, weight_update_time: 2.2877326012, other_time: 43.7290039063
epoch time: 264.4474182129s, input_fired_time: 4.4102029800, output_fired_time: 0.0000000000, propagate_forward_time: 252.3330993652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.7041015625
validation[          , genome 10889] predictions:    6955/  10000 (69.55%), best:    6985/10000 (69.85%), error:      8996.70410, best error:      8846.12598 on epoch:    83, epoch:   86/100, mu: 0.9857915640, learning_rate: 0.0000129246, weight_decay: 0.0000124523

epoch time: 3528.3525390625s, input_fired_time: 63.9150238037, output_fired_time: 23.7784748077, propagate_forward_time: 1290.9536132813, propagate_backward_time: 2106.8942871094, weight_update_time: 2.3570418358, other_time: 42.8110351563
epoch time: 265.2901000977s, input_fired_time: 4.4984526634, output_fired_time: 0.0000000000, propagate_forward_time: 253.1037139893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.6879425049
validation[          , genome 10889] predictions:    6933/  10000 (69.33%), best:    6933/10000 (69.33%), error:      8844.85449, best error:      8844.85449 on epoch:    87, epoch:   87/100, mu: 0.9860173464, learning_rate: 0.0000121085, weight_decay: 0.0000119260

epoch time: 3773.7770996094s, input_fired_time: 67.4807128906, output_fired_time: 26.8008842468, propagate_forward_time: 1376.9851074219, propagate_backward_time: 2254.4423828125, weight_update_time: 2.5737226009, other_time: 48.0681152344
epoch time: 280.4996337891s, input_fired_time: 4.7143769264, output_fired_time: 0.0000000000, propagate_forward_time: 268.1622009277, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.6230468750
validation[          , genome 10889] predictions:    6955/  10000 (69.55%), best:    6933/10000 (69.33%), error:      8912.01563, best error:      8844.85449 on epoch:    87, epoch:   88/100, mu: 0.9862309694, learning_rate: 0.0000113438, weight_decay: 0.0000114220

epoch time: 2816.7517089844s, input_fired_time: 49.7642478943, output_fired_time: 18.2160224915, propagate_forward_time: 1046.8862304688, propagate_backward_time: 1669.6322021484, weight_update_time: 1.9665542841, other_time: 32.2530517578
epoch time: 212.5245666504s, input_fired_time: 3.4971117973, output_fired_time: 0.0000000000, propagate_forward_time: 203.2699127197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.7575378418
validation[          , genome 10889] predictions:    7016/  10000 (70.16%), best:    7016/10000 (70.16%), error:      8772.77930, best error:      8772.77930 on epoch:    89, epoch:   89/100, mu: 0.9864331484, learning_rate: 0.0000106275, weight_decay: 0.0000109393

epoch time: 2776.3359375000s, input_fired_time: 49.3061676025, output_fired_time: 17.9092597961, propagate_forward_time: 1026.2791748047, propagate_backward_time: 1650.6940917969, weight_update_time: 1.8221638203, other_time: 32.1473388672
epoch time: 214.7901458740s, input_fired_time: 3.5402717590, output_fired_time: 0.0000000000, propagate_forward_time: 205.4232788086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.8265991211
validation[          , genome 10889] predictions:    6997/  10000 (69.97%), best:    7016/10000 (70.16%), error:      8799.42480, best error:      8772.77930 on epoch:    89, epoch:   90/100, mu: 0.9866244793, learning_rate: 0.0000099564, weight_decay: 0.0000104769

epoch time: 2574.6853027344s, input_fired_time: 45.1308288574, output_fired_time: 16.3438224792, propagate_forward_time: 960.4686889648, propagate_backward_time: 1524.7680664063, weight_update_time: 1.6611242294, other_time: 27.9738769531
epoch time: 200.2298736572s, input_fired_time: 3.1842072010, output_fired_time: 0.0000000000, propagate_forward_time: 192.0698699951, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9757995605
validation[          , genome 10889] predictions:    6920/  10000 (69.20%), best:    7016/10000 (70.16%), error:      8993.59863, best error:      8772.77930 on epoch:    89, epoch:   91/100, mu: 0.9868055582, learning_rate: 0.0000093277, weight_decay: 0.0000100341

epoch time: 2546.7397460938s, input_fired_time: 44.5793609619, output_fired_time: 16.2268447876, propagate_forward_time: 948.9931640625, propagate_backward_time: 1509.3997802734, weight_update_time: 1.6327195168, other_time: 27.5406494141
epoch time: 199.5777282715s, input_fired_time: 3.1666648388, output_fired_time: 0.0000000000, propagate_forward_time: 191.3807983398, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0302581787
validation[          , genome 10889] predictions:    7027/  10000 (70.27%), best:    7016/10000 (70.16%), error:      8779.91406, best error:      8772.77930 on epoch:    89, epoch:   92/100, mu: 0.9869769216, learning_rate: 0.0000087386, weight_decay: 0.0000096100

epoch time: 2551.5163574219s, input_fired_time: 44.6950073242, output_fired_time: 16.2611923218, propagate_forward_time: 951.5522460938, propagate_backward_time: 1510.9665527344, weight_update_time: 1.6278337240, other_time: 28.0412597656
epoch time: 200.1138610840s, input_fired_time: 3.1617763042, output_fired_time: 0.0000000000, propagate_forward_time: 191.9332580566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0188293457
validation[          , genome 10889] predictions:    7018/  10000 (70.18%), best:    7016/10000 (70.16%), error:      8792.71289, best error:      8772.77930 on epoch:    89, epoch:   93/100, mu: 0.9871391058, learning_rate: 0.0000081868, weight_decay: 0.0000092039

epoch time: 2543.3764648438s, input_fired_time: 44.4873580933, output_fired_time: 16.2410984039, propagate_forward_time: 947.4801025391, propagate_backward_time: 1507.0115966797, weight_update_time: 1.6299908161, other_time: 28.1562500000
epoch time: 200.5411529541s, input_fired_time: 3.1691932678, output_fired_time: 0.0000000000, propagate_forward_time: 192.2973480225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0746154785
validation[          , genome 10889] predictions:    6972/  10000 (69.72%), best:    7016/10000 (70.16%), error:      8881.02344, best error:      8772.77930 on epoch:    89, epoch:   94/100, mu: 0.9872925878, learning_rate: 0.0000076698, weight_decay: 0.0000088149

epoch time: 2546.3381347656s, input_fired_time: 44.5835609436, output_fired_time: 16.1610374451, propagate_forward_time: 948.5085449219, propagate_backward_time: 1509.0244140625, weight_update_time: 1.6295117140, other_time: 28.0605468750
epoch time: 198.6835479736s, input_fired_time: 3.1555066109, output_fired_time: 0.0000000000, propagate_forward_time: 190.5711364746, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9569091797
validation[          , genome 10889] predictions:    6989/  10000 (69.89%), best:    7016/10000 (70.16%), error:      8781.38379, best error:      8772.77930 on epoch:    89, epoch:   95/100, mu: 0.9874378443, learning_rate: 0.0000071855, weight_decay: 0.0000084424

epoch time: 2544.0270996094s, input_fired_time: 44.4986724854, output_fired_time: 16.0872039795, propagate_forward_time: 948.4274902344, propagate_backward_time: 1507.3714599609, weight_update_time: 1.6414415836, other_time: 27.6422119141
epoch time: 200.2264709473s, input_fired_time: 3.1781096458, output_fired_time: 0.0000000000, propagate_forward_time: 192.0841674805, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9641876221
validation[          , genome 10889] predictions:    6991/  10000 (69.91%), best:    7016/10000 (70.16%), error:      8778.22656, best error:      8772.77930 on epoch:    89, epoch:   96/100, mu: 0.9875752926, learning_rate: 0.0000067317, weight_decay: 0.0000080855

epoch time: 2545.4868164063s, input_fired_time: 44.4980392456, output_fired_time: 16.1018028259, propagate_forward_time: 948.1371459961, propagate_backward_time: 1509.0953369141, weight_update_time: 1.6287809610, other_time: 27.6544189453
epoch time: 199.6914672852s, input_fired_time: 3.1695396900, output_fired_time: 0.0000000000, propagate_forward_time: 191.6096191406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9123077393
validation[          , genome 10889] predictions:    7011/  10000 (70.11%), best:    7016/10000 (70.16%), error:      8786.16504, best error:      8772.77930 on epoch:    89, epoch:   97/100, mu: 0.9877053499, learning_rate: 0.0000063066, weight_decay: 0.0000077438

epoch time: 2551.2729492188s, input_fired_time: 44.5687561035, output_fired_time: 16.1632213593, propagate_forward_time: 952.3980102539, propagate_backward_time: 1510.5876464844, weight_update_time: 1.6401712894, other_time: 27.5551757813
epoch time: 200.3156890869s, input_fired_time: 3.2021694183, output_fired_time: 0.0000000000, propagate_forward_time: 192.1675415039, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9459838867
validation[          , genome 10889] predictions:    6972/  10000 (69.72%), best:    7016/10000 (70.16%), error:      8811.31250, best error:      8772.77930 on epoch:    89, epoch:   98/100, mu: 0.9878284335, learning_rate: 0.0000059084, weight_decay: 0.0000074165

epoch time: 2547.1308593750s, input_fired_time: 44.5517387390, output_fired_time: 16.1116008759, propagate_forward_time: 950.3595581055, propagate_backward_time: 1508.6557617188, weight_update_time: 1.6251323223, other_time: 27.4521484375
epoch time: 199.8735046387s, input_fired_time: 3.1749498844, output_fired_time: 0.0000000000, propagate_forward_time: 191.7628631592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9356842041
validation[          , genome 10889] predictions:    7035/  10000 (70.35%), best:    7035/10000 (70.35%), error:      8731.32910, best error:      8731.32910 on epoch:    99, epoch:   99/100, mu: 0.9879449010, learning_rate: 0.0000055353, weight_decay: 0.0000071031

epoch time: 2549.4733886719s, input_fired_time: 44.5764465332, output_fired_time: 16.1381988525, propagate_forward_time: 950.8846435547, propagate_backward_time: 1510.3350830078, weight_update_time: 1.6366848946, other_time: 27.5390625000
epoch time: 199.9527893066s, input_fired_time: 3.1565332413, output_fired_time: 0.0000000000, propagate_forward_time: 191.8594818115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9367675781
validation[          , genome 10889] predictions:    7021/  10000 (70.21%), best:    7035/10000 (70.35%), error:      8760.27832, best error:      8731.32910 on epoch:    99, epoch:  100/100, mu: 0.9880551696, learning_rate: 0.0000051857, weight_decay: 0.0000068029

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 990.2944335938s, input_fired_time: 15.7141628265, output_fired_time: 0.0000000000, propagate_forward_time: 950.2080688477, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 24.3721923828
best training[          , genome 10889] predictions:   40930/  50000 (81.86%), best:    7035/10000 (70.35%), error:     29563.90234, best error:      8731.32910 on epoch:    99, epoch:  101/100, mu: 0.9881594777, learning_rate: 0.0000048583, weight_decay: 0.0000065154
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 200.1713714600s, input_fired_time: 3.1608259678, output_fired_time: 0.0000000000, propagate_forward_time: 192.1574401855, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8531036377
   testing[          , genome 10889] predictions:    7035/  10000 (70.35%), best:    7035/10000 (70.35%), error:      8731.32910, best error:      8731.32910 on epoch:    99, epoch:  101/100, mu: 0.9881594777, learning_rate: 0.0000048583, weight_decay: 0.0000065154
backpropagation finished successfully!
02:56:16 (3668): called boinc_finish(0)

</stderr_txt>
]]>