Nameexact_genome_1542414603_189_6154_0
Workunit4095037
Created17 Nov 2018, 8:25:57 UTC
Sent17 Nov 2018, 9:10:33 UTC
Report deadline30 Jan 2019, 9:04:22 UTC
Received10 Jul 2019, 17:17:27 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID85311
Run time1 days 18 hours 29 min 10 sec
CPU time1 days 17 hours 38 min 38 sec
Validate stateValid
Credit18,498.93
Device peak FLOPS2.89 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size536.52 MB
Peak swap size556.15 MB
Peak disk usage9.87 MB

Stderr output

<core_client_version>7.10.2</core_client_version>
<![CDATA[
<stderr_txt>
other_time: 2.1711349487
validation[          , genome  6154] predictions:    5924/  10000 (59.24%), best:    5995/10000 (59.95%), error:     11748.62988, best error:     11512.83984 on epoch:    22, epoch:   28/100, mu: 0.8791882992, learning_rate: 0.0002030930, weight_decay: 0.0001151695

epoch time: 1393.9395751953s, input_fired_time: 14.8776855469, output_fired_time: 6.6071538925, propagate_forward_time: 523.3065795898, propagate_backward_time: 836.1695556641, weight_update_time: 0.3700232208, other_time: 12.9785766602
epoch time: 94.3473739624s, input_fired_time: 1.0201566219, output_fired_time: 0.0000000000, propagate_forward_time: 91.1602172852, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1669998169
validation[          , genome  6154] predictions:    5953/  10000 (59.53%), best:    5953/10000 (59.53%), error:     11512.65723, best error:     11512.65723 on epoch:    29, epoch:   29/100, mu: 0.8849003315, learning_rate: 0.0001893652, weight_decay: 0.0001097684

epoch time: 1191.9342041016s, input_fired_time: 14.5100488663, output_fired_time: 6.4581990242, propagate_forward_time: 451.6507568359, propagate_backward_time: 706.1322021484, weight_update_time: 0.3674430251, other_time: 13.1829833984
epoch time: 94.9248733521s, input_fired_time: 1.0188314915, output_fired_time: 0.0000000000, propagate_forward_time: 91.7271194458, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1789245605
validation[          , genome  6154] predictions:    5527/  10000 (55.27%), best:    5953/10000 (59.53%), error:     12993.06250, best error:     11512.65723 on epoch:    29, epoch:   30/100, mu: 0.8903179169, learning_rate: 0.0001765653, weight_decay: 0.0001046206

epoch time: 1226.6152343750s, input_fired_time: 14.8487262726, output_fired_time: 6.7631325722, propagate_forward_time: 463.2995910645, propagate_backward_time: 728.2896728516, weight_update_time: 0.3756599724, other_time: 13.4140625000
epoch time: 100.8690643311s, input_fired_time: 1.1505143642, output_fired_time: 0.0000000000, propagate_forward_time: 97.4084777832, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3100738525
validation[          , genome  6154] predictions:    6073/  10000 (60.73%), best:    6073/10000 (60.73%), error:     11229.78516, best error:     11229.78516 on epoch:    31, epoch:   31/100, mu: 0.8954562545, learning_rate: 0.0001646306, weight_decay: 0.0000997142

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_189_6154.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_189_6154_0_r1941448806_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.502308
read mu: 0.90033
read mu_delta: 0.948453
read initial_learning_rate: 0.00144128
read learning_rate: 0.000153503
read learning_rate_delta: 0.932406
read initial_weight_decay: 0.000442
read weight_decay: 9.50379e-05
read weight_decay_delta: 0.953103
read batch_size: 102
read epsilon: 1e-07
read alpha: 0.101811
read input_dropout_probability: 0.0013408
read hidden_dropout_probability: 0.108328
read velocity_reset: 1833
read epoch: 32
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 31
read number_validation_images: 10000
read best_validation_predictions: 6073
read best_validation_error: 11229.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6154
read normal distribution: '1 0 0'
generator_str: '1620734279'
read generator: 1620734279
read generated_by_map:
1 disable_node 1
reading 62 nodes.
reading 266 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_189_6154.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_189_6154_0_r1941448806_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.502308
read mu: 0.90033
read mu_delta: 0.948453
read initial_learning_rate: 0.00144128
read learning_rate: 0.000153503
read learning_rate_delta: 0.932406
read initial_weight_decay: 0.000442
read weight_decay: 9.50379e-05
read weight_decay_delta: 0.953103
read batch_size: 102
read epsilon: 1e-07
read alpha: 0.101811
read input_dropout_probability: 0.0013408
read hidden_dropout_probability: 0.108328
read velocity_reset: 1833
read epoch: 32
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 31
read number_validation_images: 10000
read best_validation_predictions: 6073
read best_validation_error: 11229.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6154
read normal distribution: '1 0 0'
generator_str: '1620734279'
read generator: 1620734279
read generated_by_map:
1 disable_node 1
reading 62 nodes.
reading 266 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 6656.33203125s, input_fired_time: 17.1812725067139, output_fired_time: 7.77285957336426, propagate_forward_time: 5177.8095703125, propagate_backward_time: 1434.00866699219, weight_update_time: 0.442844867706299, other_time: 19.5596923828125
epoch time: 126.693260192871s, input_fired_time: 1.36395394802094, output_fired_time: 0, propagate_forward_time: 122.005882263184, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.32342529296875
validation[          , genome  6154] predictions:    6049/  10000 (60.49%), best:    6073/10000 (60.73%), error:     11419.32324, best error:     11229.78516 on epoch:    31, epoch:   32/100, mu: 0.9003297091, learning_rate: 0.0001535026, weight_decay: 0.0000950379

epoch time: 1567.2487792969s, input_fired_time: 18.9113941193, output_fired_time: 8.5246534348, propagate_forward_time: 593.8574829102, propagate_backward_time: 928.7700805664, weight_update_time: 0.4813229740, other_time: 17.1851806641
epoch time: 118.0376892090s, input_fired_time: 1.2779946327, output_fired_time: 0.0000000000, propagate_forward_time: 114.0777664185, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6819305420
validation[          , genome  6154] predictions:    5985/  10000 (59.85%), best:    6073/10000 (60.73%), error:     11536.16797, best error:     11229.78516 on epoch:    31, epoch:   33/100, mu: 0.9049519897, learning_rate: 0.0001431268, weight_decay: 0.0000905809

epoch time: 1591.5670166016s, input_fired_time: 19.2609519958, output_fired_time: 8.6889343262, propagate_forward_time: 602.9256591797, propagate_backward_time: 943.2229614258, weight_update_time: 0.5238949060, other_time: 17.4684448242
epoch time: 126.7509078979s, input_fired_time: 1.4010889530, output_fired_time: 0.0000000000, propagate_forward_time: 122.4280548096, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9217605591
validation[          , genome  6154] predictions:    5962/  10000 (59.62%), best:    6073/10000 (60.73%), error:     11412.28125, best error:     11229.78516 on epoch:    31, epoch:   34/100, mu: 0.9093359709, learning_rate: 0.0001334523, weight_decay: 0.0000863329

epoch time: 1643.6940917969s, input_fired_time: 19.8617286682, output_fired_time: 9.0040378571, propagate_forward_time: 621.8002929688, propagate_backward_time: 975.1901855469, weight_update_time: 0.4886872172, other_time: 17.8378906250
epoch time: 133.5206756592s, input_fired_time: 1.4598922729, output_fired_time: 0.0000000000, propagate_forward_time: 128.9865417480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0742492676
validation[          , genome  6154] predictions:    6023/  10000 (60.23%), best:    6073/10000 (60.73%), error:     11252.80469, best error:     11229.78516 on epoch:    31, epoch:   35/100, mu: 0.9134939909, learning_rate: 0.0001244318, weight_decay: 0.0000822842

epoch time: 1685.3747558594s, input_fired_time: 20.3396339417, output_fired_time: 9.1533775330, propagate_forward_time: 637.7209472656, propagate_backward_time: 999.7581176758, weight_update_time: 0.5194162130, other_time: 18.4027709961
epoch time: 134.9998779297s, input_fired_time: 1.4688118696, output_fired_time: 0.0000000000, propagate_forward_time: 130.4117889404, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1192779541
validation[          , genome  6154] predictions:    5886/  10000 (58.86%), best:    6073/10000 (60.73%), error:     11815.92578, best error:     11229.78516 on epoch:    31, epoch:   36/100, mu: 0.9174376726, learning_rate: 0.0001160210, weight_decay: 0.0000784253

epoch time: 1726.6616210938s, input_fired_time: 21.0058288574, output_fired_time: 9.4240150452, propagate_forward_time: 655.0065917969, propagate_backward_time: 1022.5441894531, weight_update_time: 0.5159785151, other_time: 18.6809082031
epoch time: 136.3918762207s, input_fired_time: 1.4929182529, output_fired_time: 0.0000000000, propagate_forward_time: 131.7374420166, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1615142822
validation[          , genome  6154] predictions:    6143/  10000 (61.43%), best:    6143/10000 (61.43%), error:     11025.24707, best error:     11025.24707 on epoch:    37, epoch:   37/100, mu: 0.9211780429, learning_rate: 0.0001081787, weight_decay: 0.0000747474

epoch time: 1729.2976074219s, input_fired_time: 20.9535274506, output_fired_time: 9.4753341675, propagate_forward_time: 656.2678833008, propagate_backward_time: 1023.8813476563, weight_update_time: 0.5175754428, other_time: 18.7194824219
epoch time: 136.6956024170s, input_fired_time: 1.4908761978, output_fired_time: 0.0000000000, propagate_forward_time: 132.1283874512, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0763397217
validation[          , genome  6154] predictions:    6073/  10000 (60.73%), best:    6143/10000 (61.43%), error:     11135.82520, best error:     11025.24707 on epoch:    37, epoch:   38/100, mu: 0.9247256517, learning_rate: 0.0001008665, weight_decay: 0.0000712420

epoch time: 1730.3211669922s, input_fired_time: 21.0096302032, output_fired_time: 9.4169006348, propagate_forward_time: 654.9691772461, propagate_backward_time: 1026.2248535156, weight_update_time: 0.5167101026, other_time: 18.7006835938
epoch time: 137.1537170410s, input_fired_time: 1.5045578480, output_fired_time: 0.0000000000, propagate_forward_time: 132.5478210449, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1013336182
validation[          , genome  6154] predictions:    6097/  10000 (60.97%), best:    6143/10000 (61.43%), error:     11117.41602, best error:     11025.24707 on epoch:    37, epoch:   39/100, mu: 0.9280903339, learning_rate: 0.0000940486, weight_decay: 0.0000679009

epoch time: 1731.6457519531s, input_fired_time: 20.9874782562, output_fired_time: 9.4013919830, propagate_forward_time: 657.2462768555, propagate_backward_time: 1025.2457275391, weight_update_time: 0.5127508044, other_time: 18.7650146484
epoch time: 138.0816955566s, input_fired_time: 1.5116721392, output_fired_time: 0.0000000000, propagate_forward_time: 133.3399505615, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2300720215
validation[          , genome  6154] predictions:    6044/  10000 (60.44%), best:    6143/10000 (61.43%), error:     11353.41504, best error:     11025.24707 on epoch:    37, epoch:   40/100, mu: 0.9312816262, learning_rate: 0.0000876915, weight_decay: 0.0000647166

epoch time: 1765.8247070313s, input_fired_time: 21.3393268585, output_fired_time: 9.5949077606, propagate_forward_time: 670.0910034180, propagate_backward_time: 1045.7779541016, weight_update_time: 0.5350619555, other_time: 19.0216064453
epoch time: 140.0963745117s, input_fired_time: 1.5592888594, output_fired_time: 0.0000000000, propagate_forward_time: 135.3116302490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2254486084
validation[          , genome  6154] predictions:    6118/  10000 (61.18%), best:    6143/10000 (61.43%), error:     11148.59570, best error:     11025.24707 on epoch:    37, epoch:   41/100, mu: 0.9343084097, learning_rate: 0.0000817641, weight_decay: 0.0000616816

epoch time: 1771.4543457031s, input_fired_time: 21.2119255066, output_fired_time: 9.6573143005, propagate_forward_time: 671.6769409180, propagate_backward_time: 1049.7089843750, weight_update_time: 0.5411745310, other_time: 19.1992187500
epoch time: 140.2787628174s, input_fired_time: 1.5395623446, output_fired_time: 0.0000000000, propagate_forward_time: 135.5480804443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1911163330
validation[          , genome  6154] predictions:    6080/  10000 (60.80%), best:    6143/10000 (61.43%), error:     11251.84863, best error:     11025.24707 on epoch:    37, epoch:   42/100, mu: 0.9371791482, learning_rate: 0.0000762374, weight_decay: 0.0000587889

epoch time: 1774.1290283203s, input_fired_time: 21.3916282654, output_fired_time: 9.5937843323, propagate_forward_time: 671.3250122070, propagate_backward_time: 1052.7064208984, weight_update_time: 0.5228168368, other_time: 19.1121826172
epoch time: 143.5801696777s, input_fired_time: 1.5784072876, output_fired_time: 0.0000000000, propagate_forward_time: 138.7343444824, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2674255371
validation[          , genome  6154] predictions:    6188/  10000 (61.88%), best:    6188/10000 (61.88%), error:     10986.52637, best error:     10986.52637 on epoch:    43, epoch:   43/100, mu: 0.9399018884, learning_rate: 0.0000710842, weight_decay: 0.0000560319

epoch time: 1797.5173339844s, input_fired_time: 21.6369819641, output_fired_time: 9.7341995239, propagate_forward_time: 682.2282104492, propagate_backward_time: 1064.3675537109, weight_update_time: 0.5548359752, other_time: 19.5504150391
epoch time: 138.9761505127s, input_fired_time: 1.5461330414, output_fired_time: 0.0000000000, propagate_forward_time: 134.2770385742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1529846191
validation[          , genome  6154] predictions:    6210/  10000 (62.10%), best:    6210/10000 (62.10%), error:     10926.33398, best error:     10926.33398 on epoch:    44, epoch:   44/100, mu: 0.9424843192, learning_rate: 0.0000662793, weight_decay: 0.0000534041

epoch time: 1789.5645751953s, input_fired_time: 21.5100574493, output_fired_time: 9.7634973526, propagate_forward_time: 678.5706787109, propagate_backward_time: 1060.3549804688, weight_update_time: 0.5351059437, other_time: 19.3653564453
epoch time: 141.3932952881s, input_fired_time: 1.5302307606, output_fired_time: 0.0000000000, propagate_forward_time: 136.6159515381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2471160889
validation[          , genome  6154] predictions:    6250/  10000 (62.50%), best:    6250/10000 (62.50%), error:     10794.76270, best error:     10794.76270 on epoch:    45, epoch:   45/100, mu: 0.9449335933, learning_rate: 0.0000617993, weight_decay: 0.0000508996

epoch time: 1807.6296386719s, input_fired_time: 21.6847019196, output_fired_time: 9.8781719208, propagate_forward_time: 684.6619262695, propagate_backward_time: 1071.7768554688, weight_update_time: 0.5382643342, other_time: 19.6279296875
epoch time: 145.6730346680s, input_fired_time: 1.6029231548, output_fired_time: 0.0000000000, propagate_forward_time: 140.7580718994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3120422363
validation[          , genome  6154] predictions:    6169/  10000 (61.69%), best:    6250/10000 (62.50%), error:     11059.53320, best error:     10794.76270 on epoch:    45, epoch:   46/100, mu: 0.9472566247, learning_rate: 0.0000576220, weight_decay: 0.0000485126

epoch time: 1805.5295410156s, input_fired_time: 21.8031520844, output_fired_time: 9.8234739304, propagate_forward_time: 684.4663696289, propagate_backward_time: 1069.6557617188, weight_update_time: 0.6312216520, other_time: 19.7807617188
epoch time: 142.5278625488s, input_fired_time: 1.5614641905, output_fired_time: 0.0000000000, propagate_forward_time: 137.6486968994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3177032471
validation[          , genome  6154] predictions:    6009/  10000 (60.09%), best:    6250/10000 (62.50%), error:     11437.11719, best error:     10794.76270 on epoch:    45, epoch:   47/100, mu: 0.9494599104, learning_rate: 0.0000537272, weight_decay: 0.0000462375

epoch time: 1550.9749755859s, input_fired_time: 19.8106212616, output_fired_time: 8.4963235855, propagate_forward_time: 583.1316528320, propagate_backward_time: 921.9745483398, weight_update_time: 0.4710992575, other_time: 17.5617675781
epoch time: 104.3895950317s, input_fired_time: 1.2331259251, output_fired_time: 0.0000000000, propagate_forward_time: 100.5704193115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5860519409
validation[          , genome  6154] predictions:    6274/  10000 (62.74%), best:    6250/10000 (62.50%), error:     10820.37207, best error:     10794.76270 on epoch:    45, epoch:   48/100, mu: 0.9515496492, learning_rate: 0.0000500955, weight_decay: 0.0000440691

epoch time: 1321.3232421875s, input_fired_time: 18.1641845703, output_fired_time: 7.3456339836, propagate_forward_time: 492.7261657715, propagate_backward_time: 787.4296875000, weight_update_time: 0.4130569696, other_time: 15.6575927734
epoch time: 109.1344604492s, input_fired_time: 1.2736830711, output_fired_time: 0.0000000000, propagate_forward_time: 105.1404571533, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7203216553
validation[          , genome  6154] predictions:    6185/  10000 (61.85%), best:    6250/10000 (62.50%), error:     11009.07422, best error:     10794.76270 on epoch:    45, epoch:   49/100, mu: 0.9535316825, learning_rate: 0.0000467094, weight_decay: 0.0000420024

epoch time: 1331.3337402344s, input_fired_time: 18.2358722687, output_fired_time: 7.4251351357, propagate_forward_time: 496.5088500977, propagate_backward_time: 793.3580932617, weight_update_time: 0.4297531247, other_time: 15.8057861328
epoch time: 103.2512817383s, input_fired_time: 1.2222511768, output_fired_time: 0.0000000000, propagate_forward_time: 99.4561920166, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5728378296
validation[          , genome  6154] predictions:    6099/  10000 (60.99%), best:    6250/10000 (62.50%), error:     11122.15918, best error:     10794.76270 on epoch:    45, epoch:   50/100, mu: 0.9554114938, learning_rate: 0.0000435521, weight_decay: 0.0000400326

epoch time: 1372.0126953125s, input_fired_time: 18.6456260681, output_fired_time: 7.5912933350, propagate_forward_time: 512.2380371094, propagate_backward_time: 817.1755371094, weight_update_time: 0.4319141805, other_time: 16.3621826172
epoch time: 127.0853805542s, input_fired_time: 1.4167437553, output_fired_time: 0.0000000000, propagate_forward_time: 122.7347183228, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9339218140
validation[          , genome  6154] predictions:    6289/  10000 (62.89%), best:    6250/10000 (62.50%), error:     10802.28906, best error:     10794.76270 on epoch:    45, epoch:   51/100, mu: 0.9571944475, learning_rate: 0.0000406083, weight_decay: 0.0000381552

epoch time: 1382.6047363281s, input_fired_time: 18.7530384064, output_fired_time: 7.6257767677, propagate_forward_time: 522.7488403320, propagate_backward_time: 817.1668090820, weight_update_time: 0.4394742846, other_time: 16.3103027344
epoch time: 104.5569458008s, input_fired_time: 1.2425597906, output_fired_time: 0.0000000000, propagate_forward_time: 100.7211380005, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5932464600
validation[          , genome  6154] predictions:    6215/  10000 (62.15%), best:    6250/10000 (62.50%), error:     10858.73535, best error:     10794.76270 on epoch:    45, epoch:   52/100, mu: 0.9588854909, learning_rate: 0.0000378634, weight_decay: 0.0000363658

epoch time: 1359.5172119141s, input_fired_time: 18.5991630554, output_fired_time: 7.6248164177, propagate_forward_time: 507.3399963379, propagate_backward_time: 809.8881225586, weight_update_time: 0.4308507442, other_time: 16.0652465820
epoch time: 100.7321395874s, input_fired_time: 1.1755493879, output_fired_time: 0.0000000000, propagate_forward_time: 97.0346145630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5219726563
validation[          , genome  6154] predictions:    6244/  10000 (62.44%), best:    6250/10000 (62.50%), error:     10892.63379, best error:     10794.76270 on epoch:    45, epoch:   53/100, mu: 0.9604893327, learning_rate: 0.0000353041, weight_decay: 0.0000346604

epoch time: 1367.8712158203s, input_fired_time: 18.6688804626, output_fired_time: 7.6200141907, propagate_forward_time: 509.2666015625, propagate_backward_time: 816.3092041016, weight_update_time: 0.4396240413, other_time: 16.0065917969
epoch time: 111.8936767578s, input_fired_time: 1.3187919855, output_fired_time: 0.0000000000, propagate_forward_time: 107.7568511963, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8180313110
validation[          , genome  6154] predictions:    6292/  10000 (62.92%), best:    6250/10000 (62.50%), error:     10828.34570, best error:     10794.76270 on epoch:    45, epoch:   54/100, mu: 0.9620105028, learning_rate: 0.0000329178, weight_decay: 0.0000330349

epoch time: 1379.2095947266s, input_fired_time: 18.9139881134, output_fired_time: 7.7033433914, propagate_forward_time: 515.2754516602, propagate_backward_time: 821.0466308594, weight_update_time: 0.4449932873, other_time: 16.2702026367
epoch time: 97.4207077026s, input_fired_time: 1.1461492777, output_fired_time: 0.0000000000, propagate_forward_time: 93.8450698853, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4294891357
validation[          , genome  6154] predictions:    6316/  10000 (63.16%), best:    6316/10000 (63.16%), error:     10681.54004, best error:     10681.54004 on epoch:    55, epoch:   55/100, mu: 0.9634532928, learning_rate: 0.0000306927, weight_decay: 0.0000314857

epoch time: 1376.7945556641s, input_fired_time: 18.8749752045, output_fired_time: 7.6574583054, propagate_forward_time: 512.7386474609, propagate_backward_time: 821.2933959961, weight_update_time: 0.4364034832, other_time: 16.2300415039
epoch time: 92.6331634521s, input_fired_time: 1.1114842892, output_fired_time: 0.0000000000, propagate_forward_time: 89.2242279053, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2974548340
validation[          , genome  6154] predictions:    6224/  10000 (62.24%), best:    6316/10000 (63.16%), error:     10921.22070, best error:     10681.54004 on epoch:    55, epoch:   56/100, mu: 0.9648216963, learning_rate: 0.0000286181, weight_decay: 0.0000300091

epoch time: 1382.9569091797s, input_fired_time: 19.0768642426, output_fired_time: 7.6396675110, propagate_forward_time: 516.5927124023, propagate_backward_time: 823.2242431641, weight_update_time: 0.4450426400, other_time: 16.4234008789
epoch time: 113.0387344360s, input_fired_time: 1.3309553862, output_fired_time: 0.0000000000, propagate_forward_time: 108.9009857178, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8067932129
validation[          , genome  6154] predictions:    6235/  10000 (62.35%), best:    6316/10000 (63.16%), error:     10831.59277, best error:     10681.54004 on epoch:    55, epoch:   57/100, mu: 0.9661195874, learning_rate: 0.0000266837, weight_decay: 0.0000286018

epoch time: 1365.5925292969s, input_fired_time: 18.8485774994, output_fired_time: 7.6035943031, propagate_forward_time: 511.3376159668, propagate_backward_time: 811.5815429688, weight_update_time: 0.4477771819, other_time: 16.2210693359
epoch time: 111.2364730835s, input_fired_time: 1.3151667118, output_fired_time: 0.0000000000, propagate_forward_time: 107.2036972046, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7176055908
validation[          , genome  6154] predictions:    6243/  10000 (62.43%), best:    6316/10000 (63.16%), error:     10851.20410, best error:     10681.54004 on epoch:    55, epoch:   58/100, mu: 0.9673505425, learning_rate: 0.0000248800, weight_decay: 0.0000272604

epoch time: 1362.1251220703s, input_fired_time: 18.7495079041, output_fired_time: 7.5604596138, propagate_forward_time: 507.7528076172, propagate_backward_time: 811.9568481445, weight_update_time: 0.4369752407, other_time: 16.1055297852
epoch time: 115.8961639404s, input_fired_time: 1.3581923246, output_fired_time: 0.0000000000, propagate_forward_time: 111.6948547363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8431167603
validation[          , genome  6154] predictions:    6304/  10000 (63.04%), best:    6304/10000 (63.04%), error:     10629.25977, best error:     10629.25977 on epoch:    59, epoch:   59/100, mu: 0.9685180783, learning_rate: 0.0000231983, weight_decay: 0.0000259820

epoch time: 1368.1805419922s, input_fired_time: 18.7927932739, output_fired_time: 7.6305251122, propagate_forward_time: 509.8797912598, propagate_backward_time: 815.7806396484, weight_update_time: 0.4439752102, other_time: 16.0968017578
epoch time: 117.2729721069s, input_fired_time: 1.3872205019, output_fired_time: 0.0000000000, propagate_forward_time: 113.0384521484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8472976685
validation[          , genome  6154] predictions:    6314/  10000 (63.14%), best:    6304/10000 (63.04%), error:     10654.20898, best error:     10629.25977 on epoch:    59, epoch:   60/100, mu: 0.9696254134, learning_rate: 0.0000216302, weight_decay: 0.0000247635

epoch time: 1363.0509033203s, input_fired_time: 18.7913475037, output_fired_time: 7.5779957771, propagate_forward_time: 508.7764282227, propagate_backward_time: 811.6848754883, weight_update_time: 0.4493291080, other_time: 16.2202148438
epoch time: 107.9514846802s, input_fired_time: 1.2675853968, output_fired_time: 0.0000000000, propagate_forward_time: 104.0337142944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6501846313
validation[          , genome  6154] predictions:    6324/  10000 (63.24%), best:    6304/10000 (63.04%), error:     10666.70410, best error:     10629.25977 on epoch:    59, epoch:   61/100, mu: 0.9706756473, learning_rate: 0.0000201682, weight_decay: 0.0000236022

epoch time: 1351.5584716797s, input_fired_time: 18.5599403381, output_fired_time: 7.5101847649, propagate_forward_time: 504.7344665527, propagate_backward_time: 804.7368774414, weight_update_time: 0.4284999669, other_time: 16.0170288086
epoch time: 110.9027023315s, input_fired_time: 1.3090149164, output_fired_time: 0.0000000000, propagate_forward_time: 106.8660964966, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7275924683
validation[          , genome  6154] predictions:    6313/  10000 (63.13%), best:    6304/10000 (63.04%), error:     10718.25586, best error:     10629.25977 on epoch:    59, epoch:   62/100, mu: 0.9716717601, learning_rate: 0.0000188049, weight_decay: 0.0000224953

epoch time: 1353.1108398438s, input_fired_time: 18.6379814148, output_fired_time: 7.5152845383, propagate_forward_time: 504.3579406738, propagate_backward_time: 806.6578979492, weight_update_time: 0.4360822737, other_time: 15.9418334961
epoch time: 99.2514724731s, input_fired_time: 1.1716294289, output_fired_time: 0.0000000000, propagate_forward_time: 95.6031188965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4767227173
validation[          , genome  6154] predictions:    6326/  10000 (63.26%), best:    6326/10000 (63.26%), error:     10619.52832, best error:     10619.52832 on epoch:    63, epoch:   63/100, mu: 0.9726165533, learning_rate: 0.0000175338, weight_decay: 0.0000214403

epoch time: 1364.0991210938s, input_fired_time: 18.7635860443, output_fired_time: 7.5298957825, propagate_forward_time: 508.3013610840, propagate_backward_time: 813.2764282227, weight_update_time: 0.4414252937, other_time: 16.2278442383
epoch time: 106.4707794189s, input_fired_time: 1.2678363323, output_fired_time: 0.0000000000, propagate_forward_time: 102.5655364990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6374053955
validation[          , genome  6154] predictions:    6247/  10000 (62.47%), best:    6326/10000 (63.26%), error:     10826.70703, best error:     10619.52832 on epoch:    63, epoch:   64/100, mu: 0.9735126495, learning_rate: 0.0000163487, weight_decay: 0.0000204349

epoch time: 1360.2397460938s, input_fired_time: 18.7089328766, output_fired_time: 7.5054101944, propagate_forward_time: 507.7196350098, propagate_backward_time: 810.2788085938, weight_update_time: 0.4409412742, other_time: 16.0269775391
epoch time: 96.8079833984s, input_fired_time: 1.1951707602, output_fired_time: 0.0000000000, propagate_forward_time: 93.1994628906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4133529663
validation[          , genome  6154] predictions:    6315/  10000 (63.15%), best:    6326/10000 (63.26%), error:     10719.39453, best error:     10619.52832 on epoch:    63, epoch:   65/100, mu: 0.9743625522, learning_rate: 0.0000152436, weight_decay: 0.0000194765

epoch time: 1359.7264404297s, input_fired_time: 18.7875270844, output_fired_time: 7.5283384323, propagate_forward_time: 507.9488525391, propagate_backward_time: 809.4105834961, weight_update_time: 0.4279456139, other_time: 16.0512084961
epoch time: 101.3643951416s, input_fired_time: 1.2100914717, output_fired_time: 0.0000000000, propagate_forward_time: 97.6432189941, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5110855103
validation[          , genome  6154] predictions:    6338/  10000 (63.38%), best:    6338/10000 (63.38%), error:     10575.12305, best error:     10575.12305 on epoch:    66, epoch:   66/100, mu: 0.9751686454, learning_rate: 0.0000142132, weight_decay: 0.0000185631

epoch time: 1313.4503173828s, input_fired_time: 18.1385726929, output_fired_time: 7.3695836067, propagate_forward_time: 490.0526733398, propagate_backward_time: 782.3152465820, weight_update_time: 0.4197233021, other_time: 15.5742187500
epoch time: 86.5579986572s, input_fired_time: 1.0267015696, output_fired_time: 0.0000000000, propagate_forward_time: 83.3468704224, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1844253540
validation[          , genome  6154] predictions:    6350/  10000 (63.50%), best:    6338/10000 (63.38%), error:     10646.60742, best error:     10575.12305 on epoch:    66, epoch:   67/100, mu: 0.9759331346, learning_rate: 0.0000132525, weight_decay: 0.0000176926

epoch time: 986.7779541016s, input_fired_time: 13.7581996918, output_fired_time: 5.5326848030, propagate_forward_time: 368.7923583984, propagate_backward_time: 586.9822387695, weight_update_time: 0.3193685710, other_time: 11.7124633789
epoch time: 76.0718307495s, input_fired_time: 0.9156640768, output_fired_time: 0.0000000000, propagate_forward_time: 73.2851181030, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8710479736
validation[          , genome  6154] predictions:    6374/  10000 (63.74%), best:    6374/10000 (63.74%), error:     10570.25879, best error:     10570.25879 on epoch:    68, epoch:   68/100, mu: 0.9766582251, learning_rate: 0.0000123567, weight_decay: 0.0000168629

epoch time: 970.5375366211s, input_fired_time: 13.4527692795, output_fired_time: 5.4354753494, propagate_forward_time: 363.1869812012, propagate_backward_time: 576.8849487305, weight_update_time: 0.3158614039, other_time: 11.5773315430
epoch time: 75.6855163574s, input_fired_time: 0.8993564248, output_fired_time: 0.0000000000, propagate_forward_time: 72.8888320923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8973312378
validation[          , genome  6154] predictions:    6292/  10000 (62.92%), best:    6374/10000 (63.74%), error:     10681.48145, best error:     10570.25879 on epoch:    68, epoch:   69/100, mu: 0.9773459435, learning_rate: 0.0000115215, weight_decay: 0.0000160720

epoch time: 964.1451416016s, input_fired_time: 13.3784437180, output_fired_time: 5.4201707840, propagate_forward_time: 360.0000000000, propagate_backward_time: 573.9074096680, weight_update_time: 0.2975087166, other_time: 11.4391479492
epoch time: 75.7250442505s, input_fired_time: 0.9082774520, output_fired_time: 0.0000000000, propagate_forward_time: 72.9155807495, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9011840820
validation[          , genome  6154] predictions:    6388/  10000 (63.88%), best:    6388/10000 (63.88%), error:     10548.53320, best error:     10548.53320 on epoch:    70, epoch:   70/100, mu: 0.9779981971, learning_rate: 0.0000107427, weight_decay: 0.0000153183

epoch time: 964.9468383789s, input_fired_time: 13.4444293976, output_fired_time: 5.4131627083, propagate_forward_time: 360.0780334473, propagate_backward_time: 574.4767456055, weight_update_time: 0.3143835068, other_time: 11.5344848633
epoch time: 75.7455978394s, input_fired_time: 0.9192838073, output_fired_time: 0.0000000000, propagate_forward_time: 72.9358291626, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8904876709
validation[          , genome  6154] predictions:    6320/  10000 (63.20%), best:    6388/10000 (63.88%), error:     10679.78906, best error:     10548.53320 on epoch:    70, epoch:   71/100, mu: 0.9786168337, learning_rate: 0.0000100166, weight_decay: 0.0000145999

epoch time: 965.1597900391s, input_fired_time: 13.4161329269, output_fired_time: 5.4464144707, propagate_forward_time: 360.9809570313, propagate_backward_time: 573.8377075195, weight_update_time: 0.3142511249, other_time: 11.4785766602
epoch time: 75.4474487305s, input_fired_time: 0.8993039727, output_fired_time: 0.0000000000, propagate_forward_time: 72.6619644165, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8861770630
validation[          , genome  6154] predictions:    6349/  10000 (63.49%), best:    6388/10000 (63.88%), error:     10571.38574, best error:     10548.53320 on epoch:    70, epoch:   72/100, mu: 0.9792035818, learning_rate: 0.0000093395, weight_decay: 0.0000139152

epoch time: 966.1693725586s, input_fired_time: 13.4720449448, output_fired_time: 5.4231767654, propagate_forward_time: 362.3043518066, propagate_backward_time: 573.6132202148, weight_update_time: 0.3165087402, other_time: 11.3566284180
epoch time: 75.5191650391s, input_fired_time: 0.9111778140, output_fired_time: 0.0000000000, propagate_forward_time: 72.7101211548, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8978652954
validation[          , genome  6154] predictions:    6362/  10000 (63.62%), best:    6362/10000 (63.62%), error:     10547.19141, best error:     10547.19141 on epoch:    73, epoch:   73/100, mu: 0.9797601104, learning_rate: 0.0000087082, weight_decay: 0.0000132626

epoch time: 964.0614013672s, input_fired_time: 13.4578170776, output_fired_time: 5.3912897110, propagate_forward_time: 360.5075683594, propagate_backward_time: 573.2774047852, weight_update_time: 0.3143243492, other_time: 11.4273071289
epoch time: 75.6965332031s, input_fired_time: 0.9221550226, output_fired_time: 0.0000000000, propagate_forward_time: 72.8488082886, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9255676270
validation[          , genome  6154] predictions:    6383/  10000 (63.83%), best:    6383/10000 (63.83%), error:     10519.53125, best error:     10519.53125 on epoch:    74, epoch:   74/100, mu: 0.9802879691, learning_rate: 0.0000081196, weight_decay: 0.0000126407

epoch time: 970.1790771484s, input_fired_time: 13.4640712738, output_fired_time: 5.4663619995, propagate_forward_time: 363.2291564941, propagate_backward_time: 576.5620727539, weight_update_time: 0.3108648956, other_time: 11.4574584961
epoch time: 75.2839584351s, input_fired_time: 0.9040777087, output_fired_time: 0.0000000000, propagate_forward_time: 72.5195465088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8603363037
validation[          , genome  6154] predictions:    6341/  10000 (63.41%), best:    6383/10000 (63.83%), error:     10606.20020, best error:     10519.53125 on epoch:    74, epoch:   75/100, mu: 0.9807885885, learning_rate: 0.0000075708, weight_decay: 0.0000120479

epoch time: 963.9024047852s, input_fired_time: 13.4265060425, output_fired_time: 5.4151239395, propagate_forward_time: 360.5911254883, propagate_backward_time: 572.9672851563, weight_update_time: 0.3172502518, other_time: 11.5023803711
epoch time: 75.3360366821s, input_fired_time: 0.9080849290, output_fired_time: 0.0000000000, propagate_forward_time: 72.5158767700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9120712280
validation[          , genome  6154] predictions:    6370/  10000 (63.70%), best:    6383/10000 (63.83%), error:     10536.62109, best error:     10519.53125 on epoch:    74, epoch:   76/100, mu: 0.9812633991, learning_rate: 0.0000070590, weight_decay: 0.0000114829

epoch time: 963.6395263672s, input_fired_time: 13.4115066528, output_fired_time: 5.4174890518, propagate_forward_time: 360.2461853027, propagate_backward_time: 573.1075439453, weight_update_time: 0.3179821074, other_time: 11.4567871094
epoch time: 76.1862945557s, input_fired_time: 0.9354494810, output_fired_time: 0.0000000000, propagate_forward_time: 73.3469543457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9038925171
validation[          , genome  6154] predictions:    6401/  10000 (64.01%), best:    6383/10000 (63.83%), error:     10557.82324, best error:     10519.53125 on epoch:    74, epoch:   77/100, mu: 0.9817137718, learning_rate: 0.0000065819, weight_decay: 0.0000109443

epoch time: 979.6434326172s, input_fired_time: 13.6877374649, output_fired_time: 5.5029292107, propagate_forward_time: 365.1318969727, propagate_backward_time: 583.6834106445, weight_update_time: 0.3194885254, other_time: 11.6374511719
epoch time: 75.3368453979s, input_fired_time: 0.9125421643, output_fired_time: 0.0000000000, propagate_forward_time: 72.5726165771, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8516845703
validation[          , genome  6154] predictions:    6358/  10000 (63.58%), best:    6383/10000 (63.83%), error:     10630.56738, best error:     10519.53125 on epoch:    74, epoch:   78/100, mu: 0.9821408987, learning_rate: 0.0000061370, weight_decay: 0.0000104311

epoch time: 996.3813476563s, input_fired_time: 13.8734626770, output_fired_time: 5.6085586548, propagate_forward_time: 372.0702819824, propagate_backward_time: 592.9848632813, weight_update_time: 0.3195178509, other_time: 11.8441162109
epoch time: 75.8183975220s, input_fired_time: 0.9199864864, output_fired_time: 0.0000000000, propagate_forward_time: 73.0064697266, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8919448853
validation[          , genome  6154] predictions:    6353/  10000 (63.53%), best:    6383/10000 (63.83%), error:     10661.89551, best error:     10519.53125 on epoch:    74, epoch:   79/100, mu: 0.9825460315, learning_rate: 0.0000057222, weight_decay: 0.0000099419

epoch time: 965.3033447266s, input_fired_time: 13.4533357620, output_fired_time: 5.4246511459, propagate_forward_time: 361.1643371582, propagate_backward_time: 573.7968139648, weight_update_time: 0.3074505329, other_time: 11.4642944336
epoch time: 75.3835296631s, input_fired_time: 0.9211087227, output_fired_time: 0.0000000000, propagate_forward_time: 72.5880050659, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8744125366
validation[          , genome  6154] predictions:    6363/  10000 (63.63%), best:    6383/10000 (63.83%), error:     10571.16113, best error:     10519.53125 on epoch:    74, epoch:   80/100, mu: 0.9829302430, learning_rate: 0.0000053354, weight_decay: 0.0000094757

epoch time: 963.8661499023s, input_fired_time: 13.4434251785, output_fired_time: 5.4355177879, propagate_forward_time: 359.9093322754, propagate_backward_time: 573.5888671875, weight_update_time: 0.3163797259, other_time: 11.4890136719
epoch time: 75.0903701782s, input_fired_time: 0.8985013366, output_fired_time: 0.0000000000, propagate_forward_time: 72.2821884155, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9096832275
validation[          , genome  6154] predictions:    6395/  10000 (63.95%), best:    6395/10000 (63.95%), error:     10509.19141, best error:     10509.19141 on epoch:    81, epoch:   81/100, mu: 0.9832946658, learning_rate: 0.0000049747, weight_decay: 0.0000090313

epoch time: 967.2203369141s, input_fired_time: 13.4757099152, output_fired_time: 5.4464650154, propagate_forward_time: 361.2934265137, propagate_backward_time: 575.5176391602, weight_update_time: 0.3195737302, other_time: 11.4871215820
epoch time: 75.3849563599s, input_fired_time: 0.9138821959, output_fired_time: 0.0000000000, propagate_forward_time: 72.5880966187, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8829803467
validation[          , genome  6154] predictions:    6338/  10000 (63.38%), best:    6395/10000 (63.95%), error:     10533.95117, best error:     10509.19141 on epoch:    81, epoch:   82/100, mu: 0.9836403131, learning_rate: 0.0000046385, weight_decay: 0.0000086077

epoch time: 965.9077758789s, input_fired_time: 13.4677543640, output_fired_time: 5.4706478119, propagate_forward_time: 361.2948608398, propagate_backward_time: 574.1776123047, weight_update_time: 0.3189229071, other_time: 11.4968872070
epoch time: 75.7313995361s, input_fired_time: 0.9209577441, output_fired_time: 0.0000000000, propagate_forward_time: 72.8440551758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9663848877
validation[          , genome  6154] predictions:    6398/  10000 (63.98%), best:    6395/10000 (63.95%), error:     10523.01367, best error:     10509.19141 on epoch:    81, epoch:   83/100, mu: 0.9839681387, learning_rate: 0.0000043250, weight_decay: 0.0000082041

epoch time: 964.7758178711s, input_fired_time: 13.4201383591, output_fired_time: 5.4286732674, propagate_forward_time: 360.7488403320, propagate_backward_time: 573.6257324219, weight_update_time: 0.3153854311, other_time: 11.5524291992
epoch time: 75.3691253662s, input_fired_time: 0.9068250060, output_fired_time: 0.0000000000, propagate_forward_time: 72.5780563354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8842468262
validation[          , genome  6154] predictions:    6341/  10000 (63.41%), best:    6395/10000 (63.95%), error:     10558.81152, best error:     10509.19141 on epoch:    81, epoch:   84/100, mu: 0.9842790365, learning_rate: 0.0000040326, weight_decay: 0.0000078193

epoch time: 963.7893676758s, input_fired_time: 13.4673328400, output_fired_time: 5.4374623299, propagate_forward_time: 359.3756713867, propagate_backward_time: 574.0113525391, weight_update_time: 0.3141739368, other_time: 11.4975585938
epoch time: 75.1560058594s, input_fired_time: 0.9059628248, output_fired_time: 0.0000000000, propagate_forward_time: 72.3678054810, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8822402954
validation[          , genome  6154] predictions:    6380/  10000 (63.80%), best:    6395/10000 (63.95%), error:     10523.76953, best error:     10509.19141 on epoch:    81, epoch:   85/100, mu: 0.9845739603, learning_rate: 0.0000037600, weight_decay: 0.0000074526

epoch time: 966.7617797852s, input_fired_time: 13.4529800415, output_fired_time: 5.4295120239, propagate_forward_time: 361.1267089844, propagate_backward_time: 575.2287597656, weight_update_time: 0.3153369725, other_time: 11.5238037109
epoch time: 80.0536422729s, input_fired_time: 0.9514373541, output_fired_time: 0.0000000000, propagate_forward_time: 77.1249008179, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9773025513
validation[          , genome  6154] predictions:    6402/  10000 (64.02%), best:    6402/10000 (64.02%), error:     10504.37891, best error:     10504.37891 on epoch:    86, epoch:   86/100, mu: 0.9848536849, learning_rate: 0.0000035059, weight_decay: 0.0000071031

epoch time: 1094.3081054688s, input_fired_time: 14.9693288803, output_fired_time: 5.9459218979, propagate_forward_time: 421.5343933105, propagate_backward_time: 639.0232543945, weight_update_time: 0.3396554589, other_time: 12.8351440430
epoch time: 89.1333541870s, input_fired_time: 1.0808876753, output_fired_time: 0.0000000000, propagate_forward_time: 85.8659896851, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1864776611
validation[          , genome  6154] predictions:    6400/  10000 (64.00%), best:    6400/10000 (64.00%), error:     10503.76855, best error:     10503.76855 on epoch:    87, epoch:   87/100, mu: 0.9851189852, learning_rate: 0.0000032689, weight_decay: 0.0000067700

epoch time: 1249.4288330078s, input_fired_time: 17.4390773773, output_fired_time: 7.0593833923, propagate_forward_time: 467.0372619629, propagate_backward_time: 743.0501098633, weight_update_time: 0.4047079384, other_time: 14.8430786133
epoch time: 101.2999877930s, input_fired_time: 1.2094887495, output_fired_time: 0.0000000000, propagate_forward_time: 97.5271835327, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5633163452
validation[          , genome  6154] predictions:    6343/  10000 (63.43%), best:    6400/10000 (64.00%), error:     10567.30957, best error:     10503.76855 on epoch:    87, epoch:   88/100, mu: 0.9853705764, learning_rate: 0.0000030479, weight_decay: 0.0000064525

epoch time: 1302.2293701172s, input_fired_time: 18.1495704651, output_fired_time: 7.2509369850, propagate_forward_time: 486.4542541504, propagate_backward_time: 774.8994140625, weight_update_time: 0.4276331663, other_time: 15.4752197266
epoch time: 101.8394470215s, input_fired_time: 1.2314393520, output_fired_time: 0.0000000000, propagate_forward_time: 98.0809249878, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5270843506
validation[          , genome  6154] predictions:    6412/  10000 (64.12%), best:    6412/10000 (64.12%), error:     10486.12695, best error:     10486.12695 on epoch:    89, epoch:   89/100, mu: 0.9856092334, learning_rate: 0.0000028419, weight_decay: 0.0000061499

epoch time: 1327.7067871094s, input_fired_time: 18.4302711487, output_fired_time: 7.3932051659, propagate_forward_time: 496.2925415039, propagate_backward_time: 789.7781372070, weight_update_time: 0.4233274758, other_time: 15.8126220703
epoch time: 105.2213516235s, input_fired_time: 1.2623958588, output_fired_time: 0.0000000000, propagate_forward_time: 101.3575973511, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6013565063
validation[          , genome  6154] predictions:    6382/  10000 (63.82%), best:    6412/10000 (64.12%), error:     10517.11328, best error:     10486.12695 on epoch:    89, epoch:   90/100, mu: 0.9858355522, learning_rate: 0.0000026498, weight_decay: 0.0000058615

epoch time: 1311.9847412109s, input_fired_time: 18.2488231659, output_fired_time: 7.2262125015, propagate_forward_time: 490.1327819824, propagate_backward_time: 780.8029174805, weight_update_time: 0.4124082327, other_time: 15.5740356445
epoch time: 96.5908660889s, input_fired_time: 1.1601065397, output_fired_time: 0.0000000000, propagate_forward_time: 93.0273513794, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4034118652
validation[          , genome  6154] predictions:    6414/  10000 (64.14%), best:    6414/10000 (64.14%), error:     10479.00586, best error:     10479.00586 on epoch:    91, epoch:   91/100, mu: 0.9860501885, learning_rate: 0.0000024707, weight_decay: 0.0000055866

epoch time: 1322.0845947266s, input_fired_time: 18.3652858734, output_fired_time: 7.2695174217, propagate_forward_time: 494.3716430664, propagate_backward_time: 786.5003051758, weight_update_time: 0.4239189029, other_time: 15.5778808594
epoch time: 107.6943511963s, input_fired_time: 1.2755076885, output_fired_time: 0.0000000000, propagate_forward_time: 103.7715988159, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6472473145
validation[          , genome  6154] predictions:    6407/  10000 (64.07%), best:    6414/10000 (64.14%), error:     10512.44043, best error:     10479.00586 on epoch:    91, epoch:   92/100, mu: 0.9862537980, learning_rate: 0.0000023037, weight_decay: 0.0000053246

epoch time: 1360.3499755859s, input_fired_time: 18.8979015350, output_fired_time: 7.5228524208, propagate_forward_time: 507.8636169434, propagate_backward_time: 809.8295898438, weight_update_time: 0.4237423539, other_time: 16.2359619141
epoch time: 106.2399673462s, input_fired_time: 1.2646044493, output_fired_time: 0.0000000000, propagate_forward_time: 102.3242874146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6510772705
validation[          , genome  6154] predictions:    6430/  10000 (64.30%), best:    6430/10000 (64.30%), error:     10463.11328, best error:     10463.11328 on epoch:    93, epoch:   93/100, mu: 0.9864469171, learning_rate: 0.0000021480, weight_decay: 0.0000050749

epoch time: 2287.6535644531s, input_fired_time: 19.8621940613, output_fired_time: 8.0793962479, propagate_forward_time: 1313.6600341797, propagate_backward_time: 928.9271240234, weight_update_time: 0.4466630220, other_time: 17.1247558594
epoch time: 119.6092681885s, input_fired_time: 1.4384248257, output_fired_time: 0.0000000000, propagate_forward_time: 115.2037811279, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9670639038
validation[          , genome  6154] predictions:    6387/  10000 (63.87%), best:    6430/10000 (64.30%), error:     10517.74219, best error:     10463.11328 on epoch:    93, epoch:   94/100, mu: 0.9866300821, learning_rate: 0.0000020028, weight_decay: 0.0000048369

epoch time: 1516.7829589844s, input_fired_time: 19.7598533630, output_fired_time: 7.9597721100, propagate_forward_time: 615.0758056641, propagate_backward_time: 856.5306396484, weight_update_time: 0.4843807220, other_time: 17.4569091797
epoch time: 114.2883071899s, input_fired_time: 1.4115049839, output_fired_time: 0.0000000000, propagate_forward_time: 110.0981063843, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7786941528
validation[          , genome  6154] predictions:    6412/  10000 (64.12%), best:    6430/10000 (64.30%), error:     10503.72461, best error:     10463.11328 on epoch:    93, epoch:   95/100, mu: 0.9868037701, learning_rate: 0.0000018674, weight_decay: 0.0000046101

epoch time: 1416.6860351563s, input_fired_time: 19.4560508728, output_fired_time: 7.7585859299, propagate_forward_time: 536.1076660156, propagate_backward_time: 836.8550415039, weight_update_time: 0.4415785968, other_time: 16.5087280273
epoch time: 107.8597564697s, input_fired_time: 1.2945016623, output_fired_time: 0.0000000000, propagate_forward_time: 103.8733825684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6918716431
validation[          , genome  6154] predictions:    6420/  10000 (64.20%), best:    6430/10000 (64.30%), error:     10474.00684, best error:     10463.11328 on epoch:    93, epoch:   96/100, mu: 0.9869685173, learning_rate: 0.0000017412, weight_decay: 0.0000043939

epoch time: 1395.1293945313s, input_fired_time: 19.4631805420, output_fired_time: 7.7117128372, propagate_forward_time: 523.0151367188, propagate_backward_time: 828.5137329102, weight_update_time: 0.4365062118, other_time: 16.4257202148
epoch time: 101.2789993286s, input_fired_time: 1.2176481485, output_fired_time: 0.0000000000, propagate_forward_time: 97.5425720215, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5187759399
validation[          , genome  6154] predictions:    6406/  10000 (64.06%), best:    6430/10000 (64.30%), error:     10472.75586, best error:     10463.11328 on epoch:    93, epoch:   97/100, mu: 0.9871248007, learning_rate: 0.0000016235, weight_decay: 0.0000041878

epoch time: 1420.2291259766s, input_fired_time: 19.6578998566, output_fired_time: 7.8811717033, propagate_forward_time: 529.0244140625, propagate_backward_time: 846.9156494141, weight_update_time: 0.4436320364, other_time: 16.7498779297
epoch time: 111.4549331665s, input_fired_time: 1.3253524303, output_fired_time: 0.0000000000, propagate_forward_time: 107.3325958252, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7969818115
validation[          , genome  6154] predictions:    6398/  10000 (63.98%), best:    6398/10000 (63.98%), error:     10452.81152, best error:     10452.81152 on epoch:    98, epoch:   98/100, mu: 0.9872730374, learning_rate: 0.0000015138, weight_decay: 0.0000039914

epoch time: 1434.7030029297s, input_fired_time: 19.7904148102, output_fired_time: 7.9091520309, propagate_forward_time: 540.1340942383, propagate_backward_time: 849.9799194336, weight_update_time: 0.4458760321, other_time: 16.8894042969
epoch time: 116.2717361450s, input_fired_time: 1.4027999640, output_fired_time: 0.0000000000, propagate_forward_time: 111.9684219360, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9005126953
validation[          , genome  6154] predictions:    6394/  10000 (63.94%), best:    6398/10000 (63.98%), error:     10463.96777, best error:     10452.81152 on epoch:    98, epoch:   99/100, mu: 0.9874135852, learning_rate: 0.0000014114, weight_decay: 0.0000038042

epoch time: 1452.6341552734s, input_fired_time: 20.0927238464, output_fired_time: 8.0716505051, propagate_forward_time: 542.4581298828, propagate_backward_time: 864.7841796875, weight_update_time: 0.4638342857, other_time: 17.2274169922
epoch time: 110.4144287109s, input_fired_time: 1.3367651701, output_fired_time: 0.0000000000, propagate_forward_time: 106.3383102417, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7393569946
validation[          , genome  6154] predictions:    6404/  10000 (64.04%), best:    6398/10000 (63.98%), error:     10462.65430, best error:     10452.81152 on epoch:    98, epoch:  100/100, mu: 0.9875469208, learning_rate: 0.0000013160, weight_decay: 0.0000036258

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 559.7781982422s, input_fired_time: 6.7669677734, output_fired_time: 0.0000000000, propagate_forward_time: 539.0410766602, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.9701538086
best training[          , genome  6154] predictions:   34820/  50000 (69.64%), best:    6398/10000 (63.98%), error:     46299.85547, best error:     10452.81152 on epoch:    98, epoch:  101/100, mu: 0.9876733422, learning_rate: 0.0000012271, weight_decay: 0.0000034558
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 118.5307235718s, input_fired_time: 1.4072777033, output_fired_time: 0.0000000000, propagate_forward_time: 113.7276840210, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3957595825
   testing[          , genome  6154] predictions:    6398/  10000 (63.98%), best:    6398/10000 (63.98%), error:     10452.81152, best error:     10452.81152 on epoch:    98, epoch:  101/100, mu: 0.9876733422, learning_rate: 0.0000012271, weight_decay: 0.0000034558
backpropagation finished successfully!
22:46:33 (4168): called boinc_finish(0)

</stderr_txt>
]]>