Nameexact_genome_1542692703_186_6408_1
Workunit4100812
Created20 Nov 2018, 10:32:48 UTC
Sent20 Nov 2018, 11:17:21 UTC
Report deadline12 Feb 2019, 3:59:09 UTC
Received10 Jul 2019, 23:58:26 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID85311
Run time1 days 21 hours 38 min 48 sec
CPU time1 days 20 hours 57 min 5 sec
Validate stateValid
Credit20,923.93
Device peak FLOPS2.89 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size537.50 MB
Peak swap size557.11 MB
Peak disk usage10.02 MB

Stderr output

<core_client_version>7.10.2</core_client_version>
<![CDATA[
<stderr_txt>
ent_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_186_6408.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_186_6408_1_r1151615183_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.543628
read mu: 0.888936
read mu_delta: 0.934711
read initial_learning_rate: 0.00308857
read learning_rate: 0.000739443
read learning_rate_delta: 0.937086
read initial_weight_decay: 0.000382041
read weight_decay: 0.000142824
read weight_decay_delta: 0.956262
read batch_size: 89
read epsilon: 1e-07
read alpha: 0.0777069
read input_dropout_probability: 0.00116734
read hidden_dropout_probability: 0.109332
read velocity_reset: 1923
read epoch: 22
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 20
read number_validation_images: 10000
read best_validation_predictions: 6155
read best_validation_error: 11188.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6408
read normal distribution: '1 0 0'
generator_str: '1819224529'
read generator: 1819224529
read generated_by_map:
1 disable_node 1
reading 74 nodes.
reading 277 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 6903.42333984375s, input_fired_time: 21.0229587554932, output_fired_time: 9.53843212127686, propagate_forward_time: 4868.58447265625, propagate_backward_time: 1981.16052246094, weight_update_time: 0.970631241798401, other_time: 23.1168212890625
epoch time: 147.001724243164s, input_fired_time: 1.68755614757538, output_fired_time: 0, propagate_forward_time: 141.578903198242, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.73526000976563
validation[          , genome  6408] predictions:    6110/  10000 (61.10%), best:    6155/10000 (61.55%), error:     11246.40527, best error:     11188.93652 on epoch:    20, epoch:   22/100, mu: 0.8889356256, learning_rate: 0.0007394433, weight_decay: 0.0001428235

epoch time: 1818.9171142578s, input_fired_time: 22.9605045319, output_fired_time: 10.2668914795, propagate_forward_time: 677.5398559570, propagate_backward_time: 1089.5314941406, weight_update_time: 1.0626345873, other_time: 18.6184082031
epoch time: 141.5287933350s, input_fired_time: 1.6664280891, output_fired_time: 0.0000000000, propagate_forward_time: 136.7015838623, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1607818604
validation[          , genome  6408] predictions:    6002/  10000 (60.02%), best:    6155/10000 (61.55%), error:     11549.37793, best error:     11188.93652 on epoch:    20, epoch:   23/100, mu: 0.8955340385, learning_rate: 0.0006929220, weight_decay: 0.0001365767

epoch time: 1873.8096923828s, input_fired_time: 23.6602058411, output_fired_time: 10.5976657867, propagate_forward_time: 697.6293945313, propagate_backward_time: 1122.6831054688, weight_update_time: 1.0752314329, other_time: 19.2393798828
epoch time: 147.9846191406s, input_fired_time: 1.7237176895, output_fired_time: 0.0000000000, propagate_forward_time: 143.0092773438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2516174316
validation[          , genome  6408] predictions:    6215/  10000 (62.15%), best:    6215/10000 (62.15%), error:     10920.07031, best error:     10920.07031 on epoch:    24, epoch:   24/100, mu: 0.9017016292, learning_rate: 0.0006493275, weight_decay: 0.0001306031

epoch time: 1936.5100097656s, input_fired_time: 24.4434413910, output_fired_time: 11.0323066711, propagate_forward_time: 720.0404052734, propagate_backward_time: 1161.0206298828, weight_update_time: 1.0930991173, other_time: 19.9731445313
epoch time: 150.5881347656s, input_fired_time: 1.7397727966, output_fired_time: 0.0000000000, propagate_forward_time: 145.5469360352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3014221191
validation[          , genome  6408] predictions:    6206/  10000 (62.06%), best:    6215/10000 (62.15%), error:     10980.27734, best error:     10920.07031 on epoch:    24, epoch:   25/100, mu: 0.9074665308, learning_rate: 0.0006084756, weight_decay: 0.0001248907

epoch time: 1999.4096679688s, input_fired_time: 25.1883850098, output_fired_time: 11.3124952316, propagate_forward_time: 744.2117919922, propagate_backward_time: 1198.1269531250, weight_update_time: 1.1539443731, other_time: 20.5700683594
epoch time: 155.7335815430s, input_fired_time: 1.7917979956, output_fired_time: 0.0000000000, propagate_forward_time: 150.5459442139, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3958435059
validation[          , genome  6408] predictions:    5996/  10000 (59.96%), best:    6215/10000 (62.15%), error:     11487.19629, best error:     10920.07031 on epoch:    24, epoch:   26/100, mu: 0.9128550887, learning_rate: 0.0005701940, weight_decay: 0.0001194282

epoch time: 2007.8194580078s, input_fired_time: 25.3153362274, output_fired_time: 11.4173164368, propagate_forward_time: 747.0577392578, propagate_backward_time: 1203.3657226563, weight_update_time: 1.1652735472, other_time: 20.6633300781
epoch time: 155.4084472656s, input_fired_time: 1.8111519814, output_fired_time: 0.0000000000, propagate_forward_time: 150.1585083008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4387817383
validation[          , genome  6408] predictions:    6109/  10000 (61.09%), best:    6215/10000 (62.15%), error:     11287.13965, best error:     10920.07031 on epoch:    24, epoch:   27/100, mu: 0.9178918004, learning_rate: 0.0005343208, weight_decay: 0.0001142047

epoch time: 2011.9749755859s, input_fired_time: 25.3844947815, output_fired_time: 11.4047384262, propagate_forward_time: 747.5234375000, propagate_backward_time: 1207.1151123047, weight_update_time: 1.1463769674, other_time: 20.5471191406
epoch time: 155.5062408447s, input_fired_time: 1.8152446747, output_fired_time: 0.0000000000, propagate_forward_time: 150.2718200684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4191741943
validation[          , genome  6408] predictions:    6131/  10000 (61.31%), best:    6215/10000 (62.15%), error:     11095.20117, best error:     10920.07031 on epoch:    24, epoch:   28/100, mu: 0.9225996733, learning_rate: 0.0005007045, weight_decay: 0.0001092096

epoch time: 2016.4980468750s, input_fired_time: 25.4171276093, output_fired_time: 11.3771018982, propagate_forward_time: 751.5667114258, propagate_backward_time: 1207.4848632813, weight_update_time: 1.1449939013, other_time: 20.6523437500
epoch time: 158.1596069336s, input_fired_time: 1.8530058861, output_fired_time: 0.0000000000, propagate_forward_time: 152.8747253418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4318695068
validation[          , genome  6408] predictions:    6172/  10000 (61.72%), best:    6215/10000 (62.15%), error:     11045.01953, best error:     10920.07031 on epoch:    24, epoch:   29/100, mu: 0.9270001650, learning_rate: 0.0004692032, weight_decay: 0.0001044329

epoch time: 2052.5832519531s, input_fired_time: 25.9040832520, output_fired_time: 11.7679548264, propagate_forward_time: 765.1346435547, propagate_backward_time: 1228.7048339844, weight_update_time: 1.1859509945, other_time: 21.0717773438
epoch time: 158.9373626709s, input_fired_time: 1.8725063801, output_fired_time: 0.0000000000, propagate_forward_time: 153.6396484375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4252014160
validation[          , genome  6408] predictions:    6130/  10000 (61.30%), best:    6215/10000 (62.15%), error:     11189.14551, best error:     10920.07031 on epoch:    24, epoch:   30/100, mu: 0.9311133623, learning_rate: 0.0004396837, weight_decay: 0.0000998652

epoch time: 2057.7246093750s, input_fired_time: 25.9287452698, output_fired_time: 11.7266407013, propagate_forward_time: 765.5787353516, propagate_backward_time: 1233.3880615234, weight_update_time: 1.1592457294, other_time: 21.1024169922
epoch time: 158.5458068848s, input_fired_time: 1.8492918015, output_fired_time: 0.0000000000, propagate_forward_time: 153.1252136230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5713043213
validation[          , genome  6408] predictions:    6137/  10000 (61.37%), best:    6215/10000 (62.15%), error:     11130.58496, best error:     10920.07031 on epoch:    24, epoch:   31/100, mu: 0.9349580407, learning_rate: 0.0004120215, weight_decay: 0.0000954973

epoch time: 2077.9101562500s, input_fired_time: 26.1756973267, output_fired_time: 11.8287982941, propagate_forward_time: 773.7827758789, propagate_backward_time: 1244.9707031250, weight_update_time: 1.1819190979, other_time: 21.1520996094
epoch time: 161.0613555908s, input_fired_time: 1.8887941837, output_fired_time: 0.0000000000, propagate_forward_time: 155.5538177490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6187438965
validation[          , genome  6408] predictions:    6154/  10000 (61.54%), best:    6215/10000 (62.15%), error:     10936.86523, best error:     10920.07031 on epoch:    24, epoch:   32/100, mu: 0.9385516644, learning_rate: 0.0003860995, weight_decay: 0.0000913204

epoch time: 2089.5166015625s, input_fired_time: 26.2869510651, output_fired_time: 11.8680820465, propagate_forward_time: 779.7357177734, propagate_backward_time: 1250.2677001953, weight_update_time: 1.1986926794, other_time: 21.3581542969
epoch time: 156.5189819336s, input_fired_time: 1.8330221176, output_fired_time: 0.0000000000, propagate_forward_time: 151.2778320313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4081268311
validation[          , genome  6408] predictions:    6052/  10000 (60.52%), best:    6215/10000 (62.15%), error:     11206.06543, best error:     10920.07031 on epoch:    24, epoch:   33/100, mu: 0.9419106841, learning_rate: 0.0003618085, weight_decay: 0.0000873262

epoch time: 2099.7702636719s, input_fired_time: 26.4554519653, output_fired_time: 11.8840980530, propagate_forward_time: 782.8604125977, propagate_backward_time: 1257.2719726563, weight_update_time: 1.1943519115, other_time: 21.2983398438
epoch time: 153.4515075684s, input_fired_time: 1.7950091362, output_fired_time: 0.0000000000, propagate_forward_time: 148.2567291260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3997650146
validation[          , genome  6408] predictions:    6253/  10000 (62.53%), best:    6253/10000 (62.53%), error:     10877.75293, best error:     10877.75293 on epoch:    34, epoch:   34/100, mu: 0.9450504184, learning_rate: 0.0003390456, weight_decay: 0.0000835067

epoch time: 2109.7890625000s, input_fired_time: 26.5137958527, output_fired_time: 12.0237703323, propagate_forward_time: 786.5152587891, propagate_backward_time: 1263.2795410156, weight_update_time: 1.1787253618, other_time: 21.4566650391
epoch time: 166.6824951172s, input_fired_time: 1.9574872255, output_fired_time: 0.0000000000, propagate_forward_time: 161.1252593994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5997467041
validation[          , genome  6408] predictions:    6243/  10000 (62.43%), best:    6243/10000 (62.43%), error:     10734.83008, best error:     10734.83008 on epoch:    35, epoch:   35/100, mu: 0.9479851127, learning_rate: 0.0003177149, weight_decay: 0.0000798543

epoch time: 1639.9049072266s, input_fired_time: 22.6411266327, output_fired_time: 9.3735322952, propagate_forward_time: 601.1845703125, propagate_backward_time: 988.7195434570, weight_update_time: 0.9957257509, other_time: 17.9861450195
epoch time: 114.5452194214s, input_fired_time: 1.4219906330, output_fired_time: 0.0000000000, propagate_forward_time: 110.3766555786, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7465744019
validation[          , genome  6408] predictions:    6193/  10000 (61.93%), best:    6243/10000 (62.43%), error:     11022.07227, best error:     10734.83008 on epoch:    35, epoch:   36/100, mu: 0.9507282376, learning_rate: 0.0002977262, weight_decay: 0.0000763616

epoch time: 1549.3554687500s, input_fired_time: 22.0629539490, output_fired_time: 8.9945335388, propagate_forward_time: 566.6793823242, propagate_backward_time: 934.5310058594, weight_update_time: 0.9542998672, other_time: 17.0875854492
epoch time: 114.0874176025s, input_fired_time: 1.4201025963, output_fired_time: 0.0000000000, propagate_forward_time: 109.9428634644, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7244491577
validation[          , genome  6408] predictions:    6249/  10000 (62.49%), best:    6243/10000 (62.43%), error:     10823.73438, best error:     10734.83008 on epoch:    35, epoch:   37/100, mu: 0.9532922506, learning_rate: 0.0002789951, weight_decay: 0.0000730217

epoch time: 1567.2102050781s, input_fired_time: 22.3909397125, output_fired_time: 9.0672273636, propagate_forward_time: 573.4312133789, propagate_backward_time: 945.1123046875, weight_update_time: 0.9553731084, other_time: 17.2084350586
epoch time: 122.9832229614s, input_fired_time: 1.5144817829, output_fired_time: 0.0000000000, propagate_forward_time: 118.5878219604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8809204102
validation[          , genome  6408] predictions:    6088/  10000 (60.88%), best:    6243/10000 (62.43%), error:     11288.81445, best error:     10734.83008 on epoch:    35, epoch:   38/100, mu: 0.9556888938, learning_rate: 0.0002614424, weight_decay: 0.0000698279

epoch time: 1647.3743896484s, input_fired_time: 23.3809223175, output_fired_time: 9.4466810226, propagate_forward_time: 600.8064575195, propagate_backward_time: 995.9676513672, weight_update_time: 0.9925886989, other_time: 17.7726440430
epoch time: 120.9293441772s, input_fired_time: 1.5268093348, output_fired_time: 0.0000000000, propagate_forward_time: 116.5405654907, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8619689941
validation[          , genome  6408] predictions:    6033/  10000 (60.33%), best:    6243/10000 (62.43%), error:     11356.32520, best error:     10734.83008 on epoch:    35, epoch:   39/100, mu: 0.9579290152, learning_rate: 0.0002449940, weight_decay: 0.0000667737

epoch time: 1583.1203613281s, input_fired_time: 22.5496902466, output_fired_time: 9.0857524872, propagate_forward_time: 579.0081787109, propagate_backward_time: 955.1530761719, weight_update_time: 0.9853720665, other_time: 17.3237304688
epoch time: 113.9860382080s, input_fired_time: 1.4302854538, output_fired_time: 0.0000000000, propagate_forward_time: 109.8536376953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7021179199
validation[          , genome  6408] predictions:    6175/  10000 (61.75%), best:    6243/10000 (62.43%), error:     10861.64453, best error:     10734.83008 on epoch:    35, epoch:   40/100, mu: 0.9600229263, learning_rate: 0.0002295804, weight_decay: 0.0000638532

epoch time: 1594.1939697266s, input_fired_time: 22.7200984955, output_fired_time: 9.3058471680, propagate_forward_time: 582.5674438477, propagate_backward_time: 961.9641723633, weight_update_time: 0.9761306643, other_time: 17.6363525391
epoch time: 125.7862854004s, input_fired_time: 1.5633788109, output_fired_time: 0.0000000000, propagate_forward_time: 121.2968444824, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9260635376
validation[          , genome  6408] predictions:    6232/  10000 (62.32%), best:    6243/10000 (62.43%), error:     10831.15820, best error:     10734.83008 on epoch:    35, epoch:   41/100, mu: 0.9619801044, learning_rate: 0.0002151366, weight_decay: 0.0000610603

epoch time: 1607.7741699219s, input_fired_time: 22.9045791626, output_fired_time: 9.2213525772, propagate_forward_time: 588.4450683594, propagate_backward_time: 969.4888305664, weight_update_time: 0.9767270684, other_time: 17.7144165039
epoch time: 118.0073699951s, input_fired_time: 1.4984586239, output_fired_time: 0.0000000000, propagate_forward_time: 113.7686462402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7402648926
validation[          , genome  6408] predictions:    6305/  10000 (63.05%), best:    6305/10000 (63.05%), error:     10630.92188, best error:     10630.92188 on epoch:    42, epoch:   42/100, mu: 0.9638094902, learning_rate: 0.0002016015, weight_decay: 0.0000583897

epoch time: 1585.7314453125s, input_fired_time: 22.6858329773, output_fired_time: 9.1271562576, propagate_forward_time: 581.4371948242, propagate_backward_time: 955.0424194336, weight_update_time: 0.9862786531, other_time: 17.4388427734
epoch time: 125.4869766235s, input_fired_time: 1.5522816181, output_fired_time: 0.0000000000, propagate_forward_time: 120.9484558105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9862365723
validation[          , genome  6408] predictions:    6226/  10000 (62.26%), best:    6305/10000 (63.05%), error:     10809.13477, best error:     10630.92188 on epoch:    42, epoch:   43/100, mu: 0.9655194283, learning_rate: 0.0001889179, weight_decay: 0.0000558358

epoch time: 1603.6984863281s, input_fired_time: 22.9269695282, output_fired_time: 9.2392511368, propagate_forward_time: 585.7426147461, propagate_backward_time: 968.0424804688, weight_update_time: 0.9928218722, other_time: 17.7471313477
epoch time: 112.8101043701s, input_fired_time: 1.4172294140, output_fired_time: 0.0000000000, propagate_forward_time: 108.6845626831, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7083129883
validation[          , genome  6408] predictions:    6234/  10000 (62.34%), best:    6305/10000 (63.05%), error:     10873.21582, best error:     10630.92188 on epoch:    42, epoch:   44/100, mu: 0.9671177268, learning_rate: 0.0001770324, weight_decay: 0.0000533937

epoch time: 1596.8897705078s, input_fired_time: 22.7652702332, output_fired_time: 9.1960754395, propagate_forward_time: 582.7650756836, propagate_backward_time: 964.6651611328, weight_update_time: 0.9841236472, other_time: 17.4982299805
epoch time: 127.5825729370s, input_fired_time: 1.6109766960, output_fired_time: 0.0000000000, propagate_forward_time: 122.9207839966, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0508117676
validation[          , genome  6408] predictions:    6308/  10000 (63.08%), best:    6305/10000 (63.05%), error:     10655.03418, best error:     10630.92188 on epoch:    42, epoch:   45/100, mu: 0.9686117172, learning_rate: 0.0001658945, weight_decay: 0.0000510583

epoch time: 1601.0509033203s, input_fired_time: 22.9029750824, output_fired_time: 9.3441267014, propagate_forward_time: 584.8491821289, propagate_backward_time: 966.4882202148, weight_update_time: 0.9817999005, other_time: 17.4664306641
epoch time: 125.8562622070s, input_fired_time: 1.5645701885, output_fired_time: 0.0000000000, propagate_forward_time: 121.3106002808, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9810943604
validation[          , genome  6408] predictions:    6322/  10000 (63.22%), best:    6322/10000 (63.22%), error:     10588.61035, best error:     10588.61035 on epoch:    46, epoch:   46/100, mu: 0.9700081348, learning_rate: 0.0001554575, weight_decay: 0.0000488251

epoch time: 1595.5821533203s, input_fired_time: 22.8324432373, output_fired_time: 9.1404991150, propagate_forward_time: 584.7015991211, propagate_backward_time: 961.2846069336, weight_update_time: 1.0003875494, other_time: 17.6230468750
epoch time: 110.6153869629s, input_fired_time: 1.4022901058, output_fired_time: 0.0000000000, propagate_forward_time: 106.6021041870, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6109924316
validation[          , genome  6408] predictions:    6314/  10000 (63.14%), best:    6322/10000 (63.22%), error:     10629.65527, best error:     10588.61035 on epoch:    46, epoch:   47/100, mu: 0.9713133574, learning_rate: 0.0001456770, weight_decay: 0.0000466896

epoch time: 1582.0775146484s, input_fired_time: 22.6702919006, output_fired_time: 9.1559410095, propagate_forward_time: 578.8980102539, propagate_backward_time: 953.7109375000, weight_update_time: 0.9760499597, other_time: 17.6423950195
epoch time: 119.8042449951s, input_fired_time: 1.5052082539, output_fired_time: 0.0000000000, propagate_forward_time: 115.4563598633, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8426742554
validation[          , genome  6408] predictions:    6309/  10000 (63.09%), best:    6322/10000 (63.22%), error:     10712.18652, best error:     10588.61035 on epoch:    46, epoch:   48/100, mu: 0.9725334048, learning_rate: 0.0001365119, weight_decay: 0.0000446475

epoch time: 1587.4128417969s, input_fired_time: 22.6851348877, output_fired_time: 9.1626777649, propagate_forward_time: 581.7500610352, propagate_backward_time: 956.1156005859, weight_update_time: 1.0157876015, other_time: 17.6992797852
epoch time: 118.7479858398s, input_fired_time: 1.5045901537, output_fired_time: 0.0000000000, propagate_forward_time: 114.4491577148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7942352295
validation[          , genome  6408] predictions:    6312/  10000 (63.12%), best:    6322/10000 (63.22%), error:     10639.82031, best error:     10588.61035 on epoch:    46, epoch:   49/100, mu: 0.9736737609, learning_rate: 0.0001279234, weight_decay: 0.0000426947

epoch time: 1574.5877685547s, input_fired_time: 22.5399017334, output_fired_time: 9.0188884735, propagate_forward_time: 577.3815307617, propagate_backward_time: 948.1486816406, weight_update_time: 0.9684526920, other_time: 17.4987182617
epoch time: 112.7049560547s, input_fired_time: 1.4202561378, output_fired_time: 0.0000000000, propagate_forward_time: 108.6299896240, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6547088623
validation[          , genome  6408] predictions:    6256/  10000 (62.56%), best:    6322/10000 (63.22%), error:     10704.77637, best error:     10588.61035 on epoch:    46, epoch:   50/100, mu: 0.9747396708, learning_rate: 0.0001198752, weight_decay: 0.0000408273

epoch time: 1600.8017578125s, input_fired_time: 22.8855953217, output_fired_time: 9.2010641098, propagate_forward_time: 585.9337158203, propagate_backward_time: 965.1878051758, weight_update_time: 1.0083674192, other_time: 17.5935668945
epoch time: 115.9560012817s, input_fired_time: 1.4649894238, output_fired_time: 0.0000000000, propagate_forward_time: 111.7411880493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7498245239
validation[          , genome  6408] predictions:    6304/  10000 (63.04%), best:    6304/10000 (63.04%), error:     10580.42285, best error:     10580.42285 on epoch:    51, epoch:   51/100, mu: 0.9757360220, learning_rate: 0.0001123334, weight_decay: 0.0000390416

epoch time: 1528.7718505859s, input_fired_time: 21.9016761780, output_fired_time: 8.8369588852, propagate_forward_time: 557.9619140625, propagate_backward_time: 923.1761474609, weight_update_time: 0.9410145283, other_time: 16.8951416016
epoch time: 98.5775985718s, input_fired_time: 1.2588384151, output_fired_time: 0.0000000000, propagate_forward_time: 94.9362487793, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3825149536
validation[          , genome  6408] predictions:    6426/  10000 (64.26%), best:    6426/10000 (64.26%), error:     10270.99121, best error:     10270.99121 on epoch:    52, epoch:   52/100, mu: 0.9766672850, learning_rate: 0.0001052660, weight_decay: 0.0000373340

epoch time: 1148.9567871094s, input_fired_time: 16.5935287476, output_fired_time: 6.6219129562, propagate_forward_time: 421.6353454590, propagate_backward_time: 691.1478881836, weight_update_time: 0.7189382911, other_time: 12.9580688477
epoch time: 87.1406097412s, input_fired_time: 1.1155452728, output_fired_time: 0.0000000000, propagate_forward_time: 83.9031524658, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1219100952
validation[          , genome  6408] predictions:    6364/  10000 (63.64%), best:    6426/10000 (64.26%), error:     10471.65430, best error:     10270.99121 on epoch:    52, epoch:   53/100, mu: 0.9775377512, learning_rate: 0.0000986433, weight_decay: 0.0000357010

epoch time: 1126.4394531250s, input_fired_time: 16.2995300293, output_fired_time: 6.5462698936, propagate_forward_time: 413.0292968750, propagate_backward_time: 677.7995605469, weight_update_time: 0.7141161561, other_time: 12.7647705078
epoch time: 86.1247177124s, input_fired_time: 1.0943899155, output_fired_time: 0.0000000000, propagate_forward_time: 82.9385070801, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0918197632
validation[          , genome  6408] predictions:    6327/  10000 (63.27%), best:    6426/10000 (64.26%), error:     10622.91309, best error:     10270.99121 on epoch:    52, epoch:   54/100, mu: 0.9783514142, learning_rate: 0.0000924373, weight_decay: 0.0000341395

epoch time: 1124.0411376953s, input_fired_time: 16.2670726776, output_fired_time: 6.5318117142, propagate_forward_time: 412.2379150391, propagate_backward_time: 676.2862548828, weight_update_time: 0.7218755484, other_time: 12.7180175781
epoch time: 86.2137451172s, input_fired_time: 1.0990310907, output_fired_time: 0.0000000000, propagate_forward_time: 83.0599060059, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0548095703
validation[          , genome  6408] predictions:    6403/  10000 (64.03%), best:    6426/10000 (64.26%), error:     10339.89551, best error:     10270.99121 on epoch:    52, epoch:   55/100, mu: 0.9791119099, learning_rate: 0.0000866217, weight_decay: 0.0000326463

epoch time: 1125.2611083984s, input_fired_time: 16.3037071228, output_fired_time: 6.5205674171, propagate_forward_time: 412.0650329590, propagate_backward_time: 677.6275024414, weight_update_time: 0.7147212029, other_time: 12.7443237305
epoch time: 85.7660369873s, input_fired_time: 1.1075088978, output_fired_time: 0.0000000000, propagate_forward_time: 82.6481704712, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0103607178
validation[          , genome  6408] predictions:    6305/  10000 (63.05%), best:    6426/10000 (64.26%), error:     10540.64941, best error:     10270.99121 on epoch:    52, epoch:   56/100, mu: 0.9798227549, learning_rate: 0.0000811719, weight_decay: 0.0000312184

epoch time: 1126.7375488281s, input_fired_time: 16.3366737366, output_fired_time: 6.5727849007, propagate_forward_time: 413.4520263672, propagate_backward_time: 677.6217041016, weight_update_time: 0.7198874354, other_time: 12.7543945313
epoch time: 85.8391723633s, input_fired_time: 1.0991364717, output_fired_time: 0.0000000000, propagate_forward_time: 82.6909103394, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0491256714
validation[          , genome  6408] predictions:    6412/  10000 (64.12%), best:    6426/10000 (64.26%), error:     10360.29785, best error:     10270.99121 on epoch:    52, epoch:   57/100, mu: 0.9804872274, learning_rate: 0.0000760651, weight_decay: 0.0000298530

epoch time: 1122.7650146484s, input_fired_time: 16.2631015778, output_fired_time: 6.5366401672, propagate_forward_time: 412.1475524902, propagate_backward_time: 675.1619873047, weight_update_time: 0.6997745037, other_time: 12.6557617188
epoch time: 86.1937942505s, input_fired_time: 1.1031184196, output_fired_time: 0.0000000000, propagate_forward_time: 83.0287170410, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0619583130
validation[          , genome  6408] predictions:    6320/  10000 (63.20%), best:    6426/10000 (64.26%), error:     10598.11035, best error:     10270.99121 on epoch:    52, epoch:   58/100, mu: 0.9811083078, learning_rate: 0.0000712795, weight_decay: 0.0000285473

epoch time: 1129.2932128906s, input_fired_time: 16.3461856842, output_fired_time: 6.5907645226, propagate_forward_time: 414.5763854980, propagate_backward_time: 678.9810180664, weight_update_time: 0.7215650678, other_time: 12.7987670898
epoch time: 85.5354614258s, input_fired_time: 1.1004976034, output_fired_time: 0.0000000000, propagate_forward_time: 82.3244018555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1105651855
validation[          , genome  6408] predictions:    6376/  10000 (63.76%), best:    6426/10000 (64.26%), error:     10459.50781, best error:     10270.99121 on epoch:    52, epoch:   59/100, mu: 0.9816888571, learning_rate: 0.0000667951, weight_decay: 0.0000272987

epoch time: 1123.1942138672s, input_fired_time: 16.2794933319, output_fired_time: 6.5228972435, propagate_forward_time: 412.3039245605, propagate_backward_time: 675.3438110352, weight_update_time: 0.7269113064, other_time: 12.7439575195
epoch time: 85.9446868896s, input_fired_time: 1.0990951061, output_fired_time: 0.0000000000, propagate_forward_time: 82.7676696777, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0779190063
validation[          , genome  6408] predictions:    6375/  10000 (63.75%), best:    6426/10000 (64.26%), error:     10408.01074, best error:     10270.99121 on epoch:    52, epoch:   60/100, mu: 0.9822314978, learning_rate: 0.0000625927, weight_decay: 0.0000261047

epoch time: 1125.2808837891s, input_fired_time: 16.3039512634, output_fired_time: 6.5664725304, propagate_forward_time: 413.2498168945, propagate_backward_time: 676.4530639648, weight_update_time: 0.7135260701, other_time: 12.7075195313
epoch time: 88.3370208740s, input_fired_time: 1.1389719248, output_fired_time: 0.0000000000, propagate_forward_time: 85.1033477783, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0947036743
validation[          , genome  6408] predictions:    6394/  10000 (63.94%), best:    6426/10000 (64.26%), error:     10377.83691, best error:     10270.99121 on epoch:    52, epoch:   61/100, mu: 0.9827386737, learning_rate: 0.0000586547, weight_decay: 0.0000249629

epoch time: 1154.7170410156s, input_fired_time: 16.7554454803, output_fired_time: 6.6752748489, propagate_forward_time: 423.8766174316, propagate_backward_time: 694.4030151367, weight_update_time: 0.7456489205, other_time: 13.0066528320
epoch time: 87.8139114380s, input_fired_time: 1.1252307892, output_fired_time: 0.0000000000, propagate_forward_time: 84.5859832764, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1026992798
validation[          , genome  6408] predictions:    6429/  10000 (64.29%), best:    6426/10000 (64.26%), error:     10342.26465, best error:     10270.99121 on epoch:    52, epoch:   62/100, mu: 0.9832127690, learning_rate: 0.0000549645, weight_decay: 0.0000238711

epoch time: 1134.9376220703s, input_fired_time: 16.4713954926, output_fired_time: 6.6087737083, propagate_forward_time: 416.2694702148, propagate_backward_time: 682.7235107422, weight_update_time: 0.7246982455, other_time: 12.8644409180
epoch time: 86.0285110474s, input_fired_time: 1.1057047844, output_fired_time: 0.0000000000, propagate_forward_time: 82.8479766846, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0748291016
validation[          , genome  6408] predictions:    6399/  10000 (63.99%), best:    6426/10000 (64.26%), error:     10288.06348, best error:     10270.99121 on epoch:    52, epoch:   63/100, mu: 0.9836559296, learning_rate: 0.0000515065, weight_decay: 0.0000228270

epoch time: 1122.9875488281s, input_fired_time: 16.3145313263, output_fired_time: 6.4802665710, propagate_forward_time: 413.1085510254, propagate_backward_time: 674.3001098633, weight_update_time: 0.7142011523, other_time: 12.7841186523
epoch time: 85.6760406494s, input_fired_time: 1.1047838926, output_fired_time: 0.0000000000, propagate_forward_time: 82.4895477295, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0817108154
validation[          , genome  6408] predictions:    6398/  10000 (63.98%), best:    6426/10000 (64.26%), error:     10340.46680, best error:     10270.99121 on epoch:    52, epoch:   64/100, mu: 0.9840701222, learning_rate: 0.0000482660, weight_decay: 0.0000218286

epoch time: 1125.8636474609s, input_fired_time: 16.4136924744, output_fired_time: 6.5385408401, propagate_forward_time: 413.5225219727, propagate_backward_time: 676.5869750977, weight_update_time: 0.7185983062, other_time: 12.8018798828
epoch time: 86.1648178101s, input_fired_time: 1.1150954962, output_fired_time: 0.0000000000, propagate_forward_time: 83.0144348145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0352859497
validation[          , genome  6408] predictions:    6465/  10000 (64.65%), best:    6465/10000 (64.65%), error:     10223.68652, best error:     10223.68652 on epoch:    65, epoch:   65/100, mu: 0.9844572544, learning_rate: 0.0000452294, weight_decay: 0.0000208739

epoch time: 1124.7746582031s, input_fired_time: 16.3573570251, output_fired_time: 6.5430192947, propagate_forward_time: 412.8328247070, propagate_backward_time: 676.2322387695, weight_update_time: 0.7109137177, other_time: 12.8093261719
epoch time: 85.7838745117s, input_fired_time: 1.1161717176, output_fired_time: 0.0000000000, propagate_forward_time: 82.5779113770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0897903442
validation[          , genome  6408] predictions:    6409/  10000 (64.09%), best:    6465/10000 (64.65%), error:     10393.31738, best error:     10223.68652 on epoch:    65, epoch:   66/100, mu: 0.9848191142, learning_rate: 0.0000423838, weight_decay: 0.0000199609

epoch time: 1124.0993652344s, input_fired_time: 16.3222312927, output_fired_time: 6.5296297073, propagate_forward_time: 412.5198364258, propagate_backward_time: 675.9899291992, weight_update_time: 0.7133768201, other_time: 12.7376708984
epoch time: 85.5679550171s, input_fired_time: 1.1084011793, output_fired_time: 0.0000000000, propagate_forward_time: 82.3871841431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0723724365
validation[          , genome  6408] predictions:    6383/  10000 (63.83%), best:    6465/10000 (64.65%), error:     10319.48047, best error:     10223.68652 on epoch:    65, epoch:   67/100, mu: 0.9851573706, learning_rate: 0.0000397173, weight_decay: 0.0000190878

epoch time: 1123.2985839844s, input_fired_time: 16.3478374481, output_fired_time: 6.5614686012, propagate_forward_time: 412.1426086426, propagate_backward_time: 675.4772338867, weight_update_time: 0.7174445391, other_time: 12.7694702148
epoch time: 86.3689193726s, input_fired_time: 1.1089512110, output_fired_time: 0.0000000000, propagate_forward_time: 83.2045669556, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0554046631
validation[          , genome  6408] predictions:    6429/  10000 (64.29%), best:    6465/10000 (64.65%), error:     10252.35254, best error:     10223.68652 on epoch:    65, epoch:   68/100, mu: 0.9854735136, learning_rate: 0.0000372185, weight_decay: 0.0000182529

epoch time: 1199.3930664063s, input_fired_time: 17.0742740631, output_fired_time: 6.8315534592, propagate_forward_time: 430.6097717285, propagate_backward_time: 731.7427978516, weight_update_time: 0.7343006134, other_time: 13.1347656250
epoch time: 103.0762481689s, input_fired_time: 1.3252483606, output_fired_time: 0.0000000000, propagate_forward_time: 99.2594604492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4915390015
validation[          , genome  6408] predictions:    6423/  10000 (64.23%), best:    6465/10000 (64.65%), error:     10273.65332, best error:     10223.68652 on epoch:    65, epoch:   69/100, mu: 0.9857690334, learning_rate: 0.0000348770, weight_decay: 0.0000174546

epoch time: 1438.4084472656s, input_fired_time: 20.8997554779, output_fired_time: 8.3112249374, propagate_forward_time: 527.7602539063, propagate_backward_time: 865.2979125977, weight_update_time: 0.8961513042, other_time: 16.1392211914
epoch time: 117.2405319214s, input_fired_time: 1.5169523954, output_fired_time: 0.0000000000, propagate_forward_time: 112.9692153931, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7543640137
validation[          , genome  6408] predictions:    6444/  10000 (64.44%), best:    6444/10000 (64.44%), error:     10194.16602, best error:     10194.16602 on epoch:    70, epoch:   70/100, mu: 0.9860452414, learning_rate: 0.0000326827, weight_decay: 0.0000166912

epoch time: 1516.4228515625s, input_fired_time: 22.0079746246, output_fired_time: 8.7720451355, propagate_forward_time: 556.3853149414, propagate_backward_time: 912.3657836914, weight_update_time: 0.9480700493, other_time: 16.8917236328
epoch time: 117.5819168091s, input_fired_time: 1.5059957504, output_fired_time: 0.0000000000, propagate_forward_time: 113.2634201050, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8125000000
validation[          , genome  6408] predictions:    6435/  10000 (64.35%), best:    6444/10000 (64.44%), error:     10261.78613, best error:     10194.16602 on epoch:    70, epoch:   71/100, mu: 0.9863034487, learning_rate: 0.0000306265, weight_decay: 0.0000159611

epoch time: 1539.0479736328s, input_fired_time: 22.3286991119, output_fired_time: 8.8527488708, propagate_forward_time: 564.4548339844, propagate_backward_time: 926.2341308594, weight_update_time: 0.9607909322, other_time: 17.1774902344
epoch time: 129.5140075684s, input_fired_time: 1.6692625284, output_fired_time: 0.0000000000, propagate_forward_time: 124.8577957153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9869461060
validation[          , genome  6408] predictions:    6401/  10000 (64.01%), best:    6444/10000 (64.44%), error:     10271.54980, best error:     10194.16602 on epoch:    70, epoch:   72/100, mu: 0.9865447879, learning_rate: 0.0000286997, weight_decay: 0.0000152630

epoch time: 1514.4875488281s, input_fired_time: 21.9094753265, output_fired_time: 8.7680320740, propagate_forward_time: 556.8700561523, propagate_backward_time: 910.2543334961, weight_update_time: 0.9179430008, other_time: 16.6856689453
epoch time: 115.1076507568s, input_fired_time: 1.4952502251, output_fired_time: 0.0000000000, propagate_forward_time: 110.8664779663, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7459259033
validation[          , genome  6408] predictions:    6474/  10000 (64.74%), best:    6474/10000 (64.74%), error:     10180.97266, best error:     10180.97266 on epoch:    73, epoch:   73/100, mu: 0.9867703915, learning_rate: 0.0000268941, weight_decay: 0.0000145954

epoch time: 1575.1828613281s, input_fired_time: 22.7279815674, output_fired_time: 9.1348943710, propagate_forward_time: 576.9470825195, propagate_backward_time: 948.9557495117, weight_update_time: 0.9693433642, other_time: 17.4171142578
epoch time: 119.4006347656s, input_fired_time: 1.5424563885, output_fired_time: 0.0000000000, propagate_forward_time: 115.0184631348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8397140503
validation[          , genome  6408] predictions:    6463/  10000 (64.63%), best:    6463/10000 (64.63%), error:     10125.29492, best error:     10125.29492 on epoch:    74, epoch:   74/100, mu: 0.9869812727, learning_rate: 0.0000252021, weight_decay: 0.0000139571

epoch time: 2490.1398925781s, input_fired_time: 33.9935989380, output_fired_time: 9.5315771103, propagate_forward_time: 669.7720947266, propagate_backward_time: 1758.5845947266, weight_update_time: 1.0093836784, other_time: 18.2580566406
epoch time: 134.3960571289s, input_fired_time: 1.7313361168, output_fired_time: 0.0000000000, propagate_forward_time: 129.4623565674, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2023620605
validation[          , genome  6408] predictions:    6507/  10000 (65.07%), best:    6507/10000 (65.07%), error:     10069.01563, best error:     10069.01563 on epoch:    75, epoch:   75/100, mu: 0.9871783853, learning_rate: 0.0000236165, weight_decay: 0.0000133466

epoch time: 1752.9265136719s, input_fired_time: 24.0157070160, output_fired_time: 9.5858316422, propagate_forward_time: 695.2222290039, propagate_backward_time: 1005.8958129883, weight_update_time: 1.0252844095, other_time: 18.2069702148
epoch time: 132.3398284912s, input_fired_time: 1.6958236694, output_fired_time: 0.0000000000, propagate_forward_time: 127.5029678345, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1410446167
validation[          , genome  6408] predictions:    6446/  10000 (64.46%), best:    6507/10000 (65.07%), error:     10107.74121, best error:     10069.01563 on epoch:    75, epoch:   76/100, mu: 0.9873626232, learning_rate: 0.0000221307, weight_decay: 0.0000127628

epoch time: 1647.8720703125s, input_fired_time: 23.5641918182, output_fired_time: 9.4724731445, propagate_forward_time: 598.8364257813, propagate_backward_time: 998.1843261719, weight_update_time: 1.0098812580, other_time: 17.8146972656
epoch time: 125.9183044434s, input_fired_time: 1.6202828884, output_fired_time: 0.0000000000, propagate_forward_time: 121.3368530273, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9611663818
validation[          , genome  6408] predictions:    6422/  10000 (64.22%), best:    6507/10000 (65.07%), error:     10299.42480, best error:     10069.01563 on epoch:    75, epoch:   77/100, mu: 0.9875348210, learning_rate: 0.0000207384, weight_decay: 0.0000122046

epoch time: 1624.6922607422s, input_fired_time: 23.4729881287, output_fired_time: 9.3370828629, propagate_forward_time: 594.0410156250, propagate_backward_time: 980.1486206055, weight_update_time: 1.0018924475, other_time: 17.6925659180
epoch time: 123.9535598755s, input_fired_time: 1.5991015434, output_fired_time: 0.0000000000, propagate_forward_time: 119.4403228760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9141387939
validation[          , genome  6408] predictions:    6453/  10000 (64.53%), best:    6507/10000 (65.07%), error:     10105.42090, best error:     10069.01563 on epoch:    75, epoch:   78/100, mu: 0.9876957536, learning_rate: 0.0000194336, weight_decay: 0.0000116708

epoch time: 1653.9046630859s, input_fired_time: 23.9092979431, output_fired_time: 9.4811429977, propagate_forward_time: 604.7099609375, propagate_backward_time: 997.5947875977, weight_update_time: 1.0094326735, other_time: 18.2094116211
epoch time: 129.6552886963s, input_fired_time: 1.6801604033, output_fired_time: 0.0000000000, propagate_forward_time: 124.9262008667, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0489273071
validation[          , genome  6408] predictions:    6438/  10000 (64.38%), best:    6507/10000 (65.07%), error:     10171.92773, best error:     10069.01563 on epoch:    75, epoch:   79/100, mu: 0.9878461957, learning_rate: 0.0000182110, weight_decay: 0.0000111604

epoch time: 1680.7232666016s, input_fired_time: 24.1415748596, output_fired_time: 9.7159471512, propagate_forward_time: 613.0364990234, propagate_backward_time: 1015.5157470703, weight_update_time: 1.0433353186, other_time: 18.3134765625
epoch time: 128.5410156250s, input_fired_time: 1.6531598568, output_fired_time: 0.0000000000, propagate_forward_time: 123.8579864502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0298690796
validation[          , genome  6408] predictions:    6487/  10000 (64.87%), best:    6507/10000 (65.07%), error:     10082.58496, best error:     10069.01563 on epoch:    75, epoch:   80/100, mu: 0.9879868031, learning_rate: 0.0000170652, weight_decay: 0.0000106722

epoch time: 1576.8599853516s, input_fired_time: 23.1057987213, output_fired_time: 9.0901784897, propagate_forward_time: 575.7963256836, propagate_backward_time: 951.4127197266, weight_update_time: 0.9810240865, other_time: 17.4548950195
epoch time: 83.9721069336s, input_fired_time: 1.1627831459, output_fired_time: 0.0000000000, propagate_forward_time: 80.6398239136, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1695022583
validation[          , genome  6408] predictions:    6455/  10000 (64.55%), best:    6507/10000 (65.07%), error:     10125.92480, best error:     10069.01563 on epoch:    75, epoch:   81/100, mu: 0.9881182313, learning_rate: 0.0000159916, weight_decay: 0.0000102054

epoch time: 1119.8269042969s, input_fired_time: 18.2132453918, output_fired_time: 6.6678752899, propagate_forward_time: 402.7778625488, propagate_backward_time: 678.7325439453, weight_update_time: 0.7474552393, other_time: 13.4354248047
epoch time: 83.3461685181s, input_fired_time: 1.1577069759, output_fired_time: 0.0000000000, propagate_forward_time: 80.0681838989, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1202774048
validation[          , genome  6408] predictions:    6432/  10000 (64.32%), best:    6507/10000 (65.07%), error:     10108.71191, best error:     10069.01563 on epoch:    75, epoch:   82/100, mu: 0.9882410765, learning_rate: 0.0000149855, weight_decay: 0.0000097591

epoch time: 1120.6287841797s, input_fired_time: 18.2184429169, output_fired_time: 6.6194033623, propagate_forward_time: 402.5950317383, propagate_backward_time: 679.7553100586, weight_update_time: 0.7497156858, other_time: 13.4406738281
epoch time: 87.4227218628s, input_fired_time: 1.1949031353, output_fired_time: 0.0000000000, propagate_forward_time: 83.9590606689, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2687606812
validation[          , genome  6408] predictions:    6525/  10000 (65.25%), best:    6525/10000 (65.25%), error:      9986.39355, best error:      9986.39355 on epoch:    83, epoch:   83/100, mu: 0.9883559346, learning_rate: 0.0000140427, weight_decay: 0.0000093322

epoch time: 1131.5336914063s, input_fired_time: 18.4677715302, output_fired_time: 6.7108898163, propagate_forward_time: 407.2848815918, propagate_backward_time: 685.5424194336, weight_update_time: 0.7559412122, other_time: 13.5276489258
epoch time: 86.3569412231s, input_fired_time: 1.1967946291, output_fired_time: 0.0000000000, propagate_forward_time: 82.9074554443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2526931763
validation[          , genome  6408] predictions:    6513/  10000 (65.13%), best:    6525/10000 (65.25%), error:     10067.74707, best error:      9986.39355 on epoch:    83, epoch:   84/100, mu: 0.9884632826, learning_rate: 0.0000131592, weight_decay: 0.0000089241

epoch time: 1119.8189697266s, input_fired_time: 18.2471218109, output_fired_time: 6.5731239319, propagate_forward_time: 403.3316955566, propagate_backward_time: 678.2559204102, weight_update_time: 0.7502674460, other_time: 13.4111938477
epoch time: 86.1336059570s, input_fired_time: 1.1969525814, output_fired_time: 0.0000000000, propagate_forward_time: 82.7630004883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1736526489
validation[          , genome  6408] predictions:    6516/  10000 (65.16%), best:    6525/10000 (65.25%), error:     10028.45996, best error:      9986.39355 on epoch:    83, epoch:   85/100, mu: 0.9885635972, learning_rate: 0.0000123313, weight_decay: 0.0000085337

epoch time: 1138.8668212891s, input_fired_time: 18.4979934692, output_fired_time: 6.6982493401, propagate_forward_time: 409.4640197754, propagate_backward_time: 690.5555419922, weight_update_time: 0.7582014203, other_time: 13.6510009766
epoch time: 87.7826004028s, input_fired_time: 1.2120143175, output_fired_time: 0.0000000000, propagate_forward_time: 84.3442230225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2263641357
validation[          , genome  6408] predictions:    6478/  10000 (64.78%), best:    6525/10000 (65.25%), error:     10043.46973, best error:      9986.39355 on epoch:    83, epoch:   86/100, mu: 0.9886573553, learning_rate: 0.0000115555, weight_decay: 0.0000081605

epoch time: 1130.3176269531s, input_fired_time: 18.2314910889, output_fired_time: 6.5556383133, propagate_forward_time: 403.6468200684, propagate_backward_time: 688.4090576172, weight_update_time: 0.7589142323, other_time: 13.4746093750
epoch time: 86.8139572144s, input_fired_time: 1.2017279863, output_fired_time: 0.0000000000, propagate_forward_time: 83.3882217407, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2240066528
validation[          , genome  6408] predictions:    6471/  10000 (64.71%), best:    6525/10000 (65.25%), error:     10099.88379, best error:      9986.39355 on epoch:    83, epoch:   87/100, mu: 0.9887450337, learning_rate: 0.0000108285, weight_decay: 0.0000078036

epoch time: 1120.2485351563s, input_fired_time: 18.2306232452, output_fired_time: 6.5682854652, propagate_forward_time: 402.8847961426, propagate_backward_time: 679.1889648438, weight_update_time: 0.7523666024, other_time: 13.3759765625
epoch time: 85.3530883789s, input_fired_time: 1.1759572029, output_fired_time: 0.0000000000, propagate_forward_time: 81.9934005737, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1837310791
validation[          , genome  6408] predictions:    6507/  10000 (65.07%), best:    6525/10000 (65.25%), error:     10063.10059, best error:      9986.39355 on epoch:    83, epoch:   88/100, mu: 0.9888269901, learning_rate: 0.0000101472, weight_decay: 0.0000074622

epoch time: 1159.4342041016s, input_fired_time: 18.3594055176, output_fired_time: 6.6403670311, propagate_forward_time: 435.9865112305, propagate_backward_time: 684.9306640625, weight_update_time: 0.7558513880, other_time: 13.5172729492
epoch time: 87.6872558594s, input_fired_time: 1.2051794529, output_fired_time: 0.0000000000, propagate_forward_time: 84.2136077881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2684707642
validation[          , genome  6408] predictions:    6541/  10000 (65.41%), best:    6541/10000 (65.41%), error:      9961.73145, best error:      9961.73145 on epoch:    89, epoch:   89/100, mu: 0.9889035821, learning_rate: 0.0000095088, weight_decay: 0.0000071359

epoch time: 1129.3067626953s, input_fired_time: 18.3719043732, output_fired_time: 6.6375422478, propagate_forward_time: 406.3988952637, propagate_backward_time: 684.3967895508, weight_update_time: 0.7417567372, other_time: 13.5015258789
epoch time: 83.1485595703s, input_fired_time: 1.1498591900, output_fired_time: 0.0000000000, propagate_forward_time: 79.8627548218, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1359481812
validation[          , genome  6408] predictions:    6533/  10000 (65.33%), best:    6533/10000 (65.33%), error:      9939.35156, best error:      9939.35156 on epoch:    90, epoch:   90/100, mu: 0.9889751673, learning_rate: 0.0000089106, weight_decay: 0.0000068237

epoch time: 1134.8540039063s, input_fired_time: 18.4894142151, output_fired_time: 6.6020088196, propagate_forward_time: 408.0143432617, propagate_backward_time: 688.1644287109, weight_update_time: 0.7595792413, other_time: 13.5838012695
epoch time: 82.1032257080s, input_fired_time: 1.1276780367, output_fired_time: 0.0000000000, propagate_forward_time: 78.8798370361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0957107544
validation[          , genome  6408] predictions:    6514/  10000 (65.14%), best:    6533/10000 (65.33%), error:      9967.21777, best error:      9939.35156 on epoch:    90, epoch:   91/100, mu: 0.9890421033, learning_rate: 0.0000083500, weight_decay: 0.0000065253

epoch time: 1134.8769531250s, input_fired_time: 18.5417919159, output_fired_time: 6.6705927849, propagate_forward_time: 408.2702331543, propagate_backward_time: 687.8148193359, weight_update_time: 0.7470617294, other_time: 13.5795898438
epoch time: 86.7872390747s, input_fired_time: 1.2004730701, output_fired_time: 0.0000000000, propagate_forward_time: 83.3755645752, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2112045288
validation[          , genome  6408] predictions:    6536/  10000 (65.36%), best:    6533/10000 (65.33%), error:     10014.18066, best error:      9939.35156 on epoch:    90, epoch:   92/100, mu: 0.9891046286, learning_rate: 0.0000078247, weight_decay: 0.0000062399

epoch time: 1132.3983154297s, input_fired_time: 18.4761848450, output_fired_time: 6.6442403793, propagate_forward_time: 408.6049194336, propagate_backward_time: 685.1042480469, weight_update_time: 0.7513280511, other_time: 13.5686645508
epoch time: 83.1016159058s, input_fired_time: 1.1540704966, output_fired_time: 0.0000000000, propagate_forward_time: 79.7888336182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1587142944
validation[          , genome  6408] predictions:    6523/  10000 (65.23%), best:    6523/10000 (65.23%), error:      9933.63184, best error:      9933.63184 on epoch:    93, epoch:   93/100, mu: 0.9891631007, learning_rate: 0.0000073324, weight_decay: 0.0000059670

epoch time: 1135.6445312500s, input_fired_time: 18.5454463959, output_fired_time: 6.6606760025, propagate_forward_time: 409.2433776855, propagate_backward_time: 687.6925048828, weight_update_time: 0.7456988096, other_time: 13.5025634766
epoch time: 87.4866714478s, input_fired_time: 1.2103471756, output_fired_time: 0.0000000000, propagate_forward_time: 84.0559616089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2203598022
validation[          , genome  6408] predictions:    6516/  10000 (65.16%), best:    6523/10000 (65.23%), error:      9967.35059, best error:      9933.63184 on epoch:    93, epoch:   94/100, mu: 0.9892177582, learning_rate: 0.0000068711, weight_decay: 0.0000057060

epoch time: 1139.3774414063s, input_fired_time: 18.5499439240, output_fired_time: 6.7306070328, propagate_forward_time: 409.4920349121, propagate_backward_time: 691.0745239258, weight_update_time: 0.7508784533, other_time: 13.5303344727
epoch time: 86.6252059937s, input_fired_time: 1.1964672804, output_fired_time: 0.0000000000, propagate_forward_time: 83.2015533447, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2271881104
validation[          , genome  6408] predictions:    6498/  10000 (64.98%), best:    6498/10000 (64.98%), error:      9926.83594, best error:      9926.83594 on epoch:    95, epoch:   95/100, mu: 0.9892688394, learning_rate: 0.0000064388, weight_decay: 0.0000054564

epoch time: 1135.1000976563s, input_fired_time: 18.5430507660, output_fired_time: 6.7053847313, propagate_forward_time: 409.2063293457, propagate_backward_time: 687.1235351563, weight_update_time: 0.7605612278, other_time: 13.5217285156
epoch time: 84.0487213135s, input_fired_time: 1.1640516520, output_fired_time: 0.0000000000, propagate_forward_time: 80.7293014526, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1553649902
validation[          , genome  6408] predictions:    6530/  10000 (65.30%), best:    6498/10000 (64.98%), error:      9993.67676, best error:      9926.83594 on epoch:    95, epoch:   96/100, mu: 0.9893165827, learning_rate: 0.0000060337, weight_decay: 0.0000052178

epoch time: 1144.8687744141s, input_fired_time: 18.6642799377, output_fired_time: 6.7033839226, propagate_forward_time: 411.8946533203, propagate_backward_time: 693.9037475586, weight_update_time: 0.7536361814, other_time: 13.7026977539
epoch time: 87.0172500610s, input_fired_time: 1.1964948177, output_fired_time: 0.0000000000, propagate_forward_time: 83.6077804565, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2129745483
validation[          , genome  6408] predictions:    6510/  10000 (65.10%), best:    6498/10000 (64.98%), error:      9988.35840, best error:      9926.83594 on epoch:    95, epoch:   97/100, mu: 0.9893612266, learning_rate: 0.0000056541, weight_decay: 0.0000049895

epoch time: 1198.6000976563s, input_fired_time: 19.2343635559, output_fired_time: 7.0256876945, propagate_forward_time: 431.2837219238, propagate_backward_time: 726.9193725586, weight_update_time: 0.7772699594, other_time: 14.1370239258
epoch time: 83.9035339355s, input_fired_time: 1.1606438160, output_fired_time: 0.0000000000, propagate_forward_time: 80.5774307251, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1654586792
validation[          , genome  6408] predictions:    6511/  10000 (65.11%), best:    6498/10000 (64.98%), error:     10013.89160, best error:      9926.83594 on epoch:    95, epoch:   98/100, mu: 0.9894029498, learning_rate: 0.0000052984, weight_decay: 0.0000047713

epoch time: 1140.8211669922s, input_fired_time: 18.5599288940, output_fired_time: 6.7172570229, propagate_forward_time: 410.0296936035, propagate_backward_time: 691.8988037109, weight_update_time: 0.7572982311, other_time: 13.6154785156
epoch time: 86.9865036011s, input_fired_time: 1.1980433464, output_fired_time: 0.0000000000, propagate_forward_time: 83.5745773315, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2138824463
validation[          , genome  6408] predictions:    6496/  10000 (64.96%), best:    6498/10000 (64.98%), error:      9955.84180, best error:      9926.83594 on epoch:    95, epoch:   99/100, mu: 0.9894419312, learning_rate: 0.0000049650, weight_decay: 0.0000045626

epoch time: 1134.1721191406s, input_fired_time: 18.5106964111, output_fired_time: 6.6767740250, propagate_forward_time: 408.8022460938, propagate_backward_time: 686.5468750000, weight_update_time: 0.7623787522, other_time: 13.6354980469
epoch time: 88.1410598755s, input_fired_time: 1.2171868086, output_fired_time: 0.0000000000, propagate_forward_time: 84.6847534180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2391204834
validation[          , genome  6408] predictions:    6472/  10000 (64.72%), best:    6498/10000 (64.98%), error:     10030.47754, best error:      9926.83594 on epoch:    95, epoch:  100/100, mu: 0.9894783497, learning_rate: 0.0000046527, weight_decay: 0.0000043631

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 429.4652099609s, input_fired_time: 5.9767966270, output_fired_time: 0.0000000000, propagate_forward_time: 412.3668823242, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.1215209961
best training[          , genome  6408] predictions:   37354/  50000 (74.71%), best:    6498/10000 (64.98%), error:     38979.35938, best error:      9926.83594 on epoch:    95, epoch:  101/100, mu: 0.9895123839, learning_rate: 0.0000043599, weight_decay: 0.0000041722
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 87.6399230957s, input_fired_time: 1.2153656483, output_fired_time: 0.0000000000, propagate_forward_time: 83.8302307129, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5943298340
   testing[          , genome  6408] predictions:    6498/  10000 (64.98%), best:    6498/10000 (64.98%), error:      9926.83594, best error:      9926.83594 on epoch:    95, epoch:  101/100, mu: 0.9895123839, learning_rate: 0.0000043599, weight_decay: 0.0000041722
backpropagation finished successfully!
05:27:34 (11592): called boinc_finish(0)

</stderr_txt>
]]>