Nameexact_genome_1542692703_174_6399_0
Workunit4100924
Created20 Nov 2018, 11:35:44 UTC
Sent20 Nov 2018, 12:17:03 UTC
Report deadline2 Jan 2019, 6:56:12 UTC
Received23 Nov 2018, 1:44:16 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88437
Run time13 hours 6 min 25 sec
CPU time12 hours 56 min 27 sec
Validate stateValid
Credit10,694.30
Device peak FLOPS4.09 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size524.14 MB
Peak swap size545.74 MB
Peak disk usage9.83 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_174_6399.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_174_6399_0_r937835473_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542692703_174_6399.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6399
read normal distribution: '1 -0.490044683218002 -0.367206305265427'
generator_str: '1293533217'
read generator: 1293533217
read generated_by_map:
1 add_node 1
reading 98 nodes.
reading 330 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1435819538
post shuffle 1: 2074527664
epoch time: 851.36865234375s, input_fired_time: 9.19247817993164, output_fired_time: 6.63453674316406, propagate_forward_time: 298.989318847656, propagate_backward_time: 523.931701660156, weight_update_time: 1.01894021034241, other_time: 12.62060546875
epoch time: 63.2292823791504s, input_fired_time: 1.05062508583069, output_fired_time: 0, propagate_forward_time: 59.9334487915039, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.24520874023438
validation[          , genome  6399] predictions:    4760/  10000 (47.60%), best:    4760/10000 (47.60%), error:     18158.69922, best error:     18158.69922 on epoch:     0, epoch:    0/50, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 851.1232299805s, input_fired_time: 9.1533269882, output_fired_time: 6.6347351074, propagate_forward_time: 298.8299560547, propagate_backward_time: 523.3551635742, weight_update_time: 1.0425685644, other_time: 13.1500854492
epoch time: 63.1075210571s, input_fired_time: 1.0071896315, output_fired_time: 0.0000000000, propagate_forward_time: 59.8782272339, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2221031189
validation[          , genome  6399] predictions:    5069/  10000 (50.69%), best:    5069/10000 (50.69%), error:     16392.45313, best error:     16392.45313 on epoch:     1, epoch:    1/50, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 852.7131958008s, input_fired_time: 8.9699430466, output_fired_time: 6.6198272705, propagate_forward_time: 301.9893798828, propagate_backward_time: 522.4609985352, weight_update_time: 1.0223591328, other_time: 12.6730346680
epoch time: 62.8390312195s, input_fired_time: 1.0157052279, output_fired_time: 0.0000000000, propagate_forward_time: 59.5420074463, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2813186646
validation[          , genome  6399] predictions:    4963/  10000 (49.63%), best:    5069/10000 (50.69%), error:     16423.01758, best error:     16392.45313 on epoch:     1, epoch:    2/50, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 853.6143798828s, input_fired_time: 9.0642423630, output_fired_time: 6.6369962692, propagate_forward_time: 300.0760803223, propagate_backward_time: 525.4307250977, weight_update_time: 1.0113490820, other_time: 12.4063110352
epoch time: 63.3393745422s, input_fired_time: 1.0073859692, output_fired_time: 0.0000000000, propagate_forward_time: 60.0356178284, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2963714600
validation[          , genome  6399] predictions:    4886/  10000 (48.86%), best:    4886/10000 (48.86%), error:     16314.18945, best error:     16314.18945 on epoch:     3, epoch:    3/50, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 852.6903076172s, input_fired_time: 9.0359048843, output_fired_time: 6.6379437447, propagate_forward_time: 299.6392211914, propagate_backward_time: 524.9671630859, weight_update_time: 1.0151232481, other_time: 12.4100952148
epoch time: 63.4474563599s, input_fired_time: 1.0179052353, output_fired_time: 0.0000000000, propagate_forward_time: 60.1297531128, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2997970581
validation[          , genome  6399] predictions:    4645/  10000 (46.45%), best:    4886/10000 (48.86%), error:     17963.12305, best error:     16314.18945 on epoch:     3, epoch:    4/50, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 851.2313842773s, input_fired_time: 8.9527492523, output_fired_time: 6.6371626854, propagate_forward_time: 299.5551757813, propagate_backward_time: 523.5932006836, weight_update_time: 1.0022624731, other_time: 12.4931030273
epoch time: 63.5853271484s, input_fired_time: 1.0055253506, output_fired_time: 0.0000000000, propagate_forward_time: 60.3217964172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2580070496
validation[          , genome  6399] predictions:    4969/  10000 (49.69%), best:    4969/10000 (49.69%), error:     15400.80859, best error:     15400.80859 on epoch:     5, epoch:    5/50, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 860.0899047852s, input_fired_time: 9.0162076950, output_fired_time: 6.6580491066, propagate_forward_time: 304.8580322266, propagate_backward_time: 527.0394897461, weight_update_time: 1.0089219809, other_time: 12.5181274414
epoch time: 62.6269912720s, input_fired_time: 0.9684144258, output_fired_time: 0.0000000000, propagate_forward_time: 59.5568542480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1017227173
validation[          , genome  6399] predictions:    5006/  10000 (50.06%), best:    4969/10000 (49.69%), error:     15720.41016, best error:     15400.80859 on epoch:     5, epoch:    6/50, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 858.2570190430s, input_fired_time: 9.0589351654, output_fired_time: 6.6691279411, propagate_forward_time: 299.9052429199, propagate_backward_time: 529.8456420898, weight_update_time: 1.0331729650, other_time: 12.7780151367
epoch time: 63.3046913147s, input_fired_time: 0.9984624386, output_fired_time: 0.0000000000, propagate_forward_time: 60.0093994141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2968292236
validation[          , genome  6399] predictions:    5206/  10000 (52.06%), best:    5206/10000 (52.06%), error:     14790.94434, best error:     14790.94434 on epoch:     7, epoch:    7/50, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 853.4817504883s, input_fired_time: 8.9513292313, output_fired_time: 6.5998563766, propagate_forward_time: 300.1459655762, propagate_backward_time: 525.4993896484, weight_update_time: 1.0123559237, other_time: 12.2851562500
epoch time: 63.4179229736s, input_fired_time: 1.0249665976, output_fired_time: 0.0000000000, propagate_forward_time: 60.0534095764, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3395462036
validation[          , genome  6399] predictions:    5098/  10000 (50.98%), best:    5206/10000 (52.06%), error:     15197.41797, best error:     14790.94434 on epoch:     7, epoch:    8/50, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 854.4201049805s, input_fired_time: 9.1314249039, output_fired_time: 6.6549887657, propagate_forward_time: 300.2106323242, propagate_backward_time: 525.6433715820, weight_update_time: 1.0245463848, other_time: 12.7797241211
epoch time: 63.7142524719s, input_fired_time: 0.9993564487, output_fired_time: 0.0000000000, propagate_forward_time: 60.4608192444, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2540779114
validation[          , genome  6399] predictions:    4909/  10000 (49.09%), best:    5206/10000 (52.06%), error:     16656.36719, best error:     14790.94434 on epoch:     7, epoch:    9/50, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 859.0588989258s, input_fired_time: 8.9273452759, output_fired_time: 6.6301846504, propagate_forward_time: 298.8207092285, propagate_backward_time: 532.2499389648, weight_update_time: 1.0087723732, other_time: 12.4307250977
epoch time: 63.4460144043s, input_fired_time: 1.0250760317, output_fired_time: 0.0000000000, propagate_forward_time: 60.1416091919, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2793273926
validation[          , genome  6399] predictions:    5327/  10000 (53.27%), best:    5327/10000 (53.27%), error:     14563.29980, best error:     14563.29980 on epoch:    10, epoch:   10/50, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 849.1273193359s, input_fired_time: 8.9404001236, output_fired_time: 6.6198263168, propagate_forward_time: 298.5208435059, propagate_backward_time: 522.5965576172, weight_update_time: 1.0210359097, other_time: 12.4497070313
epoch time: 63.0089454651s, input_fired_time: 1.0229057074, output_fired_time: 0.0000000000, propagate_forward_time: 59.6764030457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3096351624
validation[          , genome  6399] predictions:    5241/  10000 (52.41%), best:    5327/10000 (53.27%), error:     14994.74805, best error:     14563.29980 on epoch:    10, epoch:   11/50, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 856.6674804688s, input_fired_time: 9.0378293991, output_fired_time: 6.6407771111, propagate_forward_time: 303.7491760254, propagate_backward_time: 524.6431884766, weight_update_time: 1.0237284899, other_time: 12.5965576172
epoch time: 62.9524192810s, input_fired_time: 1.0001413822, output_fired_time: 0.0000000000, propagate_forward_time: 59.6673507690, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2849273682
validation[          , genome  6399] predictions:    5339/  10000 (53.39%), best:    5339/10000 (53.39%), error:     14347.09668, best error:     14347.09668 on epoch:    12, epoch:   12/50, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 857.8961181641s, input_fired_time: 9.0115585327, output_fired_time: 6.6110372543, propagate_forward_time: 300.0179748535, propagate_backward_time: 529.4868164063, weight_update_time: 1.0352307558, other_time: 12.7687988281
epoch time: 63.2265052795s, input_fired_time: 1.0015110970, output_fired_time: 0.0000000000, propagate_forward_time: 59.9546775818, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2703170776
validation[          , genome  6399] predictions:    5127/  10000 (51.27%), best:    5339/10000 (53.39%), error:     14722.57227, best error:     14347.09668 on epoch:    12, epoch:   13/50, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 861.4263305664s, input_fired_time: 9.0760803223, output_fired_time: 6.6017184258, propagate_forward_time: 299.1793518066, propagate_backward_time: 533.6902465820, weight_update_time: 1.0273113251, other_time: 12.8788452148
epoch time: 63.3622283936s, input_fired_time: 1.0136225224, output_fired_time: 0.0000000000, propagate_forward_time: 60.0662117004, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2823944092
validation[          , genome  6399] predictions:    5065/  10000 (50.65%), best:    5339/10000 (53.39%), error:     15116.75879, best error:     14347.09668 on epoch:    12, epoch:   14/50, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 859.6486816406s, input_fired_time: 9.0637969971, output_fired_time: 6.6340613365, propagate_forward_time: 303.5541076660, propagate_backward_time: 527.7312622070, weight_update_time: 1.0254597664, other_time: 12.6654663086
epoch time: 63.3432884216s, input_fired_time: 1.0256911516, output_fired_time: 0.0000000000, propagate_forward_time: 60.0560798645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2615165710
validation[          , genome  6399] predictions:    5074/  10000 (50.74%), best:    5339/10000 (53.39%), error:     15136.13672, best error:     14347.09668 on epoch:    12, epoch:   15/50, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 853.7709350586s, input_fired_time: 9.0788555145, output_fired_time: 6.6189131737, propagate_forward_time: 300.0306091309, propagate_backward_time: 525.5418090820, weight_update_time: 1.0151132345, other_time: 12.5007934570
epoch time: 63.6361312866s, input_fired_time: 1.0217018127, output_fired_time: 0.0000000000, propagate_forward_time: 60.3869895935, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2274398804
validation[          , genome  6399] predictions:    5387/  10000 (53.87%), best:    5387/10000 (53.87%), error:     14070.71973, best error:     14070.71973 on epoch:    16, epoch:   16/50, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 856.4084472656s, input_fired_time: 8.9838571548, output_fired_time: 6.6313438416, propagate_forward_time: 298.9162597656, propagate_backward_time: 528.7999267578, weight_update_time: 1.0509119034, other_time: 13.0770263672
epoch time: 62.9582977295s, input_fired_time: 1.0076991320, output_fired_time: 0.0000000000, propagate_forward_time: 59.7239036560, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2266960144
validation[          , genome  6399] predictions:    5123/  10000 (51.23%), best:    5387/10000 (53.87%), error:     15329.38184, best error:     14070.71973 on epoch:    16, epoch:   17/50, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 854.6821289063s, input_fired_time: 9.0026569366, output_fired_time: 6.6341009140, propagate_forward_time: 298.9231872559, propagate_backward_time: 527.3182373047, weight_update_time: 1.0350829363, other_time: 12.8039550781
epoch time: 62.7175941467s, input_fired_time: 0.9691157341, output_fired_time: 0.0000000000, propagate_forward_time: 59.6743278503, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0741500854
validation[          , genome  6399] predictions:    5436/  10000 (54.36%), best:    5436/10000 (54.36%), error:     13674.01367, best error:     13674.01367 on epoch:    18, epoch:   18/50, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 853.6405639648s, input_fired_time: 8.9876279831, output_fired_time: 6.6356434822, propagate_forward_time: 298.5657958984, propagate_backward_time: 527.0551757813, weight_update_time: 1.0089236498, other_time: 12.3963623047
epoch time: 63.6295661926s, input_fired_time: 1.0490090847, output_fired_time: 0.0000000000, propagate_forward_time: 60.3802185059, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2003402710
validation[          , genome  6399] predictions:    5534/  10000 (55.34%), best:    5534/10000 (55.34%), error:     13492.34863, best error:     13492.34863 on epoch:    19, epoch:   19/50, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 857.1754760742s, input_fired_time: 9.1196889877, output_fired_time: 6.6387877464, propagate_forward_time: 299.6163330078, propagate_backward_time: 529.1605224609, weight_update_time: 1.0172915459, other_time: 12.6401367188
epoch time: 62.6295089722s, input_fired_time: 0.9802762866, output_fired_time: 0.0000000000, propagate_forward_time: 59.5888748169, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0603561401
validation[          , genome  6399] predictions:    5510/  10000 (55.10%), best:    5534/10000 (55.34%), error:     13523.90430, best error:     13492.34863 on epoch:    19, epoch:   20/50, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 853.8077392578s, input_fired_time: 9.1317176819, output_fired_time: 6.6659035683, propagate_forward_time: 300.2110900879, propagate_backward_time: 525.0348510742, weight_update_time: 1.0334576368, other_time: 12.7642211914
epoch time: 62.8779983521s, input_fired_time: 1.0081256628, output_fired_time: 0.0000000000, propagate_forward_time: 59.5991058350, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2707672119
validation[          , genome  6399] predictions:    5556/  10000 (55.56%), best:    5534/10000 (55.34%), error:     13598.14648, best error:     13492.34863 on epoch:    19, epoch:   21/50, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 852.5658569336s, input_fired_time: 9.1767082214, output_fired_time: 6.6340155602, propagate_forward_time: 299.4246520996, propagate_backward_time: 524.6619873047, weight_update_time: 1.0382335186, other_time: 12.6684570313
epoch time: 62.5615081787s, input_fired_time: 0.9892788529, output_fired_time: 0.0000000000, propagate_forward_time: 59.4323616028, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1398658752
validation[          , genome  6399] predictions:    5482/  10000 (54.82%), best:    5534/10000 (55.34%), error:     13689.32129, best error:     13492.34863 on epoch:    19, epoch:   22/50, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 855.4949951172s, input_fired_time: 9.1052732468, output_fired_time: 6.6213860512, propagate_forward_time: 303.3144531250, propagate_backward_time: 523.9039916992, weight_update_time: 1.0132776499, other_time: 12.5498657227
epoch time: 63.4191703796s, input_fired_time: 1.0245312452, output_fired_time: 0.0000000000, propagate_forward_time: 60.2386665344, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1559715271
validation[          , genome  6399] predictions:    5617/  10000 (56.17%), best:    5617/10000 (56.17%), error:     13399.29980, best error:     13399.29980 on epoch:    23, epoch:   23/50, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 849.9025268555s, input_fired_time: 8.9924688339, output_fired_time: 6.6162819862, propagate_forward_time: 298.6748657227, propagate_backward_time: 523.0364990234, weight_update_time: 1.0131582022, other_time: 12.5823974609
epoch time: 63.0809173584s, input_fired_time: 1.0295302868, output_fired_time: 0.0000000000, propagate_forward_time: 59.8074378967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2439498901
validation[          , genome  6399] predictions:    5697/  10000 (56.97%), best:    5697/10000 (56.97%), error:     13110.03613, best error:     13110.03613 on epoch:    24, epoch:   24/50, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 851.8873901367s, input_fired_time: 9.1439599991, output_fired_time: 6.6684274673, propagate_forward_time: 299.8593444824, propagate_backward_time: 523.7213134766, weight_update_time: 1.0111634731, other_time: 12.4942626953
epoch time: 62.8340797424s, input_fired_time: 1.0249197483, output_fired_time: 0.0000000000, propagate_forward_time: 59.5800590515, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2290992737
validation[          , genome  6399] predictions:    5423/  10000 (54.23%), best:    5697/10000 (56.97%), error:     14042.31934, best error:     13110.03613 on epoch:    24, epoch:   25/50, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 854.9066162109s, input_fired_time: 9.1825828552, output_fired_time: 6.6374287605, propagate_forward_time: 298.8099670410, propagate_backward_time: 527.4292602539, weight_update_time: 1.0198234320, other_time: 12.8473510742
epoch time: 62.9728469849s, input_fired_time: 1.0411679745, output_fired_time: 0.0000000000, propagate_forward_time: 59.6126022339, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3190765381
validation[          , genome  6399] predictions:    5543/  10000 (55.43%), best:    5697/10000 (56.97%), error:     13458.07324, best error:     13110.03613 on epoch:    24, epoch:   26/50, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 851.3220214844s, input_fired_time: 9.1926612854, output_fired_time: 6.6270308495, propagate_forward_time: 298.6610412598, propagate_backward_time: 524.0310668945, weight_update_time: 1.0193471909, other_time: 12.8102416992
epoch time: 63.8350601196s, input_fired_time: 1.0335568190, output_fired_time: 0.0000000000, propagate_forward_time: 60.4796028137, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3218994141
validation[          , genome  6399] predictions:    5602/  10000 (56.02%), best:    5697/10000 (56.97%), error:     13454.66309, best error:     13110.03613 on epoch:    24, epoch:   27/50, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 852.1302490234s, input_fired_time: 9.2127714157, output_fired_time: 6.6436972618, propagate_forward_time: 299.3352966309, propagate_backward_time: 524.1082153320, weight_update_time: 1.0345867872, other_time: 12.8302612305
epoch time: 63.2834320068s, input_fired_time: 1.0279185772, output_fired_time: 0.0000000000, propagate_forward_time: 59.9554252625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3000869751
validation[          , genome  6399] predictions:    5623/  10000 (56.23%), best:    5697/10000 (56.97%), error:     13424.24219, best error:     13110.03613 on epoch:    24, epoch:   28/50, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 851.1668701172s, input_fired_time: 9.1526317596, output_fired_time: 6.6606984138, propagate_forward_time: 299.1712646484, propagate_backward_time: 523.5253906250, weight_update_time: 1.0311952829, other_time: 12.6568603516
epoch time: 63.0858764648s, input_fired_time: 1.0321807861, output_fired_time: 0.0000000000, propagate_forward_time: 59.8134346008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2402610779
validation[          , genome  6399] predictions:    5790/  10000 (57.90%), best:    5790/10000 (57.90%), error:     12697.24902, best error:     12697.24902 on epoch:    29, epoch:   29/50, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 856.2445678711s, input_fired_time: 9.3005008698, output_fired_time: 6.5958762169, propagate_forward_time: 299.3858337402, propagate_backward_time: 528.3342895508, weight_update_time: 1.0140115023, other_time: 12.6281127930
epoch time: 63.3187026978s, input_fired_time: 1.0530049801, output_fired_time: 0.0000000000, propagate_forward_time: 59.9670639038, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2986335754
validation[          , genome  6399] predictions:    5555/  10000 (55.55%), best:    5790/10000 (57.90%), error:     13639.09668, best error:     12697.24902 on epoch:    29, epoch:   30/50, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 854.3827514648s, input_fired_time: 9.2032451630, output_fired_time: 6.6176114082, propagate_forward_time: 302.4656677246, propagate_backward_time: 523.4507446289, weight_update_time: 1.0175869465, other_time: 12.6454467773
epoch time: 63.3066978455s, input_fired_time: 1.0562930107, output_fired_time: 0.0000000000, propagate_forward_time: 59.9529609680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2974433899
validation[          , genome  6399] predictions:    5764/  10000 (57.64%), best:    5790/10000 (57.90%), error:     13075.14453, best error:     12697.24902 on epoch:    29, epoch:   31/50, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 852.5503540039s, input_fired_time: 9.2853479385, output_fired_time: 6.6482768059, propagate_forward_time: 299.6384582520, propagate_backward_time: 524.0519409180, weight_update_time: 1.0442279577, other_time: 12.9263305664
epoch time: 63.2348442078s, input_fired_time: 1.0493034124, output_fired_time: 0.0000000000, propagate_forward_time: 59.9134826660, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2720565796
validation[          , genome  6399] predictions:    5553/  10000 (55.53%), best:    5790/10000 (57.90%), error:     13807.59961, best error:     12697.24902 on epoch:    29, epoch:   32/50, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 850.1839599609s, input_fired_time: 9.2203788757, output_fired_time: 6.6211791039, propagate_forward_time: 298.5693664551, propagate_backward_time: 523.1593017578, weight_update_time: 1.0274943113, other_time: 12.6137695313
epoch time: 62.9283103943s, input_fired_time: 1.0411783457, output_fired_time: 0.0000000000, propagate_forward_time: 59.5953559875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2917747498
validation[          , genome  6399] predictions:    5529/  10000 (55.29%), best:    5790/10000 (57.90%), error:     13679.48926, best error:     12697.24902 on epoch:    29, epoch:   33/50, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 848.5126342773s, input_fired_time: 9.1794090271, output_fired_time: 6.6249809265, propagate_forward_time: 298.0466003418, propagate_backward_time: 522.1399536133, weight_update_time: 1.0173182487, other_time: 12.5216674805
epoch time: 62.8953361511s, input_fired_time: 1.0213243961, output_fired_time: 0.0000000000, propagate_forward_time: 59.5923652649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2816467285
validation[          , genome  6399] predictions:    5719/  10000 (57.19%), best:    5790/10000 (57.90%), error:     13321.19043, best error:     12697.24902 on epoch:    29, epoch:   34/50, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 851.6618652344s, input_fired_time: 9.3233385086, output_fired_time: 6.6235423088, propagate_forward_time: 298.9079589844, propagate_backward_time: 524.0716552734, weight_update_time: 1.0244987011, other_time: 12.7353515625
epoch time: 62.7985725403s, input_fired_time: 1.0458974838, output_fired_time: 0.0000000000, propagate_forward_time: 59.4469375610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3057365417
validation[          , genome  6399] predictions:    5782/  10000 (57.82%), best:    5790/10000 (57.90%), error:     12844.82031, best error:     12697.24902 on epoch:    29, epoch:   35/50, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 850.2586059570s, input_fired_time: 9.2607631683, output_fired_time: 6.6478152275, propagate_forward_time: 298.1946411133, propagate_backward_time: 523.5132446289, weight_update_time: 1.0387686491, other_time: 12.6421508789
epoch time: 62.8511352539s, input_fired_time: 1.0437837839, output_fired_time: 0.0000000000, propagate_forward_time: 59.6452713013, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1620788574
validation[          , genome  6399] predictions:    5595/  10000 (55.95%), best:    5790/10000 (57.90%), error:     13558.16016, best error:     12697.24902 on epoch:    29, epoch:   36/50, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 855.6818237305s, input_fired_time: 9.2264289856, output_fired_time: 6.6444725990, propagate_forward_time: 299.0399475098, propagate_backward_time: 528.1295776367, weight_update_time: 1.0097345114, other_time: 12.6414184570
epoch time: 63.1319923401s, input_fired_time: 1.0283397436, output_fired_time: 0.0000000000, propagate_forward_time: 60.0033645630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1002883911
validation[          , genome  6399] predictions:    5640/  10000 (56.40%), best:    5790/10000 (57.90%), error:     13053.44629, best error:     12697.24902 on epoch:    29, epoch:   37/50, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 852.3323364258s, input_fired_time: 9.2646179199, output_fired_time: 6.6146235466, propagate_forward_time: 298.6691284180, propagate_backward_time: 525.5440063477, weight_update_time: 0.9984421134, other_time: 12.2399902344
epoch time: 64.6507873535s, input_fired_time: 1.0645384789, output_fired_time: 0.0000000000, propagate_forward_time: 61.2403678894, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3458824158
validation[          , genome  6399] predictions:    5683/  10000 (56.83%), best:    5790/10000 (57.90%), error:     13347.26953, best error:     12697.24902 on epoch:    29, epoch:   38/50, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 849.7290649414s, input_fired_time: 9.1971998215, output_fired_time: 6.6103668213, propagate_forward_time: 298.6839599609, propagate_backward_time: 522.8657226563, weight_update_time: 0.9983381629, other_time: 12.3718261719
epoch time: 63.1805114746s, input_fired_time: 1.0668283701, output_fired_time: 0.0000000000, propagate_forward_time: 59.7850646973, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3286170959
validation[          , genome  6399] predictions:    5795/  10000 (57.95%), best:    5795/10000 (57.95%), error:     12693.42383, best error:     12693.42383 on epoch:    39, epoch:   39/50, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 850.8470458984s, input_fired_time: 9.2919034958, output_fired_time: 6.6296486855, propagate_forward_time: 298.8069763184, propagate_backward_time: 523.4789428711, weight_update_time: 1.0191934109, other_time: 12.6395874023
epoch time: 63.0979499817s, input_fired_time: 1.0626668930, output_fired_time: 0.0000000000, propagate_forward_time: 59.7546386719, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2806434631
validation[          , genome  6399] predictions:    5952/  10000 (59.52%), best:    5952/10000 (59.52%), error:     12273.91211, best error:     12273.91211 on epoch:    40, epoch:   40/50, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 859.3139648438s, input_fired_time: 9.3899106979, output_fired_time: 6.6889386177, propagate_forward_time: 299.9714355469, propagate_backward_time: 530.2825317383, weight_update_time: 1.0402907133, other_time: 12.9811401367
epoch time: 63.2595558167s, input_fired_time: 1.0509294271, output_fired_time: 0.0000000000, propagate_forward_time: 59.9786567688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2299690247
validation[          , genome  6399] predictions:    5660/  10000 (56.60%), best:    5952/10000 (59.52%), error:     13318.37695, best error:     12273.91211 on epoch:    40, epoch:   41/50, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 857.5811767578s, input_fired_time: 9.3152627945, output_fired_time: 6.6256418228, propagate_forward_time: 306.0653686523, propagate_backward_time: 522.6842651367, weight_update_time: 1.0333893299, other_time: 12.8906250000
epoch time: 63.4250679016s, input_fired_time: 1.0474607944, output_fired_time: 0.0000000000, propagate_forward_time: 60.1229667664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2546386719
validation[          , genome  6399] predictions:    5693/  10000 (56.93%), best:    5952/10000 (59.52%), error:     12940.54980, best error:     12273.91211 on epoch:    40, epoch:   42/50, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 850.9915771484s, input_fired_time: 9.3480930328, output_fired_time: 6.6517834663, propagate_forward_time: 298.7892456055, propagate_backward_time: 523.2965698242, weight_update_time: 1.0278469324, other_time: 12.9058837891
epoch time: 62.8841590881s, input_fired_time: 1.0576680899, output_fired_time: 0.0000000000, propagate_forward_time: 59.5624809265, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2640113831
validation[          , genome  6399] predictions:    5901/  10000 (59.01%), best:    5952/10000 (59.52%), error:     12612.53320, best error:     12273.91211 on epoch:    40, epoch:   43/50, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 850.1628417969s, input_fired_time: 9.3007640839, output_fired_time: 6.6087040901, propagate_forward_time: 298.7999267578, propagate_backward_time: 522.8586425781, weight_update_time: 1.0277265310, other_time: 12.5947875977
epoch time: 63.1709289551s, input_fired_time: 1.0724849701, output_fired_time: 0.0000000000, propagate_forward_time: 59.8010101318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2974319458
validation[          , genome  6399] predictions:    5918/  10000 (59.18%), best:    5952/10000 (59.52%), error:     12480.79785, best error:     12273.91211 on epoch:    40, epoch:   44/50, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 853.5337524414s, input_fired_time: 9.4195098877, output_fired_time: 6.7141714096, propagate_forward_time: 299.8143005371, propagate_backward_time: 524.6077880859, weight_update_time: 1.0365905762, other_time: 12.9780273438
epoch time: 65.9794540405s, input_fired_time: 1.0781601667, output_fired_time: 0.0000000000, propagate_forward_time: 62.7161483765, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1851425171
validation[          , genome  6399] predictions:    5925/  10000 (59.25%), best:    5952/10000 (59.52%), error:     12498.90332, best error:     12273.91211 on epoch:    40, epoch:   45/50, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 859.1856079102s, input_fired_time: 9.3720827103, output_fired_time: 6.6755032539, propagate_forward_time: 300.0532531738, propagate_backward_time: 530.4913940430, weight_update_time: 1.0195261240, other_time: 12.5934448242
epoch time: 63.1942596436s, input_fired_time: 1.0772709846, output_fired_time: 0.0000000000, propagate_forward_time: 59.8518829346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2651062012
validation[          , genome  6399] predictions:    5796/  10000 (57.96%), best:    5952/10000 (59.52%), error:     12776.31348, best error:     12273.91211 on epoch:    40, epoch:   46/50, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 851.1256103516s, input_fired_time: 9.3354196548, output_fired_time: 6.7184691429, propagate_forward_time: 299.0269165039, propagate_backward_time: 523.5072021484, weight_update_time: 1.0171327591, other_time: 12.5375976563
epoch time: 63.2290000916s, input_fired_time: 1.0776784420, output_fired_time: 0.0000000000, propagate_forward_time: 59.8743782043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2769432068
validation[          , genome  6399] predictions:    5844/  10000 (58.44%), best:    5952/10000 (59.52%), error:     12540.94824, best error:     12273.91211 on epoch:    40, epoch:   47/50, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 852.6472167969s, input_fired_time: 9.3178005219, output_fired_time: 6.6426634789, propagate_forward_time: 299.0906982422, propagate_backward_time: 524.9476318359, weight_update_time: 1.0156835318, other_time: 12.6484375000
epoch time: 63.0573959351s, input_fired_time: 1.0794308186, output_fired_time: 0.0000000000, propagate_forward_time: 59.6838798523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2940864563
validation[          , genome  6399] predictions:    5943/  10000 (59.43%), best:    5952/10000 (59.52%), error:     12398.28320, best error:     12273.91211 on epoch:    40, epoch:   48/50, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 858.1791992188s, input_fired_time: 9.3037223816, output_fired_time: 6.5867929459, propagate_forward_time: 298.0353088379, propagate_backward_time: 532.0822143555, weight_update_time: 0.9934993982, other_time: 12.1712036133
epoch time: 62.9701614380s, input_fired_time: 1.0701358318, output_fired_time: 0.0000000000, propagate_forward_time: 59.5703239441, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3297004700
validation[          , genome  6399] predictions:    5886/  10000 (58.86%), best:    5952/10000 (59.52%), error:     12551.96680, best error:     12273.91211 on epoch:    40, epoch:   49/50, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 853.2606811523s, input_fired_time: 9.4400405884, output_fired_time: 6.7012314796, propagate_forward_time: 299.4194946289, propagate_backward_time: 524.8513793945, weight_update_time: 1.0293848515, other_time: 12.8485107422
epoch time: 62.7133102417s, input_fired_time: 1.0774252415, output_fired_time: 0.0000000000, propagate_forward_time: 59.4725685120, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1633148193
validation[          , genome  6399] predictions:    5862/  10000 (58.62%), best:    5952/10000 (59.52%), error:     12569.61621, best error:     12273.91211 on epoch:    40, epoch:   50/50, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 316.1624450684s, input_fired_time: 5.3043718338, output_fired_time: 0.0000000000, propagate_forward_time: 299.5662231445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.2918395996
best training[          , genome  6399] predictions:   34505/  50000 (69.01%), best:    5952/10000 (59.52%), error:     44679.53516, best error:     12273.91211 on epoch:    40, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 63.1307678223s, input_fired_time: 1.0671045780, output_fired_time: 0.0000000000, propagate_forward_time: 59.7604446411, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3032188416
   testing[          , genome  6399] predictions:    5952/  10000 (59.52%), best:    5952/10000 (59.52%), error:     12273.91211, best error:     12273.91211 on epoch:    40, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
backpropagation finished successfully!
19:44:00 (2548): called boinc_finish(0)

</stderr_txt>
]]>