Nameexact_genome_1542127803_181_5823_2
Workunit4086881
Created20 Nov 2018, 17:45:36 UTC
Sent20 Nov 2018, 17:46:01 UTC
Report deadline10 Jan 2019, 21:39:30 UTC
Received23 Nov 2018, 5:27:25 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88437
Run time15 hours 20 min 20 sec
CPU time15 hours 7 min 22 sec
Validate stateValid
Credit12,790.54
Device peak FLOPS4.09 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size551.33 MB
Peak swap size572.80 MB
Peak disk usage11.41 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_181_5823.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_181_5823_2_r270828242_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542127803_181_5823.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.479913
read mu: 0.479913
read mu_delta: 0.942073
read initial_learning_rate: 0.00216577
read learning_rate: 0.00216577
read learning_rate_delta: 0.916696
read initial_weight_decay: 0.000438746
read weight_decay: 0.000438746
read weight_decay_delta: 0.93114
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.0576008
read input_dropout_probability: 0.00128952
read hidden_dropout_probability: 0.107221
read velocity_reset: 1217
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5823
read normal distribution: '1 0 0'
generator_str: '1356408445'
read generator: 1356408445
read generated_by_map:
1 crossover 1
reading 95 nodes.
reading 345 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1617822210
post shuffle 1: 1855059709
epoch time: 1010.67944335938s, input_fired_time: 16.0485591888428, output_fired_time: 5.49990034103394, propagate_forward_time: 371.082580566406, propagate_backward_time: 607.074279785156, weight_update_time: 0.30349937081337, other_time: 10.97412109375
epoch time: 76.338493347168s, input_fired_time: 0.928962588310242, output_fired_time: 0, propagate_forward_time: 73.6936187744141, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.71591186523438
validation[          , genome  5823] predictions:    5249/  10000 (52.49%), best:    5249/10000 (52.49%), error:     13625.56348, best error:     13625.56348 on epoch:     0, epoch:    0/50, mu: 0.4799132645, learning_rate: 0.0021657690, weight_decay: 0.0004387464

epoch time: 991.8693847656s, input_fired_time: 15.9724340439, output_fired_time: 5.4504561424, propagate_forward_time: 360.1214294434, propagate_backward_time: 599.2968750000, weight_update_time: 0.3048575521, other_time: 11.0281982422
epoch time: 75.8092193604s, input_fired_time: 0.9370267987, output_fired_time: 0.0000000000, propagate_forward_time: 73.0531082153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8190841675
validation[          , genome  5823] predictions:    5611/  10000 (56.11%), best:    5611/10000 (56.11%), error:     12734.95898, best error:     12734.95898 on epoch:     1, epoch:    1/50, mu: 0.5094611049, learning_rate: 0.0019853511, weight_decay: 0.0004085345

epoch time: 994.1715087891s, input_fired_time: 16.1407032013, output_fired_time: 5.4659452438, propagate_forward_time: 360.4289550781, propagate_backward_time: 601.1641845703, weight_update_time: 0.2960896790, other_time: 10.9717407227
epoch time: 75.7354812622s, input_fired_time: 0.9377232194, output_fired_time: 0.0000000000, propagate_forward_time: 73.0073547363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7904052734
validation[          , genome  5823] predictions:    5536/  10000 (55.36%), best:    5611/10000 (56.11%), error:     12778.50586, best error:     12734.95898 on epoch:     1, epoch:    2/50, mu: 0.5372973084, learning_rate: 0.0018199629, weight_decay: 0.0003804029

epoch time: 995.7531738281s, input_fired_time: 16.1265544891, output_fired_time: 5.4812879562, propagate_forward_time: 361.1994628906, propagate_backward_time: 602.0125732422, weight_update_time: 0.3009564281, other_time: 10.9333496094
epoch time: 76.1223449707s, input_fired_time: 0.9264205694, output_fired_time: 0.0000000000, propagate_forward_time: 73.4685211182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7274017334
validation[          , genome  5823] predictions:    5625/  10000 (56.25%), best:    5625/10000 (56.25%), error:     12578.10938, best error:     12578.10938 on epoch:     3, epoch:    3/50, mu: 0.5635210276, learning_rate: 0.0016683522, weight_decay: 0.0003542085

epoch time: 1003.8994750977s, input_fired_time: 16.0758895874, output_fired_time: 5.4864444733, propagate_forward_time: 369.6235656738, propagate_backward_time: 601.8013305664, weight_update_time: 0.2990790009, other_time: 10.9122924805
epoch time: 75.9303131104s, input_fired_time: 0.9399794936, output_fired_time: 0.0000000000, propagate_forward_time: 73.3163528442, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6739807129
validation[          , genome  5823] predictions:    5679/  10000 (56.79%), best:    5679/10000 (56.79%), error:     12267.90723, best error:     12267.90723 on epoch:     4, epoch:    4/50, mu: 0.5882257223, learning_rate: 0.0015293712, weight_decay: 0.0003298178

epoch time: 993.7120971680s, input_fired_time: 16.0466747284, output_fired_time: 5.4603428841, propagate_forward_time: 360.5669860840, propagate_backward_time: 600.8793945313, weight_update_time: 0.2975807488, other_time: 10.7586669922
epoch time: 75.6314239502s, input_fired_time: 0.9350677729, output_fired_time: 0.0000000000, propagate_forward_time: 72.9235382080, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7728195190
validation[          , genome  5823] predictions:    5618/  10000 (56.18%), best:    5679/10000 (56.79%), error:     12642.95020, best error:     12267.90723 on epoch:     4, epoch:    5/50, mu: 0.6114993095, learning_rate: 0.0014019681, weight_decay: 0.0003071066

epoch time: 991.5759277344s, input_fired_time: 16.0892963409, output_fired_time: 5.4548530579, propagate_forward_time: 359.6626892090, propagate_backward_time: 599.2622680664, weight_update_time: 0.3048329651, other_time: 11.1068725586
epoch time: 75.8942565918s, input_fired_time: 0.9342829585, output_fired_time: 0.0000000000, propagate_forward_time: 73.2458877563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7140884399
validation[          , genome  5823] predictions:    5709/  10000 (57.09%), best:    5679/10000 (56.79%), error:     12360.26270, best error:     12267.90723 on epoch:     4, epoch:    6/50, mu: 0.6334247589, learning_rate: 0.0012851781, weight_decay: 0.0002859593

epoch time: 998.1606445313s, input_fired_time: 16.1604862213, output_fired_time: 5.4990229607, propagate_forward_time: 361.9861755371, propagate_backward_time: 603.5713500977, weight_update_time: 0.2955492735, other_time: 10.9436645508
epoch time: 75.5364608765s, input_fired_time: 0.9419555664, output_fired_time: 0.0000000000, propagate_forward_time: 72.7833786011, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8111267090
validation[          , genome  5823] predictions:    5825/  10000 (58.25%), best:    5825/10000 (58.25%), error:     11964.39355, best error:     11964.39355 on epoch:     7, epoch:    7/50, mu: 0.6540800929, learning_rate: 0.0011781172, weight_decay: 0.0002662683

epoch time: 1001.9979248047s, input_fired_time: 16.1540451050, output_fired_time: 5.4671516418, propagate_forward_time: 365.4999084473, propagate_backward_time: 603.9194946289, weight_update_time: 0.2974072099, other_time: 10.9573364258
epoch time: 75.8130111694s, input_fired_time: 0.9491256475, output_fired_time: 0.0000000000, propagate_forward_time: 73.1184005737, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7454833984
validation[          , genome  5823] predictions:    5718/  10000 (57.18%), best:    5825/10000 (58.25%), error:     12217.25781, best error:     11964.39355 on epoch:     7, epoch:    8/50, mu: 0.6735389233, learning_rate: 0.0010799750, weight_decay: 0.0002479331

epoch time: 995.9306030273s, input_fired_time: 16.0025558472, output_fired_time: 5.4866852760, propagate_forward_time: 359.4038696289, propagate_backward_time: 603.9379272461, weight_update_time: 0.3070697486, other_time: 11.0995483398
epoch time: 75.8305816650s, input_fired_time: 0.9368516207, output_fired_time: 0.0000000000, propagate_forward_time: 73.2164611816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6772689819
validation[          , genome  5823] predictions:    5928/  10000 (59.28%), best:    5928/10000 (59.28%), error:     11667.35938, best error:     11667.35938 on epoch:     9, epoch:    9/50, mu: 0.6918705702, learning_rate: 0.0009900085, weight_decay: 0.0002308605

epoch time: 998.8548583984s, input_fired_time: 16.1671333313, output_fired_time: 5.4833750725, propagate_forward_time: 362.2884216309, propagate_backward_time: 603.9622192383, weight_update_time: 0.3012438416, other_time: 10.9536743164
epoch time: 76.0620040894s, input_fired_time: 0.9449074864, output_fired_time: 0.0000000000, propagate_forward_time: 73.4625015259, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6545944214
validation[          , genome  5823] predictions:    5903/  10000 (59.03%), best:    5928/10000 (59.28%), error:     11869.83496, best error:     11667.35938 on epoch:     9, epoch:   10/50, mu: 0.7091403604, learning_rate: 0.0009075365, weight_decay: 0.0002149635

epoch time: 1003.2253417969s, input_fired_time: 16.1836986542, output_fired_time: 5.4700956345, propagate_forward_time: 362.2189941406, propagate_backward_time: 608.6282958984, weight_update_time: 0.2912048399, other_time: 10.7242431641
epoch time: 75.5088272095s, input_fired_time: 0.9387805462, output_fired_time: 0.0000000000, propagate_forward_time: 72.8877029419, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6823425293
validation[          , genome  5823] predictions:    5872/  10000 (58.72%), best:    5928/10000 (59.28%), error:     11850.65723, best error:     11667.35938 on epoch:     9, epoch:   11/50, mu: 0.7254097462, learning_rate: 0.0008319349, weight_decay: 0.0002001612

epoch time: 998.3806762695s, input_fired_time: 16.1632518768, output_fired_time: 5.5201506615, propagate_forward_time: 362.2384948730, propagate_backward_time: 603.5924072266, weight_update_time: 0.2940656841, other_time: 10.8663330078
epoch time: 75.6019058228s, input_fired_time: 0.9261704087, output_fired_time: 0.0000000000, propagate_forward_time: 72.9445724487, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7311630249
validation[          , genome  5823] predictions:    5808/  10000 (58.08%), best:    5928/10000 (59.28%), error:     11960.90039, best error:     11667.35938 on epoch:     9, epoch:   12/50, mu: 0.7407366633, learning_rate: 0.0007626311, weight_decay: 0.0001863782

epoch time: 996.0888671875s, input_fired_time: 16.1531085968, output_fired_time: 5.4990539551, propagate_forward_time: 361.4962463379, propagate_backward_time: 602.0897216797, weight_update_time: 0.2957461476, other_time: 10.8507080078
epoch time: 76.1129531860s, input_fired_time: 0.9544355869, output_fired_time: 0.0000000000, propagate_forward_time: 73.4033737183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7551422119
validation[          , genome  5823] predictions:    5783/  10000 (57.83%), best:    5928/10000 (59.28%), error:     12051.02539, best error:     11667.35938 on epoch:     9, epoch:   13/50, mu: 0.7551757693, learning_rate: 0.0006991007, weight_decay: 0.0001735442

epoch time: 995.2792968750s, input_fired_time: 16.1659679413, output_fired_time: 5.4764175415, propagate_forward_time: 360.9587097168, propagate_backward_time: 601.8193969727, weight_update_time: 0.2953847349, other_time: 10.8588256836
epoch time: 75.1078414917s, input_fired_time: 0.9401761889, output_fired_time: 0.0000000000, propagate_forward_time: 72.3921356201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7755279541
validation[          , genome  5823] predictions:    5926/  10000 (59.26%), best:    5926/10000 (59.26%), error:     11664.59863, best error:     11664.59863 on epoch:    14, epoch:   14/50, mu: 0.7687784433, learning_rate: 0.0006408626, weight_decay: 0.0001615940

epoch time: 997.3087768555s, input_fired_time: 16.1413974762, output_fired_time: 5.4496412277, propagate_forward_time: 360.1865844727, propagate_backward_time: 604.8334350586, weight_update_time: 0.2897948623, other_time: 10.6976928711
epoch time: 75.5082626343s, input_fired_time: 0.9650794268, output_fired_time: 0.0000000000, propagate_forward_time: 72.8271255493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7160568237
validation[          , genome  5823] predictions:    5777/  10000 (57.77%), best:    5926/10000 (59.26%), error:     12097.60547, best error:     11664.59863 on epoch:    14, epoch:   15/50, mu: 0.7815931439, learning_rate: 0.0005874760, weight_decay: 0.0001504667

epoch time: 992.1556396484s, input_fired_time: 16.1120414734, output_fired_time: 5.4645562172, propagate_forward_time: 360.0989685059, propagate_backward_time: 599.6088867188, weight_update_time: 0.2955221236, other_time: 10.8712158203
epoch time: 76.4131469727s, input_fired_time: 0.9686047435, output_fired_time: 0.0000000000, propagate_forward_time: 73.7355270386, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7090148926
validation[          , genome  5823] predictions:    5996/  10000 (59.96%), best:    5996/10000 (59.96%), error:     11549.41797, best error:     11549.41797 on epoch:    16, epoch:   16/50, mu: 0.7936655283, learning_rate: 0.0005385367, weight_decay: 0.0001401056

epoch time: 992.5557861328s, input_fired_time: 16.1730632782, output_fired_time: 5.4904336929, propagate_forward_time: 360.0865173340, propagate_backward_time: 599.8183593750, weight_update_time: 0.2967254817, other_time: 10.9874267578
epoch time: 75.7536163330s, input_fired_time: 0.9520069361, output_fired_time: 0.0000000000, propagate_forward_time: 73.0748367310, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7267761230
validation[          , genome  5823] predictions:    5968/  10000 (59.68%), best:    5968/10000 (59.68%), error:     11516.78223, best error:     11516.78223 on epoch:    17, epoch:   17/50, mu: 0.8050386310, learning_rate: 0.0004936743, weight_decay: 0.0001304580

epoch time: 994.7227783203s, input_fired_time: 16.1922283173, output_fired_time: 5.4425745010, propagate_forward_time: 361.6022644043, propagate_backward_time: 600.5154418945, weight_update_time: 0.3066065907, other_time: 10.9702758789
epoch time: 75.7682723999s, input_fired_time: 0.9708303809, output_fired_time: 0.0000000000, propagate_forward_time: 73.0562438965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7411956787
validation[          , genome  5823] predictions:    5981/  10000 (59.81%), best:    5981/10000 (59.81%), error:     11441.64453, best error:     11441.64453 on epoch:    18, epoch:   18/50, mu: 0.8157529235, learning_rate: 0.0004525491, weight_decay: 0.0001214747

epoch time: 996.6375732422s, input_fired_time: 16.2215633392, output_fired_time: 5.4701299667, propagate_forward_time: 364.1361999512, propagate_backward_time: 599.5222167969, weight_update_time: 0.2952601910, other_time: 11.2874755859
epoch time: 75.5688781738s, input_fired_time: 0.9760718942, output_fired_time: 0.0000000000, propagate_forward_time: 72.7875442505, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8052597046
validation[          , genome  5823] predictions:    6091/  10000 (60.91%), best:    6091/10000 (60.91%), error:     11253.88672, best error:     11253.88672 on epoch:    19, epoch:   19/50, mu: 0.8258465528, learning_rate: 0.0004148498, weight_decay: 0.0001131100

epoch time: 990.8853149414s, input_fired_time: 16.1536598206, output_fired_time: 5.4553866386, propagate_forward_time: 359.1743164063, propagate_backward_time: 599.0988769531, weight_update_time: 0.3053556681, other_time: 11.0030517578
epoch time: 75.6961364746s, input_fired_time: 0.9596567154, output_fired_time: 0.0000000000, propagate_forward_time: 73.0915069580, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6449737549
validation[          , genome  5823] predictions:    6019/  10000 (60.19%), best:    6091/10000 (60.91%), error:     11539.40039, best error:     11253.88672 on epoch:    19, epoch:   20/50, mu: 0.8353554606, learning_rate: 0.0003802911, weight_decay: 0.0001053213

epoch time: 996.2072753906s, input_fired_time: 16.2131175995, output_fired_time: 5.4761886597, propagate_forward_time: 361.0623779297, propagate_backward_time: 602.8796386719, weight_update_time: 0.2817318141, other_time: 10.5759277344
epoch time: 76.3921127319s, input_fired_time: 0.9652549028, output_fired_time: 0.0000000000, propagate_forward_time: 73.7453308105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6815261841
validation[          , genome  5823] predictions:    5995/  10000 (59.95%), best:    6091/10000 (60.91%), error:     11362.96191, best error:     11253.88672 on epoch:    19, epoch:   21/50, mu: 0.8443135619, learning_rate: 0.0003486112, weight_decay: 0.0000980689

epoch time: 998.8568725586s, input_fired_time: 16.1796531677, output_fired_time: 5.4760870934, propagate_forward_time: 361.6772155762, propagate_backward_time: 604.7828369141, weight_update_time: 0.2932478487, other_time: 10.7410888672
epoch time: 76.2435684204s, input_fired_time: 0.9858177304, output_fired_time: 0.0000000000, propagate_forward_time: 73.4839706421, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7737808228
validation[          , genome  5823] predictions:    6073/  10000 (60.73%), best:    6091/10000 (60.91%), error:     11327.36914, best error:     11253.88672 on epoch:    19, epoch:   22/50, mu: 0.8527527452, learning_rate: 0.0003195704, weight_decay: 0.0000913159

epoch time: 1000.0508422852s, input_fired_time: 16.2620105743, output_fired_time: 5.4606995583, propagate_forward_time: 361.0583190918, propagate_backward_time: 606.3298339844, weight_update_time: 0.2941245437, other_time: 10.9399414063
epoch time: 79.1750106812s, input_fired_time: 0.9608257413, output_fired_time: 0.0000000000, propagate_forward_time: 76.5067672729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7074203491
validation[          , genome  5823] predictions:    6135/  10000 (61.35%), best:    6135/10000 (61.35%), error:     11137.68750, best error:     11137.68750 on epoch:    23, epoch:   23/50, mu: 0.8607030511, learning_rate: 0.0002929488, weight_decay: 0.0000850279

epoch time: 993.3999633789s, input_fired_time: 16.2841339111, output_fired_time: 5.4676423073, propagate_forward_time: 360.2527770996, propagate_backward_time: 600.3753662109, weight_update_time: 0.3053334355, other_time: 11.0200195313
epoch time: 75.7046508789s, input_fired_time: 0.9683775306, output_fired_time: 0.0000000000, propagate_forward_time: 73.0332489014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7030258179
validation[          , genome  5823] predictions:    6063/  10000 (60.63%), best:    6135/10000 (61.35%), error:     11196.83984, best error:     11137.68750 on epoch:    23, epoch:   24/50, mu: 0.8681928515, learning_rate: 0.0002685449, weight_decay: 0.0000791729

epoch time: 992.5718994141s, input_fired_time: 16.2066497803, output_fired_time: 5.4581537247, propagate_forward_time: 360.0500183105, propagate_backward_time: 600.1143188477, weight_update_time: 0.2911962867, other_time: 10.7427368164
epoch time: 75.6346817017s, input_fired_time: 0.9730489254, output_fired_time: 0.0000000000, propagate_forward_time: 72.8544540405, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8071823120
validation[          , genome  5823] predictions:    6137/  10000 (61.37%), best:    6135/10000 (61.35%), error:     11237.03223, best error:     11137.68750 on epoch:    23, epoch:   25/50, mu: 0.8752487898, learning_rate: 0.0002461740, weight_decay: 0.0000737211

epoch time: 994.6423950195s, input_fired_time: 16.3702087402, output_fired_time: 5.4636030197, propagate_forward_time: 360.7467956543, propagate_backward_time: 601.1505126953, weight_update_time: 0.2955399156, other_time: 10.9112548828
epoch time: 75.2157363892s, input_fired_time: 0.9665263891, output_fired_time: 0.0000000000, propagate_forward_time: 72.5180130005, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7311935425
validation[          , genome  5823] predictions:    6023/  10000 (60.23%), best:    6135/10000 (61.35%), error:     11474.69727, best error:     11137.68750 on epoch:    23, epoch:   26/50, mu: 0.8818960190, learning_rate: 0.0002256666, weight_decay: 0.0000686447

epoch time: 993.3030395508s, input_fired_time: 16.2652435303, output_fired_time: 5.5138316154, propagate_forward_time: 360.3124084473, propagate_backward_time: 600.3282470703, weight_update_time: 0.2944121361, other_time: 10.8833007813
epoch time: 75.5022735596s, input_fired_time: 0.9923879504, output_fired_time: 0.0000000000, propagate_forward_time: 72.7299804688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7799072266
validation[          , genome  5823] predictions:    5982/  10000 (59.82%), best:    6135/10000 (61.35%), error:     11425.91504, best error:     11137.68750 on epoch:    23, epoch:   27/50, mu: 0.8881581426, learning_rate: 0.0002068676, weight_decay: 0.0000639178

epoch time: 992.7592773438s, input_fired_time: 16.2703075409, output_fired_time: 5.4915909767, propagate_forward_time: 360.2485961914, propagate_backward_time: 599.9707641602, weight_update_time: 0.2912170887, other_time: 10.7780151367
epoch time: 75.5898666382s, input_fired_time: 0.9937203526, output_fired_time: 0.0000000000, propagate_forward_time: 72.8097381592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7864074707
validation[          , genome  5823] predictions:    6137/  10000 (61.37%), best:    6135/10000 (61.35%), error:     11168.81348, best error:     11137.68750 on epoch:    23, epoch:   28/50, mu: 0.8940575719, learning_rate: 0.0001896347, weight_decay: 0.0000595164

epoch time: 995.1431274414s, input_fired_time: 16.3858718872, output_fired_time: 5.4693269730, propagate_forward_time: 361.3200378418, propagate_backward_time: 601.1190185547, weight_update_time: 0.2981034517, other_time: 10.8488769531
epoch time: 75.7555313110s, input_fired_time: 0.9902781248, output_fired_time: 0.0000000000, propagate_forward_time: 73.0900802612, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6751708984
validation[          , genome  5823] predictions:    6163/  10000 (61.63%), best:    6163/10000 (61.63%), error:     11002.78711, best error:     11002.78711 on epoch:    29, epoch:   29/50, mu: 0.8996152282, learning_rate: 0.0001738373, weight_decay: 0.0000554182

epoch time: 998.9370727539s, input_fired_time: 16.3402767181, output_fired_time: 5.4660840034, propagate_forward_time: 360.9378356934, propagate_backward_time: 605.2231445313, weight_update_time: 0.3004020751, other_time: 10.9697265625
epoch time: 77.3162384033s, input_fired_time: 0.9727333188, output_fired_time: 0.0000000000, propagate_forward_time: 74.5490646362, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7944412231
validation[          , genome  5823] predictions:    6182/  10000 (61.82%), best:    6163/10000 (61.63%), error:     11045.21875, best error:     11002.78711 on epoch:    29, epoch:   30/50, mu: 0.9048509598, learning_rate: 0.0001593559, weight_decay: 0.0000516021

epoch time: 990.9955444336s, input_fired_time: 16.2975101471, output_fired_time: 5.4381947517, propagate_forward_time: 359.6831054688, propagate_backward_time: 598.6227416992, weight_update_time: 0.3026517034, other_time: 10.9540405273
epoch time: 75.1061248779s, input_fired_time: 0.9811768532, output_fired_time: 0.0000000000, propagate_forward_time: 72.3655700684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7593765259
validation[          , genome  5823] predictions:    6166/  10000 (61.66%), best:    6163/10000 (61.63%), error:     11007.22266, best error:     11002.78711 on epoch:    29, epoch:   31/50, mu: 0.9097834229, learning_rate: 0.0001460809, weight_decay: 0.0000480488

epoch time: 993.1777343750s, input_fired_time: 16.2757606506, output_fired_time: 5.4811615944, propagate_forward_time: 360.4558410645, propagate_backward_time: 600.3307495117, weight_update_time: 0.2918959856, other_time: 10.6342163086
epoch time: 75.6066436768s, input_fired_time: 0.9900100231, output_fired_time: 0.0000000000, propagate_forward_time: 72.8910064697, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7256240845
validation[          , genome  5823] predictions:    6111/  10000 (61.11%), best:    6163/10000 (61.63%), error:     11150.04004, best error:     11002.78711 on epoch:    29, epoch:   32/50, mu: 0.9144301414, learning_rate: 0.0001339117, weight_decay: 0.0000447402

epoch time: 995.6915283203s, input_fired_time: 19.5705337524, output_fired_time: 5.4879422188, propagate_forward_time: 360.4886474609, propagate_backward_time: 599.1615600586, weight_update_time: 0.3048890829, other_time: 10.9828491211
epoch time: 75.6072998047s, input_fired_time: 0.9860631227, output_fired_time: 0.0000000000, propagate_forward_time: 72.8850860596, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7361526489
validation[          , genome  5823] predictions:    6101/  10000 (61.01%), best:    6163/10000 (61.63%), error:     11247.51758, best error:     11002.78711 on epoch:    29, epoch:   33/50, mu: 0.9188076854, learning_rate: 0.0001227563, weight_decay: 0.0000416594

epoch time: 993.8630371094s, input_fired_time: 16.3903865814, output_fired_time: 5.4972691536, propagate_forward_time: 360.2251281738, propagate_backward_time: 600.8897094727, weight_update_time: 0.2961830497, other_time: 10.8605346680
epoch time: 76.4711990356s, input_fired_time: 0.9783489108, output_fired_time: 0.0000000000, propagate_forward_time: 73.8153076172, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6775436401
validation[          , genome  5823] predictions:    6124/  10000 (61.24%), best:    6163/10000 (61.63%), error:     11204.41016, best error:     11002.78711 on epoch:    29, epoch:   34/50, mu: 0.9229316711, learning_rate: 0.0001125301, weight_decay: 0.0000387907

epoch time: 995.4957885742s, input_fired_time: 16.3289718628, output_fired_time: 5.5726790428, propagate_forward_time: 360.8210449219, propagate_backward_time: 601.8405761719, weight_update_time: 0.3038877547, other_time: 10.9324951172
epoch time: 75.6327743530s, input_fired_time: 0.9768538475, output_fired_time: 0.0000000000, propagate_forward_time: 73.0090408325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6468811035
validation[          , genome  5823] predictions:    6109/  10000 (61.09%), best:    6163/10000 (61.63%), error:     11126.25195, best error:     11002.78711 on epoch:    29, epoch:   35/50, mu: 0.9268167615, learning_rate: 0.0001031559, weight_decay: 0.0000361196

epoch time: 990.5649414063s, input_fired_time: 16.2895317078, output_fired_time: 5.4557695389, propagate_forward_time: 359.6427307129, propagate_backward_time: 598.1636962891, weight_update_time: 0.3042480052, other_time: 11.0131835938
epoch time: 75.3205947876s, input_fired_time: 0.9683256745, output_fired_time: 0.0000000000, propagate_forward_time: 72.6428756714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7093963623
validation[          , genome  5823] predictions:    6225/  10000 (62.25%), best:    6225/10000 (62.25%), error:     10986.87988, best error:     10986.87988 on epoch:    36, epoch:   36/50, mu: 0.9304767847, learning_rate: 0.0000945626, weight_decay: 0.0000336324

epoch time: 996.3339843750s, input_fired_time: 16.4211463928, output_fired_time: 5.4604597092, propagate_forward_time: 361.1910095215, propagate_backward_time: 602.2468261719, weight_update_time: 0.2984231710, other_time: 11.0145263672
epoch time: 75.6639328003s, input_fired_time: 1.0637528896, output_fired_time: 0.0000000000, propagate_forward_time: 72.7658767700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8343048096
validation[          , genome  5823] predictions:    6213/  10000 (62.13%), best:    6213/10000 (62.13%), error:     10890.75098, best error:     10890.75098 on epoch:    37, epoch:   37/50, mu: 0.9339247942, learning_rate: 0.0000866851, weight_decay: 0.0000313165

epoch time: 994.5950927734s, input_fired_time: 16.3589839935, output_fired_time: 5.4842252731, propagate_forward_time: 360.8301086426, propagate_backward_time: 601.1437377930, weight_update_time: 0.2965651453, other_time: 10.7780151367
epoch time: 75.7030868530s, input_fired_time: 0.9823213816, output_fired_time: 0.0000000000, propagate_forward_time: 73.0904693604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6302947998
validation[          , genome  5823] predictions:    6247/  10000 (62.47%), best:    6247/10000 (62.47%), error:     10826.29883, best error:     10826.29883 on epoch:    38, epoch:   38/50, mu: 0.9371730685, learning_rate: 0.0000794639, weight_decay: 0.0000291600

epoch time: 994.3944702148s, input_fired_time: 16.4607734680, output_fired_time: 5.4631643295, propagate_forward_time: 360.3491516113, propagate_backward_time: 601.1585693359, weight_update_time: 0.2977503836, other_time: 10.9628906250
epoch time: 75.7587280273s, input_fired_time: 0.9970701337, output_fired_time: 0.0000000000, propagate_forward_time: 73.0595855713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7020721436
validation[          , genome  5823] predictions:    6198/  10000 (61.98%), best:    6198/10000 (61.98%), error:     10815.93164, best error:     10815.93164 on epoch:    39, epoch:   39/50, mu: 0.9402331710, learning_rate: 0.0000728442, weight_decay: 0.0000271521

epoch time: 997.5147094727s, input_fired_time: 16.4079017639, output_fired_time: 5.5295419693, propagate_forward_time: 362.2504577637, propagate_backward_time: 602.2874755859, weight_update_time: 0.3022744656, other_time: 11.0393066406
epoch time: 75.7272796631s, input_fired_time: 1.0083220005, output_fired_time: 0.0000000000, propagate_forward_time: 72.8931350708, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8258209229
validation[          , genome  5823] predictions:    6302/  10000 (63.02%), best:    6302/10000 (63.02%), error:     10729.82227, best error:     10729.82227 on epoch:    40, epoch:   40/50, mu: 0.9431160092, learning_rate: 0.0000667760, weight_decay: 0.0000252824

epoch time: 996.9955444336s, input_fired_time: 16.3691902161, output_fired_time: 5.4757299423, propagate_forward_time: 364.5803222656, propagate_backward_time: 599.4666748047, weight_update_time: 0.3044620454, other_time: 11.1036376953
epoch time: 74.8914718628s, input_fired_time: 0.9842873812, output_fired_time: 0.0000000000, propagate_forward_time: 72.1412429810, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7659378052
validation[          , genome  5823] predictions:    6260/  10000 (62.60%), best:    6302/10000 (63.02%), error:     10808.67090, best error:     10729.82227 on epoch:    40, epoch:   41/50, mu: 0.9458318353, learning_rate: 0.0000612132, weight_decay: 0.0000235415

epoch time: 1003.3276367188s, input_fired_time: 16.4810276031, output_fired_time: 5.4894127846, propagate_forward_time: 362.3452453613, propagate_backward_time: 608.1052246094, weight_update_time: 0.3057804108, other_time: 10.9067382813
epoch time: 76.2722549438s, input_fired_time: 1.0262359381, output_fired_time: 0.0000000000, propagate_forward_time: 73.3605499268, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8854675293
validation[          , genome  5823] predictions:    6268/  10000 (62.68%), best:    6302/10000 (63.02%), error:     10738.56055, best error:     10729.82227 on epoch:    40, epoch:   42/50, mu: 0.9483903646, learning_rate: 0.0000561139, weight_decay: 0.0000219204

epoch time: 998.1255493164s, input_fired_time: 16.5297508240, output_fired_time: 5.5011658669, propagate_forward_time: 361.6345825195, propagate_backward_time: 603.3157348633, weight_update_time: 0.3092423081, other_time: 11.1443481445
epoch time: 75.5284118652s, input_fired_time: 0.9999303222, output_fired_time: 0.0000000000, propagate_forward_time: 72.8021926880, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7262878418
validation[          , genome  5823] predictions:    6281/  10000 (62.81%), best:    6302/10000 (63.02%), error:     10733.32422, best error:     10729.82227 on epoch:    40, epoch:   43/50, mu: 0.9508007169, learning_rate: 0.0000514394, weight_decay: 0.0000204110

epoch time: 1008.0313720703s, input_fired_time: 19.5285568237, output_fired_time: 5.4721622467, propagate_forward_time: 368.0773925781, propagate_backward_time: 604.0364990234, weight_update_time: 0.3024929762, other_time: 10.9167480469
epoch time: 76.9251098633s, input_fired_time: 1.0351649523, output_fired_time: 0.0000000000, propagate_forward_time: 74.1689529419, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7209930420
validation[          , genome  5823] predictions:    6164/  10000 (61.64%), best:    6302/10000 (63.02%), error:     10928.51465, best error:     10729.82227 on epoch:    40, epoch:   44/50, mu: 0.9530714154, learning_rate: 0.0000471543, weight_decay: 0.0000190055

epoch time: 1005.4451904297s, input_fired_time: 17.4631252289, output_fired_time: 5.5614390373, propagate_forward_time: 366.2565917969, propagate_backward_time: 605.3297119141, weight_update_time: 0.2911234498, other_time: 10.8342895508
epoch time: 75.9711456299s, input_fired_time: 1.0252947807, output_fired_time: 0.0000000000, propagate_forward_time: 73.0985870361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8472671509
validation[          , genome  5823] predictions:    6262/  10000 (62.62%), best:    6302/10000 (63.02%), error:     10805.97070, best error:     10729.82227 on epoch:    40, epoch:   45/50, mu: 0.9552105665, learning_rate: 0.0000432261, weight_decay: 0.0000176968

epoch time: 1002.2763671875s, input_fired_time: 16.5107326508, output_fired_time: 5.5054521561, propagate_forward_time: 366.4664001465, propagate_backward_time: 602.7382202148, weight_update_time: 0.3060600758, other_time: 11.0556030273
epoch time: 76.8726272583s, input_fired_time: 1.0134567022, output_fired_time: 0.0000000000, propagate_forward_time: 74.1165695190, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7425994873
validation[          , genome  5823] predictions:    6236/  10000 (62.36%), best:    6302/10000 (63.02%), error:     10898.85840, best error:     10729.82227 on epoch:    40, epoch:   46/50, mu: 0.9572257996, learning_rate: 0.0000396252, weight_decay: 0.0000164782

epoch time: 1007.0187377930s, input_fired_time: 16.7524089813, output_fired_time: 5.5683159828, propagate_forward_time: 365.2861633301, propagate_backward_time: 608.3334960938, weight_update_time: 0.3078364730, other_time: 11.0783691406
epoch time: 77.0679473877s, input_fired_time: 1.0376720428, output_fired_time: 0.0000000000, propagate_forward_time: 74.1724777222, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8577957153
validation[          , genome  5823] predictions:    6267/  10000 (62.67%), best:    6267/10000 (62.67%), error:     10683.40820, best error:     10683.40820 on epoch:    47, epoch:   47/50, mu: 0.9591243267, learning_rate: 0.0000363242, weight_decay: 0.0000153435

epoch time: 1003.9003295898s, input_fired_time: 16.6029987335, output_fired_time: 5.5331888199, propagate_forward_time: 364.1646728516, propagate_backward_time: 606.4941406250, weight_update_time: 0.2980413437, other_time: 11.1052856445
epoch time: 75.7391281128s, input_fired_time: 0.9989815950, output_fired_time: 0.0000000000, propagate_forward_time: 73.0332183838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7069244385
validation[          , genome  5823] predictions:    6283/  10000 (62.83%), best:    6267/10000 (62.67%), error:     10747.60254, best error:     10683.40820 on epoch:    47, epoch:   48/50, mu: 0.9609128833, learning_rate: 0.0000332983, weight_decay: 0.0000142869

epoch time: 996.9469604492s, input_fired_time: 16.5101528168, output_fired_time: 5.4899721146, propagate_forward_time: 361.4471740723, propagate_backward_time: 602.4349975586, weight_update_time: 0.3091925681, other_time: 11.0646362305
epoch time: 75.4967956543s, input_fired_time: 1.0090217590, output_fired_time: 0.0000000000, propagate_forward_time: 72.6578063965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8299713135
validation[          , genome  5823] predictions:    6356/  10000 (63.56%), best:    6356/10000 (63.56%), error:     10595.47168, best error:     10595.47168 on epoch:    49, epoch:   49/50, mu: 0.9625977874, learning_rate: 0.0000305244, weight_decay: 0.0000133032

epoch time: 1006.1939086914s, input_fired_time: 16.6092052460, output_fired_time: 5.5167031288, propagate_forward_time: 362.7262573242, propagate_backward_time: 610.0921630859, weight_update_time: 0.3096958995, other_time: 11.2495727539
epoch time: 76.3589630127s, input_fired_time: 1.0272749662, output_fired_time: 0.0000000000, propagate_forward_time: 73.5350875854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7966003418
validation[          , genome  5823] predictions:    6313/  10000 (63.13%), best:    6356/10000 (63.56%), error:     10620.05078, best error:     10595.47168 on epoch:    49, epoch:   50/50, mu: 0.9641851187, learning_rate: 0.0000279816, weight_decay: 0.0000123871

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 376.2027587891s, input_fired_time: 5.0743441582, output_fired_time: 0.0000000000, propagate_forward_time: 362.1827697754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.9456481934
best training[          , genome  5823] predictions:   35032/  50000 (70.06%), best:    6356/10000 (63.56%), error:     45113.57422, best error:     10595.47168 on epoch:    49, epoch:   51/50, mu: 0.9656804800, learning_rate: 0.0000256506, weight_decay: 0.0000115341
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 75.0456619263s, input_fired_time: 0.9860635996, output_fired_time: 0.0000000000, propagate_forward_time: 72.2539901733, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8056106567
   testing[          , genome  5823] predictions:    6356/  10000 (63.56%), best:    6356/10000 (63.56%), error:     10595.47168, best error:     10595.47168 on epoch:    49, epoch:   51/50, mu: 0.9656804800, learning_rate: 0.0000256506, weight_decay: 0.0000115341
backpropagation finished successfully!
23:27:10 (7024): called boinc_finish(0)

</stderr_txt>
]]>