Nameexact_genome_1542727803_176_6419_2
Workunit4101301
Created20 Nov 2018, 20:14:29 UTC
Sent20 Nov 2018, 20:15:03 UTC
Report deadline24 Dec 2018, 17:05:25 UTC
Received21 Nov 2018, 10:45:12 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60660
Run time10 hours 36 min 32 sec
CPU time10 hours 17 min 32 sec
Validate stateValid
Credit8,467.08
Device peak FLOPS3.44 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size518.67 MB
Peak swap size538.32 MB
Peak disk usage8.97 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_176_6419.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_176_6419_2_r1177499395_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542727803_176_6419.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6419
read normal distribution: '1 -0.43371969461441 -0.376107901334763'
generator_str: '1317903425'
read generator: 1317903425
read generated_by_map:
1 split_node 1
reading 75 nodes.
reading 275 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 856528817
post shuffle 1: 415063791
epoch time: 678.292602539063s, input_fired_time: 6.485680103302, output_fired_time: 4.84799766540527, propagate_forward_time: 243.624420166016, propagate_backward_time: 413.878875732422, weight_update_time: 0.94337409734726, other_time: 9.45562744140625
epoch time: 50.810676574707s, input_fired_time: 0.744841635227203, output_fired_time: 0, propagate_forward_time: 48.4522361755371, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.61359786987305
validation[          , genome  6419] predictions:    4731/  10000 (47.31%), best:    4731/10000 (47.31%), error:     18289.98242, best error:     18289.98242 on epoch:     0, epoch:    0/50, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 679.1387329102s, input_fired_time: 6.4605183601, output_fired_time: 4.7913293839, propagate_forward_time: 243.5889282227, propagate_backward_time: 414.8941040039, weight_update_time: 0.9459625483, other_time: 9.4038696289
epoch time: 51.2745704651s, input_fired_time: 0.7283135653, output_fired_time: 0.0000000000, propagate_forward_time: 48.9012107849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6450462341
validation[          , genome  6419] predictions:    4858/  10000 (48.58%), best:    4858/10000 (48.58%), error:     16729.13867, best error:     16729.13867 on epoch:     1, epoch:    1/50, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 679.3158569336s, input_fired_time: 6.4185109138, output_fired_time: 4.8508114815, propagate_forward_time: 243.7144775391, propagate_backward_time: 414.8742370605, weight_update_time: 0.9455819726, other_time: 9.4577941895
epoch time: 50.9306068420s, input_fired_time: 0.7112118602, output_fired_time: 0.0000000000, propagate_forward_time: 48.5515785217, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6678161621
validation[          , genome  6419] predictions:    4963/  10000 (49.63%), best:    4963/10000 (49.63%), error:     16169.55078, best error:     16169.55078 on epoch:     2, epoch:    2/50, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 679.1118774414s, input_fired_time: 6.4373936653, output_fired_time: 4.8122024536, propagate_forward_time: 243.6780395508, propagate_backward_time: 414.6825256348, weight_update_time: 0.9482111931, other_time: 9.5017395020
epoch time: 51.0155982971s, input_fired_time: 0.7216190696, output_fired_time: 0.0000000000, propagate_forward_time: 48.6387138367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6552658081
validation[          , genome  6419] predictions:    4977/  10000 (49.77%), best:    4977/10000 (49.77%), error:     15572.24316, best error:     15572.24316 on epoch:     3, epoch:    3/50, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 680.1187133789s, input_fired_time: 6.3906946182, output_fired_time: 4.8959136009, propagate_forward_time: 244.2444763184, propagate_backward_time: 415.1432800293, weight_update_time: 0.9460176826, other_time: 9.4443359375
epoch time: 52.0204010010s, input_fired_time: 0.7400556803, output_fired_time: 0.0000000000, propagate_forward_time: 49.6532974243, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6270484924
validation[          , genome  6419] predictions:    5063/  10000 (50.63%), best:    5063/10000 (50.63%), error:     15497.54004, best error:     15497.54004 on epoch:     4, epoch:    4/50, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 678.4417114258s, input_fired_time: 6.3885998726, output_fired_time: 4.8315382004, propagate_forward_time: 243.6596221924, propagate_backward_time: 414.1129150391, weight_update_time: 0.9537678957, other_time: 9.4490356445
epoch time: 50.6460571289s, input_fired_time: 0.7099193335, output_fired_time: 0.0000000000, propagate_forward_time: 48.3141059875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6220321655
validation[          , genome  6419] predictions:    4885/  10000 (48.85%), best:    5063/10000 (50.63%), error:     15579.68652, best error:     15497.54004 on epoch:     4, epoch:    5/50, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 678.2391967773s, input_fired_time: 6.3903675079, output_fired_time: 4.8424949646, propagate_forward_time: 243.5705413818, propagate_backward_time: 413.9782409668, weight_update_time: 0.9448707700, other_time: 9.4575500488
epoch time: 50.8160781860s, input_fired_time: 0.7046034336, output_fired_time: 0.0000000000, propagate_forward_time: 48.4860000610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6254730225
validation[          , genome  6419] predictions:    5064/  10000 (50.64%), best:    5063/10000 (50.63%), error:     15769.44531, best error:     15497.54004 on epoch:     4, epoch:    6/50, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 679.2350463867s, input_fired_time: 6.3567743301, output_fired_time: 4.8663887978, propagate_forward_time: 243.9651336670, propagate_backward_time: 414.6529235840, weight_update_time: 0.9412239194, other_time: 9.3938293457
epoch time: 50.7833557129s, input_fired_time: 0.7279089093, output_fired_time: 0.0000000000, propagate_forward_time: 48.3993988037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6560478210
validation[          , genome  6419] predictions:    5174/  10000 (51.74%), best:    5174/10000 (51.74%), error:     14973.16797, best error:     14973.16797 on epoch:     7, epoch:    7/50, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 679.0421142578s, input_fired_time: 6.3429627419, output_fired_time: 4.8652606010, propagate_forward_time: 243.8377838135, propagate_backward_time: 414.6061096191, weight_update_time: 0.9507957101, other_time: 9.3900451660
epoch time: 275.2736511230s, input_fired_time: 0.7008800507, output_fired_time: 0.0000000000, propagate_forward_time: 48.6397285461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 225.9330596924
validation[          , genome  6419] predictions:    5179/  10000 (51.79%), best:    5174/10000 (51.74%), error:     15065.54004, best error:     14973.16797 on epoch:     7, epoch:    8/50, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 675.5139160156s, input_fired_time: 6.3397393227, output_fired_time: 4.7874331474, propagate_forward_time: 242.5490112305, propagate_backward_time: 412.2814331055, weight_update_time: 0.9460749030, other_time: 9.5563354492
epoch time: 50.9256896973s, input_fired_time: 0.7022983432, output_fired_time: 0.0000000000, propagate_forward_time: 48.6062469482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6171455383
validation[          , genome  6419] predictions:    5211/  10000 (52.11%), best:    5174/10000 (51.74%), error:     15212.54590, best error:     14973.16797 on epoch:     7, epoch:    9/50, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 856.1047973633s, input_fired_time: 6.3709077835, output_fired_time: 4.8519597054, propagate_forward_time: 244.3542327881, propagate_backward_time: 591.0435180664, weight_update_time: 0.9499005079, other_time: 9.4841308594
epoch time: 51.0034523010s, input_fired_time: 0.7176506519, output_fired_time: 0.0000000000, propagate_forward_time: 48.6781578064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6076431274
validation[          , genome  6419] predictions:    5284/  10000 (52.84%), best:    5284/10000 (52.84%), error:     14572.94824, best error:     14572.94824 on epoch:    10, epoch:   10/50, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 675.3919677734s, input_fired_time: 6.3136906624, output_fired_time: 4.8008894920, propagate_forward_time: 242.5441741943, propagate_backward_time: 412.4194641113, weight_update_time: 0.9408798218, other_time: 9.3136901855
epoch time: 50.9083709717s, input_fired_time: 0.7144763470, output_fired_time: 0.0000000000, propagate_forward_time: 48.5937690735, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6001243591
validation[          , genome  6419] predictions:    4940/  10000 (49.40%), best:    5284/10000 (52.84%), error:     16242.64746, best error:     14572.94824 on epoch:    10, epoch:   11/50, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 677.6831665039s, input_fired_time: 6.3917255402, output_fired_time: 4.8357534409, propagate_forward_time: 243.2434539795, propagate_backward_time: 413.8363342285, weight_update_time: 0.9436767697, other_time: 9.3758850098
epoch time: 51.0698242188s, input_fired_time: 0.7219570875, output_fired_time: 0.0000000000, propagate_forward_time: 48.7102546692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6376113892
validation[          , genome  6419] predictions:    5276/  10000 (52.76%), best:    5276/10000 (52.76%), error:     14253.14844, best error:     14253.14844 on epoch:    12, epoch:   12/50, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 678.1381835938s, input_fired_time: 6.3654999733, output_fired_time: 4.8002939224, propagate_forward_time: 243.3406524658, propagate_backward_time: 414.1515808105, weight_update_time: 0.9549129605, other_time: 9.4801940918
epoch time: 50.8653030396s, input_fired_time: 0.7122310996, output_fired_time: 0.0000000000, propagate_forward_time: 48.4611930847, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6918792725
validation[          , genome  6419] predictions:    5024/  10000 (50.24%), best:    5276/10000 (52.76%), error:     15872.53320, best error:     14253.14844 on epoch:    12, epoch:   13/50, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 684.2985839844s, input_fired_time: 6.4736905098, output_fired_time: 4.8820052147, propagate_forward_time: 245.7002105713, propagate_backward_time: 417.6860961914, weight_update_time: 0.9617044926, other_time: 9.5565795898
epoch time: 51.3676223755s, input_fired_time: 0.7348766327, output_fired_time: 0.0000000000, propagate_forward_time: 48.9321441650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7005996704
validation[          , genome  6419] predictions:    5083/  10000 (50.83%), best:    5276/10000 (52.76%), error:     15665.88965, best error:     14253.14844 on epoch:    12, epoch:   14/50, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 678.7792968750s, input_fired_time: 6.3966412544, output_fired_time: 4.8330364227, propagate_forward_time: 243.6215209961, propagate_backward_time: 414.4610595703, weight_update_time: 0.9633962512, other_time: 9.4670410156
epoch time: 50.7523536682s, input_fired_time: 0.7173365355, output_fired_time: 0.0000000000, propagate_forward_time: 48.4088020325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6262168884
validation[          , genome  6419] predictions:    5485/  10000 (54.85%), best:    5485/10000 (54.85%), error:     13698.70313, best error:     13698.70313 on epoch:    15, epoch:   15/50, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 897.4907836914s, input_fired_time: 6.4274091721, output_fired_time: 4.8120169640, propagate_forward_time: 244.1499176025, propagate_backward_time: 632.5355224609, weight_update_time: 0.9560261965, other_time: 9.5659179688
epoch time: 51.1531753540s, input_fired_time: 0.7525821924, output_fired_time: 0.0000000000, propagate_forward_time: 48.7408561707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6597366333
validation[          , genome  6419] predictions:    5455/  10000 (54.55%), best:    5485/10000 (54.85%), error:     13904.49121, best error:     13698.70313 on epoch:    15, epoch:   16/50, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 1118.3874511719s, input_fired_time: 6.4089913368, output_fired_time: 4.7970280647, propagate_forward_time: 683.9541015625, propagate_backward_time: 413.7477722168, weight_update_time: 0.9460453391, other_time: 9.4796447754
epoch time: 50.9897651672s, input_fired_time: 0.7213282585, output_fired_time: 0.0000000000, propagate_forward_time: 48.5746231079, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6938133240
validation[          , genome  6419] predictions:    5374/  10000 (53.74%), best:    5485/10000 (54.85%), error:     14341.67969, best error:     13698.70313 on epoch:    15, epoch:   17/50, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 679.2060546875s, input_fired_time: 6.4554104805, output_fired_time: 4.8345975876, propagate_forward_time: 243.9353485107, propagate_backward_time: 414.5114440918, weight_update_time: 0.9457533956, other_time: 9.4692687988
epoch time: 51.4754104614s, input_fired_time: 0.7364064455, output_fired_time: 0.0000000000, propagate_forward_time: 49.0934753418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6455268860
validation[          , genome  6419] predictions:    5490/  10000 (54.90%), best:    5490/10000 (54.90%), error:     13549.34277, best error:     13549.34277 on epoch:    18, epoch:   18/50, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 679.9967041016s, input_fired_time: 6.4634776115, output_fired_time: 4.8392839432, propagate_forward_time: 244.1432800293, propagate_backward_time: 415.1477966309, weight_update_time: 0.9432920814, other_time: 9.4028320313
epoch time: 51.2680969238s, input_fired_time: 0.7446637154, output_fired_time: 0.0000000000, propagate_forward_time: 48.8231239319, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7003097534
validation[          , genome  6419] predictions:    5407/  10000 (54.07%), best:    5490/10000 (54.90%), error:     14001.24316, best error:     13549.34277 on epoch:    18, epoch:   19/50, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 678.5560913086s, input_fired_time: 6.4100980759, output_fired_time: 4.8338174820, propagate_forward_time: 243.5792846680, propagate_backward_time: 414.2269897461, weight_update_time: 0.9434043765, other_time: 9.5059204102
epoch time: 51.2117919922s, input_fired_time: 0.7176140547, output_fired_time: 0.0000000000, propagate_forward_time: 48.8221740723, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6720046997
validation[          , genome  6419] predictions:    5425/  10000 (54.25%), best:    5490/10000 (54.90%), error:     13884.78516, best error:     13549.34277 on epoch:    18, epoch:   20/50, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 680.4573364258s, input_fired_time: 6.4303250313, output_fired_time: 4.8760504723, propagate_forward_time: 244.5534210205, propagate_backward_time: 415.1017150879, weight_update_time: 0.9481725693, other_time: 9.4958801270
epoch time: 50.8639564514s, input_fired_time: 0.7359665632, output_fired_time: 0.0000000000, propagate_forward_time: 48.5241317749, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6038589478
validation[          , genome  6419] predictions:    5464/  10000 (54.64%), best:    5490/10000 (54.90%), error:     13611.33301, best error:     13549.34277 on epoch:    18, epoch:   21/50, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 1049.7744140625s, input_fired_time: 6.3467378616, output_fired_time: 4.8151206970, propagate_forward_time: 617.3022460938, propagate_backward_time: 411.9385986328, weight_update_time: 0.9406237006, other_time: 9.3718261719
epoch time: 51.1435279846s, input_fired_time: 0.7480932474, output_fired_time: 0.0000000000, propagate_forward_time: 48.6755981445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7198371887
validation[          , genome  6419] predictions:    5511/  10000 (55.11%), best:    5511/10000 (55.11%), error:     13344.94336, best error:     13344.94336 on epoch:    22, epoch:   22/50, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 1282.1925048828s, input_fired_time: 6.5239562988, output_fired_time: 4.8515801430, propagate_forward_time: 846.5952148438, propagate_backward_time: 414.7477722168, weight_update_time: 0.9413876534, other_time: 9.4740295410
epoch time: 50.8271865845s, input_fired_time: 0.7129228711, output_fired_time: 0.0000000000, propagate_forward_time: 48.4635353088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6507301331
validation[          , genome  6419] predictions:    5365/  10000 (53.65%), best:    5511/10000 (55.11%), error:     14446.82715, best error:     13344.94336 on epoch:    22, epoch:   23/50, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 676.8804321289s, input_fired_time: 6.4444594383, output_fired_time: 4.8255238533, propagate_forward_time: 242.9721984863, propagate_backward_time: 413.3121643066, weight_update_time: 0.9403944016, other_time: 9.3261108398
epoch time: 50.6661949158s, input_fired_time: 0.6982672811, output_fired_time: 0.0000000000, propagate_forward_time: 48.3485031128, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6194229126
validation[          , genome  6419] predictions:    5352/  10000 (53.52%), best:    5511/10000 (55.11%), error:     14549.52637, best error:     13344.94336 on epoch:    22, epoch:   24/50, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 677.8250122070s, input_fired_time: 6.4788742065, output_fired_time: 4.8004679680, propagate_forward_time: 243.4382171631, propagate_backward_time: 413.7766418457, weight_update_time: 0.9407951236, other_time: 9.3307800293
epoch time: 51.1741981506s, input_fired_time: 0.7315208316, output_fired_time: 0.0000000000, propagate_forward_time: 48.7575492859, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6851272583
validation[          , genome  6419] predictions:    5615/  10000 (56.15%), best:    5615/10000 (56.15%), error:     13194.91016, best error:     13194.91016 on epoch:    25, epoch:   25/50, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 677.5591430664s, input_fired_time: 6.4603080750, output_fired_time: 4.8375921249, propagate_forward_time: 243.3067016602, propagate_backward_time: 413.4116821289, weight_update_time: 0.9499427080, other_time: 9.5428466797
epoch time: 50.8211174011s, input_fired_time: 0.7200474739, output_fired_time: 0.0000000000, propagate_forward_time: 48.4324150085, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6686553955
validation[          , genome  6419] predictions:    5624/  10000 (56.24%), best:    5624/10000 (56.24%), error:     13155.12402, best error:     13155.12402 on epoch:    26, epoch:   26/50, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 677.1254882813s, input_fired_time: 6.4510235786, output_fired_time: 4.8409705162, propagate_forward_time: 243.0835266113, propagate_backward_time: 413.3001708984, weight_update_time: 0.9426323175, other_time: 9.4497985840
epoch time: 51.1473426819s, input_fired_time: 0.7160658836, output_fired_time: 0.0000000000, propagate_forward_time: 48.7483825684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6828956604
validation[          , genome  6419] predictions:    5369/  10000 (53.69%), best:    5624/10000 (56.24%), error:     13976.72070, best error:     13155.12402 on epoch:    26, epoch:   27/50, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 677.0325927734s, input_fired_time: 6.4563674927, output_fired_time: 4.7915501595, propagate_forward_time: 243.1902770996, propagate_backward_time: 413.2260131836, weight_update_time: 0.9399451613, other_time: 9.3683776855
epoch time: 50.4960289001s, input_fired_time: 0.7305163741, output_fired_time: 0.0000000000, propagate_forward_time: 48.1652030945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6003112793
validation[          , genome  6419] predictions:    5647/  10000 (56.47%), best:    5647/10000 (56.47%), error:     13073.68555, best error:     13073.68555 on epoch:    28, epoch:   28/50, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 891.9529418945s, input_fired_time: 6.4508013725, output_fired_time: 4.7831430435, propagate_forward_time: 460.3735351563, propagate_backward_time: 411.1437988281, weight_update_time: 0.9355791211, other_time: 9.2016601563
epoch time: 50.5044288635s, input_fired_time: 0.7253746986, output_fired_time: 0.0000000000, propagate_forward_time: 48.1665115356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6125411987
validation[          , genome  6419] predictions:    5464/  10000 (54.64%), best:    5647/10000 (56.47%), error:     13791.06348, best error:     13073.68555 on epoch:    28, epoch:   29/50, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1422.6970214844s, input_fired_time: 6.4090881348, output_fired_time: 4.8340721130, propagate_forward_time: 244.8018493652, propagate_backward_time: 1157.1197509766, weight_update_time: 0.9389700294, other_time: 9.5322265625
epoch time: 50.9062271118s, input_fired_time: 0.7445831299, output_fired_time: 0.0000000000, propagate_forward_time: 48.5382499695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6233940125
validation[          , genome  6419] predictions:    5766/  10000 (57.66%), best:    5766/10000 (57.66%), error:     12545.42773, best error:     12545.42773 on epoch:    30, epoch:   30/50, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 672.6395263672s, input_fired_time: 6.3739180565, output_fired_time: 4.7841467857, propagate_forward_time: 241.6013183594, propagate_backward_time: 410.6619262695, weight_update_time: 0.9373601675, other_time: 9.2182617188
epoch time: 50.5948867798s, input_fired_time: 0.7262669206, output_fired_time: 0.0000000000, propagate_forward_time: 48.2743034363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5943145752
validation[          , genome  6419] predictions:    5744/  10000 (57.44%), best:    5766/10000 (57.66%), error:     12847.51660, best error:     12545.42773 on epoch:    30, epoch:   31/50, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 674.0236816406s, input_fired_time: 6.4025940895, output_fired_time: 4.8066716194, propagate_forward_time: 242.0964355469, propagate_backward_time: 411.4081726074, weight_update_time: 0.9381902218, other_time: 9.3097839355
epoch time: 50.9792098999s, input_fired_time: 0.7485500574, output_fired_time: 0.0000000000, propagate_forward_time: 48.5695304871, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6611289978
validation[          , genome  6419] predictions:    5703/  10000 (57.03%), best:    5766/10000 (57.66%), error:     12919.78516, best error:     12545.42773 on epoch:    30, epoch:   32/50, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 675.2399902344s, input_fired_time: 6.4764118195, output_fired_time: 4.8182020187, propagate_forward_time: 242.1990814209, propagate_backward_time: 412.3844299316, weight_update_time: 0.9445655942, other_time: 9.3618469238
epoch time: 50.6983375549s, input_fired_time: 0.7451380491, output_fired_time: 0.0000000000, propagate_forward_time: 48.3418884277, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6113128662
validation[          , genome  6419] predictions:    5700/  10000 (57.00%), best:    5766/10000 (57.66%), error:     12936.77051, best error:     12545.42773 on epoch:    30, epoch:   33/50, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 735.0598754883s, input_fired_time: 7.0731811523, output_fired_time: 5.2691860199, propagate_forward_time: 263.9726867676, propagate_backward_time: 448.4043884277, weight_update_time: 1.0017231703, other_time: 10.3404541016
epoch time: 68.1056823730s, input_fired_time: 1.0250990391, output_fired_time: 0.0000000000, propagate_forward_time: 64.8199310303, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2606506348
validation[          , genome  6419] predictions:    5807/  10000 (58.07%), best:    5807/10000 (58.07%), error:     12439.44141, best error:     12439.44141 on epoch:    34, epoch:   34/50, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 819.3395385742s, input_fired_time: 8.2087306976, output_fired_time: 5.8134212494, propagate_forward_time: 294.4789428711, propagate_backward_time: 499.0827331543, weight_update_time: 1.1238071918, other_time: 11.7557067871
epoch time: 71.8732223511s, input_fired_time: 1.0689128637, output_fired_time: 0.0000000000, propagate_forward_time: 68.5143051147, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2900009155
validation[          , genome  6419] predictions:    5568/  10000 (55.68%), best:    5807/10000 (58.07%), error:     13340.13770, best error:     12439.44141 on epoch:    34, epoch:   35/50, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 860.0260009766s, input_fired_time: 8.6661939621, output_fired_time: 6.0394773483, propagate_forward_time: 308.1596069336, propagate_backward_time: 525.0107421875, weight_update_time: 1.1660926342, other_time: 12.1499633789
epoch time: 68.9023056030s, input_fired_time: 1.1097347736, output_fired_time: 0.0000000000, propagate_forward_time: 65.6148681641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1777038574
validation[          , genome  6419] predictions:    5859/  10000 (58.59%), best:    5859/10000 (58.59%), error:     12110.68652, best error:     12110.68652 on epoch:    36, epoch:   36/50, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 886.9443969727s, input_fired_time: 8.9984560013, output_fired_time: 6.3783440590, propagate_forward_time: 317.9358825684, propagate_backward_time: 540.9328002930, weight_update_time: 1.2123196125, other_time: 12.6989135742
epoch time: 66.3195495605s, input_fired_time: 0.9631917477, output_fired_time: 0.0000000000, propagate_forward_time: 63.2317810059, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1245803833
validation[          , genome  6419] predictions:    5800/  10000 (58.00%), best:    5859/10000 (58.59%), error:     12486.84961, best error:     12110.68652 on epoch:    36, epoch:   37/50, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 692.3940429688s, input_fired_time: 6.7966508865, output_fired_time: 4.9643836021, propagate_forward_time: 248.2769470215, propagate_backward_time: 422.3074645996, weight_update_time: 0.9723542929, other_time: 10.0486450195
epoch time: 50.5855979919s, input_fired_time: 0.7539863586, output_fired_time: 0.0000000000, propagate_forward_time: 48.2106704712, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6209411621
validation[          , genome  6419] predictions:    5870/  10000 (58.70%), best:    5859/10000 (58.59%), error:     12479.54102, best error:     12110.68652 on epoch:    36, epoch:   38/50, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 678.2753295898s, input_fired_time: 6.5904803276, output_fired_time: 4.8336176872, propagate_forward_time: 243.5673675537, propagate_backward_time: 413.7017822266, weight_update_time: 0.9486664534, other_time: 9.5820922852
epoch time: 50.6993179321s, input_fired_time: 0.7649324536, output_fired_time: 0.0000000000, propagate_forward_time: 48.2415847778, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6928024292
validation[          , genome  6419] predictions:    5612/  10000 (56.12%), best:    5859/10000 (58.59%), error:     13416.82422, best error:     12110.68652 on epoch:    36, epoch:   39/50, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 934.8608398438s, input_fired_time: 6.6394667625, output_fired_time: 4.8320698738, propagate_forward_time: 243.7329254150, propagate_backward_time: 669.9581909180, weight_update_time: 0.9531657100, other_time: 9.6981811523
epoch time: 50.8229751587s, input_fired_time: 0.7706739902, output_fired_time: 0.0000000000, propagate_forward_time: 48.3241767883, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7281227112
validation[          , genome  6419] predictions:    5745/  10000 (57.45%), best:    5859/10000 (58.59%), error:     12876.27344, best error:     12110.68652 on epoch:    36, epoch:   40/50, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 842.5936889648s, input_fired_time: 6.6291399002, output_fired_time: 4.8760704994, propagate_forward_time: 408.2288818359, propagate_backward_time: 413.1585388184, weight_update_time: 0.9439964890, other_time: 9.7010192871
epoch time: 51.2474975586s, input_fired_time: 0.7494548559, output_fired_time: 0.0000000000, propagate_forward_time: 48.8278350830, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6702079773
validation[          , genome  6419] predictions:    5783/  10000 (57.83%), best:    5859/10000 (58.59%), error:     12691.09082, best error:     12110.68652 on epoch:    36, epoch:   41/50, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 680.0280761719s, input_fired_time: 6.6368737221, output_fired_time: 4.8381934166, propagate_forward_time: 244.2847290039, propagate_backward_time: 414.7196350098, weight_update_time: 0.9386134148, other_time: 9.5486145020
epoch time: 50.7405929565s, input_fired_time: 0.7491865754, output_fired_time: 0.0000000000, propagate_forward_time: 48.3712234497, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6201820374
validation[          , genome  6419] predictions:    5922/  10000 (59.22%), best:    5859/10000 (58.59%), error:     12183.08203, best error:     12110.68652 on epoch:    36, epoch:   42/50, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 679.8448486328s, input_fired_time: 6.6395535469, output_fired_time: 4.8501548767, propagate_forward_time: 244.2109527588, propagate_backward_time: 414.3917846680, weight_update_time: 0.9419254661, other_time: 9.7524414063
epoch time: 50.8713607788s, input_fired_time: 0.7599927783, output_fired_time: 0.0000000000, propagate_forward_time: 48.4950904846, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6162757874
validation[          , genome  6419] predictions:    5802/  10000 (58.02%), best:    5859/10000 (58.59%), error:     12578.67090, best error:     12110.68652 on epoch:    36, epoch:   43/50, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 676.1235961914s, input_fired_time: 6.5360512733, output_fired_time: 4.8330655098, propagate_forward_time: 243.0315399170, propagate_backward_time: 412.2568969727, weight_update_time: 0.9398324490, other_time: 9.4660034180
epoch time: 50.7362594604s, input_fired_time: 0.7389386892, output_fired_time: 0.0000000000, propagate_forward_time: 48.3682556152, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6290664673
validation[          , genome  6419] predictions:    5722/  10000 (57.22%), best:    5859/10000 (58.59%), error:     12826.76953, best error:     12110.68652 on epoch:    36, epoch:   44/50, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1838.1269531250s, input_fired_time: 6.5999903679, output_fired_time: 4.8141317368, propagate_forward_time: 242.7785034180, propagate_backward_time: 1574.3139648438, weight_update_time: 0.9448418617, other_time: 9.6203613281
epoch time: 5047.5751953125s, input_fired_time: 0.7739182711, output_fired_time: 0.0000000000, propagate_forward_time: 5045.0454101563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7558593750
validation[          , genome  6419] predictions:    5821/  10000 (58.21%), best:    5859/10000 (58.59%), error:     12763.79785, best error:     12110.68652 on epoch:    36, epoch:   45/50, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1311.9494628906s, input_fired_time: 6.6239733696, output_fired_time: 4.8209600449, propagate_forward_time: 877.4965820313, propagate_backward_time: 413.4505310059, weight_update_time: 0.9402062893, other_time: 9.5574035645
epoch time: 50.7019882202s, input_fired_time: 0.7600322366, output_fired_time: 0.0000000000, propagate_forward_time: 48.2957878113, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6461677551
validation[          , genome  6419] predictions:    5789/  10000 (57.89%), best:    5859/10000 (58.59%), error:     12651.06738, best error:     12110.68652 on epoch:    36, epoch:   46/50, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1312.4993896484s, input_fired_time: 6.5750532150, output_fired_time: 4.8376364708, propagate_forward_time: 243.7645874023, propagate_backward_time: 1047.7459716797, weight_update_time: 0.9420578480, other_time: 9.5760498047
epoch time: 51.0433616638s, input_fired_time: 0.7776595950, output_fired_time: 0.0000000000, propagate_forward_time: 48.5199470520, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7457542419
validation[          , genome  6419] predictions:    5814/  10000 (58.14%), best:    5859/10000 (58.59%), error:     12519.82129, best error:     12110.68652 on epoch:    36, epoch:   47/50, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 1173.2880859375s, input_fired_time: 6.7557215691, output_fired_time: 4.9607720375, propagate_forward_time: 247.4219055176, propagate_backward_time: 904.4343872070, weight_update_time: 0.9954632521, other_time: 9.7152709961
epoch time: 50.8929824829s, input_fired_time: 0.7650446892, output_fired_time: 0.0000000000, propagate_forward_time: 48.5153274536, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6126098633
validation[          , genome  6419] predictions:    5899/  10000 (58.99%), best:    5859/10000 (58.59%), error:     12207.90430, best error:     12110.68652 on epoch:    36, epoch:   48/50, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 677.2086181641s, input_fired_time: 6.5586400032, output_fired_time: 4.8109850883, propagate_forward_time: 243.3989410400, propagate_backward_time: 412.9892578125, weight_update_time: 0.9464234114, other_time: 9.4508056641
epoch time: 50.7942924500s, input_fired_time: 0.7574074864, output_fired_time: 0.0000000000, propagate_forward_time: 48.4016494751, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6352348328
validation[          , genome  6419] predictions:    5924/  10000 (59.24%), best:    5924/10000 (59.24%), error:     12074.25977, best error:     12074.25977 on epoch:    49, epoch:   49/50, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 677.4483032227s, input_fired_time: 6.5825881958, output_fired_time: 4.8493094444, propagate_forward_time: 243.2090606689, propagate_backward_time: 413.3007812500, weight_update_time: 0.9402598143, other_time: 9.5065917969
epoch time: 50.7230949402s, input_fired_time: 0.7592749000, output_fired_time: 0.0000000000, propagate_forward_time: 48.3474655151, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6163558960
validation[          , genome  6419] predictions:    5860/  10000 (58.60%), best:    5924/10000 (59.24%), error:     12219.63086, best error:     12074.25977 on epoch:    49, epoch:   50/50, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 254.4312286377s, input_fired_time: 3.8204724789, output_fired_time: 0.0000000000, propagate_forward_time: 242.2964935303, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.3142700195
best training[          , genome  6419] predictions:   34815/  50000 (69.63%), best:    5924/10000 (59.24%), error:     43413.23047, best error:     12074.25977 on epoch:    49, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 50.6606636047s, input_fired_time: 0.7572870255, output_fired_time: 0.0000000000, propagate_forward_time: 48.2875099182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6158676147
   testing[          , genome  6419] predictions:    5924/  10000 (59.24%), best:    5924/10000 (59.24%), error:     12074.25977, best error:     12074.25977 on epoch:    49, epoch:   51/50, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488
backpropagation finished successfully!
05:01:52 (8528): called boinc_finish(0)

</stderr_txt>
]]>