Nameexact_genome_1542031203_172_5543_1
Workunit4080311
Created12 Nov 2018, 16:33:55 UTC
Sent12 Nov 2018, 16:42:55 UTC
Report deadline7 Dec 2018, 1:11:57 UTC
Received14 Nov 2018, 2:32:47 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID71624
Run time6 hours 49 min 49 sec
CPU time6 hours 38 min 10 sec
Validate stateValid
Credit6,088.38
Device peak FLOPS3.39 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34

Stderr output

<core_client_version>7.0.80</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_172_5543.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_172_5543_1_r121325358_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542031203_172_5543.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.503767
read mu: 0.503767
read mu_delta: 0.933111
read initial_learning_rate: 0.000528231
read learning_rate: 0.000528231
read learning_rate_delta: 0.931604
read initial_weight_decay: 0.000261295
read weight_decay: 0.000261295
read weight_decay_delta: 0.939594
read batch_size: 104
read epsilon: 1e-07
read alpha: 0.0930002
read input_dropout_probability: 0.0013726
read hidden_dropout_probability: 0.0815569
read velocity_reset: 1883
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5543
read normal distribution: '0 -0.312769114971161 -0.495374619960785'
generator_str: '1418129746'
read generator: 1418129746
read generated_by_map:
1 add_edge 1
reading 90 nodes.
reading 385 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1733126616
post shuffle 1: 2045422766
epoch time: 2109.56591796875s, input_fired_time: 47.104419708252, output_fired_time: 13.3365573883057, propagate_forward_time: 608.125671386719, propagate_backward_time: 1410.9375, weight_update_time: 0.204716518521309, other_time: 30.0615234375
epoch time: 128.227691650391s, input_fired_time: 3.02116990089417, output_fired_time: 0, propagate_forward_time: 117.620544433594, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.58597564697266
validation[          , genome  5543] predictions:    6253/  10000 (62.53%), best:    6253/10000 (62.53%), error:     10823.85156, best error:     10823.85156 on epoch:     0, epoch:    0/25, mu: 0.5037667751, learning_rate: 0.0005282314, weight_decay: 0.0002612954

epoch time: 2106.7807617188s, input_fired_time: 42.9704055786, output_fired_time: 17.3726902008, propagate_forward_time: 601.2865600586, propagate_backward_time: 1425.1179199219, weight_update_time: 2.2112278938, other_time: 20.0332031250
epoch time: 147.6434173584s, input_fired_time: 1.1014212370, output_fired_time: 0.0000000000, propagate_forward_time: 143.6147460938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9272460938
validation[          , genome  5543] predictions:    6101/  10000 (61.01%), best:    6253/10000 (62.53%), error:     11138.13574, best error:     10823.85156 on epoch:     0, epoch:    1/25, mu: 0.5362901688, learning_rate: 0.0004921025, weight_decay: 0.0002455116

epoch time: 2199.1835937500s, input_fired_time: 36.5251045227, output_fired_time: 15.5003585815, propagate_forward_time: 631.1869506836, propagate_backward_time: 1495.5736083984, weight_update_time: 1.2144645452, other_time: 20.3975830078
epoch time: 131.9488830566s, input_fired_time: 2.0625228882, output_fired_time: 0.0000000000, propagate_forward_time: 126.2290344238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6573181152
validation[          , genome  5543] predictions:    6194/  10000 (61.94%), best:    6253/10000 (62.53%), error:     10955.89160, best error:     10823.85156 on epoch:     0, epoch:    2/25, mu: 0.5666381717, learning_rate: 0.0004584446, weight_decay: 0.0002306813

epoch time: 2152.2929687500s, input_fired_time: 42.9774398804, output_fired_time: 11.4417209625, propagate_forward_time: 614.1976318359, propagate_backward_time: 1454.1020507813, weight_update_time: 2.2225379944, other_time: 29.5740966797
epoch time: 133.0288848877s, input_fired_time: 2.0389406681, output_fired_time: 0.0000000000, propagate_forward_time: 125.3347244263, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.6552200317
validation[          , genome  5543] predictions:    6003/  10000 (60.03%), best:    6253/10000 (62.53%), error:     11542.48145, best error:     10823.85156 on epoch:     0, epoch:    3/25, mu: 0.5949562192, learning_rate: 0.0004270889, weight_decay: 0.0002167468

epoch time: 2154.4775390625s, input_fired_time: 43.3353919983, output_fired_time: 11.4023675919, propagate_forward_time: 611.0335693359, propagate_backward_time: 1462.1790771484, weight_update_time: 0.2065958381, other_time: 26.5270996094
epoch time: 130.3720703125s, input_fired_time: 1.0173232555, output_fired_time: 0.0000000000, propagate_forward_time: 124.8067245483, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5480270386
validation[          , genome  5543] predictions:    6073/  10000 (60.73%), best:    6253/10000 (62.53%), error:     11255.31445, best error:     10823.85156 on epoch:     0, epoch:    4/25, mu: 0.6213800907, learning_rate: 0.0003978777, weight_decay: 0.0002036540

epoch time: 2171.6938476563s, input_fired_time: 36.2621955872, output_fired_time: 11.4868278503, propagate_forward_time: 630.7036743164, propagate_backward_time: 1472.6252441406, weight_update_time: 0.2105534375, other_time: 20.6159667969
epoch time: 130.3928680420s, input_fired_time: 1.0144858360, output_fired_time: 0.0000000000, propagate_forward_time: 126.7536849976, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6247024536
validation[          , genome  5543] predictions:    6048/  10000 (60.48%), best:    6253/10000 (62.53%), error:     11424.10156, best error:     10823.85156 on epoch:     0, epoch:    5/25, mu: 0.6460365057, learning_rate: 0.0003706645, weight_decay: 0.0001913521

epoch time: 2154.0036621094s, input_fired_time: 34.1206207275, output_fired_time: 15.4952650070, propagate_forward_time: 608.3358764648, propagate_backward_time: 1463.5657958984, weight_update_time: 0.2065930814, other_time: 32.4859619141
epoch time: 130.5780334473s, input_fired_time: 3.0450789928, output_fired_time: 0.0000000000, propagate_forward_time: 124.9124832153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6204681396
validation[          , genome  5543] predictions:    6251/  10000 (62.51%), best:    6253/10000 (62.53%), error:     11018.94727, best error:     10823.85156 on epoch:     0, epoch:    6/25, mu: 0.6690437198, learning_rate: 0.0003453125, weight_decay: 0.0001797933

epoch time: 2168.3703613281s, input_fired_time: 49.3451309204, output_fired_time: 12.4664049149, propagate_forward_time: 603.4791870117, propagate_backward_time: 1478.6119384766, weight_update_time: 0.2123829275, other_time: 24.4676513672
epoch time: 133.0623016357s, input_fired_time: 1.0304270983, output_fired_time: 0.0000000000, propagate_forward_time: 127.3637237549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6681518555
validation[          , genome  5543] predictions:    6123/  10000 (61.23%), best:    6253/10000 (62.53%), error:     11145.92773, best error:     10823.85156 on epoch:     0, epoch:    7/25, mu: 0.6905120015, learning_rate: 0.0003216945, weight_decay: 0.0001689328

epoch time: 2164.1357421875s, input_fired_time: 45.3354225159, output_fired_time: 13.5183477402, propagate_forward_time: 614.3558349609, propagate_backward_time: 1468.5167236328, weight_update_time: 0.2154789418, other_time: 22.4094238281
epoch time: 131.6650695801s, input_fired_time: 1.0273697376, output_fired_time: 0.0000000000, propagate_forward_time: 127.0056686401, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6320266724
validation[          , genome  5543] predictions:    6184/  10000 (61.84%), best:    6253/10000 (62.53%), error:     11062.08594, best error:     10823.85156 on epoch:     0, epoch:    8/25, mu: 0.7105442882, learning_rate: 0.0002996919, weight_decay: 0.0001587282

epoch time: 2159.7775878906s, input_fired_time: 34.2631797791, output_fired_time: 11.4884414673, propagate_forward_time: 626.7905883789, propagate_backward_time: 1458.5982666016, weight_update_time: 0.2166364640, other_time: 28.6370849609
epoch time: 130.3827514648s, input_fired_time: 1.0315308571, output_fired_time: 0.0000000000, propagate_forward_time: 123.7863998413, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5648269653
validation[          , genome  5543] predictions:    6222/  10000 (62.22%), best:    6253/10000 (62.53%), error:     10902.14941, best error:     10823.85156 on epoch:     0, epoch:    9/25, mu: 0.7292366624, learning_rate: 0.0002791942, weight_decay: 0.0001491401

epoch time: 2162.2434082031s, input_fired_time: 51.4181671143, output_fired_time: 14.4849720001, propagate_forward_time: 616.0789794922, propagate_backward_time: 1461.7873535156, weight_update_time: 0.2095583826, other_time: 18.4739990234
epoch time: 131.3480834961s, input_fired_time: 2.0300912857, output_fired_time: 0.0000000000, propagate_forward_time: 124.6857070923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6322860718
validation[          , genome  5543] predictions:    6047/  10000 (60.47%), best:    6253/10000 (62.53%), error:     11357.90820, best error:     10823.85156 on epoch:     0, epoch:   10/25, mu: 0.7466787100, learning_rate: 0.0002600984, weight_decay: 0.0001401312

epoch time: 2157.2785644531s, input_fired_time: 45.3412094116, output_fired_time: 11.3476467133, propagate_forward_time: 615.3177490234, propagate_backward_time: 1450.8012695313, weight_update_time: 2.2250919342, other_time: 34.4705810547
epoch time: 130.3612365723s, input_fired_time: 4.0413007736, output_fired_time: 0.0000000000, propagate_forward_time: 121.7820281982, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5379104614
validation[          , genome  5543] predictions:    6288/  10000 (62.88%), best:    6288/10000 (62.88%), error:     10708.49805, best error:     10708.49805 on epoch:    11, epoch:   11/25, mu: 0.7629541159, learning_rate: 0.0002423087, weight_decay: 0.0001316664

epoch time: 2150.4340820313s, input_fired_time: 50.0356140137, output_fired_time: 11.4756298065, propagate_forward_time: 585.7043457031, propagate_backward_time: 1479.6960449219, weight_update_time: 0.2110706121, other_time: 23.5224609375
epoch time: 130.3113098145s, input_fired_time: 3.0307884216, output_fired_time: 0.0000000000, propagate_forward_time: 123.6209564209, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6595611572
validation[          , genome  5543] predictions:    6282/  10000 (62.82%), best:    6288/10000 (62.88%), error:     10774.88379, best error:     10708.49805 on epoch:    11, epoch:   12/25, mu: 0.7781409025, learning_rate: 0.0002257358, weight_decay: 0.0001237130

epoch time: 2369.9895019531s, input_fired_time: 46.2706222534, output_fired_time: 16.8957004547, propagate_forward_time: 640.6272583008, propagate_backward_time: 1644.7017822266, weight_update_time: 0.2325876653, other_time: 21.4942626953
epoch time: 173.1844940186s, input_fired_time: 6.2769398689, output_fired_time: 0.0000000000, propagate_forward_time: 161.8198852539, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0876617432
validation[          , genome  5543] predictions:    6273/  10000 (62.73%), best:    6288/10000 (62.88%), error:     10745.82715, best error:     10708.49805 on epoch:    11, epoch:   13/25, mu: 0.7923118472, learning_rate: 0.0002102964, weight_decay: 0.0001162400

epoch time: 2600.5900878906s, input_fired_time: 52.3937606812, output_fired_time: 10.5615444183, propagate_forward_time: 749.1342163086, propagate_backward_time: 1767.3542480469, weight_update_time: 0.2351568937, other_time: 21.1462402344
epoch time: 139.9265136719s, input_fired_time: 2.1489763260, output_fired_time: 0.0000000000, propagate_forward_time: 130.8710479736, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.9064941406
validation[          , genome  5543] predictions:    6335/  10000 (63.35%), best:    6335/10000 (63.35%), error:     10641.49902, best error:     10641.49902 on epoch:    14, epoch:   14/25, mu: 0.8055348992, learning_rate: 0.0001959130, weight_decay: 0.0001092185

epoch time: 2416.5058593750s, input_fired_time: 46.9331359863, output_fired_time: 19.0481395721, propagate_forward_time: 736.4423828125, propagate_backward_time: 1583.9910888672, weight_update_time: 0.2356368601, other_time: 30.0911865234
epoch time: 158.6206207275s, input_fired_time: 3.2737586498, output_fired_time: 0.0000000000, propagate_forward_time: 150.2371978760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1096649170
validation[          , genome  5543] predictions:    6037/  10000 (60.37%), best:    6335/10000 (63.35%), error:     11275.86621, best error:     10641.49902 on epoch:    14, epoch:   15/25, mu: 0.8178734779, learning_rate: 0.0001825133, weight_decay: 0.0001026210

epoch time: 2605.6604003906s, input_fired_time: 55.2841567993, output_fired_time: 19.6395835876, propagate_forward_time: 749.5468750000, propagate_backward_time: 1730.3684082031, weight_update_time: 0.2430113852, other_time: 50.8212890625
epoch time: 131.6218719482s, input_fired_time: 2.0402398109, output_fired_time: 0.0000000000, propagate_forward_time: 125.8783264160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7033081055
validation[          , genome  5543] predictions:    6314/  10000 (63.14%), best:    6314/10000 (63.14%), error:     10603.25391, best error:     10603.25391 on epoch:    16, epoch:   16/25, mu: 0.8293867707, learning_rate: 0.0001700301, weight_decay: 0.0000964221

epoch time: 2137.4135742188s, input_fired_time: 40.1299247742, output_fired_time: 14.3283720016, propagate_forward_time: 612.1953735352, propagate_backward_time: 1447.4832763672, weight_update_time: 1.2121207714, other_time: 23.2767333984
epoch time: 130.0243530273s, input_fired_time: 2.0302565098, output_fired_time: 0.0000000000, propagate_forward_time: 121.3630065918, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.6310882568
validation[          , genome  5543] predictions:    6363/  10000 (63.63%), best:    6363/10000 (63.63%), error:     10491.29395, best error:     10491.29395 on epoch:    17, epoch:   17/25, mu: 0.8401299715, learning_rate: 0.0001584008, weight_decay: 0.0000905976

epoch time: 2112.7150878906s, input_fired_time: 47.9959640503, output_fired_time: 11.4134979248, propagate_forward_time: 593.0351562500, propagate_backward_time: 1433.0520019531, weight_update_time: 0.2108478099, other_time: 27.2185058594
epoch time: 129.8162841797s, input_fired_time: 1.0199307203, output_fired_time: 0.0000000000, propagate_forward_time: 127.2172470093, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5791091919
validation[          , genome  5543] predictions:    6253/  10000 (62.53%), best:    6363/10000 (63.63%), error:     10839.24805, best error:     10491.29395 on epoch:    17, epoch:   18/25, mu: 0.8501545787, learning_rate: 0.0001475668, weight_decay: 0.0000851250

epoch time: 2369.7329101563s, input_fired_time: 40.5691452026, output_fired_time: 9.6403799057, propagate_forward_time: 695.6430664063, propagate_backward_time: 1596.1702880859, weight_update_time: 0.2197368741, other_time: 27.7100830078
epoch time: 135.3106384277s, input_fired_time: 4.0605363846, output_fired_time: 0.0000000000, propagate_forward_time: 129.6094207764, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6406860352
validation[          , genome  5543] predictions:    6242/  10000 (62.42%), best:    6363/10000 (63.63%), error:     10789.40625, best error:     10491.29395 on epoch:    17, epoch:   19/25, mu: 0.8595086336, learning_rate: 0.0001374738, weight_decay: 0.0000799830

epoch time: 2160.6806640625s, input_fired_time: 47.3887481689, output_fired_time: 12.4888458252, propagate_forward_time: 608.6592407227, propagate_backward_time: 1463.6158447266, weight_update_time: 0.2107477188, other_time: 28.5281982422
epoch time: 132.6964721680s, input_fired_time: 4.0523962975, output_fired_time: 0.0000000000, propagate_forward_time: 127.0486297607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5954437256
validation[          , genome  5543] predictions:    6301/  10000 (63.01%), best:    6363/10000 (63.63%), error:     10616.43359, best error:     10491.29395 on epoch:    17, epoch:   20/25, mu: 0.8682370186, learning_rate: 0.0001280711, weight_decay: 0.0000751515

epoch time: 2147.5100097656s, input_fired_time: 39.0863647461, output_fired_time: 9.4582061768, propagate_forward_time: 613.7462768555, propagate_backward_time: 1464.7303466797, weight_update_time: 2.2202982903, other_time: 20.4886474609
epoch time: 127.1213684082s, input_fired_time: 3.0201532841, output_fired_time: 0.0000000000, propagate_forward_time: 121.5553817749, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5458297729
validation[          , genome  5543] predictions:    6314/  10000 (63.14%), best:    6363/10000 (63.63%), error:     10644.14941, best error:     10491.29395 on epoch:    17, epoch:   21/25, mu: 0.8763815761, learning_rate: 0.0001193116, weight_decay: 0.0000706119

epoch time: 2117.0480957031s, input_fired_time: 36.8270187378, output_fired_time: 21.4216194153, propagate_forward_time: 606.5284423828, propagate_backward_time: 1429.2829589844, weight_update_time: 0.2051833272, other_time: 22.9881591797
epoch time: 127.6664047241s, input_fired_time: 1.0116505623, output_fired_time: 0.0000000000, propagate_forward_time: 124.0908966064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5638580322
validation[          , genome  5543] predictions:    6337/  10000 (63.37%), best:    6337/10000 (63.37%), error:     10483.11621, best error:     10483.11621 on epoch:    22, epoch:   22/25, mu: 0.8839813471, learning_rate: 0.0001111512, weight_decay: 0.0000663466

epoch time: 2127.8793945313s, input_fired_time: 41.0992774963, output_fired_time: 11.3959102631, propagate_forward_time: 618.5823364258, propagate_backward_time: 1433.5058593750, weight_update_time: 1.2112097740, other_time: 23.2958984375
epoch time: 127.7201080322s, input_fired_time: 5.0375986099, output_fired_time: 0.0000000000, propagate_forward_time: 120.0943756104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5881347656
validation[          , genome  5543] predictions:    6342/  10000 (63.42%), best:    6337/10000 (63.37%), error:     10486.23633, best error:     10483.11621 on epoch:    22, epoch:   23/25, mu: 0.8910727501, learning_rate: 0.0001035489, weight_decay: 0.0000623388

epoch time: 2130.7368164063s, input_fired_time: 46.8955459595, output_fired_time: 13.4144954681, propagate_forward_time: 607.5441894531, propagate_backward_time: 1438.7441406250, weight_update_time: 0.2069991529, other_time: 24.1384277344
epoch time: 129.4104003906s, input_fired_time: 2.0245320797, output_fired_time: 0.0000000000, propagate_forward_time: 123.7945709229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5913009644
validation[          , genome  5543] predictions:    6443/  10000 (64.43%), best:    6443/10000 (64.43%), error:     10374.91699, best error:     10374.91699 on epoch:    24, epoch:   24/25, mu: 0.8976898193, learning_rate: 0.0000964666, weight_decay: 0.0000585732

epoch time: 2116.1352539063s, input_fired_time: 47.8308334351, output_fired_time: 12.3609542847, propagate_forward_time: 596.6003417969, propagate_backward_time: 1433.0653076172, weight_update_time: 1.2112106085, other_time: 26.2779541016
epoch time: 129.4316101074s, input_fired_time: 1.0141963959, output_fired_time: 0.0000000000, propagate_forward_time: 123.8573684692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5600509644
validation[          , genome  5543] predictions:    6420/  10000 (64.20%), best:    6420/10000 (64.20%), error:     10354.09082, best error:     10354.09082 on epoch:    25, epoch:   25/25, mu: 0.9038643241, learning_rate: 0.0000898686, weight_decay: 0.0000550350

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 637.5390625000s, input_fired_time: 8.0933952332, output_fired_time: 0.0000000000, propagate_forward_time: 614.4117431641, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 15.0339355469
best training[          , genome  5543] predictions:   34894/  50000 (69.79%), best:    6420/10000 (64.20%), error:     45304.56250, best error:     10354.09082 on epoch:    25, epoch:   26/25, mu: 0.9096258283, learning_rate: 0.0000837220, weight_decay: 0.0000517106
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 130.2901916504s, input_fired_time: 3.0355162621, output_fired_time: 0.0000000000, propagate_forward_time: 123.1414413452, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1132354736
   testing[          , genome  5543] predictions:    6420/  10000 (64.20%), best:    6420/10000 (64.20%), error:     10354.09082, best error:     10354.09082 on epoch:    25, epoch:   26/25, mu: 0.9096258283, learning_rate: 0.0000837220, weight_decay: 0.0000517106
backpropagation finished successfully!
23:24:59 (3968): called boinc_finish(0)

</stderr_txt>
]]>