Nameexact_genome_1542605141_172_6299_0
Workunit4098455
Created19 Nov 2018, 6:39:18 UTC
Sent19 Nov 2018, 6:58:13 UTC
Report deadline18 Dec 2018, 0:50:45 UTC
Received20 Nov 2018, 3:37:52 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID85065
Run time19 hours 38 min 19 sec
CPU time18 hours 52 min 44 sec
Validate stateValid
Credit7,186.21
Device peak FLOPS4.24 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size583.61 MB
Peak swap size603.35 MB
Peak disk usage13.09 MB

Stderr output

<core_client_version>7.10.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_172_6299.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_172_6299_0_r465407038_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542605141_172_6299.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.480447
read mu: 0.480447
read mu_delta: 0.939751
read initial_learning_rate: 0.0016269
read learning_rate: 0.0016269
read learning_rate_delta: 0.922622
read initial_weight_decay: 0.000230485
read weight_decay: 0.000230485
read weight_decay_delta: 0.943521
read batch_size: 104
read epsilon: 1e-07
read alpha: 0.0892642
read input_dropout_probability: 0.00137477
read hidden_dropout_probability: 0.081123
read velocity_reset: 1682
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6299
read normal distribution: '0 0.271294146776199 -0.244710609316826'
generator_str: '1896640551'
read generator: 1896640551
read generated_by_map:
1 change_size_x 1
reading 106 nodes.
reading 429 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1737968236
post shuffle 1: 580655037
epoch time: 2151.73364257813s, input_fired_time: 34.4387512207031, output_fired_time: 14.5995569229126, propagate_forward_time: 846.506958007813, propagate_backward_time: 1234.40563964844, weight_update_time: 0.578198313713074, other_time: 21.78271484375
epoch time: 176.586303710938s, input_fired_time: 2.67434072494507, output_fired_time: 0, propagate_forward_time: 169.920120239258, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.99183654785156
validation[          , genome  6299] predictions:    6246/  10000 (62.46%), best:    6246/10000 (62.46%), error:     10894.19141, best error:     10894.19141 on epoch:     0, epoch:    0/25, mu: 0.4804465771, learning_rate: 0.0016268980, weight_decay: 0.0002304852

epoch time: 2154.4079589844s, input_fired_time: 34.6772346497, output_fired_time: 14.5335292816, propagate_forward_time: 845.9202270508, propagate_backward_time: 1237.3103027344, weight_update_time: 0.5519995689, other_time: 21.9667968750
epoch time: 175.7504882813s, input_fired_time: 2.6255338192, output_fired_time: 0.0000000000, propagate_forward_time: 169.1942138672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9307403564
validation[          , genome  6299] predictions:    6273/  10000 (62.73%), best:    6273/10000 (62.73%), error:     10772.26855, best error:     10772.26855 on epoch:     1, epoch:    1/25, mu: 0.5111467838, learning_rate: 0.0015010114, weight_decay: 0.0002174676

epoch time: 2121.8522949219s, input_fired_time: 34.0936279297, output_fired_time: 14.4571905136, propagate_forward_time: 833.6075439453, propagate_backward_time: 1218.7349853516, weight_update_time: 0.5550729036, other_time: 20.9589843750
epoch time: 176.1335754395s, input_fired_time: 2.7662692070, output_fired_time: 0.0000000000, propagate_forward_time: 169.1787872314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1885223389
validation[          , genome  6299] predictions:    6283/  10000 (62.83%), best:    6283/10000 (62.83%), error:     10665.95508, best error:     10665.95508 on epoch:     2, epoch:    2/25, mu: 0.5399973392, learning_rate: 0.0013848657, weight_decay: 0.0002051852

epoch time: 2141.3925781250s, input_fired_time: 34.3655014038, output_fired_time: 14.5494089127, propagate_forward_time: 843.7915039063, propagate_backward_time: 1227.2602539063, weight_update_time: 0.5522003174, other_time: 21.4260253906
epoch time: 180.1300964355s, input_fired_time: 2.7191345692, output_fired_time: 0.0000000000, propagate_forward_time: 173.7279052734, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6830596924
validation[          , genome  6299] predictions:    5906/  10000 (59.06%), best:    6283/10000 (62.83%), error:     11898.63477, best error:     10665.95508 on epoch:     2, epoch:    3/25, mu: 0.5671096444, learning_rate: 0.0012777072, weight_decay: 0.0001935966

epoch time: 2120.1552734375s, input_fired_time: 33.8783721924, output_fired_time: 14.3052453995, propagate_forward_time: 833.5029296875, propagate_backward_time: 1217.3565673828, weight_update_time: 0.5397829413, other_time: 21.1121826172
epoch time: 174.3977355957s, input_fired_time: 2.5906083584, output_fired_time: 0.0000000000, propagate_forward_time: 167.7642364502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0428924561
validation[          , genome  6299] predictions:    6265/  10000 (62.65%), best:    6265/10000 (62.65%), error:     10587.85645, best error:     10587.85645 on epoch:     4, epoch:    4/25, mu: 0.5925884843, learning_rate: 0.0011788404, weight_decay: 0.0001826624

epoch time: 2115.0917968750s, input_fired_time: 33.9365272522, output_fired_time: 14.3361692429, propagate_forward_time: 831.8021240234, propagate_backward_time: 1213.8802490234, weight_update_time: 0.5350738168, other_time: 21.1367187500
epoch time: 176.6230468750s, input_fired_time: 2.6158018112, output_fired_time: 0.0000000000, propagate_forward_time: 170.2526245117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7546234131
validation[          , genome  6299] predictions:    6209/  10000 (62.09%), best:    6265/10000 (62.65%), error:     10965.70898, best error:     10587.85645 on epoch:     4, epoch:    5/25, mu: 0.6165322661, learning_rate: 0.0010876238, weight_decay: 0.0001723458

epoch time: 2099.7177734375s, input_fired_time: 33.5853691101, output_fired_time: 14.1913547516, propagate_forward_time: 824.3741455078, propagate_backward_time: 1206.9710693359, weight_update_time: 0.5490014553, other_time: 20.5957031250
epoch time: 174.0494537354s, input_fired_time: 2.6335003376, output_fired_time: 0.0000000000, propagate_forward_time: 167.6894226074, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7265319824
validation[          , genome  6299] predictions:    6161/  10000 (61.61%), best:    6265/10000 (62.65%), error:     10916.16504, best error:     10587.85645 on epoch:     4, epoch:    6/25, mu: 0.6390334368, learning_rate: 0.0010034653, weight_decay: 0.0001626119

epoch time: 2101.4130859375s, input_fired_time: 33.7269210815, output_fired_time: 14.2643661499, propagate_forward_time: 828.0040893555, propagate_backward_time: 1204.0885009766, weight_update_time: 0.5355369449, other_time: 21.3292236328
epoch time: 169.4818878174s, input_fired_time: 2.5417902470, output_fired_time: 0.0000000000, propagate_forward_time: 163.0822448730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8578491211
validation[          , genome  6299] predictions:    6361/  10000 (63.61%), best:    6361/10000 (63.61%), error:     10495.53027, best error:     10495.53027 on epoch:     7, epoch:    7/25, mu: 0.6601788998, learning_rate: 0.0009258189, weight_decay: 0.0001534277

epoch time: 2069.4592285156s, input_fired_time: 33.3426704407, output_fired_time: 14.0040302277, propagate_forward_time: 814.5911254883, propagate_backward_time: 1187.1505126953, weight_update_time: 0.5480408072, other_time: 20.3709716797
epoch time: 170.5342559814s, input_fired_time: 2.6062204838, output_fired_time: 0.0000000000, propagate_forward_time: 164.2899932861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6380462646
validation[          , genome  6299] predictions:    6293/  10000 (62.93%), best:    6361/10000 (63.61%), error:     10754.71875, best error:     10495.53027 on epoch:     7, epoch:    8/25, mu: 0.6800503731, learning_rate: 0.0008541806, weight_decay: 0.0001447623

epoch time: 2065.2204589844s, input_fired_time: 33.0395736694, output_fired_time: 13.9648933411, propagate_forward_time: 813.5468750000, propagate_backward_time: 1184.1920166016, weight_update_time: 0.5254555941, other_time: 20.4771728516
epoch time: 170.7050781250s, input_fired_time: 2.5990459919, output_fired_time: 0.0000000000, propagate_forward_time: 164.5085449219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5974884033
validation[          , genome  6299] predictions:    6284/  10000 (62.84%), best:    6361/10000 (63.61%), error:     10508.34277, best error:     10495.53027 on epoch:     7, epoch:    9/25, mu: 0.6987246275, learning_rate: 0.0007880856, weight_decay: 0.0001365862

epoch time: 2048.6557617188s, input_fired_time: 32.8420791626, output_fired_time: 13.8150882721, propagate_forward_time: 805.8127441406, propagate_backward_time: 1176.1713867188, weight_update_time: 0.5247936249, other_time: 20.0145263672
epoch time: 167.3713073730s, input_fired_time: 2.6026852131, output_fired_time: 0.0000000000, propagate_forward_time: 161.2465972900, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5220184326
validation[          , genome  6299] predictions:    6410/  10000 (64.10%), best:    6410/10000 (64.10%), error:     10236.27734, best error:     10236.27734 on epoch:    10, epoch:   10/25, mu: 0.7162737846, learning_rate: 0.0007271049, weight_decay: 0.0001288720

epoch time: 2201.7585449219s, input_fired_time: 35.6058769226, output_fired_time: 14.8322410583, propagate_forward_time: 863.5255126953, propagate_backward_time: 1265.9233398438, weight_update_time: 0.5851969719, other_time: 21.8714599609
epoch time: 190.2965393066s, input_fired_time: 3.0184628963, output_fired_time: 0.0000000000, propagate_forward_time: 182.7984466553, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4796295166
validation[          , genome  6299] predictions:    6291/  10000 (62.91%), best:    6410/10000 (64.10%), error:     10585.97070, best error:     10236.27734 on epoch:    10, epoch:   11/25, mu: 0.7327656150, learning_rate: 0.0006708428, weight_decay: 0.0001215934

epoch time: 2411.0104980469s, input_fired_time: 38.8449897766, output_fired_time: 16.3163528442, propagate_forward_time: 949.9845581055, propagate_backward_time: 1381.9383544922, weight_update_time: 0.6738757491, other_time: 23.9261474609
epoch time: 194.2491607666s, input_fired_time: 2.8787581921, output_fired_time: 0.0000000000, propagate_forward_time: 187.5110015869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8594055176
validation[          , genome  6299] predictions:    6187/  10000 (61.87%), best:    6410/10000 (64.10%), error:     10886.91016, best error:     10236.27734 on epoch:    10, epoch:   12/25, mu: 0.7482637763, learning_rate: 0.0006189341, weight_decay: 0.0001147259

epoch time: 2445.7695312500s, input_fired_time: 39.2013969421, output_fired_time: 16.5315837860, propagate_forward_time: 960.4704589844, propagate_backward_time: 1406.0612792969, weight_update_time: 0.6300193667, other_time: 23.5048828125
epoch time: 205.6197052002s, input_fired_time: 3.1573691368, output_fired_time: 0.0000000000, propagate_forward_time: 198.4904479980, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9718933105
validation[          , genome  6299] predictions:    6334/  10000 (63.34%), best:    6410/10000 (64.10%), error:     10583.23535, best error:     10236.27734 on epoch:    10, epoch:   13/25, mu: 0.7628282309, learning_rate: 0.0005710421, weight_decay: 0.0001082463

epoch time: 2516.9443359375s, input_fired_time: 40.4624900818, output_fired_time: 17.0017929077, propagate_forward_time: 985.5932617188, propagate_backward_time: 1449.7145996094, weight_update_time: 0.6523004770, other_time: 24.1723632813
epoch time: 216.4114532471s, input_fired_time: 3.3011112213, output_fired_time: 0.0000000000, propagate_forward_time: 208.4337921143, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6765441895
validation[          , genome  6299] predictions:    6361/  10000 (63.61%), best:    6410/10000 (64.10%), error:     10437.39160, best error:     10236.27734 on epoch:    10, epoch:   14/25, mu: 0.7765151858, learning_rate: 0.0005268558, weight_decay: 0.0001021327

epoch time: 2604.2478027344s, input_fired_time: 41.8768615723, output_fired_time: 17.5286808014, propagate_forward_time: 1021.3344726563, propagate_backward_time: 1498.6510009766, weight_update_time: 0.6527248025, other_time: 24.8568115234
epoch time: 223.3128509521s, input_fired_time: 3.4606525898, output_fired_time: 0.0000000000, propagate_forward_time: 215.3974609375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4547424316
validation[          , genome  6299] predictions:    6443/  10000 (64.43%), best:    6410/10000 (64.10%), error:     10253.55859, best error:     10236.27734 on epoch:    10, epoch:   15/25, mu: 0.7893775105, learning_rate: 0.0004860886, weight_decay: 0.0000963643

epoch time: 2765.2497558594s, input_fired_time: 44.2078323364, output_fired_time: 18.6989192963, propagate_forward_time: 1087.8063964844, propagate_backward_time: 1587.9763183594, weight_update_time: 0.6938079000, other_time: 26.5603027344
epoch time: 230.1446533203s, input_fired_time: 3.5091090202, output_fired_time: 0.0000000000, propagate_forward_time: 222.1399230957, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4956207275
validation[          , genome  6299] predictions:    6467/  10000 (64.67%), best:    6467/10000 (64.67%), error:     10137.74609, best error:     10137.74609 on epoch:    16, epoch:   16/25, mu: 0.8014648557, learning_rate: 0.0004484759, weight_decay: 0.0000909218

epoch time: 2797.3046875000s, input_fired_time: 44.9361114502, output_fired_time: 19.0239181519, propagate_forward_time: 1101.9128417969, propagate_backward_time: 1604.8641357422, weight_update_time: 0.7051141262, other_time: 26.5677490234
epoch time: 221.0001831055s, input_fired_time: 3.3896534443, output_fired_time: 0.0000000000, propagate_forward_time: 213.0715942383, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5389404297
validation[          , genome  6299] predictions:    6298/  10000 (62.98%), best:    6467/10000 (64.67%), error:     10465.94629, best error:     10137.74609 on epoch:    16, epoch:   17/25, mu: 0.8128239512, learning_rate: 0.0004137736, weight_decay: 0.0000857866

epoch time: 2664.3059082031s, input_fired_time: 42.6260871887, output_fired_time: 17.9934425354, propagate_forward_time: 1045.9377441406, propagate_backward_time: 1532.7286376953, weight_update_time: 0.6645131111, other_time: 25.0201416016
epoch time: 236.3496398926s, input_fired_time: 3.5687868595, output_fired_time: 0.0000000000, propagate_forward_time: 227.9979400635, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7829132080
validation[          , genome  6299] predictions:    6308/  10000 (63.08%), best:    6467/10000 (64.67%), error:     10450.75195, best error:     10137.74609 on epoch:    16, epoch:   18/25, mu: 0.8234986663, learning_rate: 0.0003817565, weight_decay: 0.0000809414

epoch time: 2853.7570800781s, input_fired_time: 45.6730003357, output_fired_time: 19.1913585663, propagate_forward_time: 1124.2232666016, propagate_backward_time: 1638.7097167969, weight_update_time: 0.7133063674, other_time: 25.9595947266
epoch time: 238.3891754150s, input_fired_time: 3.5826239586, output_fired_time: 0.0000000000, propagate_forward_time: 230.0947875977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7117614746
validation[          , genome  6299] predictions:    6486/  10000 (64.86%), best:    6467/10000 (64.67%), error:     10176.17090, best error:     10137.74609 on epoch:    16, epoch:   19/25, mu: 0.8335302472, learning_rate: 0.0003522169, weight_decay: 0.0000763699

epoch time: 2863.7253417969s, input_fired_time: 45.8452072144, output_fired_time: 19.2104988098, propagate_forward_time: 1127.8911132813, propagate_backward_time: 1643.9053955078, weight_update_time: 0.7286656499, other_time: 26.8731689453
epoch time: 234.6121063232s, input_fired_time: 3.6073555946, output_fired_time: 0.0000000000, propagate_forward_time: 226.1411132813, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8636322021
validation[          , genome  6299] predictions:    6439/  10000 (64.39%), best:    6467/10000 (64.67%), error:     10281.32813, best error:     10137.74609 on epoch:    16, epoch:   20/25, mu: 0.8429574370, learning_rate: 0.0003249630, weight_decay: 0.0000720566

epoch time: 2863.0246582031s, input_fired_time: 46.2163467407, output_fired_time: 19.4104614258, propagate_forward_time: 1127.5009765625, propagate_backward_time: 1643.3752441406, weight_update_time: 0.6966252923, other_time: 26.5217285156
epoch time: 232.0104522705s, input_fired_time: 3.5727188587, output_fired_time: 0.0000000000, propagate_forward_time: 223.7599639893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6777648926
validation[          , genome  6299] predictions:    6381/  10000 (63.81%), best:    6467/10000 (64.67%), error:     10283.24121, best error:     10137.74609 on epoch:    16, epoch:   21/25, mu: 0.8518166542, learning_rate: 0.0002998179, weight_decay: 0.0000679870

epoch time: 2814.7575683594s, input_fired_time: 44.8823509216, output_fired_time: 19.0354938507, propagate_forward_time: 1105.0383300781, propagate_backward_time: 1619.7593994141, weight_update_time: 0.7005991936, other_time: 26.0421142578
epoch time: 229.1698150635s, input_fired_time: 3.5062544346, output_fired_time: 0.0000000000, propagate_forward_time: 221.0978240967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5657348633
validation[          , genome  6299] predictions:    6399/  10000 (63.99%), best:    6467/10000 (64.67%), error:     10260.95117, best error:     10137.74609 on epoch:    16, epoch:   22/25, mu: 0.8601421118, learning_rate: 0.0002766185, weight_decay: 0.0000641471

epoch time: 2831.9648437500s, input_fired_time: 45.3824424744, output_fired_time: 18.8975086212, propagate_forward_time: 1115.6738281250, propagate_backward_time: 1625.2662353516, weight_update_time: 0.6959183216, other_time: 26.7449951172
epoch time: 235.0852966309s, input_fired_time: 3.5221416950, output_fired_time: 0.0000000000, propagate_forward_time: 226.8732910156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6898651123
validation[          , genome  6299] predictions:    6420/  10000 (64.20%), best:    6467/10000 (64.67%), error:     10288.97656, best error:     10137.74609 on epoch:    16, epoch:   23/25, mu: 0.8679659367, learning_rate: 0.0002552142, weight_decay: 0.0000605242

epoch time: 2884.4079589844s, input_fired_time: 46.0869941711, output_fired_time: 19.3278980255, propagate_forward_time: 1136.3394775391, propagate_backward_time: 1656.7221679688, weight_update_time: 0.7122936249, other_time: 25.9315185547
epoch time: 240.5345001221s, input_fired_time: 3.7075405121, output_fired_time: 0.0000000000, propagate_forward_time: 232.4594116211, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3675537109
validation[          , genome  6299] predictions:    6557/  10000 (65.57%), best:    6557/10000 (65.57%), error:      9982.00195, best error:      9982.00195 on epoch:    24, epoch:   24/25, mu: 0.8753184080, learning_rate: 0.0002354662, weight_decay: 0.0000571058

epoch time: 3077.1096191406s, input_fired_time: 49.3431015015, output_fired_time: 20.6428718567, propagate_forward_time: 1210.0490722656, propagate_backward_time: 1769.6566162109, weight_update_time: 0.7630510926, other_time: 27.4180908203
epoch time: 260.6072692871s, input_fired_time: 3.9659790993, output_fired_time: 0.0000000000, propagate_forward_time: 251.8132629395, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8280334473
validation[          , genome  6299] predictions:    6458/  10000 (64.58%), best:    6557/10000 (65.57%), error:     10149.04395, best error:      9982.00195 on epoch:    24, epoch:   25/25, mu: 0.8822278976, learning_rate: 0.0002172462, weight_decay: 0.0000538805

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 1307.9158935547s, input_fired_time: 20.1513442993, output_fired_time: 0.0000000000, propagate_forward_time: 1262.4547119141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 25.3098144531
best training[          , genome  6299] predictions:   36664/  50000 (73.33%), best:    6557/10000 (65.57%), error:     39805.92578, best error:      9982.00195 on epoch:    24, epoch:   26/25, mu: 0.8887211084, learning_rate: 0.0002004361, weight_decay: 0.0000508374
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 266.4049377441s, input_fired_time: 4.1458373070, output_fired_time: 0.0000000000, propagate_forward_time: 256.8623046875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3967895508
   testing[          , genome  6299] predictions:    6557/  10000 (65.57%), best:    6557/10000 (65.57%), error:      9982.00195, best error:      9982.00195 on epoch:    24, epoch:   26/25, mu: 0.8887211084, learning_rate: 0.0002004361, weight_decay: 0.0000508374
backpropagation finished successfully!
21:36:41 (26820): called boinc_finish(0)

</stderr_txt>
]]>