Nameexact_genome_1542189904_173_5888_2
Workunit4088544
Created20 Dec 2018, 2:46:34 UTC
Sent20 Dec 2018, 2:46:35 UTC
Report deadline24 Jan 2019, 17:16:00 UTC
Received20 Dec 2018, 17:14:40 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID31505
Run time13 hours 48 min 20 sec
CPU time13 hours 37 min 30 sec
Validate stateValid
Credit8,900.94
Device peak FLOPS4.25 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size586.50 MB
Peak swap size583.96 MB
Peak disk usage13.60 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5888.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5888_2_r2013756709_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5888.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.548834
read mu: 0.548834
read mu_delta: 0.972958
read initial_learning_rate: 0.00094942
read learning_rate: 0.00094942
read learning_rate_delta: 0.91542
read initial_weight_decay: 0.000314821
read weight_decay: 0.000314821
read weight_decay_delta: 0.914631
read batch_size: 122
read epsilon: 1e-07
read alpha: 0.122319
read input_dropout_probability: 0.00167777
read hidden_dropout_probability: 0.0635782
read velocity_reset: 1775
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5888
read normal distribution: '1 0 0'
generator_str: '168142454'
read generator: 168142454
read generated_by_map:
1 crossover 1
reading 102 nodes.
reading 428 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 2029228573
post shuffle 1: 141131287
epoch time: 1688.40197753906s, input_fired_time: 18.8186836242676, output_fired_time: 9.37009429931641, propagate_forward_time: 650.557067871094, propagate_backward_time: 997.43896484375, weight_update_time: 0.338268876075745, other_time: 12.2171020507813
epoch time: 133.822662353516s, input_fired_time: 1.4311558008194, output_fired_time: 0, propagate_forward_time: 130.208236694336, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.18327331542969
validation[          , genome  5888] predictions:    5913/  10000 (59.13%), best:    5913/10000 (59.13%), error:     11891.19531, best error:     11891.19531 on epoch:     0, epoch:    0/25, mu: 0.5488342047, learning_rate: 0.0009494196, weight_decay: 0.0003148215

epoch time: 1719.6855468750s, input_fired_time: 18.9622192383, output_fired_time: 9.4015245438, propagate_forward_time: 662.3201904297, propagate_backward_time: 1016.8433227539, weight_update_time: 0.3387998641, other_time: 12.1583862305
epoch time: 135.6071624756s, input_fired_time: 1.4647555351, output_fired_time: 0.0000000000, propagate_forward_time: 132.0521240234, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0902862549
validation[          , genome  5888] predictions:    6094/  10000 (60.94%), best:    6094/10000 (60.94%), error:     11266.47363, best error:     11266.47363 on epoch:     1, epoch:    1/25, mu: 0.5607642531, learning_rate: 0.0008691173, weight_decay: 0.0002879455

epoch time: 1739.7102050781s, input_fired_time: 19.0081367493, output_fired_time: 9.4937772751, propagate_forward_time: 667.9489135742, propagate_backward_time: 1031.1623535156, weight_update_time: 0.3404588699, other_time: 12.0969238281
epoch time: 139.0468444824s, input_fired_time: 1.4985381365, output_fired_time: 0.0000000000, propagate_forward_time: 135.3817138672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1665954590
validation[          , genome  5888] predictions:    6086/  10000 (60.86%), best:    6086/10000 (60.86%), error:     11246.12988, best error:     11246.12988 on epoch:     2, epoch:    2/25, mu: 0.5723717213, learning_rate: 0.0007956069, weight_decay: 0.0002633639

epoch time: 1746.0882568359s, input_fired_time: 19.0536594391, output_fired_time: 9.5234403610, propagate_forward_time: 671.9947509766, propagate_backward_time: 1033.4029541016, weight_update_time: 0.3419629335, other_time: 12.1134033203
epoch time: 136.1464538574s, input_fired_time: 1.4666179419, output_fired_time: 0.0000000000, propagate_forward_time: 132.5943756104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0854644775
validation[          , genome  5888] predictions:    5899/  10000 (58.99%), best:    6086/10000 (60.86%), error:     11723.27148, best error:     11246.12988 on epoch:     2, epoch:    3/25, mu: 0.5836653113, learning_rate: 0.0007283141, weight_decay: 0.0002408808

epoch time: 1912.0549316406s, input_fired_time: 21.0382118225, output_fired_time: 10.6371240616, propagate_forward_time: 738.6598510742, propagate_backward_time: 1127.9696044922, weight_update_time: 0.3941345513, other_time: 13.7501220703
epoch time: 134.5699462891s, input_fired_time: 1.4546673298, output_fired_time: 0.0000000000, propagate_forward_time: 131.0532073975, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0620727539
validation[          , genome  5888] predictions:    6151/  10000 (61.51%), best:    6151/10000 (61.51%), error:     11226.01855, best error:     11226.01855 on epoch:     4, epoch:    4/25, mu: 0.5946534872, learning_rate: 0.0006667129, weight_decay: 0.0002203171

epoch time: 1731.6713867188s, input_fired_time: 18.9463481903, output_fired_time: 9.4257764816, propagate_forward_time: 666.7246704102, propagate_backward_time: 1024.4708251953, weight_update_time: 0.3451352715, other_time: 12.1038818359
epoch time: 135.1078643799s, input_fired_time: 1.4553334713, output_fired_time: 0.0000000000, propagate_forward_time: 131.5857543945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0667724609
validation[          , genome  5888] predictions:    6060/  10000 (60.60%), best:    6151/10000 (61.51%), error:     11401.48047, best error:     11226.01855 on epoch:     4, epoch:    5/25, mu: 0.6053445339, learning_rate: 0.0006103221, weight_decay: 0.0002015088

epoch time: 1716.2188720703s, input_fired_time: 18.7733726501, output_fired_time: 9.2852058411, propagate_forward_time: 660.7644653320, propagate_backward_time: 1015.5068359375, weight_update_time: 0.3373113871, other_time: 11.8891601563
epoch time: 135.1491394043s, input_fired_time: 1.4603025913, output_fired_time: 0.0000000000, propagate_forward_time: 131.6161346436, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0727081299
validation[          , genome  5888] predictions:    6212/  10000 (62.12%), best:    6212/10000 (62.12%), error:     11009.39746, best error:     11009.39746 on epoch:     6, epoch:    6/25, mu: 0.6157464385, learning_rate: 0.0005587007, weight_decay: 0.0001843062

epoch time: 1738.6175537109s, input_fired_time: 18.9399738312, output_fired_time: 9.4082670212, propagate_forward_time: 672.4225463867, propagate_backward_time: 1026.0279541016, weight_update_time: 0.3392182887, other_time: 11.8187255859
epoch time: 134.7118988037s, input_fired_time: 1.4582971334, output_fired_time: 0.0000000000, propagate_forward_time: 131.2275238037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0260772705
validation[          , genome  5888] predictions:    6080/  10000 (60.80%), best:    6212/10000 (62.12%), error:     11388.73340, best error:     11009.39746 on epoch:     6, epoch:    7/25, mu: 0.6258670688, learning_rate: 0.0005114455, weight_decay: 0.0001685722

epoch time: 1781.3006591797s, input_fired_time: 19.1864128113, output_fired_time: 9.6733236313, propagate_forward_time: 687.7990112305, propagate_backward_time: 1052.5230712891, weight_update_time: 0.3424958885, other_time: 12.1187744141
epoch time: 134.5259094238s, input_fired_time: 1.4374935627, output_fired_time: 0.0000000000, propagate_forward_time: 131.0304718018, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0579376221
validation[          , genome  5888] predictions:    6142/  10000 (61.42%), best:    6212/10000 (62.12%), error:     11263.84766, best error:     11009.39746 on epoch:     6, epoch:    8/25, mu: 0.6357139945, learning_rate: 0.0004681872, weight_decay: 0.0001541814

epoch time: 1729.0034179688s, input_fired_time: 18.9340419769, output_fired_time: 9.4157180786, propagate_forward_time: 664.7646484375, propagate_backward_time: 1023.8704223633, weight_update_time: 0.3376231492, other_time: 12.0184936523
epoch time: 136.1838684082s, input_fired_time: 1.4768751860, output_fired_time: 0.0000000000, propagate_forward_time: 132.6078643799, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0991363525
validation[          , genome  5888] predictions:    6297/  10000 (62.97%), best:    6297/10000 (62.97%), error:     10822.03125, best error:     10822.03125 on epoch:     9, epoch:    9/25, mu: 0.6452946663, learning_rate: 0.0004285877, weight_decay: 0.0001410191

epoch time: 1760.7806396484s, input_fired_time: 19.3447628021, output_fired_time: 9.6273078918, propagate_forward_time: 678.0161743164, propagate_backward_time: 1041.6556396484, weight_update_time: 0.3436672688, other_time: 12.1368408203
epoch time: 138.1994476318s, input_fired_time: 1.4852269888, output_fired_time: 0.0000000000, propagate_forward_time: 134.6348114014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0794067383
validation[          , genome  5888] predictions:    6236/  10000 (62.36%), best:    6236/10000 (62.36%), error:     10788.40234, best error:     10788.40234 on epoch:    10, epoch:   10/25, mu: 0.6546162367, learning_rate: 0.0003923376, weight_decay: 0.0001289804

epoch time: 1749.8726806641s, input_fired_time: 19.1914062500, output_fired_time: 9.5069847107, propagate_forward_time: 673.5651855469, propagate_backward_time: 1035.6080322266, weight_update_time: 0.3502686024, other_time: 12.0010986328
epoch time: 138.3908538818s, input_fired_time: 1.5022023916, output_fired_time: 0.0000000000, propagate_forward_time: 134.8104553223, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0782012939
validation[          , genome  5888] predictions:    6225/  10000 (62.25%), best:    6236/10000 (62.36%), error:     10901.13281, best error:     10788.40234 on epoch:    10, epoch:   11/25, mu: 0.6636857390, learning_rate: 0.0003591535, weight_decay: 0.0001179695

epoch time: 1755.1230468750s, input_fired_time: 19.2085151672, output_fired_time: 9.5146369934, propagate_forward_time: 675.5659179688, propagate_backward_time: 1038.8497314453, weight_update_time: 0.3431799114, other_time: 11.9842529297
epoch time: 137.6436614990s, input_fired_time: 1.4916682243, output_fired_time: 0.0000000000, propagate_forward_time: 134.0946044922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0573883057
validation[          , genome  5888] predictions:    6311/  10000 (63.11%), best:    6311/10000 (63.11%), error:     10698.92676, best error:     10698.92676 on epoch:    12, epoch:   12/25, mu: 0.6725099683, learning_rate: 0.0003287761, weight_decay: 0.0001078986

epoch time: 1742.8084716797s, input_fired_time: 19.1154422760, output_fired_time: 9.4292707443, propagate_forward_time: 670.8506469727, propagate_backward_time: 1031.5351562500, weight_update_time: 0.3367360532, other_time: 11.8779296875
epoch time: 138.2555084229s, input_fired_time: 1.4881505966, output_fired_time: 0.0000000000, propagate_forward_time: 134.6841278076, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0832366943
validation[          , genome  5888] predictions:    6249/  10000 (62.49%), best:    6311/10000 (63.11%), error:     10932.37598, best error:     10698.92676 on epoch:    12, epoch:   13/25, mu: 0.6810956001, learning_rate: 0.0003009680, weight_decay: 0.0000986874

epoch time: 1741.0523681641s, input_fired_time: 19.1141357422, output_fired_time: 9.4326791763, propagate_forward_time: 669.7500610352, propagate_backward_time: 1030.8698730469, weight_update_time: 0.3373566568, other_time: 11.8854980469
epoch time: 137.5862121582s, input_fired_time: 1.4999154806, output_fired_time: 0.0000000000, propagate_forward_time: 134.0245361328, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0617675781
validation[          , genome  5888] predictions:    6274/  10000 (62.74%), best:    6311/10000 (63.11%), error:     10896.52441, best error:     10698.92676 on epoch:    12, epoch:   14/25, mu: 0.6894490123, learning_rate: 0.0002755120, weight_decay: 0.0000902626

epoch time: 1740.5838623047s, input_fired_time: 19.1200027466, output_fired_time: 9.4282875061, propagate_forward_time: 670.2316894531, propagate_backward_time: 1029.9527587891, weight_update_time: 0.3407260776, other_time: 11.8510742188
epoch time: 137.4463806152s, input_fired_time: 1.4932695627, output_fired_time: 0.0000000000, propagate_forward_time: 133.8753204346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0777893066
validation[          , genome  5888] predictions:    6242/  10000 (62.42%), best:    6311/10000 (63.11%), error:     10794.73535, best error:     10698.92676 on epoch:    12, epoch:   15/25, mu: 0.6975765824, learning_rate: 0.0002522091, weight_decay: 0.0000825570

epoch time: 1740.8081054688s, input_fired_time: 19.1294326782, output_fired_time: 9.4533758163, propagate_forward_time: 670.1761474609, propagate_backward_time: 1030.1589355469, weight_update_time: 0.3402135074, other_time: 11.8902587891
epoch time: 137.4778137207s, input_fired_time: 1.4898264408, output_fired_time: 0.0000000000, propagate_forward_time: 133.9280242920, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0599670410
validation[          , genome  5888] predictions:    6293/  10000 (62.93%), best:    6311/10000 (63.11%), error:     10762.77441, best error:     10698.92676 on epoch:    12, epoch:   16/25, mu: 0.7054843307, learning_rate: 0.0002308771, weight_decay: 0.0000755092

epoch time: 1740.4310302734s, input_fired_time: 19.1353073120, output_fired_time: 9.4633140564, propagate_forward_time: 669.7551269531, propagate_backward_time: 1030.1885986328, weight_update_time: 0.3376698494, other_time: 11.8887939453
epoch time: 137.4670715332s, input_fired_time: 1.4921858311, output_fired_time: 0.0000000000, propagate_forward_time: 133.9243469238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0505371094
validation[          , genome  5888] predictions:    6392/  10000 (63.92%), best:    6392/10000 (63.92%), error:     10525.46973, best error:     10525.46973 on epoch:    17, epoch:   17/25, mu: 0.7131782174, learning_rate: 0.0002113494, weight_decay: 0.0000690630

epoch time: 1745.5462646484s, input_fired_time: 19.1553096771, output_fired_time: 9.5141096115, propagate_forward_time: 672.0332641602, propagate_backward_time: 1032.9606933594, weight_update_time: 0.3420840800, other_time: 11.8828125000
epoch time: 137.5158996582s, input_fired_time: 1.4876084328, output_fired_time: 0.0000000000, propagate_forward_time: 133.9580383301, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0702514648
validation[          , genome  5888] predictions:    6433/  10000 (64.33%), best:    6433/10000 (64.33%), error:     10378.70703, best error:     10378.70703 on epoch:    18, epoch:   18/25, mu: 0.7206640840, learning_rate: 0.0001934734, weight_decay: 0.0000631672

epoch time: 1740.8129882813s, input_fired_time: 19.1502552032, output_fired_time: 9.4292621613, propagate_forward_time: 670.0101928711, propagate_backward_time: 1030.3135986328, weight_update_time: 0.3425953388, other_time: 11.9095458984
epoch time: 137.3656921387s, input_fired_time: 1.5069835186, output_fired_time: 0.0000000000, propagate_forward_time: 133.7831726074, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0755310059
validation[          , genome  5888] predictions:    6426/  10000 (64.26%), best:    6426/10000 (64.26%), error:     10364.80176, best error:     10364.80176 on epoch:    19, epoch:   19/25, mu: 0.7279475331, learning_rate: 0.0001771093, weight_decay: 0.0000577747

epoch time: 1730.7694091797s, input_fired_time: 19.0906581879, output_fired_time: 9.3695125580, propagate_forward_time: 665.8770141602, propagate_backward_time: 1024.5667724609, weight_update_time: 0.3399603367, other_time: 11.8653564453
epoch time: 136.1059722900s, input_fired_time: 1.4811713696, output_fired_time: 0.0000000000, propagate_forward_time: 132.5813140869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0434875488
validation[          , genome  5888] predictions:    6306/  10000 (63.06%), best:    6426/10000 (64.26%), error:     10773.79980, best error:     10364.80176 on epoch:    19, epoch:   20/25, mu: 0.7350339890, learning_rate: 0.0001621293, weight_decay: 0.0000528425

epoch time: 1722.8934326172s, input_fired_time: 19.0545158386, output_fired_time: 9.3138036728, propagate_forward_time: 662.5513305664, propagate_backward_time: 1020.2132568359, weight_update_time: 0.3374082744, other_time: 11.7603759766
epoch time: 136.1030273438s, input_fired_time: 1.4799258709, output_fired_time: 0.0000000000, propagate_forward_time: 132.5611419678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0619659424
validation[          , genome  5888] predictions:    6385/  10000 (63.85%), best:    6385/10000 (63.85%), error:     10354.76855, best error:     10354.76855 on epoch:    21, epoch:   21/25, mu: 0.7419288158, learning_rate: 0.0001484164, weight_decay: 0.0000483314

epoch time: 1727.3055419922s, input_fired_time: 19.1093521118, output_fired_time: 9.3623771667, propagate_forward_time: 664.5586547852, propagate_backward_time: 1022.5032958984, weight_update_time: 0.3334325254, other_time: 11.7718505859
epoch time: 135.9798736572s, input_fired_time: 1.4799631834, output_fired_time: 0.0000000000, propagate_forward_time: 132.4631195068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0367889404
validation[          , genome  5888] predictions:    6421/  10000 (64.21%), best:    6421/10000 (64.21%), error:     10344.82324, best error:     10344.82324 on epoch:    22, epoch:   22/25, mu: 0.7486371994, learning_rate: 0.0001358632, weight_decay: 0.0000442054

epoch time: 1723.7365722656s, input_fired_time: 19.0760879517, output_fired_time: 9.3256664276, propagate_forward_time: 663.0017089844, propagate_backward_time: 1020.5357055664, weight_update_time: 0.3343052566, other_time: 11.7974243164
epoch time: 135.8725891113s, input_fired_time: 1.4837056398, output_fired_time: 0.0000000000, propagate_forward_time: 132.3530578613, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0358276367
validation[          , genome  5888] predictions:    6466/  10000 (64.66%), best:    6466/10000 (64.66%), error:     10237.37988, best error:     10237.37988 on epoch:    23, epoch:   23/25, mu: 0.7551641464, learning_rate: 0.0001243719, weight_decay: 0.0000404316

epoch time: 1723.7908935547s, input_fired_time: 19.0721130371, output_fired_time: 9.3221035004, propagate_forward_time: 663.5355224609, propagate_backward_time: 1020.0814819336, weight_update_time: 0.3324166834, other_time: 11.7796020508
epoch time: 136.2217864990s, input_fired_time: 1.4868669510, output_fired_time: 0.0000000000, propagate_forward_time: 132.6794433594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0554809570
validation[          , genome  5888] predictions:    6475/  10000 (64.75%), best:    6475/10000 (64.75%), error:     10186.11426, best error:     10186.11426 on epoch:    24, epoch:   24/25, mu: 0.7615146041, learning_rate: 0.0001138524, weight_decay: 0.0000369800

epoch time: 1721.9997558594s, input_fired_time: 19.0644111633, output_fired_time: 9.3788890839, propagate_forward_time: 662.4865722656, propagate_backward_time: 1019.3007202148, weight_update_time: 0.3354878128, other_time: 11.7691040039
epoch time: 135.9984588623s, input_fired_time: 1.4802083969, output_fired_time: 0.0000000000, propagate_forward_time: 132.4496765137, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0685729980
validation[          , genome  5888] predictions:    6354/  10000 (63.54%), best:    6475/10000 (64.75%), error:     10521.89941, best error:     10186.11426 on epoch:    24, epoch:   25/25, mu: 0.7676933408, learning_rate: 0.0001042227, weight_decay: 0.0000338231

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 680.8467407227s, input_fired_time: 7.4503760338, output_fired_time: 0.0000000000, propagate_forward_time: 663.0818481445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.3145141602
best training[          , genome  5888] predictions:   35529/  50000 (71.06%), best:    6475/10000 (64.75%), error:     43586.82813, best error:     10186.11426 on epoch:    24, epoch:   26/25, mu: 0.7737050056, learning_rate: 0.0000954075, weight_decay: 0.0000309356
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 135.7528839111s, input_fired_time: 1.4873137474, output_fired_time: 0.0000000000, propagate_forward_time: 132.2335052490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0320587158
   testing[          , genome  5888] predictions:    6475/  10000 (64.75%), best:    6475/10000 (64.75%), error:     10186.11426, best error:     10186.11426 on epoch:    24, epoch:   26/25, mu: 0.7737050056, learning_rate: 0.0000954075, weight_decay: 0.0000309356
backpropagation finished successfully!
02:13:28 (16220): called boinc_finish(0)

</stderr_txt>
]]>