Nameexact_genome_1542727803_172_6420_0
Workunit4101316
Created20 Nov 2018, 20:14:39 UTC
Sent20 Nov 2018, 20:15:03 UTC
Report deadline18 Dec 2018, 20:02:26 UTC
Received21 Nov 2018, 6:16:47 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60660
Run time8 hours 55 min 3 sec
CPU time8 hours 35 min 58 sec
Validate stateValid
Credit6,997.81
Device peak FLOPS3.44 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size576.32 MB
Peak swap size596.23 MB
Peak disk usage13.10 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_172_6420.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_172_6420_0_r176093485_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542727803_172_6420.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.500824
read mu: 0.500824
read mu_delta: 0.939657
read initial_learning_rate: 0.000295879
read learning_rate: 0.000295879
read learning_rate_delta: 0.932347
read initial_weight_decay: 0.000215694
read weight_decay: 0.000215694
read weight_decay_delta: 0.943328
read batch_size: 102
read epsilon: 1e-07
read alpha: 0.0953097
read input_dropout_probability: 0.00126598
read hidden_dropout_probability: 0.0887064
read velocity_reset: 2008
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6420
read normal distribution: '1 0 0'
generator_str: '2039324554'
read generator: 2039324554
read generated_by_map:
1 disable_edge 1
reading 107 nodes.
reading 447 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1088772958
post shuffle 1: 953447888
epoch time: 1113.46240234375s, input_fired_time: 24.6358909606934, output_fired_time: 8.47062397003174, propagate_forward_time: 407.117858886719, propagate_backward_time: 659.16455078125, weight_update_time: 0.34182333946228, other_time: 14.0735473632813
epoch time: 85.8739852905273s, input_fired_time: 1.53825223445892, output_fired_time: 0, propagate_forward_time: 81.9317169189453, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.40401458740234
validation[          , genome  6420] predictions:    6300/  10000 (63.00%), best:    6300/10000 (63.00%), error:     10745.33398, best error:     10745.33398 on epoch:     0, epoch:    0/25, mu: 0.5008237958, learning_rate: 0.0002958791, weight_decay: 0.0002156940

epoch time: 1107.8417968750s, input_fired_time: 24.4456787109, output_fired_time: 8.4164791107, propagate_forward_time: 404.8019409180, propagate_backward_time: 656.1149291992, weight_update_time: 0.3432628810, other_time: 14.0627441406
epoch time: 86.1161499023s, input_fired_time: 1.5491397381, output_fired_time: 0.0000000000, propagate_forward_time: 82.1292343140, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4377746582
validation[          , genome  6420] predictions:    6268/  10000 (62.68%), best:    6300/10000 (63.00%), error:     10748.25879, best error:     10745.33398 on epoch:     0, epoch:    1/25, mu: 0.5303423405, learning_rate: 0.0002758621, weight_decay: 0.0002034702

epoch time: 1114.5592041016s, input_fired_time: 24.6771774292, output_fired_time: 8.5348730087, propagate_forward_time: 407.1605834961, propagate_backward_time: 660.0126342773, weight_update_time: 0.3402567804, other_time: 14.1739501953
epoch time: 86.1408767700s, input_fired_time: 1.5461398363, output_fired_time: 0.0000000000, propagate_forward_time: 82.1629409790, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4317932129
validation[          , genome  6420] predictions:    6185/  10000 (61.85%), best:    6300/10000 (63.00%), error:     11116.17676, best error:     10745.33398 on epoch:     0, epoch:    2/25, mu: 0.5580796003, learning_rate: 0.0002571993, weight_decay: 0.0001919392

epoch time: 1111.0324707031s, input_fired_time: 24.6499023438, output_fired_time: 8.4746150970, propagate_forward_time: 405.9521789551, propagate_backward_time: 657.8751220703, weight_update_time: 0.3426924348, other_time: 14.0806884766
epoch time: 85.1558456421s, input_fired_time: 1.5428115129, output_fired_time: 0.0000000000, propagate_forward_time: 81.2324371338, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3805999756
validation[          , genome  6420] predictions:    6069/  10000 (60.69%), best:    6300/10000 (63.00%), error:     11401.35547, best error:     10745.33398 on epoch:     0, epoch:    3/25, mu: 0.5841431022, learning_rate: 0.0002397990, weight_decay: 0.0001810617

epoch time: 1105.9849853516s, input_fired_time: 24.3977909088, output_fired_time: 8.4177894592, propagate_forward_time: 404.0728149414, propagate_backward_time: 655.1157226563, weight_update_time: 0.3481144011, other_time: 13.9807739258
epoch time: 85.4051208496s, input_fired_time: 1.5278084278, output_fired_time: 0.0000000000, propagate_forward_time: 81.4647216797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4125900269
validation[          , genome  6420] predictions:    5865/  10000 (58.65%), best:    6300/10000 (63.00%), error:     11909.97754, best error:     10745.33398 on epoch:     0, epoch:    4/25, mu: 0.6086338758, learning_rate: 0.0002235759, weight_decay: 0.0001708006

epoch time: 1333.7666015625s, input_fired_time: 24.4505767822, output_fired_time: 8.4496412277, propagate_forward_time: 405.3353271484, propagate_backward_time: 881.4365234375, weight_update_time: 0.3452662826, other_time: 14.0946044922
epoch time: 86.0303726196s, input_fired_time: 1.5329673290, output_fired_time: 0.0000000000, propagate_forward_time: 82.0542144775, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4431915283
validation[          , genome  6420] predictions:    6020/  10000 (60.20%), best:    6300/10000 (63.00%), error:     11604.43750, best error:     10745.33398 on epoch:     0, epoch:    5/25, mu: 0.6316467524, learning_rate: 0.0002084504, weight_decay: 0.0001611210

epoch time: 1283.9427490234s, input_fired_time: 24.5603561401, output_fired_time: 8.4478368759, propagate_forward_time: 405.2357482910, propagate_backward_time: 831.6702880859, weight_update_time: 0.3419898152, other_time: 14.0285644531
epoch time: 85.3134994507s, input_fired_time: 1.5252100229, output_fired_time: 0.0000000000, propagate_forward_time: 81.3390731812, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4492187500
validation[          , genome  6420] predictions:    6174/  10000 (61.74%), best:    6300/10000 (63.00%), error:     11171.47461, best error:     10745.33398 on epoch:     0, epoch:    6/25, mu: 0.6532709599, learning_rate: 0.0001943482, weight_decay: 0.0001519900

epoch time: 1114.3985595703s, input_fired_time: 24.5233707428, output_fired_time: 8.4862108231, propagate_forward_time: 407.2263793945, propagate_backward_time: 660.0472412109, weight_update_time: 0.3403160870, other_time: 14.1154174805
epoch time: 86.1208496094s, input_fired_time: 1.5026270151, output_fired_time: 0.0000000000, propagate_forward_time: 82.2211380005, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3970870972
validation[          , genome  6420] predictions:    5909/  10000 (59.09%), best:    6300/10000 (63.00%), error:     11749.15234, best error:     10745.33398 on epoch:     0, epoch:    7/25, mu: 0.6735903025, learning_rate: 0.0001812000, weight_decay: 0.0001433765

epoch time: 1111.2791748047s, input_fired_time: 24.7152156830, output_fired_time: 8.4080343246, propagate_forward_time: 406.4002685547, propagate_backward_time: 657.5944213867, weight_update_time: 0.3424317241, other_time: 14.1611938477
epoch time: 85.6456527710s, input_fired_time: 1.5503895283, output_fired_time: 0.0000000000, propagate_forward_time: 81.6305847168, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4646759033
validation[          , genome  6420] predictions:    6280/  10000 (62.80%), best:    6300/10000 (63.00%), error:     10944.42480, best error:     10745.33398 on epoch:     0, epoch:    8/25, mu: 0.6926835179, learning_rate: 0.0001689413, weight_decay: 0.0001352511

epoch time: 1108.1062011719s, input_fired_time: 24.4739246368, output_fired_time: 8.4523906708, propagate_forward_time: 404.8100585938, propagate_backward_time: 656.2377319336, weight_update_time: 0.3426414132, other_time: 14.1321411133
epoch time: 87.0240859985s, input_fired_time: 1.5039216280, output_fired_time: 0.0000000000, propagate_forward_time: 83.1108398438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4093246460
validation[          , genome  6420] predictions:    5688/  10000 (56.88%), best:    6300/10000 (63.00%), error:     12834.19824, best error:     10745.33398 on epoch:     0, epoch:    9/25, mu: 0.7106245756, learning_rate: 0.0001575119, weight_decay: 0.0001275862

epoch time: 1768.3812255859s, input_fired_time: 24.4347686768, output_fired_time: 8.4748687744, propagate_forward_time: 846.6080322266, propagate_backward_time: 874.8614501953, weight_update_time: 0.3440459669, other_time: 14.0020751953
epoch time: 86.2032928467s, input_fired_time: 1.5562984943, output_fired_time: 0.0000000000, propagate_forward_time: 82.2375869751, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4094085693
validation[          , genome  6420] predictions:    6151/  10000 (61.51%), best:    6300/10000 (63.00%), error:     11184.81738, best error:     10745.33398 on epoch:     0, epoch:   10/25, mu: 0.7274830341, learning_rate: 0.0001468558, weight_decay: 0.0001203556

epoch time: 1111.2374267578s, input_fired_time: 24.4769229889, output_fired_time: 8.3945455551, propagate_forward_time: 406.1512451172, propagate_backward_time: 658.0466918945, weight_update_time: 0.3464179039, other_time: 14.1680297852
epoch time: 84.7587051392s, input_fired_time: 1.5039240122, output_fired_time: 0.0000000000, propagate_forward_time: 80.8040542603, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4507293701
validation[          , genome  6420] predictions:    6063/  10000 (60.63%), best:    6300/10000 (63.00%), error:     11475.20215, best error:     10745.33398 on epoch:     0, epoch:   11/25, mu: 0.7433241606, learning_rate: 0.0001369206, weight_decay: 0.0001135349

epoch time: 1108.3571777344s, input_fired_time: 24.4406452179, output_fired_time: 8.4608335495, propagate_forward_time: 405.4407958984, propagate_backward_time: 656.0935668945, weight_update_time: 0.3685005903, other_time: 13.9213256836
epoch time: 86.6502532959s, input_fired_time: 1.5359032154, output_fired_time: 0.0000000000, propagate_forward_time: 82.6899642944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4243850708
validation[          , genome  6420] predictions:    6124/  10000 (61.24%), best:    6300/10000 (63.00%), error:     11389.93457, best error:     10745.33398 on epoch:     0, epoch:   12/25, mu: 0.7582094073, learning_rate: 0.0001276576, weight_decay: 0.0001071006

epoch time: 1481.1134033203s, input_fired_time: 24.3349132538, output_fired_time: 8.4487581253, propagate_forward_time: 404.1576843262, propagate_backward_time: 1030.1518554688, weight_update_time: 0.3515906632, other_time: 14.0201416016
epoch time: 85.0361099243s, input_fired_time: 1.4954965115, output_fired_time: 0.0000000000, propagate_forward_time: 81.0544967651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4861145020
validation[          , genome  6420] predictions:    5846/  10000 (58.46%), best:    6300/10000 (63.00%), error:     12024.37891, best error:     10745.33398 on epoch:     0, epoch:   13/25, mu: 0.7721964121, learning_rate: 0.0001190212, weight_decay: 0.0001010311

epoch time: 1708.4311523438s, input_fired_time: 24.4109764099, output_fired_time: 8.4535932541, propagate_forward_time: 404.2185668945, propagate_backward_time: 1257.2980957031, weight_update_time: 0.3414458036, other_time: 14.0498046875
epoch time: 85.1510848999s, input_fired_time: 1.5031450987, output_fired_time: 0.0000000000, propagate_forward_time: 81.2229003906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4250411987
validation[          , genome  6420] predictions:    5806/  10000 (58.06%), best:    6300/10000 (63.00%), error:     12226.49512, best error:     10745.33398 on epoch:     0, epoch:   14/25, mu: 0.7853394151, learning_rate: 0.0001109691, weight_decay: 0.0000953055

epoch time: 1106.8952636719s, input_fired_time: 24.4758872986, output_fired_time: 8.3964080811, propagate_forward_time: 404.5237426758, propagate_backward_time: 655.5439453125, weight_update_time: 0.3426726758, other_time: 13.9553833008
epoch time: 85.3780822754s, input_fired_time: 1.5197950602, output_fired_time: 0.0000000000, propagate_forward_time: 81.4169921875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4412918091
validation[          , genome  6420] predictions:    6176/  10000 (61.76%), best:    6300/10000 (63.00%), error:     11092.53711, best error:     10745.33398 on epoch:     0, epoch:   15/25, mu: 0.7976893187, learning_rate: 0.0001034617, weight_decay: 0.0000899043

epoch time: 1104.4062500000s, input_fired_time: 24.3760070801, output_fired_time: 8.3798961639, propagate_forward_time: 403.5234985352, propagate_backward_time: 654.0134887695, weight_update_time: 0.3416870236, other_time: 14.1134033203
epoch time: 85.7187576294s, input_fired_time: 1.5367523432, output_fired_time: 0.0000000000, propagate_forward_time: 81.7614517212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4205551147
validation[          , genome  6420] predictions:    6139/  10000 (61.39%), best:    6300/10000 (63.00%), error:     11193.67383, best error:     10745.33398 on epoch:     0, epoch:   16/25, mu: 0.8092939854, learning_rate: 0.0000964622, weight_decay: 0.0000848093

epoch time: 1104.8033447266s, input_fired_time: 24.3593711853, output_fired_time: 8.3981723785, propagate_forward_time: 404.1829833984, propagate_backward_time: 653.8795776367, weight_update_time: 0.3438149989, other_time: 13.9832153320
epoch time: 85.1557769775s, input_fired_time: 1.5212900639, output_fired_time: 0.0000000000, propagate_forward_time: 81.2007522583, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4337310791
validation[          , genome  6420] predictions:    6043/  10000 (60.43%), best:    6300/10000 (63.00%), error:     11356.43359, best error:     10745.33398 on epoch:     0, epoch:   17/25, mu: 0.8201984167, learning_rate: 0.0000899363, weight_decay: 0.0000800030

epoch time: 2072.6403808594s, input_fired_time: 24.7895488739, output_fired_time: 8.5071086884, propagate_forward_time: 1147.2640380859, propagate_backward_time: 877.9788208008, weight_update_time: 0.3409739733, other_time: 14.1008911133
epoch time: 84.9166946411s, input_fired_time: 1.4849658012, output_fired_time: 0.0000000000, propagate_forward_time: 81.0790328979, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3526992798
validation[          , genome  6420] predictions:    5987/  10000 (59.87%), best:    6300/10000 (63.00%), error:     11477.21094, best error:     10745.33398 on epoch:     0, epoch:   18/25, mu: 0.8304448128, learning_rate: 0.0000838519, weight_decay: 0.0000754691

epoch time: 1105.2115478516s, input_fired_time: 24.2950210571, output_fired_time: 8.3820829391, propagate_forward_time: 403.8898620605, propagate_backward_time: 654.7342529297, weight_update_time: 0.3392927945, other_time: 13.9102783203
epoch time: 85.2250518799s, input_fired_time: 1.5171647072, output_fired_time: 0.0000000000, propagate_forward_time: 81.3064270020, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4014587402
validation[          , genome  6420] predictions:    5820/  10000 (58.20%), best:    6300/10000 (63.00%), error:     12490.20508, best error:     10745.33398 on epoch:     0, epoch:   19/25, mu: 0.8400729299, learning_rate: 0.0000781790, weight_decay: 0.0000711921

epoch time: 1104.6804199219s, input_fired_time: 24.2314510345, output_fired_time: 8.4227952957, propagate_forward_time: 403.8464660645, propagate_backward_time: 654.2628173828, weight_update_time: 0.3424763680, other_time: 13.9168701172
epoch time: 85.3168182373s, input_fired_time: 1.5179492235, output_fired_time: 0.0000000000, propagate_forward_time: 81.3606185913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4382476807
validation[          , genome  6420] predictions:    6331/  10000 (63.31%), best:    6331/10000 (63.31%), error:     10672.19531, best error:     10672.19531 on epoch:    20, epoch:   20/25, mu: 0.8491200209, learning_rate: 0.0000728900, weight_decay: 0.0000671575

epoch time: 1267.1717529297s, input_fired_time: 28.1694526672, output_fired_time: 9.7182855606, propagate_forward_time: 462.0742492676, propagate_backward_time: 750.4768066406, weight_update_time: 0.4439696968, other_time: 16.7330322266
epoch time: 118.9686050415s, input_fired_time: 2.1492428780, output_fired_time: 0.0000000000, propagate_forward_time: 113.3978500366, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4215087891
validation[          , genome  6420] predictions:    6347/  10000 (63.47%), best:    6331/10000 (63.31%), error:     10710.16602, best error:     10672.19531 on epoch:    20, epoch:   21/25, mu: 0.8576211929, learning_rate: 0.0000679588, weight_decay: 0.0000633516

epoch time: 1284.5424804688s, input_fired_time: 28.4116172791, output_fired_time: 9.7870788574, propagate_forward_time: 469.7399902344, propagate_backward_time: 759.8876953125, weight_update_time: 0.4206628799, other_time: 16.7160644531
epoch time: 114.5375747681s, input_fired_time: 2.0532271862, output_fired_time: 0.0000000000, propagate_forward_time: 109.3427810669, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1415634155
validation[          , genome  6420] predictions:    5971/  10000 (59.71%), best:    6331/10000 (63.31%), error:     11887.22266, best error:     10672.19531 on epoch:    20, epoch:   22/25, mu: 0.8656093478, learning_rate: 0.0000633612, weight_decay: 0.0000597614

epoch time: 1247.8629150391s, input_fired_time: 27.7945079803, output_fired_time: 9.5650444031, propagate_forward_time: 454.1326904297, propagate_backward_time: 740.4200439453, weight_update_time: 0.3937237561, other_time: 15.9505615234
epoch time: 85.5132598877s, input_fired_time: 1.5324307680, output_fired_time: 0.0000000000, propagate_forward_time: 81.5688018799, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4120254517
validation[          , genome  6420] predictions:    6372/  10000 (63.72%), best:    6372/10000 (63.72%), error:     10505.45020, best error:     10505.45020 on epoch:    23, epoch:   23/25, mu: 0.8731154799, learning_rate: 0.0000590746, weight_decay: 0.0000563746

epoch time: 1358.3438720703s, input_fired_time: 24.3979778290, output_fired_time: 8.4501724243, propagate_forward_time: 402.8356933594, propagate_backward_time: 908.3291015625, weight_update_time: 0.3415634632, other_time: 14.3309326172
epoch time: 87.5564727783s, input_fired_time: 1.5669584274, output_fired_time: 0.0000000000, propagate_forward_time: 83.4470825195, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5424346924
validation[          , genome  6420] predictions:    6242/  10000 (62.42%), best:    6372/10000 (63.72%), error:     10819.29199, best error:     10505.45020 on epoch:    23, epoch:   24/25, mu: 0.8801686764, learning_rate: 0.0000550781, weight_decay: 0.0000531797

epoch time: 1265.0554199219s, input_fired_time: 24.3465156555, output_fired_time: 8.3971052170, propagate_forward_time: 567.2137451172, propagate_backward_time: 651.0078735352, weight_update_time: 0.3399224281, other_time: 14.0901489258
epoch time: 85.7430953979s, input_fired_time: 1.5519713163, output_fired_time: 0.0000000000, propagate_forward_time: 81.5459747314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6451492310
validation[          , genome  6420] predictions:    6322/  10000 (63.22%), best:    6372/10000 (63.72%), error:     10756.32813, best error:     10505.45020 on epoch:    23, epoch:   25/25, mu: 0.8867962360, learning_rate: 0.0000513519, weight_decay: 0.0000501659

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 422.3007202148s, input_fired_time: 7.6203188896, output_fired_time: 0.0000000000, propagate_forward_time: 402.1973876953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.4830017090
best training[          , genome  6420] predictions:   34276/  50000 (68.55%), best:    6372/10000 (63.72%), error:     47146.27344, best error:     10505.45020 on epoch:    23, epoch:   26/25, mu: 0.8930239081, learning_rate: 0.0000478778, weight_decay: 0.0000473230
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 84.6220245361s, input_fired_time: 1.5011011362, output_fired_time: 0.0000000000, propagate_forward_time: 80.8188781738, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3020477295
   testing[          , genome  6420] predictions:    6372/  10000 (63.72%), best:    6372/10000 (63.72%), error:     10505.45020, best error:     10505.45020 on epoch:    23, epoch:   26/25, mu: 0.8930239081, learning_rate: 0.0000478778, weight_decay: 0.0000473230
backpropagation finished successfully!
01:08:34 (14032): called boinc_finish(0)

</stderr_txt>
]]>