Nameexact_genome_1542189904_173_5894_3
Workunit4088642
Created25 Dec 2018, 6:28:53 UTC
Sent25 Dec 2018, 6:29:03 UTC
Report deadline4 Feb 2019, 0:04:10 UTC
Received26 Dec 2018, 0:26:24 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID23692
Run time17 hours 24 min 11 sec
CPU time17 hours 4 min 37 sec
Validate stateValid
Credit10,183.18
Device peak FLOPS4.14 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size602.90 MB
Peak swap size622.29 MB
Peak disk usage14.41 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5894.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5894_3_r516788156_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542189904_173_5894.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.552469
read mu: 0.552469
read mu_delta: 0.972663
read initial_learning_rate: 0.00139052
read learning_rate: 0.00139052
read learning_rate_delta: 0.913062
read initial_weight_decay: 0.000309236
read weight_decay: 0.000309236
read weight_decay_delta: 0.912757
read batch_size: 124
read epsilon: 1e-07
read alpha: 0.132313
read input_dropout_probability: 0.00170485
read hidden_dropout_probability: 0.0651436
read velocity_reset: 1817
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5894
read normal distribution: '0 -0.352976232767105 0.363866001367569'
generator_str: '800114620'
read generator: 800114620
read generated_by_map:
1 add_edge 1
reading 111 nodes.
reading 474 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 2131304473
post shuffle 1: 1326055045
epoch time: 2332.35205078125s, input_fired_time: 24.1889019012451, output_fired_time: 13.0033378601074, propagate_forward_time: 935.993896484375, propagate_backward_time: 1343.78332519531, weight_update_time: 0.440274477005005, other_time: 15.3824462890625
epoch time: 188.171768188477s, input_fired_time: 1.81748974323273, output_fired_time: 0, propagate_forward_time: 183.639953613281, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.71432495117188
validation[          , genome  5894] predictions:    5903/  10000 (59.03%), best:    5903/10000 (59.03%), error:     11947.72656, best error:     11947.72656 on epoch:     0, epoch:    0/25, mu: 0.5524690747, learning_rate: 0.0013905196, weight_decay: 0.0003092362

epoch time: 2366.1635742188s, input_fired_time: 24.3177165985, output_fired_time: 13.2419042587, propagate_forward_time: 953.2086181641, propagate_backward_time: 1359.8162841797, weight_update_time: 0.4655549228, other_time: 15.5791015625
epoch time: 189.0600891113s, input_fired_time: 1.8436323404, output_fired_time: 0.0000000000, propagate_forward_time: 184.6701660156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5462951660
validation[          , genome  5894] predictions:    5944/  10000 (59.44%), best:    5903/10000 (59.03%), error:     12023.36621, best error:     11947.72656 on epoch:     0, epoch:    1/25, mu: 0.5644299984, learning_rate: 0.0012696306, weight_decay: 0.0002822574

epoch time: 2224.3479003906s, input_fired_time: 22.5404281616, output_fired_time: 12.0203742981, propagate_forward_time: 883.7981567383, propagate_backward_time: 1292.3302001953, weight_update_time: 0.3994562626, other_time: 13.6588134766
epoch time: 176.0184936523s, input_fired_time: 1.7653257847, output_fired_time: 0.0000000000, propagate_forward_time: 171.9293518066, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3238220215
validation[          , genome  5894] predictions:    6078/  10000 (60.78%), best:    6078/10000 (60.78%), error:     11471.31445, best error:     11471.31445 on epoch:     2, epoch:    2/25, mu: 0.5760639906, learning_rate: 0.0011592516, weight_decay: 0.0002576323

epoch time: 2217.1437988281s, input_fired_time: 22.6747856140, output_fired_time: 11.9350786209, propagate_forward_time: 874.4096069336, propagate_backward_time: 1294.5710449219, weight_update_time: 0.3954732120, other_time: 13.5532226563
epoch time: 174.1616210938s, input_fired_time: 1.7411872149, output_fired_time: 0.0000000000, propagate_forward_time: 170.0840301514, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3364105225
validation[          , genome  5894] predictions:    6082/  10000 (60.82%), best:    6082/10000 (60.82%), error:     11271.63477, best error:     11271.63477 on epoch:     3, epoch:    3/25, mu: 0.5873799324, learning_rate: 0.0010584686, weight_decay: 0.0002351556

epoch time: 2132.7961425781s, input_fired_time: 21.9064292908, output_fired_time: 11.3107633591, propagate_forward_time: 828.0584106445, propagate_backward_time: 1258.0686035156, weight_update_time: 0.3791859448, other_time: 13.4519042969
epoch time: 171.7826538086s, input_fired_time: 1.7289232016, output_fired_time: 0.0000000000, propagate_forward_time: 167.6444549561, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4092712402
validation[          , genome  5894] predictions:    6020/  10000 (60.20%), best:    6082/10000 (60.82%), error:     11548.52441, best error:     11271.63477 on epoch:     3, epoch:    4/25, mu: 0.5983865261, learning_rate: 0.0009664475, weight_decay: 0.0002146399

epoch time: 2141.7844238281s, input_fired_time: 22.1571197510, output_fired_time: 11.4904851913, propagate_forward_time: 830.4985961914, propagate_backward_time: 1264.0605468750, weight_update_time: 0.3799047768, other_time: 13.5776367188
epoch time: 170.3078155518s, input_fired_time: 1.7356142998, output_fired_time: 0.0000000000, propagate_forward_time: 166.1831817627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3890228271
validation[          , genome  5894] predictions:    5942/  10000 (59.42%), best:    6082/10000 (60.82%), error:     11700.00684, best error:     11271.63477 on epoch:     3, epoch:    5/25, mu: 0.6090922356, learning_rate: 0.0008824265, weight_decay: 0.0001959140

epoch time: 2136.7580566406s, input_fired_time: 22.0329837799, output_fired_time: 11.3502912521, propagate_forward_time: 829.0252075195, propagate_backward_time: 1260.7587890625, weight_update_time: 0.3807217777, other_time: 13.5908203125
epoch time: 172.3188934326s, input_fired_time: 1.7423275709, output_fired_time: 0.0000000000, propagate_forward_time: 168.1790466309, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3975219727
validation[          , genome  5894] predictions:    6173/  10000 (61.73%), best:    6173/10000 (61.73%), error:     11124.66309, best error:     11124.66309 on epoch:     6, epoch:    6/25, mu: 0.6195052266, learning_rate: 0.0008057102, weight_decay: 0.0001788218

epoch time: 2140.4418945313s, input_fired_time: 22.0313377380, output_fired_time: 11.3461017609, propagate_forward_time: 831.5853271484, propagate_backward_time: 1261.9941406250, weight_update_time: 0.3859067261, other_time: 13.4849853516
epoch time: 170.3608245850s, input_fired_time: 1.7230844498, output_fired_time: 0.0000000000, propagate_forward_time: 166.2892150879, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3485260010
validation[          , genome  5894] predictions:    6308/  10000 (63.08%), best:    6308/10000 (63.08%), error:     10756.83203, best error:     10756.83203 on epoch:     7, epoch:    7/25, mu: 0.6296336055, learning_rate: 0.0007356633, weight_decay: 0.0001632208

epoch time: 2151.3630371094s, input_fired_time: 22.4142189026, output_fired_time: 11.6472530365, propagate_forward_time: 837.9246826172, propagate_backward_time: 1265.2606201172, weight_update_time: 0.3903460205, other_time: 14.1162109375
epoch time: 170.4771575928s, input_fired_time: 1.7435077429, output_fired_time: 0.0000000000, propagate_forward_time: 166.1963043213, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5373382568
validation[          , genome  5894] predictions:    5764/  10000 (57.64%), best:    6308/10000 (63.08%), error:     12408.73242, best error:     10756.83203 on epoch:     7, epoch:    8/25, mu: 0.6394850612, learning_rate: 0.0006717063, weight_decay: 0.0001489808

epoch time: 2136.1896972656s, input_fired_time: 22.1725425720, output_fired_time: 11.6321144104, propagate_forward_time: 834.3380126953, propagate_backward_time: 1254.0319824219, weight_update_time: 0.3870281875, other_time: 14.0150146484
epoch time: 172.5894470215s, input_fired_time: 1.7499495745, output_fired_time: 0.0000000000, propagate_forward_time: 168.3927917480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4467010498
validation[          , genome  5894] predictions:    6312/  10000 (63.12%), best:    6312/10000 (63.12%), error:     10725.52051, best error:     10725.52051 on epoch:     9, epoch:    9/25, mu: 0.6490672231, learning_rate: 0.0006133095, weight_decay: 0.0001359833

epoch time: 2149.5795898438s, input_fired_time: 22.3485603333, output_fired_time: 11.5576324463, propagate_forward_time: 837.0557250977, propagate_backward_time: 1264.5753173828, weight_update_time: 0.3852606118, other_time: 14.0423583984
epoch time: 170.1870880127s, input_fired_time: 1.7496962547, output_fired_time: 0.0000000000, propagate_forward_time: 165.9716644287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4657287598
validation[          , genome  5894] predictions:    6234/  10000 (62.34%), best:    6312/10000 (63.12%), error:     10953.90430, best error:     10725.52051 on epoch:     9, epoch:   10/25, mu: 0.6583874226, learning_rate: 0.0005599896, weight_decay: 0.0001241196

epoch time: 2119.4897460938s, input_fired_time: 22.0319404602, output_fired_time: 11.4161663055, propagate_forward_time: 824.5714111328, propagate_backward_time: 1247.7539062500, weight_update_time: 0.3843563795, other_time: 13.7161865234
epoch time: 172.0013732910s, input_fired_time: 1.7380938530, output_fired_time: 0.0000000000, propagate_forward_time: 167.9494018555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3138732910
validation[          , genome  5894] predictions:    6287/  10000 (62.87%), best:    6312/10000 (63.12%), error:     10851.86328, best error:     10725.52051 on epoch:     9, epoch:   11/25, mu: 0.6674528122, learning_rate: 0.0005113052, weight_decay: 0.0001132910

epoch time: 2145.6467285156s, input_fired_time: 22.1428375244, output_fired_time: 11.3464546204, propagate_forward_time: 834.2996215820, propagate_backward_time: 1264.6715087891, weight_update_time: 0.3750028610, other_time: 13.1864013672
epoch time: 170.2644653320s, input_fired_time: 1.7497502565, output_fired_time: 0.0000000000, propagate_forward_time: 166.2884826660, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2262268066
validation[          , genome  5894] predictions:    6310/  10000 (63.10%), best:    6310/10000 (63.10%), error:     10663.89648, best error:     10663.89648 on epoch:    12, epoch:   12/25, mu: 0.6762704253, learning_rate: 0.0004668534, weight_decay: 0.0001034071

epoch time: 2162.7182617188s, input_fired_time: 22.1677017212, output_fired_time: 11.3942308426, propagate_forward_time: 839.2517700195, propagate_backward_time: 1276.7321777344, weight_update_time: 0.3772486746, other_time: 13.1723632813
epoch time: 175.9053802490s, input_fired_time: 1.7903267145, output_fired_time: 0.0000000000, propagate_forward_time: 171.7449340820, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3701171875
validation[          , genome  5894] predictions:    6072/  10000 (60.72%), best:    6310/10000 (63.10%), error:     11375.76074, best error:     10663.89648 on epoch:    12, epoch:   13/25, mu: 0.6848469973, learning_rate: 0.0004262661, weight_decay: 0.0000943855

epoch time: 2159.2243652344s, input_fired_time: 22.2373542786, output_fired_time: 11.4447326660, propagate_forward_time: 839.0018310547, propagate_backward_time: 1273.3061523438, weight_update_time: 0.3756170273, other_time: 13.2342529297
epoch time: 170.9958648682s, input_fired_time: 1.7390425205, output_fired_time: 0.0000000000, propagate_forward_time: 166.9225921631, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3342285156
validation[          , genome  5894] predictions:    6321/  10000 (63.21%), best:    6310/10000 (63.10%), error:     10722.48535, best error:     10663.89648 on epoch:    12, epoch:   14/25, mu: 0.6931890845, learning_rate: 0.0003892074, weight_decay: 0.0000861510

epoch time: 2184.9541015625s, input_fired_time: 22.5425834656, output_fired_time: 11.5540609360, propagate_forward_time: 849.5850219727, propagate_backward_time: 1287.7843017578, weight_update_time: 0.3823515177, other_time: 13.4884033203
epoch time: 174.3932647705s, input_fired_time: 1.7803519964, output_fired_time: 0.0000000000, propagate_forward_time: 170.2718353271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3410797119
validation[          , genome  5894] predictions:    6358/  10000 (63.58%), best:    6358/10000 (63.58%), error:     10632.60352, best error:     10632.60352 on epoch:    15, epoch:   15/25, mu: 0.7013031244, learning_rate: 0.0003553705, weight_decay: 0.0000786349

epoch time: 2154.8300781250s, input_fired_time: 22.0100212097, output_fired_time: 11.2849292755, propagate_forward_time: 834.8085937500, propagate_backward_time: 1273.6627197266, weight_update_time: 0.3739318848, other_time: 13.0638427734
epoch time: 174.6560821533s, input_fired_time: 1.7596026659, output_fired_time: 0.0000000000, propagate_forward_time: 170.5901794434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3063049316
validation[          , genome  5894] predictions:    6253/  10000 (62.53%), best:    6358/10000 (63.58%), error:     10865.25195, best error:     10632.60352 on epoch:    15, epoch:   16/25, mu: 0.7091953158, learning_rate: 0.0003244753, weight_decay: 0.0000717746

epoch time: 2167.0412597656s, input_fired_time: 22.2556896210, output_fired_time: 11.3467750549, propagate_forward_time: 839.8200073242, propagate_backward_time: 1280.6096191406, weight_update_time: 0.3768335283, other_time: 13.0092773438
epoch time: 170.4274597168s, input_fired_time: 1.7400897741, output_fired_time: 0.0000000000, propagate_forward_time: 166.4329223633, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2544403076
validation[          , genome  5894] predictions:    6385/  10000 (63.85%), best:    6385/10000 (63.85%), error:     10574.19727, best error:     10574.19727 on epoch:    17, epoch:   17/25, mu: 0.7168717980, learning_rate: 0.0002962661, weight_decay: 0.0000655127

epoch time: 2150.5400390625s, input_fired_time: 21.9965591431, output_fired_time: 11.2954187393, propagate_forward_time: 832.5315551758, propagate_backward_time: 1271.6623535156, weight_update_time: 0.4011449516, other_time: 13.0541992188
epoch time: 170.9408721924s, input_fired_time: 1.7585071325, output_fired_time: 0.0000000000, propagate_forward_time: 166.9408264160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2415313721
validation[          , genome  5894] predictions:    6343/  10000 (63.43%), best:    6385/10000 (63.85%), error:     10625.92578, best error:     10574.19727 on epoch:    17, epoch:   18/25, mu: 0.7243384123, learning_rate: 0.0002705093, weight_decay: 0.0000597972

epoch time: 2154.4943847656s, input_fired_time: 22.0770454407, output_fired_time: 11.2870378494, propagate_forward_time: 834.5697021484, propagate_backward_time: 1273.6175537109, weight_update_time: 0.3721957803, other_time: 12.9428710938
epoch time: 169.6278686523s, input_fired_time: 1.7412072420, output_fired_time: 0.0000000000, propagate_forward_time: 165.6197814941, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2668762207
validation[          , genome  5894] predictions:    6482/  10000 (64.82%), best:    6482/10000 (64.82%), error:     10373.13867, best error:     10373.13867 on epoch:    19, epoch:   19/25, mu: 0.7316008806, learning_rate: 0.0002469918, weight_decay: 0.0000545803

epoch time: 2152.5739746094s, input_fired_time: 22.0362548828, output_fired_time: 11.2340316772, propagate_forward_time: 833.6414184570, propagate_backward_time: 1272.6530761719, weight_update_time: 0.3724257946, other_time: 13.0090332031
epoch time: 171.6850280762s, input_fired_time: 1.7612621784, output_fired_time: 0.0000000000, propagate_forward_time: 167.6562652588, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2675018311
validation[          , genome  5894] predictions:    6475/  10000 (64.75%), best:    6475/10000 (64.75%), error:     10230.73438, best error:     10230.73438 on epoch:    20, epoch:   20/25, mu: 0.7386648059, learning_rate: 0.0002255188, weight_decay: 0.0000498185

epoch time: 2154.9521484375s, input_fired_time: 22.1036834717, output_fired_time: 11.3614835739, propagate_forward_time: 837.1706542969, propagate_backward_time: 1271.3388671875, weight_update_time: 0.3724838197, other_time: 12.9772949219
epoch time: 185.1564178467s, input_fired_time: 1.8842902184, output_fired_time: 0.0000000000, propagate_forward_time: 180.8146820068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4574432373
validation[          , genome  5894] predictions:    6429/  10000 (64.29%), best:    6475/10000 (64.75%), error:     10352.94434, best error:     10230.73438 on epoch:    20, epoch:   21/25, mu: 0.7455356717, learning_rate: 0.0002059127, weight_decay: 0.0000454722

epoch time: 2303.3930664063s, input_fired_time: 23.5202636719, output_fired_time: 12.6532773972, propagate_forward_time: 926.0231933594, propagate_backward_time: 1326.8699951172, weight_update_time: 0.4305329025, other_time: 14.3262939453
epoch time: 195.2371826172s, input_fired_time: 1.9094709158, output_fired_time: 0.0000000000, propagate_forward_time: 190.7744750977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5532379150
validation[          , genome  5894] predictions:    6423/  10000 (64.23%), best:    6475/10000 (64.75%), error:     10353.29688, best error:     10230.73438 on epoch:    20, epoch:   22/25, mu: 0.7522186637, learning_rate: 0.0001880110, weight_decay: 0.0000415050

epoch time: 2285.1713867188s, input_fired_time: 23.3687572479, output_fired_time: 12.5971479416, propagate_forward_time: 925.3077392578, propagate_backward_time: 1309.6962890625, weight_update_time: 0.4285422564, other_time: 14.2015380859
epoch time: 192.6620330811s, input_fired_time: 1.8606441021, output_fired_time: 0.0000000000, propagate_forward_time: 188.2857055664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5156860352
validation[          , genome  5894] predictions:    6485/  10000 (64.85%), best:    6485/10000 (64.85%), error:     10229.22852, best error:     10229.22852 on epoch:    23, epoch:   23/25, mu: 0.7587189674, learning_rate: 0.0001716657, weight_decay: 0.0000378840

epoch time: 2393.6645507813s, input_fired_time: 23.0149974823, output_fired_time: 12.5813093185, propagate_forward_time: 928.0198974609, propagate_backward_time: 1415.9034423828, weight_update_time: 0.4297513366, other_time: 14.1450195313
epoch time: 190.2683868408s, input_fired_time: 1.8280836344, output_fired_time: 0.0000000000, propagate_forward_time: 185.9592437744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4810638428
validation[          , genome  5894] predictions:    6518/  10000 (65.18%), best:    6518/10000 (65.18%), error:     10135.49316, best error:     10135.49316 on epoch:    24, epoch:   24/25, mu: 0.7650415897, learning_rate: 0.0001567415, weight_decay: 0.0000345789

epoch time: 2250.4665527344s, input_fired_time: 23.0900726318, output_fired_time: 12.4560756683, propagate_forward_time: 905.1388549805, propagate_backward_time: 1295.0141601563, weight_update_time: 0.4290522635, other_time: 14.7673339844
epoch time: 176.5276947021s, input_fired_time: 1.8591880798, output_fired_time: 0.0000000000, propagate_forward_time: 171.8935852051, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7749176025
validation[          , genome  5894] predictions:    6570/  10000 (65.70%), best:    6570/10000 (65.70%), error:     10075.48242, best error:     10075.48242 on epoch:    25, epoch:   25/25, mu: 0.7711913586, learning_rate: 0.0001431147, weight_decay: 0.0000315621

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 871.5749511719s, input_fired_time: 9.3066787720, output_fired_time: 0.0000000000, propagate_forward_time: 848.3922729492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.8759765625
best training[          , genome  5894] predictions:   36394/  50000 (72.79%), best:    6570/10000 (65.70%), error:     40859.10156, best error:     10075.48242 on epoch:    25, epoch:   26/25, mu: 0.7771729827, learning_rate: 0.0001306726, weight_decay: 0.0000288085
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 174.2310333252s, input_fired_time: 1.8374557495, output_fired_time: 0.0000000000, propagate_forward_time: 169.6183624268, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7752227783
   testing[          , genome  5894] predictions:    6570/  10000 (65.70%), best:    6570/10000 (65.70%), error:     10075.48242, best error:     10075.48242 on epoch:    25, epoch:   26/25, mu: 0.7771729827, learning_rate: 0.0001306726, weight_decay: 0.0000288085
backpropagation finished successfully!
20:06:39 (12000): called boinc_finish(0)

</stderr_txt>
]]>