Nameexact_genome_1542277804_173_5983_1
Workunit4090847
Created15 Nov 2018, 15:37:57 UTC
Sent15 Nov 2018, 15:43:23 UTC
Report deadline20 Dec 2018, 21:06:23 UTC
Received17 Nov 2018, 8:29:37 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID79157
Run time9 hours 6 min 46 sec
CPU time9 hours 4 min 13 sec
Validate stateValid
Credit8,806.08
Device peak FLOPS4.43 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size591.82 MB
Peak swap size611.88 MB
Peak disk usage12.93 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542277804_173_5983.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542277804_173_5983_1_r627404261_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542277804_173_5983.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.543416
read mu: 0.543416
read mu_delta: 0.963154
read initial_learning_rate: 0.00182265
read learning_rate: 0.00182265
read learning_rate_delta: 0.916718
read initial_weight_decay: 0.000361656
read weight_decay: 0.000361656
read weight_decay_delta: 0.919515
read batch_size: 124
read epsilon: 1e-07
read alpha: 0.130789
read input_dropout_probability: 0.00162631
read hidden_dropout_probability: 0.0765417
read velocity_reset: 1838
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5983
read normal distribution: '1 0.618203222751617 -1.28266310691833'
generator_str: '1764113616'
read generator: 1764113616
read generated_by_map:
1 crossover 1
reading 100 nodes.
reading 412 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1298313630
post shuffle 1: 1804632905
epoch time: 1180.92822265625s, input_fired_time: 16.5986423492432, output_fired_time: 5.79672050476074, propagate_forward_time: 339.771301269531, propagate_backward_time: 810.909423828125, weight_update_time: 0.185880243778229, other_time: 7.85211181640625
epoch time: 70.3496932983398s, input_fired_time: 1.00113832950592, output_fired_time: 0, propagate_forward_time: 68.0053482055664, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.34320831298828
validation[          , genome  5983] predictions:    5728/  10000 (57.28%), best:    5728/10000 (57.28%), error:     12363.97363, best error:     12363.97363 on epoch:     0, epoch:    0/25, mu: 0.5434156656, learning_rate: 0.0018226462, weight_decay: 0.0003616558

epoch time: 1174.5332031250s, input_fired_time: 16.6222438812, output_fired_time: 5.7699799538, propagate_forward_time: 338.2559814453, propagate_backward_time: 805.3805541992, weight_update_time: 0.1847798228, other_time: 8.5044555664
epoch time: 69.8200683594s, input_fired_time: 0.9897193909, output_fired_time: 0.0000000000, propagate_forward_time: 67.5193786621, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3109741211
validation[          , genome  5983] predictions:    5729/  10000 (57.29%), best:    5729/10000 (57.29%), error:     12281.96875, best error:     12281.96875 on epoch:     1, epoch:    1/25, mu: 0.5598703027, learning_rate: 0.0016708530, weight_decay: 0.0003325480

epoch time: 1172.7658691406s, input_fired_time: 16.6356296539, output_fired_time: 5.7255573273, propagate_forward_time: 337.0234069824, propagate_backward_time: 804.6124877930, weight_update_time: 0.1824005097, other_time: 8.7687377930
epoch time: 70.0875320435s, input_fired_time: 1.0035407543, output_fired_time: 0.0000000000, propagate_forward_time: 67.4654464722, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6185455322
validation[          , genome  5983] predictions:    5366/  10000 (53.66%), best:    5729/10000 (57.29%), error:     13446.42578, best error:     12281.96875 on epoch:     1, epoch:    2/25, mu: 0.5757186413, learning_rate: 0.0015317013, weight_decay: 0.0003057829

epoch time: 1172.5445556641s, input_fired_time: 16.6510124207, output_fired_time: 5.7236156464, propagate_forward_time: 336.8529357910, propagate_backward_time: 804.8458251953, weight_update_time: 0.1838346869, other_time: 8.4711914063
epoch time: 69.7624664307s, input_fired_time: 1.0120732784, output_fired_time: 0.0000000000, propagate_forward_time: 67.4366607666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3137359619
validation[          , genome  5983] predictions:    5869/  10000 (58.69%), best:    5869/10000 (58.69%), error:     11736.18555, best error:     11736.18555 on epoch:     3, epoch:    3/25, mu: 0.5909830332, learning_rate: 0.0014041384, weight_decay: 0.0002811719

epoch time: 1173.7966308594s, input_fired_time: 16.7018260956, output_fired_time: 5.7469897270, propagate_forward_time: 337.1564025879, propagate_backward_time: 805.6026000977, weight_update_time: 0.1839223653, other_time: 8.5889282227
epoch time: 70.1178512573s, input_fired_time: 0.9997174144, output_fired_time: 0.0000000000, propagate_forward_time: 67.8058700562, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3122634888
validation[          , genome  5983] predictions:    5738/  10000 (57.38%), best:    5869/10000 (58.69%), error:     12413.90430, best error:     11736.18555 on epoch:     3, epoch:    4/25, mu: 0.6056849957, learning_rate: 0.0012871992, weight_decay: 0.0002585418

epoch time: 1171.3977050781s, input_fired_time: 16.6425533295, output_fired_time: 5.7060480118, propagate_forward_time: 336.5057678223, propagate_backward_time: 803.8300170898, weight_update_time: 0.1829620600, other_time: 8.7133178711
epoch time: 69.7386932373s, input_fired_time: 0.9986642003, output_fired_time: 0.0000000000, propagate_forward_time: 67.4258499146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3141784668
validation[          , genome  5983] predictions:    4554/  10000 (45.54%), best:    5869/10000 (58.69%), error:     19207.32813, best error:     11736.18555 on epoch:     3, epoch:    5/25, mu: 0.6198452711, learning_rate: 0.0011799990, weight_decay: 0.0002377331

epoch time: 1173.8312988281s, input_fired_time: 16.6687488556, output_fired_time: 5.7372393608, propagate_forward_time: 337.0142517090, propagate_backward_time: 805.3351440430, weight_update_time: 0.1824568063, other_time: 9.0759887695
epoch time: 70.8457183838s, input_fired_time: 1.0117886066, output_fired_time: 0.0000000000, propagate_forward_time: 67.7992172241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0347137451
validation[          , genome  5983] predictions:    5400/  10000 (54.00%), best:    5869/10000 (58.69%), error:     13754.60059, best error:     11736.18555 on epoch:     3, epoch:    6/25, mu: 0.6334838271, learning_rate: 0.0010817265, weight_decay: 0.0002185991

epoch time: 1174.4561767578s, input_fired_time: 16.6997261047, output_fired_time: 5.7310390472, propagate_forward_time: 337.2640380859, propagate_backward_time: 805.8796386719, weight_update_time: 0.1824553460, other_time: 8.8817138672
epoch time: 70.3112258911s, input_fired_time: 1.0141881704, output_fired_time: 0.0000000000, propagate_forward_time: 67.4756164551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8214187622
validation[          , genome  5983] predictions:    6006/  10000 (60.06%), best:    6006/10000 (60.06%), error:     11623.63672, best error:     11623.63672 on epoch:     7, epoch:    7/25, mu: 0.6466198564, learning_rate: 0.0009916384, weight_decay: 0.0002010052

epoch time: 1172.7900390625s, input_fired_time: 16.7083663940, output_fired_time: 5.7161850929, propagate_forward_time: 336.9902038574, propagate_backward_time: 804.5368652344, weight_update_time: 0.1826905757, other_time: 8.8383789063
epoch time: 70.2320175171s, input_fired_time: 1.0020153522, output_fired_time: 0.0000000000, propagate_forward_time: 67.4710617065, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7589416504
validation[          , genome  5983] predictions:    5636/  10000 (56.36%), best:    6006/10000 (60.06%), error:     12640.82324, best error:     11623.63672 on epoch:     7, epoch:    8/25, mu: 0.6592718959, learning_rate: 0.0009090530, weight_decay: 0.0001848273

epoch time: 1172.3331298828s, input_fired_time: 16.6899394989, output_fired_time: 5.7166199684, propagate_forward_time: 336.5323791504, propagate_backward_time: 804.5592041016, weight_update_time: 0.1828625351, other_time: 8.8349609375
epoch time: 69.9652709961s, input_fired_time: 1.0075503588, output_fired_time: 0.0000000000, propagate_forward_time: 67.4489059448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5088119507
validation[          , genome  5983] predictions:    6046/  10000 (60.46%), best:    6046/10000 (60.46%), error:     11545.67285, best error:     11545.67285 on epoch:     9, epoch:    9/25, mu: 0.6714577675, learning_rate: 0.0008333454, weight_decay: 0.0001699515

epoch time: 1172.8389892578s, input_fired_time: 16.6879577637, output_fired_time: 5.7245836258, propagate_forward_time: 336.6759033203, propagate_backward_time: 804.8457031250, weight_update_time: 0.1830469072, other_time: 8.9047851563
epoch time: 70.3929901123s, input_fired_time: 1.0247889757, output_fired_time: 0.0000000000, propagate_forward_time: 67.4753646851, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8928375244
validation[          , genome  5983] predictions:    5927/  10000 (59.27%), best:    6046/10000 (60.46%), error:     11734.33789, best error:     11545.67285 on epoch:     9, epoch:   10/25, mu: 0.6831946373, learning_rate: 0.0007639429, weight_decay: 0.0001562729

epoch time: 1172.3474121094s, input_fired_time: 16.7196254730, output_fired_time: 5.7186999321, propagate_forward_time: 336.8528747559, propagate_backward_time: 804.2097778320, weight_update_time: 0.1824786961, other_time: 8.8463745117
epoch time: 70.3388595581s, input_fired_time: 1.0211639404, output_fired_time: 0.0000000000, propagate_forward_time: 67.4676742554, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8500213623
validation[          , genome  5983] predictions:    6118/  10000 (61.18%), best:    6118/10000 (61.18%), error:     11259.36719, best error:     11259.36719 on epoch:    11, epoch:   11/25, mu: 0.6944990158, learning_rate: 0.0007003204, weight_decay: 0.0001436953

epoch time: 1174.2917480469s, input_fired_time: 16.7456207275, output_fired_time: 5.7157168388, propagate_forward_time: 337.1983337402, propagate_backward_time: 805.4998779297, weight_update_time: 0.1825935543, other_time: 9.1322021484
epoch time: 70.0202636719s, input_fired_time: 1.0147159100, output_fired_time: 0.0000000000, propagate_forward_time: 67.4301605225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5753860474
validation[          , genome  5983] predictions:    5743/  10000 (57.43%), best:    6118/10000 (61.18%), error:     12541.41016, best error:     11259.36719 on epoch:    11, epoch:   12/25, mu: 0.7053868771, learning_rate: 0.0006419964, weight_decay: 0.0001321300

epoch time: 1172.2840576172s, input_fired_time: 16.7405815125, output_fired_time: 5.7178215981, propagate_forward_time: 336.8758544922, propagate_backward_time: 804.0920410156, weight_update_time: 0.1821818650, other_time: 8.8577880859
epoch time: 70.5959472656s, input_fired_time: 1.0333912373, output_fired_time: 0.0000000000, propagate_forward_time: 67.5833282471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9792251587
validation[          , genome  5983] predictions:    6230/  10000 (62.30%), best:    6230/10000 (62.30%), error:     10964.58984, best error:     10964.58984 on epoch:    13, epoch:   13/25, mu: 0.7158735991, learning_rate: 0.0005885297, weight_decay: 0.0001214955

epoch time: 1172.6673583984s, input_fired_time: 16.7480297089, output_fired_time: 5.7225041389, propagate_forward_time: 336.8587341309, propagate_backward_time: 804.6749267578, weight_update_time: 0.1824270487, other_time: 8.6630859375
epoch time: 69.8089065552s, input_fired_time: 1.0297420025, output_fired_time: 0.0000000000, propagate_forward_time: 67.4711761475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3079910278
validation[          , genome  5983] predictions:    6204/  10000 (62.04%), best:    6230/10000 (62.30%), error:     10971.75586, best error:     10964.58984 on epoch:    13, epoch:   14/25, mu: 0.7259739041, learning_rate: 0.0005395159, weight_decay: 0.0001117169

epoch time: 1172.4456787109s, input_fired_time: 16.7613849640, output_fired_time: 5.7279324532, propagate_forward_time: 336.7556457520, propagate_backward_time: 804.5167846680, weight_update_time: 0.1824542582, other_time: 8.6840209961
epoch time: 69.7262268066s, input_fired_time: 1.0298364162, output_fired_time: 0.0000000000, propagate_forward_time: 67.3864288330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3099594116
validation[          , genome  5983] predictions:    6268/  10000 (62.68%), best:    6268/10000 (62.68%), error:     10729.35938, best error:     10729.35938 on epoch:    15, epoch:   15/25, mu: 0.7357020974, learning_rate: 0.0004945840, weight_decay: 0.0001027254

epoch time: 1172.3692626953s, input_fired_time: 16.7582321167, output_fired_time: 5.7119054794, propagate_forward_time: 336.6605834961, propagate_backward_time: 804.3857421875, weight_update_time: 0.1834903210, other_time: 8.8528442383
epoch time: 69.8578491211s, input_fired_time: 1.0347702503, output_fired_time: 0.0000000000, propagate_forward_time: 67.5146026611, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3084793091
validation[          , genome  5983] predictions:    6089/  10000 (60.89%), best:    6268/10000 (62.68%), error:     11082.90527, best error:     10729.35938 on epoch:    15, epoch:   16/25, mu: 0.7450718284, learning_rate: 0.0004533942, weight_decay: 0.0000944575

epoch time: 1172.1212158203s, input_fired_time: 16.7825927734, output_fired_time: 5.7193331718, propagate_forward_time: 336.7939453125, propagate_backward_time: 804.0496215820, weight_update_time: 0.1822239012, other_time: 8.7756958008
epoch time: 69.8112258911s, input_fired_time: 1.0375809669, output_fired_time: 0.0000000000, propagate_forward_time: 67.4661331177, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3075103760
validation[          , genome  5983] predictions:    6323/  10000 (63.23%), best:    6323/10000 (63.23%), error:     10624.86719, best error:     10624.86719 on epoch:    17, epoch:   17/25, mu: 0.7540963292, learning_rate: 0.0004156347, weight_decay: 0.0000868551

epoch time: 1172.0899658203s, input_fired_time: 16.7752819061, output_fired_time: 5.7160620689, propagate_forward_time: 336.5870971680, propagate_backward_time: 804.1726684570, weight_update_time: 0.1824517697, other_time: 8.8388671875
epoch time: 69.8479843140s, input_fired_time: 1.0393679142, output_fired_time: 0.0000000000, propagate_forward_time: 67.4965896606, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3120269775
validation[          , genome  5983] predictions:    6305/  10000 (63.05%), best:    6323/10000 (63.23%), error:     10641.29688, best error:     10624.86719 on epoch:    17, epoch:   18/25, mu: 0.7627882957, learning_rate: 0.0003810199, weight_decay: 0.0000798646

epoch time: 1173.8326416016s, input_fired_time: 16.8118648529, output_fired_time: 5.7232532501, propagate_forward_time: 337.1905212402, propagate_backward_time: 805.3078002930, weight_update_time: 0.1833465397, other_time: 8.7991333008
epoch time: 69.7826385498s, input_fired_time: 1.0381373167, output_fired_time: 0.0000000000, propagate_forward_time: 67.4353256226, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3091735840
validation[          , genome  5983] predictions:    6422/  10000 (64.22%), best:    6422/10000 (64.22%), error:     10432.32715, best error:     10432.32715 on epoch:    19, epoch:   19/25, mu: 0.7711600065, learning_rate: 0.0003492878, weight_decay: 0.0000734367

epoch time: 1172.2003173828s, input_fired_time: 16.8268985748, output_fired_time: 5.7164082527, propagate_forward_time: 336.7587585449, propagate_backward_time: 804.0633544922, weight_update_time: 0.1826143116, other_time: 8.8348388672
epoch time: 69.7162628174s, input_fired_time: 1.0410192013, output_fired_time: 0.0000000000, propagate_forward_time: 67.3683013916, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3069458008
validation[          , genome  5983] predictions:    6113/  10000 (61.13%), best:    6422/10000 (64.22%), error:     11146.94434, best error:     10432.32715 on epoch:    19, epoch:   20/25, mu: 0.7792232633, learning_rate: 0.0003201985, weight_decay: 0.0000675261

epoch time: 1172.6837158203s, input_fired_time: 16.8173542023, output_fired_time: 5.7320094109, propagate_forward_time: 336.8662414551, propagate_backward_time: 804.5106201172, weight_update_time: 0.1825250983, other_time: 8.7574462891
epoch time: 69.8469619751s, input_fired_time: 1.0448144674, output_fired_time: 0.0000000000, propagate_forward_time: 67.4920730591, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3100738525
validation[          , genome  5983] predictions:    6389/  10000 (63.89%), best:    6422/10000 (64.22%), error:     10519.36035, best error:     10432.32715 on epoch:    19, epoch:   21/25, mu: 0.7869894505, learning_rate: 0.0002935318, weight_decay: 0.0000620913

epoch time: 1171.5887451172s, input_fired_time: 16.8226280212, output_fired_time: 5.7077493668, propagate_forward_time: 336.6722412109, propagate_backward_time: 803.7401733398, weight_update_time: 0.1825895458, other_time: 8.6459350586
epoch time: 69.8621292114s, input_fired_time: 1.0427346230, output_fired_time: 0.0000000000, propagate_forward_time: 67.5063705444, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3130264282
validation[          , genome  5983] predictions:    6254/  10000 (62.54%), best:    6422/10000 (64.22%), error:     10877.34766, best error:     10432.32715 on epoch:    19, epoch:   22/25, mu: 0.7944694757, learning_rate: 0.0002690860, weight_decay: 0.0000570939

epoch time: 1171.9226074219s, input_fired_time: 16.8332595825, output_fired_time: 5.7098841667, propagate_forward_time: 336.7403564453, propagate_backward_time: 803.9618530273, weight_update_time: 0.1831890643, other_time: 8.6773071289
epoch time: 69.8443908691s, input_fired_time: 1.0526405573, output_fired_time: 0.0000000000, propagate_forward_time: 67.4720382690, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3197097778
validation[          , genome  5983] predictions:    6423/  10000 (64.23%), best:    6422/10000 (64.22%), error:     10456.66602, best error:     10432.32715 on epoch:    19, epoch:   23/25, mu: 0.8016738892, learning_rate: 0.0002466760, weight_decay: 0.0000524987

epoch time: 1172.6901855469s, input_fired_time: 16.8355922699, output_fired_time: 5.7076807022, propagate_forward_time: 336.7394104004, propagate_backward_time: 804.7297363281, weight_update_time: 0.1833621264, other_time: 8.6778564453
epoch time: 69.7911300659s, input_fired_time: 1.0473731756, output_fired_time: 0.0000000000, propagate_forward_time: 67.4305572510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3132019043
validation[          , genome  5983] predictions:    6449/  10000 (64.49%), best:    6449/10000 (64.49%), error:     10307.77637, best error:     10307.77637 on epoch:    24, epoch:   24/25, mu: 0.8086128831, learning_rate: 0.0002261324, weight_decay: 0.0000482733

epoch time: 1172.3585205078s, input_fired_time: 16.8350658417, output_fired_time: 5.7168073654, propagate_forward_time: 336.5480041504, propagate_backward_time: 804.5764160156, weight_update_time: 0.1826095581, other_time: 8.6822509766
epoch time: 70.2022247314s, input_fired_time: 1.0479916334, output_fired_time: 0.0000000000, propagate_forward_time: 67.3990325928, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7552032471
validation[          , genome  5983] predictions:    6405/  10000 (64.05%), best:    6449/10000 (64.49%), error:     10428.93555, best error:     10307.77637 on epoch:    24, epoch:   25/25, mu: 0.8152961731, learning_rate: 0.0002072996, weight_decay: 0.0000443880

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 348.6755981445s, input_fired_time: 5.2513771057, output_fired_time: 0.0000000000, propagate_forward_time: 336.6473083496, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.7769165039
best training[          , genome  5983] predictions:   35676/  50000 (71.35%), best:    6449/10000 (64.49%), error:     42604.80078, best error:     10307.77637 on epoch:    24, epoch:   26/25, mu: 0.8217332363, learning_rate: 0.0001900354, weight_decay: 0.0000408155
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 70.5600204468s, input_fired_time: 1.0502848625, output_fired_time: 0.0000000000, propagate_forward_time: 67.4360351563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0736999512
   testing[          , genome  5983] predictions:    6449/  10000 (64.49%), best:    6449/10000 (64.49%), error:     10307.77637, best error:     10307.77637 on epoch:    24, epoch:   26/25, mu: 0.8217332363, learning_rate: 0.0001900354, weight_decay: 0.0000408155
backpropagation finished successfully!
02:28:56 (5688): called boinc_finish(0)

</stderr_txt>
]]>