Nameexact_genome_1542383404_173_6105_3
Workunit4093729
Created19 Dec 2018, 4:17:12 UTC
Sent19 Dec 2018, 4:17:15 UTC
Report deadline18 Jan 2019, 18:14:02 UTC
Received19 Dec 2018, 13:47:41 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID73073
Run time7 hours 55 min 58 sec
CPU time7 hours 17 min 43 sec
Validate stateValid
Credit7,645.28
Device peak FLOPS4.61 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size582.18 MB
Peak swap size601.40 MB
Peak disk usage12.41 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542383404_173_6105.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542383404_173_6105_3_r1915604539_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542383404_173_6105.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.550962
read mu: 0.550962
read mu_delta: 0.971381
read initial_learning_rate: 0.000529685
read learning_rate: 0.000529685
read learning_rate_delta: 0.914947
read initial_weight_decay: 0.000318221
read weight_decay: 0.000318221
read weight_decay_delta: 0.913849
read batch_size: 122
read epsilon: 1e-07
read alpha: 0.128183
read input_dropout_probability: 0.00170002
read hidden_dropout_probability: 0.064427
read velocity_reset: 1817
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6105
read normal distribution: '1 0 0'
generator_str: '1562426876'
read generator: 1562426876
read generated_by_map:
1 disable_edge 1
reading 90 nodes.
reading 376 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 278469416
post shuffle 1: 176720415
epoch time: 1044.79516601563s, input_fired_time: 16.5823822021484, output_fired_time: 5.53638219833374, propagate_forward_time: 382.836059570313, propagate_backward_time: 629.003051757813, weight_update_time: 0.214207768440247, other_time: 10.8372802734375
epoch time: 81.4542083740234s, input_fired_time: 1.00388896465302, output_fired_time: 0, propagate_forward_time: 78.5289306640625, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.92138671875
validation[          , genome  6105] predictions:    6249/  10000 (62.49%), best:    6249/10000 (62.49%), error:     10763.51660, best error:     10763.51660 on epoch:     0, epoch:    0/25, mu: 0.5509619117, learning_rate: 0.0005296853, weight_decay: 0.0003182210

epoch time: 1019.1957397461s, input_fired_time: 15.9931850433, output_fired_time: 5.5063791275, propagate_forward_time: 378.1251831055, propagate_backward_time: 607.0502929688, weight_update_time: 0.2105617374, other_time: 12.5206909180
epoch time: 75.9923400879s, input_fired_time: 1.9310007095, output_fired_time: 0.0000000000, propagate_forward_time: 72.3290405273, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7322998047
validation[          , genome  6105] predictions:    6313/  10000 (63.13%), best:    6313/10000 (63.13%), error:     10588.98438, best error:     10588.98438 on epoch:     1, epoch:    1/25, mu: 0.5635267496, learning_rate: 0.0004846338, weight_decay: 0.0002908058

epoch time: 1008.4181518555s, input_fired_time: 16.8868541718, output_fired_time: 5.3983511925, propagate_forward_time: 372.5395507813, propagate_backward_time: 601.0620727539, weight_update_time: 0.2049444765, other_time: 12.5313110352
epoch time: 81.2576828003s, input_fired_time: 0.9971286058, output_fired_time: 0.0000000000, propagate_forward_time: 78.4122619629, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8482894897
validation[          , genome  6105] predictions:    6261/  10000 (62.61%), best:    6313/10000 (63.13%), error:     10853.82520, best error:     10588.98438 on epoch:     1, epoch:    2/25, mu: 0.5757319927, learning_rate: 0.0004434141, weight_decay: 0.0002657525

epoch time: 1080.0292968750s, input_fired_time: 16.8065719604, output_fired_time: 5.7522301674, propagate_forward_time: 395.0896911621, propagate_backward_time: 649.0630493164, weight_update_time: 0.2470940650, other_time: 13.3178100586
epoch time: 80.9336395264s, input_fired_time: 1.0043053627, output_fired_time: 0.0000000000, propagate_forward_time: 78.1180267334, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8113098145
validation[          , genome  6105] predictions:    6286/  10000 (62.86%), best:    6313/10000 (63.13%), error:     10799.01270, best error:     10588.98438 on epoch:     1, epoch:    3/25, mu: 0.5875879526, learning_rate: 0.0004057002, weight_decay: 0.0002428576

epoch time: 1066.5549316406s, input_fired_time: 16.5302352905, output_fired_time: 5.7640557289, propagate_forward_time: 395.1693725586, propagate_backward_time: 637.9892578125, weight_update_time: 0.2281841636, other_time: 11.1019897461
epoch time: 89.6846771240s, input_fired_time: 1.1381127834, output_fired_time: 0.0000000000, propagate_forward_time: 86.5025939941, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0439682007
validation[          , genome  6105] predictions:    6274/  10000 (62.74%), best:    6313/10000 (63.13%), error:     10847.38281, best error:     10588.98438 on epoch:     1, epoch:    4/25, mu: 0.5991045833, learning_rate: 0.0003711940, weight_decay: 0.0002219350

epoch time: 1030.9188232422s, input_fired_time: 16.2030582428, output_fired_time: 5.5629019737, propagate_forward_time: 381.6643676758, propagate_backward_time: 616.6580810547, weight_update_time: 0.2148831636, other_time: 10.8303833008
epoch time: 75.0806427002s, input_fired_time: 0.9202110171, output_fired_time: 0.0000000000, propagate_forward_time: 72.3437652588, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8166656494
validation[          , genome  6105] predictions:    6296/  10000 (62.96%), best:    6313/10000 (63.13%), error:     10747.95117, best error:     10588.98438 on epoch:     1, epoch:    5/25, mu: 0.6102916598, learning_rate: 0.0003396227, weight_decay: 0.0002028150

epoch time: 1007.6403808594s, input_fired_time: 14.8863554001, output_fired_time: 5.4257950783, propagate_forward_time: 372.6308898926, propagate_backward_time: 604.0565795898, weight_update_time: 0.2181618065, other_time: 10.6408081055
epoch time: 82.3116073608s, input_fired_time: 0.9918089509, output_fired_time: 0.0000000000, propagate_forward_time: 79.1572875977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1625137329
validation[          , genome  6105] predictions:    5992/  10000 (59.92%), best:    6313/10000 (63.13%), error:     11650.09766, best error:     10588.98438 on epoch:     1, epoch:    6/25, mu: 0.6211585402, learning_rate: 0.0003107367, weight_decay: 0.0001853422

epoch time: 1099.3748779297s, input_fired_time: 16.3483505249, output_fired_time: 6.9958376884, propagate_forward_time: 399.8182373047, propagate_backward_time: 664.4891357422, weight_update_time: 0.2179411650, other_time: 11.7232666016
epoch time: 84.3074874878s, input_fired_time: 1.0102537870, output_fired_time: 0.0000000000, propagate_forward_time: 81.3231430054, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9740905762
validation[          , genome  6105] predictions:    6117/  10000 (61.17%), best:    6313/10000 (63.13%), error:     11166.85938, best error:     10588.98438 on epoch:     1, epoch:    7/25, mu: 0.6317144036, learning_rate: 0.0002843074, weight_decay: 0.0001693748

epoch time: 1169.2130126953s, input_fired_time: 18.3663043976, output_fired_time: 8.3116426468, propagate_forward_time: 427.5028686523, propagate_backward_time: 701.6752929688, weight_update_time: 0.2689520717, other_time: 13.3568725586
epoch time: 80.7849121094s, input_fired_time: 0.9591078162, output_fired_time: 0.0000000000, propagate_forward_time: 78.0541152954, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7716903687
validation[          , genome  6105] predictions:    6200/  10000 (62.00%), best:    6313/10000 (63.13%), error:     10949.33691, best error:     10588.98438 on epoch:     1, epoch:    8/25, mu: 0.6419681907, learning_rate: 0.0002601261, weight_decay: 0.0001547829

epoch time: 1030.7869873047s, input_fired_time: 17.2416820526, output_fired_time: 5.5664763451, propagate_forward_time: 380.8253479004, propagate_backward_time: 616.4830322266, weight_update_time: 0.2142090499, other_time: 10.6704101563
epoch time: 75.7082214355s, input_fired_time: 0.9114415646, output_fired_time: 0.0000000000, propagate_forward_time: 73.0224914551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7742919922
validation[          , genome  6105] predictions:    6251/  10000 (62.51%), best:    6313/10000 (63.13%), error:     10997.18262, best error:     10588.98438 on epoch:     1, epoch:    9/25, mu: 0.6519285440, learning_rate: 0.0002380015, weight_decay: 0.0001414481

epoch time: 984.9088745117s, input_fired_time: 14.4927396774, output_fired_time: 6.2813067436, propagate_forward_time: 364.6268005371, propagate_backward_time: 589.1774291992, weight_update_time: 0.2127009481, other_time: 10.3306274414
epoch time: 77.6339950562s, input_fired_time: 0.9477196336, output_fired_time: 0.0000000000, propagate_forward_time: 74.8690490723, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8172225952
validation[          , genome  6105] predictions:    6138/  10000 (61.38%), best:    6313/10000 (63.13%), error:     11200.07715, best error:     10588.98438 on epoch:     1, epoch:   10/25, mu: 0.6616038084, learning_rate: 0.0002177587, weight_decay: 0.0001292622

epoch time: 1069.1372070313s, input_fired_time: 17.6530685425, output_fired_time: 5.7139697075, propagate_forward_time: 391.8920288086, propagate_backward_time: 641.7708129883, weight_update_time: 1.2125995159, other_time: 12.1072998047
epoch time: 102.7201004028s, input_fired_time: 2.3682918549, output_fired_time: 0.0000000000, propagate_forward_time: 97.9971466064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3546600342
validation[          , genome  6105] predictions:    6311/  10000 (63.11%), best:    6313/10000 (63.13%), error:     10671.02246, best error:     10588.98438 on epoch:     1, epoch:   11/25, mu: 0.6710022092, learning_rate: 0.0001992376, weight_decay: 0.0001181261

epoch time: 1091.8602294922s, input_fired_time: 16.9842643738, output_fired_time: 6.9899110794, propagate_forward_time: 403.6178588867, propagate_backward_time: 652.1378784180, weight_update_time: 1.2414789200, other_time: 12.1303710938
epoch time: 79.6221313477s, input_fired_time: 0.9284546375, output_fired_time: 0.0000000000, propagate_forward_time: 76.9234313965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7702484131
validation[          , genome  6105] predictions:    6351/  10000 (63.51%), best:    6313/10000 (63.13%), error:     10665.10156, best error:     10588.98438 on epoch:     1, epoch:   12/25, mu: 0.6801316142, learning_rate: 0.0001822917, weight_decay: 0.0001079494

epoch time: 1094.1759033203s, input_fired_time: 18.0088024139, output_fired_time: 6.8343791962, propagate_forward_time: 404.2630920410, propagate_backward_time: 653.8063354492, weight_update_time: 0.2106710821, other_time: 11.2633666992
epoch time: 101.5033798218s, input_fired_time: 1.2046810389, output_fired_time: 0.0000000000, propagate_forward_time: 98.0482940674, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2504043579
validation[          , genome  6105] predictions:    6256/  10000 (62.56%), best:    6313/10000 (63.13%), error:     10863.49414, best error:     10588.98438 on epoch:     1, epoch:   13/25, mu: 0.6889997721, learning_rate: 0.0001667872, weight_decay: 0.0000986494

epoch time: 1013.1716918945s, input_fired_time: 15.0214824677, output_fired_time: 6.4026904106, propagate_forward_time: 374.0330200195, propagate_backward_time: 605.8931884766, weight_update_time: 0.2074213773, other_time: 11.8212890625
epoch time: 75.6970825195s, input_fired_time: 0.9282635450, output_fired_time: 0.0000000000, propagate_forward_time: 71.9314346313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8373870850
validation[          , genome  6105] predictions:    6302/  10000 (63.02%), best:    6313/10000 (63.13%), error:     10715.38086, best error:     10588.98438 on epoch:     1, epoch:   14/25, mu: 0.6976141334, learning_rate: 0.0001526014, weight_decay: 0.0000901506

epoch time: 1013.0536499023s, input_fired_time: 14.8306369781, output_fired_time: 6.4360947609, propagate_forward_time: 374.1889038086, propagate_backward_time: 605.7683715820, weight_update_time: 0.2063717991, other_time: 11.8296508789
epoch time: 80.1639328003s, input_fired_time: 0.9878891110, output_fired_time: 0.0000000000, propagate_forward_time: 77.2429962158, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9330444336
validation[          , genome  6105] predictions:    6309/  10000 (63.09%), best:    6313/10000 (63.13%), error:     10708.18750, best error:     10588.98438 on epoch:     1, epoch:   15/25, mu: 0.7059819102, learning_rate: 0.0001396221, weight_decay: 0.0000823840

epoch time: 1048.7927246094s, input_fired_time: 16.5419445038, output_fired_time: 5.5748400688, propagate_forward_time: 390.6933288574, propagate_backward_time: 624.8375244141, weight_update_time: 0.2162903100, other_time: 11.1450195313
epoch time: 80.1651611328s, input_fired_time: 0.9895572662, output_fired_time: 0.0000000000, propagate_forward_time: 76.2528381348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9227676392
validation[          , genome  6105] predictions:    6307/  10000 (63.07%), best:    6313/10000 (63.13%), error:     10761.30859, best error:     10588.98438 on epoch:     1, epoch:   16/25, mu: 0.7141102552, learning_rate: 0.0001277468, weight_decay: 0.0000752865

epoch time: 1186.7596435547s, input_fired_time: 17.3895263672, output_fired_time: 6.2634239197, propagate_forward_time: 440.9970397949, propagate_backward_time: 709.6000976563, weight_update_time: 0.2888981104, other_time: 12.5095214844
epoch time: 80.9549026489s, input_fired_time: 0.9524888992, output_fired_time: 0.0000000000, propagate_forward_time: 78.1569595337, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8454513550
validation[          , genome  6105] predictions:    6242/  10000 (62.42%), best:    6313/10000 (63.13%), error:     10824.36816, best error:     10588.98438 on epoch:     1, epoch:   17/25, mu: 0.7220059633, learning_rate: 0.0001168815, weight_decay: 0.0000688005

epoch time: 1012.3288574219s, input_fired_time: 15.8536701202, output_fired_time: 5.4156899452, propagate_forward_time: 374.4476623535, propagate_backward_time: 605.9707641602, weight_update_time: 0.2097760737, other_time: 10.6410522461
epoch time: 75.6483993530s, input_fired_time: 0.9205378890, output_fired_time: 0.0000000000, propagate_forward_time: 72.9613342285, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7665252686
validation[          , genome  6105] predictions:    6297/  10000 (62.97%), best:    6313/10000 (63.13%), error:     10813.32422, best error:     10588.98438 on epoch:     1, epoch:   18/25, mu: 0.7296757102, learning_rate: 0.0001069403, weight_decay: 0.0000628732

epoch time: 979.6613159180s, input_fired_time: 16.3820686340, output_fired_time: 5.2325849533, propagate_forward_time: 362.0600585938, propagate_backward_time: 585.9283447266, weight_update_time: 0.2067442387, other_time: 10.0582275391
epoch time: 74.3034362793s, input_fired_time: 0.9178609848, output_fired_time: 0.0000000000, propagate_forward_time: 71.6827926636, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7027816772
validation[          , genome  6105] predictions:    6381/  10000 (63.81%), best:    6381/10000 (63.81%), error:     10566.14453, best error:     10566.14453 on epoch:    19, epoch:   19/25, mu: 0.7371259332, learning_rate: 0.0000978447, weight_decay: 0.0000574566

epoch time: 1040.5373535156s, input_fired_time: 15.2819690704, output_fired_time: 5.5182914734, propagate_forward_time: 382.1901550293, propagate_backward_time: 624.9942016602, weight_update_time: 0.2074720412, other_time: 12.5526733398
epoch time: 79.1070480347s, input_fired_time: 0.9316676855, output_fired_time: 0.0000000000, propagate_forward_time: 76.3947601318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7806167603
validation[          , genome  6105] predictions:    6265/  10000 (62.65%), best:    6381/10000 (63.81%), error:     10749.57324, best error:     10566.14453 on epoch:    19, epoch:   20/25, mu: 0.7443629503, learning_rate: 0.0000895227, weight_decay: 0.0000525066

epoch time: 1138.5783691406s, input_fired_time: 17.6509418488, output_fired_time: 6.1919312477, propagate_forward_time: 419.1016845703, propagate_backward_time: 684.3776855469, weight_update_time: 0.2369720042, other_time: 11.2561035156
epoch time: 79.9312362671s, input_fired_time: 1.0214692354, output_fired_time: 0.0000000000, propagate_forward_time: 76.9907989502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9189682007
validation[          , genome  6105] predictions:    6348/  10000 (63.48%), best:    6381/10000 (63.81%), error:     10568.15918, best error:     10566.14453 on epoch:    19, epoch:   21/25, mu: 0.7513928413, learning_rate: 0.0000819085, weight_decay: 0.0000479831

epoch time: 1053.9090576172s, input_fired_time: 15.5927057266, output_fired_time: 5.6779241562, propagate_forward_time: 384.1352233887, propagate_backward_time: 636.1755371094, weight_update_time: 0.2330197990, other_time: 12.3276367188
epoch time: 80.3394393921s, input_fired_time: 0.9737735987, output_fired_time: 0.0000000000, propagate_forward_time: 77.4436950684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9219741821
validation[          , genome  6105] predictions:    6405/  10000 (64.05%), best:    6381/10000 (63.81%), error:     10611.36230, best error:     10566.14453 on epoch:    19, epoch:   22/25, mu: 0.7582215667, learning_rate: 0.0000749419, weight_decay: 0.0000438493

epoch time: 1006.6430664063s, input_fired_time: 15.8296566010, output_fired_time: 5.4459733963, propagate_forward_time: 370.4701843262, propagate_backward_time: 603.9269409180, weight_update_time: 0.2208286077, other_time: 10.9702758789
epoch time: 76.5347213745s, input_fired_time: 0.9272825718, output_fired_time: 0.0000000000, propagate_forward_time: 73.7152175903, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8922195435
validation[          , genome  6105] predictions:    6236/  10000 (62.36%), best:    6381/10000 (63.81%), error:     10891.22559, best error:     10566.14453 on epoch:    19, epoch:   23/25, mu: 0.7648548484, learning_rate: 0.0000685678, weight_decay: 0.0000400716

epoch time: 1050.6904296875s, input_fired_time: 18.7369651794, output_fired_time: 6.6703848839, propagate_forward_time: 386.3440856934, propagate_backward_time: 625.5670776367, weight_update_time: 0.2534157932, other_time: 13.3718872070
epoch time: 102.3843154907s, input_fired_time: 1.2317715883, output_fired_time: 0.0000000000, propagate_forward_time: 98.6162796021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5362625122
validation[          , genome  6105] predictions:    6429/  10000 (64.29%), best:    6429/10000 (64.29%), error:     10376.76660, best error:     10376.76660 on epoch:    24, epoch:   24/25, mu: 0.7712982893, learning_rate: 0.0000627359, weight_decay: 0.0000366194

epoch time: 1069.1844482422s, input_fired_time: 15.7993459702, output_fired_time: 5.6640172005, propagate_forward_time: 396.0414123535, propagate_backward_time: 640.2371215820, weight_update_time: 0.2280267924, other_time: 11.4425659180
epoch time: 81.4454040527s, input_fired_time: 1.0008208752, output_fired_time: 0.0000000000, propagate_forward_time: 78.5436630249, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9009170532
validation[          , genome  6105] predictions:    6412/  10000 (64.12%), best:    6429/10000 (64.29%), error:     10416.30566, best error:     10376.76660 on epoch:    24, epoch:   25/25, mu: 0.7775573134, learning_rate: 0.0000574000, weight_decay: 0.0000334646

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 412.4619750977s, input_fired_time: 5.0765242577, output_fired_time: 0.0000000000, propagate_forward_time: 397.2203369141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.1651000977
best training[          , genome  6105] predictions:   34877/  50000 (69.75%), best:    6429/10000 (64.29%), error:     45681.87109, best error:     10376.76660 on epoch:    24, epoch:   26/25, mu: 0.7836372256, learning_rate: 0.0000525179, weight_decay: 0.0000305816
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 82.6370239258s, input_fired_time: 1.0480329990, output_fired_time: 0.0000000000, propagate_forward_time: 79.4140777588, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1749114990
   testing[          , genome  6105] predictions:    6429/  10000 (64.29%), best:    6429/10000 (64.29%), error:     10376.76660, best error:     10376.76660 on epoch:    24, epoch:   26/25, mu: 0.7836372256, learning_rate: 0.0000525179, weight_decay: 0.0000305816
backpropagation finished successfully!
13:46:35 (13552): called boinc_finish(0)

</stderr_txt>
]]>