Nameexact_genome_1542414603_173_6164_1
Workunit4095168
Created17 Nov 2018, 10:04:29 UTC
Sent17 Nov 2018, 10:32:22 UTC
Report deadline21 Dec 2018, 16:33:05 UTC
Received19 Nov 2018, 9:39:54 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID79157
Run time8 hours 51 min 52 sec
CPU time8 hours 49 min 38 sec
Validate stateValid
Credit8,562.63
Device peak FLOPS4.43 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size584.47 MB
Peak swap size604.24 MB
Peak disk usage12.30 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_173_6164.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542414603_173_6164_1_r900375446_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542414603_173_6164.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.548989
read mu: 0.548989
read mu_delta: 0.971094
read initial_learning_rate: 0.000567238
read learning_rate: 0.000567238
read learning_rate_delta: 0.915527
read initial_weight_decay: 0.000319767
read weight_decay: 0.000319767
read weight_decay_delta: 0.915208
read batch_size: 122
read epsilon: 1e-07
read alpha: 0.123874
read input_dropout_probability: 0.00167382
read hidden_dropout_probability: 0.0658555
read velocity_reset: 1785
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6164
read normal distribution: '0 -0.591850101947784 0.704913377761841'
generator_str: '775581913'
read generator: 775581913
read generated_by_map:
1 change_size_x 1
reading 93 nodes.
reading 390 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 2126958148
post shuffle 1: 1668460003
epoch time: 1140.63891601563s, input_fired_time: 15.3179874420166, output_fired_time: 5.22213125228882, propagate_forward_time: 328.122467041016, propagate_backward_time: 783.741333007813, weight_update_time: 0.182182520627975, other_time: 8.2349853515625
epoch time: 67.7799224853516s, input_fired_time: 0.976473927497864, output_fired_time: 0, propagate_forward_time: 65.5478210449219, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.25563049316406
validation[          , genome  6164] predictions:    6176/  10000 (61.76%), best:    6176/10000 (61.76%), error:     11043.67285, best error:     11043.67285 on epoch:     0, epoch:    0/25, mu: 0.5489889979, learning_rate: 0.0005672384, weight_decay: 0.0003197667

epoch time: 1141.2624511719s, input_fired_time: 15.3010501862, output_fired_time: 5.2289652824, propagate_forward_time: 328.1608886719, propagate_backward_time: 783.9613037109, weight_update_time: 0.1877528578, other_time: 8.6102294922
epoch time: 67.7981872559s, input_fired_time: 0.9627945423, output_fired_time: 0.0000000000, propagate_forward_time: 65.5734710693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2619247437
validation[          , genome  6164] predictions:    5922/  10000 (59.22%), best:    6176/10000 (61.76%), error:     11788.92676, best error:     11043.67285 on epoch:     0, epoch:    1/25, mu: 0.5617369413, learning_rate: 0.0005193221, weight_decay: 0.0002926531

epoch time: 1141.1816406250s, input_fired_time: 15.2640390396, output_fired_time: 5.2234306335, propagate_forward_time: 327.9191284180, propagate_backward_time: 784.2531738281, weight_update_time: 0.1814852506, other_time: 8.5219116211
epoch time: 67.8544387817s, input_fired_time: 0.9701659679, output_fired_time: 0.0000000000, propagate_forward_time: 65.6313247681, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2529449463
validation[          , genome  6164] predictions:    5829/  10000 (58.29%), best:    6176/10000 (61.76%), error:     12202.08984, best error:     11043.67285 on epoch:     0, epoch:    2/25, mu: 0.5741164088, learning_rate: 0.0004754534, weight_decay: 0.0002678384

epoch time: 1140.8348388672s, input_fired_time: 15.2389202118, output_fired_time: 5.2249960899, propagate_forward_time: 327.8224792480, propagate_backward_time: 783.9621582031, weight_update_time: 0.1827077270, other_time: 8.5863037109
epoch time: 67.8535156250s, input_fired_time: 0.9591085911, output_fired_time: 0.0000000000, propagate_forward_time: 65.6245040894, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2699050903
validation[          , genome  6164] predictions:    6179/  10000 (61.79%), best:    6176/10000 (61.76%), error:     11076.83887, best error:     11043.67285 on epoch:     0, epoch:    3/25, mu: 0.5861380100, learning_rate: 0.0004352904, weight_decay: 0.0002451279

epoch time: 1141.5828857422s, input_fired_time: 15.2496967316, output_fired_time: 5.2214217186, propagate_forward_time: 328.2657165527, propagate_backward_time: 784.3266601563, weight_update_time: 0.1809565127, other_time: 8.5192871094
epoch time: 67.7264556885s, input_fired_time: 0.9607852697, output_fired_time: 0.0000000000, propagate_forward_time: 65.5108032227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2548675537
validation[          , genome  6164] predictions:    6117/  10000 (61.17%), best:    6176/10000 (61.76%), error:     11243.98340, best error:     11043.67285 on epoch:     0, epoch:    4/25, mu: 0.5978121161, learning_rate: 0.0003985201, weight_decay: 0.0002243430

epoch time: 1141.0604248047s, input_fired_time: 15.2257947922, output_fired_time: 5.2128167152, propagate_forward_time: 328.1186828613, propagate_backward_time: 783.9766235352, weight_update_time: 0.1815059185, other_time: 8.5265502930
epoch time: 67.7843627930s, input_fired_time: 0.9507567286, output_fired_time: 0.0000000000, propagate_forward_time: 65.5595245361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2740783691
validation[          , genome  6164] predictions:    6020/  10000 (60.20%), best:    6176/10000 (61.76%), error:     11450.11133, best error:     11043.67285 on epoch:     0, epoch:    5/25, mu: 0.6091488004, learning_rate: 0.0003648560, weight_decay: 0.0002053206

epoch time: 1140.7695312500s, input_fired_time: 15.2477531433, output_fired_time: 5.2168254852, propagate_forward_time: 328.1602172852, propagate_backward_time: 783.6427612305, weight_update_time: 0.1816975176, other_time: 8.5019531250
epoch time: 67.7720489502s, input_fired_time: 0.9563859701, output_fired_time: 0.0000000000, propagate_forward_time: 65.5551681519, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2604980469
validation[          , genome  6164] predictions:    6231/  10000 (62.31%), best:    6231/10000 (62.31%), error:     10977.35938, best error:     10977.35938 on epoch:     6, epoch:    6/25, mu: 0.6201577783, learning_rate: 0.0003340355, weight_decay: 0.0001879110

epoch time: 1141.3024902344s, input_fired_time: 15.2200441360, output_fired_time: 5.2119603157, propagate_forward_time: 328.3515930176, propagate_backward_time: 784.0068969727, weight_update_time: 0.1821533740, other_time: 8.5120239258
epoch time: 67.8371505737s, input_fired_time: 0.9548230767, output_fired_time: 0.0000000000, propagate_forward_time: 65.6261367798, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2561874390
validation[          , genome  6164] predictions:    6071/  10000 (60.71%), best:    6231/10000 (62.31%), error:     11403.68750, best error:     10977.35938 on epoch:     6, epoch:    7/25, mu: 0.6308485270, learning_rate: 0.0003058185, weight_decay: 0.0001719777

epoch time: 1141.1978759766s, input_fired_time: 15.2078533173, output_fired_time: 5.2095718384, propagate_forward_time: 328.0302429199, propagate_backward_time: 784.1875000000, weight_update_time: 0.1825543493, other_time: 8.5626220703
epoch time: 68.6015625000s, input_fired_time: 0.9538143277, output_fired_time: 0.0000000000, propagate_forward_time: 65.7134628296, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9342880249
validation[          , genome  6164] predictions:    6123/  10000 (61.23%), best:    6231/10000 (62.31%), error:     11300.15918, best error:     10977.35938 on epoch:     6, epoch:    8/25, mu: 0.6412302256, learning_rate: 0.0002799851, weight_decay: 0.0001573953

epoch time: 1141.3972167969s, input_fired_time: 15.2268924713, output_fired_time: 5.2160491943, propagate_forward_time: 328.3314819336, propagate_backward_time: 783.9688110352, weight_update_time: 0.1814563870, other_time: 8.6539306641
epoch time: 67.7760314941s, input_fired_time: 0.9547020793, output_fired_time: 0.0000000000, propagate_forward_time: 65.5621871948, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2591400146
validation[          , genome  6164] predictions:    6199/  10000 (61.99%), best:    6231/10000 (62.31%), error:     10992.02637, best error:     10977.35938 on epoch:     6, epoch:    9/25, mu: 0.6513118148, learning_rate: 0.0002563340, weight_decay: 0.0001440495

epoch time: 1142.0988769531s, input_fired_time: 15.2290639877, output_fired_time: 5.2155280113, propagate_forward_time: 328.2708129883, propagate_backward_time: 784.8433227539, weight_update_time: 0.1818020493, other_time: 8.5401611328
epoch time: 68.3305053711s, input_fired_time: 0.9613234401, output_fired_time: 0.0000000000, propagate_forward_time: 65.4932937622, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8758850098
validation[          , genome  6164] predictions:    6269/  10000 (62.69%), best:    6269/10000 (62.69%), error:     10975.32910, best error:     10975.32910 on epoch:    10, epoch:   10/25, mu: 0.6611019969, learning_rate: 0.0002346807, weight_decay: 0.0001318352

epoch time: 1140.8468017578s, input_fired_time: 15.2097520828, output_fired_time: 5.2047853470, propagate_forward_time: 328.1224670410, propagate_backward_time: 784.0565185547, weight_update_time: 0.1838093549, other_time: 8.2532958984
epoch time: 67.7927322388s, input_fired_time: 0.9540634155, output_fired_time: 0.0000000000, propagate_forward_time: 65.5776977539, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2609710693
validation[          , genome  6164] predictions:    6069/  10000 (60.69%), best:    6269/10000 (62.69%), error:     11576.80566, best error:     10975.32910 on epoch:    10, epoch:   11/25, mu: 0.6706091762, learning_rate: 0.0002148565, weight_decay: 0.0001206567

epoch time: 1140.8645019531s, input_fired_time: 15.2516880035, output_fired_time: 5.2169132233, propagate_forward_time: 328.1181030273, propagate_backward_time: 783.9930419922, weight_update_time: 0.1828147024, other_time: 8.2847290039
epoch time: 67.7219009399s, input_fired_time: 0.9539921284, output_fired_time: 0.0000000000, propagate_forward_time: 65.5171051025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2508010864
validation[          , genome  6164] predictions:    6090/  10000 (60.90%), best:    6269/10000 (62.69%), error:     11241.77344, best error:     10975.32910 on epoch:    10, epoch:   12/25, mu: 0.6798415780, learning_rate: 0.0001967069, weight_decay: 0.0001104260

epoch time: 1141.7026367188s, input_fired_time: 15.2490625381, output_fired_time: 5.2227997780, propagate_forward_time: 328.4073486328, propagate_backward_time: 784.4329833984, weight_update_time: 0.1823807359, other_time: 8.3905029297
epoch time: 67.7265243530s, input_fired_time: 0.9615643024, output_fired_time: 0.0000000000, propagate_forward_time: 65.5007553101, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2642059326
validation[          , genome  6164] predictions:    6217/  10000 (62.17%), best:    6217/10000 (62.17%), error:     10889.81250, best error:     10889.81250 on epoch:    13, epoch:   13/25, mu: 0.6888070703, learning_rate: 0.0001800905, weight_decay: 0.0001010627

epoch time: 1142.3759765625s, input_fired_time: 15.2416486740, output_fired_time: 5.2208485603, propagate_forward_time: 328.4308166504, propagate_backward_time: 784.7449951172, weight_update_time: 0.1824156791, other_time: 8.7376708984
epoch time: 68.4560317993s, input_fired_time: 0.9561399221, output_fired_time: 0.0000000000, propagate_forward_time: 65.5687332153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9311599731
validation[          , genome  6164] predictions:    6360/  10000 (63.60%), best:    6360/10000 (63.60%), error:     10646.34766, best error:     10646.34766 on epoch:    14, epoch:   14/25, mu: 0.6975134015, learning_rate: 0.0001648778, weight_decay: 0.0000924934

epoch time: 1141.2152099609s, input_fired_time: 15.2286853790, output_fired_time: 5.2155809402, propagate_forward_time: 328.2217712402, propagate_backward_time: 783.9603881836, weight_update_time: 0.1823724806, other_time: 8.5888061523
epoch time: 67.7669525146s, input_fired_time: 0.9595887661, output_fired_time: 0.0000000000, propagate_forward_time: 65.5526428223, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2547225952
validation[          , genome  6164] predictions:    6285/  10000 (62.85%), best:    6360/10000 (63.60%), error:     10963.71289, best error:     10646.34766 on epoch:    14, epoch:   15/25, mu: 0.7059680820, learning_rate: 0.0001509500, weight_decay: 0.0000846507

epoch time: 1144.6903076172s, input_fired_time: 15.2852973938, output_fired_time: 5.2182254791, propagate_forward_time: 329.0620422363, propagate_backward_time: 786.2963256836, weight_update_time: 0.1822769046, other_time: 8.8284301758
epoch time: 68.4364624023s, input_fired_time: 0.9590074420, output_fired_time: 0.0000000000, propagate_forward_time: 65.5144119263, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9630432129
validation[          , genome  6164] predictions:    6364/  10000 (63.64%), best:    6360/10000 (63.60%), error:     10657.96094, best error:     10646.34766 on epoch:    14, epoch:   16/25, mu: 0.7141783834, learning_rate: 0.0001381988, weight_decay: 0.0000774730

epoch time: 1141.8941650391s, input_fired_time: 15.2518672943, output_fired_time: 5.2113690376, propagate_forward_time: 328.3342895508, propagate_backward_time: 784.2106323242, weight_update_time: 0.1825413555, other_time: 8.8859863281
epoch time: 68.4778442383s, input_fired_time: 0.9612417221, output_fired_time: 0.0000000000, propagate_forward_time: 65.5521316528, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9644699097
validation[          , genome  6164] predictions:    6313/  10000 (63.13%), best:    6360/10000 (63.60%), error:     10767.41504, best error:     10646.34766 on epoch:    14, epoch:   17/25, mu: 0.7221513391, learning_rate: 0.0001265248, weight_decay: 0.0000709039

epoch time: 1141.4357910156s, input_fired_time: 15.2402143478, output_fired_time: 5.2201175690, propagate_forward_time: 327.9248962402, propagate_backward_time: 784.1257934570, weight_update_time: 0.1829223037, other_time: 8.9247436523
epoch time: 68.4765930176s, input_fired_time: 0.9631546140, output_fired_time: 0.0000000000, propagate_forward_time: 65.5776062012, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9358291626
validation[          , genome  6164] predictions:    6314/  10000 (63.14%), best:    6360/10000 (63.60%), error:     10705.61816, best error:     10646.34766 on epoch:    14, epoch:   18/25, mu: 0.7298938036, learning_rate: 0.0001158368, weight_decay: 0.0000648919

epoch time: 1141.5788574219s, input_fired_time: 15.2486763000, output_fired_time: 5.2270536423, propagate_forward_time: 328.1644897461, propagate_backward_time: 784.2105102539, weight_update_time: 0.1830681860, other_time: 8.7281494141
epoch time: 68.5427551270s, input_fired_time: 0.9562328458, output_fired_time: 0.0000000000, propagate_forward_time: 65.5697784424, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0167465210
validation[          , genome  6164] predictions:    6253/  10000 (62.53%), best:    6360/10000 (63.60%), error:     10841.88867, best error:     10646.34766 on epoch:    14, epoch:   19/25, mu: 0.7374125123, learning_rate: 0.0001060518, weight_decay: 0.0000593895

epoch time: 1142.7747802734s, input_fired_time: 15.2827367783, output_fired_time: 5.2204260826, propagate_forward_time: 328.5979919434, propagate_backward_time: 784.9409790039, weight_update_time: 0.1812393814, other_time: 8.7326049805
epoch time: 68.5141906738s, input_fired_time: 0.9735577106, output_fired_time: 0.0000000000, propagate_forward_time: 65.5236587524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0169754028
validation[          , genome  6164] predictions:    6263/  10000 (62.63%), best:    6360/10000 (63.60%), error:     10883.94727, best error:     10646.34766 on epoch:    14, epoch:   20/25, mu: 0.7447138429, learning_rate: 0.0000970933, weight_decay: 0.0000543538

epoch time: 1141.2401123047s, input_fired_time: 15.2382478714, output_fired_time: 5.2103128433, propagate_forward_time: 328.0874633789, propagate_backward_time: 784.2366943359, weight_update_time: 0.1824953854, other_time: 8.4673461914
epoch time: 67.7374801636s, input_fired_time: 0.9646091461, output_fired_time: 0.0000000000, propagate_forward_time: 65.5077590942, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2651138306
validation[          , genome  6164] predictions:    6383/  10000 (63.83%), best:    6360/10000 (63.60%), error:     10649.71094, best error:     10646.34766 on epoch:    14, epoch:   21/25, mu: 0.7518041134, learning_rate: 0.0000888915, weight_decay: 0.0000497450

epoch time: 1142.1713867188s, input_fired_time: 15.2715902328, output_fired_time: 5.2173161507, propagate_forward_time: 328.3143005371, propagate_backward_time: 784.7067260742, weight_update_time: 0.1827995479, other_time: 8.6614379883
epoch time: 68.4943847656s, input_fired_time: 0.9673638344, output_fired_time: 0.0000000000, propagate_forward_time: 65.5479202271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9791030884
validation[          , genome  6164] predictions:    6348/  10000 (63.48%), best:    6360/10000 (63.60%), error:     10652.36816, best error:     10646.34766 on epoch:    14, epoch:   22/25, mu: 0.7586894631, learning_rate: 0.0000813826, weight_decay: 0.0000455270

epoch time: 1141.4874267578s, input_fired_time: 15.2908582687, output_fired_time: 5.2094459534, propagate_forward_time: 328.4398498535, propagate_backward_time: 784.0669555664, weight_update_time: 0.1818432361, other_time: 8.4802856445
epoch time: 67.8005599976s, input_fired_time: 0.9661713243, output_fired_time: 0.0000000000, propagate_forward_time: 65.5680389404, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2663497925
validation[          , genome  6164] predictions:    6313/  10000 (63.13%), best:    6313/10000 (63.13%), error:     10639.35742, best error:     10639.35742 on epoch:    23, epoch:   23/25, mu: 0.7653757930, learning_rate: 0.0000745080, weight_decay: 0.0000416667

epoch time: 1140.7825927734s, input_fired_time: 15.2666692734, output_fired_time: 5.2122254372, propagate_forward_time: 327.9757995605, propagate_backward_time: 783.9304809570, weight_update_time: 0.1826038808, other_time: 8.3972778320
epoch time: 67.8358306885s, input_fired_time: 0.9725747108, output_fired_time: 0.0000000000, propagate_forward_time: 65.5957870483, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2674713135
validation[          , genome  6164] predictions:    6444/  10000 (64.44%), best:    6444/10000 (64.44%), error:     10398.72070, best error:     10398.72070 on epoch:    24, epoch:   24/25, mu: 0.7718688250, learning_rate: 0.0000682140, weight_decay: 0.0000381337

epoch time: 1141.5029296875s, input_fired_time: 15.2947597504, output_fired_time: 5.2073297501, propagate_forward_time: 328.2265319824, propagate_backward_time: 784.2991333008, weight_update_time: 0.1816822886, other_time: 8.4751586914
epoch time: 67.7876739502s, input_fired_time: 0.9692639709, output_fired_time: 0.0000000000, propagate_forward_time: 65.5576934814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2607192993
validation[          , genome  6164] predictions:    6406/  10000 (64.06%), best:    6444/10000 (64.44%), error:     10427.83203, best error:     10398.72070 on epoch:    24, epoch:   25/25, mu: 0.7781741619, learning_rate: 0.0000624518, weight_decay: 0.0000349003

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 339.4598083496s, input_fired_time: 4.8674559593, output_fired_time: 0.0000000000, propagate_forward_time: 327.9649658203, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.6273803711
best training[          , genome  6164] predictions:   34753/  50000 (69.51%), best:    6444/10000 (64.44%), error:     46270.74609, best error:     10398.72070 on epoch:    24, epoch:   26/25, mu: 0.7842972279, learning_rate: 0.0000571763, weight_decay: 0.0000319410
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 68.5849838257s, input_fired_time: 0.9713224769, output_fired_time: 0.0000000000, propagate_forward_time: 65.5340957642, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0795669556
   testing[          , genome  6164] predictions:    6444/  10000 (64.44%), best:    6444/10000 (64.44%), error:     10398.72070, best error:     10398.72070 on epoch:    24, epoch:   26/25, mu: 0.7842972279, learning_rate: 0.0000571763, weight_decay: 0.0000319410
backpropagation finished successfully!
03:31:08 (5316): called boinc_finish(0)

</stderr_txt>
]]>