| Name | exact_genome_1542692703_178_6371_3 |
| Workunit | 4100304 |
| Created | 4 Feb 2019, 12:48:16 UTC |
| Sent | 4 Feb 2019, 12:48:30 UTC |
| Report deadline | 21 Apr 2019, 18:26:29 UTC |
| Received | 5 Feb 2019, 6:36:03 UTC |
| Server state | Over |
| Outcome | Success |
| Client state | Done |
| Exit status | 0 (0x0) |
| Computer ID | 87483 |
| Run time | 17 hours 44 min 37 sec |
| CPU time | 17 hours 40 min 26 sec |
| Validate state | Valid |
| Credit | 19,058.68 |
| Device peak FLOPS | 4.37 GFLOPS |
| Application version | EXACT Batch Norm With Scaled FMP CNN Trainer v0.34 |
| Peak working set size | 569.86 MB |
| Peak swap size | 589.70 MB |
| Peak disk usage | 16.42 MB |
Stderr output
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
'--training_file'
'training_samples.bin'
'--validation_file'
'validation_samples.bin'
'--testing_file'
'testing_samples.bin'
'--genome_file'
'input_genome.txt'
'--output_file'
'output_genome.txt'
'--checkpoint_file'
'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_178_6371.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_178_6371_3_r1447296945_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542692703_178_6371.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.487938
read mu: 0.487938
read mu_delta: 0.95195
read initial_learning_rate: 0.000690388
read learning_rate: 0.000690388
read learning_rate_delta: 0.920021
read initial_weight_decay: 0.000503204
read weight_decay: 0.000503204
read weight_decay_delta: 0.926033
read batch_size: 102
read epsilon: 1e-07
read alpha: 0.0862348
read input_dropout_probability: 0.0011378
read hidden_dropout_probability: 0.109211
read velocity_reset: 2170
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6371
read normal distribution: '1 -0.359059572219849 0.62816196680069'
generator_str: '714351106'
read generator: 714351106
read generated_by_map:
1 crossover 1
reading 120 nodes.
reading 629 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
class 0: 5000
class 1: 5000
class 2: 5000
class 3: 5000
class 4: 5000
class 5: 5000
class 6: 5000
class 7: 5000
class 8: 5000
class 9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
class 0: 1000
class 1: 1000
class 2: 1000
class 3: 1000
class 4: 1000
class 5: 1000
class 6: 1000
class 7: 1000
class 8: 1000
class 9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
class 0: 1000
class 1: 1000
class 2: 1000
class 3: 1000
class 4: 1000
class 5: 1000
class 6: 1000
class 7: 1000
class 8: 1000
class 9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1665451812
post shuffle 1: 795412496
epoch time: 1171.10900878906s, input_fired_time: 18.3309421539307, output_fired_time: 4.95036554336548, propagate_forward_time: 418.645843505859, propagate_backward_time: 708.869934082031, weight_update_time: 0.314247220754623, other_time: 20.3119506835938
epoch time: 87.2619552612305s, input_fired_time: 1.00982618331909, output_fired_time: 0, propagate_forward_time: 82.3963394165039, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.85578918457031
validation[ , genome 6371] predictions: 5851/ 10000 (58.51%), best: 5851/10000 (58.51%), error: 12205.16016, best error: 12205.16016 on epoch: 0, epoch: 0/50, mu: 0.4879384041, learning_rate: 0.0006903875, weight_decay: 0.0005032038
epoch time: 1147.4366455078s, input_fired_time: 18.1512680054, output_fired_time: 4.8344087601, propagate_forward_time: 408.7622985840, propagate_backward_time: 694.8596801758, weight_update_time: 0.2958233953, other_time: 20.8290405273
epoch time: 87.3984069824s, input_fired_time: 0.9999215007, output_fired_time: 0.0000000000, propagate_forward_time: 82.5556869507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8427963257
validation[ , genome 6371] predictions: 5241/ 10000 (52.41%), best: 5851/10000 (58.51%), error: 14612.85449, best error: 12205.16016 on epoch: 0, epoch: 1/50, mu: 0.5120623708, learning_rate: 0.0006351707, weight_decay: 0.0004659831
epoch time: 1146.9587402344s, input_fired_time: 18.1381034851, output_fired_time: 4.8281340599, propagate_forward_time: 408.9361877441, propagate_backward_time: 694.2449951172, weight_update_time: 0.2947723866, other_time: 20.8114013672
epoch time: 87.4873886108s, input_fired_time: 0.9875639081, output_fired_time: 0.0000000000, propagate_forward_time: 82.5564575195, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9433670044
validation[ , genome 6371] predictions: 5679/ 10000 (56.79%), best: 5851/10000 (58.51%), error: 12278.61328, best error: 12205.16016 on epoch: 0, epoch: 2/50, mu: 0.5350272059, learning_rate: 0.0005843700, weight_decay: 0.0004315157
epoch time: 1147.7773437500s, input_fired_time: 18.1153488159, output_fired_time: 4.8414707184, propagate_forward_time: 408.9851379395, propagate_backward_time: 694.7136840820, weight_update_time: 0.2969872355, other_time: 21.1217651367
epoch time: 87.4324951172s, input_fired_time: 0.9958930016, output_fired_time: 0.0000000000, propagate_forward_time: 82.5045928955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9320068359
validation[ , genome 6371] predictions: 5977/ 10000 (59.77%), best: 5977/10000 (59.77%), error: 11618.38281, best error: 11618.38281 on epoch: 3, epoch: 3/50, mu: 0.5568885803, learning_rate: 0.0005376324, weight_decay: 0.0003995976
epoch time: 1147.2086181641s, input_fired_time: 18.0936355591, output_fired_time: 4.8373985291, propagate_forward_time: 408.6090698242, propagate_backward_time: 694.6453857422, weight_update_time: 0.2943717241, other_time: 21.0231323242
epoch time: 87.1927490234s, input_fired_time: 0.9861366153, output_fired_time: 0.0000000000, propagate_forward_time: 82.3136978149, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8929138184
validation[ , genome 6371] predictions: 5597/ 10000 (55.97%), best: 5977/10000 (59.77%), error: 12647.25195, best error: 11618.38281 on epoch: 3, epoch: 4/50, mu: 0.5776994824, learning_rate: 0.0004946329, weight_decay: 0.0003700405
epoch time: 1147.1717529297s, input_fired_time: 18.0660591125, output_fired_time: 4.8380298615, propagate_forward_time: 408.5275878906, propagate_backward_time: 694.6584472656, weight_update_time: 0.2932381630, other_time: 21.0816650391
epoch time: 87.2936401367s, input_fired_time: 0.9838654995, output_fired_time: 0.0000000000, propagate_forward_time: 82.4146118164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8951644897
validation[ , genome 6371] predictions: 5629/ 10000 (56.29%), best: 5977/10000 (59.77%), error: 13068.20996, best error: 11618.38281 on epoch: 3, epoch: 5/50, mu: 0.5975104570, learning_rate: 0.0004550724, weight_decay: 0.0003426696
epoch time: 1148.7604980469s, input_fired_time: 18.0952224731, output_fired_time: 4.8472127914, propagate_forward_time: 409.3087158203, propagate_backward_time: 695.4993896484, weight_update_time: 0.2978025079, other_time: 21.0100097656
epoch time: 87.2807922363s, input_fired_time: 0.9817367196, output_fired_time: 0.0000000000, propagate_forward_time: 82.4929504395, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8061065674
validation[ , genome 6371] predictions: 6037/ 10000 (60.37%), best: 6037/10000 (60.37%), error: 11459.94336, best error: 11459.94336 on epoch: 6, epoch: 6/50, mu: 0.6163694859, learning_rate: 0.0004186759, weight_decay: 0.0003173233
epoch time: 1150.1528320313s, input_fired_time: 18.0783157349, output_fired_time: 4.8631801605, propagate_forward_time: 409.6589050293, propagate_backward_time: 696.8057861328, weight_update_time: 0.3001708686, other_time: 20.7465820313
epoch time: 87.5386962891s, input_fired_time: 0.9856735468, output_fired_time: 0.0000000000, propagate_forward_time: 82.6879119873, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8651123047
validation[ , genome 6371] predictions: 5720/ 10000 (57.20%), best: 6037/10000 (60.37%), error: 12904.93555, best error: 11459.94336 on epoch: 6, epoch: 7/50, mu: 0.6343223453, learning_rate: 0.0003851904, weight_decay: 0.0002938517
epoch time: 1157.1087646484s, input_fired_time: 18.1331710815, output_fired_time: 4.8889918327, propagate_forward_time: 412.5122680664, propagate_backward_time: 700.8859863281, weight_update_time: 0.3043406308, other_time: 20.6882934570
epoch time: 87.8213958740s, input_fired_time: 0.9787873626, output_fired_time: 0.0000000000, propagate_forward_time: 83.0192642212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8233413696
validation[ , genome 6371] predictions: 5837/ 10000 (58.37%), best: 6037/10000 (60.37%), error: 12101.86328, best error: 11459.94336 on epoch: 6, epoch: 8/50, mu: 0.6514126062, learning_rate: 0.0003543831, weight_decay: 0.0002721163
epoch time: 1154.2469482422s, input_fired_time: 18.1291065216, output_fired_time: 4.8883838654, propagate_forward_time: 411.1437377930, propagate_backward_time: 699.2936401367, weight_update_time: 0.3043254614, other_time: 20.7919921875
epoch time: 87.6192703247s, input_fired_time: 0.9892877340, output_fired_time: 0.0000000000, propagate_forward_time: 82.6513442993, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9786376953
validation[ , genome 6371] predictions: 5997/ 10000 (59.97%), best: 6037/10000 (60.37%), error: 11624.63477, best error: 11459.94336 on epoch: 6, epoch: 9/50, mu: 0.6676816940, learning_rate: 0.0003260397, weight_decay: 0.0002519886
epoch time: 1154.2623291016s, input_fired_time: 18.1369113922, output_fired_time: 4.8690471649, propagate_forward_time: 411.1906433105, propagate_backward_time: 699.2697143555, weight_update_time: 0.3015769422, other_time: 20.7959594727
epoch time: 88.0776748657s, input_fired_time: 0.9844425321, output_fired_time: 0.0000000000, propagate_forward_time: 83.1797027588, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9135284424
validation[ , genome 6371] predictions: 5873/ 10000 (58.73%), best: 6037/10000 (60.37%), error: 11981.72461, best error: 11459.94336 on epoch: 6, epoch: 10/50, mu: 0.6831690073, learning_rate: 0.0002999633, weight_decay: 0.0002333497
epoch time: 1156.5800781250s, input_fired_time: 18.1558952332, output_fired_time: 4.8945698738, propagate_forward_time: 412.1633300781, propagate_backward_time: 700.8163452148, weight_update_time: 0.3050734103, other_time: 20.5499877930
epoch time: 87.7616500854s, input_fired_time: 0.9849869609, output_fired_time: 0.0000000000, propagate_forward_time: 82.9021224976, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8745422363
validation[ , genome 6371] predictions: 6041/ 10000 (60.41%), best: 6037/10000 (60.37%), error: 11605.66992, best error: 11459.94336 on epoch: 6, epoch: 11/50, mu: 0.6979121566, learning_rate: 0.0002759724, weight_decay: 0.0002160894
epoch time: 1155.9851074219s, input_fired_time: 18.0685386658, output_fired_time: 4.8909325600, propagate_forward_time: 412.0840148926, propagate_backward_time: 700.3084716797, weight_update_time: 0.3056939542, other_time: 20.6331787109
epoch time: 88.1485137939s, input_fired_time: 0.9903801084, output_fired_time: 0.0000000000, propagate_forward_time: 83.2826538086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8754806519
validation[ , genome 6371] predictions: 6079/ 10000 (60.79%), best: 6079/10000 (60.79%), error: 11390.74121, best error: 11390.74121 on epoch: 12, epoch: 12/50, mu: 0.7119469047, learning_rate: 0.0002539002, weight_decay: 0.0002001059
epoch time: 1154.5294189453s, input_fired_time: 18.0944423676, output_fired_time: 4.8868126869, propagate_forward_time: 411.2436218262, propagate_backward_time: 699.3098754883, weight_update_time: 0.3009253442, other_time: 20.9945678711
epoch time: 87.4824676514s, input_fired_time: 0.9857016802, output_fired_time: 0.0000000000, propagate_forward_time: 82.4692993164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0274658203
validation[ , genome 6371] predictions: 6139/ 10000 (61.39%), best: 6139/10000 (61.39%), error: 11370.71191, best error: 11370.71191 on epoch: 13, epoch: 13/50, mu: 0.7253072858, learning_rate: 0.0002335934, weight_decay: 0.0001853046
epoch time: 1149.7935791016s, input_fired_time: 18.0609569550, output_fired_time: 4.8644509315, propagate_forward_time: 409.7585754395, propagate_backward_time: 695.7081298828, weight_update_time: 0.2966526151, other_time: 21.4014892578
epoch time: 87.3257827759s, input_fired_time: 0.9851695895, output_fired_time: 0.0000000000, propagate_forward_time: 82.4462738037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8943405151
validation[ , genome 6371] predictions: 6205/ 10000 (62.05%), best: 6205/10000 (62.05%), error: 11146.46875, best error: 11146.46875 on epoch: 14, epoch: 14/50, mu: 0.7380257249, learning_rate: 0.0002149108, weight_decay: 0.0001715981
epoch time: 1148.9555664063s, input_fired_time: 18.0515804291, output_fired_time: 4.8579368591, propagate_forward_time: 409.0466613770, propagate_backward_time: 695.6886596680, weight_update_time: 0.2977997363, other_time: 21.3107299805
epoch time: 87.3888778687s, input_fired_time: 0.9858330488, output_fired_time: 0.0000000000, propagate_forward_time: 82.4623184204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9407272339
validation[ , genome 6371] predictions: 6152/ 10000 (61.52%), best: 6152/10000 (61.52%), error: 11131.75977, best error: 11131.75977 on epoch: 15, epoch: 15/50, mu: 0.7501330376, learning_rate: 0.0001977223, weight_decay: 0.0001589055
epoch time: 1148.5631103516s, input_fired_time: 18.0642356873, output_fired_time: 4.8612866402, propagate_forward_time: 408.9456787109, propagate_backward_time: 695.4141845703, weight_update_time: 0.2954003215, other_time: 21.2777099609
epoch time: 88.0587615967s, input_fired_time: 0.9883140326, output_fired_time: 0.0000000000, propagate_forward_time: 82.9854888916, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0849609375
validation[ , genome 6371] predictions: 6174/ 10000 (61.74%), best: 6152/10000 (61.52%), error: 11131.76660, best error: 11131.75977 on epoch: 15, epoch: 16/50, mu: 0.7616586089, learning_rate: 0.0001819086, weight_decay: 0.0001471517
epoch time: 1148.5583496094s, input_fired_time: 18.0782146454, output_fired_time: 4.8564519882, propagate_forward_time: 408.9272155762, propagate_backward_time: 695.4164428711, weight_update_time: 0.2957459688, other_time: 21.2799682617
epoch time: 87.4488677979s, input_fired_time: 0.9899655581, output_fired_time: 0.0000000000, propagate_forward_time: 82.4585952759, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0003051758
validation[ , genome 6371] predictions: 6073/ 10000 (60.73%), best: 6152/10000 (61.52%), error: 11640.44336, best error: 11131.75977 on epoch: 15, epoch: 17/50, mu: 0.7726303935, learning_rate: 0.0001673596, weight_decay: 0.0001362673
epoch time: 1148.4772949219s, input_fired_time: 18.0393848419, output_fired_time: 4.8534607887, propagate_forward_time: 409.0261840820, propagate_backward_time: 695.2284545898, weight_update_time: 0.2946647704, other_time: 21.3297119141
epoch time: 87.3952865601s, input_fired_time: 0.9834694862, output_fired_time: 0.0000000000, propagate_forward_time: 82.4404144287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9714050293
validation[ , genome 6371] predictions: 5922/ 10000 (59.22%), best: 6152/10000 (61.52%), error: 11971.86328, best error: 11131.75977 on epoch: 15, epoch: 18/50, mu: 0.7830749750, learning_rate: 0.0001539743, weight_decay: 0.0001261880
epoch time: 1148.4689941406s, input_fired_time: 18.0591659546, output_fired_time: 4.8665437698, propagate_forward_time: 408.9475402832, propagate_backward_time: 695.3703613281, weight_update_time: 0.2955916822, other_time: 21.2253417969
epoch time: 87.7844619751s, input_fired_time: 0.9954255819, output_fired_time: 0.0000000000, propagate_forward_time: 82.8424758911, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9465637207
validation[ , genome 6371] predictions: 6290/ 10000 (62.90%), best: 6290/10000 (62.90%), error: 10951.33887, best error: 10951.33887 on epoch: 19, epoch: 19/50, mu: 0.7930176854, learning_rate: 0.0001416595, weight_decay: 0.0001168542
epoch time: 1148.7050781250s, input_fired_time: 18.0624637604, output_fired_time: 4.8481092453, propagate_forward_time: 409.1722717285, propagate_backward_time: 695.3501586914, weight_update_time: 0.2947011292, other_time: 21.2720336914
epoch time: 87.3842468262s, input_fired_time: 0.9945378900, output_fired_time: 0.0000000000, propagate_forward_time: 82.4098739624, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9798355103
validation[ , genome 6371] predictions: 6255/ 10000 (62.55%), best: 6255/10000 (62.55%), error: 10813.48242, best error: 10813.48242 on epoch: 20, epoch: 20/50, mu: 0.8024826646, learning_rate: 0.0001303296, weight_decay: 0.0001082108
epoch time: 1148.7103271484s, input_fired_time: 18.0852870941, output_fired_time: 4.8577179909, propagate_forward_time: 408.9739990234, propagate_backward_time: 695.4437255859, weight_update_time: 0.2966004312, other_time: 21.3496093750
epoch time: 87.3366241455s, input_fired_time: 0.9912420511, output_fired_time: 0.0000000000, propagate_forward_time: 82.3878631592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9575195313
validation[ , genome 6371] predictions: 6309/ 10000 (63.09%), best: 6255/10000 (62.55%), error: 10864.45410, best error: 10813.48242 on epoch: 20, epoch: 21/50, mu: 0.8114928603, learning_rate: 0.0001199059, weight_decay: 0.0001002067
epoch time: 1148.0670166016s, input_fired_time: 18.0805091858, output_fired_time: 4.8554568291, propagate_forward_time: 408.7672729492, propagate_backward_time: 695.0910034180, weight_update_time: 0.2955374420, other_time: 21.2727050781
epoch time: 87.4052658081s, input_fired_time: 0.9930962324, output_fired_time: 0.0000000000, propagate_forward_time: 82.4020767212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0100936890
validation[ , genome 6371] predictions: 6234/ 10000 (62.34%), best: 6255/10000 (62.55%), error: 11021.36914, best error: 10813.48242 on epoch: 20, epoch: 22/50, mu: 0.8200700879, learning_rate: 0.0001103159, weight_decay: 0.0000927947
epoch time: 1146.9670410156s, input_fired_time: 18.0811290741, output_fired_time: 4.8348765373, propagate_forward_time: 408.4191284180, propagate_backward_time: 694.3334960938, weight_update_time: 0.2924633026, other_time: 21.2984008789
epoch time: 87.2149963379s, input_fired_time: 0.9959757924, output_fired_time: 0.0000000000, propagate_forward_time: 82.3214340210, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8975830078
validation[ , genome 6371] predictions: 6328/ 10000 (63.28%), best: 6328/10000 (63.28%), error: 10735.56934, best error: 10735.56934 on epoch: 23, epoch: 23/50, mu: 0.8282352090, learning_rate: 0.0001014929, weight_decay: 0.0000859309
epoch time: 1147.1868896484s, input_fired_time: 18.0899028778, output_fired_time: 4.8415255547, propagate_forward_time: 408.5261840820, propagate_backward_time: 694.4725952148, weight_update_time: 0.2908887267, other_time: 21.2567138672
epoch time: 87.2491912842s, input_fired_time: 0.9946820736, output_fired_time: 0.0000000000, propagate_forward_time: 82.3402938843, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9142150879
validation[ , genome 6371] predictions: 6332/ 10000 (63.32%), best: 6332/10000 (63.32%), error: 10690.27148, best error: 10690.27148 on epoch: 24, epoch: 24/50, mu: 0.8360079527, learning_rate: 0.0000933756, weight_decay: 0.0000795749
epoch time: 1146.8033447266s, input_fired_time: 18.0877704620, output_fired_time: 4.8389863968, propagate_forward_time: 408.4808654785, propagate_backward_time: 694.3153076172, weight_update_time: 0.2916831374, other_time: 21.0804443359
epoch time: 87.2485275269s, input_fired_time: 0.9901549220, output_fired_time: 0.0000000000, propagate_forward_time: 82.3295211792, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9288482666
validation[ , genome 6371] predictions: 6301/ 10000 (63.01%), best: 6332/10000 (63.32%), error: 10757.70215, best error: 10690.27148 on epoch: 24, epoch: 25/50, mu: 0.8434072137, learning_rate: 0.0000859074, weight_decay: 0.0000736889
epoch time: 1147.6359863281s, input_fired_time: 18.1106605530, output_fired_time: 4.8465628624, propagate_forward_time: 408.7738342285, propagate_backward_time: 694.5135498047, weight_update_time: 0.2906827927, other_time: 21.3913574219
epoch time: 87.3232650757s, input_fired_time: 1.0002330542, output_fired_time: 0.0000000000, propagate_forward_time: 82.3874359131, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9355926514
validation[ , genome 6371] predictions: 6368/ 10000 (63.68%), best: 6368/10000 (63.68%), error: 10538.23438, best error: 10538.23438 on epoch: 26, epoch: 26/50, mu: 0.8504509926, learning_rate: 0.0000790366, weight_decay: 0.0000682384
epoch time: 1147.2917480469s, input_fired_time: 18.1275062561, output_fired_time: 4.8423342705, propagate_forward_time: 408.7068481445, propagate_backward_time: 694.5612182617, weight_update_time: 0.2918126583, other_time: 21.0538330078
epoch time: 87.3429946899s, input_fired_time: 0.9983304143, output_fired_time: 0.0000000000, propagate_forward_time: 82.3481597900, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9965057373
validation[ , genome 6371] predictions: 6335/ 10000 (63.35%), best: 6368/10000 (63.68%), error: 10689.48730, best error: 10538.23438 on epoch: 26, epoch: 27/50, mu: 0.8571562767, learning_rate: 0.0000727153, weight_decay: 0.0000631909
epoch time: 1147.4482421875s, input_fired_time: 18.1272678375, output_fired_time: 4.8459239006, propagate_forward_time: 408.7299804688, propagate_backward_time: 694.4594116211, weight_update_time: 0.2898078561, other_time: 21.2855834961
epoch time: 87.5017547607s, input_fired_time: 0.9986122251, output_fired_time: 0.0000000000, propagate_forward_time: 82.4252243042, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0779190063
validation[ , genome 6371] predictions: 6267/ 10000 (62.67%), best: 6368/10000 (63.68%), error: 10830.14258, best error: 10538.23438 on epoch: 26, epoch: 28/50, mu: 0.8635393977, learning_rate: 0.0000668996, weight_decay: 0.0000585169
epoch time: 1146.8947753906s, input_fired_time: 18.1182270050, output_fired_time: 4.8454484940, propagate_forward_time: 408.4319763184, propagate_backward_time: 694.3144531250, weight_update_time: 0.2932167053, other_time: 21.1845703125
epoch time: 87.3265228271s, input_fired_time: 0.9965069890, output_fired_time: 0.0000000000, propagate_forward_time: 82.3653945923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9646224976
validation[ , genome 6371] predictions: 6184/ 10000 (61.84%), best: 6368/10000 (63.68%), error: 10858.52637, best error: 10538.23438 on epoch: 26, epoch: 29/50, mu: 0.8696157932, learning_rate: 0.0000615490, weight_decay: 0.0000541885
epoch time: 1147.5407714844s, input_fired_time: 18.1470165253, output_fired_time: 4.8456053734, propagate_forward_time: 408.6826782227, propagate_backward_time: 694.6006469727, weight_update_time: 0.2901221514, other_time: 21.2648925781
epoch time: 87.3518981934s, input_fired_time: 1.0009354353, output_fired_time: 0.0000000000, propagate_forward_time: 82.4590225220, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8919372559
validation[ , genome 6371] predictions: 6353/ 10000 (63.53%), best: 6368/10000 (63.68%), error: 10578.80762, best error: 10538.23438 on epoch: 26, epoch: 30/50, mu: 0.8754002452, learning_rate: 0.0000566263, weight_decay: 0.0000501804
epoch time: 1147.1917724609s, input_fired_time: 18.1418819427, output_fired_time: 4.8459010124, propagate_forward_time: 408.5787658691, propagate_backward_time: 694.3585815430, weight_update_time: 0.2915074229, other_time: 21.2666625977
epoch time: 87.5269165039s, input_fired_time: 1.0106549263, output_fired_time: 0.0000000000, propagate_forward_time: 82.5062484741, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0100097656
validation[ , genome 6371] predictions: 6421/ 10000 (64.21%), best: 6421/10000 (64.21%), error: 10487.63281, best error: 10487.63281 on epoch: 31, epoch: 31/50, mu: 0.8809067607, learning_rate: 0.0000520974, weight_decay: 0.0000464687
epoch time: 1147.5227050781s, input_fired_time: 18.1351013184, output_fired_time: 4.8478212357, propagate_forward_time: 408.6801757813, propagate_backward_time: 694.5925292969, weight_update_time: 0.2921815813, other_time: 21.2670898438
epoch time: 87.3934326172s, input_fired_time: 1.0029977560, output_fired_time: 0.0000000000, propagate_forward_time: 82.4716720581, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9187622070
validation[ , genome 6371] predictions: 6277/ 10000 (62.77%), best: 6421/10000 (64.21%), error: 10716.55859, best error: 10487.63281 on epoch: 31, epoch: 32/50, mu: 0.8861486912, learning_rate: 0.0000479307, weight_decay: 0.0000430315
epoch time: 1147.4151611328s, input_fired_time: 18.0989589691, output_fired_time: 4.8438935280, propagate_forward_time: 408.5138854980, propagate_backward_time: 694.7321777344, weight_update_time: 0.2926710844, other_time: 21.2261962891
epoch time: 87.4891738892s, input_fired_time: 1.0068082809, output_fired_time: 0.0000000000, propagate_forward_time: 82.4569320679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0254364014
validation[ , genome 6371] predictions: 6504/ 10000 (65.04%), best: 6504/10000 (65.04%), error: 10252.10742, best error: 10252.10742 on epoch: 33, epoch: 33/50, mu: 0.8911387324, learning_rate: 0.0000440972, weight_decay: 0.0000398486
epoch time: 1147.4097900391s, input_fired_time: 18.1134357452, output_fired_time: 4.8438477516, propagate_forward_time: 408.5508117676, propagate_backward_time: 694.5990600586, weight_update_time: 0.2906489372, other_time: 21.3026733398
epoch time: 87.3651199341s, input_fired_time: 1.0092374086, output_fired_time: 0.0000000000, propagate_forward_time: 82.3461074829, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0097732544
validation[ , genome 6371] predictions: 6376/ 10000 (63.76%), best: 6504/10000 (65.04%), error: 10473.71387, best error: 10252.10742 on epoch: 33, epoch: 34/50, mu: 0.8958889842, learning_rate: 0.0000405703, weight_decay: 0.0000369011
epoch time: 1147.2558593750s, input_fired_time: 18.1082305908, output_fired_time: 4.8458566666, propagate_forward_time: 408.6697998047, propagate_backward_time: 694.4639892578, weight_update_time: 0.2928123176, other_time: 21.1679687500
epoch time: 87.3221130371s, input_fired_time: 1.0098427534, output_fired_time: 0.0000000000, propagate_forward_time: 82.3477935791, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9644775391
validation[ , genome 6371] predictions: 6438/ 10000 (64.38%), best: 6504/10000 (65.04%), error: 10336.01758, best error: 10252.10742 on epoch: 33, epoch: 35/50, mu: 0.9004110098, learning_rate: 0.0000373255, weight_decay: 0.0000341716
epoch time: 1147.4915771484s, input_fired_time: 18.1350021362, output_fired_time: 4.8592429161, propagate_forward_time: 408.8365783691, propagate_backward_time: 694.5227050781, weight_update_time: 0.2899936736, other_time: 21.1380615234
epoch time: 87.4179687500s, input_fired_time: 1.0087592602, output_fired_time: 0.0000000000, propagate_forward_time: 82.4126358032, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9965744019
validation[ , genome 6371] predictions: 6509/ 10000 (65.09%), best: 6509/10000 (65.09%), error: 10245.30957, best error: 10245.30957 on epoch: 36, epoch: 36/50, mu: 0.9047157168, learning_rate: 0.0000343402, weight_decay: 0.0000316440
epoch time: 1147.4132080078s, input_fired_time: 18.1406364441, output_fired_time: 4.8398013115, propagate_forward_time: 408.6481628418, propagate_backward_time: 694.4117431641, weight_update_time: 0.2921994925, other_time: 21.3728027344
epoch time: 87.3617630005s, input_fired_time: 1.0084458590, output_fired_time: 0.0000000000, propagate_forward_time: 82.3160171509, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0373001099
validation[ , genome 6371] predictions: 6473/ 10000 (64.73%), best: 6509/10000 (65.09%), error: 10264.06543, best error: 10245.30957 on epoch: 36, epoch: 37/50, mu: 0.9088135958, learning_rate: 0.0000315937, weight_decay: 0.0000293034
epoch time: 1147.5364990234s, input_fired_time: 18.1263351440, output_fired_time: 4.8433489799, propagate_forward_time: 408.5727233887, propagate_backward_time: 694.7598266602, weight_update_time: 0.2900631726, other_time: 21.2341918945
epoch time: 87.4019088745s, input_fired_time: 1.0141310692, output_fired_time: 0.0000000000, propagate_forward_time: 82.4944458008, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8933334351
validation[ , genome 6371] predictions: 6563/ 10000 (65.63%), best: 6563/10000 (65.63%), error: 10053.33398, best error: 10053.33398 on epoch: 38, epoch: 38/50, mu: 0.9127146006, learning_rate: 0.0000290669, weight_decay: 0.0000271359
epoch time: 1146.9150390625s, input_fired_time: 18.1467781067, output_fired_time: 4.8483967781, propagate_forward_time: 408.5712890625, propagate_backward_time: 694.0884399414, weight_update_time: 0.2901195884, other_time: 21.2601928711
epoch time: 87.3743591309s, input_fired_time: 1.0164026022, output_fired_time: 0.0000000000, propagate_forward_time: 82.3982925415, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9596633911
validation[ , genome 6371] predictions: 6478/ 10000 (64.78%), best: 6563/10000 (65.63%), error: 10206.45996, best error: 10053.33398 on epoch: 38, epoch: 39/50, mu: 0.9164281487, learning_rate: 0.0000267421, weight_decay: 0.0000251287
epoch time: 1147.5771484375s, input_fired_time: 18.1520671844, output_fired_time: 4.8403801918, propagate_forward_time: 408.4711608887, propagate_backward_time: 694.9101562500, weight_update_time: 0.2931480110, other_time: 21.2033691406
epoch time: 87.4002838135s, input_fired_time: 1.0148421526, output_fired_time: 0.0000000000, propagate_forward_time: 82.3748550415, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0105895996
validation[ , genome 6371] predictions: 6641/ 10000 (66.41%), best: 6641/10000 (66.41%), error: 9956.67383, best error: 9956.67383 on epoch: 40, epoch: 40/50, mu: 0.9199632406, learning_rate: 0.0000246033, weight_decay: 0.0000232700
epoch time: 1148.0306396484s, input_fired_time: 18.1579875946, output_fired_time: 4.8533577919, propagate_forward_time: 408.6982116699, propagate_backward_time: 695.0195312500, weight_update_time: 0.2934254408, other_time: 21.3015136719
epoch time: 87.4910049438s, input_fired_time: 1.0205522776, output_fired_time: 0.0000000000, propagate_forward_time: 82.4581832886, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0122680664
validation[ , genome 6371] predictions: 6562/ 10000 (65.62%), best: 6641/10000 (66.41%), error: 10104.66504, best error: 9956.67383 on epoch: 40, epoch: 41/50, mu: 0.9233285189, learning_rate: 0.0000226355, weight_decay: 0.0000215488
epoch time: 1157.7810058594s, input_fired_time: 18.2658710480, output_fired_time: 4.8932256699, propagate_forward_time: 412.4824523926, propagate_backward_time: 701.1903686523, weight_update_time: 0.3018700480, other_time: 20.9491577148
epoch time: 87.5384445190s, input_fired_time: 1.0223317146, output_fired_time: 0.0000000000, propagate_forward_time: 82.6085052490, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9076080322
validation[ , genome 6371] predictions: 6497/ 10000 (64.97%), best: 6641/10000 (66.41%), error: 10154.91016, best error: 9956.67383 on epoch: 40, epoch: 42/50, mu: 0.9265320897, learning_rate: 0.0000208252, weight_decay: 0.0000199549
epoch time: 1153.2415771484s, input_fired_time: 18.2422904968, output_fired_time: 4.8886790276, propagate_forward_time: 411.1188964844, propagate_backward_time: 697.9524536133, weight_update_time: 0.2991687953, other_time: 21.0392456055
epoch time: 87.6751098633s, input_fired_time: 1.0213476419, output_fired_time: 0.0000000000, propagate_forward_time: 82.7393341064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9144287109
validation[ , genome 6371] predictions: 6340/ 10000 (63.40%), best: 6641/10000 (66.41%), error: 10515.34668, best error: 9956.67383 on epoch: 40, epoch: 43/50, mu: 0.9295817018, learning_rate: 0.0000191596, weight_decay: 0.0000184789
epoch time: 1148.9958496094s, input_fired_time: 18.1948242188, output_fired_time: 4.8585209846, propagate_forward_time: 409.1729125977, propagate_backward_time: 695.4137573242, weight_update_time: 0.2941549420, other_time: 21.3558349609
epoch time: 87.4432373047s, input_fired_time: 1.0196309090, output_fired_time: 0.0000000000, propagate_forward_time: 82.3594665527, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0641403198
validation[ , genome 6371] predictions: 6595/ 10000 (65.95%), best: 6641/10000 (66.41%), error: 9999.88281, best error: 9956.67383 on epoch: 40, epoch: 44/50, mu: 0.9324848056, learning_rate: 0.0000176272, weight_decay: 0.0000171121
epoch time: 1147.7321777344s, input_fired_time: 18.2007732391, output_fired_time: 4.8475937843, propagate_forward_time: 408.6801757813, propagate_backward_time: 694.5064086914, weight_update_time: 0.2922809124, other_time: 21.4972534180
epoch time: 87.8004379272s, input_fired_time: 1.0277490616, output_fired_time: 0.0000000000, propagate_forward_time: 82.8738479614, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8988418579
validation[ , genome 6371] predictions: 6487/ 10000 (64.87%), best: 6641/10000 (66.41%), error: 10212.01758, best error: 9956.67383 on epoch: 40, epoch: 45/50, mu: 0.9352483749, learning_rate: 0.0000162174, weight_decay: 0.0000158463
epoch time: 1159.9270019531s, input_fired_time: 18.3275909424, output_fired_time: 4.9357042313, propagate_forward_time: 413.8521423340, propagate_backward_time: 702.3336791992, weight_update_time: 0.3091780543, other_time: 20.4778442383
epoch time: 87.6378479004s, input_fired_time: 1.0256141424, output_fired_time: 0.0000000000, propagate_forward_time: 82.6838150024, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9284210205
validation[ , genome 6371] predictions: 6609/ 10000 (66.09%), best: 6641/10000 (66.41%), error: 9974.49219, best error: 9956.67383 on epoch: 40, epoch: 46/50, mu: 0.9378792048, learning_rate: 0.0000149203, weight_decay: 0.0000146742
epoch time: 1149.8104248047s, input_fired_time: 18.2301712036, output_fired_time: 4.8575124741, propagate_forward_time: 409.8005676270, propagate_backward_time: 695.7481079102, weight_update_time: 0.2927389741, other_time: 21.1740112305
epoch time: 87.3546295166s, input_fired_time: 1.0272654295, output_fired_time: 0.0000000000, propagate_forward_time: 82.4702453613, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8571166992
validation[ , genome 6371] predictions: 6632/ 10000 (66.32%), best: 6632/10000 (66.32%), error: 9891.79883, best error: 9891.79883 on epoch: 47, epoch: 47/50, mu: 0.9403836131, learning_rate: 0.0000137270, weight_decay: 0.0000135888
epoch time: 1149.5240478516s, input_fired_time: 18.2407341003, output_fired_time: 4.8746290207, propagate_forward_time: 409.6575317383, propagate_backward_time: 695.9300537109, weight_update_time: 0.2962021232, other_time: 20.8211059570
epoch time: 87.4011840820s, input_fired_time: 1.0293641090, output_fired_time: 0.0000000000, propagate_forward_time: 82.4924697876, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8793487549
validation[ , genome 6371] predictions: 6635/ 10000 (66.35%), best: 6632/10000 (66.32%), error: 9944.47363, best error: 9891.79883 on epoch: 47, epoch: 48/50, mu: 0.9427676797, learning_rate: 0.0000126291, weight_decay: 0.0000125837
epoch time: 1150.5384521484s, input_fired_time: 18.2545528412, output_fired_time: 4.8653192520, propagate_forward_time: 409.6050415039, propagate_backward_time: 697.1302490234, weight_update_time: 0.3000155985, other_time: 20.6832885742
epoch time: 87.6065368652s, input_fired_time: 1.0302944183, output_fired_time: 0.0000000000, propagate_forward_time: 82.6895751953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8866653442
validation[ , genome 6371] predictions: 6603/ 10000 (66.03%), best: 6632/10000 (66.32%), error: 9939.00879, best error: 9891.79883 on epoch: 47, epoch: 49/50, mu: 0.9450371861, learning_rate: 0.0000116191, weight_decay: 0.0000116529
epoch time: 1150.2725830078s, input_fired_time: 18.2920379639, output_fired_time: 4.8715543747, propagate_forward_time: 409.6109313965, propagate_backward_time: 696.6345825195, weight_update_time: 0.2971815169, other_time: 20.8634643555
epoch time: 87.4601364136s, input_fired_time: 1.0250747204, output_fired_time: 0.0000000000, propagate_forward_time: 82.4362716675, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9987869263
validation[ , genome 6371] predictions: 6554/ 10000 (65.54%), best: 6632/10000 (66.32%), error: 10078.63184, best error: 9891.79883 on epoch: 47, epoch: 50/50, mu: 0.9471976161, learning_rate: 0.0000106898, weight_decay: 0.0000107910
evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 433.8759765625s, input_fired_time: 5.1069622040, output_fired_time: 0.0000000000, propagate_forward_time: 409.0378417969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.7311706543
best training[ , genome 6371] predictions: 35672/ 50000 (71.34%), best: 6632/10000 (66.32%), error: 43073.14063, best error: 9891.79883 on epoch: 47, epoch: 51/50, mu: 0.9492542744, learning_rate: 0.0000098348, weight_decay: 0.0000099928
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 87.4531173706s, input_fired_time: 1.0260245800, output_fired_time: 0.0000000000, propagate_forward_time: 82.4395980835, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9874954224
testing[ , genome 6371] predictions: 6632/ 10000 (66.32%), best: 6632/10000 (66.32%), error: 9891.79883, best error: 9891.79883 on epoch: 47, epoch: 51/50, mu: 0.9492542744, learning_rate: 0.0000098348, weight_decay: 0.0000099928
backpropagation finished successfully!
10:34:02 (10724): called boinc_finish(0)
</stderr_txt>
]]>