Nameexact_genome_1540839302_185_5467_0
Workunit4078257
Created29 Oct 2018, 20:57:35 UTC
Sent29 Oct 2018, 22:45:46 UTC
Report deadline3 Jan 2019, 16:39:08 UTC
Received31 Oct 2018, 18:42:18 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID84153
Run time1 days 3 hours 34 min 19 sec
CPU time1 days 2 hours 47 min 42 sec
Validate stateValid
Credit16,436.35
Device peak FLOPS4.74 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size505.75 MB
Peak swap size573.19 MB
Peak disk usage4.65 MB

Stderr output

<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
84, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0905990601
validation[          , genome  5467] predictions:    5540/  10000 (55.40%), best:    5527/10000 (55.27%), error:     13725.64746, best error:     13603.43945 on epoch:    26, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 970.2706298828s, input_fired_time: 7.4697289467, output_fired_time: 5.9959392548, propagate_forward_time: 363.3443603516, propagate_backward_time: 580.9332885742, weight_update_time: 0.7412048578, other_time: 12.5273437500
epoch time: 73.8354644775s, input_fired_time: 0.7599815726, output_fired_time: 0.0000000000, propagate_forward_time: 71.0772018433, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9982833862
validation[          , genome  5467] predictions:    5524/  10000 (55.24%), best:    5524/10000 (55.24%), error:     13362.96680, best error:     13362.96680 on epoch:    28, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 974.8796386719s, input_fired_time: 7.5485510826, output_fired_time: 6.0387115479, propagate_forward_time: 353.0956420898, propagate_backward_time: 592.7220458984, weight_update_time: 0.7427353263, other_time: 15.4747314453
epoch time: 72.7214126587s, input_fired_time: 0.7768909931, output_fired_time: 0.0000000000, propagate_forward_time: 69.8785781860, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0659408569
validation[          , genome  5467] predictions:    5508/  10000 (55.08%), best:    5524/10000 (55.24%), error:     13540.64648, best error:     13362.96680 on epoch:    28, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 969.3430786133s, input_fired_time: 6.4102349281, output_fired_time: 5.9046139717, propagate_forward_time: 363.3217773438, propagate_backward_time: 581.4658203125, weight_update_time: 0.7244656682, other_time: 12.2406616211
epoch time: 73.9371795654s, input_fired_time: 0.7662425637, output_fired_time: 0.0000000000, propagate_forward_time: 70.1464080811, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0245285034
validation[          , genome  5467] predictions:    5547/  10000 (55.47%), best:    5524/10000 (55.24%), error:     13369.14648, best error:     13362.96680 on epoch:    28, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 969.4780273438s, input_fired_time: 7.5443620682, output_fired_time: 4.9686598778, propagate_forward_time: 357.1195678711, propagate_backward_time: 587.3923950195, weight_update_time: 0.7293716669, other_time: 12.4530029297
epoch time: 74.8601989746s, input_fired_time: 0.8129065037, output_fired_time: 0.0000000000, propagate_forward_time: 69.9071578979, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1401367188
validation[          , genome  5467] predictions:    5547/  10000 (55.47%), best:    5524/10000 (55.24%), error:     13574.21973, best error:     13362.96680 on epoch:    28, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 963.9734497070s, input_fired_time: 6.4659838676, output_fired_time: 4.9721484184, propagate_forward_time: 352.0850219727, propagate_backward_time: 587.8624267578, weight_update_time: 0.7392386198, other_time: 12.5878295898
epoch time: 73.8572463989s, input_fired_time: 0.7888385057, output_fired_time: 0.0000000000, propagate_forward_time: 70.9685821533, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0998229980
validation[          , genome  5467] predictions:    5698/  10000 (56.98%), best:    5698/10000 (56.98%), error:     13014.59766, best error:     13014.59766 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 973.4531250000s, input_fired_time: 6.5334553719, output_fired_time: 5.0323772430, propagate_forward_time: 357.2240905762, propagate_backward_time: 591.1688842773, weight_update_time: 0.7340192795, other_time: 13.4943237305
epoch time: 76.1560134888s, input_fired_time: 0.7950673699, output_fired_time: 0.0000000000, propagate_forward_time: 73.2181472778, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1427993774
validation[          , genome  5467] predictions:    5525/  10000 (55.25%), best:    5698/10000 (56.98%), error:     13479.49414, best error:     13014.59766 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 985.9138183594s, input_fired_time: 6.6273636818, output_fired_time: 6.0487427711, propagate_forward_time: 355.5079345703, propagate_backward_time: 601.1509399414, weight_update_time: 0.7367594838, other_time: 16.5787963867
epoch time: 74.9425659180s, input_fired_time: 0.7895423770, output_fired_time: 0.0000000000, propagate_forward_time: 72.0619430542, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0910797119
validation[          , genome  5467] predictions:    5561/  10000 (55.61%), best:    5698/10000 (56.98%), error:     13397.25879, best error:     13014.59766 on epoch:    32, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 969.1865844727s, input_fired_time: 6.4989857674, output_fired_time: 5.9890394211, propagate_forward_time: 354.9505004883, propagate_backward_time: 589.2411499023, weight_update_time: 0.7428590655, other_time: 12.5069580078
epoch time: 73.1759262085s, input_fired_time: 0.7928622365, output_fired_time: 0.0000000000, propagate_forward_time: 69.2938079834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0892562866
validation[          , genome  5467] predictions:    5667/  10000 (56.67%), best:    5698/10000 (56.98%), error:     13252.75879, best error:     13014.59766 on epoch:    32, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 960.8235473633s, input_fired_time: 6.5438857079, output_fired_time: 4.9944877625, propagate_forward_time: 347.8630676270, propagate_backward_time: 589.8486938477, weight_update_time: 0.7415560484, other_time: 11.5734252930
epoch time: 77.6033782959s, input_fired_time: 0.8113231659, output_fired_time: 0.0000000000, propagate_forward_time: 73.6507644653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1412887573
validation[          , genome  5467] predictions:    5505/  10000 (55.05%), best:    5698/10000 (56.98%), error:     13564.84277, best error:     13014.59766 on epoch:    32, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 971.4213867188s, input_fired_time: 6.6131582260, output_fired_time: 7.0699243546, propagate_forward_time: 349.9487304688, propagate_backward_time: 594.0330200195, weight_update_time: 1.7461295128, other_time: 13.7565307617
epoch time: 72.9613189697s, input_fired_time: 0.8167804480, output_fired_time: 0.0000000000, propagate_forward_time: 69.0348663330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1096725464
validation[          , genome  5467] predictions:    5678/  10000 (56.78%), best:    5678/10000 (56.78%), error:     12873.33203, best error:     12873.33203 on epoch:    37, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1540839302_185_5467.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1540839302_185_5467_0_r1100916434_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.920226
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00177995
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.11978e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 38
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 37
read number_validation_images: 10000
read best_validation_predictions: 5678
read best_validation_error: 12873.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5467
read normal distribution: '1 -0.449621319770813 -0.566973090171814'
generator_str: '317633856'
read generator: 317633856
read generated_by_map:
1 add_node 1
reading 70 nodes.
reading 261 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 972.603942871094s, input_fired_time: 7.64977836608887, output_fired_time: 7.02581453323364, propagate_forward_time: 357.106872558594, propagate_backward_time: 589.221801757812, weight_update_time: 0.750676989555359, other_time: 11.5996704101562
epoch time: 72.9001617431641s, input_fired_time: 0.824296951293945, output_fired_time: 0, propagate_forward_time: 69.9148483276367, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.16101837158203
validation[          , genome  5467] predictions:    5736/  10000 (57.36%), best:    5678/10000 (56.78%), error:     12932.62305, best error:     12873.33203 on epoch:    37, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 974.2667846680s, input_fired_time: 6.7322378159, output_fired_time: 5.0359129906, propagate_forward_time: 362.6754455566, propagate_backward_time: 587.1257934570, weight_update_time: 0.7494527102, other_time: 12.6974487305
epoch time: 72.5876770020s, input_fired_time: 0.7972097993, output_fired_time: 0.0000000000, propagate_forward_time: 69.7631759644, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0272903442
validation[          , genome  5467] predictions:    5714/  10000 (57.14%), best:    5678/10000 (56.78%), error:     12909.81836, best error:     12873.33203 on epoch:    37, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 996.7604980469s, input_fired_time: 6.8772802353, output_fired_time: 6.1489944458, propagate_forward_time: 363.9857788086, propagate_backward_time: 607.9216918945, weight_update_time: 0.7576647401, other_time: 11.8267822266
epoch time: 78.3293762207s, input_fired_time: 1.8406076431, output_fired_time: 0.0000000000, propagate_forward_time: 74.3115692139, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1772003174
validation[          , genome  5467] predictions:    5586/  10000 (55.86%), best:    5678/10000 (56.78%), error:     13146.46777, best error:     12873.33203 on epoch:    37, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1002.0904541016s, input_fired_time: 6.8815402985, output_fired_time: 6.2552790642, propagate_forward_time: 362.5360717773, propagate_backward_time: 613.2962646484, weight_update_time: 0.7791115642, other_time: 13.1213378906
epoch time: 72.1348190308s, input_fired_time: 0.8152871728, output_fired_time: 0.0000000000, propagate_forward_time: 69.2133331299, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1062011719
validation[          , genome  5467] predictions:    5734/  10000 (57.34%), best:    5678/10000 (56.78%), error:     13068.84180, best error:     12873.33203 on epoch:    37, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 969.7642822266s, input_fired_time: 6.7907505035, output_fired_time: 5.0717368126, propagate_forward_time: 348.9492797852, propagate_backward_time: 596.0603637695, weight_update_time: 0.7749479413, other_time: 12.8921508789
epoch time: 74.8974151611s, input_fired_time: 0.8664048314, output_fired_time: 0.0000000000, propagate_forward_time: 71.8119277954, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2190856934
validation[          , genome  5467] predictions:    5644/  10000 (56.44%), best:    5678/10000 (56.78%), error:     13302.89258, best error:     12873.33203 on epoch:    37, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 984.6837768555s, input_fired_time: 6.8557095528, output_fired_time: 6.1992979050, propagate_forward_time: 360.1035766602, propagate_backward_time: 596.3081054688, weight_update_time: 0.7845429182, other_time: 15.2171020508
epoch time: 75.1780624390s, input_fired_time: 0.8591517806, output_fired_time: 0.0000000000, propagate_forward_time: 72.1652145386, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1536941528
validation[          , genome  5467] predictions:    5694/  10000 (56.94%), best:    5678/10000 (56.78%), error:     13139.22656, best error:     12873.33203 on epoch:    37, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 984.6743164062s, input_fired_time: 6.9120464325, output_fired_time: 8.2275619507, propagate_forward_time: 352.3731689453, propagate_backward_time: 604.0504760742, weight_update_time: 1.7911194563, other_time: 13.1110839844
epoch time: 75.3140945435s, input_fired_time: 0.8751674294, output_fired_time: 0.0000000000, propagate_forward_time: 72.2938232422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1451034546
validation[          , genome  5467] predictions:    5657/  10000 (56.57%), best:    5678/10000 (56.78%), error:     13352.19824, best error:     12873.33203 on epoch:    37, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 976.5819091797s, input_fired_time: 8.8104286194, output_fired_time: 5.1620092392, propagate_forward_time: 352.6396789551, propagate_backward_time: 597.9018554688, weight_update_time: 0.7943246961, other_time: 12.0679931641
epoch time: 74.0899353027s, input_fired_time: 2.8547036648, output_fired_time: 0.0000000000, propagate_forward_time: 69.0763549805, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1588745117
validation[          , genome  5467] predictions:    5720/  10000 (57.20%), best:    5678/10000 (56.78%), error:     13146.31836, best error:     12873.33203 on epoch:    37, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 983.9431152344s, input_fired_time: 6.9497938156, output_fired_time: 5.2076454163, propagate_forward_time: 362.1397399902, propagate_backward_time: 596.1609497070, weight_update_time: 0.8117767572, other_time: 13.4850463867
epoch time: 74.3000793457s, input_fired_time: 0.8703129292, output_fired_time: 0.0000000000, propagate_forward_time: 71.2302246094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1995391846
validation[          , genome  5467] predictions:    5735/  10000 (57.35%), best:    5678/10000 (56.78%), error:     13069.86816, best error:     12873.33203 on epoch:    37, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 965.7618408203s, input_fired_time: 6.7532305717, output_fired_time: 7.1380615234, propagate_forward_time: 346.1333312988, propagate_backward_time: 592.8850708008, weight_update_time: 0.7700053453, other_time: 12.8521118164
epoch time: 75.2899246216s, input_fired_time: 0.8699927330, output_fired_time: 0.0000000000, propagate_forward_time: 72.2148971558, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2050323486
validation[          , genome  5467] predictions:    5781/  10000 (57.81%), best:    5781/10000 (57.81%), error:     12799.43164, best error:     12799.43164 on epoch:    47, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 969.8724975586s, input_fired_time: 7.8120589256, output_fired_time: 6.0666255951, propagate_forward_time: 352.4209899902, propagate_backward_time: 590.5562744141, weight_update_time: 0.7660622001, other_time: 13.0164794922
epoch time: 74.8935012817s, input_fired_time: 0.8522318602, output_fired_time: 0.0000000000, propagate_forward_time: 71.8908920288, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1503753662
validation[          , genome  5467] predictions:    5729/  10000 (57.29%), best:    5781/10000 (57.81%), error:     12875.48828, best error:     12799.43164 on epoch:    47, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 971.7022705078s, input_fired_time: 7.8854298592, output_fired_time: 7.1519641876, propagate_forward_time: 349.2562255859, propagate_backward_time: 594.3018188477, weight_update_time: 0.7842581868, other_time: 13.1068115234
epoch time: 75.2784118652s, input_fired_time: 0.8830114603, output_fired_time: 0.0000000000, propagate_forward_time: 72.1577911377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2376098633
validation[          , genome  5467] predictions:    5782/  10000 (57.82%), best:    5781/10000 (57.81%), error:     13190.48340, best error:     12799.43164 on epoch:    47, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 968.4125976562s, input_fired_time: 8.9402494431, output_fired_time: 7.1976647377, propagate_forward_time: 355.8591918945, propagate_backward_time: 583.9979248047, weight_update_time: 0.8084521890, other_time: 12.4175415039
epoch time: 75.6350326538s, input_fired_time: 0.9278486371, output_fired_time: 0.0000000000, propagate_forward_time: 72.3754425049, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3317413330
validation[          , genome  5467] predictions:    5730/  10000 (57.30%), best:    5781/10000 (57.81%), error:     12809.64062, best error:     12799.43164 on epoch:    47, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 980.2856445312s, input_fired_time: 8.0844326019, output_fired_time: 5.2521829605, propagate_forward_time: 358.1698913574, propagate_backward_time: 594.1596679688, weight_update_time: 0.8241132498, other_time: 14.6195068359
epoch time: 75.4565353394s, input_fired_time: 0.9426490664, output_fired_time: 0.0000000000, propagate_forward_time: 71.1746063232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3392791748
validation[          , genome  5467] predictions:    5722/  10000 (57.22%), best:    5781/10000 (57.81%), error:     12866.02441, best error:     12799.43164 on epoch:    47, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1540839302_185_5467.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1540839302_185_5467_0_r1100916434_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.955973
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000868035
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 3.47214e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 52
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 47
read number_validation_images: 10000
read best_validation_predictions: 5781
read best_validation_error: 12799.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5467
read normal distribution: '1 -0.449621319770813 -0.566973090171814'
generator_str: '356813583'
read generator: 356813583
read generated_by_map:
1 add_node 1
reading 70 nodes.
reading 261 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 980.397521972656s, input_fired_time: 10.1345148086548, output_fired_time: 6.25803136825562, propagate_forward_time: 355.377655029297, propagate_backward_time: 595.096252441406, weight_update_time: 0.82041996717453, other_time: 13.5310668945312
epoch time: 75.3743438720703s, input_fired_time: 0.957532823085785, output_fired_time: 0, propagate_forward_time: 72.0746994018555, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.34210968017578
validation[          , genome  5467] predictions:    5804/  10000 (58.04%), best:    5804/10000 (58.04%), error:     12699.37109, best error:     12699.37109 on epoch:    52, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 993.2050170898s, input_fired_time: 8.2455511093, output_fired_time: 5.4314503670, propagate_forward_time: 360.2510681152, propagate_backward_time: 606.2053222656, weight_update_time: 0.8565708995, other_time: 13.0716552734
epoch time: 76.1718368530s, input_fired_time: 0.9111104012, output_fired_time: 0.0000000000, propagate_forward_time: 71.8998641968, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3608627319
validation[          , genome  5467] predictions:    5727/  10000 (57.27%), best:    5804/10000 (58.04%), error:     12938.21680, best error:     12699.37109 on epoch:    52, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 989.1127929688s, input_fired_time: 7.2863211632, output_fired_time: 5.4211940765, propagate_forward_time: 359.7041320801, propagate_backward_time: 603.7690429688, weight_update_time: 0.8441557884, other_time: 12.9321289062
epoch time: 78.7339859009s, input_fired_time: 1.0136561394, output_fired_time: 0.0000000000, propagate_forward_time: 75.2895965576, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4307327271
validation[          , genome  5467] predictions:    5856/  10000 (58.56%), best:    5856/10000 (58.56%), error:     12385.46387, best error:     12385.46387 on epoch:    54, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 985.1138305664s, input_fired_time: 7.1499714851, output_fired_time: 5.3264651299, propagate_forward_time: 359.5608825684, propagate_backward_time: 600.4078369141, weight_update_time: 0.8353419304, other_time: 12.6687011719
epoch time: 77.0360488892s, input_fired_time: 0.9824982882, output_fired_time: 0.0000000000, propagate_forward_time: 73.6758117676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3777389526
validation[          , genome  5467] predictions:    5851/  10000 (58.51%), best:    5856/10000 (58.56%), error:     12407.50977, best error:     12385.46387 on epoch:    54, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 987.5412597656s, input_fired_time: 7.1337838173, output_fired_time: 5.3630394936, propagate_forward_time: 365.8040161133, propagate_backward_time: 595.5095825195, weight_update_time: 0.8258560896, other_time: 13.7308349609
epoch time: 74.7448577881s, input_fired_time: 0.8724572659, output_fired_time: 0.0000000000, propagate_forward_time: 71.6773986816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1949996948
validation[          , genome  5467] predictions:    5919/  10000 (59.19%), best:    5856/10000 (58.56%), error:     12423.34375, best error:     12385.46387 on epoch:    54, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 988.8774414062s, input_fired_time: 7.1705241203, output_fired_time: 5.4142570496, propagate_forward_time: 361.4329528809, propagate_backward_time: 600.9616088867, weight_update_time: 0.8390865922, other_time: 13.8981323242
epoch time: 73.9383010864s, input_fired_time: 0.8761928082, output_fired_time: 0.0000000000, propagate_forward_time: 70.8266601562, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2354507446
validation[          , genome  5467] predictions:    5757/  10000 (57.57%), best:    5856/10000 (58.56%), error:     12803.65820, best error:     12385.46387 on epoch:    54, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1000.0253906250s, input_fired_time: 8.2441024780, output_fired_time: 6.4745035172, propagate_forward_time: 370.4381103516, propagate_backward_time: 600.7734985352, weight_update_time: 0.8484047651, other_time: 14.0952148438
epoch time: 74.2235870361s, input_fired_time: 0.9243447781, output_fired_time: 0.0000000000, propagate_forward_time: 71.0022659302, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2969741821
validation[          , genome  5467] predictions:    5911/  10000 (59.11%), best:    5856/10000 (58.56%), error:     12449.89062, best error:     12385.46387 on epoch:    54, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 997.9108276367s, input_fired_time: 8.2944993973, output_fired_time: 6.4107580185, propagate_forward_time: 359.6041564941, propagate_backward_time: 609.5836181641, weight_update_time: 0.8505567312, other_time: 14.0178222656
epoch time: 74.4295501709s, input_fired_time: 0.9242671728, output_fired_time: 0.0000000000, propagate_forward_time: 71.2352066040, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2700729370
validation[          , genome  5467] predictions:    5845/  10000 (58.45%), best:    5856/10000 (58.56%), error:     12516.62695, best error:     12385.46387 on epoch:    54, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 986.8127441406s, input_fired_time: 8.2183084488, output_fired_time: 5.3285965919, propagate_forward_time: 359.8554382324, propagate_backward_time: 599.5457763672, weight_update_time: 0.8417335749, other_time: 13.8646240234
epoch time: 74.1723327637s, input_fired_time: 0.9106618166, output_fired_time: 0.0000000000, propagate_forward_time: 71.0111541748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2505187988
validation[          , genome  5467] predictions:    5838/  10000 (58.38%), best:    5856/10000 (58.56%), error:     12681.81543, best error:     12385.46387 on epoch:    54, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 995.7355346680s, input_fired_time: 7.2334079742, output_fired_time: 6.4190845490, propagate_forward_time: 357.0119628906, propagate_backward_time: 612.2116699219, weight_update_time: 0.8350529075, other_time: 12.8594360352
epoch time: 78.9632034302s, input_fired_time: 0.9533965588, output_fired_time: 0.0000000000, propagate_forward_time: 75.5629959106, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4468078613
validation[          , genome  5467] predictions:    5931/  10000 (59.31%), best:    5856/10000 (58.56%), error:     12462.95703, best error:     12385.46387 on epoch:    54, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 993.2946166992s, input_fired_time: 9.1451435089, output_fired_time: 6.3500638008, propagate_forward_time: 364.3095397949, propagate_backward_time: 599.5222778320, weight_update_time: 0.8659824133, other_time: 13.9675903320
epoch time: 77.5325241089s, input_fired_time: 0.9276469350, output_fired_time: 0.0000000000, propagate_forward_time: 74.2647247314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3401489258
validation[          , genome  5467] predictions:    5949/  10000 (59.49%), best:    5949/10000 (59.49%), error:     12076.39258, best error:     12076.39258 on epoch:    62, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 990.8989257812s, input_fired_time: 9.2509574890, output_fired_time: 6.4300856590, propagate_forward_time: 361.0362854004, propagate_backward_time: 601.2366333008, weight_update_time: 0.8447399139, other_time: 12.9448852539
epoch time: 78.0998840332s, input_fired_time: 0.9904415011, output_fired_time: 0.0000000000, propagate_forward_time: 73.6238784790, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4855651855
validation[          , genome  5467] predictions:    5883/  10000 (58.83%), best:    5949/10000 (59.49%), error:     12439.35742, best error:     12076.39258 on epoch:    62, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 997.7280883789s, input_fired_time: 7.5366897583, output_fired_time: 5.6626243591, propagate_forward_time: 360.7547302246, propagate_backward_time: 609.0339355469, weight_update_time: 0.9037493467, other_time: 14.7401123047
epoch time: 76.3368072510s, input_fired_time: 2.0127940178, output_fired_time: 0.0000000000, propagate_forward_time: 71.8748397827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4491729736
validation[          , genome  5467] predictions:    5930/  10000 (59.30%), best:    5949/10000 (59.49%), error:     12386.18262, best error:     12076.39258 on epoch:    62, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 1002.8668823242s, input_fired_time: 8.5766830444, output_fired_time: 6.6783208847, propagate_forward_time: 358.6285705566, propagate_backward_time: 615.3246459961, weight_update_time: 0.9005013108, other_time: 13.6586303711
epoch time: 73.5289611816s, input_fired_time: 0.9243662357, output_fired_time: 0.0000000000, propagate_forward_time: 70.2745208740, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3300704956
validation[          , genome  5467] predictions:    5843/  10000 (58.43%), best:    5949/10000 (59.49%), error:     12565.95508, best error:     12076.39258 on epoch:    62, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 992.1832275391s, input_fired_time: 9.5119838715, output_fired_time: 5.6575684547, propagate_forward_time: 365.5422363281, propagate_backward_time: 597.1630859375, weight_update_time: 1.8711601496, other_time: 14.3083496094
epoch time: 76.9481735229s, input_fired_time: 0.9578552842, output_fired_time: 0.0000000000, propagate_forward_time: 73.6033020020, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3870162964
validation[          , genome  5467] predictions:    5888/  10000 (58.88%), best:    5949/10000 (59.49%), error:     12376.66211, best error:     12076.39258 on epoch:    62, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 996.3728637695s, input_fired_time: 10.4340829849, output_fired_time: 5.6026492119, propagate_forward_time: 361.0816650391, propagate_backward_time: 604.7708129883, weight_update_time: 0.8795357347, other_time: 14.4836425781
epoch time: 76.0638275146s, input_fired_time: 0.9843643904, output_fired_time: 0.0000000000, propagate_forward_time: 72.6919479370, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3875122070
validation[          , genome  5467] predictions:    5845/  10000 (58.45%), best:    5949/10000 (59.49%), error:     12521.62305, best error:     12076.39258 on epoch:    62, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 992.5577392578s, input_fired_time: 8.5329370499, output_fired_time: 5.6055879593, propagate_forward_time: 358.9974975586, propagate_backward_time: 604.9772338867, weight_update_time: 0.9458428621, other_time: 14.4444580078
epoch time: 77.0814208984s, input_fired_time: 1.0569907427, output_fired_time: 0.0000000000, propagate_forward_time: 73.4759750366, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5484542847
validation[          , genome  5467] predictions:    5807/  10000 (58.07%), best:    5949/10000 (59.49%), error:     12788.25684, best error:     12076.39258 on epoch:    62, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 1006.8656005859s, input_fired_time: 9.0095396042, output_fired_time: 8.9863033295, propagate_forward_time: 361.4617004395, propagate_backward_time: 611.9962158203, weight_update_time: 1.9620820284, other_time: 15.4118652344
epoch time: 74.0452194214s, input_fired_time: 1.0030189753, output_fired_time: 0.0000000000, propagate_forward_time: 70.6243515015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4178466797
validation[          , genome  5467] predictions:    5820/  10000 (58.20%), best:    5949/10000 (59.49%), error:     12437.45801, best error:     12076.39258 on epoch:    62, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 1023.2873535156s, input_fired_time: 8.9419374466, output_fired_time: 6.9487028122, propagate_forward_time: 365.9588623047, propagate_backward_time: 627.0838623047, weight_update_time: 0.9522235990, other_time: 14.3539428711
epoch time: 76.2567977905s, input_fired_time: 1.0276303291, output_fired_time: 0.0000000000, propagate_forward_time: 70.7957534790, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4334106445
validation[          , genome  5467] predictions:    5947/  10000 (59.47%), best:    5949/10000 (59.49%), error:     12354.34961, best error:     12076.39258 on epoch:    62, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 1005.6963500977s, input_fired_time: 7.8526763916, output_fired_time: 5.8494071960, propagate_forward_time: 360.7637023926, propagate_backward_time: 614.0069580078, weight_update_time: 1.9439466000, other_time: 17.2236328125
epoch time: 76.6544189453s, input_fired_time: 1.0362529755, output_fired_time: 0.0000000000, propagate_forward_time: 73.1310653687, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4870986938
validation[          , genome  5467] predictions:    5961/  10000 (59.61%), best:    5949/10000 (59.49%), error:     12117.09961, best error:     12076.39258 on epoch:    62, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 1013.4473876953s, input_fired_time: 8.0562744141, output_fired_time: 6.0024089813, propagate_forward_time: 368.0464477539, propagate_backward_time: 613.9865722656, weight_update_time: 1.9685971737, other_time: 17.3557128906
epoch time: 76.2362289429s, input_fired_time: 2.0282456875, output_fired_time: 0.0000000000, propagate_forward_time: 71.6943740845, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5136108398
validation[          , genome  5467] predictions:    5955/  10000 (59.55%), best:    5949/10000 (59.49%), error:     12206.48633, best error:     12076.39258 on epoch:    62, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 1006.0261840820s, input_fired_time: 7.8233318329, output_fired_time: 5.9152235985, propagate_forward_time: 373.3651123047, propagate_backward_time: 601.7554931641, weight_update_time: 0.9380042553, other_time: 17.1670532227
epoch time: 74.9293365479s, input_fired_time: 1.0354981422, output_fired_time: 0.0000000000, propagate_forward_time: 71.4318542480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4619827271
validation[          , genome  5467] predictions:    5925/  10000 (59.25%), best:    5949/10000 (59.49%), error:     12234.98535, best error:     12076.39258 on epoch:    62, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 1012.1833496094s, input_fired_time: 8.9741621017, output_fired_time: 5.9880871773, propagate_forward_time: 365.4931640625, propagate_backward_time: 616.3638305664, weight_update_time: 0.9763873219, other_time: 15.3640747070
epoch time: 75.0260696411s, input_fired_time: 1.0122724771, output_fired_time: 0.0000000000, propagate_forward_time: 71.5587997437, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4549942017
validation[          , genome  5467] predictions:    5916/  10000 (59.16%), best:    5949/10000 (59.49%), error:     12327.47559, best error:     12076.39258 on epoch:    62, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 1000.5948486328s, input_fired_time: 7.8801984787, output_fired_time: 5.8624472618, propagate_forward_time: 355.9156799316, propagate_backward_time: 617.0000000000, weight_update_time: 0.9277139902, other_time: 13.9365234375
epoch time: 76.0526504517s, input_fired_time: 1.0496833324, output_fired_time: 0.0000000000, propagate_forward_time: 72.5418548584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4611129761
validation[          , genome  5467] predictions:    5897/  10000 (58.97%), best:    5949/10000 (59.49%), error:     12428.11523, best error:     12076.39258 on epoch:    62, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 1011.8046875000s, input_fired_time: 8.0287618637, output_fired_time: 6.0212755203, propagate_forward_time: 367.7403869629, propagate_backward_time: 614.5880737305, weight_update_time: 0.9637854695, other_time: 15.4262084961
epoch time: 76.0937805176s, input_fired_time: 1.0984313488, output_fired_time: 0.0000000000, propagate_forward_time: 71.4409942627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5543518066
validation[          , genome  5467] predictions:    5942/  10000 (59.42%), best:    5949/10000 (59.49%), error:     12270.53613, best error:     12076.39258 on epoch:    62, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 1006.6124877930s, input_fired_time: 7.7892556190, output_fired_time: 5.7768216133, propagate_forward_time: 359.1228942871, propagate_backward_time: 619.8600463867, weight_update_time: 0.9431027770, other_time: 14.0635375977
epoch time: 75.1271743774s, input_fired_time: 0.9856415987, output_fired_time: 0.0000000000, propagate_forward_time: 71.7601928711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3813400269
validation[          , genome  5467] predictions:    6030/  10000 (60.30%), best:    6030/10000 (60.30%), error:     11982.94531, best error:     11982.94531 on epoch:    77, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 994.2217407227s, input_fired_time: 7.5894193649, output_fired_time: 5.7382555008, propagate_forward_time: 351.8575744629, propagate_backward_time: 614.4988403320, weight_update_time: 0.9105216861, other_time: 14.5376586914
epoch time: 75.6318817139s, input_fired_time: 1.0085684061, output_fired_time: 0.0000000000, propagate_forward_time: 71.1899414062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4333724976
validation[          , genome  5467] predictions:    6028/  10000 (60.28%), best:    6028/10000 (60.28%), error:     11894.58887, best error:     11894.58887 on epoch:    78, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1023.9152221680s, input_fired_time: 7.9230194092, output_fired_time: 7.9149975777, propagate_forward_time: 372.2656250000, propagate_backward_time: 618.6939086914, weight_update_time: 0.9478487372, other_time: 17.1176757812
epoch time: 78.3634262085s, input_fired_time: 1.0079410076, output_fired_time: 0.0000000000, propagate_forward_time: 73.8708877563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4845962524
validation[          , genome  5467] predictions:    6028/  10000 (60.28%), best:    6028/10000 (60.28%), error:     11923.53516, best error:     11894.58887 on epoch:    78, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1012.4265136719s, input_fired_time: 7.9608559608, output_fired_time: 5.8055911064, propagate_forward_time: 365.3611145020, propagate_backward_time: 619.1530761719, weight_update_time: 0.9280984998, other_time: 14.1458740234
epoch time: 75.7798233032s, input_fired_time: 1.0538144112, output_fired_time: 0.0000000000, propagate_forward_time: 72.2238769531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5021286011
validation[          , genome  5467] predictions:    5915/  10000 (59.15%), best:    6028/10000 (60.28%), error:     12252.29492, best error:     11894.58887 on epoch:    78, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1006.4491577148s, input_fired_time: 7.8483934402, output_fired_time: 5.8703317642, propagate_forward_time: 360.4750366211, propagate_backward_time: 618.1302490234, weight_update_time: 0.9449466467, other_time: 14.1251220703
epoch time: 80.5243148804s, input_fired_time: 1.1051020622, output_fired_time: 0.0000000000, propagate_forward_time: 76.7528228760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6663894653
validation[          , genome  5467] predictions:    5939/  10000 (59.39%), best:    6028/10000 (60.28%), error:     12119.93652, best error:     11894.58887 on epoch:    78, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1016.9520263672s, input_fired_time: 7.9596681595, output_fired_time: 5.9132227898, propagate_forward_time: 366.7305908203, propagate_backward_time: 621.0831909180, weight_update_time: 0.9347901344, other_time: 15.2653808594
epoch time: 75.0032043457s, input_fired_time: 2.0632810593, output_fired_time: 0.0000000000, propagate_forward_time: 70.4609146118, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4790115356
validation[          , genome  5467] predictions:    5991/  10000 (59.91%), best:    6028/10000 (60.28%), error:     12072.49219, best error:     11894.58887 on epoch:    78, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1016.3798217773s, input_fired_time: 8.0923261642, output_fired_time: 7.0494041443, propagate_forward_time: 358.4690551758, propagate_backward_time: 626.4158935547, weight_update_time: 0.9641066194, other_time: 16.3531494141
epoch time: 77.7303314209s, input_fired_time: 1.0671958923, output_fired_time: 0.0000000000, propagate_forward_time: 74.1538543701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5092773438
validation[          , genome  5467] predictions:    6013/  10000 (60.13%), best:    6028/10000 (60.28%), error:     12143.08008, best error:     11894.58887 on epoch:    78, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1008.2748413086s, input_fired_time: 8.7489681244, output_fired_time: 5.7764887810, propagate_forward_time: 358.6772766113, propagate_backward_time: 616.9859619141, weight_update_time: 0.9254090190, other_time: 18.0861816406
epoch time: 75.2268829346s, input_fired_time: 1.0069319010, output_fired_time: 0.0000000000, propagate_forward_time: 71.7973327637, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4226150513
validation[          , genome  5467] predictions:    5972/  10000 (59.72%), best:    6028/10000 (60.28%), error:     12218.22461, best error:     11894.58887 on epoch:    78, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 996.3303222656s, input_fired_time: 7.6700406075, output_fired_time: 5.6013636589, propagate_forward_time: 368.0854492188, propagate_backward_time: 601.3673706055, weight_update_time: 0.8970254660, other_time: 13.6060791016
epoch time: 74.7658157349s, input_fired_time: 1.0038954020, output_fired_time: 0.0000000000, propagate_forward_time: 71.3800582886, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3818588257
validation[          , genome  5467] predictions:    5992/  10000 (59.92%), best:    6028/10000 (60.28%), error:     12079.76465, best error:     11894.58887 on epoch:    78, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1011.1515502930s, input_fired_time: 8.6863212585, output_fired_time: 7.7823185921, propagate_forward_time: 361.4215393066, propagate_backward_time: 618.5181274414, weight_update_time: 0.9156467319, other_time: 14.7432250977
epoch time: 75.8959121704s, input_fired_time: 0.9901471734, output_fired_time: 0.0000000000, propagate_forward_time: 71.5284805298, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3772811890
validation[          , genome  5467] predictions:    6012/  10000 (60.12%), best:    6028/10000 (60.28%), error:     12062.77832, best error:     11894.58887 on epoch:    78, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1007.5650634766s, input_fired_time: 8.9321718216, output_fired_time: 5.8405585289, propagate_forward_time: 361.1987915039, propagate_backward_time: 615.3636474609, weight_update_time: 0.9687123299, other_time: 16.2298583984
epoch time: 75.3163757324s, input_fired_time: 1.0822031498, output_fired_time: 0.0000000000, propagate_forward_time: 71.7355728149, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4985961914
validation[          , genome  5467] predictions:    5943/  10000 (59.43%), best:    6028/10000 (60.28%), error:     12285.59961, best error:     11894.58887 on epoch:    78, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1013.1240844727s, input_fired_time: 7.9940505028, output_fired_time: 7.0018415451, propagate_forward_time: 371.9520263672, propagate_backward_time: 610.8248901367, weight_update_time: 0.9866445065, other_time: 15.3512573242
epoch time: 76.6698226929s, input_fired_time: 1.0215942860, output_fired_time: 0.0000000000, propagate_forward_time: 71.1519699097, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4962615967
validation[          , genome  5467] predictions:    6010/  10000 (60.10%), best:    6028/10000 (60.28%), error:     12064.44727, best error:     11894.58887 on epoch:    78, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1005.4946899414s, input_fired_time: 10.8082351685, output_fired_time: 6.8121595383, propagate_forward_time: 361.3077697754, propagate_backward_time: 611.5370483398, weight_update_time: 0.9382624030, other_time: 15.0294799805
epoch time: 76.0552215576s, input_fired_time: 1.0355740786, output_fired_time: 0.0000000000, propagate_forward_time: 72.5393753052, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4802703857
validation[          , genome  5467] predictions:    6031/  10000 (60.31%), best:    6028/10000 (60.28%), error:     11942.07227, best error:     11894.58887 on epoch:    78, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1012.1218872070s, input_fired_time: 9.1442203522, output_fired_time: 7.0193657875, propagate_forward_time: 361.9366149902, propagate_backward_time: 618.5980834961, weight_update_time: 0.9758887887, other_time: 15.4236450195
epoch time: 75.9900360107s, input_fired_time: 1.0555332899, output_fired_time: 0.0000000000, propagate_forward_time: 72.4430847168, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4914169312
validation[          , genome  5467] predictions:    6026/  10000 (60.26%), best:    6028/10000 (60.28%), error:     11980.67285, best error:     11894.58887 on epoch:    78, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1009.6138916016s, input_fired_time: 7.8965063095, output_fired_time: 5.9152565002, propagate_forward_time: 365.2825012207, propagate_backward_time: 616.3952026367, weight_update_time: 0.9732214212, other_time: 14.1244506836
epoch time: 76.0487747192s, input_fired_time: 1.0938453674, output_fired_time: 0.0000000000, propagate_forward_time: 72.3596191406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5953063965
validation[          , genome  5467] predictions:    6021/  10000 (60.21%), best:    6028/10000 (60.28%), error:     12017.51855, best error:     11894.58887 on epoch:    78, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1010.8206176758s, input_fired_time: 9.0831985474, output_fired_time: 7.0117087364, propagate_forward_time: 364.8601379395, propagate_backward_time: 615.4834594727, weight_update_time: 0.9642177224, other_time: 14.3821411133
epoch time: 74.8142318726s, input_fired_time: 1.0163015127, output_fired_time: 0.0000000000, propagate_forward_time: 70.3691177368, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4288101196
validation[          , genome  5467] predictions:    6024/  10000 (60.24%), best:    6028/10000 (60.28%), error:     11935.12402, best error:     11894.58887 on epoch:    78, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1018.5870361328s, input_fired_time: 7.9385819435, output_fired_time: 7.8954916000, propagate_forward_time: 353.5359191895, propagate_backward_time: 632.9723510742, weight_update_time: 0.9195729494, other_time: 16.2446899414
epoch time: 77.3160171509s, input_fired_time: 1.0821042061, output_fired_time: 0.0000000000, propagate_forward_time: 73.6262664795, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6076431274
validation[          , genome  5467] predictions:    6005/  10000 (60.05%), best:    6028/10000 (60.28%), error:     12060.09570, best error:     11894.58887 on epoch:    78, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1010.0158691406s, input_fired_time: 9.1087169647, output_fired_time: 5.9788203239, propagate_forward_time: 368.1183776855, propagate_backward_time: 611.3461303711, weight_update_time: 0.9725418687, other_time: 15.4638061523
epoch time: 80.4901962280s, input_fired_time: 1.1227520704, output_fired_time: 0.0000000000, propagate_forward_time: 76.7625961304, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6048507690
validation[          , genome  5467] predictions:    6030/  10000 (60.30%), best:    6028/10000 (60.28%), error:     12039.24219, best error:     11894.58887 on epoch:    78, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1014.0242309570s, input_fired_time: 9.9268789291, output_fired_time: 5.8715419769, propagate_forward_time: 368.9742126465, propagate_backward_time: 614.0451049805, weight_update_time: 0.9410005808, other_time: 15.2064819336
epoch time: 75.6110382080s, input_fired_time: 1.0280544758, output_fired_time: 0.0000000000, propagate_forward_time: 72.1534881592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4294967651
validation[          , genome  5467] predictions:    6046/  10000 (60.46%), best:    6046/10000 (60.46%), error:     11847.04883, best error:     11847.04883 on epoch:    95, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1012.4818725586s, input_fired_time: 8.9781713486, output_fired_time: 7.9097023010, propagate_forward_time: 371.2906494141, propagate_backward_time: 608.9859619141, weight_update_time: 0.9538121819, other_time: 15.3173828125
epoch time: 75.7457199097s, input_fired_time: 1.1040947437, output_fired_time: 0.0000000000, propagate_forward_time: 72.0928497314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5487747192
validation[          , genome  5467] predictions:    6019/  10000 (60.19%), best:    6046/10000 (60.46%), error:     11922.32812, best error:     11847.04883 on epoch:    95, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1005.1951904297s, input_fired_time: 8.9282121658, output_fired_time: 5.8567438126, propagate_forward_time: 366.6499633789, propagate_backward_time: 608.4616699219, weight_update_time: 0.9532102942, other_time: 15.2985839844
epoch time: 76.6822509766s, input_fired_time: 1.0735944510, output_fired_time: 0.0000000000, propagate_forward_time: 73.0612716675, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5473861694
validation[          , genome  5467] predictions:    6036/  10000 (60.36%), best:    6046/10000 (60.46%), error:     11962.50586, best error:     11847.04883 on epoch:    95, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1013.4472045898s, input_fired_time: 7.9456949234, output_fired_time: 5.9116244316, propagate_forward_time: 368.4588928223, propagate_backward_time: 616.8814086914, weight_update_time: 0.9539499879, other_time: 14.2495727539
epoch time: 76.6272888184s, input_fired_time: 1.0520511866, output_fired_time: 0.0000000000, propagate_forward_time: 73.1176910400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4575500488
validation[          , genome  5467] predictions:    6038/  10000 (60.38%), best:    6046/10000 (60.46%), error:     11894.46094, best error:     11847.04883 on epoch:    95, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1005.1609497070s, input_fired_time: 7.7614383698, output_fired_time: 5.8411173820, propagate_forward_time: 360.7882690430, propagate_backward_time: 614.9123535156, weight_update_time: 0.9156553745, other_time: 15.8577880859
epoch time: 79.2402725220s, input_fired_time: 2.1007485390, output_fired_time: 0.0000000000, propagate_forward_time: 74.5981216431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5414047241
validation[          , genome  5467] predictions:    6022/  10000 (60.22%), best:    6046/10000 (60.46%), error:     11955.71484, best error:     11847.04883 on epoch:    95, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1013.9163208008s, input_fired_time: 7.9392490387, output_fired_time: 6.9208898544, propagate_forward_time: 363.0830688477, propagate_backward_time: 621.7221069336, weight_update_time: 0.9440582395, other_time: 14.2509765625
epoch time: 75.4839630127s, input_fired_time: 1.0234050751, output_fired_time: 0.0000000000, propagate_forward_time: 71.0429611206, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4175949097
validation[          , genome  5467] predictions:    5953/  10000 (59.53%), best:    6046/10000 (60.46%), error:     12081.47754, best error:     11847.04883 on epoch:    95, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 383.8760986328s, input_fired_time: 6.2306718826, output_fired_time: 0.0000000000, propagate_forward_time: 364.2085876465, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.4368286133
best training[          , genome  5467] predictions:   39286/  50000 (78.57%), best:    6046/10000 (60.46%), error:     33344.21484, best error:     11847.04883 on epoch:    95, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 77.0471801758s, input_fired_time: 1.0905473232, output_fired_time: 0.0000000000, propagate_forward_time: 72.4203109741, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5363235474
   testing[          , genome  5467] predictions:    6046/  10000 (60.46%), best:    6046/10000 (60.46%), error:     11847.04883, best error:     11847.04883 on epoch:    95, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
13:48:11 (1095): called boinc_finish(0)

</stderr_txt>
]]>