Nameexact_genome_1542006908_181_5511_0
Workunit4079393
Created12 Nov 2018, 13:44:45 UTC
Sent12 Nov 2018, 13:44:47 UTC
Report deadline6 Jan 2019, 14:11:59 UTC
Received15 Nov 2018, 21:26:32 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID13956
Run time22 hours 59 min 18 sec
CPU time21 hours 39 min 50 sec
Validate stateValid
Credit13,754.72
Device peak FLOPS2.92 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size553.26 MB
Peak swap size572.98 MB
Peak disk usage12.17 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_181_5511.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_181_5511_0_r412821258_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542006908_181_5511.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477206
read mu: 0.477206
read mu_delta: 0.944697
read initial_learning_rate: 0.000849552
read learning_rate: 0.000849552
read learning_rate_delta: 0.923362
read initial_weight_decay: 0.000505213
read weight_decay: 0.000505213
read weight_decay_delta: 0.933075
read batch_size: 100
read epsilon: 1e-07
read alpha: 0.0694823
read input_dropout_probability: 0.00117745
read hidden_dropout_probability: 0.106733
read velocity_reset: 1400
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5511
read normal distribution: '1 0.684080481529236 -0.589399755001068'
generator_str: '1904412335'
read generator: 1904412335
read generated_by_map:
1 merge_node 1
reading 94 nodes.
reading 360 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1361839457
post shuffle 1: 1920272348
epoch time: 1570.56518554688s, input_fired_time: 24.3963661193848, output_fired_time: 9.07493305206299, propagate_forward_time: 485.324371337891, propagate_backward_time: 1028.75805664063, weight_update_time: 0.543020665645599, other_time: 23.011474609375
epoch time: 125.843193054199s, input_fired_time: 1.7006311416626, output_fired_time: 0, propagate_forward_time: 119.485061645508, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.65750122070313
validation[          , genome  5511] predictions:    5195/  10000 (51.95%), best:    5195/10000 (51.95%), error:     14466.53906, best error:     14466.53906 on epoch:     0, epoch:    0/50, mu: 0.4772058129, learning_rate: 0.0008495520, weight_decay: 0.0005052128

epoch time: 1737.0462646484s, input_fired_time: 27.5588493347, output_fired_time: 9.8422813416, propagate_forward_time: 544.9219360352, propagate_backward_time: 1127.1513671875, weight_update_time: 0.8187822700, other_time: 27.5717773438
epoch time: 183.5938568115s, input_fired_time: 4.3311815262, output_fired_time: 0.0000000000, propagate_forward_time: 168.4558868408, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.8067932129
validation[          , genome  5511] predictions:    5364/  10000 (53.64%), best:    5364/10000 (53.64%), error:     13372.36719, best error:     13372.36719 on epoch:     1, epoch:    1/50, mu: 0.5055650473, learning_rate: 0.0007844443, weight_decay: 0.0004714011

epoch time: 1837.7770996094s, input_fired_time: 28.5009784698, output_fired_time: 10.6344795227, propagate_forward_time: 557.7623901367, propagate_backward_time: 1211.7814941406, weight_update_time: 0.6600832939, other_time: 29.0976562500
epoch time: 107.9592285156s, input_fired_time: 1.4287366867, output_fired_time: 0.0000000000, propagate_forward_time: 102.7985305786, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7319641113
validation[          , genome  5511] predictions:    5492/  10000 (54.92%), best:    5492/10000 (54.92%), error:     13054.39746, best error:     13054.39746 on epoch:     2, epoch:    2/50, mu: 0.5323559642, learning_rate: 0.0007243263, weight_decay: 0.0004398524

epoch time: 2257.1647949219s, input_fired_time: 26.4764671326, output_fired_time: 9.5915174484, propagate_forward_time: 565.9257812500, propagate_backward_time: 1630.2741699219, weight_update_time: 0.6117220521, other_time: 24.8967285156
epoch time: 108.1705322266s, input_fired_time: 1.5003188848, output_fired_time: 0.0000000000, propagate_forward_time: 103.0736694336, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5965423584
validation[          , genome  5511] predictions:    5445/  10000 (54.45%), best:    5492/10000 (54.92%), error:     13180.04883, best error:     13054.39746 on epoch:     2, epoch:    3/50, mu: 0.5576652288, learning_rate: 0.0006688156, weight_decay: 0.0004104151

epoch time: 1524.9001464844s, input_fired_time: 22.2898731232, output_fired_time: 7.5396037102, propagate_forward_time: 483.8826904297, propagate_backward_time: 992.0006713867, weight_update_time: 0.4743028879, other_time: 19.1873168945
epoch time: 95.3181610107s, input_fired_time: 1.2948471308, output_fired_time: 0.0000000000, propagate_forward_time: 90.7860946655, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2372207642
validation[          , genome  5511] predictions:    5596/  10000 (55.96%), best:    5596/10000 (55.96%), error:     12850.93164, best error:     12850.93164 on epoch:     4, epoch:    4/50, mu: 0.5815747976, learning_rate: 0.0006175591, weight_decay: 0.0003829479

epoch time: 4477.5771484375s, input_fired_time: 25.7686576843, output_fired_time: 10.1588087082, propagate_forward_time: 465.6357116699, propagate_backward_time: 3949.8181152344, weight_update_time: 0.6397556067, other_time: 26.1960449219
epoch time: 104.1450042725s, input_fired_time: 1.9432504177, output_fired_time: 0.0000000000, propagate_forward_time: 96.9109115601, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2908401489
validation[          , genome  5511] predictions:    5502/  10000 (55.02%), best:    5596/10000 (55.96%), error:     12985.83398, best error:     12850.93164 on epoch:     4, epoch:    5/50, mu: 0.6041620970, learning_rate: 0.0005702308, weight_decay: 0.0003573189

epoch time: 1564.5377197266s, input_fired_time: 26.1263198853, output_fired_time: 10.8460235596, propagate_forward_time: 459.7889404297, propagate_backward_time: 1035.9482421875, weight_update_time: 0.7346564531, other_time: 31.8281250000
epoch time: 109.5667724609s, input_fired_time: 2.1528761387, output_fired_time: 0.0000000000, propagate_forward_time: 100.3068313599, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.1070632935
validation[          , genome  5511] predictions:    5678/  10000 (56.78%), best:    5678/10000 (56.78%), error:     12642.91016, best error:     12642.91016 on epoch:     6, epoch:    6/50, mu: 0.6255002618, learning_rate: 0.0005265297, weight_decay: 0.0003334052

epoch time: 2455.7763671875s, input_fired_time: 27.7200565338, output_fired_time: 12.5796041489, propagate_forward_time: 512.9919433594, propagate_backward_time: 1865.0288085938, weight_update_time: 0.8321889043, other_time: 37.4560546875
epoch time: 119.1153869629s, input_fired_time: 2.6044080257, output_fired_time: 0.0000000000, propagate_forward_time: 107.7516632080, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.7593154907
validation[          , genome  5511] predictions:    5604/  10000 (56.04%), best:    5678/10000 (56.78%), error:     12683.93652, best error:     12642.91016 on epoch:     6, epoch:    7/50, mu: 0.6456583142, learning_rate: 0.0004861777, weight_decay: 0.0003110919

epoch time: 1646.3962402344s, input_fired_time: 26.0022315979, output_fired_time: 9.3586349487, propagate_forward_time: 501.4202270508, propagate_backward_time: 1084.4931640625, weight_update_time: 0.5878194571, other_time: 25.1220703125
epoch time: 102.7651596069s, input_fired_time: 1.5802155733, output_fired_time: 0.0000000000, propagate_forward_time: 96.7416381836, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4433059692
validation[          , genome  5511] predictions:    5387/  10000 (53.87%), best:    5678/10000 (56.78%), error:     13575.21875, best error:     12642.91016 on epoch:     6, epoch:    8/50, mu: 0.6647015810, learning_rate: 0.0004489181, weight_decay: 0.0002902719

epoch time: 1605.9884033203s, input_fired_time: 24.4651851654, output_fired_time: 9.8137426376, propagate_forward_time: 491.6999816895, propagate_backward_time: 1056.2174072266, weight_update_time: 0.5682082772, other_time: 23.7921142578
epoch time: 96.9899444580s, input_fired_time: 1.3938124180, output_fired_time: 0.0000000000, propagate_forward_time: 92.0736083984, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5225219727
validation[          , genome  5511] predictions:    5709/  10000 (57.09%), best:    5709/10000 (57.09%), error:     12367.59961, best error:     12367.59961 on epoch:     9, epoch:    9/50, mu: 0.6826916933, learning_rate: 0.0004145141, weight_decay: 0.0002708453

epoch time: 5861.4350585938s, input_fired_time: 22.6140136719, output_fired_time: 7.9448122978, propagate_forward_time: 466.2854309082, propagate_backward_time: 5344.1044921875, weight_update_time: 0.5027443767, other_time: 20.4863281250
epoch time: 98.9682540894s, input_fired_time: 1.5367610455, output_fired_time: 0.0000000000, propagate_forward_time: 93.3987503052, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0327453613
validation[          , genome  5511] predictions:    5747/  10000 (57.47%), best:    5747/10000 (57.47%), error:     12300.05371, best error:     12300.05371 on epoch:    10, epoch:   10/50, mu: 0.6996868849, learning_rate: 0.0003827467, weight_decay: 0.0002527189

epoch time: 1532.0308837891s, input_fired_time: 23.9217395782, output_fired_time: 8.2982826233, propagate_forward_time: 478.1415405273, propagate_backward_time: 998.9799804688, weight_update_time: 0.5774806738, other_time: 22.6892700195
epoch time: 96.7651519775s, input_fired_time: 1.3840990067, output_fired_time: 0.0000000000, propagate_forward_time: 91.8773422241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5037078857
validation[          , genome  5511] predictions:    5586/  10000 (55.86%), best:    5747/10000 (57.47%), error:     12495.17285, best error:     12300.05371 on epoch:    10, epoch:   11/50, mu: 0.7157421708, learning_rate: 0.0003534138, weight_decay: 0.0002358055

epoch time: 1479.2232666016s, input_fired_time: 23.0142898560, output_fired_time: 7.5948500633, propagate_forward_time: 464.6536865234, propagate_backward_time: 963.8776855469, weight_update_time: 0.4940795600, other_time: 20.0827636719
epoch time: 96.1264724731s, input_fired_time: 1.3898037672, output_fired_time: 0.0000000000, propagate_forward_time: 91.1130447388, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6236267090
validation[          , genome  5511] predictions:    5843/  10000 (58.43%), best:    5843/10000 (58.43%), error:     12134.03809, best error:     12134.03809 on epoch:    12, epoch:   12/50, mu: 0.7309095860, learning_rate: 0.0003263290, weight_decay: 0.0002200241

epoch time: 1458.1342773438s, input_fired_time: 22.4494876862, output_fired_time: 7.4748916626, propagate_forward_time: 456.3757324219, propagate_backward_time: 952.4765625000, weight_update_time: 0.4860447049, other_time: 19.3576660156
epoch time: 100.7615890503s, input_fired_time: 1.4173047543, output_fired_time: 0.0000000000, propagate_forward_time: 95.8630371094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4812469482
validation[          , genome  5511] predictions:    5824/  10000 (58.24%), best:    5824/10000 (58.24%), error:     12100.08496, best error:     12100.08496 on epoch:    13, epoch:   13/50, mu: 0.7452381849, learning_rate: 0.0003013199, weight_decay: 0.0002052989

epoch time: 1436.8057861328s, input_fired_time: 21.9006824493, output_fired_time: 7.3143377304, propagate_forward_time: 450.1806030273, propagate_backward_time: 938.3735351563, weight_update_time: 0.4875871241, other_time: 19.0366821289
epoch time: 93.9753646851s, input_fired_time: 1.3253455162, output_fired_time: 0.0000000000, propagate_forward_time: 89.4326400757, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2173767090
validation[          , genome  5511] predictions:    5709/  10000 (57.09%), best:    5824/10000 (58.24%), error:     12261.19043, best error:     12100.08496 on epoch:    13, epoch:   14/50, mu: 0.7587743402, learning_rate: 0.0002782275, weight_decay: 0.0001915592

epoch time: 44799.3906250000s, input_fired_time: 27.7556571960, output_fired_time: 10.0992412567, propagate_forward_time: 43564.9726562500, propagate_backward_time: 1165.4208984375, weight_update_time: 0.6166104674, other_time: 31.1455078125
epoch time: 106.9670486450s, input_fired_time: 2.0908460617, output_fired_time: 0.0000000000, propagate_forward_time: 97.8764114380, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.9997940063
validation[          , genome  5511] predictions:    5639/  10000 (56.39%), best:    5824/10000 (58.24%), error:     12590.82520, best error:     12100.08496 on epoch:    13, epoch:   15/50, mu: 0.7715619206, learning_rate: 0.0002569048, weight_decay: 0.0001787390

epoch time: 1537.0920410156s, input_fired_time: 24.6881656647, output_fired_time: 9.2690553665, propagate_forward_time: 468.4158020020, propagate_backward_time: 1008.9097900391, weight_update_time: 0.5705319047, other_time: 25.8093261719
epoch time: 92.2824172974s, input_fired_time: 1.3279585838, output_fired_time: 0.0000000000, propagate_forward_time: 87.6760101318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2784500122
validation[          , genome  5511] predictions:    5801/  10000 (58.01%), best:    5824/10000 (58.24%), error:     12244.61523, best error:     12100.08496 on epoch:    13, epoch:   16/50, mu: 0.7836422920, learning_rate: 0.0002372162, weight_decay: 0.0001667768

epoch time: 1355.1791992188s, input_fired_time: 20.9401187897, output_fired_time: 6.8455204964, propagate_forward_time: 413.5322265625, propagate_backward_time: 896.7834472656, weight_update_time: 0.4343902469, other_time: 17.0778808594
epoch time: 86.1644973755s, input_fired_time: 1.2210419178, output_fired_time: 0.0000000000, propagate_forward_time: 81.9621734619, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9812850952
validation[          , genome  5511] predictions:    5804/  10000 (58.04%), best:    5804/10000 (58.04%), error:     12095.21582, best error:     12095.21582 on epoch:    17, epoch:   17/50, mu: 0.7950545549, learning_rate: 0.0002190365, weight_decay: 0.0001556152

epoch time: 1364.3980712891s, input_fired_time: 21.1002445221, output_fired_time: 6.8685312271, propagate_forward_time: 416.4241027832, propagate_backward_time: 902.7540893555, weight_update_time: 0.4351237118, other_time: 17.2511596680
epoch time: 88.5857849121s, input_fired_time: 1.2437801361, output_fired_time: 0.0000000000, propagate_forward_time: 84.3932647705, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9487380981
validation[          , genome  5511] predictions:    5859/  10000 (58.59%), best:    5859/10000 (58.59%), error:     11999.33105, best error:     11999.33105 on epoch:    18, epoch:   18/50, mu: 0.8058357239, learning_rate: 0.0002022500, weight_decay: 0.0001452006

epoch time: 1388.1231689453s, input_fired_time: 21.2629451752, output_fired_time: 6.9791769981, propagate_forward_time: 423.9353332520, propagate_backward_time: 918.1179809570, weight_update_time: 0.4471794963, other_time: 17.8278198242
epoch time: 89.8391036987s, input_fired_time: 1.3262943029, output_fired_time: 0.0000000000, propagate_forward_time: 85.2984313965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2143783569
validation[          , genome  5511] predictions:    5944/  10000 (59.44%), best:    5944/10000 (59.44%), error:     11753.27637, best error:     11753.27637 on epoch:    19, epoch:   19/50, mu: 0.8160206079, learning_rate: 0.0001867501, weight_decay: 0.0001354830

epoch time: 1340.4803466797s, input_fired_time: 20.5971374512, output_fired_time: 6.7672791481, propagate_forward_time: 410.1062622070, propagate_backward_time: 885.8574829102, weight_update_time: 0.4348580539, other_time: 17.1520996094
epoch time: 86.0001754761s, input_fired_time: 1.2228873968, output_fired_time: 0.0000000000, propagate_forward_time: 81.8232879639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9540023804
validation[          , genome  5511] predictions:    5814/  10000 (58.14%), best:    5944/10000 (59.44%), error:     12025.19629, best error:     11753.27637 on epoch:    19, epoch:   20/50, mu: 0.8256422877, learning_rate: 0.0001724380, weight_decay: 0.0001264157

epoch time: 1374.5953369141s, input_fired_time: 21.3766403198, output_fired_time: 7.1674499512, propagate_forward_time: 422.2503967285, propagate_backward_time: 905.3958740234, weight_update_time: 0.4622147381, other_time: 18.4050292969
epoch time: 84.8109817505s, input_fired_time: 1.1959327459, output_fired_time: 0.0000000000, propagate_forward_time: 80.7561187744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8589324951
validation[          , genome  5511] predictions:    5721/  10000 (57.21%), best:    5944/10000 (59.44%), error:     12103.58594, best error:     11753.27637 on epoch:    19, epoch:   21/50, mu: 0.8347318172, learning_rate: 0.0001592227, weight_decay: 0.0001179553

epoch time: 1328.9163818359s, input_fired_time: 20.5510005951, output_fired_time: 6.5861659050, propagate_forward_time: 407.1885070801, propagate_backward_time: 877.9719238281, weight_update_time: 0.4207141101, other_time: 16.6187744141
epoch time: 85.9841995239s, input_fired_time: 1.2260409594, output_fired_time: 0.0000000000, propagate_forward_time: 81.8932952881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8648605347
validation[          , genome  5511] predictions:    5965/  10000 (59.65%), best:    5965/10000 (59.65%), error:     11688.68164, best error:     11688.68164 on epoch:    22, epoch:   22/50, mu: 0.8433187008, learning_rate: 0.0001470203, weight_decay: 0.0001100611

epoch time: 1332.3256835938s, input_fired_time: 20.4918556213, output_fired_time: 6.6209707260, propagate_forward_time: 408.0198059082, propagate_backward_time: 880.5975341797, weight_update_time: 0.4227613807, other_time: 16.5955810547
epoch time: 84.9860992432s, input_fired_time: 1.1923218966, output_fired_time: 0.0000000000, propagate_forward_time: 80.9448089600, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8489685059
validation[          , genome  5511] predictions:    5771/  10000 (57.71%), best:    5965/10000 (59.65%), error:     11929.15625, best error:     11688.68164 on epoch:    22, epoch:   23/50, mu: 0.8514306545, learning_rate: 0.0001357530, weight_decay: 0.0001026952

epoch time: 1324.7919921875s, input_fired_time: 20.3477821350, output_fired_time: 6.5317425728, propagate_forward_time: 402.9324951172, propagate_backward_time: 878.5706176758, weight_update_time: 0.4197558761, other_time: 16.4093627930
epoch time: 85.2411041260s, input_fired_time: 1.2001084089, output_fired_time: 0.0000000000, propagate_forward_time: 81.1835556030, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8574371338
validation[          , genome  5511] predictions:    5929/  10000 (59.29%), best:    5965/10000 (59.65%), error:     11761.64551, best error:     11688.68164 on epoch:    22, epoch:   24/50, mu: 0.8590940237, learning_rate: 0.0001253492, weight_decay: 0.0000958223

epoch time: 4863.8149414063s, input_fired_time: 21.6770935059, output_fired_time: 7.3307800293, propagate_forward_time: 423.8261108398, propagate_backward_time: 4391.9008789063, weight_update_time: 0.4551500678, other_time: 19.0800781250
epoch time: 86.5871276855s, input_fired_time: 1.2657079697, output_fired_time: 0.0000000000, propagate_forward_time: 82.2561721802, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0652465820
validation[          , genome  5511] predictions:    5946/  10000 (59.46%), best:    5965/10000 (59.65%), error:     11749.57520, best error:     11688.68164 on epoch:    22, epoch:   25/50, mu: 0.8663335443, learning_rate: 0.0001157427, weight_decay: 0.0000894093

epoch time: 1350.7491455078s, input_fired_time: 20.7961444855, output_fired_time: 6.8313021660, propagate_forward_time: 411.4726257324, propagate_backward_time: 894.4750976563, weight_update_time: 0.4385647178, other_time: 17.1739501953
epoch time: 85.7347030640s, input_fired_time: 1.1993669271, output_fired_time: 0.0000000000, propagate_forward_time: 81.6496810913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8856582642
validation[          , genome  5511] predictions:    5854/  10000 (58.54%), best:    5965/10000 (59.65%), error:     11770.78418, best error:     11688.68164 on epoch:    22, epoch:   26/50, mu: 0.8731727004, learning_rate: 0.0001068725, weight_decay: 0.0000834255

epoch time: 3748.9938964844s, input_fired_time: 21.9204597473, output_fired_time: 7.1372451782, propagate_forward_time: 2762.7844238281, propagate_backward_time: 939.1813354492, weight_update_time: 0.4396990836, other_time: 17.9705200195
epoch time: 87.0126495361s, input_fired_time: 1.2627453804, output_fired_time: 0.0000000000, propagate_forward_time: 82.7601394653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9897613525
validation[          , genome  5511] predictions:    5968/  10000 (59.68%), best:    5965/10000 (59.65%), error:     11693.69727, best error:     11688.68164 on epoch:    22, epoch:   27/50, mu: 0.8796336651, learning_rate: 0.0000986820, weight_decay: 0.0000778422

epoch time: 1485.9482421875s, input_fired_time: 22.8561897278, output_fired_time: 8.1925840378, propagate_forward_time: 449.3847045898, propagate_backward_time: 985.3851318359, weight_update_time: 0.5045176744, other_time: 20.1295776367
epoch time: 96.3281555176s, input_fired_time: 1.4820054770, output_fired_time: 0.0000000000, propagate_forward_time: 91.1476821899, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6984710693
validation[          , genome  5511] predictions:    5912/  10000 (59.12%), best:    5965/10000 (59.65%), error:     11774.17578, best error:     11688.68164 on epoch:    22, epoch:   28/50, mu: 0.8857372999, learning_rate: 0.0000911192, weight_decay: 0.0000726326

epoch time: 1457.1550292969s, input_fired_time: 22.2057571411, output_fired_time: 7.9883575439, propagate_forward_time: 437.3334655762, propagate_backward_time: 969.2826538086, weight_update_time: 0.4989684224, other_time: 20.3446655273
epoch time: 86.4163818359s, input_fired_time: 1.2513074875, output_fired_time: 0.0000000000, propagate_forward_time: 82.0731658936, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0919113159
validation[          , genome  5511] predictions:    5917/  10000 (59.17%), best:    5917/10000 (59.17%), error:     11675.13770, best error:     11675.13770 on epoch:    29, epoch:   29/50, mu: 0.8915033937, learning_rate: 0.0000841361, weight_decay: 0.0000677716

epoch time: 1421.8541259766s, input_fired_time: 22.2580528259, output_fired_time: 7.7368631363, propagate_forward_time: 423.1830444336, propagate_backward_time: 948.6369628906, weight_update_time: 0.4925846756, other_time: 20.0392456055
epoch time: 83.3275299072s, input_fired_time: 1.2427681684, output_fired_time: 0.0000000000, propagate_forward_time: 78.9468841553, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1378784180
validation[          , genome  5511] predictions:    5951/  10000 (59.51%), best:    5951/10000 (59.51%), error:     11657.17773, best error:     11657.17773 on epoch:    30, epoch:   30/50, mu: 0.8969506025, learning_rate: 0.0000776881, weight_decay: 0.0000632360

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_181_5511.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542006908_181_5511_0_r412821258_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.477206
read mu: 0.902097
read mu_delta: 0.944697
read initial_learning_rate: 0.000849552
read learning_rate: 7.17342e-05
read learning_rate_delta: 0.923362
read initial_weight_decay: 0.000505213
read weight_decay: 5.90039e-05
read weight_decay_delta: 0.933075
read batch_size: 100
read epsilon: 1e-07
read alpha: 0.0694823
read input_dropout_probability: 0.00117745
read hidden_dropout_probability: 0.106733
read velocity_reset: 1400
read epoch: 31
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 30
read number_validation_images: 10000
read best_validation_predictions: 5951
read best_validation_error: 11657.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5511
read normal distribution: '1 0.684080481529236 -0.589399755001068'
generator_str: '725496935'
read generator: 725496935
read generated_by_map:
1 merge_node 1
reading 94 nodes.
reading 360 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1491.78564453125s, input_fired_time: 22.1254043579102, output_fired_time: 7.60398054122925, propagate_forward_time: 486.387451171875, propagate_backward_time: 956.587341308594, weight_update_time: 0.456756800413132, other_time: 19.0814819335938
epoch time: 99.622673034668s, input_fired_time: 1.29465973377228, output_fired_time: 0, propagate_forward_time: 95.0940322875977, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.23397827148438
validation[          , genome  5511] predictions:    6054/  10000 (60.54%), best:    6054/10000 (60.54%), error:     11477.25293, best error:     11477.25293 on epoch:    31, epoch:   31/50, mu: 0.9020965695, learning_rate: 0.0000717342, weight_decay: 0.0000590039

epoch time: 6587.9135742188s, input_fired_time: 24.8144283295, output_fired_time: 9.0386171341, propagate_forward_time: 544.4139404297, propagate_backward_time: 5986.8320312500, weight_update_time: 0.5194106102, other_time: 22.8144531250
epoch time: 105.1633071899s, input_fired_time: 1.5152647495, output_fired_time: 0.0000000000, propagate_forward_time: 99.8252334595, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8228073120
validation[          , genome  5511] predictions:    6080/  10000 (60.80%), best:    6054/10000 (60.54%), error:     11538.89746, best error:     11477.25293 on epoch:    31, epoch:   32/50, mu: 0.9069579244, learning_rate: 0.0000662367, weight_decay: 0.0000550550

epoch time: 1602.4290771484s, input_fired_time: 24.3448009491, output_fired_time: 9.3392744064, propagate_forward_time: 516.0518188477, propagate_backward_time: 1028.6867675781, weight_update_time: 0.5527036190, other_time: 24.0063476563
epoch time: 108.5761566162s, input_fired_time: 1.6404296160, output_fired_time: 0.0000000000, propagate_forward_time: 102.8063735962, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1293563843
validation[          , genome  5511] predictions:    6084/  10000 (60.84%), best:    6084/10000 (60.84%), error:     11366.35938, best error:     11366.35938 on epoch:    33, epoch:   33/50, mu: 0.9115504622, learning_rate: 0.0000611605, weight_decay: 0.0000513705

epoch time: 1589.9067382813s, input_fired_time: 24.0577774048, output_fired_time: 9.1585464478, propagate_forward_time: 513.4716796875, propagate_backward_time: 1020.4151000977, weight_update_time: 0.5428864956, other_time: 22.8036499023
epoch time: 103.3621368408s, input_fired_time: 1.4185442924, output_fired_time: 0.0000000000, propagate_forward_time: 98.3582763672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5853195190
validation[          , genome  5511] predictions:    6023/  10000 (60.23%), best:    6084/10000 (60.84%), error:     11427.70703, best error:     11366.35938 on epoch:    33, epoch:   34/50, mu: 0.9158889651, learning_rate: 0.0000564733, weight_decay: 0.0000479325

epoch time: 64070.6914062500s, input_fired_time: 25.2075939178, output_fired_time: 10.0176239014, propagate_forward_time: 10213.7792968750, propagate_backward_time: 53796.2812500000, weight_update_time: 0.5742098689, other_time: 25.4062500000
epoch time: 99.5207443237s, input_fired_time: 1.5635039806, output_fired_time: 0.0000000000, propagate_forward_time: 93.9217224121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0355148315
validation[          , genome  5511] predictions:    6045/  10000 (60.45%), best:    6084/10000 (60.84%), error:     11494.18457, best error:     11366.35938 on epoch:    33, epoch:   35/50, mu: 0.9199875593, learning_rate: 0.0000521453, weight_decay: 0.0000447246

epoch time: 1604.1850585938s, input_fired_time: 24.9805603027, output_fired_time: 8.7221059799, propagate_forward_time: 496.1304016113, propagate_backward_time: 1051.3682861328, weight_update_time: 0.5278896689, other_time: 22.9837646484
epoch time: 92.2917556763s, input_fired_time: 1.3475635052, output_fired_time: 0.0000000000, propagate_forward_time: 87.5302124023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4139785767
validation[          , genome  5511] predictions:    6097/  10000 (60.97%), best:    6097/10000 (60.97%), error:     11355.03809, best error:     11355.03809 on epoch:    36, epoch:   36/50, mu: 0.9238594770, learning_rate: 0.0000481490, weight_decay: 0.0000417313

epoch time: 8018.8867187500s, input_fired_time: 22.1256885529, output_fired_time: 7.9387707710, propagate_forward_time: 7002.7622070313, propagate_backward_time: 965.5659790039, weight_update_time: 0.4743345976, other_time: 20.4940795898
epoch time: 95.2920455933s, input_fired_time: 1.4349194765, output_fired_time: 0.0000000000, propagate_forward_time: 90.2855529785, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5715713501
validation[          , genome  5511] predictions:    6086/  10000 (60.86%), best:    6086/10000 (60.86%), error:     11283.18945, best error:     11283.18945 on epoch:    37, epoch:   37/50, mu: 0.9275172949, learning_rate: 0.0000444590, weight_decay: 0.0000389385

epoch time: 1505.9031982422s, input_fired_time: 22.8087120056, output_fired_time: 8.1043653488, propagate_forward_time: 473.8218994141, propagate_backward_time: 980.6691284180, weight_update_time: 0.4913196862, other_time: 20.4990844727
epoch time: 102.4145431519s, input_fired_time: 1.4286793470, output_fired_time: 0.0000000000, propagate_forward_time: 97.5039520264, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4819107056
validation[          , genome  5511] predictions:    6100/  10000 (61.00%), best:    6086/10000 (60.86%), error:     11334.71094, best error:     11283.18945 on epoch:    37, epoch:   38/50, mu: 0.9309728146, learning_rate: 0.0000410517, weight_decay: 0.0000363325

epoch time: 1503.2880859375s, input_fired_time: 22.8356571198, output_fired_time: 8.1759834290, propagate_forward_time: 486.7360839844, propagate_backward_time: 965.1934814453, weight_update_time: 0.4865625203, other_time: 20.3468017578
epoch time: 103.4723815918s, input_fired_time: 1.4515895844, output_fired_time: 0.0000000000, propagate_forward_time: 98.2817993164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7389907837
validation[          , genome  5511] predictions:    6117/  10000 (61.17%), best:    6117/10000 (61.17%), error:     11247.08301, best error:     11247.08301 on epoch:    39, epoch:   39/50, mu: 0.9342372417, learning_rate: 0.0000379056, weight_decay: 0.0000339009

epoch time: 51153.6289062500s, input_fired_time: 25.4052066803, output_fired_time: 9.5521030426, propagate_forward_time: 524.5908813477, propagate_backward_time: 50569.7421875000, weight_update_time: 0.5349289775, other_time: 24.3398437500
epoch time: 108.5887603760s, input_fired_time: 1.8120267391, output_fired_time: 0.0000000000, propagate_forward_time: 102.3303756714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4463577271
validation[          , genome  5511] predictions:    6034/  10000 (60.34%), best:    6117/10000 (61.17%), error:     11399.85840, best error:     11247.08301 on epoch:    39, epoch:   40/50, mu: 0.9373211265, learning_rate: 0.0000350006, weight_decay: 0.0000316321

epoch time: 1459.0983886719s, input_fired_time: 22.0530433655, output_fired_time: 7.6176152229, propagate_forward_time: 452.8430786133, propagate_backward_time: 956.4215698242, weight_update_time: 0.4702970386, other_time: 20.1629638672
epoch time: 94.7037353516s, input_fired_time: 1.4000235796, output_fired_time: 0.0000000000, propagate_forward_time: 89.7755203247, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5281906128
validation[          , genome  5511] predictions:    6101/  10000 (61.01%), best:    6117/10000 (61.17%), error:     11253.75098, best error:     11247.08301 on epoch:    39, epoch:   41/50, mu: 0.9402344227, learning_rate: 0.0000323183, weight_decay: 0.0000295151

epoch time: 1553.3269042969s, input_fired_time: 23.9627895355, output_fired_time: 9.1495027542, propagate_forward_time: 474.9995117188, propagate_backward_time: 1021.2875976563, weight_update_time: 0.5659045577, other_time: 23.9274902344
epoch time: 111.6889190674s, input_fired_time: 2.2276918888, output_fired_time: 0.0000000000, propagate_forward_time: 103.4058837891, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0553436279
validation[          , genome  5511] predictions:    6095/  10000 (60.95%), best:    6095/10000 (60.95%), error:     11242.40137, best error:     11242.40137 on epoch:    42, epoch:   42/50, mu: 0.9429866076, learning_rate: 0.0000298415, weight_decay: 0.0000275398

epoch time: 1521.9874267578s, input_fired_time: 23.1575336456, output_fired_time: 8.8132238388, propagate_forward_time: 471.4280700684, propagate_backward_time: 996.9528198242, weight_update_time: 0.5370450020, other_time: 21.6356811523
epoch time: 99.0727691650s, input_fired_time: 1.5377405882, output_fired_time: 0.0000000000, propagate_forward_time: 93.7074737549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8275527954
validation[          , genome  5511] predictions:    6135/  10000 (61.35%), best:    6135/10000 (61.35%), error:     11208.85547, best error:     11208.85547 on epoch:    43, epoch:   43/50, mu: 0.9455866218, learning_rate: 0.0000275545, weight_decay: 0.0000256967

epoch time: 1585.5842285156s, input_fired_time: 24.0638008118, output_fired_time: 8.9059333801, propagate_forward_time: 485.3078002930, propagate_backward_time: 1044.7196044922, weight_update_time: 0.5774902701, other_time: 22.5870361328
epoch time: 93.8420257568s, input_fired_time: 1.3606785536, output_fired_time: 0.0000000000, propagate_forward_time: 89.2145767212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2667694092
validation[          , genome  5511] predictions:    6161/  10000 (61.61%), best:    6161/10000 (61.61%), error:     11139.28809, best error:     11139.28809 on epoch:    44, epoch:   44/50, mu: 0.9480428100, learning_rate: 0.0000254428, weight_decay: 0.0000239769

epoch time: 1486.9045410156s, input_fired_time: 22.5545692444, output_fired_time: 7.9536728859, propagate_forward_time: 458.2807922363, propagate_backward_time: 977.4954833984, weight_update_time: 0.4954687357, other_time: 20.6201171875
epoch time: 100.1935272217s, input_fired_time: 1.7535651922, output_fired_time: 0.0000000000, propagate_forward_time: 93.6309738159, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8089904785
validation[          , genome  5511] predictions:    6119/  10000 (61.19%), best:    6161/10000 (61.61%), error:     11259.89258, best error:     11139.28809 on epoch:    44, epoch:   45/50, mu: 0.9503631592, learning_rate: 0.0000234929, weight_decay: 0.0000223722

epoch time: 1397.6116943359s, input_fired_time: 21.8349533081, output_fired_time: 7.6833486557, propagate_forward_time: 415.5413208008, propagate_backward_time: 932.6082153320, weight_update_time: 0.4859485924, other_time: 19.9438476563
epoch time: 3563.3623046875s, input_fired_time: 1.3807737827, output_fired_time: 0.0000000000, propagate_forward_time: 3558.7238769531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2575683594
validation[          , genome  5511] predictions:    6112/  10000 (61.12%), best:    6161/10000 (61.61%), error:     11287.09863, best error:     11139.28809 on epoch:    44, epoch:   46/50, mu: 0.9525552392, learning_rate: 0.0000216925, weight_decay: 0.0000208750

epoch time: 1311.2222900391s, input_fired_time: 20.3730163574, output_fired_time: 6.5100474358, propagate_forward_time: 386.5472717285, propagate_backward_time: 880.0688476563, weight_update_time: 0.4284265637, other_time: 17.7231445313
epoch time: 82.4367828369s, input_fired_time: 1.3299573660, output_fired_time: 0.0000000000, propagate_forward_time: 77.8458862305, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2609405518
validation[          , genome  5511] predictions:    6171/  10000 (61.71%), best:    6171/10000 (61.71%), error:     11131.71191, best error:     11131.71191 on epoch:    47, epoch:   47/50, mu: 0.9546260834, learning_rate: 0.0000200300, weight_decay: 0.0000194779

epoch time: 1631.9824218750s, input_fired_time: 26.6380367279, output_fired_time: 11.5278739929, propagate_forward_time: 483.9984436035, propagate_backward_time: 1077.0310058594, weight_update_time: 0.6842576265, other_time: 32.7871093750
epoch time: 92.4229431152s, input_fired_time: 1.3562227488, output_fired_time: 0.0000000000, propagate_forward_time: 87.8657684326, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2009506226
validation[          , genome  5511] predictions:    6150/  10000 (61.50%), best:    6150/10000 (61.50%), error:     11126.89063, best error:     11126.89063 on epoch:    48, epoch:   48/50, mu: 0.9565823674, learning_rate: 0.0000184949, weight_decay: 0.0000181743

epoch time: 7077.3671875000s, input_fired_time: 21.8601646423, output_fired_time: 7.5789842606, propagate_forward_time: 435.7512512207, propagate_backward_time: 6592.8369140625, weight_update_time: 0.4761652648, other_time: 19.3393554688
epoch time: 95.1460342407s, input_fired_time: 1.6509749889, output_fired_time: 0.0000000000, propagate_forward_time: 89.5967636108, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8982925415
validation[          , genome  5511] predictions:    6177/  10000 (61.77%), best:    6177/10000 (61.77%), error:     11061.22559, best error:     11061.22559 on epoch:    49, epoch:   49/50, mu: 0.9584304690, learning_rate: 0.0000170775, weight_decay: 0.0000169580

epoch time: 1406.4376220703s, input_fired_time: 21.4943332672, output_fired_time: 7.4919505119, propagate_forward_time: 415.8718566895, propagate_backward_time: 942.3580932617, weight_update_time: 0.4778572023, other_time: 19.2213745117
epoch time: 86.0809860229s, input_fired_time: 1.2861200571, output_fired_time: 0.0000000000, propagate_forward_time: 81.6402282715, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1546401978
validation[          , genome  5511] predictions:    6162/  10000 (61.62%), best:    6177/10000 (61.77%), error:     11084.13672, best error:     11061.22559 on epoch:    49, epoch:   50/50, mu: 0.9601763487, learning_rate: 0.0000157688, weight_decay: 0.0000158231

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 430.5044250488s, input_fired_time: 6.4506435394, output_fired_time: 0.0000000000, propagate_forward_time: 408.1153869629, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 15.9383850098
best training[          , genome  5511] predictions:   32881/  50000 (65.76%), best:    6177/10000 (61.77%), error:     50581.35547, best error:     11061.22559 on epoch:    49, epoch:   51/50, mu: 0.9618256688, learning_rate: 0.0000145603, weight_decay: 0.0000147641
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 85.9766693115s, input_fired_time: 1.2845038176, output_fired_time: 0.0000000000, propagate_forward_time: 81.4967803955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1953887939
   testing[          , genome  5511] predictions:    6177/  10000 (61.77%), best:    6177/10000 (61.77%), error:     11061.22559, best error:     11061.22559 on epoch:    49, epoch:   51/50, mu: 0.9618256688, learning_rate: 0.0000145603, weight_decay: 0.0000147641
backpropagation finished successfully!
15:25:18 (4560): called boinc_finish(0)

</stderr_txt>
]]>