Nameexact_genome_1542531604_179_6233_2
Workunit4096948
Created18 Nov 2018, 10:13:44 UTC
Sent18 Nov 2018, 10:56:35 UTC
Report deadline9 Jan 2019, 9:34:52 UTC
Received20 Nov 2018, 2:01:04 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID80138
Run time13 hours 21 min 17 sec
CPU time13 hours 12 min 41 sec
Validate stateValid
Credit12,985.82
Device peak FLOPS4.49 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size566.63 MB
Peak swap size586.48 MB
Peak disk usage13.36 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_179_6233.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542531604_179_6233_2_r489438956_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542531604_179_6233.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.546751
read mu: 0.546751
read mu_delta: 0.951029
read initial_learning_rate: 0.00174764
read learning_rate: 0.00174764
read learning_rate_delta: 0.910008
read initial_weight_decay: 0.000472822
read weight_decay: 0.000472822
read weight_decay_delta: 0.936032
read batch_size: 118
read epsilon: 1e-07
read alpha: 0.0927476
read input_dropout_probability: 0.00120921
read hidden_dropout_probability: 0.105551
read velocity_reset: 1601
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6233
read normal distribution: '1 0 0'
generator_str: '2004309961'
read generator: 2004309961
read generated_by_map:
1 enable_edge 1
reading 90 nodes.
reading 468 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1009027685
post shuffle 1: 1094519409
epoch time: 872.306457519531s, input_fired_time: 13.1802024841309, output_fired_time: 4.51467132568359, propagate_forward_time: 263.292266845703, propagate_backward_time: 583.662719726563, weight_update_time: 0.205886468291283, other_time: 7.6566162109375
epoch time: 54.6153907775879s, input_fired_time: 0.738293170928955, output_fired_time: 0, propagate_forward_time: 52.6977424621582, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.17935562133789
validation[          , genome  6233] predictions:    4751/  10000 (47.51%), best:    4751/10000 (47.51%), error:     16104.45313, best error:     16104.45313 on epoch:     0, epoch:    0/50, mu: 0.5467506647, learning_rate: 0.0017476404, weight_decay: 0.0004728220

epoch time: 873.8356933594s, input_fired_time: 13.2732992172, output_fired_time: 4.5327749252, propagate_forward_time: 263.6875915527, propagate_backward_time: 584.4213867188, weight_update_time: 0.2068057805, other_time: 7.9206542969
epoch time: 58.1857261658s, input_fired_time: 0.8331544399, output_fired_time: 0.0000000000, propagate_forward_time: 55.4898796082, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8626937866
validation[          , genome  6233] predictions:    6016/  10000 (60.16%), best:    6016/10000 (60.16%), error:     11461.60449, best error:     11461.60449 on epoch:     1, epoch:    1/50, mu: 0.5684569478, learning_rate: 0.0015903665, weight_decay: 0.0004425767

epoch time: 881.5477294922s, input_fired_time: 13.4060688019, output_fired_time: 4.5679278374, propagate_forward_time: 266.4674682617, propagate_backward_time: 589.6107788086, weight_update_time: 0.2102687508, other_time: 7.4954833984
epoch time: 54.9615135193s, input_fired_time: 0.7726446986, output_fired_time: 0.0000000000, propagate_forward_time: 52.9763832092, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2124862671
validation[          , genome  6233] predictions:    6082/  10000 (60.82%), best:    6082/10000 (60.82%), error:     11248.53125, best error:     11248.53125 on epoch:     2, epoch:    2/50, mu: 0.5891002417, learning_rate: 0.0014472459, weight_decay: 0.0004142662

epoch time: 890.6834106445s, input_fired_time: 13.5860681534, output_fired_time: 4.6812968254, propagate_forward_time: 267.8721618652, propagate_backward_time: 597.1577148438, weight_update_time: 0.2149060220, other_time: 7.3862304688
epoch time: 55.4325141907s, input_fired_time: 0.7903368473, output_fired_time: 0.0000000000, propagate_forward_time: 53.4259223938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2162551880
validation[          , genome  6233] predictions:    6052/  10000 (60.52%), best:    6082/10000 (60.82%), error:     11275.15918, best error:     11248.53125 on epoch:     2, epoch:    3/50, mu: 0.6087325811, learning_rate: 0.0013170051, weight_decay: 0.0003877666

epoch time: 880.9019775391s, input_fired_time: 13.4534349442, output_fired_time: 4.5710525513, propagate_forward_time: 266.5144042969, propagate_backward_time: 589.1224975586, weight_update_time: 0.2106674910, other_time: 7.2406005859
epoch time: 55.0458145142s, input_fired_time: 0.7820034027, output_fired_time: 0.0000000000, propagate_forward_time: 53.0672531128, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1965560913
validation[          , genome  6233] predictions:    6032/  10000 (60.32%), best:    6032/10000 (60.32%), error:     11170.93359, best error:     11170.93359 on epoch:     4, epoch:    4/50, mu: 0.6274035573, learning_rate: 0.0011984850, weight_decay: 0.0003629621

epoch time: 885.4470825195s, input_fired_time: 13.6029787064, output_fired_time: 4.6160030365, propagate_forward_time: 267.2397460938, propagate_backward_time: 592.3856811523, weight_update_time: 0.2105580419, other_time: 7.6026611328
epoch time: 55.0176849365s, input_fired_time: 0.7898702621, output_fired_time: 0.0000000000, propagate_forward_time: 53.0394058228, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1884078979
validation[          , genome  6233] predictions:    5745/  10000 (57.45%), best:    6032/10000 (60.32%), error:     12180.13867, best error:     11170.93359 on epoch:     4, epoch:    5/50, mu: 0.6451601982, learning_rate: 0.0010906308, weight_decay: 0.0003397444

epoch time: 887.1188964844s, input_fired_time: 13.5563964844, output_fired_time: 4.6194553375, propagate_forward_time: 267.3284606934, propagate_backward_time: 594.3496704102, weight_update_time: 0.2123251110, other_time: 7.2649536133
epoch time: 56.2751808167s, input_fired_time: 0.7981069088, output_fired_time: 0.0000000000, propagate_forward_time: 54.2484474182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2286262512
validation[          , genome  6233] predictions:    6183/  10000 (61.83%), best:    6183/10000 (61.83%), error:     10989.40918, best error:     10989.40918 on epoch:     6, epoch:    6/50, mu: 0.6620472670, learning_rate: 0.0009924825, weight_decay: 0.0003180117

epoch time: 881.9722900391s, input_fired_time: 13.4874258041, output_fired_time: 4.5928931236, propagate_forward_time: 265.8765563965, propagate_backward_time: 590.8667602539, weight_update_time: 0.2116876841, other_time: 7.1486206055
epoch time: 55.3455276489s, input_fired_time: 0.7980310917, output_fired_time: 0.0000000000, propagate_forward_time: 53.3298492432, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2176475525
validation[          , genome  6233] predictions:    6209/  10000 (62.09%), best:    6209/10000 (62.09%), error:     10883.01953, best error:     10883.01953 on epoch:     7, epoch:    7/50, mu: 0.6781073809, learning_rate: 0.0009031669, weight_decay: 0.0002976693

epoch time: 883.1332397461s, input_fired_time: 13.5607423782, output_fired_time: 4.5844545364, propagate_forward_time: 266.8596801758, propagate_backward_time: 590.5914916992, weight_update_time: 0.2110809237, other_time: 7.5368652344
epoch time: 56.0411834717s, input_fired_time: 0.8206763864, output_fired_time: 0.0000000000, propagate_forward_time: 54.0051078796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2154006958
validation[          , genome  6233] predictions:    6295/  10000 (62.95%), best:    6295/10000 (62.95%), error:     10733.26074, best error:     10733.26074 on epoch:     8, epoch:    8/50, mu: 0.6933810115, learning_rate: 0.0008218890, weight_decay: 0.0002786281

epoch time: 892.4760742188s, input_fired_time: 13.7176380157, output_fired_time: 4.6538190842, propagate_forward_time: 269.2875671387, propagate_backward_time: 597.5289306641, weight_update_time: 0.2135330439, other_time: 7.2880859375
epoch time: 55.8328399658s, input_fired_time: 0.8079558611, output_fired_time: 0.0000000000, propagate_forward_time: 53.5236511230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5012321472
validation[          , genome  6233] predictions:    6266/  10000 (62.66%), best:    6266/10000 (62.66%), error:     10728.91016, best error:     10728.91016 on epoch:     9, epoch:    9/50, mu: 0.7079066634, learning_rate: 0.0007479254, weight_decay: 0.0002608050

epoch time: 883.3220214844s, input_fired_time: 13.5920019150, output_fired_time: 4.6086082458, propagate_forward_time: 266.6806335449, propagate_backward_time: 591.2734985352, weight_update_time: 0.2113411874, other_time: 7.1672973633
epoch time: 55.7186660767s, input_fired_time: 0.8151888251, output_fired_time: 0.0000000000, propagate_forward_time: 53.6727600098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2307167053
validation[          , genome  6233] predictions:    6226/  10000 (62.26%), best:    6266/10000 (62.66%), error:     10906.00293, best error:     10728.91016 on epoch:     9, epoch:   10/50, mu: 0.7217209935, learning_rate: 0.0006806180, weight_decay: 0.0002441219

epoch time: 887.9500732422s, input_fired_time: 13.6494436264, output_fired_time: 4.5974969864, propagate_forward_time: 267.8675231934, propagate_backward_time: 594.5957031250, weight_update_time: 0.2123290151, other_time: 7.2399902344
epoch time: 55.0963172913s, input_fired_time: 0.8006581068, output_fired_time: 0.0000000000, propagate_forward_time: 53.0922698975, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2033882141
validation[          , genome  6233] predictions:    6238/  10000 (62.38%), best:    6266/10000 (62.66%), error:     10743.73926, best error:     10728.91016 on epoch:     9, epoch:   11/50, mu: 0.7348588109, learning_rate: 0.0006193677, weight_decay: 0.0002285061

epoch time: 884.4203491211s, input_fired_time: 13.5653524399, output_fired_time: 4.6148104668, propagate_forward_time: 266.5477600098, propagate_backward_time: 592.4429321289, weight_update_time: 0.2140429020, other_time: 7.2494506836
epoch time: 55.4929389954s, input_fired_time: 0.8136820793, output_fired_time: 0.0000000000, propagate_forward_time: 53.4576835632, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2215728760
validation[          , genome  6233] predictions:    6298/  10000 (62.98%), best:    6298/10000 (62.98%), error:     10580.87891, best error:     10580.87891 on epoch:    12, epoch:   12/50, mu: 0.7473532557, learning_rate: 0.0005636294, weight_decay: 0.0002138891

epoch time: 883.6255493164s, input_fired_time: 13.5560264587, output_fired_time: 4.6154923439, propagate_forward_time: 266.5610351563, propagate_backward_time: 591.6602172852, weight_update_time: 0.2119143456, other_time: 7.2327880859
epoch time: 56.0760765076s, input_fired_time: 0.8155298233, output_fired_time: 0.0000000000, propagate_forward_time: 54.0394134521, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2211341858
validation[          , genome  6233] predictions:    6212/  10000 (62.12%), best:    6298/10000 (62.98%), error:     10898.43262, best error:     10580.87891 on epoch:    12, epoch:   13/50, mu: 0.7592358589, learning_rate: 0.0005129072, weight_decay: 0.0002002071

epoch time: 885.2622070313s, input_fired_time: 13.6148920059, output_fired_time: 4.6231989861, propagate_forward_time: 267.1552429199, propagate_backward_time: 592.5852661133, weight_update_time: 0.2115296721, other_time: 7.2836303711
epoch time: 55.4002113342s, input_fired_time: 0.8176399469, output_fired_time: 0.0000000000, propagate_forward_time: 53.3773193359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2052536011
validation[          , genome  6233] predictions:    6317/  10000 (63.17%), best:    6298/10000 (62.98%), error:     10758.06445, best error:     10580.87891 on epoch:    12, epoch:   14/50, mu: 0.7705365419, learning_rate: 0.0004667496, weight_decay: 0.0001874004

epoch time: 887.5360717773s, input_fired_time: 13.7354106903, output_fired_time: 4.6189837456, propagate_forward_time: 268.3426513672, propagate_backward_time: 593.5343627930, weight_update_time: 0.2156575322, other_time: 7.3046875000
epoch time: 56.2604942322s, input_fired_time: 0.8281103373, output_fired_time: 0.0000000000, propagate_forward_time: 54.1936149597, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2387695313
validation[          , genome  6233] predictions:    6280/  10000 (62.80%), best:    6298/10000 (62.98%), error:     10610.38574, best error:     10580.87891 on epoch:    12, epoch:   15/50, mu: 0.7812838554, learning_rate: 0.0004247458, weight_decay: 0.0001754128

epoch time: 881.4466552734s, input_fired_time: 13.5655412674, output_fired_time: 4.5742874146, propagate_forward_time: 266.0010375977, propagate_backward_time: 590.1520996094, weight_update_time: 0.2095978111, other_time: 7.1536865234
epoch time: 54.8283843994s, input_fired_time: 0.8069921732, output_fired_time: 0.0000000000, propagate_forward_time: 52.8371620178, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1842308044
validation[          , genome  6233] predictions:    6342/  10000 (63.42%), best:    6342/10000 (63.42%), error:     10488.04492, best error:     10488.04492 on epoch:    16, epoch:   16/50, mu: 0.7915048599, learning_rate: 0.0003865220, weight_decay: 0.0001641921

epoch time: 881.0281372070s, input_fired_time: 13.5874853134, output_fired_time: 4.5492887497, propagate_forward_time: 265.7575683594, propagate_backward_time: 589.9852294922, weight_update_time: 0.2085569501, other_time: 7.1485595703
epoch time: 54.8238372803s, input_fired_time: 0.8011105657, output_fired_time: 0.0000000000, propagate_forward_time: 52.8414192200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1813087463
validation[          , genome  6233] predictions:    6366/  10000 (63.66%), best:    6366/10000 (63.66%), error:     10478.89355, best error:     10478.89355 on epoch:    17, epoch:   17/50, mu: 0.8012253046, learning_rate: 0.0003517380, weight_decay: 0.0001536892

epoch time: 879.8421020508s, input_fired_time: 13.5685529709, output_fired_time: 4.5597505569, propagate_forward_time: 265.2304687500, propagate_backward_time: 589.2962036133, weight_update_time: 0.2109750658, other_time: 7.1871337891
epoch time: 55.6311759949s, input_fired_time: 0.8184432387, output_fired_time: 0.0000000000, propagate_forward_time: 53.6177711487, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1949615479
validation[          , genome  6233] predictions:    6550/  10000 (65.50%), best:    6550/10000 (65.50%), error:     10048.18750, best error:     10048.18750 on epoch:    18, epoch:   18/50, mu: 0.8104697466, learning_rate: 0.0003200844, weight_decay: 0.0001438581

epoch time: 885.1964721680s, input_fired_time: 13.7106723785, output_fired_time: 4.5925569534, propagate_forward_time: 267.2871093750, propagate_backward_time: 592.3259277344, weight_update_time: 0.2104103714, other_time: 7.2802124023
epoch time: 55.2306137085s, input_fired_time: 0.8110117912, output_fired_time: 0.0000000000, propagate_forward_time: 53.2187500000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2008514404
validation[          , genome  6233] predictions:    6562/  10000 (65.62%), best:    6550/10000 (65.50%), error:     10060.31738, best error:     10048.18750 on epoch:    18, epoch:   19/50, mu: 0.8192614913, learning_rate: 0.0002912793, weight_decay: 0.0001346558

epoch time: 879.6434936523s, input_fired_time: 13.5631361008, output_fired_time: 4.5649743080, propagate_forward_time: 265.4724121094, propagate_backward_time: 588.8393554688, weight_update_time: 0.2109278142, other_time: 7.2036132813
epoch time: 54.9574241638s, input_fired_time: 0.8113521934, output_fired_time: 0.0000000000, propagate_forward_time: 52.8682594299, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2778129578
validation[          , genome  6233] predictions:    6437/  10000 (64.37%), best:    6550/10000 (65.50%), error:     10261.79004, best error:     10048.18750 on epoch:    18, epoch:   20/50, mu: 0.8276227117, learning_rate: 0.0002650664, weight_decay: 0.0001260422

epoch time: 882.2848510742s, input_fired_time: 13.6455478668, output_fired_time: 4.5761466026, propagate_forward_time: 265.9650878906, propagate_backward_time: 590.9063110352, weight_update_time: 0.2104293406, other_time: 7.1917114258
epoch time: 54.9985733032s, input_fired_time: 0.8124779463, output_fired_time: 0.0000000000, propagate_forward_time: 52.9889869690, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1971092224
validation[          , genome  6233] predictions:    6514/  10000 (65.14%), best:    6550/10000 (65.50%), error:     10074.60840, best error:     10048.18750 on epoch:    18, epoch:   21/50, mu: 0.8355744481, learning_rate: 0.0002412125, weight_decay: 0.0001179796

epoch time: 883.7041625977s, input_fired_time: 13.7221221924, output_fired_time: 4.6068277359, propagate_forward_time: 267.6241455078, propagate_backward_time: 590.5355834961, weight_update_time: 0.2110612243, other_time: 7.2155151367
epoch time: 55.1663055420s, input_fired_time: 0.8183233142, output_fired_time: 0.0000000000, propagate_forward_time: 53.1581001282, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1898803711
validation[          , genome  6233] predictions:    6475/  10000 (64.75%), best:    6550/10000 (65.50%), error:     10119.95020, best error:     10048.18750 on epoch:    18, epoch:   22/50, mu: 0.8431367874, learning_rate: 0.0002195053, weight_decay: 0.0001104327

epoch time: 881.5434570313s, input_fired_time: 13.6211795807, output_fired_time: 4.5824408531, propagate_forward_time: 266.1144714355, propagate_backward_time: 590.0360717773, weight_update_time: 0.2173255533, other_time: 7.1892700195
epoch time: 54.7912483215s, input_fired_time: 0.8040952682, output_fired_time: 0.0000000000, propagate_forward_time: 52.8045997620, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1825523376
validation[          , genome  6233] predictions:    6416/  10000 (64.16%), best:    6550/10000 (65.50%), error:     10400.73535, best error:     10048.18750 on epoch:    18, epoch:   23/50, mu: 0.8503288031, learning_rate: 0.0001997515, weight_decay: 0.0001033686

epoch time: 882.9797973633s, input_fired_time: 13.6185092926, output_fired_time: 4.5729603767, propagate_forward_time: 265.9455261230, propagate_backward_time: 591.6340332031, weight_update_time: 0.2102287561, other_time: 7.2087402344
epoch time: 55.4311027527s, input_fired_time: 0.8191749454, output_fired_time: 0.0000000000, propagate_forward_time: 53.3871002197, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2248268127
validation[          , genome  6233] predictions:    6595/  10000 (65.95%), best:    6595/10000 (65.95%), error:      9971.21875, best error:      9971.21875 on epoch:    24, epoch:   24/50, mu: 0.8571686149, learning_rate: 0.0001817755, weight_decay: 0.0000967564

epoch time: 878.5888061523s, input_fired_time: 13.6014957428, output_fired_time: 4.5402274132, propagate_forward_time: 265.1787109375, propagate_backward_time: 588.1096191406, weight_update_time: 0.2086177617, other_time: 7.1587524414
epoch time: 54.8432960510s, input_fired_time: 0.8172272444, output_fired_time: 0.0000000000, propagate_forward_time: 52.8339653015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1921043396
validation[          , genome  6233] predictions:    6640/  10000 (66.40%), best:    6640/10000 (66.40%), error:      9775.22656, best error:      9775.22656 on epoch:    25, epoch:   25/50, mu: 0.8636734486, learning_rate: 0.0001654171, weight_decay: 0.0000905671

epoch time: 878.5060424805s, input_fired_time: 13.5636854172, output_fired_time: 4.5373239517, propagate_forward_time: 264.5315856934, propagate_backward_time: 588.7695312500, weight_update_time: 0.2073412836, other_time: 7.1038818359
epoch time: 55.1058464050s, input_fired_time: 0.8187077045, output_fired_time: 0.0000000000, propagate_forward_time: 53.0884284973, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1987113953
validation[          , genome  6233] predictions:    6559/  10000 (65.59%), best:    6640/10000 (66.40%), error:     10047.57227, best error:      9775.22656 on epoch:    25, epoch:   26/50, mu: 0.8698597550, learning_rate: 0.0001505308, weight_decay: 0.0000847738

epoch time: 879.5062255859s, input_fired_time: 13.5961179733, output_fired_time: 4.5564918518, propagate_forward_time: 265.3423461914, propagate_backward_time: 588.8988037109, weight_update_time: 0.2097998708, other_time: 7.1124267578
epoch time: 55.3864402771s, input_fired_time: 0.8209869266, output_fired_time: 0.0000000000, propagate_forward_time: 53.3287658691, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2366867065
validation[          , genome  6233] predictions:    6567/  10000 (65.67%), best:    6640/10000 (66.40%), error:      9929.58594, best error:      9775.22656 on epoch:    25, epoch:   27/50, mu: 0.8757430911, learning_rate: 0.0001369843, weight_decay: 0.0000793510

epoch time: 883.9630126953s, input_fired_time: 13.7400665283, output_fired_time: 4.5918335915, propagate_forward_time: 267.1819152832, propagate_backward_time: 591.2109375000, weight_update_time: 0.2145753354, other_time: 7.2382812500
epoch time: 55.3706474304s, input_fired_time: 0.8184169531, output_fired_time: 0.0000000000, propagate_forward_time: 53.3502616882, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2019691467
validation[          , genome  6233] predictions:    6476/  10000 (64.76%), best:    6640/10000 (66.40%), error:     10210.08203, best error:      9775.22656 on epoch:    25, epoch:   28/50, mu: 0.8813383579, learning_rate: 0.0001246567, weight_decay: 0.0000742751

epoch time: 880.5004272461s, input_fired_time: 13.6570196152, output_fired_time: 4.5414576530, propagate_forward_time: 266.1067504883, propagate_backward_time: 588.8920288086, weight_update_time: 0.2116226405, other_time: 7.3031616211
epoch time: 55.9498443604s, input_fired_time: 0.8512080908, output_fired_time: 0.0000000000, propagate_forward_time: 53.7507858276, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3478507996
validation[          , genome  6233] predictions:    6612/  10000 (66.12%), best:    6640/10000 (66.40%), error:      9823.25879, best error:      9775.22656 on epoch:    25, epoch:   29/50, mu: 0.8866596222, learning_rate: 0.0001134386, weight_decay: 0.0000695239

epoch time: 878.2390136719s, input_fired_time: 13.6093187332, output_fired_time: 4.5438933372, propagate_forward_time: 265.2571411133, propagate_backward_time: 587.6920776367, weight_update_time: 0.2092578262, other_time: 7.1365966797
epoch time: 55.5875663757s, input_fired_time: 0.8412154913, output_fired_time: 0.0000000000, propagate_forward_time: 53.5441284180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2022209167
validation[          , genome  6233] predictions:    6635/  10000 (66.35%), best:    6640/10000 (66.40%), error:      9782.01367, best error:      9775.22656 on epoch:    25, epoch:   30/50, mu: 0.8917202950, learning_rate: 0.0001032300, weight_decay: 0.0000650766

epoch time: 883.2290039063s, input_fired_time: 13.6538820267, output_fired_time: 4.5604262352, propagate_forward_time: 265.5307617188, propagate_backward_time: 592.2935180664, weight_update_time: 0.2084675431, other_time: 7.1904296875
epoch time: 56.1000289917s, input_fired_time: 0.8378877640, output_fired_time: 0.0000000000, propagate_forward_time: 54.0657501221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1963920593
validation[          , genome  6233] predictions:    6630/  10000 (66.30%), best:    6640/10000 (66.40%), error:      9839.53711, best error:      9775.22656 on epoch:    25, epoch:   31/50, mu: 0.8965331316, learning_rate: 0.0000939401, weight_decay: 0.0000609139

epoch time: 884.8636474609s, input_fired_time: 13.7288694382, output_fired_time: 4.6023278236, propagate_forward_time: 266.9713439941, propagate_backward_time: 592.3688354492, weight_update_time: 0.2106892169, other_time: 7.1921997070
epoch time: 57.3443603516s, input_fired_time: 0.8498217463, output_fired_time: 0.0000000000, propagate_forward_time: 55.2535667419, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2409706116
validation[          , genome  6233] predictions:    6693/  10000 (66.93%), best:    6693/10000 (66.93%), error:      9672.85938, best error:      9672.85938 on epoch:    32, epoch:   32/50, mu: 0.9011102915, learning_rate: 0.0000854863, weight_decay: 0.0000570173

epoch time: 880.2195434570s, input_fired_time: 13.6476640701, output_fired_time: 4.6241364479, propagate_forward_time: 265.7981567383, propagate_backward_time: 589.0176391602, weight_update_time: 0.2090691179, other_time: 7.1319580078
epoch time: 55.0959243774s, input_fired_time: 0.8285576701, output_fired_time: 0.0000000000, propagate_forward_time: 53.0731124878, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1942558289
validation[          , genome  6233] predictions:    6673/  10000 (66.73%), best:    6693/10000 (66.93%), error:      9758.47461, best error:      9672.85938 on epoch:    32, epoch:   33/50, mu: 0.9054632783, learning_rate: 0.0000777932, weight_decay: 0.0000533701

epoch time: 881.3015747070s, input_fired_time: 13.6881284714, output_fired_time: 4.5695390701, propagate_forward_time: 265.4967346191, propagate_backward_time: 590.4122924805, weight_update_time: 0.2113300413, other_time: 7.1349487305
epoch time: 55.3544731140s, input_fired_time: 0.8341928124, output_fired_time: 0.0000000000, propagate_forward_time: 53.3217315674, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1985473633
validation[          , genome  6233] predictions:    6617/  10000 (66.17%), best:    6693/10000 (66.93%), error:      9802.23828, best error:      9672.85938 on epoch:    32, epoch:   34/50, mu: 0.9096031189, learning_rate: 0.0000707924, weight_decay: 0.0000499561

epoch time: 877.0604248047s, input_fired_time: 13.6212444305, output_fired_time: 4.6157803535, propagate_forward_time: 264.7971801758, propagate_backward_time: 586.9393920898, weight_update_time: 0.2099621296, other_time: 7.0868530273
epoch time: 56.1717643738s, input_fired_time: 0.8494183421, output_fired_time: 0.0000000000, propagate_forward_time: 54.1224555969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1998901367
validation[          , genome  6233] predictions:    6713/  10000 (67.13%), best:    6693/10000 (66.93%), error:      9677.19238, best error:      9672.85938 on epoch:    32, epoch:   35/50, mu: 0.9135402441, learning_rate: 0.0000644216, weight_decay: 0.0000467606

epoch time: 879.6826171875s, input_fired_time: 13.6490974426, output_fired_time: 4.5693440437, propagate_forward_time: 265.8064575195, propagate_backward_time: 588.4816894531, weight_update_time: 0.2081892490, other_time: 7.1760253906
epoch time: 54.9333229065s, input_fired_time: 0.8282769322, output_fired_time: 0.0000000000, propagate_forward_time: 52.9127044678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1923408508
validation[          , genome  6233] predictions:    6709/  10000 (67.09%), best:    6709/10000 (67.09%), error:      9577.47266, best error:      9577.47266 on epoch:    36, epoch:   36/50, mu: 0.9172845483, learning_rate: 0.0000586242, weight_decay: 0.0000437694

epoch time: 873.6398925781s, input_fired_time: 13.5487403870, output_fired_time: 4.4944305420, propagate_forward_time: 263.6087341309, propagate_backward_time: 584.9107055664, weight_update_time: 0.2054972649, other_time: 7.0772094727
epoch time: 54.5780792236s, input_fired_time: 0.8190104365, output_fired_time: 0.0000000000, propagate_forward_time: 52.5881729126, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1708946228
validation[          , genome  6233] predictions:    6678/  10000 (66.78%), best:    6709/10000 (67.09%), error:      9648.74707, best error:      9577.47266 on epoch:    36, epoch:   37/50, mu: 0.9208455086, learning_rate: 0.0000533485, weight_decay: 0.0000409696

epoch time: 871.3643798828s, input_fired_time: 13.5300331116, output_fired_time: 4.4688229561, propagate_forward_time: 263.1225891113, propagate_backward_time: 582.6487426758, weight_update_time: 0.2055429965, other_time: 7.5941772461
epoch time: 54.5912933350s, input_fired_time: 0.8142233491, output_fired_time: 0.0000000000, propagate_forward_time: 52.6073379517, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1697311401
validation[          , genome  6233] predictions:    6709/  10000 (67.09%), best:    6709/10000 (67.09%), error:      9579.50098, best error:      9577.47266 on epoch:    36, epoch:   38/50, mu: 0.9242320657, learning_rate: 0.0000485475, weight_decay: 0.0000383489

epoch time: 870.2434082031s, input_fired_time: 13.5004987717, output_fired_time: 4.4649724960, propagate_forward_time: 262.7055053711, propagate_backward_time: 582.0500488281, weight_update_time: 0.2051929981, other_time: 7.5223999023
epoch time: 54.8513832092s, input_fired_time: 0.8246437311, output_fired_time: 0.0000000000, propagate_forward_time: 52.8478202820, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1789207458
validation[          , genome  6233] predictions:    6749/  10000 (67.49%), best:    6749/10000 (67.49%), error:      9552.55371, best error:      9552.55371 on epoch:    39, epoch:   39/50, mu: 0.9274527431, learning_rate: 0.0000441786, weight_decay: 0.0000358958

epoch time: 870.8930664063s, input_fired_time: 13.5031261444, output_fired_time: 4.4705877304, propagate_forward_time: 262.7201538086, propagate_backward_time: 582.6266479492, weight_update_time: 0.2059946656, other_time: 7.5725708008
epoch time: 54.5771980286s, input_fired_time: 0.8241109252, output_fired_time: 0.0000000000, propagate_forward_time: 52.5737648010, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1793212891
validation[          , genome  6233] predictions:    6739/  10000 (67.39%), best:    6749/10000 (67.49%), error:      9639.81836, best error:      9552.55371 on epoch:    39, epoch:   40/50, mu: 0.9305157065, learning_rate: 0.0000402029, weight_decay: 0.0000335996

epoch time: 870.9121093750s, input_fired_time: 13.5391616821, output_fired_time: 4.4693565369, propagate_forward_time: 262.6257019043, propagate_backward_time: 582.7237548828, weight_update_time: 0.2064992785, other_time: 7.5540771484
epoch time: 54.7806434631s, input_fired_time: 0.8282915354, output_fired_time: 0.0000000000, propagate_forward_time: 52.7809753418, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1713752747
validation[          , genome  6233] predictions:    6739/  10000 (67.39%), best:    6739/10000 (67.39%), error:      9497.30664, best error:      9497.30664 on epoch:    41, epoch:   41/50, mu: 0.9334287047, learning_rate: 0.0000365850, weight_decay: 0.0000314503

epoch time: 869.8577880859s, input_fired_time: 13.5085296631, output_fired_time: 4.4607205391, propagate_forward_time: 262.3786621094, propagate_backward_time: 581.9945678711, weight_update_time: 0.2061396539, other_time: 7.5153198242
epoch time: 54.6095542908s, input_fired_time: 0.8236802816, output_fired_time: 0.0000000000, propagate_forward_time: 52.6016769409, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1841964722
validation[          , genome  6233] predictions:    6734/  10000 (67.34%), best:    6739/10000 (67.39%), error:      9531.63086, best error:      9497.30664 on epoch:    41, epoch:   42/50, mu: 0.9361990690, learning_rate: 0.0000332926, weight_decay: 0.0000294385

epoch time: 870.9846801758s, input_fired_time: 13.5385942459, output_fired_time: 4.4677443504, propagate_forward_time: 262.7592163086, propagate_backward_time: 582.6273193359, weight_update_time: 0.2061091661, other_time: 7.5917968750
epoch time: 54.5864486694s, input_fired_time: 0.8274903893, output_fired_time: 0.0000000000, propagate_forward_time: 52.5942802429, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1646766663
validation[          , genome  6233] predictions:    6781/  10000 (67.81%), best:    6781/10000 (67.81%), error:      9464.43262, best error:      9464.43262 on epoch:    43, epoch:   43/50, mu: 0.9388337135, learning_rate: 0.0000302965, weight_decay: 0.0000275554

epoch time: 870.5316162109s, input_fired_time: 13.5333223343, output_fired_time: 4.4770479202, propagate_forward_time: 262.7166442871, propagate_backward_time: 582.2693481445, weight_update_time: 0.2051223069, other_time: 7.5352172852
epoch time: 54.6663246155s, input_fired_time: 0.8283405900, output_fired_time: 0.0000000000, propagate_forward_time: 52.6422042847, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1957778931
validation[          , genome  6233] predictions:    6775/  10000 (67.75%), best:    6775/10000 (67.75%), error:      9456.14648, best error:      9456.14648 on epoch:    44, epoch:   44/50, mu: 0.9413393736, learning_rate: 0.0000275701, weight_decay: 0.0000257928

epoch time: 871.2488403320s, input_fired_time: 13.5352315903, output_fired_time: 4.4674811363, propagate_forward_time: 262.8893737793, propagate_backward_time: 582.6987304688, weight_update_time: 0.2061571926, other_time: 7.6580810547
epoch time: 54.6866073608s, input_fired_time: 0.8300526142, output_fired_time: 0.0000000000, propagate_forward_time: 52.6865196228, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1700363159
validation[          , genome  6233] predictions:    6728/  10000 (67.28%), best:    6775/10000 (67.75%), error:      9490.78613, best error:      9456.14648 on epoch:    44, epoch:   45/50, mu: 0.9437223077, learning_rate: 0.0000250890, weight_decay: 0.0000241429

epoch time: 869.9447631836s, input_fired_time: 13.5179643631, output_fired_time: 4.4737734795, propagate_forward_time: 262.3813781738, propagate_backward_time: 582.0062866211, weight_update_time: 0.2051479816, other_time: 7.5653686523
epoch time: 54.5205154419s, input_fired_time: 0.8257412910, output_fired_time: 0.0000000000, propagate_forward_time: 52.5368804932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1578941345
validation[          , genome  6233] predictions:    6736/  10000 (67.36%), best:    6775/10000 (67.75%), error:      9530.08496, best error:      9456.14648 on epoch:    44, epoch:   46/50, mu: 0.9459885359, learning_rate: 0.0000228312, weight_decay: 0.0000225985

epoch time: 870.2942504883s, input_fired_time: 13.5716991425, output_fired_time: 4.4679317474, propagate_forward_time: 262.7659912109, propagate_backward_time: 581.9226074219, weight_update_time: 0.2058917582, other_time: 7.5659790039
epoch time: 54.8003845215s, input_fired_time: 0.8309220672, output_fired_time: 0.0000000000, propagate_forward_time: 52.7919807434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1774826050
validation[          , genome  6233] predictions:    6743/  10000 (67.43%), best:    6775/10000 (67.75%), error:      9466.18652, best error:      9456.14648 on epoch:    44, epoch:   47/50, mu: 0.9481438398, learning_rate: 0.0000207765, weight_decay: 0.0000211529

epoch time: 871.8025512695s, input_fired_time: 13.5731296539, output_fired_time: 4.4765353203, propagate_forward_time: 263.0767211914, propagate_backward_time: 583.0685424805, weight_update_time: 0.2057877034, other_time: 7.6076049805
epoch time: 54.5580024719s, input_fired_time: 0.8304320574, output_fired_time: 0.0000000000, propagate_forward_time: 52.5558090210, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1717605591
validation[          , genome  6233] predictions:    6785/  10000 (67.85%), best:    6785/10000 (67.85%), error:      9407.56836, best error:      9407.56836 on epoch:    48, epoch:   48/50, mu: 0.9501935840, learning_rate: 0.0000189068, weight_decay: 0.0000197998

epoch time: 872.9459838867s, input_fired_time: 13.7080640793, output_fired_time: 4.4841098785, propagate_forward_time: 263.2763671875, propagate_backward_time: 583.7197875977, weight_update_time: 0.2061177194, other_time: 7.7576293945
epoch time: 54.6392822266s, input_fired_time: 0.8311630487, output_fired_time: 0.0000000000, propagate_forward_time: 52.6425285339, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1655921936
validation[          , genome  6233] predictions:    6798/  10000 (67.98%), best:    6785/10000 (67.85%), error:      9451.22559, best error:      9407.56836 on epoch:    48, epoch:   49/50, mu: 0.9521429539, learning_rate: 0.0000172054, weight_decay: 0.0000185333

epoch time: 870.2601928711s, input_fired_time: 13.5412206650, output_fired_time: 4.4740476608, propagate_forward_time: 262.4984130859, propagate_backward_time: 582.0828247070, weight_update_time: 0.2058248967, other_time: 7.6636962891
epoch time: 54.5235252380s, input_fired_time: 0.8246006966, output_fired_time: 0.0000000000, propagate_forward_time: 52.5296325684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1692924500
validation[          , genome  6233] predictions:    6755/  10000 (67.55%), best:    6785/10000 (67.85%), error:      9457.30371, best error:      9407.56836 on epoch:    48, epoch:   50/50, mu: 0.9539968371, learning_rate: 0.0000156570, weight_decay: 0.0000173478

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 272.5815429688s, input_fired_time: 4.1539230347, output_fired_time: 0.0000000000, propagate_forward_time: 262.5481567383, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.8794555664
best training[          , genome  6233] predictions:   36517/  50000 (73.03%), best:    6785/10000 (67.85%), error:     40699.27344, best error:      9407.56836 on epoch:    48, epoch:   51/50, mu: 0.9557599425, learning_rate: 0.0000142480, weight_decay: 0.0000162381
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 55.3511390686s, input_fired_time: 0.8308613896, output_fired_time: 0.0000000000, propagate_forward_time: 52.5900268555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9302520752
   testing[          , genome  6233] predictions:    6785/  10000 (67.85%), best:    6785/10000 (67.85%), error:      9407.56836, best error:      9407.56836 on epoch:    48, epoch:   51/50, mu: 0.9557599425, learning_rate: 0.0000142480, weight_decay: 0.0000162381
backpropagation finished successfully!
20:11:36 (6432): called boinc_finish(0)

</stderr_txt>
]]>