Nameexact_genome_1542692703_171_6402_0
Workunit4100833
Created20 Nov 2018, 10:34:44 UTC
Sent20 Nov 2018, 11:11:03 UTC
Report deadline22 Dec 2018, 23:34:22 UTC
Received22 Nov 2018, 21:39:37 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88437
Run time10 hours 3 min 1 sec
CPU time9 hours 55 min 30 sec
Validate stateValid
Credit8,129.05
Device peak FLOPS4.09 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size592.81 MB
Peak swap size614.40 MB
Peak disk usage13.78 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_171_6402.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_171_6402_0_r1033502670_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542692703_171_6402.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.515131
read mu: 0.515131
read mu_delta: 0.960525
read initial_learning_rate: 0.00216006
read learning_rate: 0.00216006
read learning_rate_delta: 0.916619
read initial_weight_decay: 0.000164676
read weight_decay: 0.000164676
read weight_decay_delta: 0.964952
read batch_size: 122
read epsilon: 1e-07
read alpha: 0.105027
read input_dropout_probability: 0.00104322
read hidden_dropout_probability: 0.111731
read velocity_reset: 1009
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6402
read normal distribution: '1 0.0384592413902283 0.718027472496033'
generator_str: '518581707'
read generator: 518581707
read generated_by_map:
1 change_size 1
reading 114 nodes.
reading 460 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1314110023
post shuffle 1: 2137416262
epoch time: 1266.24890136719s, input_fired_time: 25.1707134246826, output_fired_time: 8.40277099609375, propagate_forward_time: 453.048553466797, propagate_backward_time: 763.988464355469, weight_update_time: 0.321098208427429, other_time: 15.6384887695313
epoch time: 94.5511093139648s, input_fired_time: 1.51439464092255, output_fired_time: 0, propagate_forward_time: 90.5723724365234, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.46434020996094
validation[          , genome  6402] predictions:    5923/  10000 (59.23%), best:    5923/10000 (59.23%), error:     11843.70410, best error:     11843.70410 on epoch:     0, epoch:    0/25, mu: 0.5151306391, learning_rate: 0.0021600579, weight_decay: 0.0001646761

epoch time: 1273.8267822266s, input_fired_time: 25.3712368011, output_fired_time: 8.4863700867, propagate_forward_time: 456.3195495605, propagate_backward_time: 768.7790527344, weight_update_time: 0.3059026301, other_time: 14.8706054688
epoch time: 93.9436569214s, input_fired_time: 1.4989703894, output_fired_time: 0.0000000000, propagate_forward_time: 90.0647201538, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3799667358
validation[          , genome  6402] predictions:    6095/  10000 (60.95%), best:    6095/10000 (60.95%), error:     11303.59375, best error:     11303.59375 on epoch:     1, epoch:    1/25, mu: 0.5338762403, learning_rate: 0.0019799492, weight_decay: 0.0001589045

epoch time: 1268.1408691406s, input_fired_time: 25.2636184692, output_fired_time: 8.3865499496, propagate_forward_time: 453.8000793457, propagate_backward_time: 765.2551269531, weight_update_time: 0.3164449632, other_time: 15.4354248047
epoch time: 94.5320587158s, input_fired_time: 1.4931509495, output_fired_time: 0.0000000000, propagate_forward_time: 90.5552825928, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4836273193
validation[          , genome  6402] predictions:    6098/  10000 (60.98%), best:    6095/10000 (60.95%), error:     11343.56348, best error:     11303.59375 on epoch:     1, epoch:    2/25, mu: 0.5518818498, learning_rate: 0.0018148582, weight_decay: 0.0001533352

epoch time: 1267.9809570313s, input_fired_time: 25.2990131378, output_fired_time: 8.4211483002, propagate_forward_time: 453.9995422363, propagate_backward_time: 765.0393676758, weight_update_time: 0.3081056178, other_time: 15.2218627930
epoch time: 95.0845489502s, input_fired_time: 1.5093959570, output_fired_time: 0.0000000000, propagate_forward_time: 91.0886688232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4864807129
validation[          , genome  6402] predictions:    6086/  10000 (60.86%), best:    6095/10000 (60.95%), error:     11351.34961, best error:     11303.59375 on epoch:     1, epoch:    3/25, mu: 0.5691766739, learning_rate: 0.0016635327, weight_decay: 0.0001479611

epoch time: 1270.5328369141s, input_fired_time: 25.4023036957, output_fired_time: 8.3993635178, propagate_forward_time: 454.8207092285, propagate_backward_time: 766.6015625000, weight_update_time: 0.3276300728, other_time: 15.3088378906
epoch time: 94.7945709229s, input_fired_time: 1.5759615898, output_fired_time: 0.0000000000, propagate_forward_time: 90.7702713013, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4483413696
validation[          , genome  6402] predictions:    6181/  10000 (61.81%), best:    6181/10000 (61.81%), error:     10990.83105, best error:     10990.83105 on epoch:     4, epoch:    4/25, mu: 0.5857887864, learning_rate: 0.0015248250, weight_decay: 0.0001427754

epoch time: 1271.3717041016s, input_fired_time: 25.4154872894, output_fired_time: 8.4079561234, propagate_forward_time: 454.9678955078, propagate_backward_time: 767.3571777344, weight_update_time: 0.3071219921, other_time: 15.2231445313
epoch time: 94.7934188843s, input_fired_time: 1.5197442770, output_fired_time: 0.0000000000, propagate_forward_time: 90.7353363037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5383377075
validation[          , genome  6402] predictions:    6072/  10000 (60.72%), best:    6181/10000 (61.81%), error:     11358.98633, best error:     10990.83105 on epoch:     4, epoch:    5/25, mu: 0.6017451286, learning_rate: 0.0013976829, weight_decay: 0.0001377714

epoch time: 1278.3077392578s, input_fired_time: 25.4632682800, output_fired_time: 8.4346656799, propagate_forward_time: 456.3118896484, propagate_backward_time: 772.4877929688, weight_update_time: 0.3193439841, other_time: 15.6101074219
epoch time: 98.8361129761s, input_fired_time: 1.5582919121, output_fired_time: 0.0000000000, propagate_forward_time: 94.6977539063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5800704956
validation[          , genome  6402] predictions:    6228/  10000 (62.28%), best:    6228/10000 (62.28%), error:     10824.37598, best error:     10824.37598 on epoch:     6, epoch:    6/25, mu: 0.6170715690, learning_rate: 0.0012811421, weight_decay: 0.0001329427

epoch time: 1276.0037841797s, input_fired_time: 25.5826473236, output_fired_time: 8.4596576691, propagate_forward_time: 456.7847900391, propagate_backward_time: 769.6315307617, weight_update_time: 0.3088274598, other_time: 15.5451049805
epoch time: 95.3422012329s, input_fired_time: 1.5695141554, output_fired_time: 0.0000000000, propagate_forward_time: 91.2063751221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5663146973
validation[          , genome  6402] predictions:    6311/  10000 (63.11%), best:    6311/10000 (63.11%), error:     10700.70703, best error:     10700.70703 on epoch:     7, epoch:    7/25, mu: 0.6317930222, learning_rate: 0.0011743187, weight_decay: 0.0001282834

epoch time: 1274.2165527344s, input_fired_time: 25.5822010040, output_fired_time: 8.4345331192, propagate_forward_time: 454.4947204590, propagate_backward_time: 770.1671752930, weight_update_time: 0.3178452849, other_time: 15.5379028320
epoch time: 95.3555908203s, input_fired_time: 1.5450017452, output_fired_time: 0.0000000000, propagate_forward_time: 91.2554397583, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5551528931
validation[          , genome  6402] predictions:    6350/  10000 (63.50%), best:    6350/10000 (63.50%), error:     10672.12500, best error:     10672.12500 on epoch:     8, epoch:    8/25, mu: 0.6459333301, learning_rate: 0.0010764024, weight_decay: 0.0001237873

epoch time: 1273.8686523438s, input_fired_time: 25.4989204407, output_fired_time: 8.3854827881, propagate_forward_time: 456.8645629883, propagate_backward_time: 767.7927246094, weight_update_time: 0.3075388074, other_time: 15.3269653320
epoch time: 94.3220672607s, input_fired_time: 1.5494736433, output_fired_time: 0.0000000000, propagate_forward_time: 90.2267837524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5458068848
validation[          , genome  6402] predictions:    6428/  10000 (64.28%), best:    6428/10000 (64.28%), error:     10475.44824, best error:     10475.44824 on epoch:     9, epoch:    9/25, mu: 0.6595154405, learning_rate: 0.0009866505, weight_decay: 0.0001194488

epoch time: 1272.9774169922s, input_fired_time: 25.4941406250, output_fired_time: 8.4314651489, propagate_forward_time: 455.4002380371, propagate_backward_time: 768.1547851563, weight_update_time: 0.3134196103, other_time: 15.4967041016
epoch time: 94.4513549805s, input_fired_time: 1.5373975039, output_fired_time: 0.0000000000, propagate_forward_time: 90.3725891113, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5413665771
validation[          , genome  6402] predictions:    6310/  10000 (63.10%), best:    6428/10000 (64.28%), error:     10748.20605, best error:     10475.44824 on epoch:     9, epoch:   10/25, mu: 0.6725614071, learning_rate: 0.0009043821, weight_decay: 0.0001152623

epoch time: 1272.4090576172s, input_fired_time: 25.5207366943, output_fired_time: 8.4549064636, propagate_forward_time: 454.9794616699, propagate_backward_time: 768.2257690430, weight_update_time: 0.3087868690, other_time: 15.2280883789
epoch time: 101.3484954834s, input_fired_time: 1.6006120443, output_fired_time: 0.0000000000, propagate_forward_time: 97.1257324219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6221542358
validation[          , genome  6402] predictions:    6342/  10000 (63.42%), best:    6428/10000 (64.28%), error:     10638.85449, best error:     10475.44824 on epoch:     9, epoch:   11/25, mu: 0.6850923896, learning_rate: 0.0008289735, weight_decay: 0.0001112226

epoch time: 1277.0065917969s, input_fired_time: 25.5463714600, output_fired_time: 8.4349498749, propagate_forward_time: 455.8422546387, propagate_backward_time: 771.7733764648, weight_update_time: 0.3116265535, other_time: 15.4096069336
epoch time: 94.7606506348s, input_fired_time: 1.5404021740, output_fired_time: 0.0000000000, propagate_forward_time: 90.7814941406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4387512207
validation[          , genome  6402] predictions:    6328/  10000 (63.28%), best:    6428/10000 (64.28%), error:     10677.76270, best error:     10475.44824 on epoch:     9, epoch:   12/25, mu: 0.6971287131, learning_rate: 0.0007598525, weight_decay: 0.0001073244

epoch time: 1273.5261230469s, input_fired_time: 25.4918079376, output_fired_time: 8.4515590668, propagate_forward_time: 454.6227416992, propagate_backward_time: 769.4357910156, weight_update_time: 0.3218178451, other_time: 15.5242309570
epoch time: 95.2699432373s, input_fired_time: 1.5652915239, output_fired_time: 0.0000000000, propagate_forward_time: 91.1906890869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5139617920
validation[          , genome  6402] predictions:    6303/  10000 (63.03%), best:    6428/10000 (64.28%), error:     10770.26953, best error:     10475.44824 on epoch:     9, epoch:   13/25, mu: 0.7086898685, learning_rate: 0.0006964949, weight_decay: 0.0001035629

epoch time: 1277.2701416016s, input_fired_time: 25.6981353760, output_fired_time: 8.4792079926, propagate_forward_time: 457.0648193359, propagate_backward_time: 770.4682617188, weight_update_time: 0.3056660891, other_time: 15.5596923828
epoch time: 95.6712799072s, input_fired_time: 1.5491310358, output_fired_time: 0.0000000000, propagate_forward_time: 91.5893402100, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5328063965
validation[          , genome  6402] predictions:    6422/  10000 (64.22%), best:    6422/10000 (64.22%), error:     10390.12891, best error:     10390.12891 on epoch:    14, epoch:   14/25, mu: 0.7197946310, learning_rate: 0.0006384202, weight_decay: 0.0000999332

epoch time: 1274.3786621094s, input_fired_time: 25.6194763184, output_fired_time: 8.4453496933, propagate_forward_time: 456.3646240234, propagate_backward_time: 768.5617065430, weight_update_time: 0.3036009073, other_time: 15.3875122070
epoch time: 95.0012817383s, input_fired_time: 1.5629072189, output_fired_time: 0.0000000000, propagate_forward_time: 90.8806533813, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5577239990
validation[          , genome  6402] predictions:    6500/  10000 (65.00%), best:    6500/10000 (65.00%), error:     10256.41797, best error:     10256.41797 on epoch:    15, epoch:   15/25, mu: 0.7304610610, learning_rate: 0.0005851879, weight_decay: 0.0000964308

epoch time: 1277.5250244141s, input_fired_time: 25.6390762329, output_fired_time: 8.4166536331, propagate_forward_time: 455.8135986328, propagate_backward_time: 772.0269775391, weight_update_time: 0.3190453649, other_time: 15.6287841797
epoch time: 96.0029678345s, input_fired_time: 1.5784677267, output_fired_time: 0.0000000000, propagate_forward_time: 91.9686431885, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4558563232
validation[          , genome  6402] predictions:    6453/  10000 (64.53%), best:    6500/10000 (65.00%), error:     10349.19238, best error:     10256.41797 on epoch:    15, epoch:   16/25, mu: 0.7407064438, learning_rate: 0.0005363941, weight_decay: 0.0000930511

epoch time: 1274.6169433594s, input_fired_time: 25.7183513641, output_fired_time: 8.4154005051, propagate_forward_time: 456.0195312500, propagate_backward_time: 769.0115966797, weight_update_time: 0.3069964349, other_time: 15.4520263672
epoch time: 95.2263793945s, input_fired_time: 1.5911467075, output_fired_time: 0.0000000000, propagate_forward_time: 90.9670715332, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6681594849
validation[          , genome  6402] predictions:    6501/  10000 (65.01%), best:    6501/10000 (65.01%), error:     10200.03418, best error:     10200.03418 on epoch:    17, epoch:   17/25, mu: 0.7505473495, learning_rate: 0.0004916688, weight_decay: 0.0000897898

epoch time: 1276.1124267578s, input_fired_time: 25.6438789368, output_fired_time: 8.4292430878, propagate_forward_time: 458.8741455078, propagate_backward_time: 767.7532958984, weight_update_time: 0.3098834157, other_time: 15.4118652344
epoch time: 95.1660232544s, input_fired_time: 1.5681031942, output_fired_time: 0.0000000000, propagate_forward_time: 91.0103759766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5875473022
validation[          , genome  6402] predictions:    6484/  10000 (64.84%), best:    6501/10000 (65.01%), error:     10279.65039, best error:     10200.03418 on epoch:    17, epoch:   18/25, mu: 0.7599998116, learning_rate: 0.0004506727, weight_decay: 0.0000866429

epoch time: 1276.8999023438s, input_fired_time: 25.5491981506, output_fired_time: 8.4135398865, propagate_forward_time: 457.8036193848, propagate_backward_time: 769.6130981445, weight_update_time: 0.3154390454, other_time: 15.5204467773
epoch time: 94.9197158813s, input_fired_time: 1.5797622204, output_fired_time: 0.0000000000, propagate_forward_time: 90.8195648193, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5203857422
validation[          , genome  6402] predictions:    6407/  10000 (64.07%), best:    6501/10000 (65.01%), error:     10390.41406, best error:     10200.03418 on epoch:    17, epoch:   19/25, mu: 0.7690791488, learning_rate: 0.0004130950, weight_decay: 0.0000836062

epoch time: 1271.1827392578s, input_fired_time: 25.6340637207, output_fired_time: 8.4277267456, propagate_forward_time: 455.5397033691, propagate_backward_time: 766.1125488281, weight_update_time: 0.3142103851, other_time: 15.4687500000
epoch time: 94.8641738892s, input_fired_time: 1.5694772005, output_fired_time: 0.0000000000, propagate_forward_time: 90.7668762207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5278167725
validation[          , genome  6402] predictions:    6532/  10000 (65.32%), best:    6532/10000 (65.32%), error:     10150.25391, best error:     10150.25391 on epoch:    20, epoch:   20/25, mu: 0.7778000832, learning_rate: 0.0003786506, weight_decay: 0.0000806760

epoch time: 1271.1873779297s, input_fired_time: 25.5303306580, output_fired_time: 8.3783674240, propagate_forward_time: 457.9056091309, propagate_backward_time: 763.8645019531, weight_update_time: 0.3157382905, other_time: 15.5085449219
epoch time: 94.8346862793s, input_fired_time: 1.5653686523, output_fired_time: 0.0000000000, propagate_forward_time: 90.7461395264, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5231781006
validation[          , genome  6402] predictions:    6510/  10000 (65.10%), best:    6532/10000 (65.32%), error:     10231.57129, best error:     10150.25391 on epoch:    20, epoch:   21/25, mu: 0.7861767411, learning_rate: 0.0003470781, weight_decay: 0.0000778484

epoch time: 1269.5874023438s, input_fired_time: 25.6068801880, output_fired_time: 8.4061555862, propagate_forward_time: 454.4739074707, propagate_backward_time: 765.8892822266, weight_update_time: 0.3080437779, other_time: 15.2111816406
epoch time: 94.8804473877s, input_fired_time: 1.5660992861, output_fired_time: 0.0000000000, propagate_forward_time: 90.8149337769, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4994125366
validation[          , genome  6402] predictions:    6407/  10000 (64.07%), best:    6532/10000 (65.32%), error:     10370.33301, best error:     10150.25391 on epoch:    20, epoch:   22/25, mu: 0.7942227125, learning_rate: 0.0003181383, weight_decay: 0.0000751200

epoch time: 1271.2950439453s, input_fired_time: 25.5882873535, output_fired_time: 8.3629331589, propagate_forward_time: 453.8649597168, propagate_backward_time: 768.2176513672, weight_update_time: 0.3067369461, other_time: 15.2612304688
epoch time: 95.4738235474s, input_fired_time: 1.5776089430, output_fired_time: 0.0000000000, propagate_forward_time: 91.3366775513, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5595397949
validation[          , genome  6402] predictions:    6543/  10000 (65.43%), best:    6543/10000 (65.43%), error:     10149.12988, best error:     10149.12988 on epoch:    23, epoch:   23/25, mu: 0.8019510508, learning_rate: 0.0002916114, weight_decay: 0.0000724872

epoch time: 1278.0910644531s, input_fired_time: 25.7276630402, output_fired_time: 8.4131097794, propagate_forward_time: 455.7613525391, propagate_backward_time: 772.8063354492, weight_update_time: 0.3107259572, other_time: 15.3826293945
epoch time: 95.0714874268s, input_fired_time: 1.5842044353, output_fired_time: 0.0000000000, propagate_forward_time: 91.0076522827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4796295166
validation[          , genome  6402] predictions:    6449/  10000 (64.49%), best:    6543/10000 (65.43%), error:     10231.56543, best error:     10149.12988 on epoch:    23, epoch:   24/25, mu: 0.8093743324, learning_rate: 0.0002672965, weight_decay: 0.0000699466

epoch time: 1274.4737548828s, input_fired_time: 25.6470184326, output_fired_time: 8.4144039154, propagate_forward_time: 454.8221130371, propagate_backward_time: 770.2722778320, weight_update_time: 0.3105378151, other_time: 15.3179321289
epoch time: 95.3059768677s, input_fired_time: 1.5969052315, output_fired_time: 0.0000000000, propagate_forward_time: 91.1393203735, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5697479248
validation[          , genome  6402] predictions:    6506/  10000 (65.06%), best:    6543/10000 (65.43%), error:     10227.43848, best error:     10149.12988 on epoch:    23, epoch:   25/25, mu: 0.8165045977, learning_rate: 0.0002450089, weight_decay: 0.0000674951

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 475.3814086914s, input_fired_time: 7.9634542465, output_fired_time: 0.0000000000, propagate_forward_time: 454.2343750000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 13.1835937500
best training[          , genome  6402] predictions:   36722/  50000 (73.44%), best:    6543/10000 (65.43%), error:     40330.96875, best error:     10149.12988 on epoch:    23, epoch:   26/25, mu: 0.8233533502, learning_rate: 0.0002245797, weight_decay: 0.0000651296
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 95.4598541260s, input_fired_time: 1.6135572195, output_fired_time: 0.0000000000, propagate_forward_time: 91.1591339111, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6871643066
   testing[          , genome  6402] predictions:    6543/  10000 (65.43%), best:    6543/10000 (65.43%), error:     10149.12988, best error:     10149.12988 on epoch:    23, epoch:   26/25, mu: 0.8233533502, learning_rate: 0.0002245797, weight_decay: 0.0000651296
backpropagation finished successfully!
15:39:19 (4796): called boinc_finish(0)

</stderr_txt>
]]>