Nameexact_genome_1542727803_173_6421_1
Workunit4101313
Created20 Nov 2018, 20:14:19 UTC
Sent20 Nov 2018, 20:15:03 UTC
Report deadline25 Dec 2018, 13:48:23 UTC
Received21 Nov 2018, 10:45:12 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60660
Run time10 hours 51 min 21 sec
CPU time10 hours 30 min 47 sec
Validate stateValid
Credit8,682.87
Device peak FLOPS3.44 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size599.76 MB
Peak swap size619.61 MB
Peak disk usage13.30 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_173_6421.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542727803_173_6421_1_r271212227_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542727803_173_6421.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.55259
read mu: 0.55259
read mu_delta: 0.973491
read initial_learning_rate: 0.00037855
read learning_rate: 0.00037855
read learning_rate_delta: 0.914146
read initial_weight_decay: 0.000305826
read weight_decay: 0.000305826
read weight_decay_delta: 0.912448
read batch_size: 122
read epsilon: 1e-07
read alpha: 0.128412
read input_dropout_probability: 0.00171136
read hidden_dropout_probability: 0.0622665
read velocity_reset: 1810
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6421
read normal distribution: '1 -0.582410156726837 0.542001068592072'
generator_str: '817584092'
read generator: 817584092
read generated_by_map:
1 change_size_y 1
reading 106 nodes.
reading 424 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1535460738
post shuffle 1: 2064552153
epoch time: 1361.55529785156s, input_fired_time: 23.9245948791504, output_fired_time: 8.48756885528564, propagate_forward_time: 500.465789794922, propagate_backward_time: 814.750427246094, weight_update_time: 0.283431679010391, other_time: 13.9269409179688
epoch time: 104.14453125s, input_fired_time: 1.57018077373505, output_fired_time: 0, propagate_forward_time: 100.114219665527, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.46012878417969
validation[          , genome  6421] predictions:    6428/  10000 (64.28%), best:    6428/10000 (64.28%), error:     10345.34375, best error:     10345.34375 on epoch:     0, epoch:    0/25, mu: 0.5525903702, learning_rate: 0.0003785498, weight_decay: 0.0003058257

epoch time: 1358.2829589844s, input_fired_time: 23.8860301971, output_fired_time: 8.5187416077, propagate_forward_time: 499.7719726563, propagate_backward_time: 812.2092285156, weight_update_time: 0.2842535675, other_time: 13.8969726563
epoch time: 103.9847259521s, input_fired_time: 1.5283367634, output_fired_time: 0.0000000000, propagate_forward_time: 100.0149917603, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4413986206
validation[          , genome  6421] predictions:    6364/  10000 (63.64%), best:    6428/10000 (64.28%), error:     10485.90723, best error:     10345.34375 on epoch:     0, epoch:    1/25, mu: 0.5641855001, learning_rate: 0.0003460498, weight_decay: 0.0002790499

epoch time: 1362.8759765625s, input_fired_time: 24.0031127930, output_fired_time: 8.4939470291, propagate_forward_time: 501.1445922852, propagate_backward_time: 815.3665161133, weight_update_time: 0.2837931812, other_time: 13.8677978516
epoch time: 104.4522399902s, input_fired_time: 1.5159469843, output_fired_time: 0.0000000000, propagate_forward_time: 100.5502929688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3860015869
validation[          , genome  6421] predictions:    6422/  10000 (64.22%), best:    6428/10000 (64.28%), error:     10364.57324, best error:     10345.34375 on epoch:     0, epoch:    2/25, mu: 0.5754733086, learning_rate: 0.0003163400, weight_decay: 0.0002546184

epoch time: 1356.0878906250s, input_fired_time: 23.8546371460, output_fired_time: 8.4568367004, propagate_forward_time: 498.4609985352, propagate_backward_time: 811.1730346680, weight_update_time: 0.2882561088, other_time: 14.1424560547
epoch time: 103.6225967407s, input_fired_time: 1.5485202074, output_fired_time: 0.0000000000, propagate_forward_time: 99.7146530151, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3594207764
validation[          , genome  6421] predictions:    6450/  10000 (64.50%), best:    6450/10000 (64.50%), error:     10298.51660, best error:     10298.51660 on epoch:     3, epoch:    3/25, mu: 0.5864618421, learning_rate: 0.0002891810, weight_decay: 0.0002323259

epoch time: 1584.3244628906s, input_fired_time: 24.0140113831, output_fired_time: 8.4794750214, propagate_forward_time: 724.9061889648, propagate_backward_time: 812.8302612305, weight_update_time: 0.2886089385, other_time: 14.0944824219
epoch time: 103.8877563477s, input_fired_time: 1.5210860968, output_fired_time: 0.0000000000, propagate_forward_time: 99.9376602173, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4290084839
validation[          , genome  6421] predictions:    6363/  10000 (63.63%), best:    6450/10000 (64.50%), error:     10605.60156, best error:     10298.51660 on epoch:     3, epoch:    4/25, mu: 0.5971590877, learning_rate: 0.0002643537, weight_decay: 0.0002119852

epoch time: 1528.8767089844s, input_fired_time: 23.9717655182, output_fired_time: 8.3628969193, propagate_forward_time: 498.8514099121, propagate_backward_time: 983.6798706055, weight_update_time: 0.3035764694, other_time: 14.0106811523
epoch time: 103.1848831177s, input_fired_time: 1.5330325365, output_fired_time: 0.0000000000, propagate_forward_time: 99.2387466431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4131011963
validation[          , genome  6421] predictions:    6362/  10000 (63.62%), best:    6450/10000 (64.50%), error:     10540.66504, best error:     10298.51660 on epoch:     3, epoch:    5/25, mu: 0.6075727940, learning_rate: 0.0002416579, weight_decay: 0.0001934254

epoch time: 1356.5596923828s, input_fired_time: 23.8966007233, output_fired_time: 8.3903751373, propagate_forward_time: 499.3580017090, propagate_backward_time: 810.9524536133, weight_update_time: 0.3028922379, other_time: 13.9622192383
epoch time: 104.4713211060s, input_fired_time: 1.5101535320, output_fired_time: 0.0000000000, propagate_forward_time: 100.5651092529, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3960571289
validation[          , genome  6421] predictions:    6376/  10000 (63.76%), best:    6450/10000 (64.50%), error:     10515.55957, best error:     10298.51660 on epoch:     3, epoch:    6/25, mu: 0.6177104115, learning_rate: 0.0002209106, weight_decay: 0.0001764905

epoch time: 1360.8322753906s, input_fired_time: 23.9451828003, output_fired_time: 8.4945907593, propagate_forward_time: 501.2111511230, propagate_backward_time: 813.0630493164, weight_update_time: 0.2862411737, other_time: 14.1182250977
epoch time: 103.8319091797s, input_fired_time: 1.5254888535, output_fired_time: 0.0000000000, propagate_forward_time: 99.7088546753, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5975646973
validation[          , genome  6421] predictions:    6382/  10000 (63.82%), best:    6450/10000 (64.50%), error:     10657.97070, best error:     10298.51660 on epoch:     3, epoch:    7/25, mu: 0.6275793314, learning_rate: 0.0002019446, weight_decay: 0.0001610383

epoch time: 2011.8149414063s, input_fired_time: 23.7564163208, output_fired_time: 8.4468641281, propagate_forward_time: 715.3094482422, propagate_backward_time: 1250.3760986328, weight_update_time: 0.2861088812, other_time: 13.9260253906
epoch time: 104.6061859131s, input_fired_time: 1.5001330376, output_fired_time: 0.0000000000, propagate_forward_time: 100.6973724365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4086837769
validation[          , genome  6421] predictions:    6313/  10000 (63.13%), best:    6450/10000 (64.50%), error:     10652.89746, best error:     10298.51660 on epoch:     3, epoch:    8/25, mu: 0.6371866465, learning_rate: 0.0001846068, weight_decay: 0.0001469390

epoch time: 1356.3140869141s, input_fired_time: 23.9692687988, output_fired_time: 8.4478569031, propagate_forward_time: 498.9692077637, propagate_backward_time: 810.8447875977, weight_update_time: 0.2832739949, other_time: 14.0829467773
epoch time: 103.0201263428s, input_fired_time: 1.4985514879, output_fired_time: 0.0000000000, propagate_forward_time: 99.1060028076, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4155731201
validation[          , genome  6421] predictions:    6204/  10000 (62.04%), best:    6450/10000 (64.50%), error:     10998.57910, best error:     10298.51660 on epoch:     3, epoch:    9/25, mu: 0.6465392709, learning_rate: 0.0001687576, weight_decay: 0.0001340741

epoch time: 1360.8193359375s, input_fired_time: 24.0662002563, output_fired_time: 8.4197845459, propagate_forward_time: 500.8125610352, propagate_backward_time: 813.5950317383, weight_update_time: 0.2851825058, other_time: 13.9257812500
epoch time: 104.4792709351s, input_fired_time: 1.5074129105, output_fired_time: 0.0000000000, propagate_forward_time: 100.6022491455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3696060181
validation[          , genome  6421] predictions:    6373/  10000 (63.73%), best:    6450/10000 (64.50%), error:     10566.91992, best error:     10298.51660 on epoch:     3, epoch:   10/25, mu: 0.6556439400, learning_rate: 0.0001542691, weight_decay: 0.0001223356

epoch time: 2334.4140625000s, input_fired_time: 23.8582439423, output_fired_time: 8.4799003601, propagate_forward_time: 1101.6405029297, propagate_backward_time: 1186.3342285156, weight_update_time: 0.2824643850, other_time: 14.1011962891
epoch time: 103.2605361938s, input_fired_time: 1.4918253422, output_fired_time: 0.0000000000, propagate_forward_time: 99.3164596558, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4522476196
validation[          , genome  6421] predictions:    6379/  10000 (63.79%), best:    6450/10000 (64.50%), error:     10655.45410, best error:     10298.51660 on epoch:     3, epoch:   11/25, mu: 0.6645072699, learning_rate: 0.0001410245, weight_decay: 0.0001116248

epoch time: 1353.0893554688s, input_fired_time: 23.7132778168, output_fired_time: 8.3956003189, propagate_forward_time: 497.2758483887, propagate_backward_time: 809.8164062500, weight_update_time: 0.2836136222, other_time: 13.8881835938
epoch time: 103.3238906860s, input_fired_time: 1.5015523434, output_fired_time: 0.0000000000, propagate_forward_time: 99.4463729858, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3759689331
validation[          , genome  6421] predictions:    6415/  10000 (64.15%), best:    6450/10000 (64.50%), error:     10506.19238, best error:     10298.51660 on epoch:     3, epoch:   12/25, mu: 0.6731356382, learning_rate: 0.0001289170, weight_decay: 0.0001018518

epoch time: 1352.4805908203s, input_fired_time: 23.8441371918, output_fired_time: 8.4146766663, propagate_forward_time: 497.6532287598, propagate_backward_time: 808.5550537109, weight_update_time: 0.2837039530, other_time: 14.0135498047
epoch time: 103.3542251587s, input_fired_time: 1.5113894939, output_fired_time: 0.0000000000, propagate_forward_time: 99.4205474854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4222869873
validation[          , genome  6421] predictions:    6313/  10000 (63.13%), best:    6450/10000 (64.50%), error:     10744.56738, best error:     10298.51660 on epoch:     3, epoch:   13/25, mu: 0.6815353036, learning_rate: 0.0001178490, weight_decay: 0.0000929344

epoch time: 1351.4936523438s, input_fired_time: 23.7084350586, output_fired_time: 8.3924036026, propagate_forward_time: 497.2006835938, propagate_backward_time: 808.2004394531, weight_update_time: 0.2884516716, other_time: 13.9915771484
epoch time: 281.3757629395s, input_fired_time: 1.4565918446, output_fired_time: 0.0000000000, propagate_forward_time: 277.5884704590, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3306884766
validation[          , genome  6421] predictions:    6249/  10000 (62.49%), best:    6450/10000 (64.50%), error:     10841.88867, best error:     10298.51660 on epoch:     3, epoch:   14/25, mu: 0.6897122860, learning_rate: 0.0001077312, weight_decay: 0.0000847978

epoch time: 2102.1606445313s, input_fired_time: 764.1201171875, output_fired_time: 8.3949937820, propagate_forward_time: 501.1920166016, propagate_backward_time: 814.3238525391, weight_update_time: 0.3118135035, other_time: 14.1296386719
epoch time: 103.6014404297s, input_fired_time: 1.4947992563, output_fired_time: 0.0000000000, propagate_forward_time: 99.6497879028, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4568557739
validation[          , genome  6421] predictions:    6320/  10000 (63.20%), best:    6450/10000 (64.50%), error:     10745.40234, best error:     10298.51660 on epoch:     3, epoch:   15/25, mu: 0.6976725459, learning_rate: 0.0000984820, weight_decay: 0.0000773735

epoch time: 1352.8059082031s, input_fired_time: 23.7058181763, output_fired_time: 8.4080715179, propagate_forward_time: 497.1624450684, propagate_backward_time: 809.6251831055, weight_update_time: 0.2841313779, other_time: 13.9043579102
epoch time: 103.2439575195s, input_fired_time: 1.5036349297, output_fired_time: 0.0000000000, propagate_forward_time: 99.3618774414, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3784484863
validation[          , genome  6421] predictions:    6401/  10000 (64.01%), best:    6450/10000 (64.50%), error:     10513.95215, best error:     10298.51660 on epoch:     3, epoch:   16/25, mu: 0.7054217458, learning_rate: 0.0000900269, weight_decay: 0.0000705993

epoch time: 1471.8920898438s, input_fired_time: 25.9466094971, output_fired_time: 9.1664056778, propagate_forward_time: 541.3752441406, propagate_backward_time: 879.7225952148, weight_update_time: 0.3173590899, other_time: 15.6812133789
epoch time: 127.1108016968s, input_fired_time: 1.8205560446, output_fired_time: 0.0000000000, propagate_forward_time: 122.2352676392, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0549774170
validation[          , genome  6421] predictions:    6390/  10000 (63.90%), best:    6450/10000 (64.50%), error:     10561.80273, best error:     10298.51660 on epoch:     3, epoch:   17/25, mu: 0.7129655480, learning_rate: 0.0000822978, weight_decay: 0.0000644181

epoch time: 1594.8045654297s, input_fired_time: 28.3675155640, output_fired_time: 10.0003042221, propagate_forward_time: 587.0390014648, propagate_backward_time: 952.4396362305, weight_update_time: 0.3071005046, other_time: 16.9581298828
epoch time: 121.4930114746s, input_fired_time: 1.9301322699, output_fired_time: 0.0000000000, propagate_forward_time: 116.6189422607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9439392090
validation[          , genome  6421] predictions:    6466/  10000 (64.66%), best:    6450/10000 (64.50%), error:     10361.31055, best error:     10298.51660 on epoch:     3, epoch:   18/25, mu: 0.7203093767, learning_rate: 0.0000752322, weight_decay: 0.0000587782

epoch time: 1473.2038574219s, input_fired_time: 25.9405517578, output_fired_time: 9.1130037308, propagate_forward_time: 542.3224487305, propagate_backward_time: 880.1220092773, weight_update_time: 0.3055657148, other_time: 15.7058105469
epoch time: 103.6433029175s, input_fired_time: 1.4961113930, output_fired_time: 0.0000000000, propagate_forward_time: 99.7192459106, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4279479980
validation[          , genome  6421] predictions:    6438/  10000 (64.38%), best:    6450/10000 (64.50%), error:     10402.68555, best error:     10298.51660 on epoch:     3, epoch:   19/25, mu: 0.7274585366, learning_rate: 0.0000687732, weight_decay: 0.0000536320

epoch time: 1608.9401855469s, input_fired_time: 23.7948131561, output_fired_time: 8.4628839493, propagate_forward_time: 496.9432983398, propagate_backward_time: 1065.6793212891, weight_update_time: 0.2814945281, other_time: 14.0599365234
epoch time: 103.7833709717s, input_fired_time: 1.4874418974, output_fired_time: 0.0000000000, propagate_forward_time: 99.8633193970, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4326095581
validation[          , genome  6421] predictions:    6344/  10000 (63.44%), best:    6450/10000 (64.50%), error:     10650.45215, best error:     10298.51660 on epoch:     3, epoch:   20/25, mu: 0.7344181538, learning_rate: 0.0000628688, weight_decay: 0.0000489364

epoch time: 1514.3140869141s, input_fired_time: 23.7813091278, output_fired_time: 8.3957719803, propagate_forward_time: 496.9086914063, propagate_backward_time: 971.3299560547, weight_update_time: 0.2789319456, other_time: 13.8984375000
epoch time: 102.6941528320s, input_fired_time: 1.5063028336, output_fired_time: 0.0000000000, propagate_forward_time: 98.7696762085, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4181747437
validation[          , genome  6421] predictions:    6408/  10000 (64.08%), best:    6450/10000 (64.50%), error:     10499.88770, best error:     10298.51660 on epoch:     3, epoch:   21/25, mu: 0.7411932945, learning_rate: 0.0000574712, weight_decay: 0.0000446519

epoch time: 2513.0776367188s, input_fired_time: 23.8730487823, output_fired_time: 8.4952850342, propagate_forward_time: 496.9949951172, propagate_backward_time: 1969.5651855469, weight_update_time: 0.2972821593, other_time: 14.1490478516
epoch time: 102.4735412598s, input_fired_time: 1.5075223446, output_fired_time: 0.0000000000, propagate_forward_time: 98.5776596069, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3883590698
validation[          , genome  6421] predictions:    6493/  10000 (64.93%), best:    6493/10000 (64.93%), error:     10293.75977, best error:     10293.75977 on epoch:    22, epoch:   22/25, mu: 0.7477888465, learning_rate: 0.0000525371, weight_decay: 0.0000407425

epoch time: 7399.6899414063s, input_fired_time: 23.8333740234, output_fired_time: 8.5437898636, propagate_forward_time: 6127.9858398438, propagate_backward_time: 1225.4149169922, weight_update_time: 0.2806212306, other_time: 13.9117431641
epoch time: 102.9153900146s, input_fired_time: 1.5130137205, output_fired_time: 0.0000000000, propagate_forward_time: 99.0761642456, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3262100220
validation[          , genome  6421] predictions:    6452/  10000 (64.52%), best:    6493/10000 (64.93%), error:     10332.49316, best error:     10293.75977 on epoch:    22, epoch:   23/25, mu: 0.7542095184, learning_rate: 0.0000480266, weight_decay: 0.0000371754

epoch time: 2049.8601074219s, input_fired_time: 23.7264175415, output_fired_time: 8.5046548843, propagate_forward_time: 981.7496337891, propagate_backward_time: 1021.9129638672, weight_update_time: 0.2813957632, other_time: 13.9664306641
epoch time: 104.1340789795s, input_fired_time: 1.5160483122, output_fired_time: 0.0000000000, propagate_forward_time: 100.1916732788, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4263610840
validation[          , genome  6421] predictions:    6451/  10000 (64.51%), best:    6493/10000 (64.93%), error:     10397.65723, best error:     10293.75977 on epoch:    22, epoch:   24/25, mu: 0.7604600191, learning_rate: 0.0000439033, weight_decay: 0.0000339206

epoch time: 2419.8305664063s, input_fired_time: 23.6625442505, output_fired_time: 8.3581809998, propagate_forward_time: 1568.4327392578, propagate_backward_time: 805.5435180664, weight_update_time: 0.2863299549, other_time: 13.8335571289
epoch time: 103.1804122925s, input_fired_time: 1.5001080036, output_fired_time: 0.0000000000, propagate_forward_time: 99.2130737305, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4672317505
validation[          , genome  6421] predictions:    6495/  10000 (64.95%), best:    6493/10000 (64.93%), error:     10305.26563, best error:     10293.75977 on epoch:    22, epoch:   25/25, mu: 0.7665448189, learning_rate: 0.0000401340, weight_decay: 0.0000309508

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 514.9334106445s, input_fired_time: 7.4348034859, output_fired_time: 0.0000000000, propagate_forward_time: 495.5624694824, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 11.9361267090
best training[          , genome  6421] predictions:   35367/  50000 (70.73%), best:    6493/10000 (64.93%), error:     45363.23438, best error:     10293.75977 on epoch:    22, epoch:   26/25, mu: 0.7724683285, learning_rate: 0.0000366884, weight_decay: 0.0000282409
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 328.8244323730s, input_fired_time: 1.5764955282, output_fired_time: 0.0000000000, propagate_forward_time: 324.6863098145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5616149902
   testing[          , genome  6421] predictions:    6493/  10000 (64.93%), best:    6493/10000 (64.93%), error:     10293.75977, best error:     10293.75977 on epoch:    22, epoch:   26/25, mu: 0.7724683285, learning_rate: 0.0000366884, weight_decay: 0.0000282409
backpropagation finished successfully!
05:37:40 (12336): called boinc_finish(0)

</stderr_txt>
]]>