Nameexact_genome_1542127803_178_5801_1
Workunit4086578
Created13 Nov 2018, 17:30:49 UTC
Sent13 Nov 2018, 17:53:58 UTC
Report deadline14 Jan 2019, 2:10:26 UTC
Received24 Nov 2018, 15:12:06 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID13956
Run time1 days 2 hours 32 min 49 sec
CPU time1 days 0 hours 46 min 52 sec
Validate stateValid
Credit15,336.19
Device peak FLOPS2.92 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size563.45 MB
Peak swap size583.36 MB
Peak disk usage14.48 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801_1_r2082023144_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.482211
read mu: 0.482211
read mu_delta: 0.946719
read initial_learning_rate: 0.00269226
read learning_rate: 0.00269226
read learning_rate_delta: 0.914606
read initial_weight_decay: 0.000639082
read weight_decay: 0.000639082
read weight_decay_delta: 0.927316
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.0775869
read input_dropout_probability: 0.00117795
read hidden_dropout_probability: 0.10697
read velocity_reset: 2135
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5801
read normal distribution: '1 0 0'
generator_str: '1698501607'
read generator: 1698501607
read generated_by_map:
1 disable_edge 1
reading 100 nodes.
reading 529 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 216389278
post shuffle 1: 1770287138
epoch time: 1697.93530273438s, input_fired_time: 23.7922134399414, output_fired_time: 9.11636924743652, propagate_forward_time: 526.678405761719, propagate_backward_time: 1115.71301269531, weight_update_time: 0.564001381397247, other_time: 22.6353759765625
epoch time: 110.151275634766s, input_fired_time: 1.42351841926575, output_fired_time: 0, propagate_forward_time: 105.267120361328, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.46063995361328
validation[          , genome  5801] predictions:    5466/  10000 (54.66%), best:    5466/10000 (54.66%), error:     13007.61621, best error:     13007.61621 on epoch:     0, epoch:    0/50, mu: 0.4822108150, learning_rate: 0.0026922580, weight_decay: 0.0006390816

epoch time: 1684.9194335938s, input_fired_time: 23.2547245026, output_fired_time: 8.8033437729, propagate_forward_time: 522.8978271484, propagate_backward_time: 1108.8319091797, weight_update_time: 0.5268852711, other_time: 21.1315917969
epoch time: 106.1286849976s, input_fired_time: 1.3620538712, output_fired_time: 0.0000000000, propagate_forward_time: 101.4083175659, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3583145142
validation[          , genome  5801] predictions:    5551/  10000 (55.51%), best:    5551/10000 (55.51%), error:     12739.56641, best error:     12739.56641 on epoch:     1, epoch:    1/50, mu: 0.5092661977, learning_rate: 0.0024623554, weight_decay: 0.0005926306

epoch time: 1635.3041992188s, input_fired_time: 22.4208316803, output_fired_time: 8.1815977097, propagate_forward_time: 512.3887939453, propagate_backward_time: 1072.8276367188, weight_update_time: 0.5017883778, other_time: 19.4853515625
epoch time: 111.1014099121s, input_fired_time: 1.5591186285, output_fired_time: 0.0000000000, propagate_forward_time: 105.9244995117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6177902222
validation[          , genome  5801] predictions:    5588/  10000 (55.88%), best:    5551/10000 (55.51%), error:     12801.85059, best error:     12739.56641 on epoch:     1, epoch:    2/50, mu: 0.5348800421, learning_rate: 0.0022520851, weight_decay: 0.0005495559

epoch time: 1915.9045410156s, input_fired_time: 30.0178642273, output_fired_time: 14.1129436493, propagate_forward_time: 568.9936523438, propagate_backward_time: 1261.7124023438, weight_update_time: 0.8850218654, other_time: 41.0677490234
epoch time: 130.1870880127s, input_fired_time: 2.7686636448, output_fired_time: 0.0000000000, propagate_forward_time: 118.6716461182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.7467803955
validation[          , genome  5801] predictions:    5637/  10000 (56.37%), best:    5637/10000 (56.37%), error:     12439.23535, best error:     12439.23535 on epoch:     3, epoch:    3/50, mu: 0.5591291785, learning_rate: 0.0020597707, weight_decay: 0.0005096121

epoch time: 4211.1655273438s, input_fired_time: 24.2820072174, output_fired_time: 9.4676256180, propagate_forward_time: 3046.8684082031, propagate_backward_time: 1100.0898437500, weight_update_time: 0.5942673087, other_time: 30.4572753906
epoch time: 106.6757659912s, input_fired_time: 1.5810500383, output_fired_time: 0.0000000000, propagate_forward_time: 99.8330078125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2617111206
validation[          , genome  5801] predictions:    5777/  10000 (57.77%), best:    5777/10000 (57.77%), error:     12187.08594, best error:     12187.08594 on epoch:     4, epoch:    4/50, mu: 0.5820862651, learning_rate: 0.0018838787, weight_decay: 0.0004725715

epoch time: 1655.4351806641s, input_fired_time: 23.9181346893, output_fired_time: 9.1563825607, propagate_forward_time: 499.4842529297, propagate_backward_time: 1093.4562988281, weight_update_time: 0.6151087880, other_time: 29.4201660156
epoch time: 107.4320678711s, input_fired_time: 1.6300227642, output_fired_time: 0.0000000000, propagate_forward_time: 100.3104248047, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4916229248
validation[          , genome  5801] predictions:    5735/  10000 (57.35%), best:    5735/10000 (57.35%), error:     12127.76563, best error:     12127.76563 on epoch:     5, epoch:    5/50, mu: 0.6038202047, learning_rate: 0.0017230068, weight_decay: 0.0004382232

epoch time: 1653.1436767578s, input_fired_time: 23.8914279938, output_fired_time: 9.0493593216, propagate_forward_time: 498.5562133789, propagate_backward_time: 1091.7658691406, weight_update_time: 0.6267822981, other_time: 29.8808593750
epoch time: 110.4509048462s, input_fired_time: 1.7290800810, output_fired_time: 0.0000000000, propagate_forward_time: 103.2637710571, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4580535889
validation[          , genome  5801] predictions:    5980/  10000 (59.80%), best:    5980/10000 (59.80%), error:     11661.49707, best error:     11661.49707 on epoch:     6, epoch:    6/50, mu: 0.6243961453, learning_rate: 0.0015758724, weight_decay: 0.0004063714

epoch time: 5316.6054687500s, input_fired_time: 26.9887371063, output_fired_time: 11.2016191483, propagate_forward_time: 4045.3005371094, propagate_backward_time: 1197.9074707031, weight_update_time: 0.7968553305, other_time: 35.2070312500
epoch time: 122.4221420288s, input_fired_time: 2.1980478764, output_fired_time: 0.0000000000, propagate_forward_time: 113.6395416260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.5845489502
validation[          , genome  5801] predictions:    5740/  10000 (57.40%), best:    5980/10000 (59.80%), error:     12205.00879, best error:     11661.49707 on epoch:     6, epoch:    7/50, mu: 0.6438757777, learning_rate: 0.0014413025, weight_decay: 0.0003768348

epoch time: 1941.0593261719s, input_fired_time: 29.8046035767, output_fired_time: 13.3252372742, propagate_forward_time: 580.8805541992, propagate_backward_time: 1276.1373291016, weight_update_time: 0.8135535121, other_time: 40.9117431641
epoch time: 124.3503799438s, input_fired_time: 2.3219263554, output_fired_time: 0.0000000000, propagate_forward_time: 114.1096954346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.9187545776
validation[          , genome  5801] predictions:    5974/  10000 (59.74%), best:    5974/10000 (59.74%), error:     11637.51660, best error:     11637.51660 on epoch:     8, epoch:    8/50, mu: 0.6623175144, learning_rate: 0.0013182239, weight_decay: 0.0003494449

epoch time: 186787.5000000000s, input_fired_time: 100009.4296875000, output_fired_time: 9.0756292343, propagate_forward_time: 85636.1093750000, propagate_backward_time: 1109.2641601563, weight_update_time: 0.5319234729, other_time: 23.6186523438
epoch time: 116.4271545410s, input_fired_time: 1.4592350721, output_fired_time: 0.0000000000, propagate_forward_time: 111.3548965454, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6130218506
validation[          , genome  5801] predictions:    6029/  10000 (60.29%), best:    6029/10000 (60.29%), error:     11527.82813, best error:     11527.82813 on epoch:     9, epoch:    9/50, mu: 0.6797766685, learning_rate: 0.0012056555, weight_decay: 0.0003240459

epoch time: 5673.7167968750s, input_fired_time: 23.0175571442, output_fired_time: 8.1747207642, propagate_forward_time: 544.6616210938, propagate_backward_time: 5078.0629882813, weight_update_time: 0.4924803078, other_time: 19.7998046875
epoch time: 112.3766708374s, input_fired_time: 1.3449041843, output_fired_time: 0.0000000000, propagate_forward_time: 107.8250122070, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2067565918
validation[          , genome  5801] predictions:    5911/  10000 (59.11%), best:    6029/10000 (60.29%), error:     11745.83887, best error:     11527.82813 on epoch:     9, epoch:   10/50, mu: 0.6963055730, learning_rate: 0.0011026998, weight_decay: 0.0003004930

epoch time: 54178.6250000000s, input_fired_time: 61.8386116028, output_fired_time: 19.1355533600, propagate_forward_time: 1235.7563476563, propagate_backward_time: 52806.8515625000, weight_update_time: 1.5216156244, other_time: 55.0390625000
epoch time: 310.0626220703s, input_fired_time: 4.8365826607, output_fired_time: 0.0000000000, propagate_forward_time: 294.3473510742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 10.8786926270
validation[          , genome  5801] predictions:    6044/  10000 (60.44%), best:    6044/10000 (60.44%), error:     11376.20508, best error:     11376.20508 on epoch:    11, epoch:   11/50, mu: 0.7119538188, learning_rate: 0.0010085359, weight_decay: 0.0002786520

epoch time: 2195.8854980469s, input_fired_time: 31.9469051361, output_fired_time: 11.9074306488, propagate_forward_time: 688.7860107422, propagate_backward_time: 1434.3759765625, weight_update_time: 0.7000010610, other_time: 28.8690185547
epoch time: 137.2602386475s, input_fired_time: 1.8636519909, output_fired_time: 0.0000000000, propagate_forward_time: 130.6797943115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7167968750
validation[          , genome  5801] predictions:    5878/  10000 (58.78%), best:    6044/10000 (60.44%), error:     11903.83984, best error:     11376.20508 on epoch:    11, epoch:   12/50, mu: 0.7267683148, learning_rate: 0.0009224130, weight_decay: 0.0002583985

epoch time: 14155.1259765625s, input_fired_time: 26.0275421143, output_fired_time: 10.4728736877, propagate_forward_time: 567.2146606445, propagate_backward_time: 13524.0068359375, weight_update_time: 0.6383389831, other_time: 27.4042968750
epoch time: 105.0503692627s, input_fired_time: 1.4307458401, output_fired_time: 0.0000000000, propagate_forward_time: 99.9742355347, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6453857422
validation[          , genome  5801] predictions:    6127/  10000 (61.27%), best:    6127/10000 (61.27%), error:     11188.14551, best error:     11188.14551 on epoch:    13, epoch:   13/50, mu: 0.7407934666, learning_rate: 0.0008436444, weight_decay: 0.0002396171

epoch time: 1688.7020263672s, input_fired_time: 24.0994968414, output_fired_time: 8.3943796158, propagate_forward_time: 521.7114868164, propagate_backward_time: 1112.0401611328, weight_update_time: 0.5425245166, other_time: 22.4564208984
epoch time: 104.2879638672s, input_fired_time: 1.3740617037, output_fired_time: 0.0000000000, propagate_forward_time: 99.5179138184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3959884644
validation[          , genome  5801] predictions:    6202/  10000 (62.02%), best:    6202/10000 (62.02%), error:     10938.89844, best error:     10938.89844 on epoch:    14, epoch:   14/50, mu: 0.7540713549, learning_rate: 0.0007716023, weight_decay: 0.0002222008

epoch time: 1771.0783691406s, input_fired_time: 24.9312858582, output_fired_time: 9.8069562912, propagate_forward_time: 559.4057617188, propagate_backward_time: 1151.6771240234, weight_update_time: 0.5810747743, other_time: 25.2572021484
epoch time: 113.4879531860s, input_fired_time: 1.9850766659, output_fired_time: 0.0000000000, propagate_forward_time: 105.2367019653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.2661743164
validation[          , genome  5801] predictions:    5963/  10000 (59.63%), best:    6202/10000 (62.02%), error:     11585.22168, best error:     10938.89844 on epoch:    14, epoch:   15/50, mu: 0.7666417956, learning_rate: 0.0007057121, weight_decay: 0.0002060504

epoch time: 1740.9031982422s, input_fired_time: 26.3114585876, output_fired_time: 10.5905456543, propagate_forward_time: 524.9714965820, propagate_backward_time: 1148.1726074219, weight_update_time: 0.7050613761, other_time: 30.8571777344
epoch time: 104.1948471069s, input_fired_time: 1.5896953344, output_fired_time: 0.0000000000, propagate_forward_time: 98.8805465698, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7246017456
validation[          , genome  5801] predictions:    6110/  10000 (61.10%), best:    6202/10000 (62.02%), error:     11123.12012, best error:     10938.89844 on epoch:    14, epoch:   16/50, mu: 0.7785424590, learning_rate: 0.0006454485, weight_decay: 0.0001910739

epoch time: 1618.7775878906s, input_fired_time: 23.0259284973, output_fired_time: 8.6218996048, propagate_forward_time: 496.6499938965, propagate_backward_time: 1069.6358642578, weight_update_time: 0.5249751210, other_time: 20.8438720703
epoch time: 100.1700210571s, input_fired_time: 1.2865383625, output_fired_time: 0.0000000000, propagate_forward_time: 95.8307113647, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0527725220
validation[          , genome  5801] predictions:    6153/  10000 (61.53%), best:    6202/10000 (62.02%), error:     11094.99414, best error:     10938.89844 on epoch:    14, epoch:   17/50, mu: 0.7898090482, learning_rate: 0.0005903311, weight_decay: 0.0001771859

epoch time: 1727.4473876953s, input_fired_time: 25.4823474884, output_fired_time: 10.3979492188, propagate_forward_time: 529.6034545898, propagate_backward_time: 1133.2839355469, weight_update_time: 0.6308817267, other_time: 28.6796875000
epoch time: 122.3581695557s, input_fired_time: 2.1377115250, output_fired_time: 0.0000000000, propagate_forward_time: 113.3620529175, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.8584060669
validation[          , genome  5801] predictions:    6233/  10000 (62.33%), best:    6233/10000 (62.33%), error:     10804.79883, best error:     10804.79883 on epoch:    18, epoch:   18/50, mu: 0.8004753590, learning_rate: 0.0005399204, weight_decay: 0.0001643073

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801_1_r2082023144_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.482211
read mu: 0.810573
read mu_delta: 0.946719
read initial_learning_rate: 0.00269226
read learning_rate: 0.000493814
read learning_rate_delta: 0.914606
read initial_weight_decay: 0.000639082
read weight_decay: 0.000152365
read weight_decay_delta: 0.927316
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.0775869
read input_dropout_probability: 0.00117795
read hidden_dropout_probability: 0.10697
read velocity_reset: 2135
read epoch: 19
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 18
read number_validation_images: 10000
read best_validation_predictions: 6233
read best_validation_error: 10804.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5801
read normal distribution: '1 0 0'
generator_str: '469981849'
read generator: 469981849
read generated_by_map:
1 disable_edge 1
reading 100 nodes.
reading 529 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1829.25793457031s, input_fired_time: 24.6825847625732, output_fired_time: 8.95759868621826, propagate_forward_time: 597.706787109375, propagate_backward_time: 1175.63073730469, weight_update_time: 0.527094125747681, other_time: 22.2801513671875
epoch time: 124.469306945801s, input_fired_time: 1.51546919345856, output_fired_time: 0, propagate_forward_time: 119.272186279297, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.68164825439453
validation[          , genome  5801] predictions:    6331/  10000 (63.31%), best:    6331/10000 (63.31%), error:     10622.55664, best error:     10622.55664 on epoch:    19, epoch:   19/50, mu: 0.8105733395, learning_rate: 0.0004938145, weight_decay: 0.0001523648

epoch time: 1651.4492187500s, input_fired_time: 22.6624355316, output_fired_time: 8.1102342606, propagate_forward_time: 524.2620849609, propagate_backward_time: 1076.5310058594, weight_update_time: 0.4906200171, other_time: 19.8834228516
epoch time: 96.5818862915s, input_fired_time: 1.2848626375, output_fired_time: 0.0000000000, propagate_forward_time: 92.2145767212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0824432373
validation[          , genome  5801] predictions:    6347/  10000 (63.47%), best:    6347/10000 (63.47%), error:     10547.78711, best error:     10547.78711 on epoch:    20, epoch:   20/50, mu: 0.8201333284, learning_rate: 0.0004516457, weight_decay: 0.0001412904

epoch time: 54436.8359375000s, input_fired_time: 21.8973388672, output_fired_time: 7.2044630051, propagate_forward_time: 466.5781250000, propagate_backward_time: 53922.6484375000, weight_update_time: 0.4523569345, other_time: 18.5078125000
epoch time: 102.5977935791s, input_fired_time: 1.4362446070, output_fired_time: 0.0000000000, propagate_forward_time: 97.4666519165, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6949005127
validation[          , genome  5801] predictions:    6322/  10000 (63.22%), best:    6347/10000 (63.47%), error:     10643.75781, best error:     10547.78711 on epoch:    20, epoch:   21/50, mu: 0.8291839361, learning_rate: 0.0004130779, weight_decay: 0.0001310208

epoch time: 1525.4082031250s, input_fired_time: 21.4196662903, output_fired_time: 7.0088582039, propagate_forward_time: 466.3602905273, propagate_backward_time: 1013.1106567383, weight_update_time: 0.4396032393, other_time: 17.5087280273
epoch time: 96.7612838745s, input_fired_time: 1.2671338320, output_fired_time: 0.0000000000, propagate_forward_time: 92.5032501221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9908981323
validation[          , genome  5801] predictions:    6294/  10000 (62.94%), best:    6347/10000 (63.47%), error:     10723.02344, best error:     10547.78711 on epoch:    20, epoch:   22/50, mu: 0.8377523422, learning_rate: 0.0003778035, weight_decay: 0.0001214977

epoch time: 2768.8320312500s, input_fired_time: 21.4711818695, output_fired_time: 7.0625205040, propagate_forward_time: 467.0794372559, propagate_backward_time: 2255.6184082031, weight_update_time: 0.4404541552, other_time: 17.6005859375
epoch time: 112.4494552612s, input_fired_time: 1.8585284948, output_fired_time: 0.0000000000, propagate_forward_time: 105.1424713135, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.4484558105
validation[          , genome  5801] predictions:    6244/  10000 (62.44%), best:    6347/10000 (63.47%), error:     10802.59277, best error:     10547.78711 on epoch:    20, epoch:   23/50, mu: 0.8458641768, learning_rate: 0.0003455414, weight_decay: 0.0001126668

epoch time: 7753.2368164063s, input_fired_time: 23.7649097443, output_fired_time: 3629.1401367188, propagate_forward_time: 3006.8015136719, propagate_backward_time: 1065.7547607422, weight_update_time: 0.6137710214, other_time: 27.7752685547
epoch time: 96.6535110474s, input_fired_time: 1.2998766899, output_fired_time: 0.0000000000, propagate_forward_time: 92.4474945068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9061431885
validation[          , genome  5801] predictions:    6357/  10000 (63.57%), best:    6347/10000 (63.47%), error:     10579.80273, best error:     10547.78711 on epoch:    20, epoch:   24/50, mu: 0.8535438180, learning_rate: 0.0003160342, weight_decay: 0.0001044777

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_178_5801_1_r2082023144_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.482211
read mu: 0.860814
read mu_delta: 0.946719
read initial_learning_rate: 0.00269226
read learning_rate: 0.000289047
read learning_rate_delta: 0.914606
read initial_weight_decay: 0.000639082
read weight_decay: 9.68839e-05
read weight_decay_delta: 0.927316
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.0775869
read input_dropout_probability: 0.00117795
read hidden_dropout_probability: 0.10697
read velocity_reset: 2135
read epoch: 25
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 20
read number_validation_images: 10000
read best_validation_predictions: 6347
read best_validation_error: 10547.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5801
read normal distribution: '1 0 0'
generator_str: '353070094'
read generator: 353070094
read generated_by_map:
1 disable_edge 1
reading 100 nodes.
reading 529 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1827.78039550781s, input_fired_time: 24.882287979126, output_fired_time: 8.81767654418945, propagate_forward_time: 601.430603027344, propagate_backward_time: 1171.05712890625, weight_update_time: 0.520254790782928, other_time: 21.5927734375
epoch time: 126.019660949707s, input_fired_time: 1.63683998584747, output_fired_time: 0, propagate_forward_time: 120.353210449219, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.02960968017578
validation[          , genome  5801] predictions:    6357/  10000 (63.57%), best:    6357/10000 (63.57%), error:     10542.81055, best error:     10542.81055 on epoch:    25, epoch:   25/50, mu: 0.8608142734, learning_rate: 0.0002890468, weight_decay: 0.0000968839

epoch time: 1834.2889404297s, input_fired_time: 24.7920818329, output_fired_time: 9.1413078308, propagate_forward_time: 597.3712768555, propagate_backward_time: 1180.0902099609, weight_update_time: 0.5519655347, other_time: 22.8939208984
epoch time: 124.0355072021s, input_fired_time: 1.6316541433, output_fired_time: 0.0000000000, propagate_forward_time: 118.2145996094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1892547607
validation[          , genome  5801] predictions:    6221/  10000 (62.21%), best:    6357/10000 (63.57%), error:     10859.04590, best error:     10542.81055 on epoch:    25, epoch:   26/50, mu: 0.8676973581, learning_rate: 0.0002643639, weight_decay: 0.0000898420

epoch time: 9682.7734375000s, input_fired_time: 23.9556293488, output_fired_time: 8.9277906418, propagate_forward_time: 554.2691650391, propagate_backward_time: 9073.2910156250, weight_update_time: 0.5226812959, other_time: 22.3291015625
epoch time: 96.9666061401s, input_fired_time: 1.4860278368, output_fired_time: 0.0000000000, propagate_forward_time: 92.3806533813, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0999221802
validation[          , genome  5801] predictions:    6403/  10000 (64.03%), best:    6403/10000 (64.03%), error:     10457.67578, best error:     10457.67578 on epoch:    27, epoch:   27/50, mu: 0.8742137551, learning_rate: 0.0002417889, weight_decay: 0.0000833119

epoch time: 1574.7108154297s, input_fired_time: 22.3513069153, output_fired_time: 7.7715983391, propagate_forward_time: 478.2442932129, propagate_backward_time: 1046.4265136719, weight_update_time: 0.4879700243, other_time: 19.9171142578
epoch time: 108.0589981079s, input_fired_time: 1.4234369993, output_fired_time: 0.0000000000, propagate_forward_time: 103.2512512207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3843078613
validation[          , genome  5801] predictions:    6409/  10000 (64.09%), best:    6409/10000 (64.09%), error:     10302.68262, best error:     10302.68262 on epoch:    28, epoch:   28/50, mu: 0.8803829551, learning_rate: 0.0002211416, weight_decay: 0.0000772565

epoch time: 48842.5703125000s, input_fired_time: 31.7234954834, output_fired_time: 11.4105243683, propagate_forward_time: 47568.2773437500, propagate_backward_time: 1196.2518310547, weight_update_time: 0.6558178663, other_time: 34.9083251953
epoch time: 111.5285339355s, input_fired_time: 1.4454691410, output_fired_time: 0.0000000000, propagate_forward_time: 106.7119674683, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3710937500
validation[          , genome  5801] predictions:    6368/  10000 (63.68%), best:    6409/10000 (64.09%), error:     10443.57910, best error:     10302.68262 on epoch:    28, epoch:   29/50, mu: 0.8862234354, learning_rate: 0.0002022574, weight_decay: 0.0000716412

epoch time: 1699.0762939453s, input_fired_time: 25.5192947388, output_fired_time: 11.2057170868, propagate_forward_time: 518.8856201172, propagate_backward_time: 1110.5466308594, weight_update_time: 0.6895014048, other_time: 32.9190673828
epoch time: 104.1383514404s, input_fired_time: 1.5024548769, output_fired_time: 0.0000000000, propagate_forward_time: 99.1386260986, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4972686768
validation[          , genome  5801] predictions:    6430/  10000 (64.30%), best:    6409/10000 (64.09%), error:     10411.98340, best error:     10302.68262 on epoch:    28, epoch:   30/50, mu: 0.8917527199, learning_rate: 0.0001849858, weight_decay: 0.0000664340

epoch time: 1717.2467041016s, input_fired_time: 24.7139625549, output_fired_time: 9.7036628723, propagate_forward_time: 524.7399291992, propagate_backward_time: 1133.4346923828, weight_update_time: 0.5801870823, other_time: 24.6544189453
epoch time: 108.4142608643s, input_fired_time: 1.6429799795, output_fired_time: 0.0000000000, propagate_forward_time: 102.7947540283, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9765243530
validation[          , genome  5801] predictions:    6351/  10000 (63.51%), best:    6409/10000 (64.09%), error:     10626.85449, best error:     10302.68262 on epoch:    28, epoch:   31/50, mu: 0.8969873786, learning_rate: 0.0001691892, weight_decay: 0.0000616054

epoch time: 1606.4730224609s, input_fired_time: 22.2893409729, output_fired_time: 7.8812813759, propagate_forward_time: 492.8747558594, propagate_backward_time: 1064.4981689453, weight_update_time: 0.4869060814, other_time: 18.9295654297
epoch time: 107.4897689819s, input_fired_time: 1.5064407587, output_fired_time: 0.0000000000, propagate_forward_time: 102.4088439941, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5744857788
validation[          , genome  5801] predictions:    6414/  10000 (64.14%), best:    6409/10000 (64.09%), error:     10400.50781, best error:     10302.68262 on epoch:    28, epoch:   32/50, mu: 0.9019431472, learning_rate: 0.0001547414, weight_decay: 0.0000571276

epoch time: 1585.9926757813s, input_fired_time: 22.0333423615, output_fired_time: 7.4968395233, propagate_forward_time: 486.6692810059, propagate_backward_time: 1051.5826416016, weight_update_time: 0.4621501863, other_time: 18.2105712891
epoch time: 104.6646575928s, input_fired_time: 1.4499748945, output_fired_time: 0.0000000000, propagate_forward_time: 99.8531494141, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3615341187
validation[          , genome  5801] predictions:    6315/  10000 (63.15%), best:    6409/10000 (64.09%), error:     10535.75195, best error:     10302.68262 on epoch:    28, epoch:   33/50, mu: 0.9066348672, learning_rate: 0.0001415274, weight_decay: 0.0000529754

epoch time: 2679.7416992188s, input_fired_time: 23.5272579193, output_fired_time: 8.7270593643, propagate_forward_time: 1519.6176757813, propagate_backward_time: 1107.3400878906, weight_update_time: 0.5218904614, other_time: 20.5295410156
epoch time: 110.2872543335s, input_fired_time: 1.6930590868, output_fired_time: 0.0000000000, propagate_forward_time: 104.7257232666, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8684692383
validation[          , genome  5801] predictions:    6442/  10000 (64.42%), best:    6442/10000 (64.42%), error:     10187.32520, best error:     10187.32520 on epoch:    34, epoch:   34/50, mu: 0.9110766053, learning_rate: 0.0001294419, weight_decay: 0.0000491249

epoch time: 1626.3972167969s, input_fired_time: 23.0559635162, output_fired_time: 8.1861391068, propagate_forward_time: 495.7090148926, propagate_backward_time: 1079.3211669922, weight_update_time: 0.5112510324, other_time: 20.1250000000
epoch time: 99.0748062134s, input_fired_time: 1.3284080029, output_fired_time: 0.0000000000, propagate_forward_time: 94.7088546753, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0375442505
validation[          , genome  5801] predictions:    6464/  10000 (64.64%), best:    6442/10000 (64.42%), error:     10287.30176, best error:     10187.32520 on epoch:    34, epoch:   35/50, mu: 0.9152817130, learning_rate: 0.0001183883, weight_decay: 0.0000455543

epoch time: 1692.4305419922s, input_fired_time: 25.1965847015, output_fired_time: 10.2870550156, propagate_forward_time: 511.7866210938, propagate_backward_time: 1115.8631591797, weight_update_time: 0.6396414638, other_time: 29.2971191406
epoch time: 119.2304611206s, input_fired_time: 2.4904873371, output_fired_time: 0.0000000000, propagate_forward_time: 109.4069976807, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.3329772949
validation[          , genome  5801] predictions:    6415/  10000 (64.15%), best:    6442/10000 (64.42%), error:     10326.19727, best error:     10187.32520 on epoch:    34, epoch:   36/50, mu: 0.9192627668, learning_rate: 0.0001082787, weight_decay: 0.0000422433

epoch time: 3158.6948242188s, input_fired_time: 26.4956111908, output_fired_time: 11.6635608673, propagate_forward_time: 1925.2807617188, propagate_backward_time: 1164.6621093750, weight_update_time: 0.7104067802, other_time: 30.5927734375
epoch time: 110.7205505371s, input_fired_time: 1.5132434368, output_fired_time: 0.0000000000, propagate_forward_time: 105.6684570313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5388488770
validation[          , genome  5801] predictions:    6430/  10000 (64.30%), best:    6430/10000 (64.30%), error:     10161.42969, best error:     10161.42969 on epoch:    37, epoch:   37/50, mu: 0.9230316877, learning_rate: 0.0000990323, weight_decay: 0.0000391729

epoch time: 1677.4071044922s, input_fired_time: 23.7583694458, output_fired_time: 8.9976005554, propagate_forward_time: 509.1764526367, propagate_backward_time: 1113.9947509766, weight_update_time: 0.5285754204, other_time: 21.4798583984
epoch time: 108.0714721680s, input_fired_time: 1.6215962172, output_fired_time: 0.0000000000, propagate_forward_time: 102.8330230713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6168518066
validation[          , genome  5801] predictions:    6444/  10000 (64.44%), best:    6444/10000 (64.44%), error:     10140.84766, best error:     10140.84766 on epoch:    38, epoch:   38/50, mu: 0.9265998006, learning_rate: 0.0000905756, weight_decay: 0.0000363256

epoch time: 9800.6005859375s, input_fired_time: 25.0456180573, output_fired_time: 9.9007883072, propagate_forward_time: 8589.0410156250, propagate_backward_time: 1151.9509277344, weight_update_time: 0.5999177694, other_time: 24.6623535156
epoch time: 102.3739776611s, input_fired_time: 1.4174228907, output_fired_time: 0.0000000000, propagate_forward_time: 97.4929199219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4636383057
validation[          , genome  5801] predictions:    6462/  10000 (64.62%), best:    6444/10000 (64.44%), error:     10247.90820, best error:     10140.84766 on epoch:    38, epoch:   39/50, mu: 0.9299778342, learning_rate: 0.0000828409, weight_decay: 0.0000336853

epoch time: 1680.5906982422s, input_fired_time: 24.6448764801, output_fired_time: 9.2104482651, propagate_forward_time: 510.5622558594, propagate_backward_time: 1112.7708740234, weight_update_time: 0.5699152946, other_time: 23.4022216797
epoch time: 100.0041885376s, input_fired_time: 1.3631811142, output_fired_time: 0.0000000000, propagate_forward_time: 95.3687667847, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2722396851
validation[          , genome  5801] predictions:    6549/  10000 (65.49%), best:    6549/10000 (65.49%), error:     10120.37500, best error:     10120.37500 on epoch:    40, epoch:   40/50, mu: 0.9331758618, learning_rate: 0.0000757668, weight_decay: 0.0000312370

epoch time: 1568.4888916016s, input_fired_time: 22.4914474487, output_fired_time: 7.6351604462, propagate_forward_time: 476.5035400391, propagate_backward_time: 1042.7714843750, weight_update_time: 0.4749405980, other_time: 19.0872802734
epoch time: 101.1606140137s, input_fired_time: 1.4232751131, output_fired_time: 0.0000000000, propagate_forward_time: 96.3110809326, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4262542725
validation[          , genome  5801] predictions:    6530/  10000 (65.30%), best:    6530/10000 (65.30%), error:     10048.46484, best error:     10048.46484 on epoch:    41, epoch:   41/50, mu: 0.9362034798, learning_rate: 0.0000692968, weight_decay: 0.0000289665

epoch time: 1574.4299316406s, input_fired_time: 22.4050884247, output_fired_time: 7.3064289093, propagate_forward_time: 481.5507812500, propagate_backward_time: 1044.6944580078, weight_update_time: 0.4612295926, other_time: 18.4732666016
epoch time: 106.0792694092s, input_fired_time: 1.4911367893, output_fired_time: 0.0000000000, propagate_forward_time: 101.0635833740, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5245513916
validation[          , genome  5801] predictions:    6558/  10000 (65.58%), best:    6558/10000 (65.58%), error:      9988.85254, best error:      9988.85254 on epoch:    42, epoch:   42/50, mu: 0.9390698075, learning_rate: 0.0000633793, weight_decay: 0.0000268611

epoch time: 1627.4205322266s, input_fired_time: 22.9012584686, output_fired_time: 8.1453189850, propagate_forward_time: 496.4787902832, propagate_backward_time: 1080.0427246094, weight_update_time: 0.4892915785, other_time: 19.8525390625
epoch time: 103.6077728271s, input_fired_time: 1.4484831095, output_fired_time: 0.0000000000, propagate_forward_time: 98.8311691284, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3281173706
validation[          , genome  5801] predictions:    6594/  10000 (65.94%), best:    6594/10000 (65.94%), error:      9954.15234, best error:      9954.15234 on epoch:    43, epoch:   43/50, mu: 0.9417834282, learning_rate: 0.0000579671, weight_decay: 0.0000249088

epoch time: 1621.7515869141s, input_fired_time: 22.6851253510, output_fired_time: 7.8529067039, propagate_forward_time: 498.4843139648, propagate_backward_time: 1074.2354736328, weight_update_time: 0.4844526649, other_time: 18.4937744141
epoch time: 101.7138137817s, input_fired_time: 1.3746716976, output_fired_time: 0.0000000000, propagate_forward_time: 97.3813781738, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9577636719
validation[          , genome  5801] predictions:    6542/  10000 (65.42%), best:    6594/10000 (65.94%), error:     10033.29297, best error:      9954.15234 on epoch:    43, epoch:   44/50, mu: 0.9443524480, learning_rate: 0.0000530170, weight_decay: 0.0000230983

epoch time: 1593.8485107422s, input_fired_time: 22.3614196777, output_fired_time: 7.4098134041, propagate_forward_time: 489.9221801758, propagate_backward_time: 1056.4875488281, weight_update_time: 0.4575650692, other_time: 17.6674804688
epoch time: 104.6030197144s, input_fired_time: 1.4740993977, output_fired_time: 0.0000000000, propagate_forward_time: 99.7823944092, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3465270996
validation[          , genome  5801] predictions:    6529/  10000 (65.29%), best:    6594/10000 (65.94%), error:     10058.44434, best error:      9954.15234 on epoch:    43, epoch:   45/50, mu: 0.9467845559, learning_rate: 0.0000484897, weight_decay: 0.0000214194

epoch time: 1618.8642578125s, input_fired_time: 23.0480937958, output_fired_time: 7.9779319763, propagate_forward_time: 493.4815979004, propagate_backward_time: 1074.3183593750, weight_update_time: 0.4895506799, other_time: 20.0383300781
epoch time: 104.7480697632s, input_fired_time: 1.4678767920, output_fired_time: 0.0000000000, propagate_forward_time: 99.8282623291, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4519271851
validation[          , genome  5801] predictions:    6582/  10000 (65.82%), best:    6582/10000 (65.82%), error:      9895.44336, best error:      9895.44336 on epoch:    46, epoch:   46/50, mu: 0.9490870833, learning_rate: 0.0000443490, weight_decay: 0.0000198626

epoch time: 1604.7315673828s, input_fired_time: 22.7186737061, output_fired_time: 7.6855473518, propagate_forward_time: 489.8790588379, propagate_backward_time: 1065.6533203125, weight_update_time: 0.4785623848, other_time: 18.7950439453
epoch time: 99.4205474854s, input_fired_time: 1.3694885969, output_fired_time: 0.0000000000, propagate_forward_time: 95.0298767090, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0211791992
validation[          , genome  5801] predictions:    6620/  10000 (66.20%), best:    6620/10000 (66.20%), error:      9873.59180, best error:      9873.59180 on epoch:    47, epoch:   47/50, mu: 0.9512669444, learning_rate: 0.0000405618, weight_decay: 0.0000184189

epoch time: 1561.0679931641s, input_fired_time: 22.2663898468, output_fired_time: 7.0927000046, propagate_forward_time: 478.2416076660, propagate_backward_time: 1036.1761474609, weight_update_time: 0.4500157237, other_time: 17.2912597656
epoch time: 102.2982330322s, input_fired_time: 1.4558166265, output_fired_time: 0.0000000000, propagate_forward_time: 97.4971008301, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3453140259
validation[          , genome  5801] predictions:    6548/  10000 (65.48%), best:    6620/10000 (66.20%), error:      9918.03223, best error:      9873.59180 on epoch:    47, epoch:   48/50, mu: 0.9533306956, learning_rate: 0.0000370981, weight_decay: 0.0000170801

epoch time: 45228.4726562500s, input_fired_time: 23.5265064240, output_fired_time: 8.2808256149, propagate_forward_time: 507.5452270508, propagate_backward_time: 43204.7617187500, weight_update_time: 0.5057985783, other_time: 1484.3554687500
epoch time: 112.7266235352s, input_fired_time: 1.5144294500, output_fired_time: 0.0000000000, propagate_forward_time: 107.7079620361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5042343140
validation[          , genome  5801] predictions:    6623/  10000 (66.23%), best:    6623/10000 (66.23%), error:      9788.38379, best error:      9788.38379 on epoch:    49, epoch:   49/50, mu: 0.9552844763, learning_rate: 0.0000339301, weight_decay: 0.0000158387

epoch time: 1700.3392333984s, input_fired_time: 23.6237640381, output_fired_time: 8.5752248764, propagate_forward_time: 539.1845703125, propagate_backward_time: 1108.4674072266, weight_update_time: 0.5073259473, other_time: 20.4882812500
epoch time: 116.9034576416s, input_fired_time: 1.6902111769, output_fired_time: 0.0000000000, propagate_forward_time: 111.0254821777, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1877670288
validation[          , genome  5801] predictions:    6642/  10000 (66.42%), best:    6623/10000 (66.23%), error:      9827.93945, best error:      9788.38379 on epoch:    49, epoch:   50/50, mu: 0.9571341276, learning_rate: 0.0000310327, weight_decay: 0.0000146875

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 607.8278808594s, input_fired_time: 8.6675243378, output_fired_time: 0.0000000000, propagate_forward_time: 577.3622436523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 21.7980957031
best training[          , genome  5801] predictions:   36942/  50000 (73.88%), best:    6623/10000 (66.23%), error:     39725.50781, best error:      9788.38379 on epoch:    49, epoch:   51/50, mu: 0.9588852525, learning_rate: 0.0000283827, weight_decay: 0.0000136199
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 117.9560852051s, input_fired_time: 1.6615135670, output_fired_time: 0.0000000000, propagate_forward_time: 112.4891204834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8054504395
   testing[          , genome  5801] predictions:    6623/  10000 (66.23%), best:    6623/10000 (66.23%), error:      9788.38379, best error:      9788.38379 on epoch:    49, epoch:   51/50, mu: 0.9588852525, learning_rate: 0.0000283827, weight_decay: 0.0000136199
backpropagation finished successfully!
10:02:29 (5124): called boinc_finish(0)

</stderr_txt>
]]>