Nameexact_genome_1542031203_181_5605_1
Workunit4081695
Created12 Nov 2018, 19:22:47 UTC
Sent12 Nov 2018, 19:27:31 UTC
Report deadline1 Jan 2019, 16:17:57 UTC
Received10 Dec 2018, 4:36:15 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64195
Run time20 hours 3 min 14 sec
CPU time20 hours 2 min 42 sec
Validate stateValid
Credit12,467.09
Device peak FLOPS3.83 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size552.11 MB
Peak swap size571.76 MB
Peak disk usage10.80 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_1_r1539389976_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.488702
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.00563086
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 0.000368483
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1515447827'
read generator: 1515447827
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 975574969
post shuffle 1: 45542896
epoch time: 1775.36706542969s, input_fired_time: 13.8135709762573, output_fired_time: 5.03761672973633, propagate_forward_time: 358.935211181641, propagate_backward_time: 1387.52075195313, weight_update_time: 0.240050241351128, other_time: 10.0599365234375
epoch time: 94.7740173339844s, input_fired_time: 0.962759852409363, output_fired_time: 0, propagate_forward_time: 91.8953857421875, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.91587066650391
validation[          , genome  5605] predictions:    4980/  10000 (49.80%), best:    4980/10000 (49.80%), error:     14860.61523, best error:     14860.61523 on epoch:     0, epoch:    0/50, mu: 0.4887023270, learning_rate: 0.0056308587, weight_decay: 0.0003684833

epoch time: 1121.4864501953s, input_fired_time: 15.1602792740, output_fired_time: 5.7198314667, propagate_forward_time: 418.0392150879, propagate_backward_time: 671.5582275391, weight_update_time: 0.2709484994, other_time: 11.0089111328
epoch time: 100.4666976929s, input_fired_time: 1.0171552896, output_fired_time: 0.0000000000, propagate_forward_time: 97.4720993042, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9774398804
validation[          , genome  5605] predictions:    5390/  10000 (53.90%), best:    5390/10000 (53.90%), error:     13248.95313, best error:     13248.95313 on epoch:     1, epoch:    1/50, mu: 0.5208209753, learning_rate: 0.0051687951, weight_decay: 0.0003427038

epoch time: 1203.1240234375s, input_fired_time: 15.5030555725, output_fired_time: 6.0671525002, propagate_forward_time: 451.3771972656, propagate_backward_time: 718.8324584961, weight_update_time: 0.2848240137, other_time: 11.3441772461
epoch time: 73.3519210815s, input_fired_time: 0.8206232786, output_fired_time: 0.0000000000, propagate_forward_time: 70.7944869995, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7368087769
validation[          , genome  5605] predictions:    5474/  10000 (54.74%), best:    5474/10000 (54.74%), error:     12975.93652, best error:     12975.93652 on epoch:     2, epoch:    2/50, mu: 0.5508817434, learning_rate: 0.0047446480, weight_decay: 0.0003187278

epoch time: 1211.1639404297s, input_fired_time: 15.4303798676, output_fired_time: 6.1003608704, propagate_forward_time: 454.9471740723, propagate_backward_time: 723.3786621094, weight_update_time: 0.2848703563, other_time: 11.3073730469
epoch time: 63.8945465088s, input_fired_time: 0.7221863866, output_fired_time: 0.0000000000, propagate_forward_time: 61.6722450256, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5001144409
validation[          , genome  5605] predictions:    5517/  10000 (55.17%), best:    5517/10000 (55.17%), error:     12849.39648, best error:     12849.39648 on epoch:     3, epoch:    3/50, mu: 0.5790165067, learning_rate: 0.0043553058, weight_decay: 0.0002964293

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_1_r1539389976_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.605349
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.00399791
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 0.000275691
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 4
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 3
read number_validation_images: 10000
read best_validation_predictions: 5517
read best_validation_error: 12849.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '269847327'
read generator: 269847327
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1058.11010742188s, input_fired_time: 14.7031669616699, output_fired_time: 5.38876438140869, propagate_forward_time: 390.436157226563, propagate_backward_time: 637.276916503906, weight_update_time: 0.253437250852585, other_time: 10.3051147460938
epoch time: 75.8376770019531s, input_fired_time: 0.831904828548431, output_fired_time: 0, propagate_forward_time: 73.3293914794922, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.67638397216797
validation[          , genome  5605] predictions:    5514/  10000 (55.14%), best:    5517/10000 (55.17%), error:     12934.84766, best error:     12849.39648 on epoch:     3, epoch:    4/50, mu: 0.6053486466, learning_rate: 0.0039979131, weight_decay: 0.0002756907

epoch time: 1148.8029785156s, input_fired_time: 15.0610857010, output_fired_time: 5.7442688942, propagate_forward_time: 428.5685119629, propagate_backward_time: 688.7807006836, weight_update_time: 0.2682125568, other_time: 10.6484985352
epoch time: 94.0196533203s, input_fired_time: 0.9674219489, output_fired_time: 0.0000000000, propagate_forward_time: 91.2395401001, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8126907349
validation[          , genome  5605] predictions:    5430/  10000 (54.30%), best:    5517/10000 (55.17%), error:     13167.00391, best error:     12849.39648 on epoch:     3, epoch:    5/50, mu: 0.6299936771, learning_rate: 0.0036698475, weight_decay: 0.0002564031

epoch time: 1179.0975341797s, input_fired_time: 15.1474227905, output_fired_time: 5.8766851425, propagate_forward_time: 440.3870239258, propagate_backward_time: 706.9602661133, weight_update_time: 0.2725893557, other_time: 10.7260742188
epoch time: 81.1659469604s, input_fired_time: 0.8823183179, output_fired_time: 0.0000000000, propagate_forward_time: 78.5604019165, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7232284546
validation[          , genome  5605] predictions:    5588/  10000 (55.88%), best:    5588/10000 (55.88%), error:     12620.12500, best error:     12620.12500 on epoch:     6, epoch:    6/50, mu: 0.6530596614, learning_rate: 0.0033687027, weight_decay: 0.0002384649

epoch time: 1237.1945800781s, input_fired_time: 15.4167194366, output_fired_time: 6.1452331543, propagate_forward_time: 465.0687255859, propagate_backward_time: 739.4866333008, weight_update_time: 0.2836180329, other_time: 11.0772094727
epoch time: 81.5597763062s, input_fired_time: 0.8855446577, output_fired_time: 0.0000000000, propagate_forward_time: 78.9301681519, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7440643311
validation[          , genome  5605] predictions:    5645/  10000 (56.45%), best:    5645/10000 (56.45%), error:     12588.16113, best error:     12588.16113 on epoch:     7, epoch:    7/50, mu: 0.6746478081, learning_rate: 0.0030922696, weight_decay: 0.0002217816

epoch time: 1210.8140869141s, input_fired_time: 15.3389911652, output_fired_time: 6.0698595047, propagate_forward_time: 453.6705017090, propagate_backward_time: 724.7460937500, weight_update_time: 0.2809198797, other_time: 10.9886474609
epoch time: 94.1994400024s, input_fired_time: 0.9862239957, output_fired_time: 0.0000000000, propagate_forward_time: 91.3424224854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8707962036
validation[          , genome  5605] predictions:    5410/  10000 (54.10%), best:    5645/10000 (56.45%), error:     13456.99902, best error:     12588.16113 on epoch:     7, epoch:    8/50, mu: 0.6948527694, learning_rate: 0.0028385206, weight_decay: 0.0002062655

epoch time: 1265.9759521484s, input_fired_time: 15.5041065216, output_fired_time: 6.2239623070, propagate_forward_time: 477.8347473145, propagate_backward_time: 755.3473510742, weight_update_time: 0.2880368829, other_time: 11.0657348633
epoch time: 73.1924514771s, input_fired_time: 0.8216335177, output_fired_time: 0.0000000000, propagate_forward_time: 70.7305831909, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6402359009
validation[          , genome  5605] predictions:    5695/  10000 (56.95%), best:    5695/10000 (56.95%), error:     12312.62988, best error:     12312.62988 on epoch:     9, epoch:    9/50, mu: 0.7137631774, learning_rate: 0.0026055938, weight_decay: 0.0001918349

epoch time: 1240.3696289063s, input_fired_time: 15.3677005768, output_fired_time: 6.0535016060, propagate_forward_time: 466.9584350586, propagate_backward_time: 741.1054077148, weight_update_time: 0.2831417918, other_time: 10.8846435547
epoch time: 106.0465774536s, input_fired_time: 1.0587911606, output_fired_time: 0.0000000000, propagate_forward_time: 103.1001052856, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8876800537
validation[          , genome  5605] predictions:    5787/  10000 (57.87%), best:    5787/10000 (57.87%), error:     12039.70605, best error:     12039.70605 on epoch:    10, epoch:   10/50, mu: 0.7314619422, learning_rate: 0.0023917807, weight_decay: 0.0001784139

epoch time: 1218.6630859375s, input_fired_time: 15.3781795502, output_fired_time: 6.0641317368, propagate_forward_time: 457.4363403320, propagate_backward_time: 728.8205566406, weight_update_time: 0.2794159055, other_time: 10.9639282227
epoch time: 99.8295288086s, input_fired_time: 1.0267819166, output_fired_time: 0.0000000000, propagate_forward_time: 96.8746185303, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9281311035
validation[          , genome  5605] predictions:    5705/  10000 (57.05%), best:    5787/10000 (57.87%), error:     12310.95410, best error:     12039.70605 on epoch:    10, epoch:   11/50, mu: 0.7480267286, learning_rate: 0.0021955131, weight_decay: 0.0001659319

epoch time: 1408.0155029297s, input_fired_time: 16.0642719269, output_fired_time: 6.8110651970, propagate_forward_time: 536.8168334961, propagate_backward_time: 836.6359863281, weight_update_time: 0.3127976060, other_time: 11.6873168945
epoch time: 127.9272155762s, input_fired_time: 1.2027796507, output_fired_time: 0.0000000000, propagate_forward_time: 124.6501541138, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0742797852
validation[          , genome  5605] predictions:    5817/  10000 (58.17%), best:    5787/10000 (57.87%), error:     12041.52734, best error:     12039.70605 on epoch:    10, epoch:   12/50, mu: 0.7635301948, learning_rate: 0.0020153511, weight_decay: 0.0001543231

epoch time: 1518.3736572266s, input_fired_time: 16.4721431732, output_fired_time: 7.2827901840, propagate_forward_time: 582.1050415039, propagate_backward_time: 900.3631591797, weight_update_time: 0.3307801485, other_time: 12.1504516602
epoch time: 107.3562927246s, input_fired_time: 1.0625784397, output_fired_time: 0.0000000000, propagate_forward_time: 104.4105224609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8831939697
validation[          , genome  5605] predictions:    5966/  10000 (59.66%), best:    5966/10000 (59.66%), error:     11762.57031, best error:     11762.57031 on epoch:    13, epoch:   13/50, mu: 0.7780403495, learning_rate: 0.0018499730, weight_decay: 0.0001435265

epoch time: 1233.1987304688s, input_fired_time: 15.4370965958, output_fired_time: 6.0869336128, propagate_forward_time: 462.6408386230, propagate_backward_time: 738.0902099609, weight_update_time: 0.2806058228, other_time: 10.9436035156
epoch time: 132.9940338135s, input_fired_time: 1.2818560600, output_fired_time: 0.0000000000, propagate_forward_time: 129.4546356201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2575378418
validation[          , genome  5605] predictions:    5698/  10000 (56.98%), best:    5966/10000 (59.66%), error:     12458.27539, best error:     11762.57031 on epoch:    13, epoch:   14/50, mu: 0.7916208506, learning_rate: 0.0016981657, weight_decay: 0.0001334852

epoch time: 1444.2940673828s, input_fired_time: 16.1745586395, output_fired_time: 6.9237861633, propagate_forward_time: 551.7728881836, propagate_backward_time: 857.6719970703, weight_update_time: 0.3183958232, other_time: 11.7507934570
epoch time: 118.5386123657s, input_fired_time: 1.1438167095, output_fired_time: 0.0000000000, propagate_forward_time: 115.4205627441, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9742355347
validation[          , genome  5605] predictions:    5919/  10000 (59.19%), best:    5919/10000 (59.19%), error:     11740.24316, best error:     11740.24316 on epoch:    15, epoch:   15/50, mu: 0.8043311834, learning_rate: 0.0015588156, weight_decay: 0.0001241464

epoch time: 1386.7548828125s, input_fired_time: 15.9879980087, output_fired_time: 6.6974787712, propagate_forward_time: 526.7145385742, propagate_backward_time: 825.7782592773, weight_update_time: 0.3085503280, other_time: 11.5765380859
epoch time: 109.2785644531s, input_fired_time: 1.0954810381, output_fired_time: 0.0000000000, propagate_forward_time: 106.2139587402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9691238403
validation[          , genome  5605] predictions:    5974/  10000 (59.74%), best:    5974/10000 (59.74%), error:     11652.47949, best error:     11652.47949 on epoch:    16, epoch:   16/50, mu: 0.8162271976, learning_rate: 0.0014309004, weight_decay: 0.0001154610

epoch time: 1477.9124755859s, input_fired_time: 16.3181381226, output_fired_time: 7.0685601234, propagate_forward_time: 565.2443847656, propagate_backward_time: 877.3988647461, weight_update_time: 0.3227675259, other_time: 11.8825073242
epoch time: 112.9461212158s, input_fired_time: 1.1284703016, output_fired_time: 0.0000000000, propagate_forward_time: 109.8096084595, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0080413818
validation[          , genome  5605] predictions:    5981/  10000 (59.81%), best:    5981/10000 (59.81%), error:     11547.37109, best error:     11547.37109 on epoch:    17, epoch:   17/50, mu: 0.8273609877, learning_rate: 0.0013134818, weight_decay: 0.0001073832

epoch time: 1391.5861816406s, input_fired_time: 16.0307483673, output_fired_time: 6.7015719414, propagate_forward_time: 530.8034667969, propagate_backward_time: 826.5109863281, weight_update_time: 0.3096363842, other_time: 11.5394287109
epoch time: 99.2464675903s, input_fired_time: 1.0178623199, output_fired_time: 0.0000000000, propagate_forward_time: 96.3792419434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8493652344
validation[          , genome  5605] predictions:    6022/  10000 (60.22%), best:    6022/10000 (60.22%), error:     11480.62207, best error:     11480.62207 on epoch:    18, epoch:   18/50, mu: 0.8377814293, learning_rate: 0.0012056986, weight_decay: 0.0000998706

epoch time: 1405.2159423828s, input_fired_time: 16.0159950256, output_fired_time: 6.7203345299, propagate_forward_time: 533.8037719727, propagate_backward_time: 837.2396850586, weight_update_time: 0.3098866343, other_time: 11.4361572266
epoch time: 105.8063430786s, input_fired_time: 1.0687948465, output_fired_time: 0.0000000000, propagate_forward_time: 102.8605117798, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8770370483
validation[          , genome  5605] predictions:    5867/  10000 (58.67%), best:    6022/10000 (60.22%), error:     11773.58203, best error:     11480.62207 on epoch:    18, epoch:   19/50, mu: 0.8475342393, learning_rate: 0.0011067599, weight_decay: 0.0000928835

epoch time: 1392.3010253906s, input_fired_time: 16.0407924652, output_fired_time: 6.6772065163, propagate_forward_time: 532.7765502930, propagate_backward_time: 825.2470092773, weight_update_time: 0.3101774156, other_time: 11.5594482422
epoch time: 116.2314682007s, input_fired_time: 1.1596037149, output_fired_time: 0.0000000000, propagate_forward_time: 113.0692901611, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0025711060
validation[          , genome  5605] predictions:    5981/  10000 (59.81%), best:    6022/10000 (60.22%), error:     11515.49902, best error:     11480.62207 on epoch:    18, epoch:   20/50, mu: 0.8566621542, learning_rate: 0.0010159401, weight_decay: 0.0000863853

epoch time: 1448.2584228516s, input_fired_time: 16.2313060760, output_fired_time: 6.8814716339, propagate_forward_time: 553.8300170898, propagate_backward_time: 859.6769409180, weight_update_time: 0.3154063225, other_time: 11.6386718750
epoch time: 115.4219436646s, input_fired_time: 1.1402180195, output_fired_time: 0.0000000000, propagate_forward_time: 112.3037643433, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9779586792
validation[          , genome  5605] predictions:    5996/  10000 (59.96%), best:    5996/10000 (59.96%), error:     11388.03711, best error:     11388.03711 on epoch:    21, epoch:   21/50, mu: 0.8652052283, learning_rate: 0.0009325729, weight_decay: 0.0000803417

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_1_r1539389976_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.873201
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000856047
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 7.47209e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 22
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 21
read number_validation_images: 10000
read best_validation_predictions: 5996
read best_validation_error: 11388
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1154572065'
read generator: 1154572065
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1824.53063964844s, input_fired_time: 41.2803077697754, output_fired_time: 28.5738067626953, propagate_forward_time: 646.083923339844, propagate_backward_time: 1035.26330566406, weight_update_time: 1.9209052324295, other_time: 73.3292236328125
epoch time: 152.950805664063s, input_fired_time: 5.47535037994385, output_fired_time: 0, propagate_forward_time: 133.444442749023, propagate_backward_time: 0, weight_update_time: 0, other_time: 14.031005859375
validation[          , genome  5605] predictions:    5857/  10000 (58.57%), best:    5996/10000 (59.96%), error:     11854.54688, best error:     11388.03711 on epoch:    21, epoch:   22/50, mu: 0.8732009530, learning_rate: 0.0008560467, weight_decay: 0.0000747209

epoch time: 1857.3472900391s, input_fired_time: 41.7028465271, output_fired_time: 28.1077499390, propagate_forward_time: 660.4255371094, propagate_backward_time: 1054.3427734375, weight_update_time: 1.8654268980, other_time: 72.7683105469
epoch time: 153.9629516602s, input_fired_time: 5.3890113831, output_fired_time: 0.0000000000, propagate_forward_time: 134.5033416748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 14.0706024170
validation[          , genome  5605] predictions:    6017/  10000 (60.17%), best:    5996/10000 (59.96%), error:     11409.91406, best error:     11388.03711 on epoch:    21, epoch:   23/50, mu: 0.8806843758, learning_rate: 0.0007858002, weight_decay: 0.0000694933

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542031203_181_5605_1_r1539389976_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488702
read mu: 0.887688
read mu_delta: 0.935929
read initial_learning_rate: 0.00563086
read learning_rate: 0.000721318
read learning_rate_delta: 0.917941
read initial_weight_decay: 0.000368483
read weight_decay: 6.46315e-05
read weight_decay_delta: 0.930039
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.047146
read input_dropout_probability: 0.00137303
read hidden_dropout_probability: 0.112932
read velocity_reset: 1030
read epoch: 24
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 21
read number_validation_images: 10000
read best_validation_predictions: 5996
read best_validation_error: 11388
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5605
read normal distribution: '1 0 0'
generator_str: '1895482414'
read generator: 1895482414
read generated_by_map:
1 enable_edge 1
reading 85 nodes.
reading 318 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1577.02746582031s, input_fired_time: 27.2372665405273, output_fired_time: 14.7288446426392, propagate_forward_time: 573.046997070313, propagate_backward_time: 917.945251464844, weight_update_time: 1.01972854137421, other_time: 44.0690307617188
epoch time: 126.664985656738s, input_fired_time: 3.05870079994202, output_fired_time: 0, propagate_forward_time: 115.175338745117, propagate_backward_time: 0, weight_update_time: 0, other_time: 8.43094635009766
validation[          , genome  5605] predictions:    6067/  10000 (60.67%), best:    6067/10000 (60.67%), error:     11381.92578, best error:     11381.92578 on epoch:    24, epoch:   24/50, mu: 0.8876883388, learning_rate: 0.0007213180, weight_decay: 0.0000646315

epoch time: 1489.1602783203s, input_fired_time: 25.9795589447, output_fired_time: 13.7686700821, propagate_forward_time: 541.2337646484, propagate_backward_time: 868.0314941406, weight_update_time: 0.9721640348, other_time: 40.1467285156
epoch time: 124.4721145630s, input_fired_time: 2.9580721855, output_fired_time: 0.0000000000, propagate_forward_time: 112.8039245605, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.7101211548
validation[          , genome  5605] predictions:    6099/  10000 (60.99%), best:    6099/10000 (60.99%), error:     11259.61328, best error:     11259.61328 on epoch:    25, epoch:   25/50, mu: 0.8942435384, learning_rate: 0.0006621272, weight_decay: 0.0000601098

epoch time: 1503.9913330078s, input_fired_time: 26.5127811432, output_fired_time: 14.1265420914, propagate_forward_time: 545.2294311523, propagate_backward_time: 877.3690795898, weight_update_time: 0.9817267656, other_time: 40.7534179688
epoch time: 116.4567337036s, input_fired_time: 2.6061067581, output_fired_time: 0.0000000000, propagate_forward_time: 107.0491256714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.8014984131
validation[          , genome  5605] predictions:    6019/  10000 (60.19%), best:    6099/10000 (60.99%), error:     11383.51465, best error:     11259.61328 on epoch:    25, epoch:   26/50, mu: 0.9003787637, learning_rate: 0.0006077936, weight_decay: 0.0000559045

epoch time: 1649.4810791016s, input_fired_time: 27.3245048523, output_fired_time: 14.7674026489, propagate_forward_time: 602.3029174805, propagate_backward_time: 960.5462036133, weight_update_time: 1.0419540405, other_time: 44.5400390625
epoch time: 136.2507934570s, input_fired_time: 2.9785304070, output_fired_time: 0.0000000000, propagate_forward_time: 124.4552001953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.8170623779
validation[          , genome  5605] predictions:    6071/  10000 (60.71%), best:    6099/10000 (60.99%), error:     11360.15820, best error:     11259.61328 on epoch:    25, epoch:   27/50, mu: 0.9061208963, learning_rate: 0.0005579185, weight_decay: 0.0000519933

epoch time: 1646.8521728516s, input_fired_time: 27.3466300964, output_fired_time: 14.9246330261, propagate_forward_time: 599.8103027344, propagate_backward_time: 959.4995727539, weight_update_time: 1.0517319441, other_time: 45.2709350586
epoch time: 123.1276855469s, input_fired_time: 2.6897435188, output_fired_time: 0.0000000000, propagate_forward_time: 112.7344589233, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.7034835815
validation[          , genome  5605] predictions:    6095/  10000 (60.95%), best:    6099/10000 (60.99%), error:     11267.19336, best error:     11259.61328 on epoch:    25, epoch:   28/50, mu: 0.9114951491, learning_rate: 0.0005121361, weight_decay: 0.0000483558

epoch time: 1676.7338867188s, input_fired_time: 27.3224105835, output_fired_time: 14.7963066101, propagate_forward_time: 611.9559936523, propagate_backward_time: 978.6770019531, weight_update_time: 1.0125710964, other_time: 43.9822387695
epoch time: 134.4122314453s, input_fired_time: 2.7580852509, output_fired_time: 0.0000000000, propagate_forward_time: 123.7287368774, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.9254074097
validation[          , genome  5605] predictions:    6100/  10000 (61.00%), best:    6099/10000 (60.99%), error:     11260.12598, best error:     11259.61328 on epoch:    25, epoch:   29/50, mu: 0.9165250659, learning_rate: 0.0004701107, weight_decay: 0.0000449728

epoch time: 1635.9166259766s, input_fired_time: 23.9250011444, output_fired_time: 12.2080869675, propagate_forward_time: 604.8018188477, propagate_backward_time: 959.9778442383, weight_update_time: 0.7846481800, other_time: 35.0037841797
epoch time: 136.6209564209s, input_fired_time: 2.2078118324, output_fired_time: 0.0000000000, propagate_forward_time: 127.0322036743, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.3809432983
validation[          , genome  5605] predictions:    6070/  10000 (60.70%), best:    6099/10000 (60.99%), error:     11298.35156, best error:     11259.61328 on epoch:    25, epoch:   30/50, mu: 0.9212327003, learning_rate: 0.0004315338, weight_decay: 0.0000418264

epoch time: 1573.8946533203s, input_fired_time: 22.8527565002, output_fired_time: 11.1351146698, propagate_forward_time: 579.5876464844, propagate_backward_time: 926.8957519531, weight_update_time: 0.7480989695, other_time: 33.4233398438
epoch time: 125.3910827637s, input_fired_time: 2.1092405319, output_fired_time: 0.0000000000, propagate_forward_time: 117.1818161011, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1000289917
validation[          , genome  5605] predictions:    6087/  10000 (60.87%), best:    6099/10000 (60.99%), error:     11287.83301, best error:     11259.61328 on epoch:    25, epoch:   31/50, mu: 0.9256386757, learning_rate: 0.0003961224, weight_decay: 0.0000389002

epoch time: 1659.3389892578s, input_fired_time: 23.3054962158, output_fired_time: 11.5239448547, propagate_forward_time: 614.2416381836, propagate_backward_time: 974.9846191406, weight_update_time: 0.7733728886, other_time: 35.2832641602
epoch time: 130.8329620361s, input_fired_time: 2.1224884987, output_fired_time: 0.0000000000, propagate_forward_time: 122.1145401001, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.5959396362
validation[          , genome  5605] predictions:    6160/  10000 (61.60%), best:    6160/10000 (61.60%), error:     11107.30566, best error:     11107.30566 on epoch:    32, epoch:   32/50, mu: 0.9297623634, learning_rate: 0.0003636169, weight_decay: 0.0000361787

epoch time: 1523.5889892578s, input_fired_time: 19.9409046173, output_fired_time: 8.9765405655, propagate_forward_time: 571.7849731445, propagate_backward_time: 898.1015625000, weight_update_time: 0.5588130951, other_time: 24.7849731445
epoch time: 136.9945678711s, input_fired_time: 1.4866870642, output_fired_time: 0.0000000000, propagate_forward_time: 131.3983306885, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1095428467
validation[          , genome  5605] predictions:    6120/  10000 (61.20%), best:    6160/10000 (61.60%), error:     11179.01660, best error:     11107.30566 on epoch:    32, epoch:   33/50, mu: 0.9336218238, learning_rate: 0.0003337788, weight_decay: 0.0000336476

epoch time: 1459.6118164063s, input_fired_time: 18.2455272675, output_fired_time: 7.8537449837, propagate_forward_time: 547.1287231445, propagate_backward_time: 867.0751342773, weight_update_time: 0.4367672503, other_time: 19.3087158203
epoch time: 84.7090148926s, input_fired_time: 1.0696808100, output_fired_time: 0.0000000000, propagate_forward_time: 81.0681762695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5711593628
validation[          , genome  5605] predictions:    6167/  10000 (61.67%), best:    6167/10000 (61.67%), error:     11090.78613, best error:     11090.78613 on epoch:    34, epoch:   34/50, mu: 0.9372340441, learning_rate: 0.0003063892, weight_decay: 0.0000312936

epoch time: 1411.6813964844s, input_fired_time: 17.9642772675, output_fired_time: 7.4957566261, propagate_forward_time: 527.8023071289, propagate_backward_time: 840.4573974609, weight_update_time: 0.4146120548, other_time: 17.9617309570
epoch time: 108.2700576782s, input_fired_time: 1.2532007694, output_fired_time: 0.0000000000, propagate_forward_time: 103.9064636230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1103897095
validation[          , genome  5605] predictions:    6127/  10000 (61.27%), best:    6167/10000 (61.67%), error:     11233.70801, best error:     11090.78613 on epoch:    34, epoch:   35/50, mu: 0.9406148195, learning_rate: 0.0002812472, weight_decay: 0.0000291042

epoch time: 1190.5623779297s, input_fired_time: 16.0169487000, output_fired_time: 6.2152218819, propagate_forward_time: 441.1092529297, propagate_backward_time: 713.8626098633, weight_update_time: 0.3098180294, other_time: 13.3583374023
epoch time: 70.7668685913s, input_fired_time: 0.8608562350, output_fired_time: 0.0000000000, propagate_forward_time: 68.2894821167, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6165313721
validation[          , genome  5605] predictions:    6147/  10000 (61.47%), best:    6167/10000 (61.67%), error:     11132.34961, best error:     11090.78613 on epoch:    34, epoch:   36/50, mu: 0.9437789917, learning_rate: 0.0002581682, weight_decay: 0.0000270681

epoch time: 1083.0017089844s, input_fired_time: 15.1243705750, output_fired_time: 5.4826340675, propagate_forward_time: 401.6445312500, propagate_backward_time: 650.3848876953, weight_update_time: 0.2596182823, other_time: 10.3652343750
epoch time: 74.5830917358s, input_fired_time: 0.8869522214, output_fired_time: 0.0000000000, propagate_forward_time: 72.0477294922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6484069824
validation[          , genome  5605] predictions:    6104/  10000 (61.04%), best:    6167/10000 (61.67%), error:     11233.72559, best error:     11090.78613 on epoch:    34, epoch:   37/50, mu: 0.9467404485, learning_rate: 0.0002369831, weight_decay: 0.0000251744

epoch time: 1003.4673461914s, input_fired_time: 14.8487396240, output_fired_time: 5.1566557884, propagate_forward_time: 368.2990417480, propagate_backward_time: 605.1081542969, weight_update_time: 0.2452868968, other_time: 10.0546875000
epoch time: 75.2346267700s, input_fired_time: 0.8934716582, output_fired_time: 0.0000000000, propagate_forward_time: 72.6874160767, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6537399292
validation[          , genome  5605] predictions:    6277/  10000 (62.77%), best:    6277/10000 (62.77%), error:     10880.24023, best error:     10880.24023 on epoch:    38, epoch:   38/50, mu: 0.9495121241, learning_rate: 0.0002175365, weight_decay: 0.0000234131

epoch time: 1031.3970947266s, input_fired_time: 14.9463205338, output_fired_time: 5.2730445862, propagate_forward_time: 379.4687805176, propagate_backward_time: 621.5270996094, weight_update_time: 0.2503964305, other_time: 10.1817626953
epoch time: 70.6813278198s, input_fired_time: 0.8661510348, output_fired_time: 0.0000000000, propagate_forward_time: 68.2031097412, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6120681763
validation[          , genome  5605] predictions:    6257/  10000 (62.57%), best:    6257/10000 (62.57%), error:     10816.17578, best error:     10816.17578 on epoch:    39, epoch:   39/50, mu: 0.9521062374, learning_rate: 0.0001996856, weight_decay: 0.0000217751

epoch time: 988.7570800781s, input_fired_time: 14.8457174301, output_fired_time: 5.1290268898, propagate_forward_time: 362.3379821777, propagate_backward_time: 596.3907470703, weight_update_time: 0.2437808514, other_time: 10.0535888672
epoch time: 80.4482650757s, input_fired_time: 0.9372475743, output_fired_time: 0.0000000000, propagate_forward_time: 77.7773971558, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7336196899
validation[          , genome  5605] predictions:    6268/  10000 (62.68%), best:    6268/10000 (62.68%), error:     10789.51660, best error:     10789.51660 on epoch:    40, epoch:   40/50, mu: 0.9545341134, learning_rate: 0.0001832996, weight_decay: 0.0000202517

epoch time: 1007.7156372070s, input_fired_time: 14.8921613693, output_fired_time: 5.2034292221, propagate_forward_time: 369.0739440918, propagate_backward_time: 608.4143066406, weight_update_time: 0.2461504191, other_time: 10.1318359375
epoch time: 81.1444168091s, input_fired_time: 0.9344822764, output_fired_time: 0.0000000000, propagate_forward_time: 78.5187911987, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6911468506
validation[          , genome  5605] predictions:    6219/  10000 (62.19%), best:    6268/10000 (62.68%), error:     10873.00293, best error:     10789.51660 on epoch:    40, epoch:   41/50, mu: 0.9568064213, learning_rate: 0.0001682582, weight_decay: 0.0000188349

epoch time: 1059.4288330078s, input_fired_time: 15.0855264664, output_fired_time: 5.4020948410, propagate_forward_time: 391.8455505371, propagate_backward_time: 636.7650146484, weight_update_time: 0.2553629577, other_time: 10.3305664063
epoch time: 76.4888687134s, input_fired_time: 0.9118570685, output_fired_time: 0.0000000000, propagate_forward_time: 73.9034576416, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6735534668
validation[          , genome  5605] predictions:    6240/  10000 (62.40%), best:    6268/10000 (62.68%), error:     10874.02246, best error:     10789.51660 on epoch:    40, epoch:   42/50, mu: 0.9589331746, learning_rate: 0.0001544510, weight_decay: 0.0000175172

epoch time: 1034.0729980469s, input_fired_time: 15.0119323730, output_fired_time: 5.3037366867, propagate_forward_time: 381.1966552734, propagate_backward_time: 622.3037719727, weight_update_time: 0.2523175776, other_time: 10.2568969727
epoch time: 70.5862731934s, input_fired_time: 0.8675875664, output_fired_time: 0.0000000000, propagate_forward_time: 68.1031341553, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6155548096
validation[          , genome  5605] predictions:    6229/  10000 (62.29%), best:    6268/10000 (62.68%), error:     10892.34375, best error:     10789.51660 on epoch:    40, epoch:   43/50, mu: 0.9609236717, learning_rate: 0.0001417769, weight_decay: 0.0000162917

epoch time: 938.4430541992s, input_fired_time: 14.6469736099, output_fired_time: 4.9034042358, propagate_forward_time: 341.2012634277, propagate_backward_time: 567.8827514648, weight_update_time: 0.2347761989, other_time: 9.8086547852
epoch time: 71.0947189331s, input_fired_time: 0.8710902333, output_fired_time: 0.0000000000, propagate_forward_time: 68.6055603027, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6180648804
validation[          , genome  5605] predictions:    6255/  10000 (62.55%), best:    6268/10000 (62.68%), error:     10807.62500, best error:     10789.51660 on epoch:    40, epoch:   44/50, mu: 0.9627866149, learning_rate: 0.0001301428, weight_decay: 0.0000151519

epoch time: 977.1654663086s, input_fired_time: 14.7859897614, output_fired_time: 5.0403385162, propagate_forward_time: 358.3076171875, propagate_backward_time: 589.0720214844, weight_update_time: 0.2412746996, other_time: 9.9594726563
epoch time: 95.2203445435s, input_fired_time: 1.0365369320, output_fired_time: 0.0000000000, propagate_forward_time: 92.3923416138, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7914657593
validation[          , genome  5605] predictions:    6222/  10000 (62.22%), best:    6268/10000 (62.68%), error:     10891.68945, best error:     10789.51660 on epoch:    40, epoch:   45/50, mu: 0.9645302296, learning_rate: 0.0001194634, weight_decay: 0.0000140918

epoch time: 1072.1164550781s, input_fired_time: 15.1141357422, output_fired_time: 5.4227485657, propagate_forward_time: 397.8096923828, propagate_backward_time: 643.4940185547, weight_update_time: 0.2604704201, other_time: 10.2758789063
epoch time: 84.8552169800s, input_fired_time: 0.9638208747, output_fired_time: 0.0000000000, propagate_forward_time: 82.1795349121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7118606567
validation[          , genome  5605] predictions:    6238/  10000 (62.38%), best:    6268/10000 (62.68%), error:     10803.82324, best error:     10789.51660 on epoch:    40, epoch:   46/50, mu: 0.9661620855, learning_rate: 0.0001096603, weight_decay: 0.0000131059

epoch time: 1189.8554687500s, input_fired_time: 15.5191669464, output_fired_time: 5.8951864243, propagate_forward_time: 447.0886535645, propagate_backward_time: 710.6417846680, weight_update_time: 0.2757534385, other_time: 10.7107543945
epoch time: 83.2561492920s, input_fired_time: 0.9563374519, output_fired_time: 0.0000000000, propagate_forward_time: 80.5973892212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7024230957
validation[          , genome  5605] predictions:    6286/  10000 (62.86%), best:    6286/10000 (62.86%), error:     10734.12402, best error:     10734.12402 on epoch:    47, epoch:   47/50, mu: 0.9676893950, learning_rate: 0.0001006617, weight_decay: 0.0000121890

epoch time: 1132.6221923828s, input_fired_time: 15.3368330002, output_fired_time: 5.6749529839, propagate_forward_time: 422.2941284180, propagate_backward_time: 678.8049926758, weight_update_time: 0.2668572068, other_time: 10.5113525391
epoch time: 123.0050201416s, input_fired_time: 1.2533880472, output_fired_time: 0.0000000000, propagate_forward_time: 119.6446456909, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1069869995
validation[          , genome  5605] predictions:    6281/  10000 (62.81%), best:    6286/10000 (62.86%), error:     10766.98438, best error:     10734.12402 on epoch:    47, epoch:   48/50, mu: 0.9691188335, learning_rate: 0.0000924015, weight_decay: 0.0000113363

epoch time: 1025.6959228516s, input_fired_time: 14.9675588608, output_fired_time: 5.2376670837, propagate_forward_time: 377.2222900391, propagate_backward_time: 618.1241455078, weight_update_time: 0.2499040812, other_time: 10.1442871094
epoch time: 88.6222229004s, input_fired_time: 0.9895998240, output_fired_time: 0.0000000000, propagate_forward_time: 85.8882217407, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7444000244
validation[          , genome  5605] predictions:    6302/  10000 (63.02%), best:    6286/10000 (62.86%), error:     10772.02441, best error:     10734.12402 on epoch:    47, epoch:   49/50, mu: 0.9704567194, learning_rate: 0.0000848191, weight_decay: 0.0000105432

epoch time: 954.1723632813s, input_fired_time: 14.7275953293, output_fired_time: 4.9559297562, propagate_forward_time: 347.8855895996, propagate_backward_time: 576.7586059570, weight_update_time: 0.2362897694, other_time: 9.8446655273
epoch time: 85.8293457031s, input_fired_time: 0.9862919450, output_fired_time: 0.0000000000, propagate_forward_time: 83.0659484863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7771072388
validation[          , genome  5605] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10693.95410, best error:     10693.95410 on epoch:    50, epoch:   50/50, mu: 0.9717088938, learning_rate: 0.0000778589, weight_decay: 0.0000098056

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 453.7507019043s, input_fired_time: 5.1149034500, output_fired_time: 0.0000000000, propagate_forward_time: 439.4967651367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.1390380859
best training[          , genome  5605] predictions:   36478/  50000 (72.96%), best:    6336/10000 (63.36%), error:     41169.80469, best error:     10693.95410 on epoch:    50, epoch:   51/50, mu: 0.9728808403, learning_rate: 0.0000714698, weight_decay: 0.0000091196
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 92.4596252441s, input_fired_time: 1.0204846859, output_fired_time: 0.0000000000, propagate_forward_time: 89.6669082642, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7722320557
   testing[          , genome  5605] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10693.95410, best error:     10693.95410 on epoch:    50, epoch:   51/50, mu: 0.9728808403, learning_rate: 0.0000714698, weight_decay: 0.0000091196
backpropagation finished successfully!
05:21:01 (17604): called boinc_finish(0)

</stderr_txt>
]]>