Nameexact_genome_1542303603_178_6008_0
Workunit4091501
Created15 Nov 2018, 20:59:26 UTC
Sent15 Nov 2018, 21:25:15 UTC
Report deadline14 Jan 2019, 22:27:12 UTC
Received20 Nov 2018, 16:26:47 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID85431
Run time22 hours 42 min 32 sec
CPU time22 hours 25 min 37 sec
Validate stateValid
Credit15,010.76
Device peak FLOPS3.94 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size541.95 MB
Peak swap size544.75 MB
Peak disk usage9.24 MB

Stderr output

<core_client_version>7.12.0</core_client_version>
<![CDATA[
<stderr_txt>
et with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 3208.2587890625s, input_fired_time: 28.7183418273926, output_fired_time: 16.4116249084473, propagate_forward_time: 1342.76733398438, propagate_backward_time: 1783.29455566406, weight_update_time: 0.464203208684921, other_time: 37.0670166015625
epoch time: 273.839599609375s, input_fired_time: 1.86636137962341, output_fired_time: 0, propagate_forward_time: 268.579345703125, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.39389038085938
validation[          , genome  6008] predictions:    5984/  10000 (59.84%), best:    6068/10000 (60.68%), error:     11573.25195, best error:     11428.09082 on epoch:     0, epoch:   12/50, mu: 0.7290121913, learning_rate: 0.0002323070, weight_decay: 0.0002308718

epoch time: 3160.8737792969s, input_fired_time: 32.5750007629, output_fired_time: 9.3430538177, propagate_forward_time: 1313.0678710938, propagate_backward_time: 1775.0168457031, weight_update_time: 0.4719611406, other_time: 30.8710937500
arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.742663
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 0.000212501
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 0.000213524
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 13
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6068
read best_validation_error: 11428.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '731622789'
read generator: 731622789
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2895.74243164062s, input_fired_time: 25.7881774902344, output_fired_time: 7.70563745498657, propagate_forward_time: 1220.74499511719, propagate_backward_time: 1620.97106933594, weight_update_time: 0.408964902162552, other_time: 20.53271484375
epoch time: 242.100250244141s, input_fired_time: 0.732370018959045, output_fired_time: 0, propagate_forward_time: 237.359085083008, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.0087890625
validation[          , genome  6008] predictions:    5928/  10000 (59.28%), best:    6068/10000 (60.68%), error:     12008.81738, best error:     11428.09082 on epoch:     0, epoch:   13/50, mu: 0.7426630855, learning_rate: 0.0002125009, weight_decay: 0.0002135244

epoch time: 2916.9965820312s, input_fired_time: 29.9142837524, output_fired_time: 8.8672895432, propagate_forward_time: 1203.8099365234, propagate_backward_time: 1650.6151123047, weight_update_time: 0.4160398841, other_time: 23.7900390625
epoch time: 257.6665344238s, input_fired_time: 0.8115263581, output_fired_time: 0.0000000000, propagate_forward_time: 251.6186828613, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2363281250
validation[          , genome  6008] predictions:    6011/  10000 (60.11%), best:    6068/10000 (60.68%), error:     11680.79297, best error:     11428.09082 on epoch:     0, epoch:   14/50, mu: 0.7555999756, learning_rate: 0.0001943834, weight_decay: 0.0001974805

epoch time: 3030.2407226562s, input_fired_time: 32.2605171204, output_fired_time: 11.0941896439, propagate_forward_time: 1272.5183105469, propagate_backward_time: 1689.0198974609, weight_update_time: 0.4487833083, other_time: 25.3477783203
epoch time: 247.5963745117s, input_fired_time: 1.7628397942, output_fired_time: 0.0000000000, propagate_forward_time: 241.7236785889, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1098632812
validation[          , genome  6008] predictions:    5987/  10000 (59.87%), best:    6068/10000 (60.68%), error:     11643.33691, best error:     11428.09082 on epoch:     0, epoch:   15/50, mu: 0.7678601742, learning_rate: 0.0001778106, weight_decay: 0.0001826421

epoch time: 2846.5671386719s, input_fired_time: 29.7097015381, output_fired_time: 8.6973562241, propagate_forward_time: 1196.4020996094, propagate_backward_time: 1574.2770996094, weight_update_time: 0.4009160399, other_time: 37.4809570312
epoch time: 242.1986083984s, input_fired_time: 2.7380664349, output_fired_time: 0.0000000000, propagate_forward_time: 231.4412841797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.0192565918
validation[          , genome  6008] predictions:    5924/  10000 (59.24%), best:    6068/10000 (60.68%), error:     11666.21973, best error:     11428.09082 on epoch:     0, epoch:   16/50, mu: 0.7794790864, learning_rate: 0.0001626508, weight_decay: 0.0001689186

epoch time: 2820.5502929688s, input_fired_time: 26.5775108337, output_fired_time: 7.5591616631, propagate_forward_time: 1199.3562011719, propagate_backward_time: 1559.9002685547, weight_update_time: 1.3958133459, other_time: 27.1573486328
epoch time: 244.2017822266s, input_fired_time: 1.7210229635, output_fired_time: 0.0000000000, propagate_forward_time: 240.5256195068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9551391602
validation[          , genome  6008] predictions:    5823/  10000 (58.23%), best:    6068/10000 (60.68%), error:     12287.04102, best error:     11428.09082 on epoch:     0, epoch:   17/50, mu: 0.7904903293, learning_rate: 0.0001487835, weight_decay: 0.0001562263

epoch time: 2820.0141601562s, input_fired_time: 34.6463203430, output_fired_time: 8.5712728500, propagate_forward_time: 1173.7186279297, propagate_backward_time: 1576.8914794922, weight_update_time: 1.4002079964, other_time: 26.1865234375
epoch time: 264.3761291504s, input_fired_time: 3.8234641552, output_fired_time: 0.0000000000, propagate_forward_time: 255.3117980957, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2408752441
validation[          , genome  6008] predictions:    6036/  10000 (60.36%), best:    6068/10000 (60.68%), error:     11516.88281, best error:     11428.09082 on epoch:     0, epoch:   18/50, mu: 0.8009256124, learning_rate: 0.0001360984, weight_decay: 0.0001444877

epoch time: 3021.9035644531s, input_fired_time: 22.2681159973, output_fired_time: 9.0731296539, propagate_forward_time: 1272.0231933594, propagate_backward_time: 1686.2055664062, weight_update_time: 1.4434664249, other_time: 32.3334960938
epoch time: 261.9356994629s, input_fired_time: 0.8260657191, output_fired_time: 0.0000000000, propagate_forward_time: 253.8489074707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2607116699
validation[          , genome  6008] predictions:    6201/  10000 (62.01%), best:    6201/10000 (62.01%), error:     11043.92871, best error:     11043.92871 on epoch:    19, epoch:   19/50, mu: 0.8108150959, learning_rate: 0.0001244949, weight_decay: 0.0001336311

epoch time: 5046.5952148438s, input_fired_time: 23.8408718109, output_fired_time: 9.7948398590, propagate_forward_time: 1234.4023437500, propagate_backward_time: 3751.0981445312, weight_update_time: 0.4141730368, other_time: 27.4589843750
epoch time: 293.8680419922s, input_fired_time: 2.9038174152, output_fired_time: 0.0000000000, propagate_forward_time: 284.5905151367, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.3737182617
validation[          , genome  6008] predictions:    6199/  10000 (61.99%), best:    6201/10000 (62.01%), error:     11062.79199, best error:     11043.92871 on epoch:    19, epoch:   20/50, mu: 0.8201872706, learning_rate: 0.0001138807, weight_decay: 0.0001235902

epoch time: 2934.5021972656s, input_fired_time: 22.9638900757, output_fired_time: 10.7976303101, propagate_forward_time: 1222.6024169922, propagate_backward_time: 1656.5029296875, weight_update_time: 0.4190847874, other_time: 21.6353759766
epoch time: 244.1463623047s, input_fired_time: 1.7343996763, output_fired_time: 0.0000000000, propagate_forward_time: 237.4472808838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9646759033
validation[          , genome  6008] predictions:    6157/  10000 (61.57%), best:    6201/10000 (62.01%), error:     11203.08105, best error:     11043.92871 on epoch:    19, epoch:   21/50, mu: 0.8290692568, learning_rate: 0.0001041714, weight_decay: 0.0001143039

epoch time: 2894.7045898438s, input_fired_time: 33.8835678101, output_fired_time: 10.7450428009, propagate_forward_time: 1201.7014160156, propagate_backward_time: 1628.8603515625, weight_update_time: 1.4138399363, other_time: 19.5141601562
epoch time: 249.6303863525s, input_fired_time: 0.7757943869, output_fired_time: 0.0000000000, propagate_forward_time: 242.7364044189, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1181945801
validation[          , genome  6008] predictions:    6076/  10000 (60.76%), best:    6201/10000 (62.01%), error:     11371.76270, best error:     11043.92871 on epoch:    19, epoch:   22/50, mu: 0.8374866843, learning_rate: 0.0000952899, weight_decay: 0.0001057152

epoch time: 2877.1035156250s, input_fired_time: 31.7186756134, output_fired_time: 16.6365489960, propagate_forward_time: 1208.2490234375, propagate_backward_time: 1594.3547363281, weight_update_time: 0.4034942389, other_time: 26.1445312500
epoch time: 242.1993560791s, input_fired_time: 0.7441628575, output_fired_time: 0.0000000000, propagate_forward_time: 237.4614410400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9937591553
validation[          , genome  6008] predictions:    6212/  10000 (62.12%), best:    6201/10000 (62.01%), error:     11068.77930, best error:     11043.92871 on epoch:    19, epoch:   23/50, mu: 0.8454638124, learning_rate: 0.0000871656, weight_decay: 0.0000977719

epoch time: 3131.6877441406s, input_fired_time: 30.5403366089, output_fired_time: 14.2146100998, propagate_forward_time: 1291.8251953125, propagate_backward_time: 1765.4975585938, weight_update_time: 0.4561260343, other_time: 29.6101074219
epoch time: 274.2655334473s, input_fired_time: 0.8621010184, output_fired_time: 0.0000000000, propagate_forward_time: 269.1280212402, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2754211426
validation[          , genome  6008] predictions:    6032/  10000 (60.32%), best:    6201/10000 (62.01%), error:     11403.09961, best error:     11043.92871 on epoch:    19, epoch:   24/50, mu: 0.8530237079, learning_rate: 0.0000797340, weight_decay: 0.0000904255

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.860188
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 7.2936e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 8.36311e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 25
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 19
read number_validation_images: 10000
read best_validation_predictions: 6201
read best_validation_error: 11043.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '115755885'
read generator: 115755885
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2872.04370117188s, input_fired_time: 35.7121810913086, output_fired_time: 10.6565275192261, propagate_forward_time: 1196.33386230469, propagate_backward_time: 1604.02185058594, weight_update_time: 1.4113005399704, other_time: 25.3193359375
epoch time: 257.970397949219s, input_fired_time: 0.785713076591492, output_fired_time: 0, propagate_forward_time: 252.05712890625, propagate_backward_time: 0, weight_update_time: 0, other_time: 5.1275634765625
validation[          , genome  6008] predictions:    6220/  10000 (62.20%), best:    6201/10000 (62.01%), error:     11050.52344, best error:     11043.92871 on epoch:    19, epoch:   25/50, mu: 0.8601881862, learning_rate: 0.0000729360, weight_decay: 0.0000836311

epoch time: 2875.2617187500s, input_fired_time: 33.7563705444, output_fired_time: 11.6823797226, propagate_forward_time: 1196.5548095703, propagate_backward_time: 1605.8703613281, weight_update_time: 0.4170592427, other_time: 27.3978271484
epoch time: 242.5646667480s, input_fired_time: 1.7310976982, output_fired_time: 0.0000000000, propagate_forward_time: 237.9010925293, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9324798584
validation[          , genome  6008] predictions:    5941/  10000 (59.41%), best:    6201/10000 (62.01%), error:     11634.22461, best error:     11043.92871 on epoch:    19, epoch:   26/50, mu: 0.8669779301, learning_rate: 0.0000667176, weight_decay: 0.0000773471

epoch time: 2830.9794921875s, input_fired_time: 32.6743011475, output_fired_time: 5.6012144089, propagate_forward_time: 1180.8891601562, propagate_backward_time: 1571.5010986328, weight_update_time: 1.4094879627, other_time: 40.3135986328
epoch time: 242.0426788330s, input_fired_time: 1.7355576754, output_fired_time: 0.0000000000, propagate_forward_time: 238.3440704346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9630432129
validation[          , genome  6008] predictions:    6138/  10000 (61.38%), best:    6201/10000 (62.01%), error:     11308.43164, best error:     11043.92871 on epoch:    19, epoch:   27/50, mu: 0.8734125495, learning_rate: 0.0000610294, weight_decay: 0.0000715354

epoch time: 2816.3837890625s, input_fired_time: 27.5974998474, output_fired_time: 11.5506868362, propagate_forward_time: 1193.5950927734, propagate_backward_time: 1564.4868164062, weight_update_time: 1.3996897936, other_time: 19.1536865234
epoch time: 240.8680877686s, input_fired_time: 0.7389408350, output_fired_time: 0.0000000000, propagate_forward_time: 236.1712188721, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9579315186
validation[          , genome  6008] predictions:    6110/  10000 (61.10%), best:    6201/10000 (62.01%), error:     11256.56250, best error:     11043.92871 on epoch:    19, epoch:   28/50, mu: 0.8795106411, learning_rate: 0.0000558261, weight_decay: 0.0000661603

epoch time: 2818.7834472656s, input_fired_time: 32.6141128540, output_fired_time: 7.5445208549, propagate_forward_time: 1189.0162353516, propagate_backward_time: 1565.5455322266, weight_update_time: 0.3943169117, other_time: 24.0632324219
epoch time: 244.1500854492s, input_fired_time: 0.7398100495, output_fired_time: 0.0000000000, propagate_forward_time: 239.4367370605, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9735412598
validation[          , genome  6008] predictions:    6151/  10000 (61.51%), best:    6201/10000 (62.01%), error:     11074.72852, best error:     11043.92871 on epoch:    19, epoch:   29/50, mu: 0.8852897286, learning_rate: 0.0000510665, weight_decay: 0.0000611891

epoch time: 2836.8090820312s, input_fired_time: 31.7720298767, output_fired_time: 10.5821657181, propagate_forward_time: 1249.7448730469, propagate_backward_time: 1527.5349121094, weight_update_time: 1.3952479362, other_time: 17.1750488281
epoch time: 244.1428222656s, input_fired_time: 0.7585307360, output_fired_time: 0.0000000000, propagate_forward_time: 241.3743591309, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0099334717
validation[          , genome  6008] predictions:    6287/  10000 (62.87%), best:    6287/10000 (62.87%), error:     10801.28125, best error:     10801.28125 on epoch:    30, epoch:   30/50, mu: 0.8907665610, learning_rate: 0.0000467126, weight_decay: 0.0000565915

epoch time: 7048.2031250000s, input_fired_time: 36.6944198608, output_fired_time: 6.6059985161, propagate_forward_time: 1188.4489746094, propagate_backward_time: 5795.3041992188, weight_update_time: 2.3972165585, other_time: 21.1499023438
epoch time: 242.1564636230s, input_fired_time: 0.7470130324, output_fired_time: 0.0000000000, propagate_forward_time: 238.4427032471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9667510986
validation[          , genome  6008] predictions:    6286/  10000 (62.86%), best:    6286/10000 (62.86%), error:     10785.77441, best error:     10785.77441 on epoch:    31, epoch:   31/50, mu: 0.8959569335, learning_rate: 0.0000427300, weight_decay: 0.0000523393

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.900876
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 3.90869e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 4.84066e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 32
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 31
read number_validation_images: 10000
read best_validation_predictions: 6286
read best_validation_error: 10785.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '135153252'
read generator: 135153252
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2843.30297851562s, input_fired_time: 25.7656955718994, output_fired_time: 7.67446041107178, propagate_forward_time: 1194.75634765625, propagate_backward_time: 1595.771484375, weight_update_time: 0.400787591934204, other_time: 19.3349609375
epoch time: 241.133102416992s, input_fired_time: 4.74833917617798, output_fired_time: 0, propagate_forward_time: 233.378662109375, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.006103515625
validation[          , genome  6008] predictions:    6274/  10000 (62.74%), best:    6286/10000 (62.86%), error:     10823.16602, best error:     10785.77441 on epoch:    31, epoch:   32/50, mu: 0.9008758068, learning_rate: 0.0000390869, weight_decay: 0.0000484066

epoch time: 2870.7680664062s, input_fired_time: 30.9111633301, output_fired_time: 5.7237577438, propagate_forward_time: 1207.7181396484, propagate_backward_time: 1602.8286132812, weight_update_time: 0.4083772600, other_time: 23.5863037109
epoch time: 246.2190856934s, input_fired_time: 0.7787061930, output_fired_time: 0.0000000000, propagate_forward_time: 241.4038848877, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0364990234
validation[          , genome  6008] predictions:    6319/  10000 (63.19%), best:    6319/10000 (63.19%), error:     10698.32812, best error:     10698.32812 on epoch:    33, epoch:   33/50, mu: 0.9055374265, learning_rate: 0.0000357544, weight_decay: 0.0000447694

epoch time: 2864.2585449219s, input_fired_time: 24.9487781525, output_fired_time: 11.7911624908, propagate_forward_time: 1172.3200683594, propagate_backward_time: 1625.5272216797, weight_update_time: 0.4050526619, other_time: 29.6712646484
epoch time: 249.4259033203s, input_fired_time: 2.8401713371, output_fired_time: 0.0000000000, propagate_forward_time: 242.4226074219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1631317139
validation[          , genome  6008] predictions:    6386/  10000 (63.86%), best:    6386/10000 (63.86%), error:     10613.79688, best error:     10613.79688 on epoch:    34, epoch:   34/50, mu: 0.9099552035, learning_rate: 0.0000327061, weight_decay: 0.0000414055

epoch time: 2890.1999511719s, input_fired_time: 38.0275192261, output_fired_time: 7.8809390068, propagate_forward_time: 1206.9454345703, propagate_backward_time: 1615.4202880859, weight_update_time: 0.4135257304, other_time: 21.9257812500
epoch time: 281.7992858887s, input_fired_time: 1.0172460079, output_fired_time: 0.0000000000, propagate_forward_time: 276.1740417480, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6080017090
validation[          , genome  6008] predictions:    6329/  10000 (63.29%), best:    6386/10000 (63.86%), error:     10668.82227, best error:     10613.79688 on epoch:    34, epoch:   35/50, mu: 0.9141418934, learning_rate: 0.0000299176, weight_decay: 0.0000382943

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.91811
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 2.73669e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 3.54169e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 36
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 34
read number_validation_images: 10000
read best_validation_predictions: 6386
read best_validation_error: 10613.8
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '1030335799'
read generator: 1030335799
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2900.21069335938s, input_fired_time: 33.9445953369141, output_fired_time: 9.73415851593018, propagate_forward_time: 1206.888671875, propagate_backward_time: 1622.29760742188, weight_update_time: 1.41127622127533, other_time: 27.345703125
epoch time: 275.718505859375s, input_fired_time: 0.871884167194366, output_fired_time: 0, propagate_forward_time: 271.620880126953, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.22573852539062
validation[          , genome  6008] predictions:    6386/  10000 (63.86%), best:    6386/10000 (63.86%), error:     10513.78711, best error:     10513.78711 on epoch:    36, epoch:   36/50, mu: 0.9181095958, learning_rate: 0.0000273669, weight_decay: 0.0000354169

epoch time: 2994.6147460938s, input_fired_time: 27.3506889343, output_fired_time: 15.0049839020, propagate_forward_time: 1269.8144531250, propagate_backward_time: 1657.3647460938, weight_update_time: 0.4326268733, other_time: 25.0800781250
epoch time: 261.8585205078s, input_fired_time: 0.8556320667, output_fired_time: 0.0000000000, propagate_forward_time: 253.7971343994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2057647705
validation[          , genome  6008] predictions:    6287/  10000 (62.87%), best:    6386/10000 (63.86%), error:     10747.20117, best error:     10513.78711 on epoch:    36, epoch:   37/50, mu: 0.9218698144, learning_rate: 0.0000250336, weight_decay: 0.0000327558

epoch time: 3013.3156738281s, input_fired_time: 28.4101028442, output_fired_time: 13.0328750610, propagate_forward_time: 1255.2037353516, propagate_backward_time: 1695.5270996094, weight_update_time: 0.4375346899, other_time: 21.1417236328
epoch time: 254.0296783447s, input_fired_time: 0.8209770322, output_fired_time: 0.0000000000, propagate_forward_time: 247.0736846924, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1350097656
validation[          , genome  6008] predictions:    6347/  10000 (63.47%), best:    6386/10000 (63.86%), error:     10650.37891, best error:     10513.78711 on epoch:    36, epoch:   38/50, mu: 0.9254333377, learning_rate: 0.0000228993, weight_decay: 0.0000302945

epoch time: 3006.2631835938s, input_fired_time: 35.3951568604, output_fired_time: 7.0137219429, propagate_forward_time: 1247.2950439453, propagate_backward_time: 1693.3160400391, weight_update_time: 0.4383758307, other_time: 23.2431640625
epoch time: 262.6817626953s, input_fired_time: 0.8943614960, output_fired_time: 0.0000000000, propagate_forward_time: 258.4455261230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3418884277
validation[          , genome  6008] predictions:    6224/  10000 (62.24%), best:    6386/10000 (63.86%), error:     10865.09277, best error:     10513.78711 on epoch:    36, epoch:   39/50, mu: 0.9288104773, learning_rate: 0.0000209469, weight_decay: 0.0000280182

epoch time: 2868.7463378906s, input_fired_time: 32.0407142639, output_fired_time: 7.7689514160, propagate_forward_time: 1268.9976806641, propagate_backward_time: 1529.2939453125, weight_update_time: 0.4044627249, other_time: 30.6448974609
epoch time: 251.8319549561s, input_fired_time: 0.8149583936, output_fired_time: 0.0000000000, propagate_forward_time: 247.8899993896, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1269989014
validation[          , genome  6008] predictions:    6437/  10000 (64.37%), best:    6437/10000 (64.37%), error:     10488.10547, best error:     10488.10547 on epoch:    40, epoch:   40/50, mu: 0.9320109487, learning_rate: 0.0000191610, weight_decay: 0.0000259130

epoch time: 3028.5490722656s, input_fired_time: 32.7669982910, output_fired_time: 9.3403644562, propagate_forward_time: 1246.6978759766, propagate_backward_time: 1708.5849609375, weight_update_time: 0.4568315744, other_time: 31.1588134766
epoch time: 267.4299316406s, input_fired_time: 0.9375781417, output_fired_time: 0.0000000000, propagate_forward_time: 262.0349731445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4573669434
validation[          , genome  6008] predictions:    6368/  10000 (63.68%), best:    6437/10000 (64.37%), error:     10598.85547, best error:     10488.10547 on epoch:    40, epoch:   41/50, mu: 0.9350440502, learning_rate: 0.0000175274, weight_decay: 0.0000239659

epoch time: 2986.0905761719s, input_fired_time: 30.4533386230, output_fired_time: 12.1379184723, propagate_forward_time: 1246.3630371094, propagate_backward_time: 1675.6859130859, weight_update_time: 0.4384941757, other_time: 21.4503173828
epoch time: 240.1529693604s, input_fired_time: 0.7575784326, output_fired_time: 0.0000000000, propagate_forward_time: 235.4420928955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9532928467
validation[          , genome  6008] predictions:    6439/  10000 (64.39%), best:    6439/10000 (64.39%), error:     10482.28711, best error:     10482.28711 on epoch:    42, epoch:   42/50, mu: 0.9379184842, learning_rate: 0.0000160330, weight_decay: 0.0000221652

epoch time: 2790.8479003906s, input_fired_time: 21.6276416779, output_fired_time: 9.4954557419, propagate_forward_time: 1172.0074462891, propagate_backward_time: 1562.7381591797, weight_update_time: 0.3902631104, other_time: 24.9792480469
epoch time: 241.1779937744s, input_fired_time: 1.7617456913, output_fired_time: 0.0000000000, propagate_forward_time: 237.4723358154, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9439086914
validation[          , genome  6008] predictions:    6344/  10000 (63.44%), best:    6439/10000 (64.39%), error:     10616.31055, best error:     10482.28711 on epoch:    42, epoch:   43/50, mu: 0.9406425953, learning_rate: 0.0000146661, weight_decay: 0.0000204997

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.943224
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 1.34157e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 1.89594e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 44
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 42
read number_validation_images: 10000
read best_validation_predictions: 6439
read best_validation_error: 10482.3
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '21339057'
read generator: 21339057
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 3228.6689453125s, input_fired_time: 30.0206680297852, output_fired_time: 9.48631191253662, propagate_forward_time: 1332.89306640625, propagate_backward_time: 1828.97265625, weight_update_time: 0.476392090320587, other_time: 27.296142578125
epoch time: 273.987060546875s, input_fired_time: 1.9049905538559, output_fired_time: 0, propagate_forward_time: 267.687408447266, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.3946533203125
validation[          , genome  6008] predictions:    6406/  10000 (64.06%), best:    6406/10000 (64.06%), error:     10477.70703, best error:     10477.70703 on epoch:    44, epoch:   44/50, mu: 0.9432241917, learning_rate: 0.0000134157, weight_decay: 0.0000189594

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.945671
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 1.22719e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 1.75348e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 45
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 44
read number_validation_images: 10000
read best_validation_predictions: 6406
read best_validation_error: 10477.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '1110389543'
read generator: 1110389543
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2939.24780273438s, input_fired_time: 32.1100311279297, output_fired_time: 6.87223339080811, propagate_forward_time: 1217.12219238281, propagate_backward_time: 1654.54602050781, weight_update_time: 0.418656647205353, other_time: 28.59716796875
epoch time: 261.338287353516s, input_fired_time: 1.85200166702271, output_fired_time: 0, propagate_forward_time: 255.270797729492, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.21549987792969
validation[          , genome  6008] predictions:    6425/  10000 (64.25%), best:    6425/10000 (64.25%), error:     10427.70312, best error:     10427.70312 on epoch:    45, epoch:   45/50, mu: 0.9456707835, learning_rate: 0.0000122719, weight_decay: 0.0000175348

arguments:
	'../../projects/csgrid.org_csg/exact_client_0.33_x86_64-pc-linux-gnu'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542303603_178_6008_0_r392359743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.492727
read mu: 0.947989
read mu_delta: 0.947695
read initial_learning_rate: 0.000676832
read learning_rate: 1.12256e-05
read learning_rate_delta: 0.914742
read initial_weight_decay: 0.000589455
read weight_decay: 1.62173e-05
read weight_decay_delta: 0.924861
read batch_size: 107
read epsilon: 1e-07
read alpha: 0.0757249
read input_dropout_probability: 0.00113281
read hidden_dropout_probability: 0.111784
read velocity_reset: 2185
read epoch: 46
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 45
read number_validation_images: 10000
read best_validation_predictions: 6425
read best_validation_error: 10427.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6008
read normal distribution: '1 0 0'
generator_str: '117703520'
read generator: 117703520
read generated_by_map:
1 disable_node 1
reading 94 nodes.
reading 490 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507812
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2889.20849609375s, input_fired_time: 27.0029830932617, output_fired_time: 10.7861862182617, propagate_forward_time: 1202.63671875, propagate_backward_time: 1623.25732421875, weight_update_time: 1.41186201572418, other_time: 25.525390625
epoch time: 240.240280151367s, input_fired_time: 2.77298831939697, output_fired_time: 0, propagate_forward_time: 233.460235595703, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.00704956054688
validation[          , genome  6008] predictions:    6438/  10000 (64.38%), best:    6438/10000 (64.38%), error:     10401.49707, best error:     10401.49707 on epoch:    46, epoch:   46/50, mu: 0.9479894042, learning_rate: 0.0000112256, weight_decay: 0.0000162173

epoch time: 2849.7849121094s, input_fired_time: 35.8867225647, output_fired_time: 12.7266445160, propagate_forward_time: 1111.1656494141, propagate_backward_time: 1675.5881347656, weight_update_time: 0.4047242999, other_time: 14.4178466797
epoch time: 245.9842376709s, input_fired_time: 0.7756567597, output_fired_time: 0.0000000000, propagate_forward_time: 242.1962890625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0122985840
validation[          , genome  6008] predictions:    6397/  10000 (63.97%), best:    6438/10000 (64.38%), error:     10513.84961, best error:     10401.49707 on epoch:    46, epoch:   47/50, mu: 0.9501867294, learning_rate: 0.0000102685, weight_decay: 0.0000149987

epoch time: 2830.4340820312s, input_fired_time: 27.8180408478, output_fired_time: 9.5736570358, propagate_forward_time: 1226.5941162109, propagate_backward_time: 1542.3511962891, weight_update_time: 0.3978574276, other_time: 24.0969238281
epoch time: 242.0348052979s, input_fired_time: 0.7690744996, output_fired_time: 0.0000000000, propagate_forward_time: 238.2575988770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0081329346
validation[          , genome  6008] predictions:    6467/  10000 (64.67%), best:    6438/10000 (64.38%), error:     10411.15137, best error:     10401.49707 on epoch:    46, epoch:   48/50, mu: 0.9522691369, learning_rate: 0.0000093930, weight_decay: 0.0000138717

epoch time: 2844.4135742188s, input_fired_time: 33.8966560364, output_fired_time: 13.6339683533, propagate_forward_time: 1248.7028808594, propagate_backward_time: 1510.8912353516, weight_update_time: 1.4050409794, other_time: 37.2886962891
epoch time: 241.6723022461s, input_fired_time: 1.7816107273, output_fired_time: 0.0000000000, propagate_forward_time: 235.8505096436, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0401763916
validation[          , genome  6008] predictions:    6453/  10000 (64.53%), best:    6438/10000 (64.38%), error:     10402.43359, best error:     10401.49707 on epoch:    46, epoch:   49/50, mu: 0.9542426467, learning_rate: 0.0000085922, weight_decay: 0.0000128294

epoch time: 2859.8491210938s, input_fired_time: 31.9247837067, output_fired_time: 8.6785879135, propagate_forward_time: 1169.7905273438, propagate_backward_time: 1626.0161132812, weight_update_time: 0.4044591188, other_time: 23.4392089844
epoch time: 244.6584472656s, input_fired_time: 2.7956023216, output_fired_time: 0.0000000000, propagate_forward_time: 237.7945404053, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0682983398
validation[          , genome  6008] predictions:    6469/  10000 (64.69%), best:    6469/10000 (64.69%), error:     10347.20410, best error:     10347.20410 on epoch:    50, epoch:   50/50, mu: 0.9561129212, learning_rate: 0.0000078597, weight_decay: 0.0000118655

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 1225.4682617188s, input_fired_time: 12.9767608643, output_fired_time: 0.0000000000, propagate_forward_time: 1189.1676025391, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 23.3238525391
best training[          , genome  6008] predictions:   34529/  50000 (69.06%), best:    6469/10000 (64.69%), error:     46602.05859, best error:     10347.20410 on epoch:    50, epoch:   51/50, mu: 0.9578853846, learning_rate: 0.0000071896, weight_decay: 0.0000109739
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 242.3826599121s, input_fired_time: 2.7982554436, output_fired_time: 0.0000000000, propagate_forward_time: 237.5195159912, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0648956299
   testing[          , genome  6008] predictions:    6469/  10000 (64.69%), best:    6469/10000 (64.69%), error:     10347.20410, best error:     10347.20410 on epoch:    50, epoch:   51/50, mu: 0.9578853846, learning_rate: 0.0000071896, weight_decay: 0.0000109739
backpropagation finished successfully!
14:30:56 (2745): called boinc_finish(0)

</stderr_txt>
]]>