Nameexact_genome_1542229504_179_5954_0
Workunit4090153
Created15 Nov 2018, 6:11:52 UTC
Sent15 Nov 2018, 6:11:57 UTC
Report deadline2 Jan 2019, 17:19:59 UTC
Received21 Nov 2018, 5:45:23 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID21895
Run time12 hours 34 min 23 sec
CPU time11 hours 50 min 20 sec
Validate stateValid
Credit12,115.98
Device peak FLOPS4.37 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size552.13 MB
Peak swap size550.51 MB
Peak disk usage12.78 MB

Stderr output

<core_client_version>7.6.22</core_client_version>
<![CDATA[
<stderr_txt>
ht_decay: 0.0003260078

epoch time: 784.1951293945s, input_fired_time: 13.9876794815, output_fired_time: 4.9839968681, propagate_forward_time: 294.6435852051, propagate_backward_time: 461.2659301758, weight_update_time: 0.3561981022, other_time: 9.3139343262
epoch time: 62.7578506470s, input_fired_time: 0.8410649300, output_fired_time: 0.0000000000, propagate_forward_time: 60.4378623962, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4789237976
validation[          , genome  5954] predictions:    6120/  10000 (61.20%), best:    6120/10000 (61.20%), error:     11146.04883, best error:     11146.04883 on epoch:     5, epoch:    5/50, mu: 0.6059018373, learning_rate: 0.0021429581, weight_decay: 0.0003081311

epoch time: 776.9432983398s, input_fired_time: 14.0864019394, output_fired_time: 4.9189286232, propagate_forward_time: 292.6911010742, propagate_backward_time: 455.9971313477, weight_update_time: 0.3715504408, other_time: 9.2496948242
epoch time: 60.2052764893s, input_fired_time: 0.8237524033, output_fired_time: 0.0000000000, propagate_forward_time: 57.9338874817, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4476356506
validation[          , genome  5954] predictions:    6176/  10000 (61.76%), best:    6176/10000 (61.76%), error:     11027.26367, best error:     11027.26367 on epoch:     6, epoch:    6/50, mu: 0.6221431494, learning_rate: 0.0019686429, weight_decay: 0.0002912347

epoch time: 911.3859863281s, input_fired_time: 15.2300271988, output_fired_time: 5.4553623199, propagate_forward_time: 363.3615722656, propagate_backward_time: 517.0091552734, weight_update_time: 0.3962043226, other_time: 10.3298339844
epoch time: 60.3967933655s, input_fired_time: 0.8389800191, output_fired_time: 0.0000000000, propagate_forward_time: 58.1059036255, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4519081116
validation[          , genome  5954] predictions:    6079/  10000 (60.79%), best:    6176/10000 (61.76%), error:     11156.30371, best error:     11027.26367 on epoch:     6, epoch:    7/50, mu: 0.6376976967, learning_rate: 0.0018085071, weight_decay: 0.0002752648

epoch time: 775.4691162109s, input_fired_time: 13.9078063965, output_fired_time: 4.9167242050, propagate_forward_time: 290.6544189453, propagate_backward_time: 456.8786010742, weight_update_time: 0.3549658954, other_time: 9.1115112305
epoch time: 60.1528511047s, input_fired_time: 0.8367364407, output_fired_time: 0.0000000000, propagate_forward_time: 57.8808898926, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4352264404
validation[          , genome  5954] predictions:    6190/  10000 (61.90%), best:    6190/10000 (61.90%), error:     10953.48633, best error:     10953.48633 on epoch:     8, epoch:    8/50, mu: 0.6525945067, learning_rate: 0.0016613973, weight_decay: 0.0002601707

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.666861
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 0.00152625
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 0.000245904
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 9
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 8
read number_validation_images: 10000
read best_validation_predictions: 6190
read best_validation_error: 10953.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '755130209'
read generator: 755130209
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1259.30749511719s, input_fired_time: 13.9916925430298, output_fired_time: 4.98932600021362, propagate_forward_time: 295.210784912109, propagate_backward_time: 863.545288085938, weight_update_time: 0.36094868183136, other_time: 81.5703125
epoch time: 60.085994720459s, input_fired_time: 0.826554894447327, output_fired_time: 0, propagate_forward_time: 57.8244552612305, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.43498611450195
validation[          , genome  5954] predictions:    6041/  10000 (60.41%), best:    6190/10000 (61.90%), error:     11206.77734, best error:     10953.48633 on epoch:     8, epoch:    9/50, mu: 0.6668614745, learning_rate: 0.0015262539, weight_decay: 0.0002459042

epoch time: 774.1616821289s, input_fired_time: 13.9978809357, output_fired_time: 4.9184284210, propagate_forward_time: 291.8552246094, propagate_backward_time: 454.1817932129, weight_update_time: 0.3546481729, other_time: 9.2083435059
epoch time: 59.9464454651s, input_fired_time: 0.8539722562, output_fired_time: 0.0000000000, propagate_forward_time: 57.6618041992, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4306678772
validation[          , genome  5954] predictions:    6214/  10000 (62.14%), best:    6214/10000 (62.14%), error:     10713.83105, best error:     10713.83105 on epoch:    10, epoch:   10/50, mu: 0.6805251241, learning_rate: 0.0014021035, weight_decay: 0.0002324200

epoch time: 768.6838378906s, input_fired_time: 13.8536262512, output_fired_time: 4.8836112022, propagate_forward_time: 289.1239318848, propagate_backward_time: 451.7703552246, weight_update_time: 0.3521384001, other_time: 9.0523071289
epoch time: 60.1481208801s, input_fired_time: 0.8425894380, output_fired_time: 0.0000000000, propagate_forward_time: 57.8731079102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4324226379
validation[          , genome  5954] predictions:    6073/  10000 (60.73%), best:    6214/10000 (62.14%), error:     11155.72461, best error:     10713.83105 on epoch:    10, epoch:   11/50, mu: 0.6936110258, learning_rate: 0.0012880518, weight_decay: 0.0002196752

epoch time: 772.4876098633s, input_fired_time: 13.9049692154, output_fired_time: 4.8985438347, propagate_forward_time: 290.8804626465, propagate_backward_time: 453.6305541992, weight_update_time: 0.3517410755, other_time: 9.1730651855
epoch time: 62.1656875610s, input_fired_time: 0.9447314143, output_fired_time: 0.0000000000, propagate_forward_time: 59.6547889709, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5661659241
validation[          , genome  5954] predictions:    6261/  10000 (62.61%), best:    6261/10000 (62.61%), error:     10686.76953, best error:     10686.76953 on epoch:    12, epoch:   12/50, mu: 0.7061436176, learning_rate: 0.0011832776, weight_decay: 0.0002076293

epoch time: 783.2427978516s, input_fired_time: 14.3556680679, output_fired_time: 5.0294799805, propagate_forward_time: 295.3433532715, propagate_backward_time: 459.2096557617, weight_update_time: 0.3650393486, other_time: 9.3046569824
epoch time: 60.4517250061s, input_fired_time: 0.8560577035, output_fired_time: 0.0000000000, propagate_forward_time: 58.1459846497, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4496841431
validation[          , genome  5954] predictions:    6320/  10000 (63.20%), best:    6320/10000 (63.20%), error:     10638.11523, best error:     10638.11523 on epoch:    13, epoch:   13/50, mu: 0.7181462646, learning_rate: 0.0010870260, weight_decay: 0.0001962439

epoch time: 779.8825073242s, input_fired_time: 14.0230188370, output_fired_time: 4.9735474586, propagate_forward_time: 294.3518371582, propagate_backward_time: 457.3419799805, weight_update_time: 0.3590598702, other_time: 9.1921081543
epoch time: 60.3775672913s, input_fired_time: 0.8545469046, output_fired_time: 0.0000000000, propagate_forward_time: 58.0826530457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4403686523
validation[          , genome  5954] predictions:    6043/  10000 (60.43%), best:    6320/10000 (63.20%), error:     11247.60742, best error:     10638.11523 on epoch:    13, epoch:   14/50, mu: 0.7296413779, learning_rate: 0.0009986038, weight_decay: 0.0001854829

epoch time: 776.1867065430s, input_fired_time: 14.0239343643, output_fired_time: 4.9468655586, propagate_forward_time: 291.5439147949, propagate_backward_time: 456.5133666992, weight_update_time: 0.3574774265, other_time: 9.1586608887
epoch time: 61.3245010376s, input_fired_time: 0.8590909839, output_fired_time: 0.0000000000, propagate_forward_time: 59.0168914795, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4485168457
validation[          , genome  5954] predictions:    6293/  10000 (62.93%), best:    6320/10000 (63.20%), error:     10645.79785, best error:     10638.11523 on epoch:    13, epoch:   15/50, mu: 0.7406504750, learning_rate: 0.0009173741, weight_decay: 0.0001753119

epoch time: 776.9583129883s, input_fired_time: 14.0584917068, output_fired_time: 4.9089121819, propagate_forward_time: 293.1223754883, propagate_backward_time: 455.6016845703, weight_update_time: 0.3536471426, other_time: 9.2668457031
epoch time: 60.3374023438s, input_fired_time: 1.0793393850, output_fired_time: 0.0000000000, propagate_forward_time: 57.8185997009, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4394645691
validation[          , genome  5954] predictions:    6260/  10000 (62.60%), best:    6260/10000 (62.60%), error:     10634.54883, best error:     10634.54883 on epoch:    16, epoch:   16/50, mu: 0.7511940598, learning_rate: 0.0008427519, weight_decay: 0.0001656987

epoch time: 780.4406127930s, input_fired_time: 14.0924434662, output_fired_time: 4.9281387329, propagate_forward_time: 293.9096679688, propagate_backward_time: 458.1288452148, weight_update_time: 0.3652566373, other_time: 9.3814697266
epoch time: 97.2659988403s, input_fired_time: 1.1206442118, output_fired_time: 0.0000000000, propagate_forward_time: 94.3129501343, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8324050903
validation[          , genome  5954] predictions:    6396/  10000 (63.96%), best:    6396/10000 (63.96%), error:     10552.52246, best error:     10552.52246 on epoch:    17, epoch:   17/50, mu: 0.7612918019, learning_rate: 0.0007741998, weight_decay: 0.0001566126

epoch time: 1038.9809570313s, input_fired_time: 14.6380081177, output_fired_time: 5.1726245880, propagate_forward_time: 326.3352050781, propagate_backward_time: 683.0997314453, weight_update_time: 0.3785599768, other_time: 9.7353515625
epoch time: 60.1533012390s, input_fired_time: 0.8534018397, output_fired_time: 0.0000000000, propagate_forward_time: 57.8634223938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4364776611
validation[          , genome  5954] predictions:    6261/  10000 (62.61%), best:    6396/10000 (63.96%), error:     10596.22656, best error:     10552.52246 on epoch:    17, epoch:   18/50, mu: 0.7709625363, learning_rate: 0.0007112239, weight_decay: 0.0001480247

epoch time: 770.5499267578s, input_fired_time: 13.9222631454, output_fired_time: 4.8904647827, propagate_forward_time: 289.6401977539, propagate_backward_time: 453.0079040527, weight_update_time: 0.3530722260, other_time: 9.0891418457
epoch time: 60.3340568542s, input_fired_time: 0.8627159595, output_fired_time: 0.0000000000, propagate_forward_time: 58.0328750610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4384651184
validation[          , genome  5954] predictions:    6437/  10000 (64.37%), best:    6437/10000 (64.37%), error:     10255.06055, best error:     10255.06055 on epoch:    19, epoch:   19/50, mu: 0.7802243829, learning_rate: 0.0006533706, weight_decay: 0.0001399078

epoch time: 775.7134399414s, input_fired_time: 13.9994993210, output_fired_time: 4.9197578430, propagate_forward_time: 292.3362121582, propagate_backward_time: 455.0487976074, weight_update_time: 0.3614034951, other_time: 9.4091796875
epoch time: 62.3199844360s, input_fired_time: 0.9532613158, output_fired_time: 0.0000000000, propagate_forward_time: 59.9076957703, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4590263367
validation[          , genome  5954] predictions:    6275/  10000 (62.75%), best:    6437/10000 (64.37%), error:     10663.33496, best error:     10255.06055 on epoch:    19, epoch:   20/50, mu: 0.7890945673, learning_rate: 0.0006002234, weight_decay: 0.0001322359

epoch time: 777.6189575195s, input_fired_time: 14.1986713409, output_fired_time: 4.9028792381, propagate_forward_time: 293.3837585449, propagate_backward_time: 455.8866882324, weight_update_time: 0.4377665520, other_time: 9.2469482422
epoch time: 60.6629447937s, input_fired_time: 0.8587353826, output_fired_time: 0.0000000000, propagate_forward_time: 58.3613128662, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4428977966
validation[          , genome  5954] predictions:    6290/  10000 (62.90%), best:    6437/10000 (64.37%), error:     10617.18750, best error:     10255.06055 on epoch:    19, epoch:   21/50, mu: 0.7975897193, learning_rate: 0.0005513993, weight_decay: 0.0001249848

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.805726
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 0.000506547
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 0.000118131
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 22
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 19
read number_validation_images: 10000
read best_validation_predictions: 6437
read best_validation_error: 10255.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '928641357'
read generator: 928641357
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1562.22619628906s, input_fired_time: 15.9542474746704, output_fired_time: 5.61716938018799, propagate_forward_time: 652.533508300781, propagate_backward_time: 877.816589355469, weight_update_time: 0.414011985063553, other_time: 10.3046875
epoch time: 73.4515991210938s, input_fired_time: 0.876896023750305, output_fired_time: 0, propagate_forward_time: 60.239070892334, propagate_backward_time: 0, weight_update_time: 0, other_time: 12.3356285095215
validation[          , genome  5954] predictions:    6451/  10000 (64.51%), best:    6451/10000 (64.51%), error:     10185.99805, best error:     10185.99805 on epoch:    22, epoch:   22/50, mu: 0.8057256341, learning_rate: 0.0005065466, weight_decay: 0.0001181312

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.813518
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 0.000465342
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 0.000111653
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 23
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 22
read number_validation_images: 10000
read best_validation_predictions: 6451
read best_validation_error: 10186
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '593785905'
read generator: 593785905
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1131.91101074219s, input_fired_time: 49.4133262634277, output_fired_time: 5.84702396392822, propagate_forward_time: 441.626556396484, propagate_backward_time: 623.168395996094, weight_update_time: 0.438135862350464, other_time: 11.8556518554688
epoch time: 61.1986541748047s, input_fired_time: 0.85912948846817, output_fired_time: 0, propagate_forward_time: 58.8146896362305, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.52483367919922
validation[          , genome  5954] predictions:    6351/  10000 (63.51%), best:    6451/10000 (64.51%), error:     10350.45605, best error:     10185.99805 on epoch:    22, epoch:   23/50, mu: 0.8135175109, learning_rate: 0.0004653425, weight_decay: 0.0001116535

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.82098
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 0.00042749
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 0.000105531
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 24
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 22
read number_validation_images: 10000
read best_validation_predictions: 6451
read best_validation_error: 10186
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '101038835'
read generator: 101038835
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1496.19079589844s, input_fired_time: 16.6043529510498, output_fired_time: 6.01959371566772, propagate_forward_time: 463.387847900391, propagate_backward_time: 998.767150878906, weight_update_time: 0.406622886657715, other_time: 11.4118041992188
epoch time: 138.934707641602s, input_fired_time: 1.06630337238312, output_fired_time: 0, propagate_forward_time: 136.106063842773, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.76234436035156
validation[          , genome  5954] predictions:    6498/  10000 (64.98%), best:    6498/10000 (64.98%), error:     10092.87500, best error:     10092.87500 on epoch:    24, epoch:   24/50, mu: 0.8209799528, learning_rate: 0.0004274900, weight_decay: 0.0001055309

epoch time: 1164.8616943359s, input_fired_time: 16.6130104065, output_fired_time: 5.8345580101, propagate_forward_time: 505.7531433105, propagate_backward_time: 573.4883422852, weight_update_time: 0.4632015526, other_time: 63.1725463867
epoch time: 67.6834030151s, input_fired_time: 0.9479218125, output_fired_time: 0.0000000000, propagate_forward_time: 65.1455993652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5898818970
validation[          , genome  5954] predictions:    6361/  10000 (63.61%), best:    6498/10000 (64.98%), error:     10416.84570, best error:     10092.87500 on epoch:    24, epoch:   25/50, mu: 0.8281268477, learning_rate: 0.0003927166, weight_decay: 0.0000997441

epoch time: 804.6319580078s, input_fired_time: 14.3244590759, output_fired_time: 5.1180553436, propagate_forward_time: 299.3058471680, propagate_backward_time: 476.4168090820, weight_update_time: 0.3678284585, other_time: 9.4667968750
epoch time: 60.9401512146s, input_fired_time: 0.8808387518, output_fired_time: 0.0000000000, propagate_forward_time: 58.6042556763, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4550552368
validation[          , genome  5954] predictions:    6492/  10000 (64.92%), best:    6492/10000 (64.92%), error:     10087.18652, best error:     10087.18652 on epoch:    26, epoch:   26/50, mu: 0.8349715471, learning_rate: 0.0003607718, weight_decay: 0.0000942747

epoch time: 782.2243041992s, input_fired_time: 14.0891313553, output_fired_time: 5.3470473289, propagate_forward_time: 294.1526794434, propagate_backward_time: 459.3968811035, weight_update_time: 0.3566674292, other_time: 9.2385864258
epoch time: 62.5418395996s, input_fired_time: 0.8982976079, output_fired_time: 0.0000000000, propagate_forward_time: 60.0968627930, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5466804504
validation[          , genome  5954] predictions:    6417/  10000 (64.17%), best:    6492/10000 (64.92%), error:     10265.05176, best error:     10087.18652 on epoch:    26, epoch:   27/50, mu: 0.8415268064, learning_rate: 0.0003314254, weight_decay: 0.0000891051

epoch time: 781.0458374023s, input_fired_time: 14.1134748459, output_fired_time: 4.9887981415, propagate_forward_time: 294.6008300781, propagate_backward_time: 458.0535888672, weight_update_time: 0.3585280478, other_time: 9.2891845703
epoch time: 60.7724037170s, input_fired_time: 0.8768582940, output_fired_time: 0.0000000000, propagate_forward_time: 58.4428520203, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4526939392
validation[          , genome  5954] predictions:    6452/  10000 (64.52%), best:    6492/10000 (64.92%), error:     10102.03027, best error:     10087.18652 on epoch:    26, epoch:   28/50, mu: 0.8478049040, learning_rate: 0.0003044662, weight_decay: 0.0000842190

epoch time: 819.6151733398s, input_fired_time: 14.3810997009, output_fired_time: 5.0520257950, propagate_forward_time: 321.6156921387, propagate_backward_time: 468.9041442871, weight_update_time: 0.3663984835, other_time: 9.6622314453
epoch time: 104.0664901733s, input_fired_time: 1.2249970436, output_fired_time: 0.0000000000, propagate_forward_time: 100.7311630249, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1103286743
validation[          , genome  5954] predictions:    6513/  10000 (65.13%), best:    6513/10000 (65.13%), error:      9978.64453, best error:      9978.64453 on epoch:    29, epoch:   29/50, mu: 0.8538175225, learning_rate: 0.0002796999, weight_decay: 0.0000796009

epoch time: 981.8365478516s, input_fired_time: 45.7219505310, output_fired_time: 5.1057505608, propagate_forward_time: 408.2608032227, propagate_backward_time: 513.2214355469, weight_update_time: 0.3605767787, other_time: 9.5266113281
epoch time: 60.6799964905s, input_fired_time: 0.8791230321, output_fired_time: 0.0000000000, propagate_forward_time: 58.3310966492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4697761536
validation[          , genome  5954] predictions:    6468/  10000 (64.68%), best:    6513/10000 (65.13%), error:     10137.15430, best error:      9978.64453 on epoch:    29, epoch:   30/50, mu: 0.8595758677, learning_rate: 0.0002569482, weight_decay: 0.0000752359

epoch time: 777.8575439453s, input_fired_time: 14.0736503601, output_fired_time: 4.9557261467, propagate_forward_time: 292.6864624023, propagate_backward_time: 456.9430847168, weight_update_time: 0.3617341816, other_time: 9.1985778809
epoch time: 60.3080863953s, input_fired_time: 0.8694822192, output_fired_time: 0.0000000000, propagate_forward_time: 57.9949798584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4436225891
validation[          , genome  5954] predictions:    6478/  10000 (64.78%), best:    6513/10000 (65.13%), error:     10140.96582, best error:      9978.64453 on epoch:    29, epoch:   31/50, mu: 0.8650907278, learning_rate: 0.0002360472, weight_decay: 0.0000711104

epoch time: 775.5053710938s, input_fired_time: 14.1290740967, output_fired_time: 4.9207692146, propagate_forward_time: 292.0125732422, propagate_backward_time: 455.2652282715, weight_update_time: 0.3569817245, other_time: 9.1777038574
epoch time: 61.7463989258s, input_fired_time: 0.8731158972, output_fired_time: 0.0000000000, propagate_forward_time: 59.4286041260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4446792603
validation[          , genome  5954] predictions:    6515/  10000 (65.15%), best:    6513/10000 (65.13%), error:     10006.07910, best error:      9978.64453 on epoch:    29, epoch:   32/50, mu: 0.8703724146, learning_rate: 0.0002168464, weight_decay: 0.0000672110

epoch time: 787.8682250977s, input_fired_time: 14.3695764542, output_fired_time: 4.9861655235, propagate_forward_time: 297.1629638672, propagate_backward_time: 461.9862060547, weight_update_time: 0.3633003235, other_time: 9.3633422852
epoch time: 61.5686531067s, input_fired_time: 0.8939542174, output_fired_time: 0.0000000000, propagate_forward_time: 59.2048034668, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4698944092
validation[          , genome  5954] predictions:    6539/  10000 (65.39%), best:    6513/10000 (65.13%), error:      9981.17188, best error:      9978.64453 on epoch:    29, epoch:   33/50, mu: 0.8754307628, learning_rate: 0.0001992074, weight_decay: 0.0000635255

epoch time: 926.2716674805s, input_fired_time: 14.9078340530, output_fired_time: 5.2542734146, propagate_forward_time: 319.8483581543, propagate_backward_time: 576.5431518555, weight_update_time: 0.3801003098, other_time: 9.7180786133
epoch time: 62.5728073120s, input_fired_time: 0.8969878554, output_fired_time: 0.0000000000, propagate_forward_time: 60.1624717712, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5133476257
validation[          , genome  5954] predictions:    6505/  10000 (65.05%), best:    6513/10000 (65.13%), error:     10045.62598, best error:      9978.64453 on epoch:    29, epoch:   34/50, mu: 0.8802752495, learning_rate: 0.0001830032, weight_decay: 0.0000600421

epoch time: 785.8951416016s, input_fired_time: 14.2762699127, output_fired_time: 5.0075445175, propagate_forward_time: 295.9815673828, propagate_backward_time: 461.3318481445, weight_update_time: 0.3561402857, other_time: 9.2979125977
epoch time: 61.6089706421s, input_fired_time: 0.8823039532, output_fired_time: 0.0000000000, propagate_forward_time: 59.2754096985, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4512557983
validation[          , genome  5954] predictions:    6529/  10000 (65.29%), best:    6513/10000 (65.13%), error:     10009.68066, best error:      9978.64453 on epoch:    29, epoch:   35/50, mu: 0.8849148750, learning_rate: 0.0001681172, weight_decay: 0.0000567497

epoch time: 801.2280273438s, input_fired_time: 14.3037958145, output_fired_time: 4.9344592094, propagate_forward_time: 303.5747375488, propagate_backward_time: 469.1835021973, weight_update_time: 0.3577800989, other_time: 9.2315673828
epoch time: 60.6857337952s, input_fired_time: 0.8821732402, output_fired_time: 0.0000000000, propagate_forward_time: 58.3711967468, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4323654175
validation[          , genome  5954] predictions:    6605/  10000 (66.05%), best:    6605/10000 (66.05%), error:      9815.50488, best error:      9815.50488 on epoch:    36, epoch:   36/50, mu: 0.8893583417, learning_rate: 0.0001544420, weight_decay: 0.0000536378

epoch time: 773.6379394531s, input_fired_time: 14.0450286865, output_fired_time: 4.9179348946, propagate_forward_time: 291.6146545410, propagate_backward_time: 453.8883361816, weight_update_time: 0.3553013504, other_time: 9.1719970703
epoch time: 60.4208030701s, input_fired_time: 0.8755702376, output_fired_time: 0.0000000000, propagate_forward_time: 58.0882263184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4570083618
validation[          , genome  5954] predictions:    6536/  10000 (65.36%), best:    6605/10000 (66.05%), error:      9931.77539, best error:      9815.50488 on epoch:    36, epoch:   37/50, mu: 0.8936138749, learning_rate: 0.0001418792, weight_decay: 0.0000506966

epoch time: 821.6128540039s, input_fired_time: 14.3726577759, output_fired_time: 5.0546236038, propagate_forward_time: 309.6804199219, propagate_backward_time: 482.8138427734, weight_update_time: 0.3648047149, other_time: 9.6912841797
epoch time: 63.4942474365s, input_fired_time: 0.9275798202, output_fired_time: 0.0000000000, propagate_forward_time: 61.0331382751, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5335311890
validation[          , genome  5954] predictions:    6546/  10000 (65.46%), best:    6605/10000 (66.05%), error:      9980.12402, best error:      9815.50488 on epoch:    36, epoch:   38/50, mu: 0.8976894617, learning_rate: 0.0001303382, weight_decay: 0.0000479166

epoch time: 811.1857910156s, input_fired_time: 14.3559865952, output_fired_time: 5.0642638206, propagate_forward_time: 308.4051208496, propagate_backward_time: 473.8286132813, weight_update_time: 0.3924373686, other_time: 9.5318298340
epoch time: 61.9385223389s, input_fired_time: 0.8979198933, output_fired_time: 0.0000000000, propagate_forward_time: 59.5225296021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5180740356
validation[          , genome  5954] predictions:    6525/  10000 (65.25%), best:    6605/10000 (66.05%), error:      9954.94434, best error:      9815.50488 on epoch:    36, epoch:   39/50, mu: 0.9015927315, learning_rate: 0.0001197361, weight_decay: 0.0000452891

epoch time: 796.8746337891s, input_fired_time: 14.6974077225, output_fired_time: 5.0312037468, propagate_forward_time: 299.0419006348, propagate_backward_time: 468.3706665039, weight_update_time: 0.3721985519, other_time: 9.7334899902
epoch time: 61.3861656189s, input_fired_time: 0.8925895095, output_fired_time: 0.0000000000, propagate_forward_time: 59.0169525146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4766235352
validation[          , genome  5954] predictions:    6544/  10000 (65.44%), best:    6544/10000 (65.44%), error:      9807.11816, best error:      9807.11816 on epoch:    40, epoch:   40/50, mu: 0.9053309560, learning_rate: 0.0001099964, weight_decay: 0.0000428057

epoch time: 809.0863037109s, input_fired_time: 14.5354728699, output_fired_time: 5.1031937599, propagate_forward_time: 310.8271789551, propagate_backward_time: 468.9832153320, weight_update_time: 0.3647429943, other_time: 9.6372375488
epoch time: 62.9995727539s, input_fired_time: 0.9565000534, output_fired_time: 0.0000000000, propagate_forward_time: 60.4871482849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5559234619
validation[          , genome  5954] predictions:    6582/  10000 (65.82%), best:    6544/10000 (65.44%), error:      9871.07129, best error:      9807.11816 on epoch:    40, epoch:   41/50, mu: 0.9089111090, learning_rate: 0.0001010489, weight_decay: 0.0000404584

epoch time: 845.3224487305s, input_fired_time: 14.2682113647, output_fired_time: 5.0351390839, propagate_forward_time: 307.3004455566, propagate_backward_time: 509.2678833008, weight_update_time: 0.3641631305, other_time: 9.4507446289
epoch time: 62.0082473755s, input_fired_time: 0.8975501657, output_fired_time: 0.0000000000, propagate_forward_time: 59.6055068970, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5051918030
validation[          , genome  5954] predictions:    6537/  10000 (65.37%), best:    6544/10000 (65.44%), error:      9860.64844, best error:      9807.11816 on epoch:    40, epoch:   42/50, mu: 0.9123398662, learning_rate: 0.0000928293, weight_decay: 0.0000382399

epoch time: 791.6829223633s, input_fired_time: 14.2276468277, output_fired_time: 5.0067982674, propagate_forward_time: 293.6768188477, propagate_backward_time: 469.3961181641, weight_update_time: 0.3561182618, other_time: 9.3755493164
epoch time: 61.8061828613s, input_fired_time: 0.8786448836, output_fired_time: 0.0000000000, propagate_forward_time: 59.3078384399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6197013855
validation[          , genome  5954] predictions:    6629/  10000 (66.29%), best:    6544/10000 (65.44%), error:      9816.58594, best error:      9807.11816 on epoch:    40, epoch:   43/50, mu: 0.9156236649, learning_rate: 0.0000852782, weight_decay: 0.0000361430

epoch time: 815.8041381836s, input_fired_time: 14.3613538742, output_fired_time: 4.9835820198, propagate_forward_time: 305.2943420410, propagate_backward_time: 481.9173583984, weight_update_time: 0.3619618714, other_time: 9.2475280762
epoch time: 62.3340950012s, input_fired_time: 0.9035451412, output_fired_time: 0.0000000000, propagate_forward_time: 59.9272460938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5033035278
validation[          , genome  5954] predictions:    6563/  10000 (65.63%), best:    6544/10000 (65.44%), error:      9970.92676, best error:      9807.11816 on epoch:    40, epoch:   44/50, mu: 0.9187685847, learning_rate: 0.0000783414, weight_decay: 0.0000341611

epoch time: 790.3083496094s, input_fired_time: 14.1818866730, output_fired_time: 4.9502201080, propagate_forward_time: 304.0625000000, propagate_backward_time: 457.8786010742, weight_update_time: 0.3572455049, other_time: 9.2351684570
epoch time: 61.6183547974s, input_fired_time: 0.8852723837, output_fired_time: 0.0000000000, propagate_forward_time: 59.2062492371, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5268325806
validation[          , genome  5954] predictions:    6537/  10000 (65.37%), best:    6544/10000 (65.44%), error:      9923.19922, best error:      9807.11816 on epoch:    40, epoch:   45/50, mu: 0.9217805266, learning_rate: 0.0000719689, weight_decay: 0.0000322879

epoch time: 778.2397460938s, input_fired_time: 14.1644315720, output_fired_time: 4.9322228432, propagate_forward_time: 292.4485168457, propagate_backward_time: 457.3765869141, weight_update_time: 0.3692304790, other_time: 9.3179626465
epoch time: 60.6021499634s, input_fired_time: 0.8799549341, output_fired_time: 0.0000000000, propagate_forward_time: 58.2835769653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4386177063
validation[          , genome  5954] predictions:    6603/  10000 (66.03%), best:    6603/10000 (66.03%), error:      9769.28320, best error:      9769.28320 on epoch:    46, epoch:   46/50, mu: 0.9246651530, learning_rate: 0.0000661147, weight_decay: 0.0000305174

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.927428
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 6.07367e-05
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 2.88439e-05
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 47
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 46
read number_validation_images: 10000
read best_validation_predictions: 6603
read best_validation_error: 9769.28
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '347663319'
read generator: 347663319
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 776.356262207031s, input_fired_time: 14.1354093551636, output_fired_time: 4.90536069869995, propagate_forward_time: 292.663787841797, propagate_backward_time: 455.507934570313, weight_update_time: 0.357126981019974, other_time: 9.14376831054688
epoch time: 60.5994529724121s, input_fired_time: 0.883156538009644, output_fired_time: 0, propagate_forward_time: 58.2873420715332, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.428955078125
validation[          , genome  5954] predictions:    6675/  10000 (66.75%), best:    6675/10000 (66.75%), error:      9648.76563, best error:      9648.76563 on epoch:    47, epoch:   47/50, mu: 0.9274277687, learning_rate: 0.0000607367, weight_decay: 0.0000288439

epoch time: 767.5245971680s, input_fired_time: 13.9948825836, output_fired_time: 4.8717093468, propagate_forward_time: 289.3570251465, propagate_backward_time: 450.2807006836, weight_update_time: 0.3663983941, other_time: 9.0202941895
epoch time: 60.0833435059s, input_fired_time: 0.8799709678, output_fired_time: 0.0000000000, propagate_forward_time: 57.7788543701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4245185852
validation[          , genome  5954] predictions:    6684/  10000 (66.84%), best:    6684/10000 (66.84%), error:      9619.06055, best error:      9619.06055 on epoch:    48, epoch:   48/50, mu: 0.9300735593, learning_rate: 0.0000557962, weight_decay: 0.0000272623

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542229504_179_5954_0_r1342040710_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.513285
read mu: 0.932608
read mu_delta: 0.957716
read initial_learning_rate: 0.00327527
read learning_rate: 5.12576e-05
read learning_rate_delta: 0.918657
read initial_weight_decay: 0.000408505
read weight_decay: 2.57673e-05
read weight_decay_delta: 0.945165
read batch_size: 109
read epsilon: 1e-07
read alpha: 0.113522
read input_dropout_probability: 0.00118054
read hidden_dropout_probability: 0.11859
read velocity_reset: 1831
read epoch: 49
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 48
read number_validation_images: 10000
read best_validation_predictions: 6684
read best_validation_error: 9619.06
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5954
read normal distribution: '1 -2.01429486274719 1.53504621982574'
generator_str: '199894419'
read generator: 199894419
read generated_by_map:
1 change_size 1
reading 87 nodes.
reading 427 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1112.65747070313s, input_fired_time: 16.2076606750488, output_fired_time: 16.6715087890625, propagate_forward_time: 394.087432861328, propagate_backward_time: 675.392822265625, weight_update_time: 0.391711801290512, other_time: 10.298095703125
epoch time: 60.0551300048828s, input_fired_time: 0.878575325012207, output_fired_time: 0, propagate_forward_time: 57.7292976379395, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.44725799560547
validation[          , genome  5954] predictions:    6625/  10000 (66.25%), best:    6625/10000 (66.25%), error:      9616.89160, best error:      9616.89160 on epoch:    49, epoch:   49/50, mu: 0.9326075315, learning_rate: 0.0000512576, weight_decay: 0.0000257673

epoch time: 1049.4310302734s, input_fired_time: 14.6220846176, output_fired_time: 5.0872683525, propagate_forward_time: 358.0401916504, propagate_backward_time: 620.8659057617, weight_update_time: 0.3672814667, other_time: 50.8156127930
epoch time: 60.3219566345s, input_fired_time: 0.8804091811, output_fired_time: 0.0000000000, propagate_forward_time: 57.9785041809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4630432129
validation[          , genome  5954] predictions:    6650/  10000 (66.50%), best:    6625/10000 (66.25%), error:      9726.76367, best error:      9616.89160 on epoch:    49, epoch:   50/50, mu: 0.9350343347, learning_rate: 0.0000470881, weight_decay: 0.0000243544

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 302.6435546875s, input_fired_time: 4.4934535027, output_fired_time: 0.0000000000, propagate_forward_time: 290.8386535645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.3114624023
best training[          , genome  5954] predictions:   37275/  50000 (74.55%), best:    6625/10000 (66.25%), error:     38737.86719, best error:      9616.89160 on epoch:    49, epoch:   51/50, mu: 0.9373584986, learning_rate: 0.0000432578, weight_decay: 0.0000230189
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 60.3278160095s, input_fired_time: 0.8831863999, output_fired_time: 0.0000000000, propagate_forward_time: 57.9959793091, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4486503601
   testing[          , genome  5954] predictions:    6625/  10000 (66.25%), best:    6625/10000 (66.25%), error:      9616.89160, best error:      9616.89160 on epoch:    49, epoch:   51/50, mu: 0.9373584986, learning_rate: 0.0000432578, weight_decay: 0.0000230189
backpropagation finished successfully!
22:44:45 (6736): called boinc_finish(0)

</stderr_txt>
]]>