Nameexact_genome_1542495604_181_6222_3
Workunit4096480
Created24 Nov 2018, 10:15:36 UTC
Sent24 Nov 2018, 10:15:46 UTC
Report deadline25 Jan 2019, 10:49:29 UTC
Received27 Nov 2018, 3:22:39 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID78968
Run time1 days 10 hours 59 min 46 sec
CPU time1 days 10 hours 16 min 51 sec
Validate stateValid
Credit15,505.85
Device peak FLOPS2.20 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size564.17 MB
Peak swap size583.84 MB
Peak disk usage12.46 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
0 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2356.07739257813s, input_fired_time: 29.8484115600586, output_fired_time: 13.0505838394165, propagate_forward_time: 894.2099609375, propagate_backward_time: 1395.74938964844, weight_update_time: 0.666561663150787, other_time: 23.2191162109375
epoch time: 177.531066894531s, input_fired_time: 1.99826836585999, output_fired_time: 0, propagate_forward_time: 171.885833740234, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.64695739746094
validation[          , genome  6222] predictions:    5548/  10000 (55.48%), best:    5585/10000 (55.85%), error:     12925.94043, best error:     12841.24902 on epoch:     3, epoch:    6/50, mu: 0.6573305130, learning_rate: 0.0039636982, weight_decay: 0.0002196717

epoch time: 2133.9216308594s, input_fired_time: 28.1903915405, output_fired_time: 11.8083267212, propagate_forward_time: 802.5245971680, propagate_backward_time: 1270.5535888672, weight_update_time: 0.5742082000, other_time: 20.8446044922
epoch time: 169.9561157227s, input_fired_time: 1.8875733614, output_fired_time: 0.0000000000, propagate_forward_time: 164.5978393555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4707031250
validation[          , genome  6222] predictions:    5676/  10000 (56.76%), best:    5676/10000 (56.76%), error:     12690.06738, best error:     12690.06738 on epoch:     7, epoch:    7/50, mu: 0.6792838573, learning_rate: 0.0036367930, weight_decay: 0.0002041664

epoch time: 2193.1965332031s, input_fired_time: 28.5963096619, output_fired_time: 11.4306039810, propagate_forward_time: 825.3135986328, propagate_backward_time: 1307.0468750000, weight_update_time: 0.5611436367, other_time: 20.8092041016
epoch time: 176.4282226563s, input_fired_time: 2.1662075520, output_fired_time: 0.0000000000, propagate_forward_time: 170.2133178711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0486907959
validation[          , genome  6222] predictions:    5679/  10000 (56.79%), best:    5676/10000 (56.76%), error:     12742.80371, best error:     12690.06738 on epoch:     7, epoch:    8/50, mu: 0.6997884512, learning_rate: 0.0033368494, weight_decay: 0.0001897556

epoch time: 2159.2631835938s, input_fired_time: 28.3287467957, output_fired_time: 11.8594989777, propagate_forward_time: 811.3772583008, propagate_backward_time: 1286.6427001953, weight_update_time: 0.5707089305, other_time: 21.0548095703
epoch time: 175.1997985840s, input_fired_time: 1.9807051420, output_fired_time: 0.0000000000, propagate_forward_time: 169.5816345215, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6374664307
validation[          , genome  6222] predictions:    5655/  10000 (56.55%), best:    5676/10000 (56.76%), error:     12728.46973, best error:     12690.06738 on epoch:     7, epoch:    9/50, mu: 0.7189399004, learning_rate: 0.0030616436, weight_decay: 0.0001763620

epoch time: 2267.2314453125s, input_fired_time: 29.2129096985, output_fired_time: 12.2230386734, propagate_forward_time: 855.3291015625, propagate_backward_time: 1348.3239746094, weight_update_time: 0.6053158045, other_time: 22.1423339844
epoch time: 181.0338134766s, input_fired_time: 2.0521821976, output_fired_time: 0.0000000000, propagate_forward_time: 175.2514343262, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7301940918
validation[          , genome  6222] predictions:    5733/  10000 (57.33%), best:    5733/10000 (57.33%), error:     12352.01563, best error:     12352.01563 on epoch:    10, epoch:   10/50, mu: 0.7368275523, learning_rate: 0.0028091352, weight_decay: 0.0001639137

epoch time: 2195.4362792969s, input_fired_time: 28.4567890167, output_fired_time: 11.8822660446, propagate_forward_time: 828.6975097656, propagate_backward_time: 1305.4626464844, weight_update_time: 0.5867829323, other_time: 20.9370117188
epoch time: 164.0987701416s, input_fired_time: 1.8522669077, output_fired_time: 0.0000000000, propagate_forward_time: 158.7650146484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4814910889
validation[          , genome  6222] predictions:    5843/  10000 (58.43%), best:    5843/10000 (58.43%), error:     12209.14941, best error:     12209.14941 on epoch:    11, epoch:   11/50, mu: 0.7535347342, learning_rate: 0.0025774525, weight_decay: 0.0001523441

epoch time: 2261.1784667969s, input_fired_time: 28.8496284485, output_fired_time: 12.0961189270, propagate_forward_time: 855.5415649414, propagate_backward_time: 1342.2484130859, weight_update_time: 0.5957902670, other_time: 22.4427490234
epoch time: 182.2933044434s, input_fired_time: 2.0213458538, output_fired_time: 0.0000000000, propagate_forward_time: 176.5311584473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7407989502
validation[          , genome  6222] predictions:    5726/  10000 (57.26%), best:    5843/10000 (58.43%), error:     12479.64844, best error:     12209.14941 on epoch:    11, epoch:   12/50, mu: 0.7691394091, learning_rate: 0.0023648776, weight_decay: 0.0001415911

epoch time: 2307.8996582031s, input_fired_time: 29.4276924133, output_fired_time: 12.5862236023, propagate_forward_time: 871.9307861328, propagate_backward_time: 1371.8615722656, weight_update_time: 0.6286200285, other_time: 22.0933837891
epoch time: 179.0225677490s, input_fired_time: 2.1713147163, output_fired_time: 0.0000000000, propagate_forward_time: 172.8547821045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9964752197
validation[          , genome  6222] predictions:    5827/  10000 (58.27%), best:    5843/10000 (58.43%), error:     12229.48633, best error:     12209.14941 on epoch:    11, epoch:   13/50, mu: 0.7837142944, learning_rate: 0.0021698349, weight_decay: 0.0001315970

epoch time: 2314.5786132813s, input_fired_time: 29.5241832733, output_fired_time: 13.0742359161, propagate_forward_time: 880.0602416992, propagate_backward_time: 1369.5115966797, weight_update_time: 0.6361450553, other_time: 22.4083251953
epoch time: 166.3925018311s, input_fired_time: 1.8666191101, output_fired_time: 0.0000000000, propagate_forward_time: 161.2565307617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2693481445
validation[          , genome  6222] predictions:    5683/  10000 (56.83%), best:    5843/10000 (58.43%), error:     12405.05664, best error:     12209.14941 on epoch:    11, epoch:   14/50, mu: 0.7973273993, learning_rate: 0.0019908783, weight_decay: 0.0001223084

epoch time: 2252.8828125000s, input_fired_time: 28.6805934906, output_fired_time: 12.2138299942, propagate_forward_time: 854.3568725586, propagate_backward_time: 1336.3488769531, weight_update_time: 0.5687875152, other_time: 21.2824707031
epoch time: 182.1592254639s, input_fired_time: 2.1340961456, output_fired_time: 0.0000000000, propagate_forward_time: 176.0989074707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9262237549
validation[          , genome  6222] predictions:    5942/  10000 (59.42%), best:    5942/10000 (59.42%), error:     11625.75098, best error:     11625.75098 on epoch:    15, epoch:   15/50, mu: 0.8100421429, learning_rate: 0.0018266811, weight_decay: 0.0001136754

epoch time: 2320.0319824219s, input_fired_time: 30.9031200409, output_fired_time: 13.8774776459, propagate_forward_time: 881.2491455078, propagate_backward_time: 1368.9875488281, weight_update_time: 0.6778643131, other_time: 25.0147705078
epoch time: 169.9442596436s, input_fired_time: 2.0832843781, output_fired_time: 0.0000000000, propagate_forward_time: 164.1010437012, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7599334717
validation[          , genome  6222] predictions:    5841/  10000 (58.41%), best:    5942/10000 (59.42%), error:     12143.89258, best error:     11625.75098 on epoch:    15, epoch:   16/50, mu: 0.8219178319, learning_rate: 0.0016760260, weight_decay: 0.0001056518

epoch time: 2254.0754394531s, input_fired_time: 28.9007320404, output_fired_time: 12.3908271790, propagate_forward_time: 850.4396362305, propagate_backward_time: 1340.3646240234, weight_update_time: 0.6224794984, other_time: 21.9796142578
epoch time: 163.1571960449s, input_fired_time: 1.9727149010, output_fired_time: 0.0000000000, propagate_forward_time: 157.6890411377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4954376221
validation[          , genome  6222] predictions:    6027/  10000 (60.27%), best:    5942/10000 (59.42%), error:     11704.00586, best error:     11625.75098 on epoch:    15, epoch:   17/50, mu: 0.8330098391, learning_rate: 0.0015377962, weight_decay: 0.0000981945

epoch time: 2219.1088867188s, input_fired_time: 28.8704814911, output_fired_time: 12.0738840103, propagate_forward_time: 839.0687866211, propagate_backward_time: 1317.3134765625, weight_update_time: 0.5768094063, other_time: 21.7822265625
epoch time: 182.0647888184s, input_fired_time: 2.1217663288, output_fired_time: 0.0000000000, propagate_forward_time: 176.0027008057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9403228760
validation[          , genome  6222] predictions:    5952/  10000 (59.52%), best:    5942/10000 (59.42%), error:     11815.21777, best error:     11625.75098 on epoch:    15, epoch:   18/50, mu: 0.8433698416, learning_rate: 0.0014109669, weight_decay: 0.0000912636

epoch time: 2303.3356933594s, input_fired_time: 32.8767623901, output_fired_time: 14.3478822708, propagate_forward_time: 868.4225463867, propagate_backward_time: 1361.4000244141, weight_update_time: 0.6962429881, other_time: 26.2884521484
epoch time: 174.7472686768s, input_fired_time: 2.0004162788, output_fired_time: 0.0000000000, propagate_forward_time: 169.2454681396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5013885498
validation[          , genome  6222] predictions:    6022/  10000 (60.22%), best:    6022/10000 (60.22%), error:     11505.89355, best error:     11505.89355 on epoch:    19, epoch:   19/50, mu: 0.8530461788, learning_rate: 0.0012945976, weight_decay: 0.0000848219

epoch time: 2257.3691406250s, input_fired_time: 29.0020980835, output_fired_time: 11.9422893524, propagate_forward_time: 858.1133422852, propagate_backward_time: 1336.5216064453, weight_update_time: 0.5974762440, other_time: 21.7896728516
epoch time: 183.3835449219s, input_fired_time: 2.2183377743, output_fired_time: 0.0000000000, propagate_forward_time: 177.0886688232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0765380859
validation[          , genome  6222] predictions:    5923/  10000 (59.23%), best:    6022/10000 (60.22%), error:     11752.98828, best error:     11505.89355 on epoch:    19, epoch:   20/50, mu: 0.8620839715, learning_rate: 0.0011878260, weight_decay: 0.0000788348

epoch time: 2337.9243164063s, input_fired_time: 31.0223369598, output_fired_time: 13.0176486969, propagate_forward_time: 882.5736083984, propagate_backward_time: 1388.6326904297, weight_update_time: 0.6351045370, other_time: 22.6782226563
epoch time: 180.8111419678s, input_fired_time: 2.2189228535, output_fired_time: 0.0000000000, propagate_forward_time: 174.5948944092, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9973297119
validation[          , genome  6222] predictions:    5982/  10000 (59.82%), best:    6022/10000 (60.22%), error:     11698.22266, best error:     11505.89355 on epoch:    19, epoch:   21/50, mu: 0.8705253005, learning_rate: 0.0010898603, weight_decay: 0.0000732704

epoch time: 2258.0244140625s, input_fired_time: 29.5318260193, output_fired_time: 11.8265428543, propagate_forward_time: 848.7817382813, propagate_backward_time: 1346.1953125000, weight_update_time: 0.5916566849, other_time: 21.6889648438
epoch time: 175.5014190674s, input_fired_time: 1.9615070820, output_fired_time: 0.0000000000, propagate_forward_time: 169.8520050049, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6879119873
validation[          , genome  6222] predictions:    6022/  10000 (60.22%), best:    6022/10000 (60.22%), error:     11578.12988, best error:     11505.89355 on epoch:    19, epoch:   22/50, mu: 0.8784096241, learning_rate: 0.0009999743, weight_decay: 0.0000680987

epoch time: 2338.9377441406s, input_fired_time: 30.8379592896, output_fired_time: 12.8852586746, propagate_forward_time: 883.8951416016, propagate_backward_time: 1388.4127197266, weight_update_time: 0.6256566048, other_time: 22.9067382813
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.885774
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000917502
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 6.3292e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 23
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 19
read number_validation_images: 10000
read best_validation_predictions: 6022
read best_validation_error: 11505.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '730323269'
read generator: 730323269
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2183.33325195313s, input_fired_time: 28.3935527801514, output_fired_time: 11.6544914245605, propagate_forward_time: 824.945190429688, propagate_backward_time: 1297.51501464844, weight_update_time: 0.554406881332397, other_time: 20.824951171875
epoch time: 172.794036865234s, input_fired_time: 1.95448577404022, output_fired_time: 0, propagate_forward_time: 167.212066650391, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.62748718261719
validation[          , genome  6222] predictions:    6091/  10000 (60.91%), best:    6091/10000 (60.91%), error:     11441.91016, best error:     11441.91016 on epoch:    23, epoch:   23/50, mu: 0.8857736588, learning_rate: 0.0009175017, weight_decay: 0.0000632920

epoch time: 2027.3353271484s, input_fired_time: 27.0839042664, output_fired_time: 10.6938257217, propagate_forward_time: 760.5520019531, propagate_backward_time: 1209.7926025391, weight_update_time: 0.5379645228, other_time: 19.2130126953
epoch time: 157.5508880615s, input_fired_time: 1.8530535698, output_fired_time: 0.0000000000, propagate_forward_time: 152.4115905762, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2862396240
validation[          , genome  6222] predictions:    5971/  10000 (59.71%), best:    6091/10000 (60.91%), error:     11582.57617, best error:     11441.91016 on epoch:    23, epoch:   24/50, mu: 0.8926516771, learning_rate: 0.0008418310, weight_decay: 0.0000588246

epoch time: 2131.2509765625s, input_fired_time: 28.1586742401, output_fired_time: 11.5015172958, propagate_forward_time: 804.8413696289, propagate_backward_time: 1266.5338134766, weight_update_time: 0.5699060559, other_time: 20.2156982422
epoch time: 169.3793640137s, input_fired_time: 1.9081448317, output_fired_time: 0.0000000000, propagate_forward_time: 164.1025390625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3686828613
validation[          , genome  6222] predictions:    6064/  10000 (60.64%), best:    6091/10000 (60.91%), error:     11559.26758, best error:     11441.91016 on epoch:    23, epoch:   25/50, mu: 0.8990758061, learning_rate: 0.0007724012, weight_decay: 0.0000546726

epoch time: 2103.2651367188s, input_fired_time: 28.0628089905, output_fired_time: 11.1345462799, propagate_forward_time: 789.2853393555, propagate_backward_time: 1254.3188476563, weight_update_time: 0.5685371757, other_time: 20.4636230469
epoch time: 175.3308868408s, input_fired_time: 2.0015320778, output_fired_time: 0.0000000000, propagate_forward_time: 169.7696380615, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5597229004
validation[          , genome  6222] predictions:    6082/  10000 (60.82%), best:    6082/10000 (60.82%), error:     11427.72656, best error:     11427.72656 on epoch:    26, epoch:   26/50, mu: 0.9050760269, learning_rate: 0.0007086975, weight_decay: 0.0000508136

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.91068
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000650248
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 4.7227e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 27
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 26
read number_validation_images: 10000
read best_validation_predictions: 6082
read best_validation_error: 11427.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '149750752'
read generator: 149750752
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.91068
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000650248
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 4.7227e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 27
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 26
read number_validation_images: 10000
read best_validation_predictions: 6082
read best_validation_error: 11427.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '149750752'
read generator: 149750752
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2145.73413085938s, input_fired_time: 27.9659194946289, output_fired_time: 11.4071588516235, propagate_forward_time: 810.413818359375, propagate_backward_time: 1270.56311035156, weight_update_time: 0.576547920703888, other_time: 25.3841552734375
epoch time: 169.310501098633s, input_fired_time: 1.98121094703674, output_fired_time: 0, propagate_forward_time: 163.703216552734, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.62606811523438
validation[          , genome  6222] predictions:    6234/  10000 (62.34%), best:    6234/10000 (62.34%), error:     10989.19336, best error:     10989.19336 on epoch:    27, epoch:   27/50, mu: 0.9106802940, learning_rate: 0.0006502478, weight_decay: 0.0000472270

epoch time: 2017.1883544922s, input_fired_time: 27.2647953033, output_fired_time: 10.6226949692, propagate_forward_time: 756.2637939453, propagate_backward_time: 1204.2153320313, weight_update_time: 0.5159915686, other_time: 18.8217773438
epoch time: 166.7435607910s, input_fired_time: 1.9974745512, output_fired_time: 0.0000000000, propagate_forward_time: 161.3608245850, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3852691650
validation[          , genome  6222] predictions:    6100/  10000 (61.00%), best:    6234/10000 (62.34%), error:     11356.40332, best error:     10989.19336 on epoch:    27, epoch:   28/50, mu: 0.9159147143, learning_rate: 0.0005966188, weight_decay: 0.0000438935

epoch time: 2254.5993652344s, input_fired_time: 30.0446109772, output_fired_time: 12.0526504517, propagate_forward_time: 848.0723876953, propagate_backward_time: 1343.0714111328, weight_update_time: 0.5875134468, other_time: 21.3581542969
epoch time: 184.6338195801s, input_fired_time: 2.1904320717, output_fired_time: 0.0000000000, propagate_forward_time: 178.3365325928, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1068572998
validation[          , genome  6222] predictions:    6061/  10000 (60.61%), best:    6234/10000 (62.34%), error:     11457.56738, best error:     10989.19336 on epoch:    27, epoch:   29/50, mu: 0.9208037257, learning_rate: 0.0005474128, weight_decay: 0.0000407953

epoch time: 2313.1464843750s, input_fired_time: 30.6139411926, output_fired_time: 12.7240581512, propagate_forward_time: 870.7225952148, propagate_backward_time: 1377.0834960938, weight_update_time: 0.5915716887, other_time: 22.0021972656
epoch time: 183.4822082520s, input_fired_time: 2.1752903461, output_fired_time: 0.0000000000, propagate_forward_time: 177.4167633057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8901519775
validation[          , genome  6222] predictions:    6213/  10000 (62.13%), best:    6234/10000 (62.34%), error:     11182.09961, best error:     10989.19336 on epoch:    27, epoch:   30/50, mu: 0.9253700972, learning_rate: 0.0005022651, weight_decay: 0.0000379159

epoch time: 2397.7971191406s, input_fired_time: 30.9341373444, output_fired_time: 13.1360540390, propagate_forward_time: 908.7346191406, propagate_backward_time: 1421.3135986328, weight_update_time: 0.6527695656, other_time: 23.6788330078
epoch time: 180.8462677002s, input_fired_time: 2.2118289471, output_fired_time: 0.0000000000, propagate_forward_time: 174.7466735840, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8877716064
validation[          , genome  6222] predictions:    6185/  10000 (61.85%), best:    6234/10000 (62.34%), error:     11120.82520, best error:     10989.19336 on epoch:    27, epoch:   31/50, mu: 0.9296351075, learning_rate: 0.0004608409, weight_decay: 0.0000352396

epoch time: 2396.9682617188s, input_fired_time: 31.3776321411, output_fired_time: 13.3610000610, propagate_forward_time: 902.8558959961, propagate_backward_time: 1425.8889160156, weight_update_time: 0.6446312070, other_time: 23.4846191406
epoch time: 179.4815368652s, input_fired_time: 2.1887383461, output_fired_time: 0.0000000000, propagate_forward_time: 173.6505279541, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6422729492
validation[          , genome  6222] predictions:    6077/  10000 (60.77%), best:    6234/10000 (62.34%), error:     11429.67969, best error:     10989.19336 on epoch:    27, epoch:   32/50, mu: 0.9336186647, learning_rate: 0.0004228331, weight_decay: 0.0000327523

epoch time: 2352.8693847656s, input_fired_time: 32.4451370239, output_fired_time: 14.1865854263, propagate_forward_time: 888.1587524414, propagate_backward_time: 1393.2468261719, weight_update_time: 0.6708865166, other_time: 24.8322753906
epoch time: 190.6240386963s, input_fired_time: 2.4542479515, output_fired_time: 0.0000000000, propagate_forward_time: 183.7694702148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.4003143311
validation[          , genome  6222] predictions:    6229/  10000 (62.29%), best:    6234/10000 (62.34%), error:     11067.32227, best error:     10989.19336 on epoch:    27, epoch:   33/50, mu: 0.9373393655, learning_rate: 0.0003879601, weight_decay: 0.0000304405

epoch time: 2474.2399902344s, input_fired_time: 32.3556976318, output_fired_time: 13.7776727676, propagate_forward_time: 935.7422485352, propagate_backward_time: 1467.4715576172, weight_update_time: 0.6554173827, other_time: 24.8929443359
epoch time: 188.9756317139s, input_fired_time: 2.2910206318, output_fired_time: 0.0000000000, propagate_forward_time: 182.4213562012, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2632598877
validation[          , genome  6222] predictions:    6185/  10000 (61.85%), best:    6234/10000 (62.34%), error:     11131.58203, best error:     10989.19336 on epoch:    27, epoch:   34/50, mu: 0.9408144951, learning_rate: 0.0003559632, weight_decay: 0.0000282919

epoch time: 2409.8251953125s, input_fired_time: 31.1038455963, output_fired_time: 13.0105466843, propagate_forward_time: 907.5639648438, propagate_backward_time: 1434.4129638672, weight_update_time: 0.6900243163, other_time: 23.7340087891
epoch time: 191.4085388184s, input_fired_time: 2.2473349571, output_fired_time: 0.0000000000, propagate_forward_time: 185.1501312256, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0110778809
validation[          , genome  6222] predictions:    6191/  10000 (61.91%), best:    6234/10000 (62.34%), error:     11083.30859, best error:     10989.19336 on epoch:    27, epoch:   35/50, mu: 0.9440603256, learning_rate: 0.0003266052, weight_decay: 0.0000262949

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.947092
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000299668
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 2.4439e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 36
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 27
read number_validation_images: 10000
read best_validation_predictions: 6234
read best_validation_error: 10989.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '1339226005'
read generator: 1339226005
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2157.2314453125s, input_fired_time: 28.6144943237305, output_fired_time: 11.3908910751343, propagate_forward_time: 808.232177734375, propagate_backward_time: 1288.46081542969, weight_update_time: 0.54891175031662, other_time: 20.5330810546875
epoch time: 176.477645874023s, input_fired_time: 2.10786128044128, output_fired_time: 0, propagate_forward_time: 170.69303894043, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.67674255371094
validation[          , genome  6222] predictions:    6262/  10000 (62.62%), best:    6234/10000 (62.34%), error:     11015.87402, best error:     10989.19336 on epoch:    27, epoch:   36/50, mu: 0.9470919371, learning_rate: 0.0002996685, weight_decay: 0.0000244390

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.949924
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000274953
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 2.2714e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 37
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 27
read number_validation_images: 10000
read best_validation_predictions: 6234
read best_validation_error: 10989.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '1709991181'
read generator: 1709991181
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_181_6222_3_r580915820_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.488919
read mu: 0.949924
read mu_delta: 0.934009
read initial_learning_rate: 0.00664343
read learning_rate: 0.000274953
read learning_rate_delta: 0.917525
read initial_weight_decay: 0.00034081
read weight_decay: 2.2714e-05
read weight_decay_delta: 0.929416
read batch_size: 110
read epsilon: 1e-07
read alpha: 0.0417636
read input_dropout_probability: 0.00138758
read hidden_dropout_probability: 0.112593
read velocity_reset: 981
read epoch: 37
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 27
read number_validation_images: 10000
read best_validation_predictions: 6234
read best_validation_error: 10989.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6222
read normal distribution: '0 -0.976721584796906 -1.21197116374969'
generator_str: '1709991181'
read generator: 1709991181
read generated_by_map:
1 crossover 1
reading 103 nodes.
reading 387 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2203.54541015625s, input_fired_time: 29.2677974700928, output_fired_time: 11.7101593017578, propagate_forward_time: 832.928833007813, propagate_backward_time: 1308.28588867188, weight_update_time: 0.555206835269928, other_time: 21.3526611328125
epoch time: 159.963317871094s, input_fired_time: 1.88553011417389, output_fired_time: 0, propagate_forward_time: 154.658538818359, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.41925048828125
validation[          , genome  6222] predictions:    6308/  10000 (63.08%), best:    6308/10000 (63.08%), error:     10819.87793, best error:     10819.87793 on epoch:    37, epoch:   37/50, mu: 0.9499235153, learning_rate: 0.0002749534, weight_decay: 0.0000227140

epoch time: 2231.7504882813s, input_fired_time: 29.5438709259, output_fired_time: 12.0247173309, propagate_forward_time: 835.8789062500, propagate_backward_time: 1333.4323730469, weight_update_time: 0.5893776417, other_time: 20.8706054688
epoch time: 160.1761779785s, input_fired_time: 1.9378638268, output_fired_time: 0.0000000000, propagate_forward_time: 154.7897796631, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4485321045
validation[          , genome  6222] predictions:    6268/  10000 (62.68%), best:    6308/10000 (63.08%), error:     10872.54199, best error:     10819.87793 on epoch:    37, epoch:   38/50, mu: 0.9525682330, learning_rate: 0.0002522767, weight_decay: 0.0000211107

epoch time: 2228.3713378906s, input_fired_time: 29.7474880219, output_fired_time: 12.2396974564, propagate_forward_time: 835.5122680664, propagate_backward_time: 1329.8262939453, weight_update_time: 0.5554963946, other_time: 21.0455322266
epoch time: 183.5386962891s, input_fired_time: 2.1977894306, output_fired_time: 0.0000000000, propagate_forward_time: 177.3151855469, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0257263184
validation[          , genome  6222] predictions:    6265/  10000 (62.65%), best:    6308/10000 (63.08%), error:     10922.33008, best error:     10819.87793 on epoch:    37, epoch:   39/50, mu: 0.9550384283, learning_rate: 0.0002314702, weight_decay: 0.0000196207

epoch time: 2163.9526367188s, input_fired_time: 29.5913219452, output_fired_time: 11.9569072723, propagate_forward_time: 812.7679443359, propagate_backward_time: 1288.9426269531, weight_update_time: 0.5947607160, other_time: 20.6939697266
epoch time: 180.3182983398s, input_fired_time: 2.1430530548, output_fired_time: 0.0000000000, propagate_forward_time: 174.3396911621, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8355560303
validation[          , genome  6222] predictions:    6285/  10000 (62.85%), best:    6308/10000 (63.08%), error:     10958.89453, best error:     10819.87793 on epoch:    37, epoch:   40/50, mu: 0.9573456049, learning_rate: 0.0002123798, weight_decay: 0.0000182358

epoch time: 2319.3828125000s, input_fired_time: 30.7472953796, output_fired_time: 12.5601444244, propagate_forward_time: 876.4243774414, propagate_backward_time: 1377.6435546875, weight_update_time: 0.6095963120, other_time: 22.0075683594
epoch time: 186.0611877441s, input_fired_time: 2.2975637913, output_fired_time: 0.0000000000, propagate_forward_time: 179.7605133057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0031127930
validation[          , genome  6222] predictions:    6272/  10000 (62.72%), best:    6272/10000 (62.72%), error:     10797.11621, best error:     10797.11621 on epoch:    41, epoch:   41/50, mu: 0.9595004916, learning_rate: 0.0001948638, weight_decay: 0.0000169486

epoch time: 2232.8669433594s, input_fired_time: 29.4879722595, output_fired_time: 11.9348592758, propagate_forward_time: 840.2992553711, propagate_backward_time: 1330.0589599609, weight_update_time: 0.5755869150, other_time: 21.0858154297
epoch time: 187.4610443115s, input_fired_time: 2.2492222786, output_fired_time: 0.0000000000, propagate_forward_time: 181.3106079102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9012145996
validation[          , genome  6222] predictions:    6237/  10000 (62.37%), best:    6272/10000 (62.72%), error:     11002.11621, best error:     10797.11621 on epoch:    41, epoch:   42/50, mu: 0.9615132213, learning_rate: 0.0001787925, weight_decay: 0.0000157523

epoch time: 2300.7741699219s, input_fired_time: 29.9910526276, output_fired_time: 12.1519594193, propagate_forward_time: 860.7803344727, propagate_backward_time: 1376.1573486328, weight_update_time: 0.6006068587, other_time: 21.6937255859
epoch time: 179.5661926270s, input_fired_time: 2.3183219433, output_fired_time: 0.0000000000, propagate_forward_time: 173.2481994629, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9996643066
validation[          , genome  6222] predictions:    6307/  10000 (63.07%), best:    6272/10000 (62.72%), error:     10886.37891, best error:     10797.11621 on epoch:    41, epoch:   43/50, mu: 0.9633930922, learning_rate: 0.0001640466, weight_decay: 0.0000146405

epoch time: 2347.7153320313s, input_fired_time: 31.1261215210, output_fired_time: 12.8659057617, propagate_forward_time: 884.1707153320, propagate_backward_time: 1395.9996337891, weight_update_time: 0.6593204737, other_time: 23.5528564453
epoch time: 168.9716186523s, input_fired_time: 2.0515372753, output_fired_time: 0.0000000000, propagate_forward_time: 163.4832000732, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4368743896
validation[          , genome  6222] predictions:    6232/  10000 (62.32%), best:    6272/10000 (62.72%), error:     10998.77441, best error:     10797.11621 on epoch:    41, epoch:   44/50, mu: 0.9651489258, learning_rate: 0.0001505169, weight_decay: 0.0000136071

epoch time: 2358.4887695313s, input_fired_time: 31.3025894165, output_fired_time: 12.7713184357, propagate_forward_time: 886.9577026367, propagate_backward_time: 1404.3701171875, weight_update_time: 0.6271185875, other_time: 23.0871582031
epoch time: 177.8603363037s, input_fired_time: 2.1602737904, output_fired_time: 0.0000000000, propagate_forward_time: 172.2199554443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4801025391
validation[          , genome  6222] predictions:    6301/  10000 (63.01%), best:    6272/10000 (62.72%), error:     10839.35645, best error:     10797.11621 on epoch:    41, epoch:   45/50, mu: 0.9667888880, learning_rate: 0.0001381030, weight_decay: 0.0000126466

epoch time: 2387.9934082031s, input_fired_time: 31.8649654388, output_fired_time: 12.8642654419, propagate_forward_time: 897.9701538086, propagate_backward_time: 1422.2443847656, weight_update_time: 0.6257272363, other_time: 23.0495605469
epoch time: 181.0797424316s, input_fired_time: 2.2683494091, output_fired_time: 0.0000000000, propagate_forward_time: 174.8473663330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9640197754
validation[          , genome  6222] predictions:    6320/  10000 (63.20%), best:    6320/10000 (63.20%), error:     10749.43359, best error:     10749.43359 on epoch:    46, epoch:   46/50, mu: 0.9683206081, learning_rate: 0.0001267130, weight_decay: 0.0000117540

epoch time: 2364.1884765625s, input_fired_time: 31.7863712311, output_fired_time: 12.9495229721, propagate_forward_time: 893.1898803711, propagate_backward_time: 1403.2839355469, weight_update_time: 0.6290493608, other_time: 22.9787597656
epoch time: 182.9528808594s, input_fired_time: 2.1899971962, output_fired_time: 0.0000000000, propagate_forward_time: 176.9733581543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7895202637
validation[          , genome  6222] predictions:    6304/  10000 (63.04%), best:    6304/10000 (63.04%), error:     10743.55469, best error:     10743.55469 on epoch:    47, epoch:   47/50, mu: 0.9697512388, learning_rate: 0.0001162624, weight_decay: 0.0000109244

epoch time: 2315.6491699219s, input_fired_time: 31.7969627380, output_fired_time: 12.8661899567, propagate_forward_time: 869.6121215820, propagate_backward_time: 1378.6274414063, weight_update_time: 0.6252053976, other_time: 22.7465820313
epoch time: 186.8281860352s, input_fired_time: 2.2442319393, output_fired_time: 0.0000000000, propagate_forward_time: 180.6389312744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9450225830
validation[          , genome  6222] predictions:    6265/  10000 (62.65%), best:    6304/10000 (63.04%), error:     10868.63965, best error:     10743.55469 on epoch:    47, epoch:   48/50, mu: 0.9710874557, learning_rate: 0.0001066737, weight_decay: 0.0000101533

epoch time: 2325.5710449219s, input_fired_time: 31.6787166595, output_fired_time: 12.5966768265, propagate_forward_time: 873.3389282227, propagate_backward_time: 1385.3229980469, weight_update_time: 0.6326441765, other_time: 22.6337890625
epoch time: 187.2058105469s, input_fired_time: 2.2492728233, output_fired_time: 0.0000000000, propagate_forward_time: 181.2556152344, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7009277344
validation[          , genome  6222] predictions:    6348/  10000 (63.48%), best:    6348/10000 (63.48%), error:     10661.02637, best error:     10661.02637 on epoch:    49, epoch:   49/50, mu: 0.9723355174, learning_rate: 0.0000978758, weight_decay: 0.0000094366

epoch time: 2392.8339843750s, input_fired_time: 32.3208885193, output_fired_time: 13.2767667770, propagate_forward_time: 900.7570800781, propagate_backward_time: 1421.9368896484, weight_update_time: 0.6512678862, other_time: 24.5423583984
epoch time: 177.0406036377s, input_fired_time: 2.2114858627, output_fired_time: 0.0000000000, propagate_forward_time: 171.1426849365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6864318848
validation[          , genome  6222] predictions:    6327/  10000 (63.27%), best:    6348/10000 (63.48%), error:     10813.08301, best error:     10661.02637 on epoch:    49, epoch:   50/50, mu: 0.9735012054, learning_rate: 0.0000898035, weight_decay: 0.0000087705

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 909.6245727539s, input_fired_time: 11.0293560028, output_fired_time: 0.0000000000, propagate_forward_time: 878.7980346680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 19.7971801758
best training[          , genome  6222] predictions:   37324/  50000 (74.65%), best:    6348/10000 (63.48%), error:     38687.91016, best error:     10661.02637 on epoch:    49, epoch:   51/50, mu: 0.9745900035, learning_rate: 0.0000823970, weight_decay: 0.0000081515
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 183.1244049072s, input_fired_time: 2.1847445965, output_fired_time: 0.0000000000, propagate_forward_time: 176.2507476807, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6889190674
   testing[          , genome  6222] predictions:    6348/  10000 (63.48%), best:    6348/10000 (63.48%), error:     10661.02637, best error:     10661.02637 on epoch:    49, epoch:   51/50, mu: 0.9745900035, learning_rate: 0.0000823970, weight_decay: 0.0000081515
backpropagation finished successfully!
19:22:19 (3444): called boinc_finish(0)

</stderr_txt>
]]>