Nameexact_genome_1542495604_172_6196_1
Workunit4095940
Created18 Nov 2018, 3:02:51 UTC
Sent18 Nov 2018, 3:17:03 UTC
Report deadline15 Dec 2018, 11:38:34 UTC
Received19 Nov 2018, 13:15:40 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60920
Run time12 hours 6 min 22 sec
CPU time12 hours 2 min 26 sec
Validate stateValid
Credit6,837.07
Device peak FLOPS3.88 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size571.34 MB
Peak swap size569.62 MB
Peak disk usage12.75 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_172_6196.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_172_6196_1_r1249575589_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542495604_172_6196.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.496698
read mu: 0.496698
read mu_delta: 0.93551
read initial_learning_rate: 0.000290716
read learning_rate: 0.000290716
read learning_rate_delta: 0.930071
read initial_weight_decay: 0.000221546
read weight_decay: 0.000221546
read weight_decay_delta: 0.941912
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.092828
read input_dropout_probability: 0.00134115
read hidden_dropout_probability: 0.0827877
read velocity_reset: 1876
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6196
read normal distribution: '1 0.0908117964863777 -1.11697173118591'
generator_str: '1638339267'
read generator: 1638339267
read generated_by_map:
1 add_edge 1
reading 105 nodes.
reading 415 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 532738635
post shuffle 1: 1706553764
epoch time: 1532.16918945313s, input_fired_time: 24.3494052886963, output_fired_time: 11.9767427444458, propagate_forward_time: 585.321105957031, propagate_backward_time: 896.228637695313, weight_update_time: 0.456936925649643, other_time: 14.2933959960938
epoch time: 121.794563293457s, input_fired_time: 1.92348992824554, output_fired_time: 0, propagate_forward_time: 117.391105651855, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.47996520996094
validation[          , genome  6196] predictions:    6229/  10000 (62.29%), best:    6229/10000 (62.29%), error:     10925.18848, best error:     10925.18848 on epoch:     0, epoch:    0/25, mu: 0.4966982603, learning_rate: 0.0002907157, weight_decay: 0.0002215457

epoch time: 1525.2358398438s, input_fired_time: 24.3171977997, output_fired_time: 11.8865613937, propagate_forward_time: 581.4772949219, propagate_backward_time: 893.2713623047, weight_update_time: 0.4528579712, other_time: 14.2834472656
epoch time: 121.2966232300s, input_fired_time: 1.9081872702, output_fired_time: 0.0000000000, propagate_forward_time: 116.8727569580, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5156784058
validation[          , genome  6196] predictions:    6312/  10000 (63.12%), best:    6312/10000 (63.12%), error:     10896.05469, best error:     10896.05469 on epoch:     1, epoch:    1/25, mu: 0.5285115242, learning_rate: 0.0002703863, weight_decay: 0.0002086765

epoch time: 1523.1406250000s, input_fired_time: 24.2989006042, output_fired_time: 11.8897275925, propagate_forward_time: 581.2996826172, propagate_backward_time: 891.3061523438, weight_update_time: 0.4534136653, other_time: 14.3460693359
epoch time: 121.7087173462s, input_fired_time: 1.9179620743, output_fired_time: 0.0000000000, propagate_forward_time: 117.2775878906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5131683350
validation[          , genome  6196] predictions:    6366/  10000 (63.66%), best:    6366/10000 (63.66%), error:     10719.66504, best error:     10719.66504 on epoch:     2, epoch:    2/25, mu: 0.5582731366, learning_rate: 0.0002514784, weight_decay: 0.0001965548

epoch time: 1521.8847656250s, input_fired_time: 24.3019447327, output_fired_time: 11.9253139496, propagate_forward_time: 581.1957397461, propagate_backward_time: 890.1022338867, weight_update_time: 0.4533518255, other_time: 14.3594970703
epoch time: 121.6405868530s, input_fired_time: 1.9056987762, output_fired_time: 0.0000000000, propagate_forward_time: 117.2108993530, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5239868164
validation[          , genome  6196] predictions:    6190/  10000 (61.90%), best:    6366/10000 (63.66%), error:     11038.76074, best error:     10719.66504 on epoch:     2, epoch:    3/25, mu: 0.5861154199, learning_rate: 0.0002338927, weight_decay: 0.0001851372

epoch time: 1526.5191650391s, input_fired_time: 24.2971744537, output_fired_time: 11.9101133347, propagate_forward_time: 582.5371093750, propagate_backward_time: 893.5009765625, weight_update_time: 0.4544705451, other_time: 14.2738037109
epoch time: 122.3620071411s, input_fired_time: 1.9155193567, output_fired_time: 0.0000000000, propagate_forward_time: 117.9343261719, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5121612549
validation[          , genome  6196] predictions:    6353/  10000 (63.53%), best:    6353/10000 (63.53%), error:     10679.07031, best error:     10679.07031 on epoch:     4, epoch:    4/25, mu: 0.6121621132, learning_rate: 0.0002175368, weight_decay: 0.0001743829

epoch time: 1525.0715332031s, input_fired_time: 24.2978153229, output_fired_time: 11.9399127960, propagate_forward_time: 582.0490722656, propagate_backward_time: 892.5413818359, weight_update_time: 0.4501100183, other_time: 14.2432861328
epoch time: 121.1962051392s, input_fired_time: 1.9023922682, output_fired_time: 0.0000000000, propagate_forward_time: 116.7894592285, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5043563843
validation[          , genome  6196] predictions:    6188/  10000 (61.88%), best:    6353/10000 (63.53%), error:     11139.26074, best error:     10679.07031 on epoch:     4, epoch:    5/25, mu: 0.6365290284, learning_rate: 0.0002023247, weight_decay: 0.0001642533

epoch time: 1524.5198974609s, input_fired_time: 24.2654857635, output_fired_time: 11.9578876495, propagate_forward_time: 581.7598266602, propagate_backward_time: 892.1496582031, weight_update_time: 0.4576871395, other_time: 14.3870239258
epoch time: 121.6206359863s, input_fired_time: 1.8933677673, output_fired_time: 0.0000000000, propagate_forward_time: 117.2110366821, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5162277222
validation[          , genome  6196] predictions:    6109/  10000 (61.09%), best:    6353/10000 (63.53%), error:     11500.56641, best error:     10679.07031 on epoch:     4, epoch:    6/25, mu: 0.6593245268, learning_rate: 0.0001881763, weight_decay: 0.0001547120

epoch time: 1523.1188964844s, input_fired_time: 24.2959213257, output_fired_time: 11.9499464035, propagate_forward_time: 581.0480957031, propagate_backward_time: 891.5073852539, weight_update_time: 0.4520313144, other_time: 14.3175659180
epoch time: 122.2353897095s, input_fired_time: 1.9070734978, output_fired_time: 0.0000000000, propagate_forward_time: 117.8563385010, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4719772339
validation[          , genome  6196] predictions:    6090/  10000 (60.90%), best:    6353/10000 (63.53%), error:     11436.71387, best error:     10679.07031 on epoch:     4, epoch:    7/25, mu: 0.6806499362, learning_rate: 0.0001750173, weight_decay: 0.0001457251

epoch time: 1527.3652343750s, input_fired_time: 24.2958354950, output_fired_time: 11.9449577332, propagate_forward_time: 584.3750000000, propagate_backward_time: 892.4362792969, weight_update_time: 0.4588416815, other_time: 14.3132324219
epoch time: 122.3906173706s, input_fired_time: 1.8968997002, output_fired_time: 0.0000000000, propagate_forward_time: 117.9960250854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4976959229
validation[          , genome  6196] predictions:    6227/  10000 (62.27%), best:    6353/10000 (63.53%), error:     10965.23047, best error:     10679.07031 on epoch:     4, epoch:    8/25, mu: 0.7006000280, learning_rate: 0.0001627785, weight_decay: 0.0001372601

epoch time: 1529.6641845703s, input_fired_time: 24.2984123230, output_fired_time: 11.9537248611, propagate_forward_time: 584.7680664063, propagate_backward_time: 894.4592285156, weight_update_time: 0.4519015849, other_time: 14.1846923828
epoch time: 122.0753784180s, input_fired_time: 1.9008150101, output_fired_time: 0.0000000000, propagate_forward_time: 117.6823577881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4922027588
validation[          , genome  6196] predictions:    6238/  10000 (62.38%), best:    6353/10000 (63.53%), error:     10982.45508, best error:     10679.07031 on epoch:     4, epoch:    9/25, mu: 0.7192635536, learning_rate: 0.0001513956, weight_decay: 0.0001292869

epoch time: 1526.7072753906s, input_fired_time: 24.2430076599, output_fired_time: 11.9575443268, propagate_forward_time: 583.0843505859, propagate_backward_time: 893.1721801758, weight_update_time: 0.4562590420, other_time: 14.2501831055
epoch time: 122.1996994019s, input_fired_time: 1.9063358307, output_fired_time: 0.0000000000, propagate_forward_time: 117.7931060791, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5002593994
validation[          , genome  6196] predictions:    6180/  10000 (61.80%), best:    6353/10000 (63.53%), error:     11109.42969, best error:     10679.07031 on epoch:     4, epoch:   10/25, mu: 0.7367234826, learning_rate: 0.0001408086, weight_decay: 0.0001217768

epoch time: 1525.4382324219s, input_fired_time: 24.2917098999, output_fired_time: 12.0481834412, propagate_forward_time: 582.9977416992, propagate_backward_time: 891.8078002930, weight_update_time: 0.4587601423, other_time: 14.2927246094
epoch time: 121.6311569214s, input_fired_time: 1.8855670691, output_fired_time: 0.0000000000, propagate_forward_time: 117.2369842529, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5086059570
validation[          , genome  6196] predictions:    6282/  10000 (62.82%), best:    6353/10000 (63.53%), error:     10817.43066, best error:     10679.07031 on epoch:     4, epoch:   11/25, mu: 0.7530574203, learning_rate: 0.0001309620, weight_decay: 0.0001147030

epoch time: 1524.7902832031s, input_fired_time: 24.2428321838, output_fired_time: 11.9408245087, propagate_forward_time: 582.7661132813, propagate_backward_time: 891.4521484375, weight_update_time: 0.4530838430, other_time: 14.3884277344
epoch time: 122.1678314209s, input_fired_time: 1.8937861919, output_fired_time: 0.0000000000, propagate_forward_time: 117.7614059448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5126419067
validation[          , genome  6196] predictions:    6279/  10000 (62.79%), best:    6353/10000 (63.53%), error:     10936.70215, best error:     10679.07031 on epoch:     4, epoch:   12/25, mu: 0.7683379650, learning_rate: 0.0001218040, weight_decay: 0.0001080401

epoch time: 1521.9552001953s, input_fired_time: 24.2338504791, output_fired_time: 11.9618911743, propagate_forward_time: 581.5004272461, propagate_backward_time: 889.9288940430, weight_update_time: 0.4567381144, other_time: 14.3300781250
epoch time: 121.4919281006s, input_fired_time: 1.8888576031, output_fired_time: 0.0000000000, propagate_forward_time: 117.0956954956, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5073776245
validation[          , genome  6196] predictions:    6275/  10000 (62.75%), best:    6353/10000 (63.53%), error:     10809.91797, best error:     10679.07031 on epoch:     4, epoch:   13/25, mu: 0.7826330662, learning_rate: 0.0001132863, weight_decay: 0.0001017642

epoch time: 1518.8232421875s, input_fired_time: 24.1924686432, output_fired_time: 11.9236307144, propagate_forward_time: 580.2731323242, propagate_backward_time: 888.0911254883, weight_update_time: 0.4570416212, other_time: 14.3428955078
epoch time: 121.3316192627s, input_fired_time: 1.8809429407, output_fired_time: 0.0000000000, propagate_forward_time: 116.9449081421, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5057678223
validation[          , genome  6196] predictions:    6313/  10000 (63.13%), best:    6353/10000 (63.53%), error:     10692.34961, best error:     10679.07031 on epoch:     4, epoch:   14/25, mu: 0.7960062623, learning_rate: 0.0001053643, weight_decay: 0.0000958529

epoch time: 1519.9150390625s, input_fired_time: 24.2370491028, output_fired_time: 11.9404544830, propagate_forward_time: 581.2276611328, propagate_backward_time: 888.0955200195, weight_update_time: 0.4639677107, other_time: 14.4143676758
epoch time: 121.4074401855s, input_fired_time: 1.8896303177, output_fired_time: 0.0000000000, propagate_forward_time: 117.0124893188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5053176880
validation[          , genome  6196] predictions:    6095/  10000 (60.95%), best:    6353/10000 (63.53%), error:     11236.52930, best error:     10679.07031 on epoch:     4, epoch:   15/25, mu: 0.8085169792, learning_rate: 0.0000979963, weight_decay: 0.0000902850

epoch time: 1517.6884765625s, input_fired_time: 24.2299365997, output_fired_time: 11.9547700882, propagate_forward_time: 580.0948486328, propagate_backward_time: 887.0408325195, weight_update_time: 0.4579551518, other_time: 14.3681030273
epoch time: 121.2577209473s, input_fired_time: 1.8981953859, output_fired_time: 0.0000000000, propagate_forward_time: 116.8581390381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5013885498
validation[          , genome  6196] predictions:    6128/  10000 (61.28%), best:    6353/10000 (63.53%), error:     11094.92383, best error:     10679.07031 on epoch:     4, epoch:   16/25, mu: 0.8202208877, learning_rate: 0.0000911435, weight_decay: 0.0000850404

epoch time: 1520.8969726563s, input_fired_time: 24.2414016724, output_fired_time: 11.9548988342, propagate_forward_time: 581.3800048828, propagate_backward_time: 888.9995117188, weight_update_time: 0.4505224526, other_time: 14.3210449219
epoch time: 122.0439376831s, input_fired_time: 1.8920087814, output_fired_time: 0.0000000000, propagate_forward_time: 117.6658706665, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4860610962
validation[          , genome  6196] predictions:    6209/  10000 (62.09%), best:    6353/10000 (63.53%), error:     11291.57715, best error:     10679.07031 on epoch:     4, epoch:   17/25, mu: 0.8311700225, learning_rate: 0.0000847699, weight_decay: 0.0000801006

epoch time: 1527.0594482422s, input_fired_time: 24.3135986328, output_fired_time: 11.9822015762, propagate_forward_time: 583.7520751953, propagate_backward_time: 892.6846313477, weight_update_time: 0.4569007456, other_time: 14.3269653320
epoch time: 122.0519866943s, input_fired_time: 1.8936787844, output_fired_time: 0.0000000000, propagate_forward_time: 117.6642913818, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4940185547
validation[          , genome  6196] predictions:    6225/  10000 (62.25%), best:    6353/10000 (63.53%), error:     10947.20996, best error:     10679.07031 on epoch:     4, epoch:   18/25, mu: 0.8414130211, learning_rate: 0.0000788420, weight_decay: 0.0000754477

epoch time: 1528.9257812500s, input_fired_time: 24.2993412018, output_fired_time: 11.9870481491, propagate_forward_time: 584.3350830078, propagate_backward_time: 894.1046752930, weight_update_time: 0.4526137710, other_time: 14.1996459961
epoch time: 122.1343612671s, input_fired_time: 1.9066674709, output_fired_time: 0.0000000000, propagate_forward_time: 117.7556381226, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4720535278
validation[          , genome  6196] predictions:    6312/  10000 (63.12%), best:    6353/10000 (63.53%), error:     10719.83105, best error:     10679.07031 on epoch:     4, epoch:   19/25, mu: 0.8509954810, learning_rate: 0.0000733287, weight_decay: 0.0000710650

epoch time: 1525.2478027344s, input_fired_time: 24.3144016266, output_fired_time: 11.9974279404, propagate_forward_time: 583.3297119141, propagate_backward_time: 891.3279418945, weight_update_time: 0.4539683759, other_time: 14.2782592773
epoch time: 121.8404998779s, input_fired_time: 1.9089083672, output_fired_time: 0.0000000000, propagate_forward_time: 117.4513626099, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4802322388
validation[          , genome  6196] predictions:    6375/  10000 (63.75%), best:    6375/10000 (63.75%), error:     10478.47363, best error:     10478.47363 on epoch:    20, epoch:   20/25, mu: 0.8599599600, learning_rate: 0.0000682009, weight_decay: 0.0000669370

epoch time: 1525.1458740234s, input_fired_time: 24.3025245667, output_fired_time: 11.9816055298, propagate_forward_time: 583.5361938477, propagate_backward_time: 891.0607299805, weight_update_time: 0.4605438411, other_time: 14.2648925781
epoch time: 120.8221511841s, input_fired_time: 1.8826543093, output_fired_time: 0.0000000000, propagate_forward_time: 116.4623107910, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4771881104
validation[          , genome  6196] predictions:    6340/  10000 (63.40%), best:    6375/10000 (63.75%), error:     10644.91113, best error:     10478.47363 on epoch:    20, epoch:   21/25, mu: 0.8683462739, learning_rate: 0.0000634317, weight_decay: 0.0000630487

epoch time: 1523.5932617188s, input_fired_time: 24.2741127014, output_fired_time: 11.9483556747, propagate_forward_time: 582.2034301758, propagate_backward_time: 890.8641967773, weight_update_time: 0.4562812150, other_time: 14.3031005859
epoch time: 121.5025558472s, input_fired_time: 1.8965879679, output_fired_time: 0.0000000000, propagate_forward_time: 117.1168365479, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4891281128
validation[          , genome  6196] predictions:    6360/  10000 (63.60%), best:    6375/10000 (63.75%), error:     10594.04590, best error:     10478.47363 on epoch:    20, epoch:   22/25, mu: 0.8761917949, learning_rate: 0.0000589959, weight_decay: 0.0000593863

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_172_6196.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_172_6196_1_r1249575589_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.496698
read mu: 0.883531
read mu_delta: 0.93551
read initial_learning_rate: 0.000290716
read learning_rate: 5.48704e-05
read learning_rate_delta: 0.930071
read initial_weight_decay: 0.000221546
read weight_decay: 5.59366e-05
read weight_decay_delta: 0.941912
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.092828
read input_dropout_probability: 0.00134115
read hidden_dropout_probability: 0.0827877
read velocity_reset: 1876
read epoch: 23
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 20
read number_validation_images: 10000
read best_validation_predictions: 6375
read best_validation_error: 10478.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6196
read normal distribution: '1 0.0908117964863777 -1.11697173118591'
generator_str: '509949254'
read generator: 509949254
read generated_by_map:
1 add_edge 1
reading 105 nodes.
reading 415 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1529.52307128906s, input_fired_time: 24.5037040710449, output_fired_time: 12.0731315612793, propagate_forward_time: 588.072143554688, propagate_backward_time: 890.54296875, weight_update_time: 0.461038500070572, other_time: 14.3311767578125
epoch time: 122.643424987793s, input_fired_time: 1.89799094200134, output_fired_time: 0, propagate_forward_time: 118.261222839355, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.48421478271484
validation[          , genome  6196] predictions:    6443/  10000 (64.43%), best:    6443/10000 (64.43%), error:     10414.18066, best error:     10414.18066 on epoch:    23, epoch:   23/25, mu: 0.8835313320, learning_rate: 0.0000548704, weight_decay: 0.0000559366

epoch time: 1526.9008789063s, input_fired_time: 24.4698257446, output_fired_time: 12.0816602707, propagate_forward_time: 586.3215332031, propagate_backward_time: 889.6343994141, weight_update_time: 0.4644911885, other_time: 14.3934326172
epoch time: 122.8175888062s, input_fired_time: 1.9115343094, output_fired_time: 0.0000000000, propagate_forward_time: 118.4193344116, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4867172241
validation[          , genome  6196] predictions:    6372/  10000 (63.72%), best:    6443/10000 (64.43%), error:     10451.48242, best error:     10414.18066 on epoch:    23, epoch:   24/25, mu: 0.8903975487, learning_rate: 0.0000510334, weight_decay: 0.0000526874

epoch time: 1529.4165039063s, input_fired_time: 24.3898620605, output_fired_time: 12.0293216705, propagate_forward_time: 586.0579833984, propagate_backward_time: 892.6108398438, weight_update_time: 0.4633249044, other_time: 14.3284912109
epoch time: 127.0634078979s, input_fired_time: 1.9544149637, output_fired_time: 0.0000000000, propagate_forward_time: 122.5720062256, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5369873047
validation[          , genome  6196] predictions:    6512/  10000 (65.12%), best:    6512/10000 (65.12%), error:     10308.89453, best error:     10308.89453 on epoch:    25, epoch:   25/25, mu: 0.8968209624, learning_rate: 0.0000474647, weight_decay: 0.0000496269

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 604.3012695313s, input_fired_time: 9.4906682968, output_fired_time: 0.0000000000, propagate_forward_time: 582.3569335938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.4536743164
best training[          , genome  6196] predictions:   34246/  50000 (68.49%), best:    6512/10000 (65.12%), error:     46988.83594, best error:     10308.89453 on epoch:    25, epoch:   26/25, mu: 0.9028301239, learning_rate: 0.0000441455, weight_decay: 0.0000467441
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 122.0276794434s, input_fired_time: 1.9144426584, output_fired_time: 0.0000000000, propagate_forward_time: 117.6154937744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4977416992
   testing[          , genome  6196] predictions:    6512/  10000 (65.12%), best:    6512/10000 (65.12%), error:     10308.89453, best error:     10308.89453 on epoch:    25, epoch:   26/25, mu: 0.9028301239, learning_rate: 0.0000441455, weight_decay: 0.0000467441
backpropagation finished successfully!
05:11:38 (20016): called boinc_finish(0)

</stderr_txt>
]]>