Nameexact_genome_1542495604_171_6197_1
Workunit4095964
Created18 Nov 2018, 3:04:32 UTC
Sent18 Nov 2018, 3:17:03 UTC
Report deadline21 Dec 2018, 2:33:33 UTC
Received19 Nov 2018, 13:15:40 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60920
Run time14 hours 38 min 4 sec
CPU time14 hours 33 min 10 sec
Validate stateValid
Credit8,242.45
Device peak FLOPS3.88 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size579.23 MB
Peak swap size577.62 MB
Peak disk usage13.87 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197_1_r2108479300_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.47116
read mu: 0.47116
read mu_delta: 0.962087
read initial_learning_rate: 0.000293997
read learning_rate: 0.000293997
read learning_rate_delta: 0.927304
read initial_weight_decay: 0.000252887
read weight_decay: 0.000252887
read weight_decay_delta: 0.929412
read batch_size: 116
read epsilon: 1e-07
read alpha: 0.0966498
read input_dropout_probability: 0.00135109
read hidden_dropout_probability: 0.117905
read velocity_reset: 1553
read epoch: 0
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6197
read normal distribution: '1 0 0'
generator_str: '1303744274'
read generator: 1303744274
read generated_by_map:
1 disable_edge 1
reading 116 nodes.
reading 455 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1254362777
post shuffle 1: 1063082626
epoch time: 1853.21923828125s, input_fired_time: 24.1272506713867, output_fired_time: 11.5233612060547, propagate_forward_time: 716.880676269531, propagate_backward_time: 1086.66748046875, weight_update_time: 0.452173382043839, other_time: 14.0205078125
epoch time: 149.037857055664s, input_fired_time: 1.82885324954987, output_fired_time: 0, propagate_forward_time: 144.75553894043, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.45346069335938
validation[          , genome  6197] predictions:    5802/  10000 (58.02%), best:    5802/10000 (58.02%), error:     12406.80859, best error:     12406.80859 on epoch:     0, epoch:    0/25, mu: 0.4711603522, learning_rate: 0.0002939974, weight_decay: 0.0002528868

epoch time: 1860.4278564453s, input_fired_time: 24.1553096771, output_fired_time: 11.5877408981, propagate_forward_time: 720.7290039063, propagate_backward_time: 1089.8398437500, weight_update_time: 0.4583381414, other_time: 14.1159667969
epoch time: 149.4046020508s, input_fired_time: 1.8197106123, output_fired_time: 0.0000000000, propagate_forward_time: 145.1447448730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4401397705
validation[          , genome  6197] predictions:    6084/  10000 (60.84%), best:    6084/10000 (60.84%), error:     11589.84473, best error:     11589.84473 on epoch:     1, epoch:    1/25, mu: 0.4908312559, learning_rate: 0.0002726249, weight_decay: 0.0002350360

epoch time: 1858.5356445313s, input_fired_time: 24.1697998047, output_fired_time: 11.5566082001, propagate_forward_time: 719.3803710938, propagate_backward_time: 1089.3316650391, weight_update_time: 0.4581909180, other_time: 14.0971679688
epoch time: 148.6094360352s, input_fired_time: 1.8327521086, output_fired_time: 0.0000000000, propagate_forward_time: 144.3058471680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4708404541
validation[          , genome  6197] predictions:    6312/  10000 (63.12%), best:    6312/10000 (63.12%), error:     10879.02246, best error:     10879.02246 on epoch:     2, epoch:    2/25, mu: 0.5097563863, learning_rate: 0.0002528062, weight_decay: 0.0002184453

epoch time: 1851.8522949219s, input_fired_time: 24.0168132782, output_fired_time: 11.4989728928, propagate_forward_time: 714.7052612305, propagate_backward_time: 1087.5069580078, weight_update_time: 0.4570935965, other_time: 14.1241455078
epoch time: 147.4259185791s, input_fired_time: 1.8128701448, output_fired_time: 0.0000000000, propagate_forward_time: 143.1343231201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4787292480
validation[          , genome  6197] predictions:    6030/  10000 (60.30%), best:    6312/10000 (63.12%), error:     11434.27539, best error:     10879.02246 on epoch:     2, epoch:    3/25, mu: 0.5279639959, learning_rate: 0.0002344282, weight_decay: 0.0002030257

epoch time: 1841.2957763672s, input_fired_time: 23.9589900970, output_fired_time: 11.3839550018, propagate_forward_time: 708.3959350586, propagate_backward_time: 1083.4156494141, weight_update_time: 0.4499080181, other_time: 14.1412353516
epoch time: 147.2627716064s, input_fired_time: 1.8112607002, output_fired_time: 0.0000000000, propagate_forward_time: 142.9725799561, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4789276123
validation[          , genome  6197] predictions:    6086/  10000 (60.86%), best:    6312/10000 (63.12%), error:     11183.91113, best error:     10879.02246 on epoch:     2, epoch:    4/25, mu: 0.5454812646, learning_rate: 0.0002173862, weight_decay: 0.0001886946

epoch time: 1838.5688476563s, input_fired_time: 23.9301834106, output_fired_time: 11.4126691818, propagate_forward_time: 707.4887695313, propagate_backward_time: 1081.5179443359, weight_update_time: 0.4447946548, other_time: 14.2192382813
epoch time: 147.0135192871s, input_fired_time: 1.8125615120, output_fired_time: 0.0000000000, propagate_forward_time: 142.7298889160, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4710693359
validation[          , genome  6197] predictions:    6215/  10000 (62.15%), best:    6312/10000 (63.12%), error:     10991.25879, best error:     10879.02246 on epoch:     2, epoch:    5/25, mu: 0.5623344183, learning_rate: 0.0002015831, weight_decay: 0.0001753750

epoch time: 1841.4844970703s, input_fired_time: 23.9327316284, output_fired_time: 11.4235715866, propagate_forward_time: 709.4338989258, propagate_backward_time: 1082.5397949219, weight_update_time: 0.4482257962, other_time: 14.1545410156
epoch time: 147.4430999756s, input_fired_time: 1.8027578592, output_fired_time: 0.0000000000, propagate_forward_time: 143.1837768555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4565582275
validation[          , genome  6197] predictions:    5999/  10000 (59.99%), best:    6312/10000 (63.12%), error:     11741.87988, best error:     10879.02246 on epoch:     2, epoch:    6/25, mu: 0.5785486102, learning_rate: 0.0001869288, weight_decay: 0.0001629957

epoch time: 1845.4504394531s, input_fired_time: 23.9387836456, output_fired_time: 11.4259519577, propagate_forward_time: 711.4057617188, propagate_backward_time: 1084.5939941406, weight_update_time: 0.4420094192, other_time: 14.0859375000
epoch time: 147.7224731445s, input_fired_time: 1.8143858910, output_fired_time: 0.0000000000, propagate_forward_time: 143.4391479492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4689331055
validation[          , genome  6197] predictions:    6248/  10000 (62.48%), best:    6312/10000 (63.12%), error:     10998.08789, best error:     10879.02246 on epoch:     2, epoch:    7/25, mu: 0.5941480994, learning_rate: 0.0001733398, weight_decay: 0.0001514902

epoch time: 1839.6810302734s, input_fired_time: 23.8862628937, output_fired_time: 11.3761224747, propagate_forward_time: 709.1532592773, propagate_backward_time: 1081.0737304688, weight_update_time: 0.4456882477, other_time: 14.1916503906
epoch time: 147.1544799805s, input_fired_time: 1.7927453518, output_fired_time: 0.0000000000, propagate_forward_time: 142.8314819336, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5302581787
validation[          , genome  6197] predictions:    6061/  10000 (60.61%), best:    6312/10000 (63.12%), error:     11554.96387, best error:     10879.02246 on epoch:     2, epoch:    8/25, mu: 0.6091561317, learning_rate: 0.0001607387, weight_decay: 0.0001407968

epoch time: 1840.4931640625s, input_fired_time: 23.9264717102, output_fired_time: 11.3915843964, propagate_forward_time: 709.3429565430, propagate_backward_time: 1081.6134033203, weight_update_time: 0.4500632286, other_time: 14.2186279297
epoch time: 147.9363708496s, input_fired_time: 1.8055438995, output_fired_time: 0.0000000000, propagate_forward_time: 143.6472778320, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4835510254
validation[          , genome  6197] predictions:    6119/  10000 (61.19%), best:    6312/10000 (63.12%), error:     11240.30664, best error:     10879.02246 on epoch:     2, epoch:    9/25, mu: 0.6235951781, learning_rate: 0.0001490536, weight_decay: 0.0001308582

epoch time: 1846.3687744141s, input_fired_time: 23.9622383118, output_fired_time: 11.4463262558, propagate_forward_time: 712.1156616211, propagate_backward_time: 1084.7178955078, weight_update_time: 0.4503478706, other_time: 14.1265869141
epoch time: 147.9814758301s, input_fired_time: 1.8157159090, output_fired_time: 0.0000000000, propagate_forward_time: 143.6909179688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4748382568
validation[          , genome  6197] predictions:    6275/  10000 (62.75%), best:    6312/10000 (63.12%), error:     10991.89551, best error:     10879.02246 on epoch:     2, epoch:   10/25, mu: 0.6374867558, learning_rate: 0.0001382180, weight_decay: 0.0001216212

epoch time: 1847.3366699219s, input_fired_time: 23.9579601288, output_fired_time: 11.4526557922, propagate_forward_time: 712.2855834961, propagate_backward_time: 1085.5867919922, weight_update_time: 0.4477357268, other_time: 14.0535888672
epoch time: 147.5092163086s, input_fired_time: 1.7866861820, output_fired_time: 0.0000000000, propagate_forward_time: 143.2671661377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4553680420
validation[          , genome  6197] predictions:    5906/  10000 (59.06%), best:    6312/10000 (63.12%), error:     11856.62793, best error:     10879.02246 on epoch:     2, epoch:   11/25, mu: 0.6508516669, learning_rate: 0.0001281701, weight_decay: 0.0001130363

epoch time: 1842.9044189453s, input_fired_time: 23.9016628265, output_fired_time: 11.4994096756, propagate_forward_time: 710.6350708008, propagate_backward_time: 1082.6174316406, weight_update_time: 0.4494377971, other_time: 14.2509765625
epoch time: 147.7015838623s, input_fired_time: 1.7930291891, output_fired_time: 0.0000000000, propagate_forward_time: 143.4277191162, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4808349609
validation[          , genome  6197] predictions:    5903/  10000 (59.03%), best:    6312/10000 (63.12%), error:     11910.67676, best error:     10879.02246 on epoch:     2, epoch:   12/25, mu: 0.6637098789, learning_rate: 0.0001188527, weight_decay: 0.0001050573

epoch time: 1841.6618652344s, input_fired_time: 23.9718990326, output_fired_time: 11.4403028488, propagate_forward_time: 710.3649291992, propagate_backward_time: 1081.6862792969, weight_update_time: 0.4537898302, other_time: 14.1984863281
epoch time: 147.3149414063s, input_fired_time: 1.7978066206, output_fired_time: 0.0000000000, propagate_forward_time: 143.0336303711, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4835052490
validation[          , genome  6197] predictions:    6198/  10000 (61.98%), best:    6312/10000 (63.12%), error:     11044.54297, best error:     10879.02246 on epoch:     2, epoch:   13/25, mu: 0.6760805845, learning_rate: 0.0001102125, weight_decay: 0.0000976415

epoch time: 1837.8575439453s, input_fired_time: 23.8907279968, output_fired_time: 11.4408473969, propagate_forward_time: 708.8213500977, propagate_backward_time: 1079.4338378906, weight_update_time: 0.4499467313, other_time: 14.2707519531
epoch time: 146.8724365234s, input_fired_time: 1.7918438911, output_fired_time: 0.0000000000, propagate_forward_time: 142.5834350586, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4971618652
validation[          , genome  6197] predictions:    6027/  10000 (60.27%), best:    6312/10000 (63.12%), error:     11487.84961, best error:     10879.02246 on epoch:     2, epoch:   14/25, mu: 0.6879823208, learning_rate: 0.0001022005, weight_decay: 0.0000907492

epoch time: 1834.2690429688s, input_fired_time: 23.8961372375, output_fired_time: 11.4234828949, propagate_forward_time: 707.4183959961, propagate_backward_time: 1077.3187255859, weight_update_time: 0.4479244351, other_time: 14.2122802734
epoch time: 147.0030670166s, input_fired_time: 1.7888280153, output_fired_time: 0.0000000000, propagate_forward_time: 142.7482147217, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4660186768
validation[          , genome  6197] predictions:    5849/  10000 (58.49%), best:    6312/10000 (63.12%), error:     12221.60840, best error:     10879.02246 on epoch:     2, epoch:   15/25, mu: 0.6994327903, learning_rate: 0.0000947709, weight_decay: 0.0000843434

epoch time: 1834.7536621094s, input_fired_time: 23.9034786224, output_fired_time: 11.4717979431, propagate_forward_time: 707.8645629883, propagate_backward_time: 1077.2612304688, weight_update_time: 0.4512255788, other_time: 14.2526855469
epoch time: 146.7038116455s, input_fired_time: 1.7890180349, output_fired_time: 0.0000000000, propagate_forward_time: 142.4376068115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4771881104
validation[          , genome  6197] predictions:    6168/  10000 (61.68%), best:    6312/10000 (63.12%), error:     11217.56250, best error:     10879.02246 on epoch:     2, epoch:   16/25, mu: 0.7104491591, learning_rate: 0.0000878815, weight_decay: 0.0000783898

epoch time: 1835.4138183594s, input_fired_time: 23.9068355560, output_fired_time: 11.4518070221, propagate_forward_time: 708.2871704102, propagate_backward_time: 1077.5571289063, weight_update_time: 0.4527753890, other_time: 14.2109375000
epoch time: 147.2347259521s, input_fired_time: 1.8188090324, output_fired_time: 0.0000000000, propagate_forward_time: 142.9451599121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4707641602
validation[          , genome  6197] predictions:    6349/  10000 (63.49%), best:    6349/10000 (63.49%), error:     10847.44824, best error:     10847.44824 on epoch:    17, epoch:   17/25, mu: 0.7210478187, learning_rate: 0.0000814928, weight_decay: 0.0000728564

epoch time: 1844.5576171875s, input_fired_time: 23.9880104065, output_fired_time: 11.5035772324, propagate_forward_time: 710.8702392578, propagate_backward_time: 1084.0104980469, weight_update_time: 0.4525075555, other_time: 14.1853027344
epoch time: 147.4995422363s, input_fired_time: 1.8091737032, output_fired_time: 0.0000000000, propagate_forward_time: 143.2070465088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4833221436
validation[          , genome  6197] predictions:    6208/  10000 (62.08%), best:    6349/10000 (63.49%), error:     11090.59277, best error:     10847.44824 on epoch:    17, epoch:   18/25, mu: 0.7312446833, learning_rate: 0.0000755686, weight_decay: 0.0000677136

epoch time: 1846.7861328125s, input_fired_time: 23.9720840454, output_fired_time: 11.4235353470, propagate_forward_time: 713.0670166016, propagate_backward_time: 1084.2573242188, weight_update_time: 0.4518590868, other_time: 14.0661621094
epoch time: 147.3526000977s, input_fired_time: 1.8050646782, output_fired_time: 0.0000000000, propagate_forward_time: 143.0863189697, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4612121582
validation[          , genome  6197] predictions:    6141/  10000 (61.41%), best:    6349/10000 (63.49%), error:     11292.73828, best error:     10847.44824 on epoch:    17, epoch:   19/25, mu: 0.7410549521, learning_rate: 0.0000700751, weight_decay: 0.0000629339

epoch time: 1841.6529541016s, input_fired_time: 23.9639854431, output_fired_time: 11.5031747818, propagate_forward_time: 711.5879516602, propagate_backward_time: 1080.4919433594, weight_update_time: 0.4448707998, other_time: 14.1059570313
epoch time: 147.5871582031s, input_fired_time: 1.8055787086, output_fired_time: 0.0000000000, propagate_forward_time: 143.3146057129, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4669799805
validation[          , genome  6197] predictions:    5990/  10000 (59.90%), best:    6349/10000 (63.49%), error:     11721.83008, best error:     10847.44824 on epoch:    17, epoch:   20/25, mu: 0.7504932880, learning_rate: 0.0000649809, weight_decay: 0.0000584915

epoch time: 1840.7465820313s, input_fired_time: 23.9619655609, output_fired_time: 11.4568777084, propagate_forward_time: 710.8558349609, propagate_backward_time: 1080.3156738281, weight_update_time: 0.4492464960, other_time: 14.1562500000
epoch time: 147.5254516602s, input_fired_time: 1.8095798492, output_fired_time: 0.0000000000, propagate_forward_time: 143.2443389893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4715270996
validation[          , genome  6197] predictions:    6215/  10000 (62.15%), best:    6349/10000 (63.49%), error:     11112.40039, best error:     10847.44824 on epoch:    17, epoch:   21/25, mu: 0.7595737576, learning_rate: 0.0000602571, weight_decay: 0.0000543627

epoch time: 1838.7147216797s, input_fired_time: 23.9223289490, output_fired_time: 11.4438409805, propagate_forward_time: 709.2994995117, propagate_backward_time: 1079.8819580078, weight_update_time: 0.4442212284, other_time: 14.1671142578
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197_1_r2108479300_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.47116
read mu: 0.76831
read mu_delta: 0.962087
read initial_learning_rate: 0.000293997
read learning_rate: 5.58766e-05
read learning_rate_delta: 0.927304
read initial_weight_decay: 0.000252887
read weight_decay: 5.05254e-05
read weight_decay_delta: 0.929412
read batch_size: 116
read epsilon: 1e-07
read alpha: 0.0966498
read input_dropout_probability: 0.00135109
read hidden_dropout_probability: 0.117905
read velocity_reset: 1553
read epoch: 22
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 17
read number_validation_images: 10000
read best_validation_predictions: 6349
read best_validation_error: 10847.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6197
read normal distribution: '1 0 0'
generator_str: '1015417613'
read generator: 1015417613
read generated_by_map:
1 disable_edge 1
reading 116 nodes.
reading 455 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1847.77355957031s, input_fired_time: 24.0742530822754, output_fired_time: 11.5898199081421, propagate_forward_time: 717.553161621094, propagate_backward_time: 1080.31494140625, weight_update_time: 0.454640209674835, other_time: 14.241455078125
epoch time: 148.56071472168s, input_fired_time: 1.82445502281189, output_fired_time: 0, propagate_forward_time: 144.271255493164, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.46501159667969
validation[          , genome  6197] predictions:    6286/  10000 (62.86%), best:    6286/10000 (62.86%), error:     10809.20703, best error:     10809.20703 on epoch:    22, epoch:   22/25, mu: 0.7683099508, learning_rate: 0.0000558766, weight_decay: 0.0000505254

epoch time: 1845.4228515625s, input_fired_time: 24.0321826935, output_fired_time: 11.5616207123, propagate_forward_time: 714.7002563477, propagate_backward_time: 1080.9455566406, weight_update_time: 0.4591139853, other_time: 14.1831054688
epoch time: 148.7080688477s, input_fired_time: 1.8219450712, output_fired_time: 0.0000000000, propagate_forward_time: 144.4220733643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4640502930
validation[          , genome  6197] predictions:    6217/  10000 (62.17%), best:    6286/10000 (62.86%), error:     11048.86719, best error:     10809.20703 on epoch:    22, epoch:   23/25, mu: 0.7767149210, learning_rate: 0.0000518146, weight_decay: 0.0000469589

epoch time: 1850.7456054688s, input_fired_time: 24.0741596222, output_fired_time: 11.5553350449, propagate_forward_time: 715.4655151367, propagate_backward_time: 1085.5433349609, weight_update_time: 0.4537972510, other_time: 14.1072998047
epoch time: 147.3920135498s, input_fired_time: 1.8215888739, output_fired_time: 0.0000000000, propagate_forward_time: 143.1314544678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4389648438
validation[          , genome  6197] predictions:    6346/  10000 (63.46%), best:    6346/10000 (63.46%), error:     10658.08203, best error:     10658.08203 on epoch:    24, epoch:   24/25, mu: 0.7848012447, learning_rate: 0.0000480479, weight_decay: 0.0000436442

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197_1_r2108479300_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.47116
read mu: 0.792581
read mu_delta: 0.962087
read initial_learning_rate: 0.000293997
read learning_rate: 4.4555e-05
read learning_rate_delta: 0.927304
read initial_weight_decay: 0.000252887
read weight_decay: 4.05634e-05
read weight_decay_delta: 0.929412
read batch_size: 116
read epsilon: 1e-07
read alpha: 0.0966498
read input_dropout_probability: 0.00135109
read hidden_dropout_probability: 0.117905
read velocity_reset: 1553
read epoch: 25
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 24
read number_validation_images: 10000
read best_validation_predictions: 6346
read best_validation_error: 10658.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6197
read normal distribution: '1 0 0'
generator_str: '2067944469'
read generator: 2067944469
read generated_by_map:
1 disable_edge 1
reading 116 nodes.
reading 455 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542495604_171_6197_1_r2108479300_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.47116
read mu: 0.792581
read mu_delta: 0.962087
read initial_learning_rate: 0.000293997
read learning_rate: 4.4555e-05
read learning_rate_delta: 0.927304
read initial_weight_decay: 0.000252887
read weight_decay: 4.05634e-05
read weight_decay_delta: 0.929412
read batch_size: 116
read epsilon: 1e-07
read alpha: 0.0966498
read input_dropout_probability: 0.00135109
read hidden_dropout_probability: 0.117905
read velocity_reset: 1553
read epoch: 25
read max_epochs: 25
read reset_weights: 0
read padding: 2
read best_epoch: 24
read number_validation_images: 10000
read best_validation_predictions: 6346
read best_validation_error: 10658.1
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6197
read normal distribution: '1 0 0'
generator_str: '2067944469'
read generator: 2067944469
read generated_by_map:
1 disable_edge 1
reading 116 nodes.
reading 455 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1822.66003417969s, input_fired_time: 23.9816513061523, output_fired_time: 11.4628038406372, propagate_forward_time: 705.252197265625, propagate_backward_time: 1067.70056152344, weight_update_time: 0.456077098846436, other_time: 14.2628173828125
epoch time: 145.025314331055s, input_fired_time: 1.80155527591705, output_fired_time: 0, propagate_forward_time: 140.72380065918, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.49995422363281
validation[          , genome  6197] predictions:    6318/  10000 (63.18%), best:    6346/10000 (63.46%), error:     10774.31055, best error:     10658.08203 on epoch:    24, epoch:   25/25, mu: 0.7925810218, learning_rate: 0.0000445550, weight_decay: 0.0000405634

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 721.5401000977s, input_fired_time: 8.9873828888, output_fired_time: 0.0000000000, propagate_forward_time: 700.0042114258, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.5485229492
best training[          , genome  6197] predictions:   33876/  50000 (67.75%), best:    6346/10000 (63.46%), error:     48436.12500, best error:     10658.08203 on epoch:    24, epoch:   26/25, mu: 0.8000658154, learning_rate: 0.0000413160, weight_decay: 0.0000377001
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 145.2711334229s, input_fired_time: 1.8040146828, output_fired_time: 0.0000000000, propagate_forward_time: 140.9570465088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5100708008
   testing[          , genome  6197] predictions:    6346/  10000 (63.46%), best:    6346/10000 (63.46%), error:     10658.08203, best error:     10658.08203 on epoch:    24, epoch:   26/25, mu: 0.8000658154, learning_rate: 0.0000413160, weight_decay: 0.0000377001
backpropagation finished successfully!
06:15:29 (21424): called boinc_finish(0)

</stderr_txt>
]]>