Nameexact_genome_1542692703_179_6400_3
Workunit4100903
Created27 Nov 2018, 21:42:00 UTC
Sent27 Nov 2018, 21:42:22 UTC
Report deadline21 Jan 2019, 11:53:27 UTC
Received28 Nov 2018, 16:22:28 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID73073
Run time14 hours 53 min 51 sec
CPU time13 hours 8 min 56 sec
Validate stateValid
Credit13,647.76
Device peak FLOPS4.61 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size581.13 MB
Peak swap size601.04 MB
Peak disk usage14.75 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400_3_r1629005478_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.536456
read mu: 0.536456
read mu_delta: 0.954359
read initial_learning_rate: 0.000574317
read learning_rate: 0.000574317
read learning_rate_delta: 0.910044
read initial_weight_decay: 0.000455629
read weight_decay: 0.000455629
read weight_decay_delta: 0.933831
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.106298
read input_dropout_probability: 0.00125594
read hidden_dropout_probability: 0.100995
read velocity_reset: 1709
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6400
read normal distribution: '0 1.86046957969666 0.455630153417587'
generator_str: '82087135'
read generator: 82087135
read generated_by_map:
1 merge_node 1
reading 103 nodes.
reading 521 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 953976571
post shuffle 1: 264464516
epoch time: 952.0576171875s, input_fired_time: 17.7818508148193, output_fired_time: 6.01708507537842, propagate_forward_time: 354.605133056641, propagate_backward_time: 561.315002441406, weight_update_time: 0.327817559242249, other_time: 12.3385620117188
epoch time: 75.9217529296875s, input_fired_time: 1.07124173641205, output_fired_time: 0, propagate_forward_time: 72.870475769043, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.98003387451172
validation[          , genome  6400] predictions:    5599/  10000 (55.99%), best:    5599/10000 (55.99%), error:     12542.14063, best error:     12542.14063 on epoch:     0, epoch:    0/50, mu: 0.5364555717, learning_rate: 0.0005743168, weight_decay: 0.0004556293

epoch time: 967.4843750000s, input_fired_time: 20.1311397552, output_fired_time: 7.2899708748, propagate_forward_time: 356.4267578125, propagate_backward_time: 571.1701660156, weight_update_time: 0.3468888700, other_time: 12.4663085938
epoch time: 75.2047500610s, input_fired_time: 1.0987474918, output_fired_time: 0.0000000000, propagate_forward_time: 72.0778732300, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0281295776
validation[          , genome  6400] predictions:    6038/  10000 (60.38%), best:    6038/10000 (60.38%), error:     11798.21875, best error:     11798.21875 on epoch:     1, epoch:    1/50, mu: 0.5571556687, learning_rate: 0.0005226538, weight_decay: 0.0004254808

epoch time: 971.4711914063s, input_fired_time: 18.1028690338, output_fired_time: 7.1855077744, propagate_forward_time: 357.2225341797, propagate_backward_time: 576.4557495117, weight_update_time: 0.3579851091, other_time: 12.5045776367
epoch time: 76.1919937134s, input_fired_time: 1.0134956837, output_fired_time: 0.0000000000, propagate_forward_time: 72.1724166870, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0060806274
validation[          , genome  6400] predictions:    6193/  10000 (61.93%), best:    6193/10000 (61.93%), error:     11343.92676, best error:     11343.92676 on epoch:     2, epoch:    2/50, mu: 0.5769109726, learning_rate: 0.0004756382, weight_decay: 0.0003973272

epoch time: 969.6507568359s, input_fired_time: 18.3044776917, output_fired_time: 6.1680088043, propagate_forward_time: 363.9882812500, propagate_backward_time: 569.7347412109, weight_update_time: 0.3858308196, other_time: 11.4552001953
epoch time: 75.7274551392s, input_fired_time: 2.0553426743, output_fired_time: 0.0000000000, propagate_forward_time: 71.7039947510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9681167603
validation[          , genome  6400] predictions:    5734/  10000 (57.34%), best:    6193/10000 (61.93%), error:     12506.63574, best error:     11343.92676 on epoch:     2, epoch:    3/50, mu: 0.5957646370, learning_rate: 0.0004328519, weight_decay: 0.0003710364

epoch time: 969.0954589844s, input_fired_time: 17.3966159821, output_fired_time: 6.2221632004, propagate_forward_time: 354.5794677734, propagate_backward_time: 578.3842773438, weight_update_time: 0.4031764269, other_time: 12.5129394531
epoch time: 73.6097564697s, input_fired_time: 0.9631699324, output_fired_time: 0.0000000000, propagate_forward_time: 70.7604675293, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8861160278
validation[          , genome  6400] predictions:    6039/  10000 (60.39%), best:    6193/10000 (61.93%), error:     11539.37598, best error:     11343.92676 on epoch:     2, epoch:    4/50, mu: 0.6137577891, learning_rate: 0.0003939145, weight_decay: 0.0003464853

epoch time: 959.0509643555s, input_fired_time: 18.0609016418, output_fired_time: 6.1599493027, propagate_forward_time: 357.5787048340, propagate_backward_time: 565.4320678711, weight_update_time: 0.4284299016, other_time: 11.8192749023
epoch time: 75.6846542358s, input_fired_time: 1.0220443010, output_fired_time: 0.0000000000, propagate_forward_time: 72.7089004517, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9537124634
validation[          , genome  6400] predictions:    6246/  10000 (62.46%), best:    6246/10000 (62.46%), error:     10997.93945, best error:     10997.93945 on epoch:     5, epoch:    5/50, mu: 0.6309297681, learning_rate: 0.0003584797, weight_decay: 0.0003235587

epoch time: 966.5286865234s, input_fired_time: 16.9842090607, output_fired_time: 8.1591377258, propagate_forward_time: 359.2837829590, propagate_backward_time: 570.6563110352, weight_update_time: 0.4494384527, other_time: 11.4452514648
epoch time: 74.6906051636s, input_fired_time: 0.9993278980, output_fired_time: 0.0000000000, propagate_forward_time: 71.7453536987, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9459228516
validation[          , genome  6400] predictions:    6191/  10000 (61.91%), best:    6246/10000 (62.46%), error:     11172.76758, best error:     10997.93945 on epoch:     5, epoch:    6/50, mu: 0.6473180056, learning_rate: 0.0003262324, weight_decay: 0.0003021491

epoch time: 960.3104248047s, input_fired_time: 16.9704627991, output_fired_time: 7.2526845932, propagate_forward_time: 360.1326293945, propagate_backward_time: 564.4845581055, weight_update_time: 0.4429382086, other_time: 11.4700927734
epoch time: 75.3010787964s, input_fired_time: 1.0575575829, output_fired_time: 0.0000000000, propagate_forward_time: 72.3134613037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9300613403
validation[          , genome  6400] predictions:    6128/  10000 (61.28%), best:    6246/10000 (62.46%), error:     11290.04395, best error:     10997.93945 on epoch:     5, epoch:    7/50, mu: 0.6629582644, learning_rate: 0.0002968860, weight_decay: 0.0002821562

epoch time: 968.6259155273s, input_fired_time: 19.1737861633, output_fired_time: 6.1399655342, propagate_forward_time: 357.0349426270, propagate_backward_time: 572.7201538086, weight_update_time: 0.4689543247, other_time: 13.5570678711
epoch time: 75.7565383911s, input_fired_time: 1.0111883879, output_fired_time: 0.0000000000, propagate_forward_time: 72.7825927734, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9627609253
validation[          , genome  6400] predictions:    6113/  10000 (61.13%), best:    6246/10000 (62.46%), error:     11447.21094, best error:     10997.93945 on epoch:     5, epoch:    8/50, mu: 0.6778846979, learning_rate: 0.0002701795, weight_decay: 0.0002634862

epoch time: 961.6123657227s, input_fired_time: 19.1545219421, output_fired_time: 6.2256035805, propagate_forward_time: 359.3255310059, propagate_backward_time: 563.4650268555, weight_update_time: 0.4421592951, other_time: 13.4417114258
epoch time: 75.1168289185s, input_fired_time: 1.0159770250, output_fired_time: 0.0000000000, propagate_forward_time: 72.1984252930, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9024276733
validation[          , genome  6400] predictions:    6038/  10000 (60.38%), best:    6246/10000 (62.46%), error:     11559.17676, best error:     10997.93945 on epoch:     5, epoch:    9/50, mu: 0.6921298504, learning_rate: 0.0002458753, weight_decay: 0.0002460516

epoch time: 960.2285766602s, input_fired_time: 19.0330085754, output_fired_time: 7.1953964233, propagate_forward_time: 356.2690734863, propagate_backward_time: 566.2482299805, weight_update_time: 0.4416563511, other_time: 11.4828491211
epoch time: 74.2597274780s, input_fired_time: 0.9887890816, output_fired_time: 0.0000000000, propagate_forward_time: 71.3438186646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9271163940
validation[          , genome  6400] predictions:    6146/  10000 (61.46%), best:    6246/10000 (62.46%), error:     11271.49512, best error:     10997.93945 on epoch:     5, epoch:   10/50, mu: 0.7057248354, learning_rate: 0.0002237575, weight_decay: 0.0002297706

epoch time: 963.6079101563s, input_fired_time: 17.0894165039, output_fired_time: 6.1724596024, propagate_forward_time: 352.9865417480, propagate_backward_time: 575.8573608398, weight_update_time: 0.4522716701, other_time: 11.5021362305
epoch time: 75.5865249634s, input_fired_time: 1.0037966967, output_fired_time: 0.0000000000, propagate_forward_time: 72.6261825562, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9565429688
validation[          , genome  6400] predictions:    5770/  10000 (57.70%), best:    6246/10000 (62.46%), error:     12224.04883, best error:     10997.93945 on epoch:     5, epoch:   11/50, mu: 0.7186993361, learning_rate: 0.0002036292, weight_decay: 0.0002145669

epoch time: 963.5184936523s, input_fired_time: 17.0275173187, output_fired_time: 8.0979986191, propagate_forward_time: 354.8722534180, propagate_backward_time: 570.9260864258, weight_update_time: 0.4533025026, other_time: 12.5946044922
epoch time: 74.4103469849s, input_fired_time: 0.9994959235, output_fired_time: 0.0000000000, propagate_forward_time: 71.5080032349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9028472900
validation[          , genome  6400] predictions:    6149/  10000 (61.49%), best:    6246/10000 (62.46%), error:     11380.79297, best error:     10997.93945 on epoch:     5, epoch:   12/50, mu: 0.7310816646, learning_rate: 0.0001853117, weight_decay: 0.0002003692

epoch time: 960.9766845703s, input_fired_time: 18.2337417603, output_fired_time: 7.1638097763, propagate_forward_time: 357.2398071289, propagate_backward_time: 566.9418945313, weight_update_time: 0.4542182386, other_time: 11.3973999023
epoch time: 73.6916198730s, input_fired_time: 1.0325433016, output_fired_time: 0.0000000000, propagate_forward_time: 70.7567138672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9023590088
validation[          , genome  6400] predictions:    6266/  10000 (62.66%), best:    6246/10000 (62.46%), error:     11090.20898, best error:     10997.93945 on epoch:     5, epoch:   13/50, mu: 0.7428988814, learning_rate: 0.0001686418, weight_decay: 0.0001871110

epoch time: 975.2465209961s, input_fired_time: 17.2823181152, output_fired_time: 6.1441774368, propagate_forward_time: 364.7851867676, propagate_backward_time: 572.5697021484, weight_update_time: 0.4778267443, other_time: 14.4652099609
epoch time: 76.6809463501s, input_fired_time: 1.0090496540, output_fired_time: 0.0000000000, propagate_forward_time: 73.7014312744, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9704666138
validation[          , genome  6400] predictions:    6061/  10000 (60.61%), best:    6246/10000 (62.46%), error:     11377.88281, best error:     10997.93945 on epoch:     5, epoch:   14/50, mu: 0.7541767359, learning_rate: 0.0001534716, weight_decay: 0.0001747301

epoch time: 1003.5187377930s, input_fired_time: 19.7859630585, output_fired_time: 6.4576420784, propagate_forward_time: 372.0925292969, propagate_backward_time: 593.2129516602, weight_update_time: 0.4812474847, other_time: 11.9696655273
epoch time: 81.3992996216s, input_fired_time: 1.0694400072, output_fired_time: 0.0000000000, propagate_forward_time: 75.0928268433, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2370300293
validation[          , genome  6400] predictions:    6246/  10000 (62.46%), best:    6246/10000 (62.46%), error:     10954.15527, best error:     10954.15527 on epoch:    15, epoch:   15/50, mu: 0.7649399042, learning_rate: 0.0001396659, weight_decay: 0.0001631683

epoch time: 971.8216552734s, input_fired_time: 17.1981410980, output_fired_time: 8.2572059631, propagate_forward_time: 358.9393005371, propagate_backward_time: 575.5373535156, weight_update_time: 0.4535098374, other_time: 11.8896484375
epoch time: 69.6397399902s, input_fired_time: 0.9560328722, output_fired_time: 0.0000000000, propagate_forward_time: 66.9159622192, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7677459717
validation[          , genome  6400] predictions:    6238/  10000 (62.38%), best:    6238/10000 (62.38%), error:     10829.53809, best error:     10829.53809 on epoch:    16, epoch:   16/50, mu: 0.7752118111, learning_rate: 0.0001271022, weight_decay: 0.0001523716

epoch time: 905.6843261719s, input_fired_time: 17.1279621124, output_fired_time: 5.7440781593, propagate_forward_time: 337.6499633789, propagate_backward_time: 533.5049438477, weight_update_time: 0.4207706153, other_time: 11.6573486328
epoch time: 69.6752777100s, input_fired_time: 0.9651595354, output_fired_time: 0.0000000000, propagate_forward_time: 66.9353103638, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7748107910
validation[          , genome  6400] predictions:    6156/  10000 (61.56%), best:    6238/10000 (62.38%), error:     11096.35742, best error:     10829.53809 on epoch:    16, epoch:   17/50, mu: 0.7850148678, learning_rate: 0.0001156687, weight_decay: 0.0001422894

epoch time: 903.9157714844s, input_fired_time: 17.1055889130, output_fired_time: 5.7484216690, propagate_forward_time: 336.1404113770, propagate_backward_time: 532.2752075195, weight_update_time: 0.4314049780, other_time: 12.6461791992
epoch time: 70.9420547485s, input_fired_time: 1.9771771431, output_fired_time: 0.0000000000, propagate_forward_time: 67.1507644653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8141098022
validation[          , genome  6400] predictions:    6272/  10000 (62.72%), best:    6238/10000 (62.38%), error:     11093.31836, best error:     10829.53809 on epoch:    16, epoch:   18/50, mu: 0.7943705320, learning_rate: 0.0001052636, weight_decay: 0.0001328742

epoch time: 936.9859008789s, input_fired_time: 16.7414703369, output_fired_time: 5.9332079887, propagate_forward_time: 348.3343505859, propagate_backward_time: 555.0225830078, weight_update_time: 0.4472289681, other_time: 10.9542846680
epoch time: 75.9296722412s, input_fired_time: 1.0386486053, output_fired_time: 0.0000000000, propagate_forward_time: 72.9408950806, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9501266479
validation[          , genome  6400] predictions:    6164/  10000 (61.64%), best:    6238/10000 (62.38%), error:     11324.35254, best error:     10829.53809 on epoch:    16, epoch:   19/50, mu: 0.8032991886, learning_rate: 0.0000957946, weight_decay: 0.0001240821

epoch time: 982.2729492188s, input_fired_time: 18.5815792084, output_fired_time: 6.1897406578, propagate_forward_time: 368.0631408691, propagate_backward_time: 576.8420410156, weight_update_time: 0.4566761851, other_time: 12.5964355469
epoch time: 75.5871582031s, input_fired_time: 1.0465608835, output_fired_time: 0.0000000000, propagate_forward_time: 72.6074142456, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9331817627
validation[          , genome  6400] predictions:    6322/  10000 (63.22%), best:    6322/10000 (63.22%), error:     10685.17969, best error:     10685.17969 on epoch:    20, epoch:   20/50, mu: 0.8118203282, learning_rate: 0.0000871773, weight_decay: 0.0001158717

epoch time: 963.9494628906s, input_fired_time: 18.1559486389, output_fired_time: 7.0488381386, propagate_forward_time: 360.5332336426, propagate_backward_time: 566.9888916016, weight_update_time: 0.4737175405, other_time: 11.2225341797
epoch time: 74.9813690186s, input_fired_time: 1.0711619854, output_fired_time: 0.0000000000, propagate_forward_time: 72.0362625122, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8739471436
validation[          , genome  6400] predictions:    6425/  10000 (64.25%), best:    6425/10000 (64.25%), error:     10444.17090, best error:     10444.17090 on epoch:    21, epoch:   21/50, mu: 0.8199525476, learning_rate: 0.0000793352, weight_decay: 0.0001082046

epoch time: 969.6475830078s, input_fired_time: 17.3796787262, output_fired_time: 7.1670241356, propagate_forward_time: 360.6865844727, propagate_backward_time: 573.1140747070, weight_update_time: 0.4638711810, other_time: 11.3001708984
epoch time: 75.0363616943s, input_fired_time: 1.0458300114, output_fired_time: 0.0000000000, propagate_forward_time: 72.1035079956, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8870239258
validation[          , genome  6400] predictions:    6408/  10000 (64.08%), best:    6425/10000 (64.25%), error:     10594.76953, best error:     10444.17090 on epoch:    21, epoch:   22/50, mu: 0.8277136087, learning_rate: 0.0000721986, weight_decay: 0.0001010448

epoch time: 972.7325439453s, input_fired_time: 17.4599723816, output_fired_time: 7.2837538719, propagate_forward_time: 361.9565429688, propagate_backward_time: 574.7802124023, weight_update_time: 0.4600230455, other_time: 11.2520751953
epoch time: 74.1292343140s, input_fired_time: 1.0171895027, output_fired_time: 0.0000000000, propagate_forward_time: 71.1827392578, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9293060303
validation[          , genome  6400] predictions:    6283/  10000 (62.83%), best:    6425/10000 (64.25%), error:     10790.69629, best error:     10444.17090 on epoch:    21, epoch:   23/50, mu: 0.8351204991, learning_rate: 0.0000657039, weight_decay: 0.0000943587

epoch time: 978.0548095703s, input_fired_time: 17.3051834106, output_fired_time: 6.2196664810, propagate_forward_time: 361.1180725098, propagate_backward_time: 580.7163696289, weight_update_time: 0.4744323492, other_time: 12.6954956055
epoch time: 76.7050933838s, input_fired_time: 1.0891283751, output_fired_time: 0.0000000000, propagate_forward_time: 73.6028060913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0131607056
validation[          , genome  6400] predictions:    6415/  10000 (64.15%), best:    6415/10000 (64.15%), error:     10425.74316, best error:     10425.74316 on epoch:    24, epoch:   24/50, mu: 0.8421893120, learning_rate: 0.0000597935, weight_decay: 0.0000881151

epoch time: 980.1633911133s, input_fired_time: 17.4052219391, output_fired_time: 6.1584076881, propagate_forward_time: 353.1545715332, propagate_backward_time: 590.6004028320, weight_update_time: 0.5006332994, other_time: 12.8447875977
epoch time: 76.1424255371s, input_fired_time: 1.0872819424, output_fired_time: 0.0000000000, propagate_forward_time: 73.0497283936, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0054168701
validation[          , genome  6400] predictions:    6366/  10000 (63.66%), best:    6415/10000 (64.15%), error:     10571.07617, best error:     10425.74316 on epoch:    24, epoch:   25/50, mu: 0.8489354849, learning_rate: 0.0000544147, weight_decay: 0.0000822846

epoch time: 980.3748779297s, input_fired_time: 18.4494152069, output_fired_time: 7.2176117897, propagate_forward_time: 362.8076171875, propagate_backward_time: 580.1452636719, weight_update_time: 0.4456685185, other_time: 11.7550048828
epoch time: 76.6966705322s, input_fired_time: 1.0969762802, output_fired_time: 0.0000000000, propagate_forward_time: 73.5530853271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0466079712
validation[          , genome  6400] predictions:    6527/  10000 (65.27%), best:    6527/10000 (65.27%), error:     10224.66992, best error:     10224.66992 on epoch:    26, epoch:   26/50, mu: 0.8553737402, learning_rate: 0.0000495198, weight_decay: 0.0000768399

epoch time: 974.4052124023s, input_fired_time: 17.2763538361, output_fired_time: 6.2713027000, propagate_forward_time: 360.0108947754, propagate_backward_time: 579.1426391602, weight_update_time: 0.4647462666, other_time: 11.7040405273
epoch time: 75.0938644409s, input_fired_time: 1.0333195925, output_fired_time: 0.0000000000, propagate_forward_time: 72.0662765503, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9942703247
validation[          , genome  6400] predictions:    6533/  10000 (65.33%), best:    6533/10000 (65.33%), error:     10186.61035, best error:     10186.61035 on epoch:    27, epoch:   27/50, mu: 0.8615182042, learning_rate: 0.0000450652, weight_decay: 0.0000717555

epoch time: 971.5386352539s, input_fired_time: 17.3474063873, output_fired_time: 6.1788148880, propagate_forward_time: 357.5708007813, propagate_backward_time: 578.6175537109, weight_update_time: 0.4719062448, other_time: 11.8240356445
epoch time: 74.5206375122s, input_fired_time: 1.0413013697, output_fired_time: 0.0000000000, propagate_forward_time: 71.5118026733, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9675369263
validation[          , genome  6400] predictions:    6539/  10000 (65.39%), best:    6539/10000 (65.39%), error:     10076.53027, best error:     10076.53027 on epoch:    28, epoch:   28/50, mu: 0.8673822284, learning_rate: 0.0000410114, weight_decay: 0.0000670075

epoch time: 966.8621826172s, input_fired_time: 17.1874847412, output_fired_time: 7.1591911316, propagate_forward_time: 358.6848754883, propagate_backward_time: 571.0466918945, weight_update_time: 0.4572395980, other_time: 12.7839355469
epoch time: 75.6188278198s, input_fired_time: 1.0370486975, output_fired_time: 0.0000000000, propagate_forward_time: 72.5881500244, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9936294556
validation[          , genome  6400] predictions:    6532/  10000 (65.32%), best:    6539/10000 (65.39%), error:     10181.97949, best error:     10076.53027 on epoch:    28, epoch:   29/50, mu: 0.8729785681, learning_rate: 0.0000373222, weight_decay: 0.0000625737

epoch time: 981.8400878906s, input_fired_time: 18.7607460022, output_fired_time: 7.4145689011, propagate_forward_time: 365.0630798340, propagate_backward_time: 577.6785888672, weight_update_time: 0.4548732936, other_time: 12.9230957031
epoch time: 75.7315444946s, input_fired_time: 1.0540587902, output_fired_time: 0.0000000000, propagate_forward_time: 72.6662139893, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0112686157
validation[          , genome  6400] predictions:    6583/  10000 (65.83%), best:    6583/10000 (65.83%), error:      9985.23340, best error:      9985.23340 on epoch:    30, epoch:   30/50, mu: 0.8783195019, learning_rate: 0.0000339648, weight_decay: 0.0000584333

epoch time: 975.0291137695s, input_fired_time: 19.4775352478, output_fired_time: 6.1730637550, propagate_forward_time: 362.6975708008, propagate_backward_time: 574.9995727539, weight_update_time: 0.4704585671, other_time: 11.6813964844
epoch time: 76.5238571167s, input_fired_time: 2.0846543312, output_fired_time: 0.0000000000, propagate_forward_time: 72.4183273315, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0208740234
validation[          , genome  6400] predictions:    6494/  10000 (64.94%), best:    6583/10000 (65.83%), error:     10226.05664, best error:      9985.23340 on epoch:    30, epoch:   31/50, mu: 0.8834166527, learning_rate: 0.0000309095, weight_decay: 0.0000545668

epoch time: 974.7910766602s, input_fired_time: 19.5998611450, output_fired_time: 6.2076497078, propagate_forward_time: 362.1336059570, propagate_backward_time: 574.1617431641, weight_update_time: 0.4492575526, other_time: 12.6882324219
epoch time: 76.5394058228s, input_fired_time: 1.0483510494, output_fired_time: 0.0000000000, propagate_forward_time: 73.4266662598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0643920898
validation[          , genome  6400] predictions:    6423/  10000 (64.23%), best:    6583/10000 (65.83%), error:     10403.98340, best error:      9985.23340 on epoch:    30, epoch:   32/50, mu: 0.8882811666, learning_rate: 0.0000281290, weight_decay: 0.0000509561

epoch time: 968.1639404297s, input_fired_time: 18.4476032257, output_fired_time: 6.1744270325, propagate_forward_time: 357.5053405762, propagate_backward_time: 573.3925781250, weight_update_time: 0.4476318359, other_time: 12.6439208984
epoch time: 74.9144668579s, input_fired_time: 1.0456707478, output_fired_time: 0.0000000000, propagate_forward_time: 71.9129791260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9558181763
validation[          , genome  6400] predictions:    6463/  10000 (64.63%), best:    6583/10000 (65.83%), error:     10304.29102, best error:      9985.23340 on epoch:    30, epoch:   33/50, mu: 0.8929237127, learning_rate: 0.0000255987, weight_decay: 0.0000475844

epoch time: 968.3739624023s, input_fired_time: 18.2781333923, output_fired_time: 7.1884164810, propagate_forward_time: 358.5151672363, propagate_backward_time: 572.4842529297, weight_update_time: 0.4643478096, other_time: 11.9079589844
epoch time: 75.6728210449s, input_fired_time: 1.0712925196, output_fired_time: 0.0000000000, propagate_forward_time: 72.6020431519, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9994888306
validation[          , genome  6400] predictions:    6529/  10000 (65.29%), best:    6583/10000 (65.83%), error:     10203.98438, best error:      9985.23340 on epoch:    30, epoch:   34/50, mu: 0.8973543644, learning_rate: 0.0000232959, weight_decay: 0.0000444358

epoch time: 967.8305664063s, input_fired_time: 18.4353713989, output_fired_time: 6.1703991890, propagate_forward_time: 355.2082519531, propagate_backward_time: 574.2822875977, weight_update_time: 0.4568426311, other_time: 13.7342529297
epoch time: 76.2975616455s, input_fired_time: 1.0348657370, output_fired_time: 0.0000000000, propagate_forward_time: 73.2403717041, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0223236084
validation[          , genome  6400] predictions:    6576/  10000 (65.76%), best:    6583/10000 (65.83%), error:     10062.50293, best error:      9985.23340 on epoch:    30, epoch:   35/50, mu: 0.9015827775, learning_rate: 0.0000212003, weight_decay: 0.0000414955

epoch time: 969.0996093750s, input_fired_time: 17.2355861664, output_fired_time: 6.1663384438, propagate_forward_time: 359.3211669922, propagate_backward_time: 572.5107421875, weight_update_time: 0.4666011930, other_time: 13.8657836914
epoch time: 76.5025253296s, input_fired_time: 1.0656561852, output_fired_time: 0.0000000000, propagate_forward_time: 73.4214782715, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0153884888
validation[          , genome  6400] predictions:    6591/  10000 (65.91%), best:    6583/10000 (65.83%), error:     10048.29199, best error:      9985.23340 on epoch:    30, epoch:   36/50, mu: 0.9056181908, learning_rate: 0.0000192932, weight_decay: 0.0000387498

epoch time: 971.9512939453s, input_fired_time: 20.3275661469, output_fired_time: 6.1402153969, propagate_forward_time: 356.0258178711, propagate_backward_time: 577.6619873047, weight_update_time: 0.4608716667, other_time: 11.7957153320
epoch time: 75.2398071289s, input_fired_time: 1.0586907864, output_fired_time: 0.0000000000, propagate_forward_time: 72.1730041504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0081100464
validation[          , genome  6400] predictions:    6575/  10000 (65.75%), best:    6583/10000 (65.83%), error:     10028.67773, best error:      9985.23340 on epoch:    30, epoch:   37/50, mu: 0.9094694257, learning_rate: 0.0000175577, weight_decay: 0.0000361858

epoch time: 972.6101684570s, input_fired_time: 18.3638248444, output_fired_time: 6.1530632973, propagate_forward_time: 355.6961669922, propagate_backward_time: 580.9047851563, weight_update_time: 0.4531556666, other_time: 11.4923095703
epoch time: 74.9295120239s, input_fired_time: 1.0615520477, output_fired_time: 0.0000000000, propagate_forward_time: 71.9462890625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9216690063
validation[          , genome  6400] predictions:    6638/  10000 (66.38%), best:    6638/10000 (66.38%), error:      9853.96973, best error:      9853.96973 on epoch:    38, epoch:   38/50, mu: 0.9131448865, learning_rate: 0.0000159783, weight_decay: 0.0000337914

epoch time: 966.6571044922s, input_fired_time: 18.3621768951, output_fired_time: 7.1601934433, propagate_forward_time: 356.8965454102, propagate_backward_time: 573.0040893555, weight_update_time: 0.4786634743, other_time: 11.2340698242
epoch time: 74.8213195801s, input_fired_time: 1.0848218203, output_fired_time: 0.0000000000, propagate_forward_time: 71.7359695435, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0005264282
validation[          , genome  6400] predictions:    6489/  10000 (64.89%), best:    6638/10000 (66.38%), error:     10250.11328, best error:      9853.96973 on epoch:    38, epoch:   39/50, mu: 0.9166526198, learning_rate: 0.0000145410, weight_decay: 0.0000315555

epoch time: 976.6323242188s, input_fired_time: 17.3230113983, output_fired_time: 6.1586351395, propagate_forward_time: 365.3284301758, propagate_backward_time: 576.2214965820, weight_update_time: 0.4814860225, other_time: 11.6007690430
epoch time: 76.7440338135s, input_fired_time: 2.0544204712, output_fired_time: 0.0000000000, propagate_forward_time: 72.7373504639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9522628784
validation[          , genome  6400] predictions:    6597/  10000 (65.97%), best:    6638/10000 (66.38%), error:      9932.99805, best error:      9853.96973 on epoch:    38, epoch:   40/50, mu: 0.9200002551, learning_rate: 0.0000132329, weight_decay: 0.0000294675

epoch time: 1221.9892578125s, input_fired_time: 24.1098747253, output_fired_time: 10.7431497574, propagate_forward_time: 446.6480102539, propagate_backward_time: 724.0240478516, weight_update_time: 0.5855633616, other_time: 16.4641723633
epoch time: 95.1096725464s, input_fired_time: 2.2360265255, output_fired_time: 0.0000000000, propagate_forward_time: 90.3375015259, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5361480713
validation[          , genome  6400] predictions:    6646/  10000 (66.46%), best:    6638/10000 (66.38%), error:      9858.92090, best error:      9853.96973 on epoch:    38, epoch:   41/50, mu: 0.9231950641, learning_rate: 0.0000120425, weight_decay: 0.0000275176

epoch time: 1225.7071533203s, input_fired_time: 25.2446002960, output_fired_time: 7.8934397697, propagate_forward_time: 452.0831604004, propagate_backward_time: 724.6463623047, weight_update_time: 0.6842707992, other_time: 15.8395996094
epoch time: 94.9800262451s, input_fired_time: 1.4043291807, output_fired_time: 0.0000000000, propagate_forward_time: 91.1498947144, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4258041382
validation[          , genome  6400] predictions:    6582/  10000 (65.82%), best:    6638/10000 (66.38%), error:     10015.14746, best error:      9853.96973 on epoch:    38, epoch:   42/50, mu: 0.9262440801, learning_rate: 0.0000109592, weight_decay: 0.0000256968

epoch time: 1235.5950927734s, input_fired_time: 24.7152004242, output_fired_time: 7.9775276184, propagate_forward_time: 458.9123535156, propagate_backward_time: 728.6436157227, weight_update_time: 0.6062244177, other_time: 15.3463745117
epoch time: 95.4938735962s, input_fired_time: 1.3403755426, output_fired_time: 0.0000000000, propagate_forward_time: 91.6278457642, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5256500244
validation[          , genome  6400] predictions:    6626/  10000 (66.26%), best:    6638/10000 (66.38%), error:      9929.10645, best error:      9853.96973 on epoch:    38, epoch:   43/50, mu: 0.9291539192, learning_rate: 0.0000099734, weight_decay: 0.0000239965

epoch time: 1228.6462402344s, input_fired_time: 24.4222068787, output_fired_time: 8.8131332397, propagate_forward_time: 455.7938537598, propagate_backward_time: 723.8031616211, weight_update_time: 0.6523702741, other_time: 15.8139038086
epoch time: 95.0313339233s, input_fired_time: 1.4150937796, output_fired_time: 0.0000000000, propagate_forward_time: 91.0330047607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5832366943
validation[          , genome  6400] predictions:    6667/  10000 (66.67%), best:    6667/10000 (66.67%), error:      9824.53125, best error:      9824.53125 on epoch:    44, epoch:   44/50, mu: 0.9319309592, learning_rate: 0.0000090762, weight_decay: 0.0000224087

epoch time: 1234.7919921875s, input_fired_time: 22.1873321533, output_fired_time: 7.3655881882, propagate_forward_time: 461.4614562988, propagate_backward_time: 729.1567382813, weight_update_time: 0.6921982169, other_time: 14.6208496094
epoch time: 95.4413909912s, input_fired_time: 1.3946930170, output_fired_time: 0.0000000000, propagate_forward_time: 91.5383758545, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5083236694
validation[          , genome  6400] predictions:    6608/  10000 (66.08%), best:    6667/10000 (66.67%), error:      9968.09863, best error:      9824.53125 on epoch:    44, epoch:   45/50, mu: 0.9345812798, learning_rate: 0.0000082598, weight_decay: 0.0000209259

epoch time: 1230.0565185547s, input_fired_time: 22.0257358551, output_fired_time: 8.9499320984, propagate_forward_time: 455.9412841797, propagate_backward_time: 728.0844116211, weight_update_time: 0.5305964947, other_time: 15.0551147461
epoch time: 96.3972930908s, input_fired_time: 1.2907820940, output_fired_time: 0.0000000000, propagate_forward_time: 92.6769790649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4295349121
validation[          , genome  6400] predictions:    6639/  10000 (66.39%), best:    6667/10000 (66.67%), error:      9832.70996, best error:      9824.53125 on epoch:    44, epoch:   46/50, mu: 0.9371106029, learning_rate: 0.0000075168, weight_decay: 0.0000195413

epoch time: 1227.3897705078s, input_fired_time: 22.8207454681, output_fired_time: 8.0037059784, propagate_forward_time: 450.7828063965, propagate_backward_time: 730.6293945313, weight_update_time: 0.5464875102, other_time: 15.1530761719
epoch time: 96.4287872314s, input_fired_time: 1.3861169815, output_fired_time: 0.0000000000, propagate_forward_time: 92.5308303833, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5118408203
validation[          , genome  6400] predictions:    6555/  10000 (65.55%), best:    6667/10000 (66.67%), error:      9928.90918, best error:      9824.53125 on epoch:    44, epoch:   47/50, mu: 0.9395245314, learning_rate: 0.0000068406, weight_decay: 0.0000182482

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400_3_r1629005478_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.536456
read mu: 0.941828
read mu_delta: 0.954359
read initial_learning_rate: 0.000574317
read learning_rate: 6.22524e-06
read learning_rate_delta: 0.910044
read initial_weight_decay: 0.000455629
read weight_decay: 1.70408e-05
read weight_decay_delta: 0.933831
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.106298
read input_dropout_probability: 0.00125594
read hidden_dropout_probability: 0.100995
read velocity_reset: 1709
read epoch: 48
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 44
read number_validation_images: 10000
read best_validation_predictions: 6667
read best_validation_error: 9824.53
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6400
read normal distribution: '0 1.86046957969666 0.455630153417587'
generator_str: '1166756060'
read generator: 1166756060
read generated_by_map:
1 merge_node 1
reading 103 nodes.
reading 521 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400_3_r1629005478_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.536456
read mu: 0.941828
read mu_delta: 0.954359
read initial_learning_rate: 0.000574317
read learning_rate: 6.22524e-06
read learning_rate_delta: 0.910044
read initial_weight_decay: 0.000455629
read weight_decay: 1.70408e-05
read weight_decay_delta: 0.933831
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.106298
read input_dropout_probability: 0.00125594
read hidden_dropout_probability: 0.100995
read velocity_reset: 1709
read epoch: 48
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 44
read number_validation_images: 10000
read best_validation_predictions: 6667
read best_validation_error: 9824.53
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6400
read normal distribution: '0 1.86046957969666 0.455630153417587'
generator_str: '1166756060'
read generator: 1166756060
read generated_by_map:
1 merge_node 1
reading 103 nodes.
reading 521 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_179_6400_3_r1629005478_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.536456
read mu: 0.941828
read mu_delta: 0.954359
read initial_learning_rate: 0.000574317
read learning_rate: 6.22524e-06
read learning_rate_delta: 0.910044
read initial_weight_decay: 0.000455629
read weight_decay: 1.70408e-05
read weight_decay_delta: 0.933831
read batch_size: 113
read epsilon: 1e-07
read alpha: 0.106298
read input_dropout_probability: 0.00125594
read hidden_dropout_probability: 0.100995
read velocity_reset: 1709
read epoch: 48
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 44
read number_validation_images: 10000
read best_validation_predictions: 6667
read best_validation_error: 9824.53
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6400
read normal distribution: '0 1.86046957969666 0.455630153417587'
generator_str: '1166756060'
read generator: 1166756060
read generated_by_map:
1 merge_node 1
reading 103 nodes.
reading 521 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1203.55285644531s, input_fired_time: 23.0063400268555, output_fired_time: 9.25132465362549, propagate_forward_time: 447.408325195313, propagate_backward_time: 709.448669433594, weight_update_time: 0.516029298305511, other_time: 14.4381713867188
epoch time: 92.0166625976563s, input_fired_time: 1.20017743110657, output_fired_time: 0, propagate_forward_time: 88.3923187255859, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.42416381835938
validation[          , genome  6400] predictions:    6691/  10000 (66.91%), best:    6691/10000 (66.91%), error:      9762.45801, best error:      9762.45801 on epoch:    48, epoch:   48/50, mu: 0.9418282509, learning_rate: 0.0000062252, weight_decay: 0.0000170408

epoch time: 1195.0906982422s, input_fired_time: 24.2253570557, output_fired_time: 7.6283645630, propagate_forward_time: 442.2034606934, propagate_backward_time: 705.5978393555, weight_update_time: 0.5758941770, other_time: 15.4356079102
epoch time: 93.4137878418s, input_fired_time: 1.2662963867, output_fired_time: 0.0000000000, propagate_forward_time: 89.6794662476, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4680252075
validation[          , genome  6400] predictions:    6667/  10000 (66.67%), best:    6667/10000 (66.67%), error:      9760.29395, best error:      9760.29395 on epoch:    49, epoch:   49/50, mu: 0.9440268278, learning_rate: 0.0000056652, weight_decay: 0.0000159132

epoch time: 1194.4742431641s, input_fired_time: 21.5437812805, output_fired_time: 7.8976998329, propagate_forward_time: 442.8069152832, propagate_backward_time: 708.0692138672, weight_update_time: 0.5812206864, other_time: 14.1566162109
epoch time: 93.5533294678s, input_fired_time: 1.2960016727, output_fired_time: 0.0000000000, propagate_forward_time: 89.7961959839, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4611282349
validation[          , genome  6400] predictions:    6612/  10000 (66.12%), best:    6667/10000 (66.67%), error:      9867.62305, best error:      9760.29395 on epoch:    49, epoch:   50/50, mu: 0.9461250901, learning_rate: 0.0000051556, weight_decay: 0.0000148602

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 460.6430053711s, input_fired_time: 6.8195610046, output_fired_time: 0.0000000000, propagate_forward_time: 441.5218505859, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.3016052246
best training[          , genome  6400] predictions:   35127/  50000 (70.25%), best:    6667/10000 (66.67%), error:     44998.36719, best error:      9760.29395 on epoch:    49, epoch:   51/50, mu: 0.9481275678, learning_rate: 0.0000046918, weight_decay: 0.0000138769
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 93.2469329834s, input_fired_time: 3.2459499836, output_fired_time: 0.0000000000, propagate_forward_time: 87.5696792603, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4313049316
   testing[          , genome  6400] predictions:    6667/  10000 (66.67%), best:    6667/10000 (66.67%), error:      9760.29395, best error:      9760.29395 on epoch:    49, epoch:   51/50, mu: 0.9481275678, learning_rate: 0.0000046918, weight_decay: 0.0000138769
backpropagation finished successfully!
16:20:53 (2912): called boinc_finish(0)

</stderr_txt>
]]>