Nameexact_genome_1542127803_179_5798_2
Workunit4086485
Created13 Jan 2019, 8:10:28 UTC
Sent13 Jan 2019, 8:10:40 UTC
Report deadline14 Mar 2019, 22:47:19 UTC
Received14 Jan 2019, 5:00:39 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88573
Run time19 hours 42 min 59 sec
CPU time19 hours 42 min 54 sec
Validate stateValid
Credit15,152.20
Device peak FLOPS3.61 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size562.85 MB
Peak swap size561.66 MB
Peak disk usage14.92 MB

Stderr output

<core_client_version>7.6.22</core_client_version>
<![CDATA[
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_179_5798.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542127803_179_5798_2_r1290296143_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542127803_179_5798.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.50532
read mu: 0.50532
read mu_delta: 0.956639
read initial_learning_rate: 0.00164046
read learning_rate: 0.00164046
read learning_rate_delta: 0.914664
read initial_weight_decay: 0.000425836
read weight_decay: 0.000425836
read weight_decay_delta: 0.936139
read batch_size: 103
read epsilon: 1e-07
read alpha: 0.12091
read input_dropout_probability: 0.00134873
read hidden_dropout_probability: 0.093361
read velocity_reset: 1805
read epoch: 0
read max_epochs: 50
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5798
read normal distribution: '0 0.372576624155045 0.61262983083725'
generator_str: '350121683'
read generator: 350121683
read generated_by_map:
1 crossover 1
reading 107 nodes.
reading 530 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 389933401
post shuffle 1: 1867055289
epoch time: 1273.71545410156s, input_fired_time: 20.4616641998291, output_fired_time: 8.24356651306152, propagate_forward_time: 495.623870849609, propagate_backward_time: 734.66015625, weight_update_time: 0.510596692562103, other_time: 14.7261962890625
epoch time: 104.00666809082s, input_fired_time: 1.39685046672821, output_fired_time: 0, propagate_forward_time: 100.060440063477, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.54937744140625
validation[          , genome  5798] predictions:    5602/  10000 (56.02%), best:    5602/10000 (56.02%), error:     12683.43066, best error:     12683.43066 on epoch:     0, epoch:    0/50, mu: 0.5053198934, learning_rate: 0.0016404595, weight_decay: 0.0004258365

epoch time: 1275.6697998047s, input_fired_time: 20.5292816162, output_fired_time: 8.2628908157, propagate_forward_time: 496.2198486328, propagate_backward_time: 735.9223022461, weight_update_time: 0.5104115009, other_time: 14.7354125977
epoch time: 104.0406494141s, input_fired_time: 1.4172235727, output_fired_time: 0.0000000000, propagate_forward_time: 100.0685577393, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5548706055
validation[          , genome  5798] predictions:    5997/  10000 (59.97%), best:    5997/10000 (59.97%), error:     11383.18262, best error:     11383.18262 on epoch:     1, epoch:    1/50, mu: 0.5263358951, learning_rate: 0.0015004688, weight_decay: 0.0003986420

epoch time: 1275.5416259766s, input_fired_time: 20.5647983551, output_fired_time: 8.2767229080, propagate_forward_time: 496.2062683105, propagate_backward_time: 735.7625732422, weight_update_time: 0.5095630884, other_time: 14.7312011719
epoch time: 104.0205535889s, input_fired_time: 1.4182876348, output_fired_time: 0.0000000000, propagate_forward_time: 100.0473709106, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5548934937
validation[          , genome  5798] predictions:    6009/  10000 (60.09%), best:    5997/10000 (59.97%), error:     11469.48242, best error:     11383.18262 on epoch:     1, epoch:    2/50, mu: 0.5464406610, learning_rate: 0.0013724244, weight_decay: 0.0003731842

epoch time: 1274.8974609375s, input_fired_time: 20.5596199036, output_fired_time: 8.2617874146, propagate_forward_time: 495.9162292480, propagate_backward_time: 735.4270019531, weight_update_time: 0.5095637441, other_time: 14.7327880859
epoch time: 104.0455856323s, input_fired_time: 1.3983695507, output_fired_time: 0.0000000000, propagate_forward_time: 100.0503311157, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5968856812
validation[          , genome  5798] predictions:    5581/  10000 (55.81%), best:    5997/10000 (59.97%), error:     12816.90918, best error:     11383.18262 on epoch:     1, epoch:    3/50, mu: 0.5656736493, learning_rate: 0.0012553069, weight_decay: 0.0003493521

epoch time: 1273.8415527344s, input_fired_time: 20.5444335938, output_fired_time: 8.2538824081, propagate_forward_time: 495.5148620605, propagate_backward_time: 734.7865600586, weight_update_time: 0.5095262527, other_time: 14.7417602539
epoch time: 103.8189926147s, input_fired_time: 1.4089666605, output_fired_time: 0.0000000000, propagate_forward_time: 99.8504028320, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5596237183
validation[          , genome  5798] predictions:    5944/  10000 (59.44%), best:    5997/10000 (59.97%), error:     11517.78516, best error:     11383.18262 on epoch:     1, epoch:    4/50, mu: 0.5840727091, learning_rate: 0.0011481837, weight_decay: 0.0003270420

epoch time: 1274.1826171875s, input_fired_time: 20.5539436340, output_fired_time: 8.2642812729, propagate_forward_time: 495.6300048828, propagate_backward_time: 734.9811401367, weight_update_time: 0.5108807683, other_time: 14.7532348633
epoch time: 103.9040145874s, input_fired_time: 1.4131515026, output_fired_time: 0.0000000000, propagate_forward_time: 99.9437789917, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5470809937
validation[          , genome  5798] predictions:    5998/  10000 (59.98%), best:    5998/10000 (59.98%), error:     11365.09082, best error:     11365.09082 on epoch:     5, epoch:    5/50, mu: 0.6016739607, learning_rate: 0.0010502019, weight_decay: 0.0003061567

epoch time: 1274.4587402344s, input_fired_time: 20.5611267090, output_fired_time: 8.2668571472, propagate_forward_time: 495.7111206055, propagate_backward_time: 735.1713867188, weight_update_time: 0.5103312731, other_time: 14.7482299805
epoch time: 103.8589019775s, input_fired_time: 1.3995761871, output_fired_time: 0.0000000000, propagate_forward_time: 99.9147949219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5445327759
validation[          , genome  5798] predictions:    5646/  10000 (56.46%), best:    5998/10000 (59.98%), error:     12792.01953, best error:     11365.09082 on epoch:     5, epoch:    6/50, mu: 0.6185120344, learning_rate: 0.0009605816, weight_decay: 0.0002866051

epoch time: 1274.0710449219s, input_fired_time: 20.5444755554, output_fired_time: 8.2419128418, propagate_forward_time: 495.4941101074, propagate_backward_time: 735.0703735352, weight_update_time: 0.5101185441, other_time: 14.7201538086
epoch time: 103.8524703979s, input_fired_time: 1.4204820395, output_fired_time: 0.0000000000, propagate_forward_time: 99.8749694824, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5570220947
validation[          , genome  5798] predictions:    6115/  10000 (61.15%), best:    6115/10000 (61.15%), error:     11174.87500, best error:     11174.87500 on epoch:     7, epoch:    7/50, mu: 0.6346199512, learning_rate: 0.0008786092, weight_decay: 0.0002683021

epoch time: 1276.5964355469s, input_fired_time: 20.5766582489, output_fired_time: 8.2710065842, propagate_forward_time: 496.6898193359, propagate_backward_time: 736.3494873047, weight_update_time: 0.5109127760, other_time: 14.7094726563
epoch time: 104.1088562012s, input_fired_time: 1.4166550636, output_fired_time: 0.0000000000, propagate_forward_time: 100.1485671997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5436325073
validation[          , genome  5798] predictions:    6208/  10000 (62.08%), best:    6208/10000 (62.08%), error:     10920.86621, best error:     10920.86621 on epoch:     8, epoch:    8/50, mu: 0.6500294209, learning_rate: 0.0008036320, weight_decay: 0.0002511679

epoch time: 1277.8173828125s, input_fired_time: 20.5938568115, output_fired_time: 8.2828722000, propagate_forward_time: 497.0115356445, propagate_backward_time: 737.2225341797, weight_update_time: 0.5101293325, other_time: 14.7066040039
epoch time: 104.2483291626s, input_fired_time: 1.4060428143, output_fired_time: 0.0000000000, propagate_forward_time: 100.2883987427, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5538864136
validation[          , genome  5798] predictions:    5875/  10000 (58.75%), best:    6208/10000 (62.08%), error:     11954.66016, best error:     10920.86621 on epoch:     8, epoch:    9/50, mu: 0.6647707224, learning_rate: 0.0007350530, weight_decay: 0.0002351280

epoch time: 1275.3873291016s, input_fired_time: 20.5807685852, output_fired_time: 8.2411727905, propagate_forward_time: 495.9848632813, propagate_backward_time: 735.8627929688, weight_update_time: 0.5114131570, other_time: 14.7176513672
epoch time: 104.1473693848s, input_fired_time: 1.4251049757, output_fired_time: 0.0000000000, propagate_forward_time: 100.1630706787, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5591964722
validation[          , genome  5798] predictions:    6148/  10000 (61.48%), best:    6208/10000 (62.08%), error:     11147.06934, best error:     10920.86621 on epoch:     8, epoch:   10/50, mu: 0.6788728833, learning_rate: 0.0006723264, weight_decay: 0.0002201124

epoch time: 1273.9440917969s, input_fired_time: 20.5775814056, output_fired_time: 8.2452735901, propagate_forward_time: 495.6784973145, propagate_backward_time: 734.7221679688, weight_update_time: 0.5088239908, other_time: 14.7205810547
epoch time: 103.7605438232s, input_fired_time: 1.4154015779, output_fired_time: 0.0000000000, propagate_forward_time: 99.7986297607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5465087891
validation[          , genome  5798] predictions:    6347/  10000 (63.47%), best:    6347/10000 (63.47%), error:     10475.08496, best error:     10475.08496 on epoch:    11, epoch:   11/50, mu: 0.6923635602, learning_rate: 0.0006149525, weight_decay: 0.0002060557

epoch time: 1273.8424072266s, input_fired_time: 20.5860919952, output_fired_time: 8.2556676865, propagate_forward_time: 495.6281127930, propagate_backward_time: 734.6495361328, weight_update_time: 0.5115112662, other_time: 14.7230834961
epoch time: 103.9153518677s, input_fired_time: 1.4230759144, output_fired_time: 0.0000000000, propagate_forward_time: 99.9399337769, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5523452759
validation[          , genome  5798] predictions:    6043/  10000 (60.43%), best:    6347/10000 (63.47%), error:     11459.81543, best error:     10475.08496 on epoch:    11, epoch:   12/50, mu: 0.7052692771, learning_rate: 0.0005624748, weight_decay: 0.0001928967

epoch time: 1275.6966552734s, input_fired_time: 20.6088485718, output_fired_time: 8.2779579163, propagate_forward_time: 496.2795104980, propagate_backward_time: 735.7896118164, weight_update_time: 0.5109999180, other_time: 14.7406616211
epoch time: 103.9450759888s, input_fired_time: 1.4359887838, output_fired_time: 0.0000000000, propagate_forward_time: 99.9573669434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5517196655
validation[          , genome  5798] predictions:    6340/  10000 (63.40%), best:    6347/10000 (63.47%), error:     10602.07910, best error:     10475.08496 on epoch:    11, epoch:   13/50, mu: 0.7176153660, learning_rate: 0.0005144753, weight_decay: 0.0001805781

epoch time: 1275.0714111328s, input_fired_time: 20.6136379242, output_fired_time: 8.2684612274, propagate_forward_time: 495.7873535156, propagate_backward_time: 735.6257934570, weight_update_time: 0.5122641325, other_time: 14.7761840820
epoch time: 103.8047943115s, input_fired_time: 1.4261445999, output_fired_time: 0.0000000000, propagate_forward_time: 99.8265228271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5521240234
validation[          , genome  5798] predictions:    6422/  10000 (64.22%), best:    6422/10000 (64.22%), error:     10392.93945, best error:     10392.93945 on epoch:    14, epoch:   14/50, mu: 0.7294261456, learning_rate: 0.0004705719, weight_decay: 0.0001690461

epoch time: 1272.7862548828s, input_fired_time: 20.5920562744, output_fired_time: 8.2393999100, propagate_forward_time: 495.0776672363, propagate_backward_time: 734.1690063477, weight_update_time: 0.5111476183, other_time: 14.7081909180
epoch time: 103.7934494019s, input_fired_time: 1.4257673025, output_fired_time: 0.0000000000, propagate_forward_time: 99.8109588623, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5567245483
validation[          , genome  5798] predictions:    6401/  10000 (64.01%), best:    6422/10000 (64.22%), error:     10454.53223, best error:     10392.93945 on epoch:    14, epoch:   15/50, mu: 0.7407248020, learning_rate: 0.0004304150, weight_decay: 0.0001582506

epoch time: 1275.1076660156s, input_fired_time: 20.6292572021, output_fired_time: 8.2687520981, propagate_forward_time: 496.1072082520, propagate_backward_time: 735.3619995117, weight_update_time: 0.5127580762, other_time: 14.7404174805
epoch time: 103.7677688599s, input_fired_time: 1.4262350798, output_fired_time: 0.0000000000, propagate_forward_time: 99.7963027954, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5452346802
validation[          , genome  5798] predictions:    6442/  10000 (64.42%), best:    6442/10000 (64.42%), error:     10278.19824, best error:     10278.19824 on epoch:    16, epoch:   16/50, mu: 0.7515335083, learning_rate: 0.0003936850, weight_decay: 0.0001481445

epoch time: 1273.9052734375s, input_fired_time: 20.6189785004, output_fired_time: 8.2552394867, propagate_forward_time: 495.5386962891, propagate_backward_time: 734.7815551758, weight_update_time: 0.5121545792, other_time: 14.7107543945
epoch time: 103.9481124878s, input_fired_time: 1.4225196838, output_fired_time: 0.0000000000, propagate_forward_time: 99.9695892334, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5559997559
validation[          , genome  5798] predictions:    6211/  10000 (62.11%), best:    6442/10000 (64.42%), error:     10917.90625, best error:     10278.19824 on epoch:    16, epoch:   17/50, mu: 0.7618735433, learning_rate: 0.0003600894, weight_decay: 0.0001386838

epoch time: 1275.6103515625s, input_fired_time: 20.6645755768, output_fired_time: 8.2650346756, propagate_forward_time: 496.1405029297, propagate_backward_time: 735.7998657227, weight_update_time: 0.5139641166, other_time: 14.7404174805
epoch time: 103.9423141479s, input_fired_time: 1.4312754869, output_fired_time: 0.0000000000, propagate_forward_time: 99.9475402832, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5634994507
validation[          , genome  5798] predictions:    6148/  10000 (61.48%), best:    6442/10000 (64.42%), error:     11014.58398, best error:     10278.19824 on epoch:    16, epoch:   18/50, mu: 0.7717652321, learning_rate: 0.0003293607, weight_decay: 0.0001298273

epoch time: 1275.1616210938s, input_fired_time: 20.6557941437, output_fired_time: 8.2511711121, propagate_forward_time: 496.0070800781, propagate_backward_time: 735.5051879883, weight_update_time: 0.5144919753, other_time: 14.7423706055
epoch time: 103.9614105225s, input_fired_time: 1.4289827347, output_fired_time: 0.0000000000, propagate_forward_time: 99.9812469482, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5511779785
validation[          , genome  5798] predictions:    6429/  10000 (64.29%), best:    6429/10000 (64.29%), error:     10255.08008, best error:     10255.08008 on epoch:    19, epoch:   19/50, mu: 0.7812280059, learning_rate: 0.0003012543, weight_decay: 0.0001215363

epoch time: 1275.1097412109s, input_fired_time: 20.6698608398, output_fired_time: 8.2655048370, propagate_forward_time: 495.7883605957, propagate_backward_time: 735.6732177734, weight_update_time: 0.5130261779, other_time: 14.7128906250
epoch time: 104.0525817871s, input_fired_time: 1.4336615801, output_fired_time: 0.0000000000, propagate_forward_time: 100.0674743652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5514450073
validation[          , genome  5798] predictions:    6464/  10000 (64.64%), best:    6464/10000 (64.64%), error:     10218.78418, best error:     10218.78418 on epoch:    20, epoch:   20/50, mu: 0.7902804613, learning_rate: 0.0002755464, weight_decay: 0.0001137748

epoch time: 1274.2879638672s, input_fired_time: 20.6613292694, output_fired_time: 8.2556076050, propagate_forward_time: 495.5787048340, propagate_backward_time: 735.0536499023, weight_update_time: 0.5154981017, other_time: 14.7385864258
epoch time: 103.8905715942s, input_fired_time: 1.4511488676, output_fired_time: 0.0000000000, propagate_forward_time: 99.8784561157, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5609664917
validation[          , genome  5798] predictions:    6375/  10000 (63.75%), best:    6464/10000 (64.64%), error:     10468.58398, best error:     10218.78418 on epoch:    20, epoch:   21/50, mu: 0.7989404202, learning_rate: 0.0002520323, weight_decay: 0.0001065090

epoch time: 1274.0329589844s, input_fired_time: 20.6647567749, output_fired_time: 8.2500934601, propagate_forward_time: 495.4010314941, propagate_backward_time: 734.9951782227, weight_update_time: 0.5146080852, other_time: 14.7218627930
epoch time: 103.8826370239s, input_fired_time: 1.4358052015, output_fired_time: 0.0000000000, propagate_forward_time: 99.8967742920, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5500564575
validation[          , genome  5798] predictions:    6413/  10000 (64.13%), best:    6464/10000 (64.64%), error:     10403.98438, best error:     10218.78418 on epoch:    20, epoch:   22/50, mu: 0.8072248697, learning_rate: 0.0002305248, weight_decay: 0.0000997072

epoch time: 1274.0006103516s, input_fired_time: 20.6635303497, output_fired_time: 8.2505245209, propagate_forward_time: 495.5470275879, propagate_backward_time: 734.8013916016, weight_update_time: 0.5160222054, other_time: 14.7381591797
epoch time: 103.8823699951s, input_fired_time: 1.4404476881, output_fired_time: 0.0000000000, propagate_forward_time: 99.8858489990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5560760498
validation[          , genome  5798] predictions:    6486/  10000 (64.86%), best:    6486/10000 (64.86%), error:     10176.87305, best error:     10176.87305 on epoch:    23, epoch:   23/50, mu: 0.8151500821, learning_rate: 0.0002108527, weight_decay: 0.0000933398

epoch time: 1275.4975585938s, input_fired_time: 20.7045803070, output_fired_time: 8.2552127838, propagate_forward_time: 495.8999328613, propagate_backward_time: 735.9064331055, weight_update_time: 0.5175318122, other_time: 14.7313842773
epoch time: 103.9235458374s, input_fired_time: 1.4427595139, output_fired_time: 0.0000000000, propagate_forward_time: 99.9297714233, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5510177612
validation[          , genome  5798] predictions:    6437/  10000 (64.37%), best:    6486/10000 (64.86%), error:     10205.26270, best error:     10176.87305 on epoch:    23, epoch:   24/50, mu: 0.8227316737, learning_rate: 0.0001928593, weight_decay: 0.0000873790

epoch time: 1273.8188476563s, input_fired_time: 20.6837863922, output_fired_time: 8.2456302643, propagate_forward_time: 495.3813781738, propagate_backward_time: 734.8062133789, weight_update_time: 0.5158974528, other_time: 14.7018432617
epoch time: 103.9119033813s, input_fired_time: 1.4488409758, output_fired_time: 0.0000000000, propagate_forward_time: 99.9138488770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5492172241
validation[          , genome  5798] predictions:    6479/  10000 (64.79%), best:    6479/10000 (64.79%), error:     10160.80566, best error:     10160.80566 on epoch:    25, epoch:   25/50, mu: 0.8299845457, learning_rate: 0.0001764014, weight_decay: 0.0000817988

epoch time: 1277.8898925781s, input_fired_time: 20.7310676575, output_fired_time: 8.2781648636, propagate_forward_time: 497.1049804688, propagate_backward_time: 737.0394287109, weight_update_time: 0.5188428760, other_time: 14.7362060547
epoch time: 103.7989425659s, input_fired_time: 1.4423197508, output_fired_time: 0.0000000000, propagate_forward_time: 99.8170852661, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5395355225
validation[          , genome  5798] predictions:    6547/  10000 (65.47%), best:    6547/10000 (65.47%), error:     10078.60156, best error:     10078.60156 on epoch:    26, epoch:   26/50, mu: 0.8369228840, learning_rate: 0.0001613480, weight_decay: 0.0000765750

epoch time: 1274.2596435547s, input_fired_time: 20.7142219543, output_fired_time: 8.2570466995, propagate_forward_time: 495.5359802246, propagate_backward_time: 735.0305786133, weight_update_time: 0.5195892453, other_time: 14.7217407227
epoch time: 103.8141403198s, input_fired_time: 1.4497156143, output_fired_time: 0.0000000000, propagate_forward_time: 99.8268890381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5375366211
validation[          , genome  5798] predictions:    6481/  10000 (64.81%), best:    6481/10000 (64.81%), error:     10040.50391, best error:     10040.50391 on epoch:    27, epoch:   27/50, mu: 0.8435603976, learning_rate: 0.0001475791, weight_decay: 0.0000716849

epoch time: 1274.9074707031s, input_fired_time: 20.7205047607, output_fired_time: 8.2616682053, propagate_forward_time: 495.6948547363, propagate_backward_time: 735.4792480469, weight_update_time: 0.5203602314, other_time: 14.7512207031
epoch time: 103.9863281250s, input_fired_time: 1.4543676376, output_fired_time: 0.0000000000, propagate_forward_time: 99.9786453247, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5533142090
validation[          , genome  5798] predictions:    6556/  10000 (65.56%), best:    6556/10000 (65.56%), error:     10016.32910, best error:     10016.32910 on epoch:    28, epoch:   28/50, mu: 0.8499100804, learning_rate: 0.0001349853, weight_decay: 0.0000671070

epoch time: 1274.3405761719s, input_fired_time: 20.7184925079, output_fired_time: 8.2509908676, propagate_forward_time: 495.5696716309, propagate_backward_time: 735.0809936523, weight_update_time: 0.5186107755, other_time: 14.7203979492
epoch time: 103.8130264282s, input_fired_time: 1.4481060505, output_fired_time: 0.0000000000, propagate_forward_time: 99.8247756958, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5401458740
validation[          , genome  5798] predictions:    6557/  10000 (65.57%), best:    6557/10000 (65.57%), error:      9875.64941, best error:      9875.64941 on epoch:    29, epoch:   29/50, mu: 0.8559844494, learning_rate: 0.0001234661, weight_decay: 0.0000628214

epoch time: 1274.1934814453s, input_fired_time: 20.7214813232, output_fired_time: 8.2503576279, propagate_forward_time: 495.4835205078, propagate_backward_time: 735.0140380859, weight_update_time: 0.5231255293, other_time: 14.7241210938
epoch time: 103.9131927490s, input_fired_time: 1.4520206451, output_fired_time: 0.0000000000, propagate_forward_time: 99.9197311401, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5414428711
validation[          , genome  5798] predictions:    6577/  10000 (65.77%), best:    6577/10000 (65.77%), error:      9837.05859, best error:      9837.05859 on epoch:    30, epoch:   30/50, mu: 0.8617954254, learning_rate: 0.0001129300, weight_decay: 0.0000588096

epoch time: 1274.9113769531s, input_fired_time: 20.7344703674, output_fired_time: 8.2554092407, propagate_forward_time: 495.6818237305, propagate_backward_time: 735.5221557617, weight_update_time: 0.5222145915, other_time: 14.7175292969
epoch time: 103.8159637451s, input_fired_time: 1.4467072487, output_fired_time: 0.0000000000, propagate_forward_time: 99.8239517212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5453033447
validation[          , genome  5798] predictions:    6632/  10000 (66.32%), best:    6632/10000 (66.32%), error:      9804.88184, best error:      9804.88184 on epoch:    31, epoch:   31/50, mu: 0.8673544526, learning_rate: 0.0001032930, weight_decay: 0.0000550539

epoch time: 1275.1810302734s, input_fired_time: 20.7522621155, output_fired_time: 8.2453536987, propagate_forward_time: 495.7972106934, propagate_backward_time: 735.6655883789, weight_update_time: 0.5251201391, other_time: 14.7205200195
epoch time: 103.9722213745s, input_fired_time: 1.4627180099, output_fired_time: 0.0000000000, propagate_forward_time: 99.9585494995, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5509567261
validation[          , genome  5798] predictions:    6556/  10000 (65.56%), best:    6632/10000 (66.32%), error:      9913.47656, best error:      9804.88184 on epoch:    31, epoch:   32/50, mu: 0.8726724386, learning_rate: 0.0000944783, weight_decay: 0.0000515381

epoch time: 1275.4614257813s, input_fired_time: 20.7730274200, output_fired_time: 8.2456932068, propagate_forward_time: 495.9051208496, propagate_backward_time: 735.8181762695, weight_update_time: 0.5246127844, other_time: 14.7192993164
epoch time: 103.8127746582s, input_fired_time: 1.4534691572, output_fired_time: 0.0000000000, propagate_forward_time: 99.8127593994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5465469360
validation[          , genome  5798] predictions:    6618/  10000 (66.18%), best:    6618/10000 (66.18%), error:      9795.63184, best error:      9795.63184 on epoch:    33, epoch:   33/50, mu: 0.8777598143, learning_rate: 0.0000864159, weight_decay: 0.0000482468

epoch time: 1275.0706787109s, input_fired_time: 20.7637977600, output_fired_time: 8.2477951050, propagate_forward_time: 495.8601074219, propagate_backward_time: 735.4635620117, weight_update_time: 0.5270750523, other_time: 14.7354125977
epoch time: 103.9081115723s, input_fired_time: 1.4605635405, output_fired_time: 0.0000000000, propagate_forward_time: 99.8904418945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5571060181
validation[          , genome  5798] predictions:    6594/  10000 (65.94%), best:    6618/10000 (66.18%), error:      9909.23926, best error:      9795.63184 on epoch:    33, epoch:   34/50, mu: 0.8826265931, learning_rate: 0.0000790415, weight_decay: 0.0000451657

epoch time: 1275.1806640625s, input_fired_time: 20.7738857269, output_fired_time: 8.2564506531, propagate_forward_time: 495.7550659180, propagate_backward_time: 735.6637573242, weight_update_time: 0.5278139710, other_time: 14.7314453125
epoch time: 104.0939941406s, input_fired_time: 1.4670207500, output_fired_time: 0.0000000000, propagate_forward_time: 100.0827407837, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5442352295
validation[          , genome  5798] predictions:    6656/  10000 (66.56%), best:    6656/10000 (66.56%), error:      9646.43066, best error:      9646.43066 on epoch:    35, epoch:   35/50, mu: 0.8872823715, learning_rate: 0.0000722964, weight_decay: 0.0000422813

epoch time: 1276.5177001953s, input_fired_time: 20.8037986755, output_fired_time: 8.2659034729, propagate_forward_time: 496.1484985352, propagate_backward_time: 736.5464477539, weight_update_time: 0.5299405456, other_time: 14.7530517578
epoch time: 104.0146636963s, input_fired_time: 1.4585413933, output_fired_time: 0.0000000000, propagate_forward_time: 100.0075988770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5485229492
validation[          , genome  5798] predictions:    6595/  10000 (65.95%), best:    6656/10000 (66.56%), error:      9907.25586, best error:      9646.43066 on epoch:    35, epoch:   36/50, mu: 0.8917362690, learning_rate: 0.0000661269, weight_decay: 0.0000395812

epoch time: 1276.5004882813s, input_fired_time: 20.8081264496, output_fired_time: 8.2525587082, propagate_forward_time: 496.2703857422, propagate_backward_time: 736.4118652344, weight_update_time: 0.5319908857, other_time: 14.7575683594
epoch time: 104.0754699707s, input_fired_time: 1.4710952044, output_fired_time: 0.0000000000, propagate_forward_time: 100.0564651489, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5479125977
validation[          , genome  5798] predictions:    6593/  10000 (65.93%), best:    6656/10000 (66.56%), error:      9855.19727, best error:      9646.43066 on epoch:    35, epoch:   37/50, mu: 0.8959970474, learning_rate: 0.0000604839, weight_decay: 0.0000370535

epoch time: 1276.4365234375s, input_fired_time: 20.8274707794, output_fired_time: 8.2612295151, propagate_forward_time: 496.3005676270, propagate_backward_time: 736.2771606445, weight_update_time: 0.5358965993, other_time: 14.7700805664
epoch time: 103.9372177124s, input_fired_time: 1.4666523933, output_fired_time: 0.0000000000, propagate_forward_time: 99.9224319458, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5481338501
validation[          , genome  5798] predictions:    6669/  10000 (66.69%), best:    6656/10000 (66.56%), error:      9659.76953, best error:      9646.43066 on epoch:    35, epoch:   38/50, mu: 0.9000730515, learning_rate: 0.0000553224, weight_decay: 0.0000346872

epoch time: 1276.6003417969s, input_fired_time: 20.8304615021, output_fired_time: 8.2607231140, propagate_forward_time: 496.3550720215, propagate_backward_time: 736.4097900391, weight_update_time: 0.5373253822, other_time: 14.7442626953
epoch time: 104.0917587280s, input_fired_time: 1.4723762274, output_fired_time: 0.0000000000, propagate_forward_time: 100.0693130493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5500717163
validation[          , genome  5798] predictions:    6513/  10000 (65.13%), best:    6656/10000 (66.56%), error:     10138.04492, best error:      9646.43066 on epoch:    35, epoch:   39/50, mu: 0.9039723277, learning_rate: 0.0000506014, weight_decay: 0.0000324720

epoch time: 1277.1318359375s, input_fired_time: 20.8410644531, output_fired_time: 8.2649879456, propagate_forward_time: 496.6330261230, propagate_backward_time: 736.6436767578, weight_update_time: 0.5397869349, other_time: 14.7490234375
epoch time: 104.1414108276s, input_fired_time: 1.4734854698, output_fired_time: 0.0000000000, propagate_forward_time: 100.1061782837, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5617446899
validation[          , genome  5798] predictions:    6706/  10000 (67.06%), best:    6706/10000 (67.06%), error:      9553.16504, best error:      9553.16504 on epoch:    40, epoch:   40/50, mu: 0.9077025652, learning_rate: 0.0000462833, weight_decay: 0.0000303983

epoch time: 1275.7342529297s, input_fired_time: 20.8328590393, output_fired_time: 8.2648086548, propagate_forward_time: 495.9151000977, propagate_backward_time: 735.9617309570, weight_update_time: 0.5419303775, other_time: 14.7597656250
epoch time: 103.9883804321s, input_fired_time: 1.4789835215, output_fired_time: 0.0000000000, propagate_forward_time: 99.9424362183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5669631958
validation[          , genome  5798] predictions:    6681/  10000 (66.81%), best:    6706/10000 (67.06%), error:      9654.90918, best error:      9553.16504 on epoch:    40, epoch:   41/50, mu: 0.9112710357, learning_rate: 0.0000423336, weight_decay: 0.0000284570

epoch time: 1276.7894287109s, input_fired_time: 20.8584289551, output_fired_time: 8.2690782547, propagate_forward_time: 496.4750061035, propagate_backward_time: 736.4262695313, weight_update_time: 0.5439840555, other_time: 14.7607421875
epoch time: 104.0693588257s, input_fired_time: 1.4687254429, output_fired_time: 0.0000000000, propagate_forward_time: 100.0440139771, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5566177368
validation[          , genome  5798] predictions:    6666/  10000 (66.66%), best:    6706/10000 (67.06%), error:      9767.38574, best error:      9553.16504 on epoch:    40, epoch:   42/50, mu: 0.9146847725, learning_rate: 0.0000387210, weight_decay: 0.0000266397

epoch time: 1274.9313964844s, input_fired_time: 20.8327274323, output_fired_time: 8.2496557236, propagate_forward_time: 495.8231201172, propagate_backward_time: 735.2814331055, weight_update_time: 0.5477330685, other_time: 14.7444458008
epoch time: 103.7423934937s, input_fired_time: 1.4665236473, output_fired_time: 0.0000000000, propagate_forward_time: 99.7392349243, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5366363525
validation[          , genome  5798] predictions:    6730/  10000 (67.30%), best:    6730/10000 (67.30%), error:      9526.05664, best error:      9526.05664 on epoch:    43, epoch:   43/50, mu: 0.9179505110, learning_rate: 0.0000354167, weight_decay: 0.0000249385

epoch time: 1274.2144775391s, input_fired_time: 20.8279857635, output_fired_time: 8.2598810196, propagate_forward_time: 495.3285522461, propagate_backward_time: 735.0316772461, weight_update_time: 0.5496124029, other_time: 14.7663574219
epoch time: 103.9027709961s, input_fired_time: 1.4732223749, output_fired_time: 0.0000000000, propagate_forward_time: 99.8830108643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5465393066
validation[          , genome  5798] predictions:    6748/  10000 (67.48%), best:    6748/10000 (67.48%), error:      9512.60840, best error:      9512.60840 on epoch:    44, epoch:   44/50, mu: 0.9210746288, learning_rate: 0.0000323944, weight_decay: 0.0000233459

epoch time: 1274.5245361328s, input_fired_time: 20.8304347992, output_fired_time: 8.2478446960, propagate_forward_time: 495.7574462891, propagate_backward_time: 734.9521484375, weight_update_time: 0.5515531898, other_time: 14.7366943359
epoch time: 103.8745574951s, input_fired_time: 1.4768996239, output_fired_time: 0.0000000000, propagate_forward_time: 99.8440856934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5535736084
validation[          , genome  5798] predictions:    6761/  10000 (67.61%), best:    6761/10000 (67.61%), error:      9474.85254, best error:      9474.85254 on epoch:    45, epoch:   45/50, mu: 0.9240632653, learning_rate: 0.0000296300, weight_decay: 0.0000218550

epoch time: 1275.2044677734s, input_fired_time: 20.8542175293, output_fired_time: 8.2770624161, propagate_forward_time: 495.8305053711, propagate_backward_time: 735.4723510742, weight_update_time: 0.5569885969, other_time: 14.7702636719
epoch time: 103.9187316895s, input_fired_time: 1.4836397171, output_fired_time: 0.0000000000, propagate_forward_time: 99.8796997070, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5553894043
validation[          , genome  5798] predictions:    6692/  10000 (66.92%), best:    6761/10000 (67.61%), error:      9581.42969, best error:      9474.85254 on epoch:    45, epoch:   46/50, mu: 0.9269223213, learning_rate: 0.0000271015, weight_decay: 0.0000204593

epoch time: 1274.4724121094s, input_fired_time: 20.8453903198, output_fired_time: 8.2643432617, propagate_forward_time: 495.4359130859, propagate_backward_time: 735.1734619141, weight_update_time: 0.5594082475, other_time: 14.7534179688
epoch time: 103.8910598755s, input_fired_time: 1.4791507721, output_fired_time: 0.0000000000, propagate_forward_time: 99.8634338379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5484771729
validation[          , genome  5798] predictions:    6694/  10000 (66.94%), best:    6761/10000 (67.61%), error:      9584.83594, best error:      9474.85254 on epoch:    45, epoch:   47/50, mu: 0.9296573997, learning_rate: 0.0000247887, weight_decay: 0.0000191527

epoch time: 1274.9536132813s, input_fired_time: 20.8563327789, output_fired_time: 8.2435417175, propagate_forward_time: 495.6616210938, propagate_backward_time: 735.4150390625, weight_update_time: 0.5613912940, other_time: 14.7770996094
epoch time: 103.9918594360s, input_fired_time: 1.4760645628, output_fired_time: 0.0000000000, propagate_forward_time: 99.9641342163, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5516586304
validation[          , genome  5798] predictions:    6750/  10000 (67.50%), best:    6761/10000 (67.61%), error:      9568.23926, best error:      9474.85254 on epoch:    45, epoch:   48/50, mu: 0.9322738647, learning_rate: 0.0000226733, weight_decay: 0.0000179296

epoch time: 1275.3609619141s, input_fired_time: 20.8637638092, output_fired_time: 8.2598886490, propagate_forward_time: 495.8334350586, propagate_backward_time: 735.6027221680, weight_update_time: 0.5668475628, other_time: 14.8011474609
epoch time: 103.8962326050s, input_fired_time: 1.4745616913, output_fired_time: 0.0000000000, propagate_forward_time: 99.8761978149, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5454711914
validation[          , genome  5798] predictions:    6692/  10000 (66.92%), best:    6761/10000 (67.61%), error:      9613.25684, best error:      9474.85254 on epoch:    45, epoch:   49/50, mu: 0.9347769022, learning_rate: 0.0000207385, weight_decay: 0.0000167846

epoch time: 1275.9658203125s, input_fired_time: 20.8852348328, output_fired_time: 8.2543916702, propagate_forward_time: 496.0299682617, propagate_backward_time: 735.9983520508, weight_update_time: 0.5702307820, other_time: 14.7978515625
epoch time: 104.0635986328s, input_fired_time: 1.4849091768, output_fired_time: 0.0000000000, propagate_forward_time: 100.0292816162, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5494079590
validation[          , genome  5798] predictions:    6744/  10000 (67.44%), best:    6744/10000 (67.44%), error:      9446.01855, best error:      9446.01855 on epoch:    50, epoch:   50/50, mu: 0.9371713996, learning_rate: 0.0000189687, weight_decay: 0.0000157127

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 516.0822753906s, input_fired_time: 7.3739614487, output_fired_time: 0.0000000000, propagate_forward_time: 495.9174194336, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 12.7908935547
best training[          , genome  5798] predictions:   36972/  50000 (73.94%), best:    6744/10000 (67.44%), error:     39619.22266, best error:      9446.01855 on epoch:    50, epoch:   51/50, mu: 0.9394620657, learning_rate: 0.0000173500, weight_decay: 0.0000147093
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 104.2469940186s, input_fired_time: 1.4881062508, output_fired_time: 0.0000000000, propagate_forward_time: 100.2017593384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5571289063
   testing[          , genome  5798] predictions:    6744/  10000 (67.44%), best:    6744/10000 (67.44%), error:      9446.01855, best error:      9446.01855 on epoch:    50, epoch:   51/50, mu: 0.9394620657, learning_rate: 0.0000173500, weight_decay: 0.0000147093
backpropagation finished successfully!
22:13:15 (2572): called boinc_finish(0)

</stderr_txt>
]]>