Nameexact_genome_1542605141_186_6310_2
Workunit4098414
Created19 Nov 2018, 7:29:45 UTC
Sent19 Nov 2018, 8:03:55 UTC
Report deadline2 Feb 2019, 8:26:31 UTC
Received23 Nov 2018, 2:05:56 UTC
Server stateOver
OutcomeComputation error
Client stateAborted by user
Exit status203 (0xcb) EXIT_ABORTED_VIA_GUI
Computer ID92895
Run time23 hours 17 min 56 sec
CPU time23 hours 6 min 53 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS2.94 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size525.92 MB
Peak swap size524.19 MB
Peak disk usage9.36 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_186_6310.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542605141_186_6310_2_r648354266_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1542605141_186_6310.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.514966
read mu: 0.514966
read mu_delta: 0.950388
read initial_learning_rate: 1e-05
read learning_rate: 1e-05
read learning_rate_delta: 0.933987
read initial_weight_decay: 0.00055041
read weight_decay: 0.00055041
read weight_decay_delta: 0.947923
read batch_size: 96
read epsilon: 1e-07
read alpha: 0.0893167
read input_dropout_probability: 0.00124555
read hidden_dropout_probability: 0.103381
read velocity_reset: 1757
read epoch: 0
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6310
read normal distribution: '0 1.00602424144745 -0.537872135639191'
generator_str: '1979674064'
read generator: 1979674064
read generated_by_map:
1 add_edge 1
reading 66 nodes.
reading 251 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1417850677
post shuffle 1: 881882028
epoch time: 1570.38879394531s, input_fired_time: 15.6892433166504, output_fired_time: 8.76891803741455, propagate_forward_time: 623.010375976563, propagate_backward_time: 908.042053222656, weight_update_time: 0.925469815731049, other_time: 14.8782348632813
epoch time: 142.860397338867s, input_fired_time: 1.43775916099548, output_fired_time: 0, propagate_forward_time: 138.712112426758, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.71052551269531
validation[          , genome  6310] predictions:    6037/  10000 (60.37%), best:    6037/10000 (60.37%), error:     11815.21875, best error:     11815.21875 on epoch:     0, epoch:    0/100, mu: 0.5149655342, learning_rate: 0.0000100000, weight_decay: 0.0005504102

epoch time: 1660.7043457031s, input_fired_time: 16.3252029419, output_fired_time: 9.0733079910, propagate_forward_time: 660.6869506836, propagate_backward_time: 958.8229370117, weight_update_time: 0.9999668598, other_time: 15.7958984375
epoch time: 130.0352630615s, input_fired_time: 1.3422101736, output_fired_time: 0.0000000000, propagate_forward_time: 126.1950607300, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4979934692
validation[          , genome  6310] predictions:    5959/  10000 (59.59%), best:    6037/10000 (60.37%), error:     12492.02539, best error:     11815.21875 on epoch:     0, epoch:    1/100, mu: 0.5385327339, learning_rate: 0.0000093399, weight_decay: 0.0005217465

epoch time: 1613.0103759766s, input_fired_time: 15.9459886551, output_fired_time: 9.1728544235, propagate_forward_time: 637.8553466797, propagate_backward_time: 934.7194824219, weight_update_time: 0.9484159350, other_time: 15.3166503906
epoch time: 131.0458068848s, input_fired_time: 1.3421336412, output_fired_time: 0.0000000000, propagate_forward_time: 127.2177581787, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4859161377
validation[          , genome  6310] predictions:    5865/  10000 (58.65%), best:    6037/10000 (60.37%), error:     13280.06152, best error:     11815.21875 on epoch:     0, epoch:    2/100, mu: 0.5609307289, learning_rate: 0.0000087233, weight_decay: 0.0004945756

epoch time: 1589.7941894531s, input_fired_time: 15.7718353271, output_fired_time: 8.8035020828, propagate_forward_time: 631.5010986328, propagate_backward_time: 918.6463012695, weight_update_time: 0.9776927233, other_time: 15.0714721680
epoch time: 129.6743011475s, input_fired_time: 1.3533877134, output_fired_time: 0.0000000000, propagate_forward_time: 125.8453750610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4755325317
validation[          , genome  6310] predictions:    5697/  10000 (56.97%), best:    6037/10000 (60.37%), error:     14071.13184, best error:     11815.21875 on epoch:     0, epoch:    3/100, mu: 0.5822175145, learning_rate: 0.0000081475, weight_decay: 0.0004688196

epoch time: 1663.2443847656s, input_fired_time: 16.2648181915, output_fired_time: 9.1630859375, propagate_forward_time: 660.0169677734, propagate_backward_time: 961.9923706055, weight_update_time: 1.0429428816, other_time: 15.8071899414
epoch time: 120.0412902832s, input_fired_time: 1.2497378588, output_fired_time: 0.0000000000, propagate_forward_time: 116.4121704102, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3793792725
validation[          , genome  6310] predictions:    5621/  10000 (56.21%), best:    6037/10000 (60.37%), error:     14724.49707, best error:     11815.21875 on epoch:     0, epoch:    4/100, mu: 0.6024482250, learning_rate: 0.0000076096, weight_decay: 0.0004444050

epoch time: 1461.0002441406s, input_fired_time: 14.8862390518, output_fired_time: 8.0724506378, propagate_forward_time: 577.5052490234, propagate_backward_time: 846.2081909180, weight_update_time: 0.8713797927, other_time: 14.3280639648
epoch time: 120.0382995605s, input_fired_time: 1.2363885641, output_fired_time: 0.0000000000, propagate_forward_time: 116.4210662842, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3808441162
validation[          , genome  6310] predictions:    5453/  10000 (54.53%), best:    6037/10000 (60.37%), error:     15373.82910, best error:     11815.21875 on epoch:     0, epoch:    5/100, mu: 0.6216752529, learning_rate: 0.0000071073, weight_decay: 0.0004212618

epoch time: 1461.0225830078s, input_fired_time: 14.8719301224, output_fired_time: 8.0993070602, propagate_forward_time: 577.0307617188, propagate_backward_time: 846.6992797852, weight_update_time: 0.8809873462, other_time: 14.3212280273
epoch time: 119.6371765137s, input_fired_time: 1.2400650978, output_fired_time: 0.0000000000, propagate_forward_time: 116.0022735596, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3948364258
validation[          , genome  6310] predictions:    5222/  10000 (52.22%), best:    6037/10000 (60.37%), error:     15863.29199, best error:     11815.21875 on epoch:     0, epoch:    6/100, mu: 0.6399484277, learning_rate: 0.0000066381, weight_decay: 0.0003993238

epoch time: 1469.8848876953s, input_fired_time: 14.8408412933, output_fired_time: 8.1112318039, propagate_forward_time: 579.9490966797, propagate_backward_time: 852.5971679688, weight_update_time: 0.8787827492, other_time: 14.3865966797
epoch time: 119.5567474365s, input_fired_time: 1.2184129953, output_fired_time: 0.0000000000, propagate_forward_time: 115.9415130615, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3968200684
validation[          , genome  6310] predictions:    5063/  10000 (50.63%), best:    6037/10000 (60.37%), error:     16269.79980, best error:     11815.21875 on epoch:     0, epoch:    7/100, mu: 0.6573150158, learning_rate: 0.0000061999, weight_decay: 0.0003785282

epoch time: 1451.0634765625s, input_fired_time: 14.6971282959, output_fired_time: 8.0111160278, propagate_forward_time: 572.2234497070, propagate_backward_time: 841.9102172852, weight_update_time: 0.8661445379, other_time: 14.2215576172
epoch time: 113.8460388184s, input_fired_time: 1.1567238569, output_fired_time: 0.0000000000, propagate_forward_time: 110.3307571411, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3585586548
validation[          , genome  6310] predictions:    4950/  10000 (49.50%), best:    6037/10000 (60.37%), error:     16563.89648, best error:     11815.21875 on epoch:     0, epoch:    8/100, mu: 0.6738200188, learning_rate: 0.0000057906, weight_decay: 0.0003588157

epoch time: 1391.0683593750s, input_fired_time: 14.3140182495, output_fired_time: 7.7692604065, propagate_forward_time: 546.2278442383, propagate_backward_time: 808.7433471680, weight_update_time: 0.8562631607, other_time: 14.0139160156
epoch time: 113.6176528931s, input_fired_time: 1.1419363022, output_fired_time: 0.0000000000, propagate_forward_time: 110.1543960571, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3213195801
validation[          , genome  6310] predictions:    4858/  10000 (48.58%), best:    6037/10000 (60.37%), error:     16849.00781, best error:     11815.21875 on epoch:     0, epoch:    9/100, mu: 0.6895062327, learning_rate: 0.0000054084, weight_decay: 0.0003401297

epoch time: 1391.8117675781s, input_fired_time: 14.2696838379, output_fired_time: 7.6884913445, propagate_forward_time: 546.7364501953, propagate_backward_time: 809.1002807617, weight_update_time: 0.8496072888, other_time: 14.0169067383
epoch time: 113.1341247559s, input_fired_time: 1.1272985935, output_fired_time: 0.0000000000, propagate_forward_time: 109.6969833374, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3098449707
validation[          , genome  6310] predictions:    4704/  10000 (47.04%), best:    6037/10000 (60.37%), error:     17015.01758, best error:     11815.21875 on epoch:     0, epoch:   10/100, mu: 0.7044141889, learning_rate: 0.0000050513, weight_decay: 0.0003224168

epoch time: 1390.7684326172s, input_fired_time: 14.2290143967, output_fired_time: 7.7058000565, propagate_forward_time: 546.6989746094, propagate_backward_time: 808.1307983398, weight_update_time: 0.8487476707, other_time: 14.0038452148
epoch time: 112.8170547485s, input_fired_time: 1.1115235090, output_fired_time: 0.0000000000, propagate_forward_time: 109.3736114502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3319168091
validation[          , genome  6310] predictions:    4724/  10000 (47.24%), best:    6037/10000 (60.37%), error:     17111.63281, best error:     11815.21875 on epoch:     0, epoch:   11/100, mu: 0.7185825706, learning_rate: 0.0000047179, weight_decay: 0.0003056263

epoch time: 1389.5367431641s, input_fired_time: 14.1529302597, output_fired_time: 7.6430621147, propagate_forward_time: 546.6064453125, propagate_backward_time: 807.1112670898, weight_update_time: 0.8488662839, other_time: 14.0230102539
epoch time: 113.2614669800s, input_fired_time: 1.1083776951, output_fired_time: 0.0000000000, propagate_forward_time: 109.8348464966, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3182449341
validation[          , genome  6310] predictions:    4607/  10000 (46.07%), best:    6037/10000 (60.37%), error:     17241.25977, best error:     11815.21875 on epoch:     0, epoch:   12/100, mu: 0.7320480347, learning_rate: 0.0000044064, weight_decay: 0.0002897102

epoch time: 1388.3143310547s, input_fired_time: 14.0686216354, output_fired_time: 7.6847991943, propagate_forward_time: 545.2059936523, propagate_backward_time: 807.3534545898, weight_update_time: 0.8578261733, other_time: 14.0014648438
epoch time: 113.0534820557s, input_fired_time: 1.1023330688, output_fired_time: 0.0000000000, propagate_forward_time: 109.6205978394, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3305511475
validation[          , genome  6310] predictions:    4623/  10000 (46.23%), best:    6037/10000 (60.37%), error:     17309.84375, best error:     11815.21875 on epoch:     0, epoch:   13/100, mu: 0.7448454499, learning_rate: 0.0000041156, weight_decay: 0.0002746230

epoch time: 1386.5855712891s, input_fired_time: 14.0146131516, output_fired_time: 7.6936283112, propagate_forward_time: 544.9780273438, propagate_backward_time: 805.9498901367, weight_update_time: 0.8444768786, other_time: 13.9494018555
epoch time: 113.6427536011s, input_fired_time: 1.0927848816, output_fired_time: 0.0000000000, propagate_forward_time: 110.2207870483, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3291854858
validation[          , genome  6310] predictions:    4540/  10000 (45.40%), best:    6037/10000 (60.37%), error:     17340.52148, best error:     11815.21875 on epoch:     0, epoch:   14/100, mu: 0.7570079565, learning_rate: 0.0000038439, weight_decay: 0.0002603215

epoch time: 1387.2861328125s, input_fired_time: 13.9866161346, output_fired_time: 7.6326994896, propagate_forward_time: 545.0272216797, propagate_backward_time: 806.6313476563, weight_update_time: 0.8416947722, other_time: 14.0083007813
epoch time: 113.2007217407s, input_fired_time: 1.0815486908, output_fired_time: 0.0000000000, propagate_forward_time: 109.7911453247, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3280258179
validation[          , genome  6310] predictions:    4476/  10000 (44.76%), best:    6037/10000 (60.37%), error:     17292.32617, best error:     11815.21875 on epoch:     0, epoch:   15/100, mu: 0.7685670853, learning_rate: 0.0000035901, weight_decay: 0.0002467648

epoch time: 1389.2451171875s, input_fired_time: 13.9564065933, output_fired_time: 7.6865119934, propagate_forward_time: 545.9547119141, propagate_backward_time: 807.6589965820, weight_update_time: 0.8487403393, other_time: 13.9884643555
epoch time: 113.1743850708s, input_fired_time: 1.0735297203, output_fired_time: 0.0000000000, propagate_forward_time: 109.7757492065, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3251037598
validation[          , genome  6310] predictions:    4452/  10000 (44.52%), best:    6037/10000 (60.37%), error:     17321.65625, best error:     11815.21875 on epoch:     0, epoch:   16/100, mu: 0.7795526981, learning_rate: 0.0000033531, weight_decay: 0.0002339141

epoch time: 1388.3745117188s, input_fired_time: 13.9556941986, output_fired_time: 7.7181310654, propagate_forward_time: 545.7079467773, propagate_backward_time: 807.0541992188, weight_update_time: 0.8444300890, other_time: 13.9385375977
epoch time: 112.7669906616s, input_fired_time: 1.0580457449, output_fired_time: 0.0000000000, propagate_forward_time: 109.3517074585, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3572387695
validation[          , genome  6310] predictions:    4419/  10000 (44.19%), best:    6037/10000 (60.37%), error:     17304.50195, best error:     11815.21875 on epoch:     0, epoch:   17/100, mu: 0.7899933457, learning_rate: 0.0000031318, weight_decay: 0.0002217326

epoch time: 1386.7475585938s, input_fired_time: 13.8899850845, output_fired_time: 7.6441473961, propagate_forward_time: 544.4357299805, propagate_backward_time: 806.7807006836, weight_update_time: 0.8401252031, other_time: 13.9969482422
epoch time: 113.5937881470s, input_fired_time: 1.0829188824, output_fired_time: 0.0000000000, propagate_forward_time: 110.1887130737, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3221588135
validation[          , genome  6310] predictions:    4500/  10000 (45.00%), best:    6037/10000 (60.37%), error:     17294.79102, best error:     11815.21875 on epoch:     0, epoch:   18/100, mu: 0.7999159694, learning_rate: 0.0000029250, weight_decay: 0.0002101854

epoch time: 1386.8229980469s, input_fired_time: 13.8799734116, output_fired_time: 7.6635241508, propagate_forward_time: 545.0739135742, propagate_backward_time: 806.2439575195, weight_update_time: 0.8447068930, other_time: 13.9615478516
epoch time: 113.4878234863s, input_fired_time: 1.0640709400, output_fired_time: 0.0000000000, propagate_forward_time: 110.0960388184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3277130127
validation[          , genome  6310] predictions:    4540/  10000 (45.40%), best:    6037/10000 (60.37%), error:     17276.86914, best error:     11815.21875 on epoch:     0, epoch:   19/100, mu: 0.8093463182, learning_rate: 0.0000027320, weight_decay: 0.0001992396

epoch time: 1385.8068847656s, input_fired_time: 13.8598394394, output_fired_time: 7.6636071205, propagate_forward_time: 544.1502075195, propagate_backward_time: 806.1776123047, weight_update_time: 0.8449937105, other_time: 13.9556274414
epoch time: 112.9736404419s, input_fired_time: 1.0579767227, output_fired_time: 0.0000000000, propagate_forward_time: 109.5994644165, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3162002563
validation[          , genome  6310] predictions:    4550/  10000 (45.50%), best:    6037/10000 (60.37%), error:     17193.71680, best error:     11815.21875 on epoch:     0, epoch:   20/100, mu: 0.8183088303, learning_rate: 0.0000025516, weight_decay: 0.0001888638

epoch time: 1388.0756835938s, input_fired_time: 13.8618316650, output_fired_time: 7.7077994347, propagate_forward_time: 545.6102905273, propagate_backward_time: 806.8871459961, weight_update_time: 0.8491370678, other_time: 14.0086669922
epoch time: 113.4369659424s, input_fired_time: 1.0530650616, output_fired_time: 0.0000000000, propagate_forward_time: 110.0680694580, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3158340454
validation[          , genome  6310] predictions:    4371/  10000 (43.71%), best:    6037/10000 (60.37%), error:     17292.40820, best error:     11815.21875 on epoch:     0, epoch:   21/100, mu: 0.8268266916, learning_rate: 0.0000023832, weight_decay: 0.0001790284

epoch time: 1357.0167236328s, input_fired_time: 13.7149209976, output_fired_time: 7.5370306969, propagate_forward_time: 532.7801513672, propagate_backward_time: 789.0947875977, weight_update_time: 0.8408380151, other_time: 13.8898315430
epoch time: 106.7415084839s, input_fired_time: 1.0086801052, output_fired_time: 0.0000000000, propagate_forward_time: 103.4452438354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2875823975
validation[          , genome  6310] predictions:    4521/  10000 (45.21%), best:    6037/10000 (60.37%), error:     17196.31250, best error:     11815.21875 on epoch:     0, epoch:   22/100, mu: 0.8349219561, learning_rate: 0.0000022259, weight_decay: 0.0001697052

epoch time: 1319.0270996094s, input_fired_time: 13.5424442291, output_fired_time: 7.3021807671, propagate_forward_time: 516.4398193359, propagate_backward_time: 768.0117187500, weight_update_time: 0.8233551979, other_time: 13.7309570313
epoch time: 107.6676940918s, input_fired_time: 1.0103665590, output_fired_time: 0.0000000000, propagate_forward_time: 104.3887863159, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2685394287
validation[          , genome  6310] predictions:    4481/  10000 (44.81%), best:    6037/10000 (60.37%), error:     17161.58203, best error:     11815.21875 on epoch:     0, epoch:   23/100, mu: 0.8426156044, learning_rate: 0.0000020789, weight_decay: 0.0001608675

epoch time: 1318.4350585938s, input_fired_time: 13.5273218155, output_fired_time: 7.2738609314, propagate_forward_time: 516.2918701172, propagate_backward_time: 767.6731567383, weight_update_time: 0.8175680041, other_time: 13.6688842773
epoch time: 106.8489456177s, input_fired_time: 1.0048868656, output_fired_time: 0.0000000000, propagate_forward_time: 103.5739135742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2701416016
validation[          , genome  6310] predictions:    4521/  10000 (45.21%), best:    6037/10000 (60.37%), error:     17075.66406, best error:     11815.21875 on epoch:     0, epoch:   24/100, mu: 0.8499276042, learning_rate: 0.0000019417, weight_decay: 0.0001524900

epoch time: 1315.0693359375s, input_fired_time: 13.5156555176, output_fired_time: 7.3778862953, propagate_forward_time: 514.2807617188, propagate_backward_time: 766.2245483398, weight_update_time: 0.8160766363, other_time: 13.6704711914
epoch time: 106.6041412354s, input_fired_time: 1.0069611073, output_fired_time: 0.0000000000, propagate_forward_time: 103.3421707153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2550125122
validation[          , genome  6310] predictions:    4493/  10000 (44.93%), best:    6037/10000 (60.37%), error:     17033.49609, best error:     11815.21875 on epoch:     0, epoch:   25/100, mu: 0.8568767905, learning_rate: 0.0000018135, weight_decay: 0.0001445488

epoch time: 1318.0866699219s, input_fired_time: 13.4848995209, output_fired_time: 7.3305311203, propagate_forward_time: 514.2291259766, propagate_backward_time: 769.3864746094, weight_update_time: 0.8210644722, other_time: 13.6556396484
epoch time: 106.7936859131s, input_fired_time: 1.0083585978, output_fired_time: 0.0000000000, propagate_forward_time: 103.5013504028, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2839736938
validation[          , genome  6310] predictions:    4483/  10000 (44.83%), best:    6037/10000 (60.37%), error:     17062.39453, best error:     11815.21875 on epoch:     0, epoch:   26/100, mu: 0.8634812236, learning_rate: 0.0000016938, weight_decay: 0.0001370211

epoch time: 1317.1652832031s, input_fired_time: 13.5114927292, output_fired_time: 7.3378734589, propagate_forward_time: 514.2746582031, propagate_backward_time: 768.3582763672, weight_update_time: 0.8279354572, other_time: 13.6829833984
epoch time: 106.8014907837s, input_fired_time: 1.0051212311, output_fired_time: 0.0000000000, propagate_forward_time: 103.5207672119, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2756042480
validation[          , genome  6310] predictions:    4539/  10000 (45.39%), best:    6037/10000 (60.37%), error:     17008.03320, best error:     11815.21875 on epoch:     0, epoch:   27/100, mu: 0.8697580099, learning_rate: 0.0000015820, weight_decay: 0.0001298855

epoch time: 1318.1750488281s, input_fired_time: 13.5084238052, output_fired_time: 7.3536696434, propagate_forward_time: 515.6637573242, propagate_backward_time: 768.0138549805, weight_update_time: 0.8217107654, other_time: 13.6353759766
epoch time: 106.8358917236s, input_fired_time: 1.0092097521, output_fired_time: 0.0000000000, propagate_forward_time: 103.5246734619, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3020095825
validation[          , genome  6310] predictions:    4453/  10000 (44.53%), best:    6037/10000 (60.37%), error:     17029.47852, best error:     11815.21875 on epoch:     0, epoch:   28/100, mu: 0.8757234216, learning_rate: 0.0000014775, weight_decay: 0.0001231215

epoch time: 1316.3809814453s, input_fired_time: 13.4939346313, output_fired_time: 7.3129091263, propagate_forward_time: 515.3846435547, propagate_backward_time: 766.5522460938, weight_update_time: 0.8166205883, other_time: 13.6373291016
epoch time: 106.5290908813s, input_fired_time: 0.9982109666, output_fired_time: 0.0000000000, propagate_forward_time: 103.2693786621, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2614974976
validation[          , genome  6310] predictions:    4454/  10000 (44.54%), best:    6037/10000 (60.37%), error:     17033.64844, best error:     11815.21875 on epoch:     0, epoch:   29/100, mu: 0.8813928366, learning_rate: 0.0000013800, weight_decay: 0.0001167097

epoch time: 1320.3845214844s, input_fired_time: 13.4983396530, output_fired_time: 7.3080334663, propagate_forward_time: 516.6444702148, propagate_backward_time: 769.2380981445, weight_update_time: 0.8204038143, other_time: 13.6956787109
epoch time: 106.3047180176s, input_fired_time: 1.0000594854, output_fired_time: 0.0000000000, propagate_forward_time: 103.0235595703, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2810974121
validation[          , genome  6310] predictions:    4631/  10000 (46.31%), best:    6037/10000 (60.37%), error:     16954.08203, best error:     11815.21875 on epoch:     0, epoch:   30/100, mu: 0.8867810369, learning_rate: 0.0000012889, weight_decay: 0.0001106318

epoch time: 1317.5709228516s, input_fired_time: 13.5139331818, output_fired_time: 7.3290328979, propagate_forward_time: 515.6600341797, propagate_backward_time: 767.3729248047, weight_update_time: 0.8239999413, other_time: 13.6950683594
epoch time: 107.3521499634s, input_fired_time: 1.0082640648, output_fired_time: 0.0000000000, propagate_forward_time: 104.0866622925, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2572250366
validation[          , genome  6310] predictions:    4582/  10000 (45.82%), best:    6037/10000 (60.37%), error:     16955.04492, best error:     11815.21875 on epoch:     0, epoch:   31/100, mu: 0.8919019103, learning_rate: 0.0000012038, weight_decay: 0.0001048704

epoch time: 1322.3939208984s, input_fired_time: 13.4938421249, output_fired_time: 7.3031611443, propagate_forward_time: 517.4743652344, propagate_backward_time: 770.4349365234, weight_update_time: 0.8217690587, other_time: 13.6876220703
epoch time: 107.4014587402s, input_fired_time: 1.0117442608, output_fired_time: 0.0000000000, propagate_forward_time: 104.0883102417, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3014068604
validation[          , genome  6310] predictions:    4495/  10000 (44.95%), best:    6037/10000 (60.37%), error:     16916.43555, best error:     11815.21875 on epoch:     0, epoch:   32/100, mu: 0.8967686892, learning_rate: 0.0000011244, weight_decay: 0.0000994091

epoch time: 1323.4804687500s, input_fired_time: 13.5199718475, output_fired_time: 7.3149571419, propagate_forward_time: 517.8852539063, propagate_backward_time: 771.0415039063, weight_update_time: 0.8246070147, other_time: 13.7187500000
epoch time: 108.5457000732s, input_fired_time: 1.0158396959, output_fired_time: 0.0000000000, propagate_forward_time: 105.2306137085, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2992477417
validation[          , genome  6310] predictions:    4553/  10000 (45.53%), best:    6037/10000 (60.37%), error:     16882.28711, best error:     11815.21875 on epoch:     0, epoch:   33/100, mu: 0.9013940692, learning_rate: 0.0000010501, weight_decay: 0.0000942322

epoch time: 1321.8034667969s, input_fired_time: 13.5123758316, output_fired_time: 7.3550276756, propagate_forward_time: 516.7725830078, propagate_backward_time: 770.3865966797, weight_update_time: 0.8238735795, other_time: 13.7769775391
epoch time: 107.5213470459s, input_fired_time: 1.0054345131, output_fired_time: 0.0000000000, propagate_forward_time: 104.2537689209, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2621459961
validation[          , genome  6310] predictions:    4467/  10000 (44.67%), best:    6037/10000 (60.37%), error:     16902.63086, best error:     11815.21875 on epoch:     0, epoch:   34/100, mu: 0.9057899714, learning_rate: 0.0000009808, weight_decay: 0.0000893249

epoch time: 1321.2238769531s, input_fired_time: 13.5035991669, output_fired_time: 7.2804841995, propagate_forward_time: 517.2008056641, propagate_backward_time: 769.4629516602, weight_update_time: 0.8290285468, other_time: 13.7760620117
epoch time: 107.1953430176s, input_fired_time: 1.0091271400, output_fired_time: 0.0000000000, propagate_forward_time: 103.9285049438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2577133179
validation[          , genome  6310] predictions:    4556/  10000 (45.56%), best:    6037/10000 (60.37%), error:     16829.66211, best error:     11815.21875 on epoch:     0, epoch:   35/100, mu: 0.9099677801, learning_rate: 0.0000009161, weight_decay: 0.0000846731

epoch time: 1319.8129882813s, input_fired_time: 13.5024709702, output_fired_time: 7.3138251305, propagate_forward_time: 516.1528320313, propagate_backward_time: 769.0913696289, weight_update_time: 0.8304145932, other_time: 13.7525024414
epoch time: 106.8812332153s, input_fired_time: 1.0061410666, output_fired_time: 0.0000000000, propagate_forward_time: 103.6223449707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2527465820
validation[          , genome  6310] predictions:    4559/  10000 (45.59%), best:    6037/10000 (60.37%), error:     16870.07227, best error:     11815.21875 on epoch:     0, epoch:   36/100, mu: 0.9139382839, learning_rate: 0.0000008556, weight_decay: 0.0000802636

epoch time: 1273.7253417969s, input_fired_time: 13.2862434387, output_fired_time: 7.0724439621, propagate_forward_time: 496.2450561523, propagate_backward_time: 743.6669311523, weight_update_time: 0.8076692820, other_time: 13.4547119141
epoch time: 100.7307815552s, input_fired_time: 0.9564663768, output_fired_time: 0.0000000000, propagate_forward_time: 97.5880813599, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1862335205
validation[          , genome  6310] predictions:    4532/  10000 (45.32%), best:    6037/10000 (60.37%), error:     16830.10938, best error:     11815.21875 on epoch:     0, epoch:   37/100, mu: 0.9177118540, learning_rate: 0.0000007991, weight_decay: 0.0000760837

epoch time: 1235.6895751953s, input_fired_time: 13.0954494476, output_fired_time: 6.9004483223, propagate_forward_time: 479.9906311035, propagate_backward_time: 722.4104614258, weight_update_time: 0.7953754663, other_time: 13.2926635742
epoch time: 99.7357559204s, input_fired_time: 0.9457612634, output_fired_time: 0.0000000000, propagate_forward_time: 96.5708236694, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2191696167
validation[          , genome  6310] predictions:    4529/  10000 (45.29%), best:    6037/10000 (60.37%), error:     16798.65234, best error:     11815.21875 on epoch:     0, epoch:   38/100, mu: 0.9212982059, learning_rate: 0.0000007464, weight_decay: 0.0000721215

epoch time: 1237.6346435547s, input_fired_time: 13.1175489426, output_fired_time: 6.9044160843, propagate_forward_time: 481.3807373047, propagate_backward_time: 722.8371582031, weight_update_time: 0.7958119512, other_time: 13.3947753906
epoch time: 100.2266387939s, input_fired_time: 0.9409977794, output_fired_time: 0.0000000000, propagate_forward_time: 97.1041030884, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1815414429
validation[          , genome  6310] predictions:    4608/  10000 (46.08%), best:    6037/10000 (60.37%), error:     16788.29492, best error:     11815.21875 on epoch:     0, epoch:   39/100, mu: 0.9247066379, learning_rate: 0.0000006971, weight_decay: 0.0000683657

epoch time: 1236.0592041016s, input_fired_time: 13.1489543915, output_fired_time: 6.9025101662, propagate_forward_time: 481.6849975586, propagate_backward_time: 720.9493408203, weight_update_time: 0.7928776741, other_time: 13.3734741211
epoch time: 99.5112609863s, input_fired_time: 0.9475708008, output_fired_time: 0.0000000000, propagate_forward_time: 96.3959732056, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1677169800
validation[          , genome  6310] predictions:    4601/  10000 (46.01%), best:    6037/10000 (60.37%), error:     16752.85547, best error:     11815.21875 on epoch:     0, epoch:   40/100, mu: 0.9279459715, learning_rate: 0.0000006511, weight_decay: 0.0000648054

epoch time: 1238.4958496094s, input_fired_time: 13.1079931259, output_fired_time: 6.7979030609, propagate_forward_time: 480.5147705078, propagate_backward_time: 724.6910400391, weight_update_time: 0.8016460538, other_time: 13.3841552734
epoch time: 99.9910659790s, input_fired_time: 0.9488866925, output_fired_time: 0.0000000000, propagate_forward_time: 96.8630676270, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1791152954
validation[          , genome  6310] predictions:    4638/  10000 (46.38%), best:    6037/10000 (60.37%), error:     16803.24023, best error:     11815.21875 on epoch:     0, epoch:   41/100, mu: 0.9310245514, learning_rate: 0.0000006081, weight_decay: 0.0000614305

epoch time: 1237.1517333984s, input_fired_time: 13.1266298294, output_fired_time: 6.9313058853, propagate_forward_time: 481.7441406250, propagate_backward_time: 722.0189819336, weight_update_time: 0.7875829935, other_time: 13.3307495117
epoch time: 92.1530685425s, input_fired_time: 0.8962798715, output_fired_time: 0.0000000000, propagate_forward_time: 89.1141204834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1426696777
validation[          , genome  6310] predictions:    4560/  10000 (45.60%), best:    6037/10000 (60.37%), error:     16733.77734, best error:     11815.21875 on epoch:     0, epoch:   42/100, mu: 0.9339504242, learning_rate: 0.0000005680, weight_decay: 0.0000582314

epoch time: 1148.2675781250s, input_fired_time: 12.7393255234, output_fired_time: 6.4500398636, propagate_forward_time: 443.8386535645, propagate_backward_time: 672.3364257813, weight_update_time: 0.7561268210, other_time: 12.9030761719
epoch time: 93.4243240356s, input_fired_time: 0.9001803398, output_fired_time: 0.0000000000, propagate_forward_time: 90.3566513062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1674957275
validation[          , genome  6310] predictions:    4566/  10000 (45.66%), best:    6037/10000 (60.37%), error:     16748.02148, best error:     11815.21875 on epoch:     0, epoch:   43/100, mu: 0.9367311597, learning_rate: 0.0000005305, weight_decay: 0.0000551989

epoch time: 1155.0512695313s, input_fired_time: 12.7531862259, output_fired_time: 6.4675922394, propagate_forward_time: 445.0662841797, propagate_backward_time: 677.7264404297, weight_update_time: 0.7674577832, other_time: 13.0377197266
epoch time: 91.9017868042s, input_fired_time: 0.8879770041, output_fired_time: 0.0000000000, propagate_forward_time: 88.8689193726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1448898315
validation[          , genome  6310] predictions:    4671/  10000 (46.71%), best:    6037/10000 (60.37%), error:     16769.07813, best error:     11815.21875 on epoch:     0, epoch:   44/100, mu: 0.9393739104, learning_rate: 0.0000004954, weight_decay: 0.0000523243

epoch time: 1155.3165283203s, input_fired_time: 12.7550458908, output_fired_time: 6.4890322685, propagate_forward_time: 445.7991638184, propagate_backward_time: 677.2151489258, weight_update_time: 0.7651683092, other_time: 13.0581665039
epoch time: 93.5353393555s, input_fired_time: 0.9019199610, output_fired_time: 0.0000000000, propagate_forward_time: 90.4929656982, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1404571533
validation[          , genome  6310] predictions:    4626/  10000 (46.26%), best:    6037/10000 (60.37%), error:     16740.27344, best error:     11815.21875 on epoch:     0, epoch:   45/100, mu: 0.9418855309, learning_rate: 0.0000004627, weight_decay: 0.0000495994

epoch time: 1159.0568847656s, input_fired_time: 12.7559614182, output_fired_time: 6.4495897293, propagate_forward_time: 447.3207397461, propagate_backward_time: 679.5258789063, weight_update_time: 0.7662470341, other_time: 13.0046997070
epoch time: 93.3838729858s, input_fired_time: 0.8992235661, output_fired_time: 0.0000000000, propagate_forward_time: 90.3649902344, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1196594238
validation[          , genome  6310] predictions:    4643/  10000 (46.43%), best:    6037/10000 (60.37%), error:     16730.84180, best error:     11815.21875 on epoch:     0, epoch:   46/100, mu: 0.9442725778, learning_rate: 0.0000004322, weight_decay: 0.0000470164

epoch time: 1154.2902832031s, input_fired_time: 12.7636489868, output_fired_time: 6.4414873123, propagate_forward_time: 445.8577880859, propagate_backward_time: 676.1449584961, weight_update_time: 0.7758182287, other_time: 13.0823364258
epoch time: 93.2503356934s, input_fired_time: 0.8958445787, output_fired_time: 0.0000000000, propagate_forward_time: 90.1998214722, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1546707153
validation[          , genome  6310] predictions:    4646/  10000 (46.46%), best:    6037/10000 (60.37%), error:     16738.62305, best error:     11815.21875 on epoch:     0, epoch:   47/100, mu: 0.9465411901, learning_rate: 0.0000004037, weight_decay: 0.0000445680

epoch time: 1154.1162109375s, input_fired_time: 12.8001470566, output_fired_time: 6.4624071121, propagate_forward_time: 447.4499511719, propagate_backward_time: 674.3692016602, weight_update_time: 0.7741104364, other_time: 13.0344848633
epoch time: 92.9666671753s, input_fired_time: 0.9000095129, output_fired_time: 0.0000000000, propagate_forward_time: 89.9550170898, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1116409302
validation[          , genome  6310] predictions:    4649/  10000 (46.49%), best:    6037/10000 (60.37%), error:     16707.83203, best error:     11815.21875 on epoch:     0, epoch:   48/100, mu: 0.9486972690, learning_rate: 0.0000003770, weight_decay: 0.0000422470

epoch time: 1151.3360595703s, input_fired_time: 12.7402696609, output_fired_time: 6.4339303970, propagate_forward_time: 444.2803955078, propagate_backward_time: 675.0068969727, weight_update_time: 0.7620921135, other_time: 12.8745727539
epoch time: 92.2641830444s, input_fired_time: 0.8896652460, output_fired_time: 0.0000000000, propagate_forward_time: 89.2406387329, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1338806152
validation[          , genome  6310] predictions:    4616/  10000 (46.16%), best:    6037/10000 (60.37%), error:     16731.76172, best error:     11815.21875 on epoch:     0, epoch:   49/100, mu: 0.9507463574, learning_rate: 0.0000003521, weight_decay: 0.0000400469

epoch time: 1158.1188964844s, input_fired_time: 12.7994680405, output_fired_time: 6.4501318932, propagate_forward_time: 448.8096313477, propagate_backward_time: 677.0568847656, weight_update_time: 0.7674812078, other_time: 13.0028686523
epoch time: 92.2670211792s, input_fired_time: 0.9001834393, output_fired_time: 0.0000000000, propagate_forward_time: 89.2461242676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1207122803
validation[          , genome  6310] predictions:    4659/  10000 (46.59%), best:    6037/10000 (60.37%), error:     16712.66211, best error:     11815.21875 on epoch:     0, epoch:   50/100, mu: 0.9526938200, learning_rate: 0.0000003289, weight_decay: 0.0000379614

epoch time: 1147.4063720703s, input_fired_time: 12.7257642746, output_fired_time: 6.4081068039, propagate_forward_time: 443.1599426270, propagate_backward_time: 672.1144409180, weight_update_time: 0.7606180906, other_time: 12.9982299805
epoch time: 93.9445953369s, input_fired_time: 0.9013470411, output_fired_time: 0.0000000000, propagate_forward_time: 90.9071502686, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1361007690
validation[          , genome  6310] predictions:    4652/  10000 (46.52%), best:    6037/10000 (60.37%), error:     16717.13086, best error:     11815.21875 on epoch:     0, epoch:   51/100, mu: 0.9545446634, learning_rate: 0.0000003072, weight_decay: 0.0000359845

epoch time: 1153.2387695313s, input_fired_time: 12.7436389923, output_fired_time: 6.4306707382, propagate_forward_time: 445.6268920898, propagate_backward_time: 675.4727172852, weight_update_time: 0.7686010599, other_time: 12.9648437500
epoch time: 93.6592407227s, input_fired_time: 0.9047171474, output_fired_time: 0.0000000000, propagate_forward_time: 90.6136856079, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1408386230
validation[          , genome  6310] predictions:    4670/  10000 (46.70%), best:    6037/10000 (60.37%), error:     16704.26953, best error:     11815.21875 on epoch:     0, epoch:   52/100, mu: 0.9563036561, learning_rate: 0.0000002869, weight_decay: 0.0000341105

epoch time: 1129.3018798828s, input_fired_time: 12.6368169785, output_fired_time: 6.3453903198, propagate_forward_time: 434.4719238281, propagate_backward_time: 662.9439086914, weight_update_time: 0.7570494413, other_time: 12.9038696289
epoch time: 84.2483749390s, input_fired_time: 0.8290433884, output_fired_time: 0.0000000000, propagate_forward_time: 81.3820800781, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0372543335
validation[          , genome  6310] predictions:    4669/  10000 (46.69%), best:    6037/10000 (60.37%), error:     16659.32227, best error:     11815.21875 on epoch:     0, epoch:   53/100, mu: 0.9579753876, learning_rate: 0.0000002680, weight_decay: 0.0000323342

epoch time: 1060.1042480469s, input_fired_time: 12.3461904526, output_fired_time: 5.9820199013, propagate_forward_time: 406.1693725586, propagate_backward_time: 623.0729370117, weight_update_time: 0.7279174328, other_time: 12.5336914063
epoch time: 85.1374435425s, input_fired_time: 0.8321920633, output_fired_time: 0.0000000000, propagate_forward_time: 82.2478637695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0573883057
validation[          , genome  6310] predictions:    4660/  10000 (46.60%), best:    6037/10000 (60.37%), error:     16688.42969, best error:     11815.21875 on epoch:     0, epoch:   54/100, mu: 0.9595641494, learning_rate: 0.0000002503, weight_decay: 0.0000306503

epoch time: 1062.8033447266s, input_fired_time: 12.3424873352, output_fired_time: 6.0027179718, propagate_forward_time: 407.1844177246, propagate_backward_time: 624.6909179688, weight_update_time: 0.7333309650, other_time: 12.5827636719
epoch time: 84.2666015625s, input_fired_time: 0.8298090100, output_fired_time: 0.0000000000, propagate_forward_time: 81.3817062378, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0550842285
validation[          , genome  6310] predictions:    4674/  10000 (46.74%), best:    6037/10000 (60.37%), error:     16690.19922, best error:     11815.21875 on epoch:     0, epoch:   55/100, mu: 0.9610741138, learning_rate: 0.0000002337, weight_decay: 0.0000290541

epoch time: 1070.6207275391s, input_fired_time: 12.3854484558, output_fired_time: 6.0741448402, propagate_forward_time: 409.3367309570, propagate_backward_time: 630.1591796875, weight_update_time: 0.7231277823, other_time: 12.6652221680
epoch time: 87.6656341553s, input_fired_time: 0.8580507636, output_fired_time: 0.0000000000, propagate_forward_time: 84.7376403809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0699462891
validation[          , genome  6310] predictions:    4707/  10000 (47.07%), best:    6037/10000 (60.37%), error:     16706.21289, best error:     11815.21875 on epoch:     0, epoch:   56/100, mu: 0.9625091553, learning_rate: 0.0000002183, weight_decay: 0.0000275411

epoch time: 1091.3555908203s, input_fired_time: 12.5299806595, output_fired_time: 6.1286792755, propagate_forward_time: 419.9533691406, propagate_backward_time: 639.9839477539, weight_update_time: 0.7377290726, other_time: 12.7595825195
epoch time: 87.5079956055s, input_fired_time: 0.8582353592, output_fired_time: 0.0000000000, propagate_forward_time: 84.5464706421, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1032867432
validation[          , genome  6310] predictions:    4597/  10000 (45.97%), best:    6037/10000 (60.37%), error:     16704.24805, best error:     11815.21875 on epoch:     0, epoch:   57/100, mu: 0.9638730288, learning_rate: 0.0000002039, weight_decay: 0.0000261068

epoch time: 1090.0441894531s, input_fired_time: 12.4987468719, output_fired_time: 6.1539702415, propagate_forward_time: 418.3120422363, propagate_backward_time: 640.3732910156, weight_update_time: 0.7313308716, other_time: 12.7061767578
epoch time: 87.9649734497s, input_fired_time: 0.8623344898, output_fired_time: 0.0000000000, propagate_forward_time: 84.9977035522, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1049346924
validation[          , genome  6310] predictions:    4660/  10000 (46.60%), best:    6037/10000 (60.37%), error:     16694.26367, best error:     11815.21875 on epoch:     0, epoch:   58/100, mu: 0.9651692510, learning_rate: 0.0000001904, weight_decay: 0.0000247473



Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x000007FEFDB932D2

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 11/23/18 10:03:57
Install Directory : D:\Boinc\
Data Directory    : D:\
Project Symstore  : 
LoadLibraryA( D:\Boinc\\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\Boinc\\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\Boinc\\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\Boinc\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\slots\2;D:\projects\csgrid.org_csg


ModLoad: 000000003fbe0000 00000000001da000 D:\projects\csgrid.org_csg\exact_client_0.34_windows_x86_64.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : C:\Users\travi\Code\boinc_2015\win_build\Build\x64\Release\exact.pdb

ModLoad: 00000000773b0000 000000000019f000 C:\Windows\SYSTEM32\ntdll.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000076f30000 000000000011f000 C:\Windows\system32\kernel32.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 00000000fdb60000 000000000006a000 C:\Windows\system32\KERNELBASE.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : kernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 0000000076e30000 00000000000fa000 C:\Windows\system32\USER32.dll (6.1.7601.23594) (-exported- Symbols Loaded)
    Linked PDB Filename   : user32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 00000000fdd00000 0000000000067000 C:\Windows\system32\GDI32.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : gdi32.pdb
    File Version          : 6.1.7601.24260 (win7sp1_ldr.180908-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.24260

ModLoad: 00000000fe180000 000000000000e000 C:\Windows\system32\LPK.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : lpk.pdb
    File Version          : 6.1.7601.24231 (win7sp1_ldr.180810-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.24231

ModLoad: 00000000ff5d0000 00000000000cb000 C:\Windows\system32\USP10.dll (1.626.7601.23807) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.23807 (win7sp1_ldr.170512-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.23807

ModLoad: 00000000fe190000 000000000009f000 C:\Windows\system32\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 00000000fe0a0000 00000000000db000 C:\Windows\system32\ADVAPI32.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7601.24260 (win7sp1_ldr.180908-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.24260

ModLoad: 00000000fdf10000 000000000001f000 C:\Windows\SYSTEM32\sechost.dll (6.1.7601.18869) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000fdbd0000 000000000012d000 C:\Windows\system32\RPCRT4.dll (6.1.7601.24260) (-exported- Symbols Loaded)
    Linked PDB Filename   : rpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000fe230000 0000000000d8a000 C:\Windows\system32\SHELL32.dll (6.1.7601.24234) (-exported- Symbols Loaded)
    Linked PDB Filename   : shell32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 00000000fe020000 0000000000071000 C:\Windows\system32\SHLWAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : shlwapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000fdee0000 000000000002e000 C:\Windows\system32\IMM32.DLL (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : imm32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000fddd0000 0000000000109000 C:\Windows\system32\MSCTF.dll (6.1.7601.23572) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000fcad0000 0000000000003000 C:\Windows\system32\api-ms-win-core-synch-l1-2-0.DLL (10.0.10586.9) (-exported- Symbols Loaded)
    Linked PDB Filename   : api-ms-win-core-synch-l1-2-0.pdb
    File Version          : 10.0.10586.9 (th2_release.151110-1756)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 10.0.10586.9

ModLoad: 00000000ecf10000 0000000000125000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 00000000fcae0000 000000000000c000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft? Windows? Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 53633, Write: 395795, Other 6009

- I/O Transfers Counters -
Read: 219702541, Write: 575665393, Other 390678

- Paged Pool Usage -
QuotaPagedPoolUsage: 107928, QuotaPeakPagedPoolUsage: 108064
QuotaNonPagedPoolUsage: 9600, QuotaPeakNonPagedPoolUsage: 9720

- Virtual Memory Usage -
VirtualSize: 550027264, PeakVirtualSize: 606953472

- Pagefile Usage -
PagefileUsage: 550027264, PeakPagefileUsage: 550027264

- Working Set Size -
WorkingSetSize: 551936000, PeakWorkingSetSize: 551936000, PageFaultCount: 147008

*** Dump of thread ID 6260 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x000007FEFDB932D2

- Registers -
rax=0000000000000000 rbx=0000000000000001 rcx=000000003fd39440 rdx=0000000001dff100 rsi=0000000000000000 rdi=0000000000000000
r8=0000000001dff100 r9=000000003fd39430 r10=0000000000000001 r11=0000000000000fff r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000fdb932d2 rsp=0000000001dff0d8 rbp=0000000000000000
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
01dff0d0 3fc6e421 00000001 01dff100 01dff100 3fd39430 KERNELBASE!DebugBreak+0x0 
01dff510 3fc6f820 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
01dff770 76f459cd 00000000 00000000 00000000 00000000 exact_client_0.34_windows_x86_6!+0x0 
01dff7a0 7740385d 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
01dff7f0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...

</stderr_txt>
]]>