Nameexact_genome_1540823703_189_5440_1
Workunit4077947
Created29 Oct 2018, 18:02:56 UTC
Sent29 Oct 2018, 18:56:29 UTC
Report deadline10 Jan 2019, 2:38:17 UTC
Received10 Dec 2018, 1:54:45 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64195
Run time1 days 5 hours 39 min 44 sec
CPU time1 days 5 hours 38 min 40 sec
Validate stateValid
Credit18,080.17
Device peak FLOPS3.80 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size529.65 MB
Peak swap size549.08 MB
Peak disk usage10.16 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
 number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5440
read normal distribution: '1 0 0'
generator_str: '1358573325'
read generator: 1358573325
read generated_by_map:
1 crossover 1
reading 63 nodes.
reading 279 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 794.926574707031s, input_fired_time: 10.5617647171021, output_fired_time: 3.84459447860718, propagate_forward_time: 293.281311035156, propagate_backward_time: 478.237731933594, weight_update_time: 0.277292937040329, other_time: 9.00115966796875
epoch time: 60.762565612793s, input_fired_time: 0.674302458763123, output_fired_time: 0, propagate_forward_time: 58.6084976196289, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.47976684570313
validation[          , genome  5440] predictions:    5823/  10000 (58.23%), best:    5971/10000 (59.71%), error:     11854.20215, best error:     11743.89551 on epoch:    25, epoch:   26/100, mu: 0.8729785085, learning_rate: 0.0002647228, weight_decay: 0.0001365032

epoch time: 858.3212280273s, input_fired_time: 10.8068094254, output_fired_time: 4.1157021523, propagate_forward_time: 320.2818298340, propagate_backward_time: 513.6885375977, weight_update_time: 0.2961729765, other_time: 9.4282836914
epoch time: 55.7899818420s, input_fired_time: 0.6245844960, output_fired_time: 0.0000000000, propagate_forward_time: 53.7752761841, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3901214600
validation[          , genome  5440] predictions:    5642/  10000 (56.42%), best:    5971/10000 (59.71%), error:     12549.04980, best error:     11743.89551 on epoch:    25, epoch:   27/100, mu: 0.8789724112, learning_rate: 0.0002453278, weight_decay: 0.0001297275

epoch time: 840.6182250977s, input_fired_time: 10.7286968231, output_fired_time: 4.0203948021, propagate_forward_time: 312.0586547852, propagate_backward_time: 504.5574340820, weight_update_time: 0.2913081348, other_time: 9.2530517578
epoch time: 54.8334083557s, input_fired_time: 0.6291356087, output_fired_time: 0.0000000000, propagate_forward_time: 52.8092803955, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3949928284
validation[          , genome  5440] predictions:    5913/  10000 (59.13%), best:    5971/10000 (59.71%), error:     11754.17578, best error:     11743.89551 on epoch:    25, epoch:   28/100, mu: 0.8846592903, learning_rate: 0.0002273538, weight_decay: 0.0001232881

epoch time: 844.4005737305s, input_fired_time: 10.7588129044, output_fired_time: 4.0525503159, propagate_forward_time: 314.1510009766, propagate_backward_time: 506.1389465332, weight_update_time: 0.2916201055, other_time: 9.2992858887
epoch time: 57.6175155640s, input_fired_time: 0.6450474262, output_fired_time: 0.0000000000, propagate_forward_time: 55.5655822754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4068870544
validation[          , genome  5440] predictions:    5934/  10000 (59.34%), best:    5971/10000 (59.71%), error:     11767.21680, best error:     11743.89551 on epoch:    25, epoch:   29/100, mu: 0.8900548816, learning_rate: 0.0002106967, weight_decay: 0.0001171684

epoch time: 800.2832641602s, input_fired_time: 10.6177463531, output_fired_time: 3.8765010834, propagate_forward_time: 295.3503723145, propagate_backward_time: 481.3410644531, weight_update_time: 0.2804571092, other_time: 9.0975646973
epoch time: 77.4736785889s, input_fired_time: 0.7850705981, output_fired_time: 0.0000000000, propagate_forward_time: 75.0875930786, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6010131836
validation[          , genome  5440] predictions:    5962/  10000 (59.62%), best:    5962/10000 (59.62%), error:     11549.31348, best error:     11549.31348 on epoch:    30, epoch:   30/100, mu: 0.8951741457, learning_rate: 0.0001952600, weight_decay: 0.0001113524

epoch time: 923.5012207031s, input_fired_time: 11.0405387878, output_fired_time: 4.3646492958, propagate_forward_time: 346.2834777832, propagate_backward_time: 552.0987548828, weight_update_time: 0.3129534721, other_time: 9.7138671875
epoch time: 100.7605438232s, input_fired_time: 0.9306932688, output_fired_time: 0.0000000000, propagate_forward_time: 98.0123748779, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8174743652
validation[          , genome  5440] predictions:    5989/  10000 (59.89%), best:    5962/10000 (59.62%), error:     11620.39746, best error:     11549.31348 on epoch:    30, epoch:   31/100, mu: 0.9000312090, learning_rate: 0.0001809542, weight_decay: 0.0001058252

epoch time: 850.6358032227s, input_fired_time: 10.7994775772, output_fired_time: 4.0865478516, propagate_forward_time: 316.3917236328, propagate_backward_time: 509.9945373535, weight_update_time: 0.2938523889, other_time: 9.3634948730
epoch time: 99.7648010254s, input_fired_time: 0.9416282177, output_fired_time: 0.0000000000, propagate_forward_time: 96.9453277588, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8778457642
validation[          , genome  5440] predictions:    5964/  10000 (59.64%), best:    5962/10000 (59.62%), error:     11685.76660, best error:     11549.31348 on epoch:    30, epoch:   32/100, mu: 0.9046394825, learning_rate: 0.0001676966, weight_decay: 0.0001005723

epoch time: 833.2739257813s, input_fired_time: 10.7426471710, output_fired_time: 3.9996776581, propagate_forward_time: 309.6732788086, propagate_backward_time: 499.6479187012, weight_update_time: 0.2875053287, other_time: 9.2103576660
epoch time: 73.9650573730s, input_fired_time: 0.7726455927, output_fired_time: 0.0000000000, propagate_forward_time: 71.5493392944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6430740356
validation[          , genome  5440] predictions:    5828/  10000 (58.28%), best:    5962/10000 (59.62%), error:     12020.52734, best error:     11549.31348 on epoch:    30, epoch:   33/100, mu: 0.9090117216, learning_rate: 0.0001554102, weight_decay: 0.0000955801

epoch time: 831.1238403320s, input_fired_time: 10.7490262985, output_fired_time: 3.9925088882, propagate_forward_time: 308.8128356934, propagate_backward_time: 498.3162536621, weight_update_time: 0.2888896465, other_time: 9.2532348633
epoch time: 51.4686431885s, input_fired_time: 0.6111179590, output_fired_time: 0.0000000000, propagate_forward_time: 49.5079994202, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3495254517
validation[          , genome  5440] predictions:    5921/  10000 (59.21%), best:    5962/10000 (59.62%), error:     11703.85938, best error:     11549.31348 on epoch:    30, epoch:   34/100, mu: 0.9131599665, learning_rate: 0.0001440241, weight_decay: 0.0000908357

epoch time: 884.3352050781s, input_fired_time: 10.9383163452, output_fired_time: 4.2325038910, propagate_forward_time: 330.0173950195, propagate_backward_time: 529.6172485352, weight_update_time: 0.3020522296, other_time: 9.5297851563
epoch time: 80.8281860352s, input_fired_time: 0.8330514431, output_fired_time: 0.0000000000, propagate_forward_time: 78.2471466064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7479858398
validation[          , genome  5440] predictions:    5930/  10000 (59.30%), best:    5962/10000 (59.62%), error:     11790.00000, best error:     11549.31348 on epoch:    30, epoch:   35/100, mu: 0.9170957804, learning_rate: 0.0001334721, weight_decay: 0.0000863269

epoch time: 779.8245239258s, input_fired_time: 10.5866680145, output_fired_time: 3.7976713181, propagate_forward_time: 286.8348999023, propagate_backward_time: 469.6072082520, weight_update_time: 0.2756813169, other_time: 8.9980773926
epoch time: 51.5391921997s, input_fired_time: 0.6126269698, output_fired_time: 0.0000000000, propagate_forward_time: 49.5773735046, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3491935730
validation[          , genome  5440] predictions:    5961/  10000 (59.61%), best:    5962/10000 (59.62%), error:     11689.99707, best error:     11549.31348 on epoch:    30, epoch:   36/100, mu: 0.9208299518, learning_rate: 0.0001236933, weight_decay: 0.0000820418

epoch time: 859.1489868164s, input_fired_time: 10.8792486191, output_fired_time: 4.1499123573, propagate_forward_time: 319.5641479492, propagate_backward_time: 515.0992431641, weight_update_time: 0.2964404523, other_time: 9.4564208984
epoch time: 58.0476531982s, input_fired_time: 0.6579018235, output_fired_time: 0.0000000000, propagate_forward_time: 55.9683532715, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4213981628
validation[          , genome  5440] predictions:    6002/  10000 (60.02%), best:    5962/10000 (59.62%), error:     11571.02051, best error:     11549.31348 on epoch:    30, epoch:   37/100, mu: 0.9243729115, learning_rate: 0.0001146309, weight_decay: 0.0000779694

epoch time: 822.7035522461s, input_fired_time: 10.7468843460, output_fired_time: 3.9840338230, propagate_forward_time: 304.8072509766, propagate_backward_time: 493.9329223633, weight_update_time: 0.2864053845, other_time: 9.2324829102
epoch time: 75.4333038330s, input_fired_time: 0.7931346297, output_fired_time: 0.0000000000, propagate_forward_time: 72.9661178589, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6740493774
validation[          , genome  5440] predictions:    6022/  10000 (60.22%), best:    6022/10000 (60.22%), error:     11449.79395, best error:     11449.79395 on epoch:    38, epoch:   38/100, mu: 0.9277343750, learning_rate: 0.0001062324, weight_decay: 0.0000740992

epoch time: 901.4873046875s, input_fired_time: 10.9736309052, output_fired_time: 4.2400722504, propagate_forward_time: 337.2748107910, propagate_backward_time: 539.4827880859, weight_update_time: 0.3063661158, other_time: 9.5159912109
epoch time: 70.2636566162s, input_fired_time: 0.7567640543, output_fired_time: 0.0000000000, propagate_forward_time: 67.9397506714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5671386719
validation[          , genome  5440] predictions:    5985/  10000 (59.85%), best:    6022/10000 (60.22%), error:     11608.04785, best error:     11449.79395 on epoch:    38, epoch:   39/100, mu: 0.9309236407, learning_rate: 0.0000984493, weight_decay: 0.0000704211

epoch time: 771.8404541016s, input_fired_time: 10.5531272888, output_fired_time: 3.7480826378, propagate_forward_time: 283.5938415527, propagate_backward_time: 465.0363159180, weight_update_time: 0.2722241580, other_time: 8.9090881348
epoch time: 79.9174270630s, input_fired_time: 0.8140158653, output_fired_time: 0.0000000000, propagate_forward_time: 77.4388809204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6645278931
validation[          , genome  5440] predictions:    5999/  10000 (59.99%), best:    6022/10000 (60.22%), error:     11536.21777, best error:     11449.79395 on epoch:    38, epoch:   40/100, mu: 0.9339495897, learning_rate: 0.0000912364, weight_decay: 0.0000669256

epoch time: 805.8289794922s, input_fired_time: 10.6794404984, output_fired_time: 3.8901417255, propagate_forward_time: 298.3153686523, propagate_backward_time: 483.8553466797, weight_update_time: 0.2816244066, other_time: 9.0886840820
epoch time: 60.7349853516s, input_fired_time: 0.6812123060, output_fired_time: 0.0000000000, propagate_forward_time: 58.6173019409, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4364700317
validation[          , genome  5440] predictions:    6114/  10000 (61.14%), best:    6114/10000 (61.14%), error:     11216.81055, best error:     11216.81055 on epoch:    41, epoch:   41/100, mu: 0.9368205070, learning_rate: 0.0000845519, weight_decay: 0.0000636035

epoch time: 925.8081054688s, input_fired_time: 11.1313762665, output_fired_time: 4.4123215675, propagate_forward_time: 347.0283508301, propagate_backward_time: 553.4241943359, weight_update_time: 0.3143388629, other_time: 9.8118896484
epoch time: 99.7097167969s, input_fired_time: 0.9715253711, output_fired_time: 0.0000000000, propagate_forward_time: 96.7737731934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9644165039
validation[          , genome  5440] predictions:    5951/  10000 (59.51%), best:    6114/10000 (61.14%), error:     11640.42871, best error:     11216.81055 on epoch:    41, epoch:   42/100, mu: 0.9395443797, learning_rate: 0.0000783572, weight_decay: 0.0000604464

epoch time: 769.4019165039s, input_fired_time: 10.5791931152, output_fired_time: 3.7641642094, propagate_forward_time: 283.1464843750, propagate_backward_time: 462.9885559082, weight_update_time: 0.2716013193, other_time: 8.9235534668
epoch time: 69.1526412964s, input_fired_time: 0.7550051212, output_fired_time: 0.0000000000, propagate_forward_time: 66.8201141357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5775222778
validation[          , genome  5440] predictions:    6038/  10000 (60.38%), best:    6114/10000 (61.14%), error:     11443.65039, best error:     11216.81055 on epoch:    41, epoch:   43/100, mu: 0.9421287179, learning_rate: 0.0000726163, weight_decay: 0.0000574460

epoch time: 848.8941040039s, input_fired_time: 10.8645572662, output_fired_time: 4.0928988457, propagate_forward_time: 315.9147033691, propagate_backward_time: 508.6463928223, weight_update_time: 0.2934446633, other_time: 9.3755798340
epoch time: 68.7021942139s, input_fired_time: 0.7343363762, output_fired_time: 0.0000000000, propagate_forward_time: 66.4586486816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5092086792
validation[          , genome  5440] predictions:    5964/  10000 (59.64%), best:    6114/10000 (61.14%), error:     11470.76855, best error:     11216.81055 on epoch:    41, epoch:   44/100, mu: 0.9445807338, learning_rate: 0.0000672961, weight_decay: 0.0000545945

epoch time: 835.9820556641s, input_fired_time: 10.8311223984, output_fired_time: 4.0356655121, propagate_forward_time: 309.9341430664, propagate_backward_time: 501.8682861328, weight_update_time: 0.2910321653, other_time: 9.3128662109
epoch time: 58.1236305237s, input_fired_time: 0.6679975390, output_fired_time: 0.0000000000, propagate_forward_time: 56.0475006104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4081306458
validation[          , genome  5440] predictions:    6044/  10000 (60.44%), best:    6114/10000 (61.14%), error:     11409.98145, best error:     11216.81055 on epoch:    41, epoch:   45/100, mu: 0.9469071627, learning_rate: 0.0000623656, weight_decay: 0.0000518846

epoch time: 830.1945800781s, input_fired_time: 10.7828845978, output_fired_time: 3.9784812927, propagate_forward_time: 307.7761535645, propagate_backward_time: 498.4775695801, weight_update_time: 0.2862077057, other_time: 9.1795043945
epoch time: 67.2247543335s, input_fired_time: 0.7206671238, output_fired_time: 0.0000000000, propagate_forward_time: 65.0068435669, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4972457886
validation[          , genome  5440] predictions:    6020/  10000 (60.20%), best:    6114/10000 (61.14%), error:     11343.11230, best error:     11216.81055 on epoch:    41, epoch:   46/100, mu: 0.9491143823, learning_rate: 0.0000577964, weight_decay: 0.0000493092

epoch time: 963.2424316406s, input_fired_time: 11.2583827972, output_fired_time: 4.5309343338, propagate_forward_time: 362.6340332031, propagate_backward_time: 574.8844604492, weight_update_time: 0.3221154809, other_time: 9.9346313477
epoch time: 68.7291870117s, input_fired_time: 0.7545001507, output_fired_time: 0.0000000000, propagate_forward_time: 66.3970565796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5776290894
validation[          , genome  5440] predictions:    6040/  10000 (60.40%), best:    6114/10000 (61.14%), error:     11274.80469, best error:     11216.81055 on epoch:    41, epoch:   47/100, mu: 0.9512085915, learning_rate: 0.0000535619, weight_decay: 0.0000468616

epoch time: 854.8765258789s, input_fired_time: 10.8822612762, output_fired_time: 4.0755400658, propagate_forward_time: 318.2792053223, propagate_backward_time: 512.3303833008, weight_update_time: 0.2932146788, other_time: 9.3091430664
epoch time: 73.4642791748s, input_fired_time: 0.7776839137, output_fired_time: 0.0000000000, propagate_forward_time: 71.1137084961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5728836060
validation[          , genome  5440] predictions:    6160/  10000 (61.60%), best:    6160/10000 (61.60%), error:     11121.06836, best error:     11121.06836 on epoch:    48, epoch:   48/100, mu: 0.9531955123, learning_rate: 0.0000496377, weight_decay: 0.0000445355

epoch time: 995.5487670898s, input_fired_time: 11.3746318817, output_fired_time: 4.6689510345, propagate_forward_time: 376.1908874512, propagate_backward_time: 593.2500610352, weight_update_time: 0.3300968409, other_time: 10.0642700195
epoch time: 61.3607254028s, input_fired_time: 0.6918371916, output_fired_time: 0.0000000000, propagate_forward_time: 59.2273941040, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4414939880
validation[          , genome  5440] predictions:    6108/  10000 (61.08%), best:    6160/10000 (61.60%), error:     11164.23730, best error:     11121.06836 on epoch:    48, epoch:   49/100, mu: 0.9550806880, learning_rate: 0.0000460010, weight_decay: 0.0000423248

epoch time: 1020.6319580078s, input_fired_time: 11.4764137268, output_fired_time: 4.7743492126, propagate_forward_time: 386.4451904297, propagate_backward_time: 607.6835327148, weight_update_time: 0.3376764953, other_time: 10.2524414063
epoch time: 70.6241302490s, input_fired_time: 0.7668186426, output_fired_time: 0.0000000000, propagate_forward_time: 68.2701873779, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5871276855
validation[          , genome  5440] predictions:    6146/  10000 (61.46%), best:    6160/10000 (61.60%), error:     11154.10352, best error:     11121.06836 on epoch:    48, epoch:   50/100, mu: 0.9568693042, learning_rate: 0.0000426307, weight_decay: 0.0000402239

epoch time: 1023.7684936523s, input_fired_time: 11.4431848526, output_fired_time: 4.7134675980, propagate_forward_time: 388.8619384766, propagate_backward_time: 608.6611938477, weight_update_time: 0.3364090323, other_time: 10.0887451172
epoch time: 74.1599349976s, input_fired_time: 0.7823715210, output_fired_time: 0.0000000000, propagate_forward_time: 71.7929458618, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5846176147
validation[          , genome  5440] predictions:    6233/  10000 (62.33%), best:    6233/10000 (62.33%), error:     11044.58008, best error:     11044.58008 on epoch:    51, epoch:   51/100, mu: 0.9585662484, learning_rate: 0.0000395074, weight_decay: 0.0000382273

epoch time: 1049.1730957031s, input_fired_time: 11.5719032288, output_fired_time: 4.8686432838, propagate_forward_time: 397.5971069336, propagate_backward_time: 624.7953491211, weight_update_time: 0.3450021744, other_time: 10.3400878906
epoch time: 83.0244598389s, input_fired_time: 0.8543801904, output_fired_time: 0.0000000000, propagate_forward_time: 80.4142227173, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7558593750
validation[          , genome  5440] predictions:    6176/  10000 (61.76%), best:    6233/10000 (62.33%), error:     11141.28809, best error:     11044.58008 on epoch:    51, epoch:   52/100, mu: 0.9601762891, learning_rate: 0.0000366129, weight_decay: 0.0000363298

epoch time: 1078.2884521484s, input_fired_time: 11.6541118622, output_fired_time: 4.9500613213, propagate_forward_time: 410.0252990723, propagate_backward_time: 641.2575073242, weight_update_time: 0.3525403142, other_time: 10.4015502930
epoch time: 71.9916381836s, input_fired_time: 0.7758551240, output_fired_time: 0.0000000000, propagate_forward_time: 69.6209182739, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5948638916
validation[          , genome  5440] predictions:    6085/  10000 (60.85%), best:    6233/10000 (62.33%), error:     11381.31836, best error:     11044.58008 on epoch:    51, epoch:   53/100, mu: 0.9617038965, learning_rate: 0.0000339304, weight_decay: 0.0000345265

epoch time: 1061.3813476563s, input_fired_time: 11.5883789063, output_fired_time: 4.8860034943, propagate_forward_time: 402.8721923828, propagate_backward_time: 631.7614135742, weight_update_time: 0.3458192647, other_time: 10.2733764648
epoch time: 100.6041259766s, input_fired_time: 0.9702700377, output_fired_time: 0.0000000000, propagate_forward_time: 97.7325592041, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9012985229
validation[          , genome  5440] predictions:    5956/  10000 (59.56%), best:    6233/10000 (62.33%), error:     11562.23242, best error:     11044.58008 on epoch:    51, epoch:   54/100, mu: 0.9631532431, learning_rate: 0.0000314445, weight_decay: 0.0000328127

epoch time: 1063.3980712891s, input_fired_time: 11.6178283691, output_fired_time: 4.9019494057, propagate_forward_time: 403.9626159668, propagate_backward_time: 632.5879516602, weight_update_time: 0.3474007547, other_time: 10.3276977539
epoch time: 82.6249389648s, input_fired_time: 0.8415795565, output_fired_time: 0.0000000000, propagate_forward_time: 80.1040115356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6793441772
validation[          , genome  5440] predictions:    6162/  10000 (61.62%), best:    6233/10000 (62.33%), error:     11122.12695, best error:     11044.58008 on epoch:    51, epoch:   55/100, mu: 0.9645283222, learning_rate: 0.0000291407, weight_decay: 0.0000311839

epoch time: 1110.7425537109s, input_fired_time: 11.7908887863, output_fired_time: 5.0916390419, propagate_forward_time: 423.6672058105, propagate_backward_time: 659.5606079102, weight_update_time: 0.3609580398, other_time: 10.6321411133
epoch time: 78.3873748779s, input_fired_time: 0.8187931776, output_fired_time: 0.0000000000, propagate_forward_time: 75.9303359985, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6382446289
validation[          , genome  5440] predictions:    6231/  10000 (62.31%), best:    6231/10000 (62.31%), error:     11029.22754, best error:     11029.22754 on epoch:    56, epoch:   56/100, mu: 0.9658330083, learning_rate: 0.0000270057, weight_decay: 0.0000296360

epoch time: 1006.4365234375s, input_fired_time: 11.4110860825, output_fired_time: 4.6570906639, propagate_forward_time: 381.7941284180, propagate_backward_time: 598.5783081055, weight_update_time: 0.3326833844, other_time: 9.9959106445
epoch time: 84.5704498291s, input_fired_time: 0.8618904948, output_fired_time: 0.0000000000, propagate_forward_time: 81.9938964844, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7146606445
validation[          , genome  5440] predictions:    6274/  10000 (62.74%), best:    6231/10000 (62.31%), error:     11029.53809, best error:     11029.22754 on epoch:    56, epoch:   57/100, mu: 0.9670708776, learning_rate: 0.0000250271, weight_decay: 0.0000281650

epoch time: 1076.6655273438s, input_fired_time: 11.6343545914, output_fired_time: 4.8948349953, propagate_forward_time: 411.5924682617, propagate_backward_time: 638.2629394531, weight_update_time: 0.3491710424, other_time: 10.2809448242
epoch time: 79.3685760498s, input_fired_time: 0.8194361925, output_fired_time: 0.0000000000, propagate_forward_time: 76.9206924438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6284484863
validation[          , genome  5440] predictions:    6246/  10000 (62.46%), best:    6246/10000 (62.46%), error:     10912.64746, best error:     10912.64746 on epoch:    58, epoch:   58/100, mu: 0.9682453275, learning_rate: 0.0000231935, weight_decay: 0.0000267669

epoch time: 1071.9532470703s, input_fired_time: 11.6321258545, output_fired_time: 4.8776717186, propagate_forward_time: 407.8840026855, propagate_backward_time: 637.2448120117, weight_update_time: 0.3489918709, other_time: 10.3146362305
epoch time: 98.5043106079s, input_fired_time: 0.9492450356, output_fired_time: 0.0000000000, propagate_forward_time: 95.7183074951, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8367614746
validation[          , genome  5440] predictions:    6213/  10000 (62.13%), best:    6246/10000 (62.46%), error:     10989.51172, best error:     10912.64746 on epoch:    58, epoch:   59/100, mu: 0.9693596363, learning_rate: 0.0000214942, weight_decay: 0.0000254383

epoch time: 1022.7592163086s, input_fired_time: 11.4929141998, output_fired_time: 4.7296342850, propagate_forward_time: 386.4366149902, propagate_backward_time: 609.9224243164, weight_update_time: 0.3371227086, other_time: 10.1776733398
epoch time: 63.8766021729s, input_fired_time: 0.7287724614, output_fired_time: 0.0000000000, propagate_forward_time: 61.6434898376, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5043411255
validation[          , genome  5440] predictions:    6184/  10000 (61.84%), best:    6246/10000 (62.46%), error:     10995.68359, best error:     10912.64746 on epoch:    58, epoch:   60/100, mu: 0.9704168439, learning_rate: 0.0000199195, weight_decay: 0.0000241756

epoch time: 978.7107543945s, input_fired_time: 11.3358373642, output_fired_time: 4.5485215187, propagate_forward_time: 369.0380554199, propagate_backward_time: 583.8693237305, weight_update_time: 0.3257554173, other_time: 9.9190063477
epoch time: 74.4725646973s, input_fired_time: 0.7939344049, output_fired_time: 0.0000000000, propagate_forward_time: 72.1013641357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5772628784
validation[          , genome  5440] predictions:    6199/  10000 (61.99%), best:    6246/10000 (62.46%), error:     11063.42383, best error:     10912.64746 on epoch:    58, epoch:   61/100, mu: 0.9714199305, learning_rate: 0.0000184601, weight_decay: 0.0000229756

epoch time: 1039.9304199219s, input_fired_time: 11.5225448608, output_fired_time: 4.7958784103, propagate_forward_time: 394.2599487305, propagate_backward_time: 619.1894531250, weight_update_time: 0.3404950202, other_time: 10.1625366211
epoch time: 89.1375732422s, input_fired_time: 0.8917855620, output_fired_time: 0.0000000000, propagate_forward_time: 86.5070266724, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7387619019
validation[          , genome  5440] predictions:    6235/  10000 (62.35%), best:    6246/10000 (62.46%), error:     11024.41602, best error:     10912.64746 on epoch:    58, epoch:   62/100, mu: 0.9723716378, learning_rate: 0.0000171076, weight_decay: 0.0000218351

epoch time: 1063.0257568359s, input_fired_time: 11.6142148972, output_fired_time: 4.8621654510, propagate_forward_time: 403.4888305664, propagate_backward_time: 632.7994384766, weight_update_time: 0.3451474309, other_time: 10.2610473633
epoch time: 77.1040954590s, input_fired_time: 0.8115193248, output_fired_time: 0.0000000000, propagate_forward_time: 74.6641998291, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6283798218
validation[          , genome  5440] predictions:    6212/  10000 (62.12%), best:    6246/10000 (62.46%), error:     10967.86621, best error:     10912.64746 on epoch:    58, epoch:   63/100, mu: 0.9732745886, learning_rate: 0.0000158542, weight_decay: 0.0000207513

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_189_5440.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1540823703_189_5440_1_r1168131320_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.530834
read mu: 0.974131
read mu_delta: 0.948779
read initial_learning_rate: 0.00191403
read learning_rate: 1.46926e-05
read learning_rate_delta: 0.926735
read initial_weight_decay: 0.000512888
read weight_decay: 1.97212e-05
read weight_decay_delta: 0.950362
read batch_size: 88
read epsilon: 1e-07
read alpha: 0.108379
read input_dropout_probability: 0.00139765
read hidden_dropout_probability: 0.111816
read velocity_reset: 1822
read epoch: 64
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 58
read number_validation_images: 10000
read best_validation_predictions: 6246
read best_validation_error: 10912.6
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 5440
read normal distribution: '1 0 0'
generator_str: '1854399298'
read generator: 1854399298
read generated_by_map:
1 crossover 1
reading 63 nodes.
reading 279 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1299.60534667969s, input_fired_time: 19.8044834136963, output_fired_time: 10.6975965499878, propagate_forward_time: 475.598846435547, propagate_backward_time: 755.085632324219, weight_update_time: 1.07168197631836, other_time: 38.4187622070313
epoch time: 105.876838684082s, input_fired_time: 2.16943907737732, output_fired_time: 0, propagate_forward_time: 96.400520324707, propagate_backward_time: 0, weight_update_time: 0, other_time: 7.30687713623047
validation[          , genome  5440] predictions:    6197/  10000 (61.97%), best:    6246/10000 (62.46%), error:     11056.28809, best error:     10912.64746 on epoch:    58, epoch:   64/100, mu: 0.9741312861, learning_rate: 0.0000146926, weight_decay: 0.0000197212

epoch time: 1302.2049560547s, input_fired_time: 19.9005775452, output_fired_time: 10.5553236008, propagate_forward_time: 475.5603332520, propagate_backward_time: 756.9801635742, weight_update_time: 1.1068035364, other_time: 39.2085571289
epoch time: 104.1590728760s, input_fired_time: 2.1524097919, output_fired_time: 0.0000000000, propagate_forward_time: 94.7531967163, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2534637451
validation[          , genome  5440] predictions:    6252/  10000 (62.52%), best:    6246/10000 (62.46%), error:     10920.85352, best error:     10912.64746 on epoch:    58, epoch:   65/100, mu: 0.9749441147, learning_rate: 0.0000136162, weight_decay: 0.0000187423

epoch time: 1251.4573974609s, input_fired_time: 19.2144355774, output_fired_time: 10.1265430450, propagate_forward_time: 456.4462890625, propagate_backward_time: 729.8350219727, weight_update_time: 1.0322154760, other_time: 35.8350219727
epoch time: 106.6096954346s, input_fired_time: 2.1863718033, output_fired_time: 0.0000000000, propagate_forward_time: 96.8003692627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.6229553223
validation[          , genome  5440] predictions:    6177/  10000 (61.77%), best:    6246/10000 (62.46%), error:     11019.19238, best error:     10912.64746 on epoch:    58, epoch:   66/100, mu: 0.9757152796, learning_rate: 0.0000126186, weight_decay: 0.0000178120

epoch time: 1239.4227294922s, input_fired_time: 19.4683742523, output_fired_time: 10.2600116730, propagate_forward_time: 449.2066650391, propagate_backward_time: 722.6981811523, weight_update_time: 1.0569020510, other_time: 37.7894897461
epoch time: 96.7579803467s, input_fired_time: 1.9800076485, output_fired_time: 0.0000000000, propagate_forward_time: 88.4472579956, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.3307113647
validation[          , genome  5440] predictions:    6205/  10000 (62.05%), best:    6246/10000 (62.46%), error:     10987.10352, best error:     10912.64746 on epoch:    58, epoch:   67/100, mu: 0.9764469266, learning_rate: 0.0000116941, weight_decay: 0.0000169278

epoch time: 1230.1077880859s, input_fired_time: 19.3539199829, output_fired_time: 10.3583860397, propagate_forward_time: 446.9801025391, propagate_backward_time: 716.7024536133, weight_update_time: 1.0350309610, other_time: 36.7129516602
epoch time: 97.2857894897s, input_fired_time: 2.0084130764, output_fired_time: 0.0000000000, propagate_forward_time: 88.8228836060, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.4544906616
validation[          , genome  5440] predictions:    6260/  10000 (62.60%), best:    6260/10000 (62.60%), error:     10882.91504, best error:     10882.91504 on epoch:    68, epoch:   68/100, mu: 0.9771411419, learning_rate: 0.0000108373, weight_decay: 0.0000160876

epoch time: 1232.2574462891s, input_fired_time: 19.2110805511, output_fired_time: 10.0786962509, propagate_forward_time: 449.7388305664, propagate_backward_time: 716.4217529297, weight_update_time: 1.0627558231, other_time: 36.8070678711
epoch time: 88.8276290894s, input_fired_time: 1.9341722727, output_fired_time: 0.0000000000, propagate_forward_time: 81.2581863403, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.6352691650
validation[          , genome  5440] predictions:    6277/  10000 (62.77%), best:    6277/10000 (62.77%), error:     10863.08984, best error:     10863.08984 on epoch:    69, epoch:   69/100, mu: 0.9777997732, learning_rate: 0.0000100433, weight_decay: 0.0000152890

epoch time: 1235.4412841797s, input_fired_time: 19.7215118408, output_fired_time: 10.4662990570, propagate_forward_time: 449.4543762207, propagate_backward_time: 719.1156005859, weight_update_time: 1.0236086845, other_time: 36.6834716797
epoch time: 96.7284164429s, input_fired_time: 2.0175931454, output_fired_time: 0.0000000000, propagate_forward_time: 88.0177993774, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.6930236816
validation[          , genome  5440] predictions:    6267/  10000 (62.67%), best:    6267/10000 (62.67%), error:     10846.25000, best error:     10846.25000 on epoch:    70, epoch:   70/100, mu: 0.9784246683, learning_rate: 0.0000093075, weight_decay: 0.0000145301

epoch time: 1216.1374511719s, input_fired_time: 19.2679576874, output_fired_time: 10.0623397827, propagate_forward_time: 443.6466064453, propagate_backward_time: 706.1348266602, weight_update_time: 1.0546504259, other_time: 37.0256958008
epoch time: 94.8254699707s, input_fired_time: 2.0161373615, output_fired_time: 0.0000000000, propagate_forward_time: 86.2614288330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.5479049683
validation[          , genome  5440] predictions:    6247/  10000 (62.47%), best:    6267/10000 (62.67%), error:     10953.06934, best error:     10846.25000 on epoch:    70, epoch:   71/100, mu: 0.9790175557, learning_rate: 0.0000086256, weight_decay: 0.0000138089

epoch time: 1199.7963867188s, input_fired_time: 19.1924419403, output_fired_time: 10.1132316589, propagate_forward_time: 434.2848205566, propagate_backward_time: 699.9677734375, weight_update_time: 1.0574839115, other_time: 36.2381591797
epoch time: 85.3475646973s, input_fired_time: 1.8440824747, output_fired_time: 0.0000000000, propagate_forward_time: 78.1450042725, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3584747314
validation[          , genome  5440] predictions:    6293/  10000 (62.93%), best:    6267/10000 (62.67%), error:     10854.27441, best error:     10846.25000 on epoch:    70, epoch:   72/100, mu: 0.9795801044, learning_rate: 0.0000079936, weight_decay: 0.0000131234

epoch time: 1180.8082275391s, input_fired_time: 19.2629032135, output_fired_time: 10.1835603714, propagate_forward_time: 428.8260803223, propagate_backward_time: 686.4401855469, weight_update_time: 1.0257531404, other_time: 36.0954589844
epoch time: 87.7656250000s, input_fired_time: 1.8706611395, output_fired_time: 0.0000000000, propagate_forward_time: 80.0656356812, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.8293304443
validation[          , genome  5440] predictions:    6267/  10000 (62.67%), best:    6267/10000 (62.67%), error:     10925.28516, best error:     10846.25000 on epoch:    70, epoch:   73/100, mu: 0.9801138043, learning_rate: 0.0000074080, weight_decay: 0.0000124720

epoch time: 1125.0932617188s, input_fired_time: 19.0622920990, output_fired_time: 9.8719491959, propagate_forward_time: 406.6491394043, propagate_backward_time: 654.8888549805, weight_update_time: 1.0271695852, other_time: 34.6210327148
epoch time: 80.3773498535s, input_fired_time: 1.8581645489, output_fired_time: 0.0000000000, propagate_forward_time: 73.4393234253, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0798645020
validation[          , genome  5440] predictions:    6303/  10000 (63.03%), best:    6303/10000 (63.03%), error:     10829.60156, best error:     10829.60156 on epoch:    74, epoch:   74/100, mu: 0.9806202054, learning_rate: 0.0000068652, weight_decay: 0.0000118529

epoch time: 1110.8858642578s, input_fired_time: 18.5047283173, output_fired_time: 9.5844678879, propagate_forward_time: 399.3257446289, propagate_backward_time: 650.2703247070, weight_update_time: 0.9946525097, other_time: 33.2005615234
epoch time: 88.0511398315s, input_fired_time: 1.8894637823, output_fired_time: 0.0000000000, propagate_forward_time: 80.1780014038, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.9836730957
validation[          , genome  5440] predictions:    6279/  10000 (62.79%), best:    6303/10000 (63.03%), error:     10912.66797, best error:     10829.60156 on epoch:    74, epoch:   75/100, mu: 0.9811006188, learning_rate: 0.0000063622, weight_decay: 0.0000112646

epoch time: 1115.2673339844s, input_fired_time: 19.0816440582, output_fired_time: 10.0745782852, propagate_forward_time: 403.4047241211, propagate_backward_time: 648.0463867188, weight_update_time: 1.0263228416, other_time: 34.6599731445
epoch time: 88.3622741699s, input_fired_time: 1.8897855282, output_fired_time: 0.0000000000, propagate_forward_time: 80.7564315796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.7160568237
validation[          , genome  5440] predictions:    6284/  10000 (62.84%), best:    6284/10000 (62.84%), error:     10817.70898, best error:     10817.70898 on epoch:    76, epoch:   76/100, mu: 0.9815564752, learning_rate: 0.0000058961, weight_decay: 0.0000107054

epoch time: 1060.9652099609s, input_fired_time: 18.8094882965, output_fired_time: 9.8246278763, propagate_forward_time: 383.1090087891, propagate_backward_time: 616.8056030273, weight_update_time: 0.9963793755, other_time: 32.4165649414
epoch time: 72.2349777222s, input_fired_time: 1.7645380497, output_fired_time: 0.0000000000, propagate_forward_time: 65.5883026123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8821334839
validation[          , genome  5440] predictions:    6199/  10000 (61.99%), best:    6284/10000 (62.84%), error:     11056.02637, best error:     10817.70898 on epoch:    76, epoch:   77/100, mu: 0.9819889665, learning_rate: 0.0000054641, weight_decay: 0.0000101741

epoch time: 1137.2839355469s, input_fired_time: 19.6134872437, output_fired_time: 10.3681783676, propagate_forward_time: 411.0420837402, propagate_backward_time: 659.9086914063, weight_update_time: 1.0617759228, other_time: 36.3514404297
epoch time: 85.1687545776s, input_fired_time: 1.9293226004, output_fired_time: 0.0000000000, propagate_forward_time: 77.6493988037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5900344849
validation[          , genome  5440] predictions:    6309/  10000 (63.09%), best:    6284/10000 (62.84%), error:     10855.26367, best error:     10817.70898 on epoch:    76, epoch:   78/100, mu: 0.9823992848, learning_rate: 0.0000050638, weight_decay: 0.0000096690

epoch time: 1082.1892089844s, input_fired_time: 18.5987339020, output_fired_time: 9.7568264008, propagate_forward_time: 391.4306945801, propagate_backward_time: 629.9922485352, weight_update_time: 0.9896944165, other_time: 32.4107055664
epoch time: 102.7420349121s, input_fired_time: 2.4388718605, output_fired_time: 0.0000000000, propagate_forward_time: 92.0587997437, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.2443618774
validation[          , genome  5440] predictions:    6324/  10000 (63.24%), best:    6324/10000 (63.24%), error:     10775.62988, best error:     10775.62988 on epoch:    79, epoch:   79/100, mu: 0.9827886224, learning_rate: 0.0000046928, weight_decay: 0.0000091891

epoch time: 1098.4252929688s, input_fired_time: 18.8821334839, output_fired_time: 9.8475179672, propagate_forward_time: 398.8579711914, propagate_backward_time: 637.7717895508, weight_update_time: 1.0118693113, other_time: 33.0659179688
epoch time: 99.8842697144s, input_fired_time: 2.2092204094, output_fired_time: 0.0000000000, propagate_forward_time: 90.0483932495, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.6266555786
validation[          , genome  5440] predictions:    6304/  10000 (63.04%), best:    6324/10000 (63.24%), error:     10808.55078, best error:     10775.62988 on epoch:    79, epoch:   80/100, mu: 0.9831579924, learning_rate: 0.0000043490, weight_decay: 0.0000087330

epoch time: 1066.8039550781s, input_fired_time: 18.7399444580, output_fired_time: 9.8135375977, propagate_forward_time: 382.5592956543, propagate_backward_time: 623.3273315430, weight_update_time: 0.9857399464, other_time: 32.3638305664
epoch time: 84.8883056641s, input_fired_time: 1.9856228828, output_fired_time: 0.0000000000, propagate_forward_time: 76.8292465210, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0734329224
validation[          , genome  5440] predictions:    6304/  10000 (63.04%), best:    6324/10000 (63.24%), error:     10789.89355, best error:     10775.62988 on epoch:    79, epoch:   81/100, mu: 0.9835084677, learning_rate: 0.0000040303, weight_decay: 0.0000082995

epoch time: 1052.2270507813s, input_fired_time: 18.7106952667, output_fired_time: 9.6980724335, propagate_forward_time: 377.5302124023, propagate_backward_time: 614.5676879883, weight_update_time: 0.9916598797, other_time: 31.7203979492
epoch time: 100.0160369873s, input_fired_time: 2.2883663177, output_fired_time: 0.0000000000, propagate_forward_time: 90.2374191284, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.4902496338
validation[          , genome  5440] predictions:    6286/  10000 (62.86%), best:    6324/10000 (63.24%), error:     10839.93652, best error:     10775.62988 on epoch:    79, epoch:   82/100, mu: 0.9838409424, learning_rate: 0.0000037351, weight_decay: 0.0000078875

epoch time: 1195.0812988281s, input_fired_time: 20.1714954376, output_fired_time: 10.4358959198, propagate_forward_time: 432.2024230957, propagate_backward_time: 696.3822631836, weight_update_time: 1.0633902550, other_time: 35.8892211914
epoch time: 93.4641799927s, input_fired_time: 2.1057331562, output_fired_time: 0.0000000000, propagate_forward_time: 85.2681350708, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.0903091431
validation[          , genome  5440] predictions:    6288/  10000 (62.88%), best:    6324/10000 (63.24%), error:     10796.78906, best error:     10775.62988 on epoch:    79, epoch:   83/100, mu: 0.9841564298, learning_rate: 0.0000034614, weight_decay: 0.0000074960

epoch time: 1193.4111328125s, input_fired_time: 19.8550071716, output_fired_time: 10.3457260132, propagate_forward_time: 433.0099792480, propagate_backward_time: 694.2285766602, weight_update_time: 1.0738439560, other_time: 35.9718627930
epoch time: 93.1454772949s, input_fired_time: 2.1145603657, output_fired_time: 0.0000000000, propagate_forward_time: 84.8435516357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.1873626709
validation[          , genome  5440] predictions:    6282/  10000 (62.82%), best:    6324/10000 (63.24%), error:     10815.94922, best error:     10775.62988 on epoch:    79, epoch:   84/100, mu: 0.9844557643, learning_rate: 0.0000032078, weight_decay: 0.0000071239

epoch time: 1193.9733886719s, input_fired_time: 20.1530628204, output_fired_time: 10.4234485626, propagate_forward_time: 431.4485778809, propagate_backward_time: 695.5658569336, weight_update_time: 1.0643628836, other_time: 36.3823852539
epoch time: 99.1728134155s, input_fired_time: 2.1597359180, output_fired_time: 0.0000000000, propagate_forward_time: 90.3108978271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.7021789551
validation[          , genome  5440] predictions:    6305/  10000 (63.05%), best:    6324/10000 (63.24%), error:     10795.71875, best error:     10775.62988 on epoch:    79, epoch:   85/100, mu: 0.9847397208, learning_rate: 0.0000029728, weight_decay: 0.0000067703

epoch time: 1234.8917236328s, input_fired_time: 20.2750282288, output_fired_time: 10.6369047165, propagate_forward_time: 446.1388854980, propagate_backward_time: 721.4993896484, weight_update_time: 1.0818643570, other_time: 36.3414306641
epoch time: 97.9735183716s, input_fired_time: 2.2113873959, output_fired_time: 0.0000000000, propagate_forward_time: 88.8564987183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.9056320190
validation[          , genome  5440] predictions:    6296/  10000 (62.96%), best:    6324/10000 (63.24%), error:     10782.88770, best error:     10775.62988 on epoch:    79, epoch:   86/100, mu: 0.9850091338, learning_rate: 0.0000027550, weight_decay: 0.0000064342

epoch time: 1237.3100585938s, input_fired_time: 17.8323783875, output_fired_time: 8.7383451462, propagate_forward_time: 454.0997924805, propagate_backward_time: 725.0034179688, weight_update_time: 0.9022971988, other_time: 31.6360473633
epoch time: 100.3803558350s, input_fired_time: 1.7689454556, output_fired_time: 0.0000000000, propagate_forward_time: 92.1435012817, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 6.4679107666
validation[          , genome  5440] predictions:    6300/  10000 (63.00%), best:    6324/10000 (63.24%), error:     10790.44531, best error:     10775.62988 on epoch:    79, epoch:   87/100, mu: 0.9852647781, learning_rate: 0.0000025531, weight_decay: 0.0000061149

epoch time: 1143.9421386719s, input_fired_time: 16.6369819641, output_fired_time: 7.9177641869, propagate_forward_time: 422.8352661133, propagate_backward_time: 668.6077880859, weight_update_time: 0.8149666190, other_time: 27.9443969727
epoch time: 88.6193771362s, input_fired_time: 1.5199387074, output_fired_time: 0.0000000000, propagate_forward_time: 82.1844024658, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9150390625
validation[          , genome  5440] predictions:    6315/  10000 (63.15%), best:    6324/10000 (63.24%), error:     10793.15527, best error:     10775.62988 on epoch:    79, epoch:   88/100, mu: 0.9855073094, learning_rate: 0.0000023661, weight_decay: 0.0000058113

epoch time: 1166.7062988281s, input_fired_time: 16.5497016907, output_fired_time: 7.9326729774, propagate_forward_time: 425.9023132324, propagate_backward_time: 688.0968017578, weight_update_time: 0.8111410141, other_time: 28.2248535156
epoch time: 96.5980606079s, input_fired_time: 1.5489293337, output_fired_time: 0.0000000000, propagate_forward_time: 90.0399856567, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.0091476440
validation[          , genome  5440] predictions:    6336/  10000 (63.36%), best:    6336/10000 (63.36%), error:     10771.52246, best error:     10771.52246 on epoch:    89, epoch:   89/100, mu: 0.9857374430, learning_rate: 0.0000021927, weight_decay: 0.0000055229

epoch time: 1184.1158447266s, input_fired_time: 16.7332916260, output_fired_time: 8.0437107086, propagate_forward_time: 436.3613586426, propagate_backward_time: 694.5621948242, weight_update_time: 0.7933745980, other_time: 28.4153442383
epoch time: 88.2156753540s, input_fired_time: 1.4738527536, output_fired_time: 0.0000000000, propagate_forward_time: 82.0568161011, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6850051880
validation[          , genome  5440] predictions:    6330/  10000 (63.30%), best:    6330/10000 (63.30%), error:     10752.67676, best error:     10752.67676 on epoch:    90, epoch:   90/100, mu: 0.9859557748, learning_rate: 0.0000020321, weight_decay: 0.0000052487

epoch time: 1156.0167236328s, input_fired_time: 15.1597709656, output_fired_time: 6.9337801933, propagate_forward_time: 429.6878662109, propagate_backward_time: 681.1192016602, weight_update_time: 0.6626299024, other_time: 23.1160278320
epoch time: 95.6527252197s, input_fired_time: 1.1003224850, output_fired_time: 0.0000000000, propagate_forward_time: 91.1821594238, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3702468872
validation[          , genome  5440] predictions:    6317/  10000 (63.17%), best:    6330/10000 (63.30%), error:     10767.89941, best error:     10752.67676 on epoch:    90, epoch:   91/100, mu: 0.9861629009, learning_rate: 0.0000018832, weight_decay: 0.0000049882

epoch time: 947.5178222656s, input_fired_time: 12.5101070404, output_fired_time: 4.8212471008, propagate_forward_time: 350.3648986816, propagate_backward_time: 565.9539794922, weight_update_time: 0.4102750719, other_time: 13.8675537109
epoch time: 73.2443237305s, input_fired_time: 0.8756746054, output_fired_time: 0.0000000000, propagate_forward_time: 70.3146820068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0539703369
validation[          , genome  5440] predictions:    6327/  10000 (63.27%), best:    6330/10000 (63.30%), error:     10769.44922, best error:     10752.67676 on epoch:    90, epoch:   92/100, mu: 0.9863594174, learning_rate: 0.0000017452, weight_decay: 0.0000047406

epoch time: 966.9959106445s, input_fired_time: 12.5601902008, output_fired_time: 4.9275913239, propagate_forward_time: 357.7944335938, propagate_backward_time: 577.6231689453, weight_update_time: 0.4126472771, other_time: 14.0905151367
epoch time: 77.0613784790s, input_fired_time: 0.9092480540, output_fired_time: 0.0000000000, propagate_forward_time: 74.0500869751, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1020431519
validation[          , genome  5440] predictions:    6341/  10000 (63.41%), best:    6341/10000 (63.41%), error:     10742.72461, best error:     10742.72461 on epoch:    93, epoch:   93/100, mu: 0.9865458608, learning_rate: 0.0000016174, weight_decay: 0.0000045053

epoch time: 953.7045898438s, input_fired_time: 12.5709543228, output_fired_time: 4.9169716835, propagate_forward_time: 352.8973693848, propagate_backward_time: 568.9465332031, weight_update_time: 0.4145488143, other_time: 14.3726806641
epoch time: 71.9428176880s, input_fired_time: 0.8730910420, output_fired_time: 0.0000000000, propagate_forward_time: 68.7450485229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3246765137
validation[          , genome  5440] predictions:    6304/  10000 (63.04%), best:    6341/10000 (63.41%), error:     10764.34668, best error:     10742.72461 on epoch:    93, epoch:   94/100, mu: 0.9867227674, learning_rate: 0.0000014989, weight_decay: 0.0000042816

epoch time: 1090.3537597656s, input_fired_time: 12.9868202209, output_fired_time: 5.5811767578, propagate_forward_time: 409.4423828125, propagate_backward_time: 646.1532592773, weight_update_time: 0.4560225606, other_time: 16.1901245117
epoch time: 84.9965667725s, input_fired_time: 0.8719021082, output_fired_time: 0.0000000000, propagate_forward_time: 82.4187240601, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7059402466
validation[          , genome  5440] predictions:    6311/  10000 (63.11%), best:    6341/10000 (63.41%), error:     10784.67285, best error:     10742.72461 on epoch:    93, epoch:   95/100, mu: 0.9868906140, learning_rate: 0.0000013891, weight_decay: 0.0000040691

epoch time: 789.3490600586s, input_fired_time: 10.7985076904, output_fired_time: 3.8276133537, propagate_forward_time: 291.0964050293, propagate_backward_time: 474.5571594238, weight_update_time: 0.2786711454, other_time: 9.0693359375
epoch time: 61.5421371460s, input_fired_time: 0.7163842916, output_fired_time: 0.0000000000, propagate_forward_time: 59.3933219910, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4324302673
validation[          , genome  5440] predictions:    6323/  10000 (63.23%), best:    6341/10000 (63.41%), error:     10759.94336, best error:     10742.72461 on epoch:    93, epoch:   96/100, mu: 0.9870498776, learning_rate: 0.0000012873, weight_decay: 0.0000038671

epoch time: 740.3833618164s, input_fired_time: 10.6183881760, output_fired_time: 3.6514506340, propagate_forward_time: 270.8800964355, propagate_backward_time: 446.4249267578, weight_update_time: 0.2646696866, other_time: 8.8085021973
epoch time: 59.2716445923s, input_fired_time: 0.7031812668, output_fired_time: 0.0000000000, propagate_forward_time: 57.1307601929, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4377021790
validation[          , genome  5440] predictions:    6335/  10000 (63.35%), best:    6341/10000 (63.41%), error:     10791.96191, best error:     10742.72461 on epoch:    93, epoch:   97/100, mu: 0.9872009754, learning_rate: 0.0000011930, weight_decay: 0.0000036752

epoch time: 705.8798828125s, input_fired_time: 10.5089874268, output_fired_time: 3.5164701939, propagate_forward_time: 256.4426879883, propagate_backward_time: 426.7761230469, weight_update_time: 0.2559746504, other_time: 8.6356201172
epoch time: 87.0390167236s, input_fired_time: 0.8811847568, output_fired_time: 0.0000000000, propagate_forward_time: 84.4728698730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6849594116
validation[          , genome  5440] predictions:    6324/  10000 (63.24%), best:    6341/10000 (63.41%), error:     10787.14941, best error:     10742.72461 on epoch:    93, epoch:   98/100, mu: 0.9873443246, learning_rate: 0.0000011056, weight_decay: 0.0000034928

epoch time: 763.4112548828s, input_fired_time: 10.6981143951, output_fired_time: 3.7459301949, propagate_forward_time: 280.3175354004, propagate_backward_time: 459.7224731445, weight_update_time: 0.2707900405, other_time: 8.9272155762
epoch time: 58.0253562927s, input_fired_time: 0.6917102933, output_fired_time: 0.0000000000, propagate_forward_time: 55.9254112244, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4082336426
validation[          , genome  5440] predictions:    6311/  10000 (63.11%), best:    6341/10000 (63.41%), error:     10780.43555, best error:     10742.72461 on epoch:    93, epoch:   99/100, mu: 0.9874803424, learning_rate: 0.0000010246, weight_decay: 0.0000033194

epoch time: 719.6042480469s, input_fired_time: 10.5555257797, output_fired_time: 3.5618171692, propagate_forward_time: 262.2428894043, propagate_backward_time: 434.5367736816, weight_update_time: 0.2590228617, other_time: 8.7072143555
epoch time: 58.8759002686s, input_fired_time: 0.6979733109, output_fired_time: 0.0000000000, propagate_forward_time: 56.7605400085, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4173851013
validation[          , genome  5440] predictions:    6329/  10000 (63.29%), best:    6341/10000 (63.41%), error:     10763.62012, best error:     10742.72461 on epoch:    93, epoch:  100/100, mu: 0.9876093864, learning_rate: 0.0000009495, weight_decay: 0.0000031546

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 293.5196838379s, input_fired_time: 3.5060443878, output_fired_time: 0.0000000000, propagate_forward_time: 282.7801513672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.2334899902
best training[          , genome  5440] predictions:   34435/  50000 (68.87%), best:    6341/10000 (63.41%), error:     47588.77734, best error:     10742.72461 on epoch:    93, epoch:  101/100, mu: 0.9877318144, learning_rate: 0.0000008799, weight_decay: 0.0000029980
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 62.4851837158s, input_fired_time: 0.7201526165, output_fired_time: 0.0000000000, propagate_forward_time: 60.3065376282, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4584922791
   testing[          , genome  5440] predictions:    6341/  10000 (63.41%), best:    6341/10000 (63.41%), error:     10742.72461, best error:     10742.72461 on epoch:    93, epoch:  101/100, mu: 0.9877318144, learning_rate: 0.0000008799, weight_decay: 0.0000029980
backpropagation finished successfully!
01:54:21 (4620): called boinc_finish(0)

</stderr_txt>
]]>