Nameexact_genome_1542692703_186_6400_2
Workunit4100620
Created20 Nov 2018, 9:18:58 UTC
Sent20 Nov 2018, 9:58:45 UTC
Report deadline6 Mar 2019, 9:56:54 UTC
Received28 Nov 2018, 2:03:28 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88000
Run time2 days 4 hours 46 min 50 sec
CPU time2 days 4 hours 45 min 9 sec
Validate stateValid
Credit26,499.68
Device peak FLOPS3.82 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size540.29 MB
Peak swap size538.59 MB
Peak disk usage11.85 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
d_by_map:
1 crossover 1
reading 92 nodes.
reading 348 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1732.11645507813s, input_fired_time: 18.725471496582, output_fired_time: 9.92523956298828, propagate_forward_time: 694.068908691406, propagate_backward_time: 995.375671386719, weight_update_time: 0.953831970691681, other_time: 14.0211181640625
epoch time: 142.614288330078s, input_fired_time: 1.5663355588913, output_fired_time: 0, propagate_forward_time: 138.620651245117, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.42730712890625
validation[          , genome  6400] predictions:    5636/  10000 (56.36%), best:    6175/10000 (61.75%), error:     12689.65234, best error:     11320.15820 on epoch:     0, epoch:   26/100, mu: 0.8739089966, learning_rate: 0.0000269139, weight_decay: 0.0001340490

epoch time: 1738.3275146484s, input_fired_time: 18.7910118103, output_fired_time: 9.9555311203, propagate_forward_time: 696.8068237305, propagate_backward_time: 998.6437988281, weight_update_time: 0.9585655928, other_time: 14.1303100586
epoch time: 141.2452087402s, input_fired_time: 1.5240945816, output_fired_time: 0.0000000000, propagate_forward_time: 137.4209747314, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3001403809
validation[          , genome  6400] predictions:    5727/  10000 (57.27%), best:    6175/10000 (61.75%), error:     12563.08594, best error:     11320.15820 on epoch:     0, epoch:   27/100, mu: 0.8801099658, learning_rate: 0.0000251672, weight_decay: 0.0001267503

epoch time: 1723.4111328125s, input_fired_time: 18.9569053650, output_fired_time: 9.9135675430, propagate_forward_time: 689.7200927734, propagate_backward_time: 990.3802490234, weight_update_time: 0.9636037946, other_time: 14.4403076172
epoch time: 143.0989074707s, input_fired_time: 1.5322345495, output_fired_time: 0.0000000000, propagate_forward_time: 139.2562103271, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3104553223
validation[          , genome  6400] predictions:    5872/  10000 (58.72%), best:    6175/10000 (61.75%), error:     12212.26953, best error:     11320.15820 on epoch:     0, epoch:   28/100, mu: 0.8859797120, learning_rate: 0.0000235338, weight_decay: 0.0001198490

epoch time: 1724.5196533203s, input_fired_time: 18.8829936981, output_fired_time: 9.9090890884, propagate_forward_time: 689.9892578125, propagate_backward_time: 991.2921142578, weight_update_time: 0.9612931609, other_time: 14.4462890625
epoch time: 143.7557067871s, input_fired_time: 1.5391967297, output_fired_time: 0.0000000000, propagate_forward_time: 139.8865509033, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3299560547
validation[          , genome  6400] predictions:    5816/  10000 (58.16%), best:    6175/10000 (61.75%), error:     12203.69336, best error:     11320.15820 on epoch:     0, epoch:   29/100, mu: 0.8915359378, learning_rate: 0.0000220065, weight_decay: 0.0001133235

epoch time: 1719.8979492188s, input_fired_time: 18.9368209839, output_fired_time: 9.9026536942, propagate_forward_time: 686.9602050781, propagate_backward_time: 988.8908691406, weight_update_time: 0.9644629955, other_time: 15.2073974609
epoch time: 143.2364959717s, input_fired_time: 1.5793577433, output_fired_time: 0.0000000000, propagate_forward_time: 139.2235717773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4335632324
validation[          , genome  6400] predictions:    5810/  10000 (58.10%), best:    6175/10000 (61.75%), error:     12322.55957, best error:     11320.15820 on epoch:     0, epoch:   30/100, mu: 0.8967953920, learning_rate: 0.0000205783, weight_decay: 0.0001071532

epoch time: 1723.4329833984s, input_fired_time: 18.9932765961, output_fired_time: 9.9151601791, propagate_forward_time: 689.3237304688, propagate_backward_time: 990.6297607422, weight_update_time: 0.9654417634, other_time: 14.5710449219
epoch time: 146.5484924316s, input_fired_time: 1.5144479275, output_fired_time: 0.0000000000, propagate_forward_time: 142.6305236816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4035186768
validation[          , genome  6400] predictions:    5627/  10000 (56.27%), best:    6175/10000 (61.75%), error:     12505.52832, best error:     11320.15820 on epoch:     0, epoch:   31/100, mu: 0.9017738700, learning_rate: 0.0000192427, weight_decay: 0.0001013190

epoch time: 1732.3304443359s, input_fired_time: 19.0452365875, output_fired_time: 9.9610729218, propagate_forward_time: 692.2985839844, propagate_backward_time: 995.4630737305, weight_update_time: 0.9746332169, other_time: 15.5624389648
epoch time: 142.8059844971s, input_fired_time: 1.5347812176, output_fired_time: 0.0000000000, propagate_forward_time: 138.9497528076, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3214569092
validation[          , genome  6400] predictions:    5588/  10000 (55.88%), best:    6175/10000 (61.75%), error:     12748.70410, best error:     11320.15820 on epoch:     0, epoch:   32/100, mu: 0.9064864516, learning_rate: 0.0000179939, weight_decay: 0.0000958023

epoch time: 1725.6655273438s, input_fired_time: 19.0033950806, output_fired_time: 9.9150886536, propagate_forward_time: 690.0385131836, propagate_backward_time: 991.7145385742, weight_update_time: 0.9665753841, other_time: 14.9940185547
epoch time: 142.1276550293s, input_fired_time: 1.5534752607, output_fired_time: 0.0000000000, propagate_forward_time: 138.2287597656, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3454132080
validation[          , genome  6400] predictions:    5842/  10000 (58.42%), best:    6175/10000 (61.75%), error:     12210.00879, best error:     11320.15820 on epoch:     0, epoch:   33/100, mu: 0.9109473228, learning_rate: 0.0000168261, weight_decay: 0.0000905861

epoch time: 1722.6280517578s, input_fired_time: 18.8118820190, output_fired_time: 9.8751173019, propagate_forward_time: 688.2741699219, propagate_backward_time: 991.0356445313, weight_update_time: 0.9559777379, other_time: 14.6312255859
epoch time: 144.1830902100s, input_fired_time: 1.5782148838, output_fired_time: 0.0000000000, propagate_forward_time: 140.1418762207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4629974365
validation[          , genome  6400] predictions:    5877/  10000 (58.77%), best:    6175/10000 (61.75%), error:     12111.25586, best error:     11320.15820 on epoch:     0, epoch:   34/100, mu: 0.9151698947, learning_rate: 0.0000157341, weight_decay: 0.0000856539

epoch time: 1730.7808837891s, input_fired_time: 18.9229831696, output_fired_time: 9.9142217636, propagate_forward_time: 692.7261962891, propagate_backward_time: 994.8067626953, weight_update_time: 0.9621584415, other_time: 14.4107666016
epoch time: 142.7119750977s, input_fired_time: 1.5434138775, output_fired_time: 0.0000000000, propagate_forward_time: 138.8203887939, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3481750488
validation[          , genome  6400] predictions:    5850/  10000 (58.50%), best:    6175/10000 (61.75%), error:     12240.41406, best error:     11320.15820 on epoch:     0, epoch:   35/100, mu: 0.9191669226, learning_rate: 0.0000147129, weight_decay: 0.0000809902

epoch time: 1733.7475585938s, input_fired_time: 18.9566612244, output_fired_time: 9.9236268997, propagate_forward_time: 692.8562011719, propagate_backward_time: 997.3008422852, weight_update_time: 0.9657235742, other_time: 14.7102661133
epoch time: 142.6463928223s, input_fired_time: 1.5655032396, output_fired_time: 0.0000000000, propagate_forward_time: 138.7163543701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3645324707
validation[          , genome  6400] predictions:    5977/  10000 (59.77%), best:    6175/10000 (61.75%), error:     11826.02832, best error:     11320.15820 on epoch:     0, epoch:   36/100, mu: 0.9229504466, learning_rate: 0.0000137581, weight_decay: 0.0000765804

epoch time: 1739.1810302734s, input_fired_time: 18.9151954651, output_fired_time: 9.9579544067, propagate_forward_time: 694.4507446289, propagate_backward_time: 1000.8484497070, weight_update_time: 0.9644155502, other_time: 15.0086669922
epoch time: 141.5548248291s, input_fired_time: 1.5310080051, output_fired_time: 0.0000000000, propagate_forward_time: 137.7227020264, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3011169434
validation[          , genome  6400] predictions:    5909/  10000 (59.09%), best:    6175/10000 (61.75%), error:     12000.91992, best error:     11320.15820 on epoch:     0, epoch:   37/100, mu: 0.9265319109, learning_rate: 0.0000128652, weight_decay: 0.0000724108

epoch time: 1732.9016113281s, input_fired_time: 18.8641796112, output_fired_time: 9.9163627625, propagate_forward_time: 692.6471557617, propagate_backward_time: 997.0596923828, weight_update_time: 0.9590496421, other_time: 14.4142456055
epoch time: 142.6251831055s, input_fired_time: 1.5605417490, output_fired_time: 0.0000000000, propagate_forward_time: 138.5964508057, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4681854248
validation[          , genome  6400] predictions:    5895/  10000 (58.95%), best:    6175/10000 (61.75%), error:     12090.58008, best error:     11320.15820 on epoch:     0, epoch:   38/100, mu: 0.9299220443, learning_rate: 0.0000120302, weight_decay: 0.0000684681

epoch time: 1723.5654296875s, input_fired_time: 18.8058967590, output_fired_time: 9.8697309494, propagate_forward_time: 689.1628417969, propagate_backward_time: 991.4848632813, weight_update_time: 0.9539150596, other_time: 14.2420654297
epoch time: 139.4728698730s, input_fired_time: 1.4922754765, output_fired_time: 0.0000000000, propagate_forward_time: 135.0484924316, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9320983887
validation[          , genome  6400] predictions:    5831/  10000 (58.31%), best:    6175/10000 (61.75%), error:     12175.18066, best error:     11320.15820 on epoch:     0, epoch:   39/100, mu: 0.9331310987, learning_rate: 0.0000112495, weight_decay: 0.0000647402

epoch time: 1723.4409179688s, input_fired_time: 18.8524188995, output_fired_time: 9.9007148743, propagate_forward_time: 687.3148193359, propagate_backward_time: 991.0704956055, weight_update_time: 0.9616712928, other_time: 16.3024291992
epoch time: 143.8191833496s, input_fired_time: 1.5698300600, output_fired_time: 0.0000000000, propagate_forward_time: 139.3392181396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9101409912
validation[          , genome  6400] predictions:    5893/  10000 (58.93%), best:    6175/10000 (61.75%), error:     12130.27246, best error:     11320.15820 on epoch:     0, epoch:   40/100, mu: 0.9361687303, learning_rate: 0.0000105194, weight_decay: 0.0000612152

epoch time: 1717.8703613281s, input_fired_time: 18.7963695526, output_fired_time: 9.8607616425, propagate_forward_time: 684.6118774414, propagate_backward_time: 988.8235473633, weight_update_time: 0.9571651220, other_time: 15.7778320313
epoch time: 141.4778900146s, input_fired_time: 1.5233978033, output_fired_time: 0.0000000000, propagate_forward_time: 136.7588806152, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1956176758
validation[          , genome  6400] predictions:    5756/  10000 (57.56%), best:    6175/10000 (61.75%), error:     12363.86816, best error:     11320.15820 on epoch:     0, epoch:   41/100, mu: 0.9390441179, learning_rate: 0.0000098367, weight_decay: 0.0000578822

epoch time: 1708.6965332031s, input_fired_time: 18.8241786957, output_fired_time: 9.8447942734, propagate_forward_time: 680.6895751953, propagate_backward_time: 983.1160888672, weight_update_time: 0.9593387246, other_time: 16.2218017578
epoch time: 142.2916107178s, input_fired_time: 1.5848498344, output_fired_time: 0.0000000000, propagate_forward_time: 138.2331695557, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4735870361
validation[          , genome  6400] predictions:    5963/  10000 (59.63%), best:    6175/10000 (61.75%), error:     11829.98242, best error:     11320.15820 on epoch:     0, epoch:   42/100, mu: 0.9417659044, learning_rate: 0.0000091983, weight_decay: 0.0000547306

epoch time: 1723.0510253906s, input_fired_time: 18.9646930695, output_fired_time: 9.9113883972, propagate_forward_time: 688.0109863281, propagate_backward_time: 990.7819213867, weight_update_time: 0.9634401202, other_time: 15.3820190430
epoch time: 141.3495330811s, input_fired_time: 1.5341851711, output_fired_time: 0.0000000000, propagate_forward_time: 137.3147735596, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5005798340
validation[          , genome  6400] predictions:    5666/  10000 (56.66%), best:    6175/10000 (61.75%), error:     12494.36328, best error:     11320.15820 on epoch:     0, epoch:   43/100, mu: 0.9443423152, learning_rate: 0.0000086013, weight_decay: 0.0000517506

epoch time: 1723.8250732422s, input_fired_time: 18.9500198364, output_fired_time: 9.9041757584, propagate_forward_time: 688.6693115234, propagate_backward_time: 990.7761230469, weight_update_time: 0.9616123438, other_time: 15.5253906250
epoch time: 141.7346343994s, input_fired_time: 1.5518133640, output_fired_time: 0.0000000000, propagate_forward_time: 137.3260803223, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8567352295
validation[          , genome  6400] predictions:    5923/  10000 (59.23%), best:    6175/10000 (61.75%), error:     11904.25488, best error:     11320.15820 on epoch:     0, epoch:   44/100, mu: 0.9467810988, learning_rate: 0.0000080431, weight_decay: 0.0000489329

epoch time: 1721.7575683594s, input_fired_time: 18.9610176086, output_fired_time: 9.9032897949, propagate_forward_time: 686.6301879883, propagate_backward_time: 990.1129760742, weight_update_time: 0.9655442834, other_time: 16.1500244141
epoch time: 144.1833648682s, input_fired_time: 1.5834839344, output_fired_time: 0.0000000000, propagate_forward_time: 139.9995269775, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6003570557
validation[          , genome  6400] predictions:    5979/  10000 (59.79%), best:    6175/10000 (61.75%), error:     11809.01465, best error:     11320.15820 on epoch:     0, epoch:   45/100, mu: 0.9490896463, learning_rate: 0.0000075211, weight_decay: 0.0000462686

epoch time: 1721.9035644531s, input_fired_time: 18.9516429901, output_fired_time: 9.9007844925, propagate_forward_time: 686.9794311523, propagate_backward_time: 990.7008666992, weight_update_time: 0.9639214277, other_time: 15.3708496094
epoch time: 143.1716003418s, input_fired_time: 1.5975487232, output_fired_time: 0.0000000000, propagate_forward_time: 139.1255645752, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4484863281
validation[          , genome  6400] predictions:    5972/  10000 (59.72%), best:    6175/10000 (61.75%), error:     11934.12305, best error:     11320.15820 on epoch:     0, epoch:   46/100, mu: 0.9512748718, learning_rate: 0.0000070330, weight_decay: 0.0000437494

epoch time: 1723.1179199219s, input_fired_time: 18.9360618591, output_fired_time: 9.8958387375, propagate_forward_time: 687.2448120117, propagate_backward_time: 991.5009765625, weight_update_time: 0.9627491236, other_time: 15.5402221680
epoch time: 139.5084075928s, input_fired_time: 1.5191916227, output_fired_time: 0.0000000000, propagate_forward_time: 134.2440032959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7452087402
validation[          , genome  6400] predictions:    5981/  10000 (59.81%), best:    6175/10000 (61.75%), error:     11808.62402, best error:     11320.15820 on epoch:     0, epoch:   47/100, mu: 0.9533433914, learning_rate: 0.0000065765, weight_decay: 0.0000413673

epoch time: 1703.0230712891s, input_fired_time: 18.6756343842, output_fired_time: 9.7929363251, propagate_forward_time: 678.2355346680, propagate_backward_time: 980.9671630859, weight_update_time: 0.9481029510, other_time: 15.3517456055
epoch time: 141.9502563477s, input_fired_time: 1.5407278538, output_fired_time: 0.0000000000, propagate_forward_time: 138.1095733643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2999572754
validation[          , genome  6400] predictions:    5926/  10000 (59.26%), best:    6175/10000 (61.75%), error:     11960.54199, best error:     11320.15820 on epoch:     0, epoch:   48/100, mu: 0.9553014040, learning_rate: 0.0000061497, weight_decay: 0.0000391149

epoch time: 1713.1614990234s, input_fired_time: 18.9991130829, output_fired_time: 9.8776807785, propagate_forward_time: 681.7125854492, propagate_backward_time: 985.8391113281, weight_update_time: 0.9687221050, other_time: 16.7329711914
epoch time: 141.3151245117s, input_fired_time: 1.5361577272, output_fired_time: 0.0000000000, propagate_forward_time: 137.4814300537, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2975311279
validation[          , genome  6400] predictions:    5973/  10000 (59.73%), best:    6175/10000 (61.75%), error:     11876.96289, best error:     11320.15820 on epoch:     0, epoch:   49/100, mu: 0.9571548104, learning_rate: 0.0000057506, weight_decay: 0.0000369852

epoch time: 1724.4237060547s, input_fired_time: 18.9745235443, output_fired_time: 9.9087257385, propagate_forward_time: 687.7423706055, propagate_backward_time: 991.3457641602, weight_update_time: 0.9631007314, other_time: 16.4523925781
epoch time: 142.2032623291s, input_fired_time: 1.5927979946, output_fired_time: 0.0000000000, propagate_forward_time: 137.9659118652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6445465088
validation[          , genome  6400] predictions:    6039/  10000 (60.39%), best:    6175/10000 (61.75%), error:     11684.03418, best error:     11320.15820 on epoch:     0, epoch:   50/100, mu: 0.9589092135, learning_rate: 0.0000053774, weight_decay: 0.0000349714

epoch time: 1724.2631835938s, input_fired_time: 18.9153442383, output_fired_time: 9.9014997482, propagate_forward_time: 687.5383911133, propagate_backward_time: 992.2620239258, weight_update_time: 0.9608773589, other_time: 15.6459960938
epoch time: 147.6675872803s, input_fired_time: 1.5823862553, output_fired_time: 0.0000000000, propagate_forward_time: 143.4300384521, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6551666260
validation[          , genome  6400] predictions:    5796/  10000 (57.96%), best:    6175/10000 (61.75%), error:     12080.00879, best error:     11320.15820 on epoch:     0, epoch:   51/100, mu: 0.9605699182, learning_rate: 0.0000050284, weight_decay: 0.0000330673

epoch time: 1736.0186767578s, input_fired_time: 19.0289001465, output_fired_time: 9.9617452621, propagate_forward_time: 692.8899536133, propagate_backward_time: 998.4391479492, weight_update_time: 0.9670248628, other_time: 15.6988525391
epoch time: 143.3828125000s, input_fired_time: 1.5586552620, output_fired_time: 0.0000000000, propagate_forward_time: 139.4495697021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3745880127
validation[          , genome  6400] predictions:    5954/  10000 (59.54%), best:    6175/10000 (61.75%), error:     11946.15625, best error:     11320.15820 on epoch:     0, epoch:   52/100, mu: 0.9621419311, learning_rate: 0.0000047021, weight_decay: 0.0000312669

epoch time: 1723.8381347656s, input_fired_time: 18.8828887939, output_fired_time: 9.8906478882, propagate_forward_time: 688.5827636719, propagate_backward_time: 991.3081665039, weight_update_time: 0.9580304027, other_time: 15.1736450195
epoch time: 141.3175354004s, input_fired_time: 1.5401829481, output_fired_time: 0.0000000000, propagate_forward_time: 137.2951354980, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4822235107
validation[          , genome  6400] predictions:    6045/  10000 (60.45%), best:    6175/10000 (61.75%), error:     11743.13379, best error:     11320.15820 on epoch:     0, epoch:   53/100, mu: 0.9636299610, learning_rate: 0.0000043969, weight_decay: 0.0000295644

epoch time: 1715.9997558594s, input_fired_time: 18.8221931458, output_fired_time: 9.8995170593, propagate_forward_time: 684.2669067383, propagate_backward_time: 988.1420288086, weight_update_time: 0.9542978406, other_time: 14.8691406250
epoch time: 142.7380981445s, input_fired_time: 1.5775378942, output_fired_time: 0.0000000000, propagate_forward_time: 138.6799011230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4806518555
validation[          , genome  6400] predictions:    5982/  10000 (59.82%), best:    6175/10000 (61.75%), error:     11812.68848, best error:     11320.15820 on epoch:     0, epoch:   54/100, mu: 0.9650385380, learning_rate: 0.0000041115, weight_decay: 0.0000279547

epoch time: 1721.5002441406s, input_fired_time: 18.9833736420, output_fired_time: 9.9125576019, propagate_forward_time: 686.3750000000, propagate_backward_time: 989.6953125000, weight_update_time: 0.9817326665, other_time: 16.5339355469
epoch time: 144.0453643799s, input_fired_time: 1.5737251043, output_fired_time: 0.0000000000, propagate_forward_time: 140.0146484375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4569854736
validation[          , genome  6400] predictions:    6028/  10000 (60.28%), best:    6175/10000 (61.75%), error:     11728.60645, best error:     11320.15820 on epoch:     0, epoch:   55/100, mu: 0.9663718343, learning_rate: 0.0000038447, weight_decay: 0.0000264326

epoch time: 1727.2541503906s, input_fired_time: 18.9058322906, output_fired_time: 9.9227361679, propagate_forward_time: 689.3633422852, propagate_backward_time: 993.7020874023, weight_update_time: 0.9582534432, other_time: 15.3601074219
epoch time: 142.3910217285s, input_fired_time: 1.6004090309, output_fired_time: 0.0000000000, propagate_forward_time: 138.2420654297, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5485534668
validation[          , genome  6400] predictions:    6030/  10000 (60.30%), best:    6175/10000 (61.75%), error:     11675.79102, best error:     11320.15820 on epoch:     0, epoch:   56/100, mu: 0.9676339030, learning_rate: 0.0000035952, weight_decay: 0.0000249934

epoch time: 1735.5839843750s, input_fired_time: 19.1421604156, output_fired_time: 9.9849615097, propagate_forward_time: 693.5355834961, propagate_backward_time: 997.1380615234, weight_update_time: 0.9765083194, other_time: 15.7831420898
epoch time: 142.4874114990s, input_fired_time: 1.5429742336, output_fired_time: 0.0000000000, propagate_forward_time: 138.4945373535, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4499053955
validation[          , genome  6400] predictions:    5976/  10000 (59.76%), best:    6175/10000 (61.75%), error:     11753.87891, best error:     11320.15820 on epoch:     0, epoch:   57/100, mu: 0.9688285589, learning_rate: 0.0000033618, weight_decay: 0.0000236326

epoch time: 1720.1287841797s, input_fired_time: 19.0117225647, output_fired_time: 9.9172887802, propagate_forward_time: 685.0812988281, propagate_backward_time: 990.0379638672, weight_update_time: 0.9673066735, other_time: 16.0805664063
epoch time: 144.3364410400s, input_fired_time: 1.5558093786, output_fired_time: 0.0000000000, propagate_forward_time: 140.3956451416, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3849792480
validation[          , genome  6400] predictions:    6006/  10000 (60.06%), best:    6175/10000 (61.75%), error:     11743.90723, best error:     11320.15820 on epoch:     0, epoch:   58/100, mu: 0.9699594378, learning_rate: 0.0000031437, weight_decay: 0.0000223458

epoch time: 1725.4119873047s, input_fired_time: 19.1637592316, output_fired_time: 9.9373102188, propagate_forward_time: 688.8349609375, propagate_backward_time: 991.5900878906, weight_update_time: 0.9705935121, other_time: 15.8858642578
epoch time: 141.6673583984s, input_fired_time: 1.5423309803, output_fired_time: 0.0000000000, propagate_forward_time: 137.6837463379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4412841797
validation[          , genome  6400] predictions:    6112/  10000 (61.12%), best:    6175/10000 (61.75%), error:     11601.70215, best error:     11320.15820 on epoch:     0, epoch:   59/100, mu: 0.9710298777, learning_rate: 0.0000029396, weight_decay: 0.0000211292

epoch time: 1726.8474121094s, input_fired_time: 19.0827732086, output_fired_time: 9.9332408905, propagate_forward_time: 688.4216918945, propagate_backward_time: 992.3786010742, weight_update_time: 0.9698420167, other_time: 17.0311279297
epoch time: 141.5580596924s, input_fired_time: 1.5652714968, output_fired_time: 0.0000000000, propagate_forward_time: 136.2772979736, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7154846191
validation[          , genome  6400] predictions:    6027/  10000 (60.27%), best:    6175/10000 (61.75%), error:     11671.84570, best error:     11320.15820 on epoch:     0, epoch:   60/100, mu: 0.9720431566, learning_rate: 0.0000027489, weight_decay: 0.0000199787

epoch time: 1728.4094238281s, input_fired_time: 19.0544109344, output_fired_time: 9.9377365112, propagate_forward_time: 690.4608154297, propagate_backward_time: 993.0940551758, weight_update_time: 0.9687514901, other_time: 15.8623657227
epoch time: 143.3406829834s, input_fired_time: 1.5300860405, output_fired_time: 0.0000000000, propagate_forward_time: 139.5095062256, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3010864258
validation[          , genome  6400] predictions:    5898/  10000 (58.98%), best:    6175/10000 (61.75%), error:     11960.58203, best error:     11320.15820 on epoch:     0, epoch:   61/100, mu: 0.9730023146, learning_rate: 0.0000025705, weight_decay: 0.0000188909

epoch time: 1727.8240966797s, input_fired_time: 18.9425563812, output_fired_time: 9.9201660156, propagate_forward_time: 690.0319824219, propagate_backward_time: 993.5122070313, weight_update_time: 0.9601217508, other_time: 15.4172363281
epoch time: 142.3214874268s, input_fired_time: 1.5965156555, output_fired_time: 0.0000000000, propagate_forward_time: 137.9204559326, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8045196533
validation[          , genome  6400] predictions:    6059/  10000 (60.59%), best:    6175/10000 (61.75%), error:     11707.76270, best error:     11320.15820 on epoch:     0, epoch:   62/100, mu: 0.9739102125, learning_rate: 0.0000024036, weight_decay: 0.0000178623

epoch time: 1728.4356689453s, input_fired_time: 18.8874816895, output_fired_time: 9.9199628830, propagate_forward_time: 689.3884277344, propagate_backward_time: 994.3732910156, weight_update_time: 0.9585380554, other_time: 15.8665771484
epoch time: 142.0153503418s, input_fired_time: 1.5104002953, output_fired_time: 0.0000000000, propagate_forward_time: 137.4579315186, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0470123291
validation[          , genome  6400] predictions:    6060/  10000 (60.60%), best:    6175/10000 (61.75%), error:     11578.81348, best error:     11320.15820 on epoch:     0, epoch:   63/100, mu: 0.9747696519, learning_rate: 0.0000022476, weight_decay: 0.0000168898

epoch time: 1731.2207031250s, input_fired_time: 19.1097087860, output_fired_time: 9.9387979507, propagate_forward_time: 692.6037597656, propagate_backward_time: 994.9192504883, weight_update_time: 0.9667530060, other_time: 14.6491088867
epoch time: 146.4328918457s, input_fired_time: 1.5762952566, output_fired_time: 0.0000000000, propagate_forward_time: 142.3090209961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5475769043
validation[          , genome  6400] predictions:    6121/  10000 (61.21%), best:    6175/10000 (61.75%), error:     11534.19531, best error:     11320.15820 on epoch:     0, epoch:   64/100, mu: 0.9755831957, learning_rate: 0.0000021018, weight_decay: 0.0000159702

epoch time: 1722.1098632813s, input_fired_time: 18.8889827728, output_fired_time: 9.8847837448, propagate_forward_time: 688.1863403320, propagate_backward_time: 990.0554199219, weight_update_time: 0.9531176686, other_time: 15.0942993164
epoch time: 145.6081237793s, input_fired_time: 1.5855122805, output_fired_time: 0.0000000000, propagate_forward_time: 141.5081176758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5144958496
validation[          , genome  6400] predictions:    6066/  10000 (60.66%), best:    6175/10000 (61.75%), error:     11594.32813, best error:     11320.15820 on epoch:     0, epoch:   65/100, mu: 0.9763532877, learning_rate: 0.0000019654, weight_decay: 0.0000151006

epoch time: 1727.5128173828s, input_fired_time: 18.8842964172, output_fired_time: 9.9146499634, propagate_forward_time: 690.6480102539, propagate_backward_time: 993.3931884766, weight_update_time: 0.9540874958, other_time: 14.6726684570
epoch time: 142.7706756592s, input_fired_time: 1.6067183018, output_fired_time: 0.0000000000, propagate_forward_time: 138.7327575684, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4311981201
validation[          , genome  6400] predictions:    6126/  10000 (61.26%), best:    6175/10000 (61.75%), error:     11489.91602, best error:     11320.15820 on epoch:     0, epoch:   66/100, mu: 0.9770822525, learning_rate: 0.0000018378, weight_decay: 0.0000142784

epoch time: 1726.1201171875s, input_fired_time: 19.0928287506, output_fired_time: 9.9337253571, propagate_forward_time: 689.8895874023, propagate_backward_time: 992.2661132813, weight_update_time: 0.9661780596, other_time: 14.9379272461
epoch time: 143.1165008545s, input_fired_time: 1.5647182465, output_fired_time: 0.0000000000, propagate_forward_time: 139.2020263672, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3497619629
validation[          , genome  6400] predictions:    6077/  10000 (60.77%), best:    6175/10000 (61.75%), error:     11531.47070, best error:     11320.15820 on epoch:     0, epoch:   67/100, mu: 0.9777722359, learning_rate: 0.0000017185, weight_decay: 0.0000135010

epoch time: 1720.7960205078s, input_fired_time: 18.9765815735, output_fired_time: 9.9122552872, propagate_forward_time: 686.0272827148, propagate_backward_time: 989.7571411133, weight_update_time: 0.9668998718, other_time: 16.1228027344
epoch time: 141.2629547119s, input_fired_time: 1.5733892918, output_fired_time: 0.0000000000, propagate_forward_time: 136.5479278564, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1416320801
validation[          , genome  6400] predictions:    6017/  10000 (60.17%), best:    6175/10000 (61.75%), error:     11627.14355, best error:     11320.15820 on epoch:     0, epoch:   68/100, mu: 0.9784253836, learning_rate: 0.0000016070, weight_decay: 0.0000127659

epoch time: 1720.0865478516s, input_fired_time: 19.0073375702, output_fired_time: 9.9036121368, propagate_forward_time: 685.6657104492, propagate_backward_time: 989.2183837891, weight_update_time: 0.9655811191, other_time: 16.2915649414
epoch time: 141.1946716309s, input_fired_time: 1.5284471512, output_fired_time: 0.0000000000, propagate_forward_time: 137.1413574219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5248718262
validation[          , genome  6400] predictions:    6104/  10000 (61.04%), best:    6175/10000 (61.75%), error:     11491.51660, best error:     11320.15820 on epoch:     0, epoch:   69/100, mu: 0.9790436625, learning_rate: 0.0000015027, weight_decay: 0.0000120708

epoch time: 1715.2236328125s, input_fired_time: 18.9449729919, output_fired_time: 9.8773994446, propagate_forward_time: 682.7114257813, propagate_backward_time: 987.1241455078, weight_update_time: 0.9608506560, other_time: 16.5656738281
epoch time: 142.6797637939s, input_fired_time: 1.5397416353, output_fired_time: 0.0000000000, propagate_forward_time: 138.6368103027, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5032043457
validation[          , genome  6400] predictions:    6010/  10000 (60.10%), best:    6175/10000 (61.75%), error:     11779.83008, best error:     11320.15820 on epoch:     0, epoch:   70/100, mu: 0.9796289206, learning_rate: 0.0000014052, weight_decay: 0.0000114136

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_186_6400.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542692703_186_6400_2_r217679239_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.506223
read mu: 0.980183
read mu_delta: 0.946585
read initial_learning_rate: 0.000154051
read learning_rate: 1.31399e-06
read learning_rate_delta: 0.9351
read initial_weight_decay: 0.000574702
read weight_decay: 1.07921e-05
read weight_decay_delta: 0.945552
read batch_size: 89
read epsilon: 1e-07
read alpha: 0.0873235
read input_dropout_probability: 0.00118544
read hidden_dropout_probability: 0.10684
read velocity_reset: 1795
read epoch: 71
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 6175
read best_validation_error: 11320.2
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6400
read normal distribution: '1 0 0'
generator_str: '164429972'
read generator: 164429972
read generated_by_map:
1 crossover 1
reading 92 nodes.
reading 348 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1769.95080566406s, input_fired_time: 19.3350811004639, output_fired_time: 10.1882781982422, propagate_forward_time: 713.113098144531, propagate_backward_time: 1012.28399658203, weight_update_time: 0.977355599403381, other_time: 15.0304565429688
epoch time: 145.281234741211s, input_fired_time: 1.62682402133942, output_fired_time: 0, propagate_forward_time: 141.180999755859, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.47340393066406
validation[          , genome  6400] predictions:    6117/  10000 (61.17%), best:    6175/10000 (61.75%), error:     11466.25195, best error:     11320.15820 on epoch:     0, epoch:   71/100, mu: 0.9801828861, learning_rate: 0.0000013140, weight_decay: 0.0000107921

epoch time: 1758.7587890625s, input_fired_time: 19.1405563354, output_fired_time: 10.0816583633, propagate_forward_time: 706.7285766602, propagate_backward_time: 1008.1011352539, weight_update_time: 0.9675068855, other_time: 14.7069091797
epoch time: 145.5693054199s, input_fired_time: 1.5929470062, output_fired_time: 0.0000000000, propagate_forward_time: 141.5162811279, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4600830078
validation[          , genome  6400] predictions:    6105/  10000 (61.05%), best:    6175/10000 (61.75%), error:     11530.81934, best error:     11320.15820 on epoch:     0, epoch:   72/100, mu: 0.9807072878, learning_rate: 0.0000012287, weight_decay: 0.0000102045

epoch time: 1729.6563720703s, input_fired_time: 18.8837738037, output_fired_time: 9.9329671860, propagate_forward_time: 690.2957763672, propagate_backward_time: 994.9138793945, weight_update_time: 0.9599764347, other_time: 15.6299438477
epoch time: 141.7604522705s, input_fired_time: 1.5560901165, output_fired_time: 0.0000000000, propagate_forward_time: 136.9253234863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2790374756
validation[          , genome  6400] predictions:    6111/  10000 (61.11%), best:    6175/10000 (61.75%), error:     11568.93262, best error:     11320.15820 on epoch:     0, epoch:   73/100, mu: 0.9812036753, learning_rate: 0.0000011490, weight_decay: 0.0000096489

epoch time: 1744.5133056641s, input_fired_time: 19.0927028656, output_fired_time: 10.0151767731, propagate_forward_time: 698.6961059570, propagate_backward_time: 1001.2607421875, weight_update_time: 0.9715272784, other_time: 15.4486694336
epoch time: 144.2882080078s, input_fired_time: 1.5686043501, output_fired_time: 0.0000000000, propagate_forward_time: 140.3449707031, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3746337891
validation[          , genome  6400] predictions:    6021/  10000 (60.21%), best:    6175/10000 (61.75%), error:     11687.44727, best error:     11320.15820 on epoch:     0, epoch:   74/100, mu: 0.9816735387, learning_rate: 0.0000010744, weight_decay: 0.0000091235

epoch time: 1729.1507568359s, input_fired_time: 18.9791870117, output_fired_time: 9.9255142212, propagate_forward_time: 691.0496826172, propagate_backward_time: 994.2880249023, weight_update_time: 0.9610586166, other_time: 14.9083862305
epoch time: 145.4368743896s, input_fired_time: 1.5392206907, output_fired_time: 0.0000000000, propagate_forward_time: 141.5713653564, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3262939453
validation[          , genome  6400] predictions:    6108/  10000 (61.08%), best:    6175/10000 (61.75%), error:     11441.79004, best error:     11320.15820 on epoch:     0, epoch:   75/100, mu: 0.9821183085, learning_rate: 0.0000010047, weight_decay: 0.0000086268

epoch time: 1742.6241455078s, input_fired_time: 19.2716484070, output_fired_time: 9.9666604996, propagate_forward_time: 697.2133789063, propagate_backward_time: 1000.7942504883, weight_update_time: 0.9597332478, other_time: 15.3782348633
epoch time: 141.9927368164s, input_fired_time: 1.6011548042, output_fired_time: 0.0000000000, propagate_forward_time: 137.9650878906, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4264984131
validation[          , genome  6400] predictions:    6069/  10000 (60.69%), best:    6175/10000 (61.75%), error:     11582.28125, best error:     11320.15820 on epoch:     0, epoch:   76/100, mu: 0.9825392962, learning_rate: 0.0000009395, weight_decay: 0.0000081571

epoch time: 1735.4323730469s, input_fired_time: 19.0381069183, output_fired_time: 9.9694309235, propagate_forward_time: 693.2726440430, propagate_backward_time: 997.0269775391, weight_update_time: 0.9705446959, other_time: 16.1251831055
epoch time: 144.9820404053s, input_fired_time: 1.5829292536, output_fired_time: 0.0000000000, propagate_forward_time: 140.9880065918, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4111022949
validation[          , genome  6400] predictions:    6161/  10000 (61.61%), best:    6175/10000 (61.75%), error:     11506.87402, best error:     11320.15820 on epoch:     0, epoch:   77/100, mu: 0.9829378128, learning_rate: 0.0000008785, weight_decay: 0.0000077129

epoch time: 1738.7150878906s, input_fired_time: 19.1409015656, output_fired_time: 9.9987802505, propagate_forward_time: 694.0418701172, propagate_backward_time: 999.4066162109, weight_update_time: 0.9733665586, other_time: 16.1269531250
epoch time: 143.9162750244s, input_fired_time: 1.5564669371, output_fired_time: 0.0000000000, propagate_forward_time: 139.7231750488, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6366271973
validation[          , genome  6400] predictions:    6082/  10000 (60.82%), best:    6175/10000 (61.75%), error:     11609.87988, best error:     11320.15820 on epoch:     0, epoch:   78/100, mu: 0.9833150506, learning_rate: 0.0000008215, weight_decay: 0.0000072930

epoch time: 1702.2130126953s, input_fired_time: 19.0336742401, output_fired_time: 9.8535814285, propagate_forward_time: 675.0180664063, propagate_backward_time: 979.4264526367, weight_update_time: 0.9734236598, other_time: 18.8811645508
epoch time: 145.5755157471s, input_fired_time: 1.5447567701, output_fired_time: 0.0000000000, propagate_forward_time: 141.5416412354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4891204834
validation[          , genome  6400] predictions:    6169/  10000 (61.69%), best:    6175/10000 (61.75%), error:     11421.54980, best error:     11320.15820 on epoch:     0, epoch:   79/100, mu: 0.9836721420, learning_rate: 0.0000007682, weight_decay: 0.0000068959

epoch time: 1723.3186035156s, input_fired_time: 18.9900856018, output_fired_time: 9.9264860153, propagate_forward_time: 685.5076293945, propagate_backward_time: 992.2800292969, weight_update_time: 0.9644637108, other_time: 16.6143188477
epoch time: 145.0788269043s, input_fired_time: 1.5502854586, output_fired_time: 0.0000000000, propagate_forward_time: 141.1322021484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3963317871
validation[          , genome  6400] predictions:    6147/  10000 (61.47%), best:    6175/10000 (61.75%), error:     11453.57617, best error:     11320.15820 on epoch:     0, epoch:   80/100, mu: 0.9840101600, learning_rate: 0.0000007183, weight_decay: 0.0000065204

epoch time: 1727.8937988281s, input_fired_time: 19.1633071899, output_fired_time: 9.9886064529, propagate_forward_time: 689.5712280273, propagate_backward_time: 992.3206176758, weight_update_time: 0.9780904055, other_time: 16.8499755859
epoch time: 142.4012298584s, input_fired_time: 1.5292185545, output_fired_time: 0.0000000000, propagate_forward_time: 138.1209259033, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7510833740
validation[          , genome  6400] predictions:    6157/  10000 (61.57%), best:    6175/10000 (61.75%), error:     11412.41699, best error:     11320.15820 on epoch:     0, epoch:   81/100, mu: 0.9843301177, learning_rate: 0.0000006717, weight_decay: 0.0000061654

epoch time: 1704.1279296875s, input_fired_time: 19.0461254120, output_fired_time: 9.8756246567, propagate_forward_time: 676.7200927734, propagate_backward_time: 980.2760620117, weight_update_time: 0.9706542492, other_time: 18.2100219727
epoch time: 140.6030578613s, input_fired_time: 1.5170216560, output_fired_time: 0.0000000000, propagate_forward_time: 136.1564636230, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9295654297
validation[          , genome  6400] predictions:    6147/  10000 (61.47%), best:    6175/10000 (61.75%), error:     11431.56348, best error:     11320.15820 on epoch:     0, epoch:   82/100, mu: 0.9846329689, learning_rate: 0.0000006281, weight_decay: 0.0000058297

epoch time: 1725.3564453125s, input_fired_time: 19.1032333374, output_fired_time: 9.9503078461, propagate_forward_time: 686.3586425781, propagate_backward_time: 992.8804321289, weight_update_time: 0.9758940935, other_time: 17.0637817383
epoch time: 144.2776489258s, input_fired_time: 1.5838729143, output_fired_time: 0.0000000000, propagate_forward_time: 139.9110717773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7826995850
validation[          , genome  6400] predictions:    6147/  10000 (61.47%), best:    6175/10000 (61.75%), error:     11421.54199, best error:     11320.15820 on epoch:     0, epoch:   83/100, mu: 0.9849196672, learning_rate: 0.0000005873, weight_decay: 0.0000055123

epoch time: 1728.5377197266s, input_fired_time: 19.0005912781, output_fired_time: 9.9500293732, propagate_forward_time: 688.3801879883, propagate_backward_time: 994.5527343750, weight_update_time: 0.9676841497, other_time: 16.6541137695
epoch time: 143.7536621094s, input_fired_time: 1.5657776594, output_fired_time: 0.0000000000, propagate_forward_time: 139.8689117432, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3189697266
validation[          , genome  6400] predictions:    6196/  10000 (61.96%), best:    6175/10000 (61.75%), error:     11361.73438, best error:     11320.15820 on epoch:     0, epoch:   84/100, mu: 0.9851910472, learning_rate: 0.0000005492, weight_decay: 0.0000052122

epoch time: 1745.5600585938s, input_fired_time: 19.2092666626, output_fired_time: 10.0216417313, propagate_forward_time: 699.5518798828, propagate_backward_time: 1001.5706787109, weight_update_time: 0.9676982164, other_time: 15.2066650391
epoch time: 145.0063476563s, input_fired_time: 1.5643098354, output_fired_time: 0.0000000000, propagate_forward_time: 140.9227600098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5192718506
validation[          , genome  6400] predictions:    6211/  10000 (62.11%), best:    6175/10000 (61.75%), error:     11356.69922, best error:     11320.15820 on epoch:     0, epoch:   85/100, mu: 0.9854479432, learning_rate: 0.0000005136, weight_decay: 0.0000049284

epoch time: 1762.8234863281s, input_fired_time: 19.2785358429, output_fired_time: 10.1313009262, propagate_forward_time: 708.2821044922, propagate_backward_time: 1009.9977416992, weight_update_time: 0.9786927700, other_time: 15.1337280273
epoch time: 143.9370574951s, input_fired_time: 1.5295228958, output_fired_time: 0.0000000000, propagate_forward_time: 139.7486724854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6588592529
validation[          , genome  6400] predictions:    6164/  10000 (61.64%), best:    6175/10000 (61.75%), error:     11381.06250, best error:     11320.15820 on epoch:     0, epoch:   86/100, mu: 0.9856910706, learning_rate: 0.0000004802, weight_decay: 0.0000046600

epoch time: 1759.0793457031s, input_fired_time: 19.1106433868, output_fired_time: 10.1056089401, propagate_forward_time: 706.6393432617, propagate_backward_time: 1008.2575683594, weight_update_time: 0.9709675312, other_time: 14.9662475586
epoch time: 144.8958892822s, input_fired_time: 1.6122217178, output_fired_time: 0.0000000000, propagate_forward_time: 140.8244018555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4592590332
validation[          , genome  6400] predictions:    6199/  10000 (61.99%), best:    6175/10000 (61.75%), error:     11363.93945, best error:     11320.15820 on epoch:     0, epoch:   87/100, mu: 0.9859212041, learning_rate: 0.0000004491, weight_decay: 0.0000044063

epoch time: 1725.6490478516s, input_fired_time: 19.1366806030, output_fired_time: 9.9621047974, propagate_forward_time: 689.2466430664, propagate_backward_time: 991.5344238281, weight_update_time: 0.9684168696, other_time: 15.7691040039
epoch time: 143.7580108643s, input_fired_time: 1.5740259886, output_fired_time: 0.0000000000, propagate_forward_time: 139.7963104248, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3876800537
validation[          , genome  6400] predictions:    6223/  10000 (62.23%), best:    6175/10000 (61.75%), error:     11343.68066, best error:     11320.15820 on epoch:     0, epoch:   88/100, mu: 0.9861390591, learning_rate: 0.0000004199, weight_decay: 0.0000041664

epoch time: 1747.2171630859s, input_fired_time: 19.1207866669, output_fired_time: 10.0576925278, propagate_forward_time: 699.0045166016, propagate_backward_time: 1002.5571899414, weight_update_time: 0.9781975150, other_time: 16.4769897461
epoch time: 146.2580871582s, input_fired_time: 1.5476260185, output_fired_time: 0.0000000000, propagate_forward_time: 142.3673400879, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3431243896
validation[          , genome  6400] predictions:    6172/  10000 (61.72%), best:    6175/10000 (61.75%), error:     11337.48730, best error:     11320.15820 on epoch:     0, epoch:   89/100, mu: 0.9863452911, learning_rate: 0.0000003927, weight_decay: 0.0000039395

epoch time: 1742.2050781250s, input_fired_time: 18.9361076355, output_fired_time: 9.9940719604, propagate_forward_time: 698.9992065430, propagate_backward_time: 999.9385375977, weight_update_time: 0.9577394724, other_time: 14.3371582031
epoch time: 143.6830139160s, input_fired_time: 1.5367763042, output_fired_time: 0.0000000000, propagate_forward_time: 139.4749755859, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6712646484
validation[          , genome  6400] predictions:    6158/  10000 (61.58%), best:    6175/10000 (61.75%), error:     11399.05664, best error:     11320.15820 on epoch:     0, epoch:   90/100, mu: 0.9865404963, learning_rate: 0.0000003672, weight_decay: 0.0000037250

epoch time: 1753.3519287109s, input_fired_time: 19.1329822540, output_fired_time: 10.0723533630, propagate_forward_time: 703.2974243164, propagate_backward_time: 1005.4419555664, weight_update_time: 0.9768479466, other_time: 15.4072265625
epoch time: 140.5907592773s, input_fired_time: 1.5247958899, output_fired_time: 0.0000000000, propagate_forward_time: 135.9365539551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1294097900
validation[          , genome  6400] predictions:    6127/  10000 (61.27%), best:    6175/10000 (61.75%), error:     11431.55762, best error:     11320.15820 on epoch:     0, epoch:   91/100, mu: 0.9867252707, learning_rate: 0.0000003434, weight_decay: 0.0000035222

epoch time: 1783.6511230469s, input_fired_time: 19.4141635895, output_fired_time: 10.2147626877, propagate_forward_time: 719.3442993164, propagate_backward_time: 1019.5138549805, weight_update_time: 0.9874756336, other_time: 15.1641235352
epoch time: 146.1532592773s, input_fired_time: 1.6083779335, output_fired_time: 0.0000000000, propagate_forward_time: 141.8022613525, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7426147461
validation[          , genome  6400] predictions:    6187/  10000 (61.87%), best:    6175/10000 (61.75%), error:     11396.16309, best error:     11320.15820 on epoch:     0, epoch:   92/100, mu: 0.9869002104, learning_rate: 0.0000003211, weight_decay: 0.0000033304

epoch time: 1746.3461914063s, input_fired_time: 19.1425018311, output_fired_time: 10.0405235291, propagate_forward_time: 699.3238525391, propagate_backward_time: 1002.3868408203, weight_update_time: 0.9735943675, other_time: 15.4525146484
epoch time: 144.6002807617s, input_fired_time: 1.5771274567, output_fired_time: 0.0000000000, propagate_forward_time: 140.5327911377, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4903564453
validation[          , genome  6400] predictions:    6116/  10000 (61.16%), best:    6175/10000 (61.75%), error:     11438.33691, best error:     11320.15820 on epoch:     0, epoch:   93/100, mu: 0.9870657921, learning_rate: 0.0000003002, weight_decay: 0.0000031491

epoch time: 1748.1374511719s, input_fired_time: 19.2494411469, output_fired_time: 10.0417728424, propagate_forward_time: 700.5573730469, propagate_backward_time: 1002.0668334961, weight_update_time: 0.9795898199, other_time: 16.2221069336
epoch time: 144.3151855469s, input_fired_time: 1.5828562975, output_fired_time: 0.0000000000, propagate_forward_time: 139.9893493652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7429809570
validation[          , genome  6400] predictions:    6139/  10000 (61.39%), best:    6175/10000 (61.75%), error:     11376.84668, best error:     11320.15820 on epoch:     0, epoch:   94/100, mu: 0.9872224927, learning_rate: 0.0000002808, weight_decay: 0.0000029776

epoch time: 1751.6193847656s, input_fired_time: 19.2617969513, output_fired_time: 10.0702257156, propagate_forward_time: 699.9130859375, propagate_backward_time: 1005.8326416016, weight_update_time: 0.9820356369, other_time: 16.5416259766
epoch time: 153.0563049316s, input_fired_time: 1.5660313368, output_fired_time: 0.0000000000, propagate_forward_time: 149.1160125732, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3742675781
validation[          , genome  6400] predictions:    6200/  10000 (62.00%), best:    6175/10000 (61.75%), error:     11342.02832, best error:     11320.15820 on epoch:     0, epoch:   95/100, mu: 0.9873708487, learning_rate: 0.0000002625, weight_decay: 0.0000028155

epoch time: 1740.8676757813s, input_fired_time: 19.0466880798, output_fired_time: 9.9924697876, propagate_forward_time: 696.4327392578, propagate_backward_time: 999.8746337891, weight_update_time: 0.9670752883, other_time: 15.5212402344
epoch time: 145.8925628662s, input_fired_time: 1.5595712662, output_fired_time: 0.0000000000, propagate_forward_time: 141.7477722168, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5852203369
validation[          , genome  6400] predictions:    6173/  10000 (61.73%), best:    6175/10000 (61.75%), error:     11400.99219, best error:     11320.15820 on epoch:     0, epoch:   96/100, mu: 0.9875112772, learning_rate: 0.0000002455, weight_decay: 0.0000026622

epoch time: 1746.7283935547s, input_fired_time: 19.1361522675, output_fired_time: 10.0417518616, propagate_forward_time: 699.1900634766, propagate_backward_time: 1002.4467163086, weight_update_time: 0.9715102315, other_time: 15.9137573242
epoch time: 145.1206817627s, input_fired_time: 1.6117441654, output_fired_time: 0.0000000000, propagate_forward_time: 140.6478729248, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8610687256
validation[          , genome  6400] predictions:    6174/  10000 (61.74%), best:    6175/10000 (61.75%), error:     11352.16699, best error:     11320.15820 on epoch:     0, epoch:   97/100, mu: 0.9876441956, learning_rate: 0.0000002296, weight_decay: 0.0000025173

epoch time: 1718.7655029297s, input_fired_time: 19.0913314819, output_fired_time: 9.9470872879, propagate_forward_time: 684.6563720703, propagate_backward_time: 988.2877807617, weight_update_time: 0.9738458991, other_time: 16.7828979492
epoch time: 137.9567413330s, input_fired_time: 1.5003079176, output_fired_time: 0.0000000000, propagate_forward_time: 133.5496978760, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9067382813
validation[          , genome  6400] predictions:    6184/  10000 (61.84%), best:    6175/10000 (61.75%), error:     11375.86426, best error:     11320.15820 on epoch:     0, epoch:   98/100, mu: 0.9877700210, learning_rate: 0.0000002147, weight_decay: 0.0000023802

epoch time: 1700.7940673828s, input_fired_time: 18.9905643463, output_fired_time: 9.8736877441, propagate_forward_time: 676.2652587891, propagate_backward_time: 978.6920166016, weight_update_time: 0.9633218646, other_time: 16.9725341797
epoch time: 143.8545074463s, input_fired_time: 1.6234196424, output_fired_time: 0.0000000000, propagate_forward_time: 139.6766967773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5543975830
validation[          , genome  6400] predictions:    6212/  10000 (62.12%), best:    6175/10000 (61.75%), error:     11336.13965, best error:     11320.15820 on epoch:     0, epoch:   99/100, mu: 0.9878891110, learning_rate: 0.0000002007, weight_decay: 0.0000022506

epoch time: 1725.0781250000s, input_fired_time: 19.1048736572, output_fired_time: 9.9626064301, propagate_forward_time: 687.8816528320, propagate_backward_time: 991.0290527344, weight_update_time: 0.9738655686, other_time: 17.0999145508
epoch time: 145.5834045410s, input_fired_time: 1.5908604860, output_fired_time: 0.0000000000, propagate_forward_time: 141.2078704834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7846679688
validation[          , genome  6400] predictions:    6171/  10000 (61.71%), best:    6175/10000 (61.75%), error:     11357.53516, best error:     11320.15820 on epoch:     0, epoch:  100/100, mu: 0.9880018830, learning_rate: 0.0000001877, weight_decay: 0.0000021281

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 678.4500732422s, input_fired_time: 7.6910471916, output_fired_time: 0.0000000000, propagate_forward_time: 655.3050537109, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 15.4539794922
best training[          , genome  6400] predictions:   33582/  50000 (67.16%), best:    6175/10000 (61.75%), error:     50527.87891, best error:     11320.15820 on epoch:     0, epoch:  101/100, mu: 0.9881086349, learning_rate: 0.0000001755, weight_decay: 0.0000020122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 139.2187652588s, input_fired_time: 1.5548480749, output_fired_time: 0.0000000000, propagate_forward_time: 134.9007873535, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7631225586
   testing[          , genome  6400] predictions:    6175/  10000 (61.75%), best:    6175/10000 (61.75%), error:     11320.15820, best error:     11320.15820 on epoch:     0, epoch:  101/100, mu: 0.9881086349, learning_rate: 0.0000001755, weight_decay: 0.0000020122
backpropagation finished successfully!
02:53:12 (6408): called boinc_finish(0)

</stderr_txt>
]]>