Nameexact_genome_1542649503_187_6349_2
Workunit4099733
Created20 Nov 2018, 3:55:52 UTC
Sent20 Nov 2018, 4:18:10 UTC
Report deadline4 Feb 2019, 11:35:10 UTC
Received22 Nov 2018, 0:18:29 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID60920
Run time1 days 8 hours 0 min 45 sec
CPU time1 days 7 hours 48 min 8 sec
Validate stateValid
Credit19,075.87
Device peak FLOPS3.88 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size538.02 MB
Peak swap size536.29 MB
Peak disk usage10.27 MB

Stderr output

<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>
_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.508671
read mu: 0.918615
read mu_delta: 0.945415
read initial_learning_rate: 0.00219902
read learning_rate: 0.000173711
read learning_rate_delta: 0.928061
read initial_weight_decay: 0.000598364
read weight_decay: 9.50863e-05
read weight_decay_delta: 0.947337
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.085487
read input_dropout_probability: 0.00124867
read hidden_dropout_probability: 0.0993689
read velocity_reset: 1745
read epoch: 34
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 33
read number_validation_images: 10000
read best_validation_predictions: 6323
read best_validation_error: 10607.5
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6349
read normal distribution: '1 0 0'
generator_str: '1338106389'
read generator: 1338106389
read generated_by_map:
1 enable_node 1
reading 76 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1072.58459472656s, input_fired_time: 14.4780683517456, output_fired_time: 7.14745426177979, propagate_forward_time: 413.895538330078, propagate_backward_time: 626.369506835938, weight_update_time: 0.399456202983856, other_time: 10.694091796875
epoch time: 85.7186737060547s, input_fired_time: 1.11163520812988, output_fired_time: 0, propagate_forward_time: 82.8278427124023, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.77919769287109
validation[          , genome  6349] predictions:    6268/  10000 (62.68%), best:    6323/10000 (63.23%), error:     10849.56836, best error:     10607.48535 on epoch:    33, epoch:   34/100, mu: 0.9186148643, learning_rate: 0.0001737106, weight_decay: 0.0000950863

epoch time: 1071.7049560547s, input_fired_time: 14.5008668900, output_fired_time: 7.1138648987, propagate_forward_time: 413.5592956543, propagate_backward_time: 625.8604125977, weight_update_time: 0.3990291059, other_time: 10.6704711914
epoch time: 85.3362350464s, input_fired_time: 1.1102883816, output_fired_time: 0.0000000000, propagate_forward_time: 82.4468688965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7790756226
validation[          , genome  6349] predictions:    6234/  10000 (62.34%), best:    6323/10000 (63.23%), error:     10800.24121, best error:     10607.48535 on epoch:    33, epoch:   35/100, mu: 0.9225113988, learning_rate: 0.0001612140, weight_decay: 0.0000900788

epoch time: 1070.3068847656s, input_fired_time: 14.4965639114, output_fired_time: 7.0982151031, propagate_forward_time: 412.1591796875, propagate_backward_time: 625.8781127930, weight_update_time: 0.3993647397, other_time: 10.6747436523
epoch time: 85.4151306152s, input_fired_time: 1.1028050184, output_fired_time: 0.0000000000, propagate_forward_time: 82.5097656250, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8025588989
validation[          , genome  6349] predictions:    6277/  10000 (62.77%), best:    6323/10000 (63.23%), error:     10692.61230, best error:     10607.48535 on epoch:    33, epoch:   36/100, mu: 0.9261952639, learning_rate: 0.0001496164, weight_decay: 0.0000853350

epoch time: 1070.5694580078s, input_fired_time: 14.4873552322, output_fired_time: 7.1177177429, propagate_forward_time: 412.5584716797, propagate_backward_time: 625.7202148438, weight_update_time: 0.3990136981, other_time: 10.6857910156
epoch time: 85.5181427002s, input_fired_time: 1.1199976206, output_fired_time: 0.0000000000, propagate_forward_time: 82.6137542725, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7843933105
validation[          , genome  6349] predictions:    6275/  10000 (62.75%), best:    6323/10000 (63.23%), error:     10690.06836, best error:     10607.48535 on epoch:    33, epoch:   37/100, mu: 0.9296780229, learning_rate: 0.0001388531, weight_decay: 0.0000808410

epoch time: 1060.4160156250s, input_fired_time: 14.4768762589, output_fired_time: 7.0516767502, propagate_forward_time: 408.0385131836, propagate_backward_time: 620.0999755859, weight_update_time: 0.4002146721, other_time: 10.7489624023
epoch time: 84.1747207642s, input_fired_time: 1.1031115055, output_fired_time: 0.0000000000, propagate_forward_time: 81.2627487183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8088607788
validation[          , genome  6349] predictions:    6345/  10000 (63.45%), best:    6345/10000 (63.45%), error:     10498.07129, best error:     10498.07129 on epoch:    38, epoch:   38/100, mu: 0.9329707026, learning_rate: 0.0001288641, weight_decay: 0.0000765837

epoch time: 1048.2205810547s, input_fired_time: 14.4128665924, output_fired_time: 7.0109977722, propagate_forward_time: 403.6213378906, propagate_backward_time: 612.3416137695, weight_update_time: 0.4001949430, other_time: 10.8338012695
epoch time: 83.8149642944s, input_fired_time: 1.1059142351, output_fired_time: 0.0000000000, propagate_forward_time: 80.9059524536, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8031005859
validation[          , genome  6349] predictions:    6334/  10000 (63.34%), best:    6345/10000 (63.45%), error:     10528.50684, best error:     10498.07129 on epoch:    38, epoch:   39/100, mu: 0.9360836148, learning_rate: 0.0001195937, weight_decay: 0.0000725505

epoch time: 1045.5093994141s, input_fired_time: 14.4139966965, output_fired_time: 6.9849948883, propagate_forward_time: 402.2813415527, propagate_backward_time: 610.9648437500, weight_update_time: 0.4021155834, other_time: 10.8642578125
epoch time: 83.9420852661s, input_fired_time: 1.1092385054, output_fired_time: 0.0000000000, propagate_forward_time: 81.0269546509, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8058929443
validation[          , genome  6349] predictions:    6319/  10000 (63.19%), best:    6345/10000 (63.45%), error:     10514.24609, best error:     10498.07129 on epoch:    38, epoch:   40/100, mu: 0.9390266538, learning_rate: 0.0001109902, weight_decay: 0.0000687298

epoch time: 1046.3409423828s, input_fired_time: 14.4583787918, output_fired_time: 7.0183801651, propagate_forward_time: 403.4017944336, propagate_backward_time: 610.5961914063, weight_update_time: 0.4011621773, other_time: 10.8661499023
epoch time: 84.0426177979s, input_fired_time: 1.1034924984, output_fired_time: 0.0000000000, propagate_forward_time: 81.1412582397, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7978668213
validation[          , genome  6349] predictions:    6251/  10000 (62.51%), best:    6345/10000 (63.45%), error:     10678.57129, best error:     10498.07129 on epoch:    38, epoch:   41/100, mu: 0.9418090582, learning_rate: 0.0001030057, weight_decay: 0.0000651103

epoch time: 1058.6655273438s, input_fired_time: 14.5464992523, output_fired_time: 7.0895032883, propagate_forward_time: 408.1209411621, propagate_backward_time: 618.0690917969, weight_update_time: 0.4054013193, other_time: 10.8394775391
epoch time: 83.6769790649s, input_fired_time: 1.1036864519, output_fired_time: 0.0000000000, propagate_forward_time: 80.7732772827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8000183105
validation[          , genome  6349] predictions:    6372/  10000 (63.72%), best:    6372/10000 (63.72%), error:     10497.53418, best error:     10497.53418 on epoch:    42, epoch:   42/100, mu: 0.9444395304, learning_rate: 0.0000955955, weight_decay: 0.0000616814

epoch time: 1054.3648681641s, input_fired_time: 14.5002365112, output_fired_time: 7.0598564148, propagate_forward_time: 406.2065124512, propagate_backward_time: 615.8314819336, weight_update_time: 0.4056693614, other_time: 10.7667846680
epoch time: 84.3501052856s, input_fired_time: 1.1106638908, output_fired_time: 0.0000000000, propagate_forward_time: 81.4434661865, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7959747314
validation[          , genome  6349] predictions:    6388/  10000 (63.88%), best:    6388/10000 (63.88%), error:     10391.86328, best error:     10391.86328 on epoch:    43, epoch:   43/100, mu: 0.9469264746, learning_rate: 0.0000887184, weight_decay: 0.0000584330

epoch time: 1057.4172363281s, input_fired_time: 14.5159263611, output_fired_time: 7.0713677406, propagate_forward_time: 407.2045288086, propagate_backward_time: 617.8829345703, weight_update_time: 0.4077276886, other_time: 10.7424926758
epoch time: 84.2422866821s, input_fired_time: 1.1034294367, output_fired_time: 0.0000000000, propagate_forward_time: 81.3320083618, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8068466187
validation[          , genome  6349] predictions:    6303/  10000 (63.03%), best:    6388/10000 (63.88%), error:     10611.95313, best error:     10391.86328 on epoch:    43, epoch:   44/100, mu: 0.9492776394, learning_rate: 0.0000823361, weight_decay: 0.0000553558

epoch time: 1056.1735839844s, input_fired_time: 14.5002212524, output_fired_time: 7.0254092216, propagate_forward_time: 406.8895263672, propagate_backward_time: 617.0086669922, weight_update_time: 0.4061852098, other_time: 10.7497558594
epoch time: 84.3919296265s, input_fired_time: 1.1054240465, output_fired_time: 0.0000000000, propagate_forward_time: 81.4773483276, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8091583252
validation[          , genome  6349] predictions:    6413/  10000 (64.13%), best:    6388/10000 (63.88%), error:     10443.53027, best error:     10391.86328 on epoch:    43, epoch:   45/100, mu: 0.9515004754, learning_rate: 0.0000764129, weight_decay: 0.0000524406

epoch time: 1055.5010986328s, input_fired_time: 14.5011920929, output_fired_time: 7.0773363113, propagate_forward_time: 406.5649414063, propagate_backward_time: 616.5456542969, weight_update_time: 0.4134299159, other_time: 10.8118896484
epoch time: 84.6061401367s, input_fired_time: 1.1180530787, output_fired_time: 0.0000000000, propagate_forward_time: 81.6893539429, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7987365723
validation[          , genome  6349] predictions:    6376/  10000 (63.76%), best:    6388/10000 (63.88%), error:     10426.44336, best error:     10391.86328 on epoch:    43, epoch:   46/100, mu: 0.9536019564, learning_rate: 0.0000709158, weight_decay: 0.0000496789

epoch time: 1052.8570556641s, input_fired_time: 14.5052394867, output_fired_time: 7.0336174965, propagate_forward_time: 405.7939758301, propagate_backward_time: 614.7413330078, weight_update_time: 0.4122419953, other_time: 10.7829589844
epoch time: 84.0472030640s, input_fired_time: 1.1130689383, output_fired_time: 0.0000000000, propagate_forward_time: 81.1317749023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8023605347
validation[          , genome  6349] predictions:    6377/  10000 (63.77%), best:    6388/10000 (63.88%), error:     10514.50488, best error:     10391.86328 on epoch:    43, epoch:   47/100, mu: 0.9555887580, learning_rate: 0.0000658142, weight_decay: 0.0000470627

epoch time: 1048.0544433594s, input_fired_time: 14.5264091492, output_fired_time: 7.0429978371, propagate_forward_time: 404.3233032227, propagate_backward_time: 611.2812500000, weight_update_time: 0.4146659374, other_time: 10.8805541992
epoch time: 83.3562927246s, input_fired_time: 1.1066025496, output_fired_time: 0.0000000000, propagate_forward_time: 80.4368667603, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8128204346
validation[          , genome  6349] predictions:    6341/  10000 (63.41%), best:    6388/10000 (63.88%), error:     10520.98730, best error:     10391.86328 on epoch:    43, epoch:   48/100, mu: 0.9574670792, learning_rate: 0.0000610796, weight_decay: 0.0000445842

epoch time: 1040.3726806641s, input_fired_time: 14.4768981934, output_fired_time: 7.0051856041, propagate_forward_time: 401.2188720703, propagate_backward_time: 606.7944946289, weight_update_time: 0.4150230587, other_time: 10.8771972656
epoch time: 83.4256286621s, input_fired_time: 1.1036113501, output_fired_time: 0.0000000000, propagate_forward_time: 80.5042572021, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8177566528
validation[          , genome  6349] predictions:    6416/  10000 (64.16%), best:    6416/10000 (64.16%), error:     10371.36719, best error:     10371.36719 on epoch:    49, epoch:   49/100, mu: 0.9592428803, learning_rate: 0.0000566855, weight_decay: 0.0000422363

epoch time: 1040.9001464844s, input_fired_time: 14.4921731949, output_fired_time: 7.0672216415, propagate_forward_time: 401.6727600098, propagate_backward_time: 606.7984619141, weight_update_time: 0.4193618894, other_time: 10.8695068359
epoch time: 83.6482162476s, input_fired_time: 1.1008598804, output_fired_time: 0.0000000000, propagate_forward_time: 80.7278060913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8195495605
validation[          , genome  6349] predictions:    6353/  10000 (63.53%), best:    6416/10000 (64.16%), error:     10541.60840, best error:     10371.36719 on epoch:    49, epoch:   50/100, mu: 0.9609217644, learning_rate: 0.0000526076, weight_decay: 0.0000400120

epoch time: 1069.8657226563s, input_fired_time: 14.7719058990, output_fired_time: 7.2595720291, propagate_forward_time: 414.0880737305, propagate_backward_time: 622.4878540039, weight_update_time: 0.4303000569, other_time: 11.2584228516
epoch time: 95.3740997314s, input_fired_time: 1.2614852190, output_fired_time: 0.0000000000, propagate_forward_time: 91.9911499023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1214675903
validation[          , genome  6349] predictions:    6392/  10000 (63.92%), best:    6392/10000 (63.92%), error:     10355.57910, best error:     10355.57910 on epoch:    51, epoch:   51/100, mu: 0.9625089765, learning_rate: 0.0000488231, weight_decay: 0.0000379048

epoch time: 1067.1093750000s, input_fired_time: 14.8018474579, output_fired_time: 7.3174815178, propagate_forward_time: 414.2639160156, propagate_backward_time: 619.2402954102, weight_update_time: 0.4397930503, other_time: 11.4857788086
epoch time: 83.2691040039s, input_fired_time: 1.1040198803, output_fired_time: 0.0000000000, propagate_forward_time: 80.3357696533, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8293151855
validation[          , genome  6349] predictions:    6447/  10000 (64.47%), best:    6447/10000 (64.47%), error:     10278.51270, best error:     10278.51270 on epoch:    52, epoch:   52/100, mu: 0.9640095830, learning_rate: 0.0000453108, weight_decay: 0.0000359086

epoch time: 1039.7609863281s, input_fired_time: 14.4842281342, output_fired_time: 7.0136432648, propagate_forward_time: 401.2082214355, propagate_backward_time: 606.2023925781, weight_update_time: 0.4220657945, other_time: 10.8524169922
epoch time: 82.6501541138s, input_fired_time: 1.1006519794, output_fired_time: 0.0000000000, propagate_forward_time: 79.7338638306, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8156356812
validation[          , genome  6349] predictions:    6398/  10000 (63.98%), best:    6447/10000 (64.47%), error:     10315.52344, best error:     10278.51270 on epoch:    52, epoch:   53/100, mu: 0.9654282928, learning_rate: 0.0000420511, weight_decay: 0.0000340176

epoch time: 1040.7188720703s, input_fired_time: 14.4982366562, output_fired_time: 7.0333838463, propagate_forward_time: 401.7471618652, propagate_backward_time: 606.5897827148, weight_update_time: 0.4246086478, other_time: 10.8502807617
epoch time: 83.5478363037s, input_fired_time: 1.1081244946, output_fired_time: 0.0000000000, propagate_forward_time: 80.6207504272, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8189620972
validation[          , genome  6349] predictions:    6510/  10000 (65.10%), best:    6510/10000 (65.10%), error:     10143.32520, best error:     10143.32520 on epoch:    54, epoch:   54/100, mu: 0.9667695165, learning_rate: 0.0000390260, weight_decay: 0.0000322261

epoch time: 1042.5252685547s, input_fired_time: 14.5092697144, output_fired_time: 7.0335206985, propagate_forward_time: 402.3047180176, propagate_backward_time: 607.8098144531, weight_update_time: 0.4289646447, other_time: 10.8679199219
epoch time: 83.5411071777s, input_fired_time: 1.1117899418, output_fired_time: 0.0000000000, propagate_forward_time: 80.6194381714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8098754883
validation[          , genome  6349] predictions:    6495/  10000 (64.95%), best:    6510/10000 (65.10%), error:     10193.81738, best error:     10143.32520 on epoch:    54, epoch:   55/100, mu: 0.9680375457, learning_rate: 0.0000362185, weight_decay: 0.0000305290

epoch time: 1042.0733642578s, input_fired_time: 14.5234050751, output_fired_time: 7.0283350945, propagate_forward_time: 402.2111206055, propagate_backward_time: 607.4066162109, weight_update_time: 0.4301753342, other_time: 10.9038696289
epoch time: 83.1211853027s, input_fired_time: 1.1143382788, output_fired_time: 0.0000000000, propagate_forward_time: 80.1862945557, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8205490112
validation[          , genome  6349] predictions:    6482/  10000 (64.82%), best:    6510/10000 (65.10%), error:     10194.67871, best error:     10143.32520 on epoch:    54, epoch:   56/100, mu: 0.9692363739, learning_rate: 0.0000336130, weight_decay: 0.0000289212

epoch time: 1045.8083496094s, input_fired_time: 14.5481891632, output_fired_time: 7.0348267555, propagate_forward_time: 403.4341735840, propagate_backward_time: 609.9398803711, weight_update_time: 0.4348236024, other_time: 10.8512573242
epoch time: 83.2718811035s, input_fired_time: 1.1153409481, output_fired_time: 0.0000000000, propagate_forward_time: 80.3409576416, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8155822754
validation[          , genome  6349] predictions:    6473/  10000 (64.73%), best:    6510/10000 (65.10%), error:     10170.23047, best error:     10143.32520 on epoch:    54, epoch:   57/100, mu: 0.9703697562, learning_rate: 0.0000311949, weight_decay: 0.0000273982

epoch time: 1050.1473388672s, input_fired_time: 14.5504817963, output_fired_time: 7.0299034119, propagate_forward_time: 404.9349975586, propagate_backward_time: 612.8328857422, weight_update_time: 0.4376836121, other_time: 10.7990112305
epoch time: 84.0665054321s, input_fired_time: 1.1186223030, output_fired_time: 0.0000000000, propagate_forward_time: 81.1421432495, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8057403564
validation[          , genome  6349] predictions:    6480/  10000 (64.80%), best:    6480/10000 (64.80%), error:     10120.78027, best error:     10120.78027 on epoch:    58, epoch:   58/100, mu: 0.9714412689, learning_rate: 0.0000289507, weight_decay: 0.0000259553

epoch time: 1041.7772216797s, input_fired_time: 14.5250968933, output_fired_time: 7.0143423080, propagate_forward_time: 402.2359924316, propagate_backward_time: 607.0881347656, weight_update_time: 0.4435528517, other_time: 10.9135742188
epoch time: 82.9439010620s, input_fired_time: 1.1113409996, output_fired_time: 0.0000000000, propagate_forward_time: 80.0165939331, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8159637451
validation[          , genome  6349] predictions:    6451/  10000 (64.51%), best:    6480/10000 (64.80%), error:     10215.35156, best error:     10120.78027 on epoch:    58, epoch:   59/100, mu: 0.9724543095, learning_rate: 0.0000268680, weight_decay: 0.0000245884

epoch time: 1041.2877197266s, input_fired_time: 14.5186815262, output_fired_time: 7.0009150505, propagate_forward_time: 402.2731323242, propagate_backward_time: 606.6468505859, weight_update_time: 0.4438225329, other_time: 10.8481445313
epoch time: 83.4874267578s, input_fired_time: 1.1189317703, output_fired_time: 0.0000000000, propagate_forward_time: 80.5549087524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8135833740
validation[          , genome  6349] predictions:    6507/  10000 (65.07%), best:    6480/10000 (64.80%), error:     10133.94336, best error:     10120.78027 on epoch:    58, epoch:   60/100, mu: 0.9734120369, learning_rate: 0.0000249352, weight_decay: 0.0000232935

epoch time: 1045.2742919922s, input_fired_time: 14.5628595352, output_fired_time: 7.0160508156, propagate_forward_time: 403.3699340820, propagate_backward_time: 609.5162353516, weight_update_time: 0.4476924837, other_time: 10.8092041016
epoch time: 84.0399246216s, input_fired_time: 1.1344312429, output_fired_time: 0.0000000000, propagate_forward_time: 81.1297454834, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7757492065
validation[          , genome  6349] predictions:    6516/  10000 (65.16%), best:    6516/10000 (65.16%), error:     10057.11914, best error:     10057.11914 on epoch:    61, epoch:   61/100, mu: 0.9743174911, learning_rate: 0.0000231414, weight_decay: 0.0000220668

epoch time: 1041.7954101563s, input_fired_time: 14.5543441772, output_fired_time: 7.0322713852, propagate_forward_time: 402.5049133301, propagate_backward_time: 606.8751831055, weight_update_time: 0.4519214630, other_time: 10.8286743164
epoch time: 83.1259002686s, input_fired_time: 1.1192827225, output_fired_time: 0.0000000000, propagate_forward_time: 80.1815872192, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8250274658
validation[          , genome  6349] predictions:    6572/  10000 (65.72%), best:    6572/10000 (65.72%), error:      9985.20117, best error:      9985.20117 on epoch:    62, epoch:   62/100, mu: 0.9751735330, learning_rate: 0.0000214766, weight_decay: 0.0000209047

epoch time: 1051.3798828125s, input_fired_time: 14.6394271851, output_fired_time: 7.0487685204, propagate_forward_time: 405.6361389160, propagate_backward_time: 613.1152343750, weight_update_time: 0.4561411440, other_time: 10.9403076172
epoch time: 83.2861175537s, input_fired_time: 1.1202095747, output_fired_time: 0.0000000000, propagate_forward_time: 80.3485336304, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8173751831
validation[          , genome  6349] predictions:    6494/  10000 (64.94%), best:    6572/10000 (65.72%), error:     10204.47266, best error:      9985.20117 on epoch:    62, epoch:   63/100, mu: 0.9759828448, learning_rate: 0.0000199316, weight_decay: 0.0000198038

epoch time: 1037.7656250000s, input_fired_time: 14.5460567474, output_fired_time: 7.0134911537, propagate_forward_time: 401.2288208008, propagate_backward_time: 604.1006469727, weight_update_time: 0.4604181051, other_time: 10.8765869141
epoch time: 83.1748428345s, input_fired_time: 1.1166951656, output_fired_time: 0.0000000000, propagate_forward_time: 80.2467041016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8114471436
validation[          , genome  6349] predictions:    6571/  10000 (65.71%), best:    6572/10000 (65.72%), error:     10046.47070, best error:      9985.20117 on epoch:    62, epoch:   64/100, mu: 0.9767479897, learning_rate: 0.0000184977, weight_decay: 0.0000187609

epoch time: 1033.1671142578s, input_fired_time: 14.5511589050, output_fired_time: 6.9945621490, propagate_forward_time: 399.3694763184, propagate_backward_time: 601.3318481445, weight_update_time: 0.4673117101, other_time: 10.9201049805
epoch time: 82.8418579102s, input_fired_time: 1.1076312065, output_fired_time: 0.0000000000, propagate_forward_time: 79.9249038696, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8093261719
validation[          , genome  6349] predictions:    6565/  10000 (65.65%), best:    6572/10000 (65.72%), error:      9989.97266, best error:      9985.20117 on epoch:    62, epoch:   65/100, mu: 0.9774713516, learning_rate: 0.0000171670, weight_decay: 0.0000177729

epoch time: 1037.1337890625s, input_fired_time: 14.5393657684, output_fired_time: 7.0057106018, propagate_forward_time: 401.2911682129, propagate_backward_time: 603.3941040039, weight_update_time: 0.4710616171, other_time: 10.9033813477
epoch time: 84.1831359863s, input_fired_time: 1.1210047007, output_fired_time: 0.0000000000, propagate_forward_time: 81.2618331909, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8003005981
validation[          , genome  6349] predictions:    6561/  10000 (65.61%), best:    6561/10000 (65.61%), error:      9964.19043, best error:      9964.19043 on epoch:    66, epoch:   66/100, mu: 0.9781552553, learning_rate: 0.0000159320, weight_decay: 0.0000168369

epoch time: 1032.0435791016s, input_fired_time: 14.5380945206, output_fired_time: 7.0164709091, propagate_forward_time: 399.4775085449, propagate_backward_time: 600.0637817383, weight_update_time: 0.4756268859, other_time: 10.9476928711
epoch time: 82.2013015747s, input_fired_time: 1.1044970751, output_fired_time: 0.0000000000, propagate_forward_time: 79.2642517090, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8325500488
validation[          , genome  6349] predictions:    6476/  10000 (64.76%), best:    6561/10000 (65.61%), error:     10192.47461, best error:      9964.19043 on epoch:    66, epoch:   67/100, mu: 0.9788017869, learning_rate: 0.0000147859, weight_decay: 0.0000159502

epoch time: 1033.7550048828s, input_fired_time: 14.5424327850, output_fired_time: 7.0160360336, propagate_forward_time: 400.4623107910, propagate_backward_time: 600.7600708008, weight_update_time: 0.4796714783, other_time: 10.9741821289
epoch time: 83.2369461060s, input_fired_time: 1.1164301634, output_fired_time: 0.0000000000, propagate_forward_time: 80.2945861816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8259277344
validation[          , genome  6349] predictions:    6521/  10000 (65.21%), best:    6561/10000 (65.61%), error:     10098.00977, best error:      9964.19043 on epoch:    66, epoch:   68/100, mu: 0.9794130325, learning_rate: 0.0000137222, weight_decay: 0.0000151102

epoch time: 1038.9753417969s, input_fired_time: 14.5543460846, output_fired_time: 7.0205421448, propagate_forward_time: 402.6809997559, propagate_backward_time: 603.8062133789, weight_update_time: 0.4805485904, other_time: 10.9132690430
epoch time: 84.4394226074s, input_fired_time: 1.1263352633, output_fired_time: 0.0000000000, propagate_forward_time: 81.5041885376, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8088989258
validation[          , genome  6349] predictions:    6519/  10000 (65.19%), best:    6561/10000 (65.61%), error:     10006.26465, best error:      9964.19043 on epoch:    66, epoch:   69/100, mu: 0.9799908996, learning_rate: 0.0000127350, weight_decay: 0.0000143145

epoch time: 1043.7016601563s, input_fired_time: 14.5836305618, output_fired_time: 7.0475401878, propagate_forward_time: 403.7803344727, propagate_backward_time: 607.4011840820, weight_update_time: 0.4905433357, other_time: 10.8889770508
epoch time: 84.7565231323s, input_fired_time: 1.1311565638, output_fired_time: 0.0000000000, propagate_forward_time: 81.8317947388, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7935714722
validation[          , genome  6349] predictions:    6629/  10000 (66.29%), best:    6629/10000 (66.29%), error:      9912.22168, best error:      9912.22168 on epoch:    70, epoch:   70/100, mu: 0.9805372357, learning_rate: 0.0000118189, weight_decay: 0.0000135606

epoch time: 1046.2839355469s, input_fired_time: 14.5723237991, output_fired_time: 7.0486612320, propagate_forward_time: 405.0679931641, propagate_backward_time: 608.6856689453, weight_update_time: 0.4955831468, other_time: 10.9093017578
epoch time: 84.4974594116s, input_fired_time: 1.1269125938, output_fired_time: 0.0000000000, propagate_forward_time: 81.5631790161, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8073654175
validation[          , genome  6349] predictions:    6560/  10000 (65.60%), best:    6629/10000 (66.29%), error:     10027.61621, best error:      9912.22168 on epoch:    70, epoch:   71/100, mu: 0.9810537696, learning_rate: 0.0000109686, weight_decay: 0.0000128465

epoch time: 1046.0314941406s, input_fired_time: 14.6004495621, output_fired_time: 7.0336647034, propagate_forward_time: 405.1561584473, propagate_backward_time: 608.3677368164, weight_update_time: 0.4983470440, other_time: 10.8734741211
epoch time: 84.3972320557s, input_fired_time: 1.1400164366, output_fired_time: 0.0000000000, propagate_forward_time: 81.4570083618, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8002090454
validation[          , genome  6349] predictions:    6648/  10000 (66.48%), best:    6648/10000 (66.48%), error:      9899.11816, best error:      9899.11816 on epoch:    72, epoch:   72/100, mu: 0.9815421104, learning_rate: 0.0000101796, weight_decay: 0.0000121699

epoch time: 1051.4598388672s, input_fired_time: 14.6334619522, output_fired_time: 7.1025857925, propagate_forward_time: 407.0460815430, propagate_backward_time: 611.8013916016, weight_update_time: 0.5048152208, other_time: 10.8764038086
epoch time: 83.9740219116s, input_fired_time: 1.1289551258, output_fired_time: 0.0000000000, propagate_forward_time: 81.0527267456, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7923431396
validation[          , genome  6349] predictions:    6591/  10000 (65.91%), best:    6648/10000 (66.48%), error:      9940.92188, best error:      9899.11816 on epoch:    72, epoch:   73/100, mu: 0.9820038080, learning_rate: 0.0000094473, weight_decay: 0.0000115290

epoch time: 1045.2819824219s, input_fired_time: 14.6173162460, output_fired_time: 7.0392913818, propagate_forward_time: 404.1914367676, propagate_backward_time: 608.5401611328, weight_update_time: 0.5151824355, other_time: 10.8937988281
epoch time: 83.8318557739s, input_fired_time: 1.1262004375, output_fired_time: 0.0000000000, propagate_forward_time: 80.8948287964, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8108291626
validation[          , genome  6349] predictions:    6538/  10000 (65.38%), best:    6648/10000 (66.48%), error:     10031.38379, best error:      9899.11816 on epoch:    72, epoch:   74/100, mu: 0.9824402928, learning_rate: 0.0000087676, weight_decay: 0.0000109219

epoch time: 1046.0405273438s, input_fired_time: 14.6291847229, output_fired_time: 7.0393838882, propagate_forward_time: 404.2686767578, propagate_backward_time: 609.1629028320, weight_update_time: 0.5155640841, other_time: 10.9403686523
epoch time: 84.1641159058s, input_fired_time: 1.1340615749, output_fired_time: 0.0000000000, propagate_forward_time: 81.2216491699, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8084030151
validation[          , genome  6349] predictions:    6598/  10000 (65.98%), best:    6648/10000 (66.48%), error:      9910.89941, best error:      9899.11816 on epoch:    72, epoch:   75/100, mu: 0.9828529358, learning_rate: 0.0000081369, weight_decay: 0.0000103467

epoch time: 1049.8526611328s, input_fired_time: 14.6569013596, output_fired_time: 7.0749735832, propagate_forward_time: 405.9745788574, propagate_backward_time: 611.2286376953, weight_update_time: 0.5212574005, other_time: 10.9176025391
epoch time: 84.1194381714s, input_fired_time: 1.1311786175, output_fired_time: 0.0000000000, propagate_forward_time: 81.1853103638, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8029479980
validation[          , genome  6349] predictions:    6637/  10000 (66.37%), best:    6637/10000 (66.37%), error:      9856.92676, best error:      9856.92676 on epoch:    76, epoch:   76/100, mu: 0.9832430482, learning_rate: 0.0000075515, weight_decay: 0.0000098018

epoch time: 1046.6324462891s, input_fired_time: 14.5969686508, output_fired_time: 7.0384101868, propagate_forward_time: 404.4577636719, propagate_backward_time: 609.6329956055, weight_update_time: 0.5294392705, other_time: 10.9063110352
epoch time: 83.7344207764s, input_fired_time: 1.1259261370, output_fired_time: 0.0000000000, propagate_forward_time: 80.8118972778, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7966003418
validation[          , genome  6349] predictions:    6608/  10000 (66.08%), best:    6637/10000 (66.37%), error:      9905.21484, best error:      9856.92676 on epoch:    76, epoch:   77/100, mu: 0.9836118817, learning_rate: 0.0000070083, weight_decay: 0.0000092856

epoch time: 1084.1734619141s, input_fired_time: 14.9350128174, output_fired_time: 7.2876534462, propagate_forward_time: 418.7042541504, propagate_backward_time: 631.9235839844, weight_update_time: 0.5452957749, other_time: 11.3229980469
epoch time: 88.6997146606s, input_fired_time: 1.1602275372, output_fired_time: 0.0000000000, propagate_forward_time: 85.6681365967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8713531494
validation[          , genome  6349] predictions:    6589/  10000 (65.89%), best:    6637/10000 (66.37%), error:      9929.24121, best error:      9856.92676 on epoch:    76, epoch:   78/100, mu: 0.9839605689, learning_rate: 0.0000065041, weight_decay: 0.0000087966

epoch time: 1077.0004882813s, input_fired_time: 14.8547487259, output_fired_time: 7.2065486908, propagate_forward_time: 418.5280761719, propagate_backward_time: 625.2089843750, weight_update_time: 0.5521158576, other_time: 11.2021484375
epoch time: 88.1666030884s, input_fired_time: 1.1848753691, output_fired_time: 0.0000000000, propagate_forward_time: 85.0482177734, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9335098267
validation[          , genome  6349] predictions:    6628/  10000 (66.28%), best:    6637/10000 (66.37%), error:      9898.32910, best error:      9856.92676 on epoch:    76, epoch:   79/100, mu: 0.9842902422, learning_rate: 0.0000060362, weight_decay: 0.0000083334

epoch time: 1041.5234375000s, input_fired_time: 14.6069183350, output_fired_time: 7.0173439980, propagate_forward_time: 403.3258972168, propagate_backward_time: 605.6146850586, weight_update_time: 0.5493581891, other_time: 10.9585571289
epoch time: 83.8233642578s, input_fired_time: 1.1263276339, output_fired_time: 0.0000000000, propagate_forward_time: 80.8894729614, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8075637817
validation[          , genome  6349] predictions:    6616/  10000 (66.16%), best:    6616/10000 (66.16%), error:      9849.92285, best error:      9849.92285 on epoch:    80, epoch:   80/100, mu: 0.9846019149, learning_rate: 0.0000056020, weight_decay: 0.0000078945

epoch time: 1031.9948730469s, input_fired_time: 14.5950984955, output_fired_time: 7.0036129951, propagate_forward_time: 399.5686340332, propagate_backward_time: 599.7803344727, weight_update_time: 0.5577819347, other_time: 11.0471801758
epoch time: 82.2167053223s, input_fired_time: 1.1048508883, output_fired_time: 0.0000000000, propagate_forward_time: 79.2710266113, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8408279419
validation[          , genome  6349] predictions:    6610/  10000 (66.10%), best:    6616/10000 (66.16%), error:      9858.59570, best error:      9849.92285 on epoch:    80, epoch:   81/100, mu: 0.9848965406, learning_rate: 0.0000051990, weight_decay: 0.0000074788

epoch time: 1026.0789794922s, input_fired_time: 14.5986404419, output_fired_time: 7.0128483772, propagate_forward_time: 397.3669433594, propagate_backward_time: 596.0195922852, weight_update_time: 0.5638471246, other_time: 11.0809326172
epoch time: 82.4785385132s, input_fired_time: 1.1158015728, output_fired_time: 0.0000000000, propagate_forward_time: 79.5307846069, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8319549561
validation[          , genome  6349] predictions:    6627/  10000 (66.27%), best:    6627/10000 (66.27%), error:      9835.45215, best error:      9835.45215 on epoch:    82, epoch:   82/100, mu: 0.9851751328, learning_rate: 0.0000048250, weight_decay: 0.0000070849

epoch time: 1029.4858398438s, input_fired_time: 14.5828657150, output_fired_time: 7.0130062103, propagate_forward_time: 398.7017517090, propagate_backward_time: 598.0930786133, weight_update_time: 0.5687122345, other_time: 11.0951538086
epoch time: 82.8824615479s, input_fired_time: 1.1163660288, output_fired_time: 0.0000000000, propagate_forward_time: 79.9409942627, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8251037598
validation[          , genome  6349] predictions:    6652/  10000 (66.52%), best:    6627/10000 (66.27%), error:      9852.04395, best error:      9835.45215 on epoch:    82, epoch:   83/100, mu: 0.9854385257, learning_rate: 0.0000044778, weight_decay: 0.0000067118

epoch time: 1028.2631835938s, input_fired_time: 14.5942468643, output_fired_time: 7.0393319130, propagate_forward_time: 398.3600769043, propagate_backward_time: 597.1009521484, weight_update_time: 0.5755428672, other_time: 11.1685791016
epoch time: 82.4655456543s, input_fired_time: 1.1143128872, output_fired_time: 0.0000000000, propagate_forward_time: 79.5338058472, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8174285889
validation[          , genome  6349] predictions:    6589/  10000 (65.89%), best:    6627/10000 (66.27%), error:      9941.89258, best error:      9835.45215 on epoch:    82, epoch:   84/100, mu: 0.9856874943, learning_rate: 0.0000041557, weight_decay: 0.0000063583

epoch time: 1033.5811767578s, input_fired_time: 14.5940771103, output_fired_time: 7.0131754875, propagate_forward_time: 400.5188903809, propagate_backward_time: 600.4380493164, weight_update_time: 0.5830523968, other_time: 11.0170288086
epoch time: 83.5829010010s, input_fired_time: 1.1327395439, output_fired_time: 0.0000000000, propagate_forward_time: 80.6439666748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8061981201
validation[          , genome  6349] predictions:    6602/  10000 (66.02%), best:    6627/10000 (66.27%), error:      9920.38965, best error:      9835.45215 on epoch:    82, epoch:   85/100, mu: 0.9859228730, learning_rate: 0.0000038568, weight_decay: 0.0000060235

epoch time: 1038.2014160156s, input_fired_time: 14.6055402756, output_fired_time: 7.0455832481, propagate_forward_time: 402.1878356934, propagate_backward_time: 603.3744506836, weight_update_time: 0.5896238685, other_time: 10.9879760742
epoch time: 83.1316299438s, input_fired_time: 1.1176086664, output_fired_time: 0.0000000000, propagate_forward_time: 80.2099685669, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8040542603
validation[          , genome  6349] predictions:    6580/  10000 (65.80%), best:    6627/10000 (66.27%), error:      9886.60547, best error:      9835.45215 on epoch:    82, epoch:   86/100, mu: 0.9861454368, learning_rate: 0.0000035793, weight_decay: 0.0000057063

epoch time: 1038.6118164063s, input_fired_time: 14.6324243546, output_fired_time: 7.0451602936, propagate_forward_time: 402.3620605469, propagate_backward_time: 603.5733032227, weight_update_time: 0.5976799726, other_time: 10.9988403320
epoch time: 84.0428390503s, input_fired_time: 1.1327195168, output_fired_time: 0.0000000000, propagate_forward_time: 81.1136474609, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7964706421
validation[          , genome  6349] predictions:    6633/  10000 (66.33%), best:    6633/10000 (66.33%), error:      9828.69434, best error:      9828.69434 on epoch:    87, epoch:   87/100, mu: 0.9863558412, learning_rate: 0.0000033218, weight_decay: 0.0000054058

epoch time: 1046.6806640625s, input_fired_time: 14.6620388031, output_fired_time: 7.0803761482, propagate_forward_time: 405.5547180176, propagate_backward_time: 608.4092407227, weight_update_time: 0.6057980061, other_time: 10.9744262695
epoch time: 83.9860076904s, input_fired_time: 1.1308922768, output_fired_time: 0.0000000000, propagate_forward_time: 81.0653381348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7897796631
validation[          , genome  6349] predictions:    6611/  10000 (66.11%), best:    6633/10000 (66.33%), error:      9835.18750, best error:      9828.69434 on epoch:    87, epoch:   88/100, mu: 0.9865547419, learning_rate: 0.0000030828, weight_decay: 0.0000051211

epoch time: 1040.2800292969s, input_fired_time: 14.6262950897, output_fired_time: 7.0669550896, propagate_forward_time: 403.0362243652, propagate_backward_time: 604.5293579102, weight_update_time: 0.6056568027, other_time: 11.0211791992
epoch time: 83.8294677734s, input_fired_time: 1.1280257702, output_fired_time: 0.0000000000, propagate_forward_time: 80.8999176025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8015213013
validation[          , genome  6349] predictions:    6620/  10000 (66.20%), best:    6633/10000 (66.33%), error:      9833.96484, best error:      9828.69434 on epoch:    87, epoch:   89/100, mu: 0.9867427945, learning_rate: 0.0000028611, weight_decay: 0.0000048514

epoch time: 1041.8729248047s, input_fired_time: 14.6565818787, output_fired_time: 7.0582284927, propagate_forward_time: 403.7536621094, propagate_backward_time: 605.4073486328, weight_update_time: 0.6110225320, other_time: 10.9970703125
epoch time: 84.4030838013s, input_fired_time: 1.1371815205, output_fired_time: 0.0000000000, propagate_forward_time: 81.4667968750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7991027832
validation[          , genome  6349] predictions:    6659/  10000 (66.59%), best:    6659/10000 (66.59%), error:      9820.73633, best error:      9820.73633 on epoch:    90, epoch:   90/100, mu: 0.9869205952, learning_rate: 0.0000026552, weight_decay: 0.0000045959

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6349_2_r1718044197_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.508671
read mu: 0.987089
read mu_delta: 0.945415
read initial_learning_rate: 0.00219902
read learning_rate: 2.46423e-06
read learning_rate_delta: 0.928061
read initial_weight_decay: 0.000598364
read weight_decay: 4.35385e-06
read weight_decay_delta: 0.947337
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.085487
read input_dropout_probability: 0.00124867
read hidden_dropout_probability: 0.0993689
read velocity_reset: 1745
read epoch: 91
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 90
read number_validation_images: 10000
read best_validation_predictions: 6659
read best_validation_error: 9820.74
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6349
read normal distribution: '1 0 0'
generator_str: '525631989'
read generator: 525631989
read generated_by_map:
1 enable_node 1
reading 76 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6349.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_187_6349_2_r1718044197_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.508671
read mu: 0.987089
read mu_delta: 0.945415
read initial_learning_rate: 0.00219902
read learning_rate: 2.46423e-06
read learning_rate_delta: 0.928061
read initial_weight_decay: 0.000598364
read weight_decay: 4.35385e-06
read weight_decay_delta: 0.947337
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.085487
read input_dropout_probability: 0.00124867
read hidden_dropout_probability: 0.0993689
read velocity_reset: 1745
read epoch: 91
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 90
read number_validation_images: 10000
read best_validation_predictions: 6659
read best_validation_error: 9820.74
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6349
read normal distribution: '1 0 0'
generator_str: '525631989'
read generator: 525631989
read generated_by_map:
1 enable_node 1
reading 76 nodes.
reading 314 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1055.99792480469s, input_fired_time: 14.7481441497803, output_fired_time: 7.12504148483276, propagate_forward_time: 409.907897949219, propagate_backward_time: 613.275756835938, weight_update_time: 0.617911756038666, other_time: 10.9411010742188
epoch time: 84.0453720092773s, input_fired_time: 1.13241052627563, output_fired_time: 0, propagate_forward_time: 81.1170043945313, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.79595947265625
validation[          , genome  6349] predictions:    6645/  10000 (66.45%), best:    6645/10000 (66.45%), error:      9819.41797, best error:      9819.41797 on epoch:    91, epoch:   91/100, mu: 0.9870886803, learning_rate: 0.0000024642, weight_decay: 0.0000043539

epoch time: 1046.6126708984s, input_fired_time: 14.6520261765, output_fired_time: 7.0965390205, propagate_forward_time: 405.7189941406, propagate_backward_time: 608.1684570313, weight_update_time: 0.6234941483, other_time: 10.9766845703
epoch time: 83.9775848389s, input_fired_time: 1.1322543621, output_fired_time: 0.0000000000, propagate_forward_time: 81.0609893799, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7843399048
validation[          , genome  6349] predictions:    6678/  10000 (66.78%), best:    6678/10000 (66.78%), error:      9792.97754, best error:      9792.97754 on epoch:    92, epoch:   92/100, mu: 0.9872475863, learning_rate: 0.0000022870, weight_decay: 0.0000041246

epoch time: 1046.3376464844s, input_fired_time: 14.6467866898, output_fired_time: 7.0541696548, propagate_forward_time: 405.6742858887, propagate_backward_time: 607.9717407227, weight_update_time: 0.6300472021, other_time: 10.9906616211
epoch time: 84.0746536255s, input_fired_time: 1.1321011782, output_fired_time: 0.0000000000, propagate_forward_time: 81.1426162720, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7999343872
validation[          , genome  6349] predictions:    6661/  10000 (66.61%), best:    6678/10000 (66.78%), error:      9808.26953, best error:      9792.97754 on epoch:    92, epoch:   93/100, mu: 0.9873978496, learning_rate: 0.0000021224, weight_decay: 0.0000039074

epoch time: 1052.2814941406s, input_fired_time: 14.6589488983, output_fired_time: 7.0541529655, propagate_forward_time: 407.4030456543, propagate_backward_time: 612.2687988281, weight_update_time: 0.6334291697, other_time: 10.8964843750
epoch time: 84.6333160400s, input_fired_time: 1.1401875019, output_fired_time: 0.0000000000, propagate_forward_time: 81.7058563232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7872695923
validation[          , genome  6349] predictions:    6686/  10000 (66.86%), best:    6686/10000 (66.86%), error:      9772.09375, best error:      9772.09375 on epoch:    94, epoch:   94/100, mu: 0.9875398874, learning_rate: 0.0000019697, weight_decay: 0.0000037016

epoch time: 1058.0097656250s, input_fired_time: 14.6802473068, output_fired_time: 7.0772566795, propagate_forward_time: 408.7888488770, propagate_backward_time: 616.6035156250, weight_update_time: 0.6311972737, other_time: 10.8598632813
epoch time: 85.0479507446s, input_fired_time: 1.1471494436, output_fired_time: 0.0000000000, propagate_forward_time: 82.1439056396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7568969727
validation[          , genome  6349] predictions:    6681/  10000 (66.81%), best:    6686/10000 (66.86%), error:      9782.29785, best error:      9772.09375 on epoch:    94, epoch:   95/100, mu: 0.9876741767, learning_rate: 0.0000018280, weight_decay: 0.0000035066

epoch time: 1055.0227050781s, input_fired_time: 14.6733093262, output_fired_time: 7.0697360039, propagate_forward_time: 408.1936035156, propagate_backward_time: 614.1931762695, weight_update_time: 0.6358460188, other_time: 10.8928833008
epoch time: 84.8427505493s, input_fired_time: 1.1443132162, output_fired_time: 0.0000000000, propagate_forward_time: 81.9173202515, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7811203003
validation[          , genome  6349] predictions:    6644/  10000 (66.44%), best:    6686/10000 (66.86%), error:      9818.96973, best error:      9772.09375 on epoch:    94, epoch:   96/100, mu: 0.9878011346, learning_rate: 0.0000016965, weight_decay: 0.0000033220

epoch time: 1059.6611328125s, input_fired_time: 14.7451686859, output_fired_time: 7.1724157333, propagate_forward_time: 410.9079589844, propagate_backward_time: 615.8809204102, weight_update_time: 0.6416794658, other_time: 10.9547729492
epoch time: 84.4434967041s, input_fired_time: 1.1416273117, output_fired_time: 0.0000000000, propagate_forward_time: 81.4919204712, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8099517822
validation[          , genome  6349] predictions:    6661/  10000 (66.61%), best:    6686/10000 (66.86%), error:      9782.49609, best error:      9772.09375 on epoch:    94, epoch:   97/100, mu: 0.9879211783, learning_rate: 0.0000015745, weight_decay: 0.0000031470

epoch time: 1049.5966796875s, input_fired_time: 14.6571359634, output_fired_time: 7.0785455704, propagate_forward_time: 406.1592712402, propagate_backward_time: 610.7568359375, weight_update_time: 0.6335818172, other_time: 10.9449462891
epoch time: 83.1972961426s, input_fired_time: 1.1216999292, output_fired_time: 0.0000000000, propagate_forward_time: 80.2768020630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7987976074
validation[          , genome  6349] predictions:    6657/  10000 (66.57%), best:    6686/10000 (66.86%), error:      9808.99023, best error:      9772.09375 on epoch:    94, epoch:   98/100, mu: 0.9880346656, learning_rate: 0.0000014612, weight_decay: 0.0000029813

epoch time: 1042.9674072266s, input_fired_time: 14.6749382019, output_fired_time: 7.0740242004, propagate_forward_time: 404.0838317871, propagate_backward_time: 606.0842285156, weight_update_time: 0.6393904090, other_time: 11.0504150391
epoch time: 83.4581298828s, input_fired_time: 1.1276857853, output_fired_time: 0.0000000000, propagate_forward_time: 80.5253677368, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8050765991
validation[          , genome  6349] predictions:    6659/  10000 (66.59%), best:    6686/10000 (66.86%), error:      9785.91797, best error:      9772.09375 on epoch:    94, epoch:   99/100, mu: 0.9881419539, learning_rate: 0.0000013561, weight_decay: 0.0000028243

epoch time: 1030.2813720703s, input_fired_time: 14.5977735519, output_fired_time: 7.0287408829, propagate_forward_time: 399.1910095215, propagate_backward_time: 598.3267822266, weight_update_time: 0.6437649727, other_time: 11.1370849609
epoch time: 81.7249374390s, input_fired_time: 1.1130369902, output_fired_time: 0.0000000000, propagate_forward_time: 78.7801284790, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8317718506
validation[          , genome  6349] predictions:    6680/  10000 (66.80%), best:    6686/10000 (66.86%), error:      9772.50586, best error:      9772.09375 on epoch:    94, epoch:  100/100, mu: 0.9882434011, learning_rate: 0.0000012585, weight_decay: 0.0000026756

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 410.2761535645s, input_fired_time: 5.5813279152, output_fired_time: 0.0000000000, propagate_forward_time: 395.5478515625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.1469726563
best training[          , genome  6349] predictions:   36207/  50000 (72.41%), best:    6686/10000 (66.86%), error:     42081.47656, best error:      9772.09375 on epoch:    94, epoch:  101/100, mu: 0.9883393049, learning_rate: 0.0000011680, weight_decay: 0.0000025347
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 81.8094940186s, input_fired_time: 1.1130968332, output_fired_time: 0.0000000000, propagate_forward_time: 78.8847808838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8116149902
   testing[          , genome  6349] predictions:    6686/  10000 (66.86%), best:    6686/10000 (66.86%), error:      9772.09375, best error:      9772.09375 on epoch:    94, epoch:  101/100, mu: 0.9883393049, learning_rate: 0.0000011680, weight_decay: 0.0000025347
backpropagation finished successfully!
17:18:12 (21276): called boinc_finish(0)

</stderr_txt>
]]>