Nameexact_genome_1542649503_185_6354_2
Workunit4099628
Created20 Nov 2018, 3:19:56 UTC
Sent20 Nov 2018, 3:30:00 UTC
Report deadline7 Feb 2019, 18:19:42 UTC
Received26 Nov 2018, 21:08:19 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID88000
Run time1 days 15 hours 19 min 49 sec
CPU time1 days 15 hours 18 min 31 sec
Validate stateValid
Credit19,904.46
Device peak FLOPS3.82 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size516.98 MB
Peak swap size515.41 MB
Peak disk usage10.33 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
ther_time: 2.1906585693
validation[          , genome  6354] predictions:    5642/  10000 (56.42%), best:    5617/10000 (56.17%), error:     13624.34961, best error:     13220.60059 on epoch:    22, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 1282.5139160156s, input_fired_time: 9.7332019806, output_fired_time: 9.1501312256, propagate_forward_time: 511.4234924316, propagate_backward_time: 739.5425415039, weight_update_time: 1.4656895399, other_time: 12.6646118164
epoch time: 105.3891525269s, input_fired_time: 1.2594548464, output_fired_time: 0.0000000000, propagate_forward_time: 101.9567794800, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1729202271
validation[          , genome  6354] predictions:    5744/  10000 (57.44%), best:    5744/10000 (57.44%), error:     12831.86816, best error:     12831.86816 on epoch:    29, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 1285.6875000000s, input_fired_time: 9.9236011505, output_fired_time: 9.1897706985, propagate_forward_time: 513.1127929688, propagate_backward_time: 740.5106811523, weight_update_time: 1.4756028652, other_time: 12.9506225586
epoch time: 106.5335922241s, input_fired_time: 1.2805862427, output_fired_time: 0.0000000000, propagate_forward_time: 103.0291290283, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2238769531
validation[          , genome  6354] predictions:    5799/  10000 (57.99%), best:    5744/10000 (57.44%), error:     12952.69531, best error:     12831.86816 on epoch:    29, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 1286.5245361328s, input_fired_time: 9.9020366669, output_fired_time: 9.1884050369, propagate_forward_time: 513.0573120117, propagate_backward_time: 741.4441528320, weight_update_time: 1.4727836847, other_time: 12.9327392578
epoch time: 105.9282073975s, input_fired_time: 1.2958588600, output_fired_time: 0.0000000000, propagate_forward_time: 102.3530502319, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2792968750
validation[          , genome  6354] predictions:    5553/  10000 (55.53%), best:    5744/10000 (57.44%), error:     13808.01758, best error:     12831.86816 on epoch:    29, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 1284.7091064453s, input_fired_time: 9.7830657959, output_fired_time: 9.1662788391, propagate_forward_time: 512.7559204102, propagate_backward_time: 740.3135375977, weight_update_time: 1.4627205133, other_time: 12.6903076172
epoch time: 106.1603851318s, input_fired_time: 1.2374534607, output_fired_time: 0.0000000000, propagate_forward_time: 102.7898559570, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1330718994
validation[          , genome  6354] predictions:    5640/  10000 (56.40%), best:    5744/10000 (57.44%), error:     13174.32031, best error:     12831.86816 on epoch:    29, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 1286.5041503906s, input_fired_time: 9.8649225235, output_fired_time: 9.1725215912, propagate_forward_time: 513.3703613281, propagate_backward_time: 741.2625122070, weight_update_time: 1.4694108963, other_time: 12.8339233398
epoch time: 105.9154815674s, input_fired_time: 1.2867542505, output_fired_time: 0.0000000000, propagate_forward_time: 102.3690719604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2596588135
validation[          , genome  6354] predictions:    5786/  10000 (57.86%), best:    5744/10000 (57.44%), error:     12935.86230, best error:     12831.86816 on epoch:    29, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 1283.4776611328s, input_fired_time: 9.7934980392, output_fired_time: 9.1751136780, propagate_forward_time: 512.1293334961, propagate_backward_time: 739.6988525391, weight_update_time: 1.4626029730, other_time: 12.6808471680
epoch time: 105.3666534424s, input_fired_time: 1.2943983078, output_fired_time: 0.0000000000, propagate_forward_time: 101.8122406006, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2600173950
validation[          , genome  6354] predictions:    5661/  10000 (56.61%), best:    5744/10000 (57.44%), error:     13328.43945, best error:     12831.86816 on epoch:    29, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 1287.5466308594s, input_fired_time: 9.9444208145, output_fired_time: 9.1798830032, propagate_forward_time: 513.9835815430, propagate_backward_time: 741.5312500000, weight_update_time: 1.4723919630, other_time: 12.9074096680
epoch time: 106.0952758789s, input_fired_time: 1.3031414747, output_fired_time: 0.0000000000, propagate_forward_time: 102.5324935913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2596435547
validation[          , genome  6354] predictions:    5488/  10000 (54.88%), best:    5744/10000 (57.44%), error:     14223.11914, best error:     12831.86816 on epoch:    29, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 1284.7675781250s, input_fired_time: 9.7572193146, output_fired_time: 9.1874923706, propagate_forward_time: 512.1353759766, propagate_backward_time: 741.1057739258, weight_update_time: 1.4615368843, other_time: 12.5817260742
epoch time: 106.0247192383s, input_fired_time: 1.2628618479, output_fired_time: 0.0000000000, propagate_forward_time: 102.5783691406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1834869385
validation[          , genome  6354] predictions:    5815/  10000 (58.15%), best:    5815/10000 (58.15%), error:     12688.38477, best error:     12688.38477 on epoch:    36, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 1286.3493652344s, input_fired_time: 9.7253389359, output_fired_time: 9.1601285934, propagate_forward_time: 513.7648925781, propagate_backward_time: 741.1631469727, weight_update_time: 1.4589265585, other_time: 12.5358276367
epoch time: 106.5471878052s, input_fired_time: 1.2874753475, output_fired_time: 0.0000000000, propagate_forward_time: 103.0639572144, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1957550049
validation[          , genome  6354] predictions:    5746/  10000 (57.46%), best:    5815/10000 (58.15%), error:     13310.70508, best error:     12688.38477 on epoch:    36, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 1285.8803710938s, input_fired_time: 9.9255466461, output_fired_time: 9.1936416626, propagate_forward_time: 513.2942504883, propagate_backward_time: 740.6608276367, weight_update_time: 1.4682930708, other_time: 12.8061523438
epoch time: 106.5837936401s, input_fired_time: 1.2976188660, output_fired_time: 0.0000000000, propagate_forward_time: 103.0190658569, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2671127319
validation[          , genome  6354] predictions:    5719/  10000 (57.19%), best:    5815/10000 (58.15%), error:     12986.12891, best error:     12688.38477 on epoch:    36, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_185_6354.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_185_6354_2_r122885010_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.923715
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00169095
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 6.76379e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 39
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 36
read number_validation_images: 10000
read best_validation_predictions: 5815
read best_validation_error: 12688.4
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6354
read normal distribution: '0 0.583398282527924 0.171483263373375'
generator_str: '322009518'
read generator: 322009518
read generated_by_map:
1 add_node 1
reading 82 nodes.
reading 333 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1298.28430175781s, input_fired_time: 9.99283599853516, output_fired_time: 9.29272937774658, propagate_forward_time: 519.093933105469, propagate_backward_time: 746.839111328125, weight_update_time: 1.47786116600037, other_time: 13.0657348632813
epoch time: 107.380104064941s, input_fired_time: 1.32393276691437, output_fired_time: 0, propagate_forward_time: 103.776573181152, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.27959442138672
validation[          , genome  6354] predictions:    5458/  10000 (54.58%), best:    5815/10000 (58.15%), error:     14250.06641, best error:     12688.38477 on epoch:    36, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 1285.9108886719s, input_fired_time: 10.0188951492, output_fired_time: 9.2136754990, propagate_forward_time: 512.8995361328, propagate_backward_time: 740.6950683594, weight_update_time: 1.4758995771, other_time: 13.0837402344
epoch time: 106.4265289307s, input_fired_time: 1.2817472219, output_fired_time: 0.0000000000, propagate_forward_time: 102.9477539063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1970291138
validation[          , genome  6354] predictions:    5771/  10000 (57.71%), best:    5815/10000 (58.15%), error:     13033.74805, best error:     12688.38477 on epoch:    36, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 1295.0515136719s, input_fired_time: 10.0327205658, output_fired_time: 9.2530384064, propagate_forward_time: 517.5469970703, propagate_backward_time: 745.1265869141, weight_update_time: 1.4807591438, other_time: 13.0921630859
epoch time: 106.1442184448s, input_fired_time: 1.3108307123, output_fired_time: 0.0000000000, propagate_forward_time: 102.5875625610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2458267212
validation[          , genome  6354] predictions:    5882/  10000 (58.82%), best:    5882/10000 (58.82%), error:     12643.79492, best error:     12643.79492 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 1287.2714843750s, input_fired_time: 9.8815889359, output_fired_time: 9.1842489243, propagate_forward_time: 513.5888061523, propagate_backward_time: 741.8354492188, weight_update_time: 1.4651913643, other_time: 12.7814331055
epoch time: 108.1029663086s, input_fired_time: 1.2385509014, output_fired_time: 0.0000000000, propagate_forward_time: 104.7111892700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1532287598
validation[          , genome  6354] predictions:    5848/  10000 (58.48%), best:    5882/10000 (58.82%), error:     12720.51172, best error:     12643.79492 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 1289.5418701172s, input_fired_time: 9.8907814026, output_fired_time: 9.1989555359, propagate_forward_time: 515.0463867188, propagate_backward_time: 742.6374511719, weight_update_time: 1.4657611847, other_time: 12.7683105469
epoch time: 105.7378234863s, input_fired_time: 1.3146973848, output_fired_time: 0.0000000000, propagate_forward_time: 102.1642532349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2588729858
validation[          , genome  6354] predictions:    5881/  10000 (58.81%), best:    5882/10000 (58.82%), error:     12824.23730, best error:     12643.79492 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 1284.7515869141s, input_fired_time: 10.0569515228, output_fired_time: 9.1608524323, propagate_forward_time: 511.9701538086, propagate_backward_time: 740.5645751953, weight_update_time: 1.4720335007, other_time: 12.9989624023
epoch time: 106.7160263062s, input_fired_time: 1.2652597427, output_fired_time: 0.0000000000, propagate_forward_time: 103.3126296997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1381378174
validation[          , genome  6354] predictions:    5809/  10000 (58.09%), best:    5882/10000 (58.82%), error:     12851.07324, best error:     12643.79492 on epoch:    41, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 1282.5362548828s, input_fired_time: 9.8124361038, output_fired_time: 9.1528301239, propagate_forward_time: 511.7150573730, propagate_backward_time: 739.2656250000, weight_update_time: 1.4564918280, other_time: 12.5903320313
epoch time: 105.7960128784s, input_fired_time: 1.3020741940, output_fired_time: 0.0000000000, propagate_forward_time: 102.2732162476, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2207260132
validation[          , genome  6354] predictions:    5865/  10000 (58.65%), best:    5882/10000 (58.82%), error:     12806.39746, best error:     12643.79492 on epoch:    41, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 1288.1495361328s, input_fired_time: 9.8395977020, output_fired_time: 9.1690855026, propagate_forward_time: 514.7794799805, propagate_backward_time: 741.7971801758, weight_update_time: 1.4580934048, other_time: 12.5642089844
epoch time: 105.1844711304s, input_fired_time: 1.2612987757, output_fired_time: 0.0000000000, propagate_forward_time: 101.7935714722, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1296005249
validation[          , genome  6354] predictions:    5715/  10000 (57.15%), best:    5882/10000 (58.82%), error:     13183.24414, best error:     12643.79492 on epoch:    41, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 1284.9775390625s, input_fired_time: 9.8871259689, output_fired_time: 9.1943960190, propagate_forward_time: 512.5878295898, propagate_backward_time: 740.6426391602, weight_update_time: 1.4599266052, other_time: 12.6656494141
epoch time: 105.8849105835s, input_fired_time: 1.3189303875, output_fired_time: 0.0000000000, propagate_forward_time: 102.3086318970, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2573471069
validation[          , genome  6354] predictions:    5874/  10000 (58.74%), best:    5874/10000 (58.74%), error:     12606.65820, best error:     12606.65820 on epoch:    47, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_185_6354.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1542649503_185_6354_2_r122885010_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.948224
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00106572
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 4.26288e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 48
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 47
read number_validation_images: 10000
read best_validation_predictions: 5874
read best_validation_error: 12606.7
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6354
read normal distribution: '0 0.583398282527924 0.171483263373375'
generator_str: '982736273'
read generator: 982736273
read generated_by_map:
1 add_node 1
reading 82 nodes.
reading 333 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 1297.98498535156s, input_fired_time: 10.0486450195313, output_fired_time: 9.29008102416992, propagate_forward_time: 518.54296875, propagate_backward_time: 746.954711914063, weight_update_time: 1.48623919487, other_time: 13.1485595703125
epoch time: 106.951507568359s, input_fired_time: 1.32516825199127, output_fired_time: 0, propagate_forward_time: 103.35905456543, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.26728820800781
validation[          , genome  6354] predictions:    5836/  10000 (58.36%), best:    5874/10000 (58.74%), error:     12740.70313, best error:     12606.65820 on epoch:    47, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 1302.5067138672s, input_fired_time: 9.8591699600, output_fired_time: 9.2923336029, propagate_forward_time: 520.9599609375, propagate_backward_time: 749.5421142578, weight_update_time: 1.4718933105, other_time: 12.8531494141
epoch time: 106.5484008789s, input_fired_time: 1.2821726799, output_fired_time: 0.0000000000, propagate_forward_time: 103.0715408325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1946868896
validation[          , genome  6354] predictions:    5953/  10000 (59.53%), best:    5953/10000 (59.53%), error:     12547.36230, best error:     12547.36230 on epoch:    49, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 1283.5556640625s, input_fired_time: 10.1145858765, output_fired_time: 9.1720771790, propagate_forward_time: 511.7037658691, propagate_backward_time: 739.5186157227, weight_update_time: 1.4789093733, other_time: 13.0465698242
epoch time: 105.0332489014s, input_fired_time: 1.2951170206, output_fired_time: 0.0000000000, propagate_forward_time: 101.5079116821, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2302169800
validation[          , genome  6354] predictions:    5856/  10000 (58.56%), best:    5953/10000 (59.53%), error:     12712.32422, best error:     12547.36230 on epoch:    49, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 1286.1595458984s, input_fired_time: 9.9498548508, output_fired_time: 9.1853694916, propagate_forward_time: 513.5242309570, propagate_backward_time: 740.7406005859, weight_update_time: 1.4687962532, other_time: 12.7594604492
epoch time: 106.1180419922s, input_fired_time: 1.3266036510, output_fired_time: 0.0000000000, propagate_forward_time: 102.5302734375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2611618042
validation[          , genome  6354] predictions:    6093/  10000 (60.93%), best:    6093/10000 (60.93%), error:     12040.73535, best error:     12040.73535 on epoch:    51, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 1282.1394042969s, input_fired_time: 10.0950689316, output_fired_time: 9.1724023819, propagate_forward_time: 510.9764404297, propagate_backward_time: 738.8965454102, weight_update_time: 1.4753389359, other_time: 12.9989624023
epoch time: 106.0443725586s, input_fired_time: 1.2675056458, output_fired_time: 0.0000000000, propagate_forward_time: 102.6296081543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1472625732
validation[          , genome  6354] predictions:    5962/  10000 (59.62%), best:    6093/10000 (60.93%), error:     12404.91699, best error:     12040.73535 on epoch:    51, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 1285.8895263672s, input_fired_time: 9.9179258347, output_fired_time: 9.1764316559, propagate_forward_time: 513.3805541992, propagate_backward_time: 740.7312011719, weight_update_time: 1.4645659924, other_time: 12.6834106445
epoch time: 105.9521713257s, input_fired_time: 1.3122235537, output_fired_time: 0.0000000000, propagate_forward_time: 102.4198532104, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2200927734
validation[          , genome  6354] predictions:    5948/  10000 (59.48%), best:    6093/10000 (60.93%), error:     12366.31152, best error:     12040.73535 on epoch:    51, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 1285.8227539063s, input_fired_time: 10.1367330551, output_fired_time: 9.1883630753, propagate_forward_time: 512.8723144531, propagate_backward_time: 740.5509033203, weight_update_time: 1.4817779064, other_time: 13.0744628906
epoch time: 105.4506378174s, input_fired_time: 1.2724034786, output_fired_time: 0.0000000000, propagate_forward_time: 102.0050811768, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1731567383
validation[          , genome  6354] predictions:    5928/  10000 (59.28%), best:    6093/10000 (60.93%), error:     12480.02734, best error:     12040.73535 on epoch:    51, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 1284.0465087891s, input_fired_time: 9.9659566879, output_fired_time: 9.1744356155, propagate_forward_time: 512.6062011719, propagate_backward_time: 739.4501342773, weight_update_time: 1.4675737619, other_time: 12.8497924805
epoch time: 105.9871597290s, input_fired_time: 1.2937345505, output_fired_time: 0.0000000000, propagate_forward_time: 102.5264663696, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1669616699
validation[          , genome  6354] predictions:    5973/  10000 (59.73%), best:    6093/10000 (60.93%), error:     12200.30664, best error:     12040.73535 on epoch:    51, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 1286.9124755859s, input_fired_time: 10.1105966568, output_fired_time: 9.1980276108, propagate_forward_time: 513.7670288086, propagate_backward_time: 740.8854370117, weight_update_time: 1.4780348539, other_time: 12.9514160156
epoch time: 106.1090164185s, input_fired_time: 1.2835559845, output_fired_time: 0.0000000000, propagate_forward_time: 102.6882324219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1372299194
validation[          , genome  6354] predictions:    5987/  10000 (59.87%), best:    6093/10000 (60.93%), error:     12611.72559, best error:     12040.73535 on epoch:    51, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 1285.7156982422s, input_fired_time: 10.0902872086, output_fired_time: 9.1929340363, propagate_forward_time: 513.2250976563, propagate_backward_time: 740.2523193359, weight_update_time: 1.4764344692, other_time: 12.9549560547
epoch time: 105.8015365601s, input_fired_time: 1.2777913809, output_fired_time: 0.0000000000, propagate_forward_time: 102.3694381714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1543045044
validation[          , genome  6354] predictions:    5983/  10000 (59.83%), best:    6093/10000 (60.93%), error:     12242.32031, best error:     12040.73535 on epoch:    51, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 1279.8251953125s, input_fired_time: 10.2087030411, output_fired_time: 9.1648817062, propagate_forward_time: 509.6542663574, propagate_backward_time: 737.6034545898, weight_update_time: 1.4826842546, other_time: 13.1937866211
epoch time: 106.1624526978s, input_fired_time: 1.3005948067, output_fired_time: 0.0000000000, propagate_forward_time: 102.6731491089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1887054443
validation[          , genome  6354] predictions:    5991/  10000 (59.91%), best:    6093/10000 (60.93%), error:     12162.08887, best error:     12040.73535 on epoch:    51, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 1282.3557128906s, input_fired_time: 10.1476135254, output_fired_time: 9.1707601547, propagate_forward_time: 510.8972167969, propagate_backward_time: 739.0955810547, weight_update_time: 1.4776334763, other_time: 13.0445556641
epoch time: 106.5930023193s, input_fired_time: 1.3091166019, output_fired_time: 0.0000000000, propagate_forward_time: 103.0741195679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2097625732
validation[          , genome  6354] predictions:    6056/  10000 (60.56%), best:    6093/10000 (60.93%), error:     12160.10547, best error:     12040.73535 on epoch:    51, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 1289.8209228516s, input_fired_time: 9.9774818420, output_fired_time: 9.1953773499, propagate_forward_time: 515.1071166992, propagate_backward_time: 742.8192749023, weight_update_time: 1.4678000212, other_time: 12.7215576172
epoch time: 105.5963592529s, input_fired_time: 1.2819027901, output_fired_time: 0.0000000000, propagate_forward_time: 102.1785354614, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1359176636
validation[          , genome  6354] predictions:    6055/  10000 (60.55%), best:    6093/10000 (60.93%), error:     12072.83594, best error:     12040.73535 on epoch:    51, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 1285.5913085938s, input_fired_time: 10.1140127182, output_fired_time: 9.1872930527, propagate_forward_time: 512.9772949219, propagate_backward_time: 740.3449707031, weight_update_time: 1.4766309261, other_time: 12.9677734375
epoch time: 106.7598953247s, input_fired_time: 1.2835203409, output_fired_time: 0.0000000000, propagate_forward_time: 103.3403396606, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1360321045
validation[          , genome  6354] predictions:    5942/  10000 (59.42%), best:    6093/10000 (60.93%), error:     12567.91992, best error:     12040.73535 on epoch:    51, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 1282.7366943359s, input_fired_time: 9.9322481155, output_fired_time: 9.1468639374, propagate_forward_time: 511.5176391602, propagate_backward_time: 739.4702148438, weight_update_time: 1.4618619680, other_time: 12.6697387695
epoch time: 105.8388519287s, input_fired_time: 1.2875541449, output_fired_time: 0.0000000000, propagate_forward_time: 102.4143753052, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1369247437
validation[          , genome  6354] predictions:    6036/  10000 (60.36%), best:    6093/10000 (60.93%), error:     12216.53223, best error:     12040.73535 on epoch:    51, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 1290.1142578125s, input_fired_time: 10.0560626984, output_fired_time: 9.1941270828, propagate_forward_time: 514.8761596680, propagate_backward_time: 743.1727905273, weight_update_time: 1.4726475477, other_time: 12.8151855469
epoch time: 106.0711746216s, input_fired_time: 1.2968142033, output_fired_time: 0.0000000000, propagate_forward_time: 102.6207275391, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1536331177
validation[          , genome  6354] predictions:    6058/  10000 (60.58%), best:    6093/10000 (60.93%), error:     12297.64160, best error:     12040.73535 on epoch:    51, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 1287.8955078125s, input_fired_time: 10.1629590988, output_fired_time: 9.2041521072, propagate_forward_time: 514.1134643555, propagate_backward_time: 741.4340820313, weight_update_time: 1.4783917665, other_time: 12.9808959961
epoch time: 105.0579986572s, input_fired_time: 1.2788457870, output_fired_time: 0.0000000000, propagate_forward_time: 101.6463470459, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1328048706
validation[          , genome  6354] predictions:    5949/  10000 (59.49%), best:    6093/10000 (60.93%), error:     12465.80176, best error:     12040.73535 on epoch:    51, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 1281.2867431641s, input_fired_time: 9.9266681671, output_fired_time: 9.1534414291, propagate_forward_time: 510.8677673340, propagate_backward_time: 738.6903686523, weight_update_time: 1.4611953497, other_time: 12.6484985352
epoch time: 105.5190505981s, input_fired_time: 1.2919607162, output_fired_time: 0.0000000000, propagate_forward_time: 102.0432739258, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1838150024
validation[          , genome  6354] predictions:    6009/  10000 (60.09%), best:    6093/10000 (60.93%), error:     12376.09766, best error:     12040.73535 on epoch:    51, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 1280.6130371094s, input_fired_time: 10.1805877686, output_fired_time: 9.1606121063, propagate_forward_time: 510.0459594727, propagate_backward_time: 738.1291503906, weight_update_time: 1.4772208929, other_time: 13.0967407227
epoch time: 106.3884735107s, input_fired_time: 1.2987798452, output_fired_time: 0.0000000000, propagate_forward_time: 102.9204788208, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1692123413
validation[          , genome  6354] predictions:    6085/  10000 (60.85%), best:    6085/10000 (60.85%), error:     12012.24707, best error:     12012.24707 on epoch:    66, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 1286.5858154297s, input_fired_time: 10.1617727280, output_fired_time: 9.1722822189, propagate_forward_time: 512.9569091797, propagate_backward_time: 741.3005981445, weight_update_time: 1.4767293930, other_time: 12.9943237305
epoch time: 106.2471313477s, input_fired_time: 1.3292679787, output_fired_time: 0.0000000000, propagate_forward_time: 102.6742172241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2436447144
validation[          , genome  6354] predictions:    6105/  10000 (61.05%), best:    6105/10000 (61.05%), error:     11985.94531, best error:     11985.94531 on epoch:    67, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 1288.1801757813s, input_fired_time: 10.1253042221, output_fired_time: 9.2133302689, propagate_forward_time: 513.8922729492, propagate_backward_time: 742.0195312500, weight_update_time: 1.4757622480, other_time: 12.9297485352
epoch time: 106.2761459351s, input_fired_time: 1.2874503136, output_fired_time: 0.0000000000, propagate_forward_time: 102.8529434204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1357498169
validation[          , genome  6354] predictions:    6058/  10000 (60.58%), best:    6105/10000 (61.05%), error:     12132.74609, best error:     11985.94531 on epoch:    67, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 1284.0334472656s, input_fired_time: 10.0444288254, output_fired_time: 9.1768913269, propagate_forward_time: 511.8027648926, propagate_backward_time: 740.1340942383, weight_update_time: 1.4711892605, other_time: 12.8753051758
epoch time: 106.8075561523s, input_fired_time: 1.2889091969, output_fired_time: 0.0000000000, propagate_forward_time: 103.3817977905, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1368484497
validation[          , genome  6354] predictions:    6080/  10000 (60.80%), best:    6080/10000 (60.80%), error:     11936.74316, best error:     11936.74316 on epoch:    69, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 1284.1336669922s, input_fired_time: 10.0510425568, output_fired_time: 9.1872444153, propagate_forward_time: 511.6073913574, propagate_backward_time: 740.4089965820, weight_update_time: 1.4708807468, other_time: 12.8789672852
epoch time: 105.1466979980s, input_fired_time: 1.2871918678, output_fired_time: 0.0000000000, propagate_forward_time: 101.7220840454, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1374206543
validation[          , genome  6354] predictions:    6161/  10000 (61.61%), best:    6080/10000 (60.80%), error:     11973.28906, best error:     11936.74316 on epoch:    69, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 1282.8397216797s, input_fired_time: 10.1241950989, output_fired_time: 9.1633501053, propagate_forward_time: 511.2960510254, propagate_backward_time: 739.2736206055, weight_update_time: 1.4734741449, other_time: 12.9826049805
epoch time: 105.4563217163s, input_fired_time: 1.2858499289, output_fired_time: 0.0000000000, propagate_forward_time: 102.0315780640, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1388931274
validation[          , genome  6354] predictions:    6143/  10000 (61.43%), best:    6080/10000 (60.80%), error:     12062.67871, best error:     11936.74316 on epoch:    69, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 1315.6582031250s, input_fired_time: 10.1291828156, output_fired_time: 9.4137945175, propagate_forward_time: 527.8124389648, propagate_backward_time: 755.2859497070, weight_update_time: 1.4819308519, other_time: 13.0168457031
epoch time: 105.3296813965s, input_fired_time: 1.3017628193, output_fired_time: 0.0000000000, propagate_forward_time: 101.8686599731, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1592559814
validation[          , genome  6354] predictions:    6113/  10000 (61.13%), best:    6080/10000 (60.80%), error:     12010.02637, best error:     11936.74316 on epoch:    69, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 1308.9038085938s, input_fired_time: 10.2265634537, output_fired_time: 9.3524179459, propagate_forward_time: 523.9258422852, propagate_backward_time: 752.1546020508, weight_update_time: 1.4884309769, other_time: 13.2443847656
epoch time: 106.6285934448s, input_fired_time: 1.3339633942, output_fired_time: 0.0000000000, propagate_forward_time: 103.0641326904, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2304992676
validation[          , genome  6354] predictions:    6112/  10000 (61.12%), best:    6080/10000 (60.80%), error:     12004.40820, best error:     11936.74316 on epoch:    69, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 1304.0921630859s, input_fired_time: 10.3402948380, output_fired_time: 9.3788604736, propagate_forward_time: 521.7351074219, propagate_backward_time: 749.0971679688, weight_update_time: 1.4988237619, other_time: 13.5406494141
epoch time: 105.4683151245s, input_fired_time: 1.3033584356, output_fired_time: 0.0000000000, propagate_forward_time: 102.0135726929, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1513824463
validation[          , genome  6354] predictions:    6110/  10000 (61.10%), best:    6080/10000 (60.80%), error:     12147.62695, best error:     11936.74316 on epoch:    69, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 1307.4218750000s, input_fired_time: 10.1866149902, output_fired_time: 9.3538970947, propagate_forward_time: 523.7681884766, propagate_backward_time: 750.9923095703, weight_update_time: 1.4825614691, other_time: 13.1208496094
epoch time: 106.1543045044s, input_fired_time: 1.3343129158, output_fired_time: 0.0000000000, propagate_forward_time: 102.5845794678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2354125977
validation[          , genome  6354] predictions:    6094/  10000 (60.94%), best:    6080/10000 (60.80%), error:     12034.67090, best error:     11936.74316 on epoch:    69, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 1290.7769775391s, input_fired_time: 10.2697372437, output_fired_time: 9.2287902832, propagate_forward_time: 515.5166015625, propagate_backward_time: 742.6144409180, weight_update_time: 1.4867516756, other_time: 13.1473999023
epoch time: 105.1291198730s, input_fired_time: 1.3309968710, output_fired_time: 0.0000000000, propagate_forward_time: 101.5505523682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2475738525
validation[          , genome  6354] predictions:    6159/  10000 (61.59%), best:    6080/10000 (60.80%), error:     12052.58008, best error:     11936.74316 on epoch:    69, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 1290.3648681641s, input_fired_time: 10.1556234360, output_fired_time: 9.2255210876, propagate_forward_time: 515.4619140625, propagate_backward_time: 742.5122680664, weight_update_time: 1.4787688255, other_time: 13.0095825195
epoch time: 106.9455871582s, input_fired_time: 1.3091325760, output_fired_time: 0.0000000000, propagate_forward_time: 103.4183654785, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2180862427
validation[          , genome  6354] predictions:    6128/  10000 (61.28%), best:    6128/10000 (61.28%), error:     11915.48730, best error:     11915.48730 on epoch:    77, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 1286.8137207031s, input_fired_time: 10.2215042114, output_fired_time: 9.2165718079, propagate_forward_time: 513.2293701172, propagate_backward_time: 740.9728393555, weight_update_time: 1.4837393761, other_time: 13.1734008789
epoch time: 105.2655944824s, input_fired_time: 1.2948838472, output_fired_time: 0.0000000000, propagate_forward_time: 101.8350143433, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1356964111
validation[          , genome  6354] predictions:    6076/  10000 (60.76%), best:    6128/10000 (61.28%), error:     12062.65820, best error:     11915.48730 on epoch:    77, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 1285.7156982422s, input_fired_time: 10.0431194305, output_fired_time: 9.1839075089, propagate_forward_time: 512.9314575195, propagate_backward_time: 740.7653198242, weight_update_time: 1.4686799049, other_time: 12.7918701172
epoch time: 105.0931930542s, input_fired_time: 1.2774519920, output_fired_time: 0.0000000000, propagate_forward_time: 101.3969879150, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4187545776
validation[          , genome  6354] predictions:    6031/  10000 (60.31%), best:    6128/10000 (61.28%), error:     12411.29297, best error:     11915.48730 on epoch:    77, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1279.8752441406s, input_fired_time: 10.1507434845, output_fired_time: 9.1943988800, propagate_forward_time: 509.0327758789, propagate_backward_time: 737.3286743164, weight_update_time: 1.4803166389, other_time: 14.1685791016
epoch time: 105.2676086426s, input_fired_time: 1.2988275290, output_fired_time: 0.0000000000, propagate_forward_time: 101.8227996826, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1459808350
validation[          , genome  6354] predictions:    6136/  10000 (61.36%), best:    6128/10000 (61.28%), error:     12084.22559, best error:     11915.48730 on epoch:    77, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1283.0026855469s, input_fired_time: 9.9928512573, output_fired_time: 9.1675567627, propagate_forward_time: 511.5744628906, propagate_backward_time: 739.5153808594, weight_update_time: 1.4615901709, other_time: 12.7524414063
epoch time: 106.0585327148s, input_fired_time: 1.2928837538, output_fired_time: 0.0000000000, propagate_forward_time: 102.5643692017, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2012786865
validation[          , genome  6354] predictions:    6182/  10000 (61.82%), best:    6128/10000 (61.28%), error:     12014.16113, best error:     11915.48730 on epoch:    77, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1281.8258056641s, input_fired_time: 10.2392349243, output_fired_time: 9.2063541412, propagate_forward_time: 510.0353698730, propagate_backward_time: 738.4733886719, weight_update_time: 1.4826046228, other_time: 13.8714599609
epoch time: 107.3731231689s, input_fired_time: 1.3385241032, output_fired_time: 0.0000000000, propagate_forward_time: 103.6876983643, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3469009399
validation[          , genome  6354] predictions:    6077/  10000 (60.77%), best:    6128/10000 (61.28%), error:     12148.48633, best error:     11915.48730 on epoch:    77, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1291.6971435547s, input_fired_time: 10.2207288742, output_fired_time: 9.2574501038, propagate_forward_time: 514.3950805664, propagate_backward_time: 744.0839843750, weight_update_time: 1.4856371880, other_time: 13.7399291992
epoch time: 106.5703430176s, input_fired_time: 1.3494560719, output_fired_time: 0.0000000000, propagate_forward_time: 102.9242401123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2966461182
validation[          , genome  6354] predictions:    6179/  10000 (61.79%), best:    6179/10000 (61.79%), error:     11725.97168, best error:     11725.97168 on epoch:    83, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1292.4228515625s, input_fired_time: 10.0571613312, output_fired_time: 9.2445039749, propagate_forward_time: 514.4533081055, propagate_backward_time: 745.0383911133, weight_update_time: 1.4742619991, other_time: 13.6295166016
epoch time: 106.0937728882s, input_fired_time: 1.3012422323, output_fired_time: 0.0000000000, propagate_forward_time: 102.5721130371, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2204208374
validation[          , genome  6354] predictions:    6156/  10000 (61.56%), best:    6179/10000 (61.79%), error:     11949.29297, best error:     11725.97168 on epoch:    83, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 1281.7215576172s, input_fired_time: 10.2354125977, output_fired_time: 9.1869974136, propagate_forward_time: 509.9838867188, propagate_backward_time: 738.5660400391, weight_update_time: 1.4820286036, other_time: 13.7491455078
epoch time: 107.3834686279s, input_fired_time: 1.3297134638, output_fired_time: 0.0000000000, propagate_forward_time: 103.7727890015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2809677124
validation[          , genome  6354] predictions:    6137/  10000 (61.37%), best:    6179/10000 (61.79%), error:     11985.48633, best error:     11725.97168 on epoch:    83, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 1289.0677490234s, input_fired_time: 10.2864656448, output_fired_time: 9.2179861069, propagate_forward_time: 514.2645263672, propagate_backward_time: 741.9243164063, weight_update_time: 1.4860535860, other_time: 13.3743896484
epoch time: 105.6969909668s, input_fired_time: 1.3517708778, output_fired_time: 0.0000000000, propagate_forward_time: 102.0419235229, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3032989502
validation[          , genome  6354] predictions:    6169/  10000 (61.69%), best:    6179/10000 (61.79%), error:     11946.95313, best error:     11725.97168 on epoch:    83, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1283.9577636719s, input_fired_time: 10.0680894852, output_fired_time: 9.1617689133, propagate_forward_time: 511.9733276367, propagate_backward_time: 739.9191894531, weight_update_time: 1.4657136202, other_time: 12.8353881836
epoch time: 106.8601608276s, input_fired_time: 1.3435362577, output_fired_time: 0.0000000000, propagate_forward_time: 103.1063690186, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4102554321
validation[          , genome  6354] predictions:    6177/  10000 (61.77%), best:    6179/10000 (61.79%), error:     11941.67969, best error:     11725.97168 on epoch:    83, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1284.0697021484s, input_fired_time: 10.0069942474, output_fired_time: 9.1667003632, propagate_forward_time: 511.5507202148, propagate_backward_time: 740.3964233398, weight_update_time: 1.4628050327, other_time: 12.9488525391
epoch time: 106.4788208008s, input_fired_time: 1.3445162773, output_fired_time: 0.0000000000, propagate_forward_time: 102.9088592529, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2254486084
validation[          , genome  6354] predictions:    6210/  10000 (62.10%), best:    6179/10000 (61.79%), error:     11855.91699, best error:     11725.97168 on epoch:    83, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1288.3851318359s, input_fired_time: 10.3221979141, output_fired_time: 9.1962137222, propagate_forward_time: 513.5563354492, propagate_backward_time: 742.0471191406, weight_update_time: 1.4858611822, other_time: 13.2633666992
epoch time: 105.4649581909s, input_fired_time: 1.3495475054, output_fired_time: 0.0000000000, propagate_forward_time: 101.8560256958, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2593841553
validation[          , genome  6354] predictions:    6186/  10000 (61.86%), best:    6179/10000 (61.79%), error:     11883.47168, best error:     11725.97168 on epoch:    83, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1291.3311767578s, input_fired_time: 10.1693754196, output_fired_time: 9.2426614761, propagate_forward_time: 513.6019897461, propagate_backward_time: 744.7795410156, weight_update_time: 1.4810048342, other_time: 13.5375366211
epoch time: 106.8280105591s, input_fired_time: 1.3473114967, output_fired_time: 0.0000000000, propagate_forward_time: 103.1663894653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3143081665
validation[          , genome  6354] predictions:    6141/  10000 (61.41%), best:    6179/10000 (61.79%), error:     11912.59180, best error:     11725.97168 on epoch:    83, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1294.3725585938s, input_fired_time: 10.3009204865, output_fired_time: 9.2529973984, propagate_forward_time: 515.6242065430, propagate_backward_time: 745.4326782227, weight_update_time: 1.4921309948, other_time: 13.7617187500
epoch time: 106.3186950684s, input_fired_time: 1.3322439194, output_fired_time: 0.0000000000, propagate_forward_time: 102.4842453003, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5022048950
validation[          , genome  6354] predictions:    6167/  10000 (61.67%), best:    6179/10000 (61.79%), error:     11981.57031, best error:     11725.97168 on epoch:    83, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1286.3107910156s, input_fired_time: 10.0355272293, output_fired_time: 9.2093877792, propagate_forward_time: 512.2863159180, propagate_backward_time: 741.5454711914, weight_update_time: 1.4701637030, other_time: 13.2341308594
epoch time: 105.3913803101s, input_fired_time: 1.2981685400, output_fired_time: 0.0000000000, propagate_forward_time: 101.0176239014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0755844116
validation[          , genome  6354] predictions:    6160/  10000 (61.60%), best:    6179/10000 (61.79%), error:     11870.67480, best error:     11725.97168 on epoch:    83, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 1290.1495361328s, input_fired_time: 10.1812820435, output_fired_time: 9.2301120758, propagate_forward_time: 513.7385253906, propagate_backward_time: 743.5140380859, weight_update_time: 1.4808984995, other_time: 13.4855957031
epoch time: 106.7341842651s, input_fired_time: 1.3671269417, output_fired_time: 0.0000000000, propagate_forward_time: 103.0985183716, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2685394287
validation[          , genome  6354] predictions:    6206/  10000 (62.06%), best:    6179/10000 (61.79%), error:     11831.20508, best error:     11725.97168 on epoch:    83, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1285.1696777344s, input_fired_time: 10.1000547409, output_fired_time: 9.1902065277, propagate_forward_time: 512.2896118164, propagate_backward_time: 740.5806884766, weight_update_time: 1.4708323479, other_time: 13.0090942383
epoch time: 104.6313858032s, input_fired_time: 1.2764008045, output_fired_time: 0.0000000000, propagate_forward_time: 100.9158630371, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4391250610
validation[          , genome  6354] predictions:    6157/  10000 (61.57%), best:    6179/10000 (61.79%), error:     11878.13184, best error:     11725.97168 on epoch:    83, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1288.3410644531s, input_fired_time: 10.3019256592, output_fired_time: 9.2207841873, propagate_forward_time: 514.2543334961, propagate_backward_time: 741.4130859375, weight_update_time: 1.4846105576, other_time: 13.1509399414
epoch time: 104.7670822144s, input_fired_time: 1.2736878395, output_fired_time: 0.0000000000, propagate_forward_time: 100.8743515015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6190414429
validation[          , genome  6354] predictions:    6198/  10000 (61.98%), best:    6179/10000 (61.79%), error:     11861.52051, best error:     11725.97168 on epoch:    83, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1289.4312744141s, input_fired_time: 10.1198806763, output_fired_time: 9.2380361557, propagate_forward_time: 514.3212890625, propagate_backward_time: 742.4465942383, weight_update_time: 1.4730026722, other_time: 13.3054809570
epoch time: 106.3968811035s, input_fired_time: 1.3444284201, output_fired_time: 0.0000000000, propagate_forward_time: 102.8136291504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2388229370
validation[          , genome  6354] predictions:    6182/  10000 (61.82%), best:    6179/10000 (61.79%), error:     11858.95996, best error:     11725.97168 on epoch:    83, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1279.1522216797s, input_fired_time: 10.2461185455, output_fired_time: 9.2273569107, propagate_forward_time: 506.8596801758, propagate_backward_time: 736.4805297852, weight_update_time: 1.4929060936, other_time: 16.3385009766
epoch time: 103.7020797729s, input_fired_time: 1.2695980072, output_fired_time: 0.0000000000, propagate_forward_time: 99.4815292358, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9509506226
validation[          , genome  6354] predictions:    6193/  10000 (61.93%), best:    6179/10000 (61.79%), error:     11883.76465, best error:     11725.97168 on epoch:    83, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1281.9417724609s, input_fired_time: 10.0313768387, output_fired_time: 9.1961212158, propagate_forward_time: 509.7005615234, propagate_backward_time: 738.9509887695, weight_update_time: 1.4861960411, other_time: 14.0626831055
epoch time: 104.8825988770s, input_fired_time: 1.2912501097, output_fired_time: 0.0000000000, propagate_forward_time: 100.8409118652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7504348755
validation[          , genome  6354] predictions:    6204/  10000 (62.04%), best:    6179/10000 (61.79%), error:     11884.59570, best error:     11725.97168 on epoch:    83, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1283.0461425781s, input_fired_time: 10.2525014877, output_fired_time: 9.1909837723, propagate_forward_time: 510.7519836426, propagate_backward_time: 739.5073852539, weight_update_time: 1.4808667898, other_time: 13.3433227539
epoch time: 104.9309844971s, input_fired_time: 1.3173978329, output_fired_time: 0.0000000000, propagate_forward_time: 100.7273483276, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8862380981
validation[          , genome  6354] predictions:    6220/  10000 (62.20%), best:    6179/10000 (61.79%), error:     11893.45313, best error:     11725.97168 on epoch:    83, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1287.9497070313s, input_fired_time: 10.2995367050, output_fired_time: 9.2586908340, propagate_forward_time: 511.4196777344, propagate_backward_time: 742.0025634766, weight_update_time: 1.4909843206, other_time: 14.9692382813
epoch time: 104.7336807251s, input_fired_time: 1.3013913631, output_fired_time: 0.0000000000, propagate_forward_time: 99.9752883911, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4570007324
validation[          , genome  6354] predictions:    6162/  10000 (61.62%), best:    6179/10000 (61.79%), error:     11951.95215, best error:     11725.97168 on epoch:    83, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 524.5445556641s, input_fired_time: 6.5066342354, output_fired_time: 0.0000000000, propagate_forward_time: 503.8851623535, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 14.1527404785
best training[          , genome  6354] predictions:   40611/  50000 (81.22%), best:    6179/10000 (61.79%), error:     29177.08789, best error:     11725.97168 on epoch:    83, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 105.9792327881s, input_fired_time: 1.3076376915, output_fired_time: 0.0000000000, propagate_forward_time: 102.4755630493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1960296631
   testing[          , genome  6354] predictions:    6179/  10000 (61.79%), best:    6179/10000 (61.79%), error:     11725.97168, best error:     11725.97168 on epoch:    83, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
21:06:43 (12944): called boinc_finish(0)

</stderr_txt>
]]>