Nameexact_genome_1535714404_162_11868_2
Workunit3938746
Created1 Mar 2019, 17:55:09 UTC
Sent1 Mar 2019, 18:01:53 UTC
Report deadline17 Aug 2019, 17:15:11 UTC
Received8 Mar 2019, 10:51:31 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID31505
Run time3 days 0 hours 17 min 4 sec
CPU time2 days 21 hours 0 min 17 sec
Validate stateInitial
Credit0.00
Device peak FLOPS4.23 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size586.27 MB
Peak swap size584.00 MB
Peak disk usage16.02 MB

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
or channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2421.15234375s, input_fired_time: 33.1453056335449, output_fired_time: 16.6488971710205, propagate_forward_time: 924.445251464844, propagate_backward_time: 1430.05908203125, weight_update_time: 0.8499675989151, other_time: 16.853759765625
epoch time: 189.333908081055s, input_fired_time: 2.53613138198853, output_fired_time: 0, propagate_forward_time: 183.86930847168, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.928466796875
validation[          , genome 11868] predictions:    6449/  10000 (64.49%), best:    6631/10000 (66.31%), error:     10337.39551, best error:     10095.10547 on epoch:    51, epoch:   52/100, mu: 0.9325550199, learning_rate: 0.0000085302, weight_decay: 0.0000376068

epoch time: 2428.9931640625s, input_fired_time: 33.2839317322, output_fired_time: 16.6985225677, propagate_forward_time: 928.5224609375, propagate_backward_time: 1433.6031494141, weight_update_time: 0.9004767537, other_time: 16.8851318359
epoch time: 203.3228302002s, input_fired_time: 2.6930272579, output_fired_time: 0.0000000000, propagate_forward_time: 197.5419006348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0879058838
validation[          , genome 11868] predictions:    6588/  10000 (65.88%), best:    6631/10000 (66.31%), error:     10215.60938, best error:     10095.10547 on epoch:    51, epoch:   53/100, mu: 0.9348228574, learning_rate: 0.0000079453, weight_decay: 0.0000357768

epoch time: 2499.2971191406s, input_fired_time: 33.9554138184, output_fired_time: 17.8785438538, propagate_forward_time: 950.6257934570, propagate_backward_time: 1479.3374023438, weight_update_time: 0.9433974028, other_time: 17.5000000000
epoch time: 237.3586273193s, input_fired_time: 3.1567082405, output_fired_time: 0.0000000000, propagate_forward_time: 230.3404541016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8614654541
validation[          , genome 11868] predictions:    6511/  10000 (65.11%), best:    6631/10000 (66.31%), error:     10376.86816, best error:     10095.10547 on epoch:    51, epoch:   54/100, mu: 0.9370011687, learning_rate: 0.0000074005, weight_decay: 0.0000340358

epoch time: 2613.7973632813s, input_fired_time: 35.1124496460, output_fired_time: 18.1559772491, propagate_forward_time: 989.8562011719, propagate_backward_time: 1551.3518066406, weight_update_time: 1.0954957008, other_time: 19.3208007813
epoch time: 187.4641723633s, input_fired_time: 2.5281996727, output_fired_time: 0.0000000000, propagate_forward_time: 181.8847808838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0511932373
validation[          , genome 11868] predictions:    6629/  10000 (66.29%), best:    6631/10000 (66.31%), error:     10122.54688, best error:     10095.10547 on epoch:    51, epoch:   55/100, mu: 0.9390934706, learning_rate: 0.0000068931, weight_decay: 0.0000323796

epoch time: 2404.7275390625s, input_fired_time: 32.8958549500, output_fired_time: 16.8607387543, propagate_forward_time: 920.1895751953, propagate_backward_time: 1417.6531982422, weight_update_time: 0.8786379099, other_time: 17.1281738281
epoch time: 190.3538970947s, input_fired_time: 2.6355514526, output_fired_time: 0.0000000000, propagate_forward_time: 184.4962615967, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2220916748
validation[          , genome 11868] predictions:    6631/  10000 (66.31%), best:    6631/10000 (66.31%), error:     10118.88770, best error:     10095.10547 on epoch:    51, epoch:   56/100, mu: 0.9411031604, learning_rate: 0.0000064204, weight_decay: 0.0000308039

epoch time: 2371.1164550781s, input_fired_time: 32.9717178345, output_fired_time: 17.1107749939, propagate_forward_time: 905.0009155273, propagate_backward_time: 1398.4283447266, weight_update_time: 0.8725450635, other_time: 17.6046142578
epoch time: 189.1196899414s, input_fired_time: 2.5615282059, output_fired_time: 0.0000000000, propagate_forward_time: 183.3552703857, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2028961182
validation[          , genome 11868] predictions:    6606/  10000 (66.06%), best:    6631/10000 (66.31%), error:     10148.81250, best error:     10095.10547 on epoch:    51, epoch:   57/100, mu: 0.9430335164, learning_rate: 0.0000059802, weight_decay: 0.0000293049

epoch time: 2383.4633789063s, input_fired_time: 33.1107177734, output_fired_time: 16.7153415680, propagate_forward_time: 908.2664794922, propagate_backward_time: 1407.5584716797, weight_update_time: 0.8854312301, other_time: 17.8122558594
epoch time: 183.5256805420s, input_fired_time: 2.5025069714, output_fired_time: 0.0000000000, propagate_forward_time: 178.0509338379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9722442627
validation[          , genome 11868] predictions:    6584/  10000 (65.84%), best:    6631/10000 (66.31%), error:     10208.88574, best error:     10095.10547 on epoch:    51, epoch:   58/100, mu: 0.9448876381, learning_rate: 0.0000055701, weight_decay: 0.0000278789

epoch time: 2376.4672851563s, input_fired_time: 32.7891159058, output_fired_time: 17.3134269714, propagate_forward_time: 903.8217773438, propagate_backward_time: 1404.7363281250, weight_update_time: 0.8741207123, other_time: 17.8066406250
epoch time: 183.4422302246s, input_fired_time: 2.5077278614, output_fired_time: 0.0000000000, propagate_forward_time: 177.9566345215, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9778747559
validation[          , genome 11868] predictions:    6611/  10000 (66.11%), best:    6631/10000 (66.31%), error:     10146.59277, best error:     10095.10547 on epoch:    51, epoch:   59/100, mu: 0.9466685653, learning_rate: 0.0000051882, weight_decay: 0.0000265223

epoch time: 2366.9938964844s, input_fired_time: 32.8945846558, output_fired_time: 16.4494094849, propagate_forward_time: 899.9121093750, propagate_backward_time: 1398.9100341797, weight_update_time: 0.8642638326, other_time: 18.8277587891
epoch time: 191.8900146484s, input_fired_time: 2.5631201267, output_fired_time: 0.0000000000, propagate_forward_time: 186.1694335938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1574554443
validation[          , genome 11868] predictions:    6605/  10000 (66.05%), best:    6605/10000 (66.05%), error:     10047.04102, best error:     10047.04102 on epoch:    60, epoch:   60/100, mu: 0.9483792186, learning_rate: 0.0000048324, weight_decay: 0.0000252317

epoch time: 2399.4716796875s, input_fired_time: 33.7699165344, output_fired_time: 16.8974971771, propagate_forward_time: 915.9728393555, propagate_backward_time: 1415.4516601563, weight_update_time: 0.8666466475, other_time: 17.3796386719
epoch time: 220.4869384766s, input_fired_time: 2.9217109680, output_fired_time: 0.0000000000, propagate_forward_time: 213.9851989746, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5800323486
validation[          , genome 11868] predictions:    6654/  10000 (66.54%), best:    6605/10000 (66.05%), error:     10088.74121, best error:     10047.04102 on epoch:    60, epoch:   61/100, mu: 0.9500223398, learning_rate: 0.0000045011, weight_decay: 0.0000240038

epoch time: 2743.0119628906s, input_fired_time: 38.1725883484, output_fired_time: 19.2194404602, propagate_forward_time: 1049.8316650391, propagate_backward_time: 1615.3555908203, weight_update_time: 1.0464470387, other_time: 20.4326171875
epoch time: 221.7290496826s, input_fired_time: 3.0403051376, output_fired_time: 0.0000000000, propagate_forward_time: 215.1382141113, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5505371094
validation[          , genome 11868] predictions:    6587/  10000 (65.87%), best:    6605/10000 (66.05%), error:     10155.57813, best error:     10047.04102 on epoch:    60, epoch:   62/100, mu: 0.9516006112, learning_rate: 0.0000041925, weight_decay: 0.0000228358

epoch time: 2645.9892578125s, input_fired_time: 36.2575645447, output_fired_time: 18.4056663513, propagate_forward_time: 1010.6135253906, propagate_backward_time: 1560.9174804688, weight_update_time: 0.9529526830, other_time: 19.7949218750
epoch time: 211.0741271973s, input_fired_time: 2.9982151985, output_fired_time: 0.0000000000, propagate_forward_time: 204.3138122559, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7621002197
validation[          , genome 11868] predictions:    6645/  10000 (66.45%), best:    6605/10000 (66.05%), error:     10059.67480, best error:     10047.04102 on epoch:    60, epoch:   63/100, mu: 0.9531165361, learning_rate: 0.0000039050, weight_decay: 0.0000217245

epoch time: 2541.9147949219s, input_fired_time: 34.8581886292, output_fired_time: 17.8229713440, propagate_forward_time: 974.6234741211, propagate_backward_time: 1496.1516113281, weight_update_time: 0.8882919550, other_time: 18.4584960938
epoch time: 183.1011810303s, input_fired_time: 2.4614052773, output_fired_time: 0.0000000000, propagate_forward_time: 177.7966308594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8431396484
validation[          , genome 11868] predictions:    6673/  10000 (66.73%), best:    6673/10000 (66.73%), error:      9943.96387, best error:      9943.96387 on epoch:    64, epoch:   64/100, mu: 0.9545726180, learning_rate: 0.0000036372, weight_decay: 0.0000206674

epoch time: 2316.9260253906s, input_fired_time: 31.7602920532, output_fired_time: 16.0195255280, propagate_forward_time: 884.8341674805, propagate_backward_time: 1367.7617187500, weight_update_time: 0.8335739374, other_time: 16.5502929688
epoch time: 183.6793975830s, input_fired_time: 2.4858798981, output_fired_time: 0.0000000000, propagate_forward_time: 178.2633819580, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9301300049
validation[          , genome 11868] predictions:    6619/  10000 (66.19%), best:    6673/10000 (66.73%), error:      9980.52246, best error:      9943.96387 on epoch:    64, epoch:   65/100, mu: 0.9559712410, learning_rate: 0.0000033878, weight_decay: 0.0000196617

epoch time: 2335.8608398438s, input_fired_time: 32.1667213440, output_fired_time: 16.2739124298, propagate_forward_time: 893.4679565430, propagate_backward_time: 1377.2519531250, weight_update_time: 0.9111588001, other_time: 16.7001953125
epoch time: 182.0989685059s, input_fired_time: 2.4366893768, output_fired_time: 0.0000000000, propagate_forward_time: 176.8541412354, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8081359863
validation[          , genome 11868] predictions:    6657/  10000 (66.57%), best:    6673/10000 (66.73%), error:     10047.47949, best error:      9943.96387 on epoch:    64, epoch:   66/100, mu: 0.9573146105, learning_rate: 0.0000031555, weight_decay: 0.0000187049

epoch time: 2324.7631835938s, input_fired_time: 32.3135337830, output_fired_time: 16.0914745331, propagate_forward_time: 889.5094604492, propagate_backward_time: 1370.4688720703, weight_update_time: 0.8316577077, other_time: 16.3797607422
epoch time: 180.0274658203s, input_fired_time: 2.4106116295, output_fired_time: 0.0000000000, propagate_forward_time: 174.7659759521, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8508758545
validation[          , genome 11868] predictions:    6688/  10000 (66.88%), best:    6673/10000 (66.73%), error:      9971.60352, best error:      9943.96387 on epoch:    64, epoch:   67/100, mu: 0.9586049914, learning_rate: 0.0000029392, weight_decay: 0.0000177947

epoch time: 2321.4885253906s, input_fired_time: 31.9511680603, output_fired_time: 16.1874046326, propagate_forward_time: 887.7963867188, propagate_backward_time: 1368.7878417969, weight_update_time: 0.8685127497, other_time: 16.7656250000
epoch time: 181.6464843750s, input_fired_time: 2.4552223682, output_fired_time: 0.0000000000, propagate_forward_time: 176.3591308594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8321380615
validation[          , genome 11868] predictions:    6654/  10000 (66.54%), best:    6673/10000 (66.73%), error:      9991.70508, best error:      9943.96387 on epoch:    64, epoch:   68/100, mu: 0.9598444104, learning_rate: 0.0000027376, weight_decay: 0.0000169288

epoch time: 2312.9902343750s, input_fired_time: 31.6066799164, output_fired_time: 16.0447292328, propagate_forward_time: 881.2710571289, propagate_backward_time: 1367.6668701172, weight_update_time: 0.8465351462, other_time: 16.4010009766
epoch time: 181.5010223389s, input_fired_time: 2.4517598152, output_fired_time: 0.0000000000, propagate_forward_time: 176.2414398193, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8078155518
validation[          , genome 11868] predictions:    6671/  10000 (66.71%), best:    6673/10000 (66.73%), error:      9956.88379, best error:      9943.96387 on epoch:    64, epoch:   69/100, mu: 0.9610348940, learning_rate: 0.0000025499, weight_decay: 0.0000161050

epoch time: 2324.3896484375s, input_fired_time: 31.7894229889, output_fired_time: 16.0520915985, propagate_forward_time: 887.4548339844, propagate_backward_time: 1372.5321044922, weight_update_time: 0.8312618732, other_time: 16.5614013672
epoch time: 186.2665100098s, input_fired_time: 2.4917085171, output_fired_time: 0.0000000000, propagate_forward_time: 180.8909454346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8838500977
validation[          , genome 11868] predictions:    6660/  10000 (66.60%), best:    6673/10000 (66.73%), error:      9982.65820, best error:      9943.96387 on epoch:    64, epoch:   70/100, mu: 0.9621784091, learning_rate: 0.0000023751, weight_decay: 0.0000153213

epoch time: 2329.9448242188s, input_fired_time: 31.9044742584, output_fired_time: 15.9166650772, propagate_forward_time: 895.0779418945, propagate_backward_time: 1370.4713134766, weight_update_time: 0.8526588678, other_time: 16.5743408203
epoch time: 182.7304534912s, input_fired_time: 2.5094270706, output_fired_time: 0.0000000000, propagate_forward_time: 177.2384643555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9825592041
validation[          , genome 11868] predictions:    6692/  10000 (66.92%), best:    6692/10000 (66.92%), error:      9923.40234, best error:      9923.40234 on epoch:    71, epoch:   71/100, mu: 0.9632767439, learning_rate: 0.0000022122, weight_decay: 0.0000145757

epoch time: 2294.7983398438s, input_fired_time: 31.5626277924, output_fired_time: 15.8836727142, propagate_forward_time: 876.3377075195, propagate_backward_time: 1354.9055175781, weight_update_time: 0.8131932020, other_time: 16.1086425781
epoch time: 190.1037445068s, input_fired_time: 2.5946862698, output_fired_time: 0.0000000000, propagate_forward_time: 184.0206909180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4883728027
validation[          , genome 11868] predictions:    6636/  10000 (66.36%), best:    6692/10000 (66.92%), error:     10079.21387, best error:      9923.40234 on epoch:    71, epoch:   72/100, mu: 0.9643317461, learning_rate: 0.0000020605, weight_decay: 0.0000138664

epoch time: 2333.1628417969s, input_fired_time: 31.9327659607, output_fired_time: 15.9659929276, propagate_forward_time: 893.7841186523, propagate_backward_time: 1375.1175537109, weight_update_time: 0.8246473074, other_time: 16.3621826172
epoch time: 179.1048889160s, input_fired_time: 2.4080071449, output_fired_time: 0.0000000000, propagate_forward_time: 173.9174652100, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7794189453
validation[          , genome 11868] predictions:    6679/  10000 (66.79%), best:    6692/10000 (66.92%), error:      9928.21680, best error:      9923.40234 on epoch:    71, epoch:   73/100, mu: 0.9653450847, learning_rate: 0.0000019192, weight_decay: 0.0000131917

epoch time: 2320.6916503906s, input_fired_time: 31.8751792908, output_fired_time: 16.1015968323, propagate_forward_time: 887.1057128906, propagate_backward_time: 1369.1015625000, weight_update_time: 0.8293937445, other_time: 16.5075683594
epoch time: 179.9212341309s, input_fired_time: 2.4347870350, output_fired_time: 0.0000000000, propagate_forward_time: 174.6730804443, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8133697510
validation[          , genome 11868] predictions:    6617/  10000 (66.17%), best:    6692/10000 (66.92%), error:     10047.28223, best error:      9923.40234 on epoch:    71, epoch:   74/100, mu: 0.9663184285, learning_rate: 0.0000017876, weight_decay: 0.0000125497

epoch time: 2309.5180664063s, input_fired_time: 31.8355464935, output_fired_time: 15.9025125504, propagate_forward_time: 881.9570312500, propagate_backward_time: 1363.1757812500, weight_update_time: 0.8590806723, other_time: 16.6472167969
epoch time: 180.0257263184s, input_fired_time: 2.4427866936, output_fired_time: 0.0000000000, propagate_forward_time: 174.7956237793, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7873229980
validation[          , genome 11868] predictions:    6753/  10000 (67.53%), best:    6753/10000 (67.53%), error:      9856.72754, best error:      9856.72754 on epoch:    75, epoch:   75/100, mu: 0.9672533274, learning_rate: 0.0000016651, weight_decay: 0.0000119391

epoch time: 2290.7954101563s, input_fired_time: 31.6223049164, output_fired_time: 15.8565073013, propagate_forward_time: 875.1633911133, propagate_backward_time: 1351.9234619141, weight_update_time: 0.8313621283, other_time: 16.2298583984
epoch time: 178.7315673828s, input_fired_time: 2.4050803185, output_fired_time: 0.0000000000, propagate_forward_time: 173.4239501953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9025421143
validation[          , genome 11868] predictions:    6719/  10000 (67.19%), best:    6753/10000 (67.53%), error:      9901.60449, best error:      9856.72754 on epoch:    75, epoch:   76/100, mu: 0.9681513309, learning_rate: 0.0000015509, weight_decay: 0.0000113581

epoch time: 2315.3869628906s, input_fired_time: 31.7854633331, output_fired_time: 15.8973073959, propagate_forward_time: 881.8082275391, propagate_backward_time: 1369.3684082031, weight_update_time: 0.8647676110, other_time: 16.5277099609
epoch time: 182.9805145264s, input_fired_time: 2.4727506638, output_fired_time: 0.0000000000, propagate_forward_time: 177.6549377441, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8528289795
validation[          , genome 11868] predictions:    6686/  10000 (66.86%), best:    6753/10000 (67.53%), error:      9927.57715, best error:      9856.72754 on epoch:    75, epoch:   77/100, mu: 0.9690138698, learning_rate: 0.0000014446, weight_decay: 0.0000108054

epoch time: 2311.6794433594s, input_fired_time: 31.6489734650, output_fired_time: 16.0421237946, propagate_forward_time: 880.7655029297, propagate_backward_time: 1367.0349121094, weight_update_time: 0.8227486014, other_time: 16.1878662109
epoch time: 182.3505706787s, input_fired_time: 2.5043833256, output_fired_time: 0.0000000000, propagate_forward_time: 176.9537048340, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8924865723
validation[          , genome 11868] predictions:    6663/  10000 (66.63%), best:    6753/10000 (67.53%), error:      9974.04395, best error:      9856.72754 on epoch:    75, epoch:   78/100, mu: 0.9698423743, learning_rate: 0.0000013455, weight_decay: 0.0000102796

epoch time: 2314.8366699219s, input_fired_time: 31.7999153137, output_fired_time: 15.9627199173, propagate_forward_time: 883.9570922852, propagate_backward_time: 1366.6667480469, weight_update_time: 0.8263844848, other_time: 16.4504394531
epoch time: 185.8184967041s, input_fired_time: 2.5416200161, output_fired_time: 0.0000000000, propagate_forward_time: 180.4037780762, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8730926514
validation[          , genome 11868] predictions:    6752/  10000 (67.52%), best:    6753/10000 (67.53%), error:      9882.05859, best error:      9856.72754 on epoch:    75, epoch:   79/100, mu: 0.9706381559, learning_rate: 0.0000012532, weight_decay: 0.0000097793

epoch time: 2307.5925292969s, input_fired_time: 31.7019462585, output_fired_time: 15.9189367294, propagate_forward_time: 882.0553588867, propagate_backward_time: 1361.5340576172, weight_update_time: 0.8431500793, other_time: 16.3822021484
epoch time: 180.7767333984s, input_fired_time: 2.4675710201, output_fired_time: 0.0000000000, propagate_forward_time: 175.5107269287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7984313965
validation[          , genome 11868] predictions:    6700/  10000 (67.00%), best:    6753/10000 (67.53%), error:      9927.28809, best error:      9856.72754 on epoch:    75, epoch:   80/100, mu: 0.9714025259, learning_rate: 0.0000011673, weight_decay: 0.0000093035

epoch time: 2306.8989257813s, input_fired_time: 31.8779830933, output_fired_time: 15.8857383728, propagate_forward_time: 880.9639282227, propagate_backward_time: 1361.9705810547, weight_update_time: 0.8349289298, other_time: 16.2008056641
epoch time: 179.8111114502s, input_fired_time: 2.4367761612, output_fired_time: 0.0000000000, propagate_forward_time: 174.5681762695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8061523438
validation[          , genome 11868] predictions:    6735/  10000 (67.35%), best:    6735/10000 (67.35%), error:      9834.33594, best error:      9834.33594 on epoch:    81, epoch:   81/100, mu: 0.9721367359, learning_rate: 0.0000010873, weight_decay: 0.0000088507

epoch time: 2300.7397460938s, input_fired_time: 31.6262588501, output_fired_time: 15.8901863098, propagate_forward_time: 877.7870483398, propagate_backward_time: 1359.0203857422, weight_update_time: 0.8324239850, other_time: 16.4158935547
epoch time: 180.1170196533s, input_fired_time: 2.4733748436, output_fired_time: 0.0000000000, propagate_forward_time: 174.7949371338, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8487091064
validation[          , genome 11868] predictions:    6702/  10000 (67.02%), best:    6735/10000 (67.35%), error:      9879.67578, best error:      9834.33594 on epoch:    81, epoch:   82/100, mu: 0.9728419185, learning_rate: 0.0000010127, weight_decay: 0.0000084200

epoch time: 2469.8886718750s, input_fired_time: 37.4414329529, output_fired_time: 16.9719219208, propagate_forward_time: 938.3652343750, propagate_backward_time: 1459.6494140625, weight_update_time: 0.8956517577, other_time: 17.4606933594
epoch time: 186.1044311523s, input_fired_time: 2.5022225380, output_fired_time: 0.0000000000, propagate_forward_time: 180.8045349121, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7976684570
validation[          , genome 11868] predictions:    6721/  10000 (67.21%), best:    6735/10000 (67.35%), error:      9897.07520, best error:      9834.33594 on epoch:    81, epoch:   83/100, mu: 0.9735192657, learning_rate: 0.0000009433, weight_decay: 0.0000080103

epoch time: 2431.1665039063s, input_fired_time: 33.3795776367, output_fired_time: 16.6159706116, propagate_forward_time: 931.6542358398, propagate_backward_time: 1431.9584960938, weight_update_time: 0.8627341390, other_time: 17.5581054688
epoch time: 183.4121398926s, input_fired_time: 2.4764223099, output_fired_time: 0.0000000000, propagate_forward_time: 178.0774078369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8583068848
validation[          , genome 11868] predictions:    6661/  10000 (66.61%), best:    6735/10000 (67.35%), error:      9979.09961, best error:      9834.33594 on epoch:    81, epoch:   84/100, mu: 0.9741699100, learning_rate: 0.0000008786, weight_decay: 0.0000076205

epoch time: 2437.5852050781s, input_fired_time: 33.8210334778, output_fired_time: 16.8427009583, propagate_forward_time: 932.2337036133, propagate_backward_time: 1436.6678466797, weight_update_time: 0.9983836412, other_time: 18.0198974609
epoch time: 210.3639221191s, input_fired_time: 2.6065335274, output_fired_time: 0.0000000000, propagate_forward_time: 204.5947875977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1625976563
validation[          , genome 11868] predictions:    6712/  10000 (67.12%), best:    6735/10000 (67.35%), error:      9890.81055, best error:      9834.33594 on epoch:    81, epoch:   85/100, mu: 0.9747948647, learning_rate: 0.0000008184, weight_decay: 0.0000072497

epoch time: 2413.8037109375s, input_fired_time: 33.5201377869, output_fired_time: 16.7358016968, propagate_forward_time: 922.6137695313, propagate_backward_time: 1423.4783935547, weight_update_time: 0.9146339893, other_time: 17.4556884766
epoch time: 191.8129119873s, input_fired_time: 2.6070690155, output_fired_time: 0.0000000000, propagate_forward_time: 186.1946105957, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0112304688
validation[          , genome 11868] predictions:    6683/  10000 (66.83%), best:    6735/10000 (67.35%), error:      9942.85840, best error:      9834.33594 on epoch:    81, epoch:   86/100, mu: 0.9753951430, learning_rate: 0.0000007622, weight_decay: 0.0000068969

epoch time: 2620.2719726563s, input_fired_time: 36.1724433899, output_fired_time: 18.8697929382, propagate_forward_time: 994.8836059570, propagate_backward_time: 1551.3983154297, weight_update_time: 0.9192934036, other_time: 18.9478759766
epoch time: 210.7547760010s, input_fired_time: 2.8362107277, output_fired_time: 0.0000000000, propagate_forward_time: 204.3305816650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5879821777
validation[          , genome 11868] predictions:    6739/  10000 (67.39%), best:    6735/10000 (67.35%), error:      9854.55566, best error:      9834.33594 on epoch:    81, epoch:   87/100, mu: 0.9759716988, learning_rate: 0.0000007100, weight_decay: 0.0000065613

epoch time: 2783.9616699219s, input_fired_time: 38.9934349060, output_fired_time: 18.9449119568, propagate_forward_time: 1109.1938476563, propagate_backward_time: 1596.5527343750, weight_update_time: 1.2197494507, other_time: 20.2768554688
epoch time: 202.1427917480s, input_fired_time: 2.6499795914, output_fired_time: 0.0000000000, propagate_forward_time: 196.4224853516, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0703277588
validation[          , genome 11868] predictions:    6742/  10000 (67.42%), best:    6735/10000 (67.35%), error:      9839.21875, best error:      9834.33594 on epoch:    81, epoch:   88/100, mu: 0.9765254855, learning_rate: 0.0000006613, weight_decay: 0.0000062420

epoch time: 2597.4899902344s, input_fired_time: 35.7201118469, output_fired_time: 17.9898281097, propagate_forward_time: 987.7057495117, propagate_backward_time: 1537.5590820313, weight_update_time: 0.9431201220, other_time: 18.5151367188
epoch time: 197.8319396973s, input_fired_time: 2.6891229153, output_fired_time: 0.0000000000, propagate_forward_time: 192.0029602051, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1398620605
validation[          , genome 11868] predictions:    6727/  10000 (67.27%), best:    6735/10000 (67.35%), error:      9839.02148, best error:      9834.33594 on epoch:    81, epoch:   89/100, mu: 0.9770574570, learning_rate: 0.0000006160, weight_decay: 0.0000059383

epoch time: 2656.3002929688s, input_fired_time: 36.1723327637, output_fired_time: 18.1979713440, propagate_forward_time: 1017.8371582031, propagate_backward_time: 1564.3156738281, weight_update_time: 0.9657215476, other_time: 19.7770996094
epoch time: 217.9831085205s, input_fired_time: 2.9775736332, output_fired_time: 0.0000000000, propagate_forward_time: 211.1127777100, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8927612305
validation[          , genome 11868] predictions:    6740/  10000 (67.40%), best:    6735/10000 (67.35%), error:      9912.13281, best error:      9834.33594 on epoch:    81, epoch:   90/100, mu: 0.9775683880, learning_rate: 0.0000005737, weight_decay: 0.0000056493

epoch time: 2623.0024414063s, input_fired_time: 38.2844924927, output_fired_time: 18.6126651764, propagate_forward_time: 1031.1516113281, propagate_backward_time: 1516.5504150391, weight_update_time: 0.9319679737, other_time: 18.4034423828
epoch time: 210.6374664307s, input_fired_time: 2.7927844524, output_fired_time: 0.0000000000, propagate_forward_time: 204.2992858887, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5453948975
validation[          , genome 11868] predictions:    6771/  10000 (67.71%), best:    6735/10000 (67.35%), error:      9849.96191, best error:      9834.33594 on epoch:    81, epoch:   91/100, mu: 0.9780591726, learning_rate: 0.0000005344, weight_decay: 0.0000053744

epoch time: 2602.6220703125s, input_fired_time: 36.2976264954, output_fired_time: 19.4450778961, propagate_forward_time: 996.6137084961, propagate_backward_time: 1531.6735839844, weight_update_time: 0.8956298828, other_time: 18.5920410156
epoch time: 201.9722137451s, input_fired_time: 2.7196912766, output_fired_time: 0.0000000000, propagate_forward_time: 196.1450195313, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1074981689
validation[          , genome 11868] predictions:    6741/  10000 (67.41%), best:    6735/10000 (67.35%), error:      9849.92090, best error:      9834.33594 on epoch:    81, epoch:   92/100, mu: 0.9785305858, learning_rate: 0.0000004977, weight_decay: 0.0000051129

epoch time: 2625.6003417969s, input_fired_time: 34.8568801880, output_fired_time: 17.5444488525, propagate_forward_time: 979.7541503906, propagate_backward_time: 1572.8426513672, weight_update_time: 1.0546493530, other_time: 20.6021728516
epoch time: 220.9688262939s, input_fired_time: 2.9900503159, output_fired_time: 0.0000000000, propagate_forward_time: 214.1100769043, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8686981201
validation[          , genome 11868] predictions:    6772/  10000 (67.72%), best:    6772/10000 (67.72%), error:      9826.33496, best error:      9826.33496 on epoch:    93, epoch:   93/100, mu: 0.9789834023, learning_rate: 0.0000004636, weight_decay: 0.0000048641

epoch time: 2402.8781738281s, input_fired_time: 33.3017959595, output_fired_time: 16.5574512482, propagate_forward_time: 920.2628784180, propagate_backward_time: 1415.6584472656, weight_update_time: 0.8651588559, other_time: 17.0976562500
epoch time: 186.1150512695s, input_fired_time: 2.5240318775, output_fired_time: 0.0000000000, propagate_forward_time: 180.6854858398, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9055328369
validation[          , genome 11868] predictions:    6759/  10000 (67.59%), best:    6772/10000 (67.72%), error:      9840.80176, best error:      9826.33496 on epoch:    93, epoch:   94/100, mu: 0.9794183373, learning_rate: 0.0000004318, weight_decay: 0.0000046274

epoch time: 2495.1059570313s, input_fired_time: 33.9219856262, output_fired_time: 16.9905242920, propagate_forward_time: 948.3541259766, propagate_backward_time: 1478.0031738281, weight_update_time: 0.8989309669, other_time: 17.8363037109
epoch time: 188.5584411621s, input_fired_time: 2.5320541859, output_fired_time: 0.0000000000, propagate_forward_time: 183.0673828125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9589996338
validation[          , genome 11868] predictions:    6787/  10000 (67.87%), best:    6772/10000 (67.72%), error:      9855.80762, best error:      9826.33496 on epoch:    93, epoch:   95/100, mu: 0.9798361063, learning_rate: 0.0000004022, weight_decay: 0.0000044022

epoch time: 2375.3532714844s, input_fired_time: 33.7363166809, output_fired_time: 16.0080127716, propagate_forward_time: 912.4977416992, propagate_backward_time: 1396.0848388672, weight_update_time: 1.1567307711, other_time: 17.0262451172
epoch time: 179.3219604492s, input_fired_time: 2.4187726974, output_fired_time: 0.0000000000, propagate_forward_time: 174.1874389648, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7157440186
validation[          , genome 11868] predictions:    6717/  10000 (67.17%), best:    6772/10000 (67.72%), error:      9864.19336, best error:      9826.33496 on epoch:    93, epoch:   96/100, mu: 0.9802373648, learning_rate: 0.0000003746, weight_decay: 0.0000041880

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868_2_r1274161476_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523476
read mu: 0.980623
read mu_delta: 0.960522
read initial_learning_rate: 0.000342851
read learning_rate: 3.48943e-07
read learning_rate_delta: 0.931432
read initial_weight_decay: 0.000503325
read weight_decay: 3.98418e-06
read weight_decay_delta: 0.951338
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0838315
read input_dropout_probability: 0.00122995
read hidden_dropout_probability: 0.103756
read velocity_reset: 2135
read epoch: 97
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 93
read number_validation_images: 10000
read best_validation_predictions: 6772
read best_validation_error: 9826.33
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11868
read normal distribution: '0 0.235368803143501 -0.0996157824993134'
generator_str: '614020559'
read generator: 614020559
read generated_by_map:
1 add_edge 1
reading 142 nodes.
reading 533 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868_2_r1274161476_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523476
read mu: 0.980623
read mu_delta: 0.960522
read initial_learning_rate: 0.000342851
read learning_rate: 3.48943e-07
read learning_rate_delta: 0.931432
read initial_weight_decay: 0.000503325
read weight_decay: 3.98418e-06
read weight_decay_delta: 0.951338
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0838315
read input_dropout_probability: 0.00122995
read hidden_dropout_probability: 0.103756
read velocity_reset: 2135
read epoch: 97
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 93
read number_validation_images: 10000
read best_validation_predictions: 6772
read best_validation_error: 9826.33
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11868
read normal distribution: '0 0.235368803143501 -0.0996157824993134'
generator_str: '614020559'
read generator: 614020559
read generated_by_map:
1 add_edge 1
reading 142 nodes.
reading 533 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868_2_r1274161476_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523476
read mu: 0.980623
read mu_delta: 0.960522
read initial_learning_rate: 0.000342851
read learning_rate: 3.48943e-07
read learning_rate_delta: 0.931432
read initial_weight_decay: 0.000503325
read weight_decay: 3.98418e-06
read weight_decay_delta: 0.951338
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0838315
read input_dropout_probability: 0.00122995
read hidden_dropout_probability: 0.103756
read velocity_reset: 2135
read epoch: 97
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 93
read number_validation_images: 10000
read best_validation_predictions: 6772
read best_validation_error: 9826.33
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11868
read normal distribution: '0 0.235368803143501 -0.0996157824993134'
generator_str: '614020559'
read generator: 614020559
read generated_by_map:
1 add_edge 1
reading 142 nodes.
reading 533 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2301.44555664063s, input_fired_time: 31.7581691741943, output_fired_time: 15.7526712417603, propagate_forward_time: 880.588684082031, propagate_backward_time: 1358.02294921875, weight_update_time: 0.808804631233215, other_time: 15.3232421875
epoch time: 182.15559387207s, input_fired_time: 2.45185422897339, output_fired_time: 0, propagate_forward_time: 177.027313232422, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.67642211914063
validation[          , genome 11868] predictions:    6759/  10000 (67.59%), best:    6759/10000 (67.59%), error:      9820.90332, best error:      9820.90332 on epoch:    97, epoch:   97/100, mu: 0.9806227684, learning_rate: 0.0000003489, weight_decay: 0.0000039842

epoch time: 2288.1774902344s, input_fired_time: 31.3416748047, output_fired_time: 15.6969680786, propagate_forward_time: 875.6557617188, propagate_backward_time: 1350.0557861328, weight_update_time: 0.8217639923, other_time: 15.4273681641
epoch time: 179.5818481445s, input_fired_time: 2.4672126770, output_fired_time: 0.0000000000, propagate_forward_time: 174.4134063721, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7012329102
validation[          , genome 11868] predictions:    6754/  10000 (67.54%), best:    6759/10000 (67.59%), error:      9852.91699, best error:      9820.90332 on epoch:    97, epoch:   98/100, mu: 0.9809929729, learning_rate: 0.0000003250, weight_decay: 0.0000037903

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868_2_r1274161476_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523476
read mu: 0.981349
read mu_delta: 0.960522
read initial_learning_rate: 0.000342851
read learning_rate: 3.02731e-07
read learning_rate_delta: 0.931432
read initial_weight_decay: 0.000503325
read weight_decay: 3.60586e-06
read weight_decay_delta: 0.951338
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0838315
read input_dropout_probability: 0.00122995
read hidden_dropout_probability: 0.103756
read velocity_reset: 2135
read epoch: 99
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 97
read number_validation_images: 10000
read best_validation_predictions: 6759
read best_validation_error: 9820.9
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11868
read normal distribution: '0 0.235368803143501 -0.0996157824993134'
generator_str: '957594134'
read generator: 957594134
read generated_by_map:
1 add_edge 1
reading 142 nodes.
reading 533 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2287.7998046875s, input_fired_time: 31.2874202728271, output_fired_time: 15.6242227554321, propagate_forward_time: 875.2626953125, propagate_backward_time: 1350.10363769531, weight_update_time: 0.808060169219971, other_time: 15.5218505859375
epoch time: 177.50895690918s, input_fired_time: 2.40580272674561, output_fired_time: 0, propagate_forward_time: 172.450012207031, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.65313720703125
validation[          , genome 11868] predictions:    6797/  10000 (67.97%), best:    6759/10000 (67.59%), error:      9823.38672, best error:      9820.90332 on epoch:    97, epoch:   99/100, mu: 0.9813485742, learning_rate: 0.0000003027, weight_decay: 0.0000036059

epoch time: 2373.1516113281s, input_fired_time: 32.6547737122, output_fired_time: 17.1337432861, propagate_forward_time: 907.5673217773, propagate_backward_time: 1399.2772216797, weight_update_time: 0.8281561136, other_time: 16.5184326172
epoch time: 205.5318145752s, input_fired_time: 2.6708941460, output_fired_time: 0.0000000000, propagate_forward_time: 199.7078247070, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1530914307
validation[          , genome 11868] predictions:    6761/  10000 (67.61%), best:    6761/10000 (67.61%), error:      9806.42676, best error:      9806.42676 on epoch:   100, epoch:  100/100, mu: 0.9816901088, learning_rate: 0.0000002820, weight_decay: 0.0000034304

evaluating best weights on full training data.
evaluting training set with running mean/variance:
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/cifar_10_training.bin'
boincified validation filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified testing filename: '../../projects/csgrid.org_csg/cifar_10_testing.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1535714404_162_11868_2_r1274161476_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.523476
read mu: 0.982018
read mu_delta: 0.960522
read initial_learning_rate: 0.000342851
read learning_rate: 2.62639e-07
read learning_rate_delta: 0.931432
read initial_weight_decay: 0.000503325
read weight_decay: 3.26346e-06
read weight_decay_delta: 0.951338
read batch_size: 92
read epsilon: 1e-07
read alpha: 0.0838315
read input_dropout_probability: 0.00122995
read hidden_dropout_probability: 0.103756
read velocity_reset: 2135
read epoch: 101
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 100
read number_validation_images: 10000
read best_validation_predictions: 6761
read best_validation_error: 9806.43
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 11868
read normal distribution: '0 0.235368803143501 -0.0996157824993134'
generator_str: '810913027'
read generator: 810913027
read generated_by_map:
1 add_edge 1
reading 142 nodes.
reading 533 edges.
number input nodes: 3
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
reading image set with 5000 images.
image_size: 3x32x32 = 3072
read 50000 images.
    class    0: 5000
    class    1: 5000
    class    2: 5000
    class    3: 5000
    class    4: 5000
    class    5: 5000
    class    6: 5000
    class    7: 5000
    class    8: 5000
    class    9: 5000
calculating averages and standard deviations for images
average pixel value for channel 0: 0.491399973630905
average pixel value for channel 1: 0.482159107923508
average pixel value for channel 2: 0.446530401706696
pixel variance for channel 0: 0.0610253289341927
pixel standard deviation for channel 0: 0.247033059597015
pixel variance for channel 1: 0.0592849254608154
pixel standard deviation for channel 1: 0.24348495900631
pixel variance for channel 2: 0.0684280395507813
pixel standard deviation for channel 2: 0.261587530374527
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 3
width: 32
height: 32
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 3x32x32 = 3072
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
loaded images
starting backpropagation!
epoch time: 2307.53662109375s, input_fired_time: 31.4457416534424, output_fired_time: 15.9148855209351, propagate_forward_time: 881.155639648438, propagate_backward_time: 1362.91223144531, weight_update_time: 0.82589715719223, other_time: 16.108154296875
epoch time: 177.218688964844s, input_fired_time: 2.39118194580078, output_fired_time: 0, propagate_forward_time: 172.006759643555, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.82075500488281
validation[          , genome 11868] predictions:    6742/  10000 (67.42%), best:    6761/10000 (67.61%), error:      9834.60742, best error:      9806.42676 on epoch:   100, epoch:  101/100, mu: 0.9820181727, learning_rate: 0.0000002626, weight_decay: 0.0000032635

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 911.6330566406s, input_fired_time: 12.3776836395, output_fired_time: 0.0000000000, propagate_forward_time: 885.0601196289, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 14.1952514648
best training[          , genome 11868] predictions:   35436/  50000 (70.87%), best:    6761/10000 (67.61%), error:     45412.05469, best error:      9806.42676 on epoch:   100, epoch:  102/100, mu: 0.9823333025, learning_rate: 0.0000002446, weight_decay: 0.0000031047
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 188.9548797607s, input_fired_time: 2.5410788059, output_fired_time: 0.0000000000, propagate_forward_time: 183.5314788818, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8823242188
   testing[          , genome 11868] predictions:    6761/  10000 (67.61%), best:    6761/10000 (67.61%), error:      9806.42676, best error:      9806.42676 on epoch:   100, epoch:  102/100, mu: 0.9823333025, learning_rate: 0.0000002446, weight_decay: 0.0000031047
backpropagation finished successfully!
19:49:59 (18104): called boinc_finish(0)

</stderr_txt>
]]>