Advanced search

Message boards : News : [wildlife] new app/type version 0.31 and an update

Author Message
Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1789
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 699

              
Message 7215 - Posted: 22 Jul 2017, 14:52:24 UTC
Last modified: 22 Jul 2017, 14:52:45 UTC

I've made an update to the application which I'm hoping will allow us to get even better results. Currently we've hit 99.45% on the MNIST dataset, which is actually quite good considering the current version of the application only performs convolutional operations and doesn't do any pooling. Here's a list of some of the best published results on this dataset:

http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#4d4e495354

And that puts is pretty solidly in the middle of the list. The new application now lets EXACT also evolve CNNs with pooling operations, which I'm hoping will bump us even higher up the list. This is pretty exciting as we're evolving the structure of the convolutional neural networks (CNNs) from scratch, and we're getting competitive with some of the best human designed CNNs. Anywho, let me know if you are having any errors with the new v0.31 applications.

On another note, I just returned from this years Genetic and Evolutionary Computation Conference in Berlin, where I presented a poster on this work. There was quite a bit of interest in this approach so that was really good. I just want to thank everyone again for contributing all your cycles to this work.

mmonnin
Send message
Joined: 31 May 16
Posts: 24
Combined Credit: 15,522,198
DNA@Home: 0
SubsetSum@Home: 1,023,200
Wildlife@Home: 14,498,998
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 54
Images Observed: 0

      
Message 7218 - Posted: 22 Jul 2017, 19:49:02 UTC - in response to Message 7215.

Could the deadline be increased? I allowed for work from CSG from the new app and nearly all the tasks went to high priority and interrupted other work. Some tasks have only about 10 hours before deadline. D/L about 3:45PM, deadline of next morning 1:42AM. They will all complete but I only had 0.5 days of work.

GLeeM
Send message
Joined: 1 Jul 13
Posts: 118
Combined Credit: 28,200,843
DNA@Home: 28,994
SubsetSum@Home: 231,079
Wildlife@Home: 27,940,769
Wildlife@Home Watched: 3,888,714s
Wildlife@Home Events: 628
Climate Tweets: 0
Images Observed: 0

          
Message 7220 - Posted: 23 Jul 2017, 0:42:30 UTC - in response to Message 7215.

Can anything be done about the bandwidth usage of these WUs?
They use in one day what the old ones used in one month!

My monthly limit would be used in 4 days :(

I guess I'll be back to try the next batch of new WUs :)

Profile Skivelitis2
Avatar
Send message
Joined: 16 May 15
Posts: 60
Combined Credit: 2,590,952
DNA@Home: 19,068
SubsetSum@Home: 575,552
Wildlife@Home: 1,996,332
Wildlife@Home Watched: 9,158s
Wildlife@Home Events: 4
Climate Tweets: 382
Images Observed: 52

            
Message 7221 - Posted: 23 Jul 2017, 12:17:30 UTC

The shorter run times on this batch are quite welcome! I do have one however that looks like it will take about 20 hrs to complete. Hopefully the credit scales accordingly and we don't have another "monster" issue. Will report upon completion.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1789
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 699

              
Message 7222 - Posted: 23 Jul 2017, 13:27:45 UTC - in response to Message 7220.

Can anything be done about the bandwidth usage of these WUs?
They use in one day what the old ones used in one month!

My monthly limit would be used in 4 days :(

I guess I'll be back to try the next batch of new WUs :)


These shouldn't be using any more bandwidth than the last WUs. They're all working on the same data and i only made slight tweaks to the input file. What file is taking up all the bandwidth?

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1789
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 699

              
Message 7223 - Posted: 23 Jul 2017, 13:50:22 UTC - in response to Message 7221.

The shorter run times on this batch are quite welcome! I do have one however that looks like it will take about 20 hrs to complete. Hopefully the credit scales accordingly and we don't have another "monster" issue. Will report upon completion.


Well each set of runs starts off small and gets larger and larger as the CNNs being trained get more and more complex. Is the 20 hour workunit from v0.31 or still from v0.30? The remaining v0.30 WUs will be longer until the v0.31 WUs get more complex.

JugNut
Send message
Joined: 4 Feb 17
Posts: 4
Combined Credit: 61,983,258
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 61,983,258
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

  
Message 7224 - Posted: 23 Jul 2017, 15:41:37 UTC - in response to Message 7223.

All my PC will be dry by morning, most within the next hour or two.

Any idea when we'll see new work?

Profile Skivelitis2
Avatar
Send message
Joined: 16 May 15
Posts: 60
Combined Credit: 2,590,952
DNA@Home: 19,068
SubsetSum@Home: 575,552
Wildlife@Home: 1,996,332
Wildlife@Home Watched: 9,158s
Wildlife@Home Events: 4
Climate Tweets: 382
Images Observed: 52

            
Message 7225 - Posted: 23 Jul 2017, 19:01:05 UTC - in response to Message 7223.

The shorter run times on this batch are quite welcome! I do have one however that looks like it will take about 20 hrs to complete. Hopefully the credit scales accordingly and we don't have another "monster" issue. Will report upon completion.


Well each set of runs starts off small and gets larger and larger as the CNNs being trained get more and more complex. Is the 20 hour workunit from v0.31 or still from v0.30? The remaining v0.30 WUs will be longer until the v0.31 WUs get more complex.

It is work unit # 1397275 (v 0.31).
72988.76 sec run time for 240.09 credits
A few other tasks on same host:
5090.22 sec....229.97cr
5089.31 sec....233.42cr both also v 0.31

GLeeM
Send message
Joined: 1 Jul 13
Posts: 118
Combined Credit: 28,200,843
DNA@Home: 28,994
SubsetSum@Home: 231,079
Wildlife@Home: 27,940,769
Wildlife@Home Watched: 3,888,714s
Wildlife@Home Events: 628
Climate Tweets: 0
Images Observed: 0

          
Message 7227 - Posted: 24 Jul 2017, 0:49:37 UTC - in response to Message 7222.

Can anything be done about the bandwidth usage of these WUs?
They use in one day what the old ones used in one month!

My monthly limit would be used in 4 days :(

I guess I'll be back to try the next batch of new WUs :)


These shouldn't be using any more bandwidth than the last WUs. They're all working on the same data and i only made slight tweaks to the input file. What file is taking up all the bandwidth?

I mean upload and download bandwidth. The WU download size was 0.8 MB, the Result upload size was 1.34 MB for a total of over 2 MB. The unit ran for just over 20 minutes. That adds up to 3,000 MB per day (or 90,000 MB per month) for 21 threads from three computers.
The old WUs took up to 24 hours to finish and the upload was 6 MB. That adds up to 126 MB per day (or 3,780 MB per month) for 21 threads. Hughesnet satellite internet gives me 14,000 MB per month.
If the WUs start taking longer as more are done then things will get better soon.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1789
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 699

              
Message 7235 - Posted: 25 Jul 2017, 1:00:45 UTC - in response to Message 7218.

Could the deadline be increased? I allowed for work from CSG from the new app and nearly all the tasks went to high priority and interrupted other work. Some tasks have only about 10 hours before deadline. D/L about 3:45PM, deadline of next morning 1:42AM. They will all complete but I only had 0.5 days of work.


The deadlines are generated somewhat automatically depending on the size of the work unit. The ones that initially get sent out at the start of a search are really small and should only take a few minutes, that's why the deadline is so short.

What I could do is keep that the same except also have a minimum deadline of 1 day -- so the deadline would always be at least one day. Would that work better?

mmonnin
Send message
Joined: 31 May 16
Posts: 24
Combined Credit: 15,522,198
DNA@Home: 0
SubsetSum@Home: 1,023,200
Wildlife@Home: 14,498,998
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 54
Images Observed: 0

      
Message 7237 - Posted: 25 Jul 2017, 1:57:25 UTC - in response to Message 7235.
Last modified: 25 Jul 2017, 1:59:52 UTC

Could the deadline be increased? I allowed for work from CSG from the new app and nearly all the tasks went to high priority and interrupted other work. Some tasks have only about 10 hours before deadline. D/L about 3:45PM, deadline of next morning 1:42AM. They will all complete but I only had 0.5 days of work.


The deadlines are generated somewhat automatically depending on the size of the work unit. The ones that initially get sent out at the start of a search are really small and should only take a few minutes, that's why the deadline is so short.

What I could do is keep that the same except also have a minimum deadline of 1 day -- so the deadline would always be at least one day. Would that work better?


Multiple that factor several times or a fixed length. A several hour deadline no matter the length is unrealistic. Just got one with about 9 hours til deadline, just shy of 10PM and it's due at 5am. 32 for 32 interrupted tasks. 27 other tasks started just to be interrupted by other CSG tasks.

Edit: It was mentioned about keeping CPU RAC about the same across projects as like a gentleman's code. Same should be applied here. Short deadlines just make tasks jump other projects.

tom7up
Send message
Joined: 23 Nov 14
Posts: 15
Combined Credit: 5,746,525
DNA@Home: 101,983
SubsetSum@Home: 100,103
Wildlife@Home: 5,544,440
Wildlife@Home Watched: 8,771,275s
Wildlife@Home Events: 2,515
Climate Tweets: 2,888
Images Observed: 40,768

              
Message 7238 - Posted: 25 Jul 2017, 5:16:08 UTC - in response to Message 7237.

1. The deadline of Pooling-WUs is just 1-3 days. That is really too short. It should be not less than a week.

2. You add the credit for Pooling-WU to combined credit but not to Wildlife@Home credit.

Profile Gibson Praise
Avatar
Send message
Joined: 5 Aug 16
Posts: 52
Combined Credit: 23,806,756
DNA@Home: 0
SubsetSum@Home: 73,256
Wildlife@Home: 23,733,501
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 767
Images Observed: 23,280

        
Message 7240 - Posted: 25 Jul 2017, 13:27:55 UTC - in response to Message 7238.

2. You add the credit for Pooling-WU to combined credit but not to Wildlife@Home credit.

That does appear to be the case.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1789
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 699

              
Message 7243 - Posted: 25 Jul 2017, 18:38:59 UTC - in response to Message 7240.

2. You add the credit for Pooling-WU to combined credit but not to Wildlife@Home credit.

That does appear to be the case.


Yup, it's fixed now. Forgot to update some of the webpages to include that appid.

ku4jb
Send message
Joined: 18 May 13
Posts: 3
Combined Credit: 37,678,232
DNA@Home: 114,462
SubsetSum@Home: 2,367,590
Wildlife@Home: 35,196,179
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 7250 - Posted: 31 Jul 2017, 0:40:13 UTC
Last modified: 31 Jul 2017, 0:40:40 UTC

Longer one here https://csgrid.org/csg/workunit.php?wuid=1451777


Post to thread

Message boards : News : [wildlife] new app/type version 0.31 and an update