Advanced search

Message boards : News : [wildlife] EXACT2 v0.19

Author Message
Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1791
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 710

              
Message 6817 - Posted: 19 Mar 2017, 22:11:26 UTC

I noticed an issue where the datafiles for v0.18 were missing a parameter, which was leading to some WUs not validating and some errors on the server side. I've updated the application to v0.19 which should fix this issue -- although may cause some of the WUs generated for v0.18 to error out. Sorry about that!

Chris Skull
Avatar
Send message
Joined: 11 Apr 15
Posts: 20
Combined Credit: 4,766,966
DNA@Home: 55,861
SubsetSum@Home: 1,272,523
Wildlife@Home: 3,438,582
Wildlife@Home Watched: 1,312,789s
Wildlife@Home Events: 475
Climate Tweets: 0
Images Observed: 46

            
Message 6820 - Posted: 21 Mar 2017, 5:42:04 UTC - in response to Message 6817.

still high error rate.. maybe because high RAM usage ? 1.4 GB per unit.... we have to reduce max units per host....
____________
Greetz
Chris

mmonnin
Send message
Joined: 31 May 16
Posts: 25
Combined Credit: 17,062,809
DNA@Home: 0
SubsetSum@Home: 1,023,200
Wildlife@Home: 16,039,608
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 54
Images Observed: 0

      
Message 6821 - Posted: 21 Mar 2017, 10:29:13 UTC
Last modified: 21 Mar 2017, 10:32:47 UTC

Just allowed CSG work and 1st task error'd in 3 seconds. 32GB of RAM for 8 tasks and I'm not using half.

starting from input file: '../../projects/csgrid.org_csg/exact_genome_1489898564_3_94.txt' read CNN_Genome file with version string: 'v0.18' breaking because version_str '0.18' did not match EXACT_VERSION '0.19': 1 parsed input file ERROR: exact application with version '0.19' trying to process workunit with incompatible input version: '0.18'


http://csgrid.org/csg/result.php?resultid=2044730

If these are processing images, would a GPU app work well? Maybe even add more images if it needs to be more parallel.

Jozef J
Send message
Joined: 19 Apr 14
Posts: 13
Combined Credit: 19,105,815
DNA@Home: 1,006,826
SubsetSum@Home: 115,384
Wildlife@Home: 17,983,605
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 3,230
Images Observed: 1

        
Message 6822 - Posted: 21 Mar 2017, 15:20:09 UTC

http://csgrid.org/csg/top_hosts.php soon will start some of "fun"
As i see most computers have only 32 gigabytes of memory so soon when they will finnish old work and "suck" the new 0.19 application ram killer ))). starts most machines connected to this project to fail and be "dysfunctional" becouse the lack of ram . ..
Now need aprx. 1,5 gb per core..
I see one of my host with 32 cores and 32gb ram completly freeze (and its ssd there as sys drive) soo its complete waste of work becouse i must delete project to "unfreeze" and connect other project , or disable project ,but i dont have plan put more ram to this host ,so i dont will finnish work later ..for exmpl..
Maybe they have to share some app or config to limit the cores for project until is this situation )
Or better implement feature like amicable number actually have
↑ You can set CPU core limits and fine-tune GPU here ↑ in your pref.
or BEst if you can quickly set - cores per project on every computer/host .. in "your" settings on project page . to avoid that waste of work

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1791
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 710

              
Message 6823 - Posted: 21 Mar 2017, 15:37:24 UTC - in response to Message 6822.
Last modified: 21 Mar 2017, 15:38:02 UTC

http://csgrid.org/csg/top_hosts.php soon will start some of "fun"
As i see most computers have only 32 gigabytes of memory so soon when they will finnish old work and "suck" the new 0.19 application ram killer ))). starts most machines connected to this project to fail and be "dysfunctional" becouse the lack of ram . ..
Now need aprx. 1,5 gb per core..
I see one of my host with 32 cores and 32gb ram completly freeze (and its ssd there as sys drive) soo its complete waste of work becouse i must delete project to "unfreeze" and connect other project , or disable project ,but i dont have plan put more ram to this host ,so i dont will finnish work later ..for exmpl..
Maybe they have to share some app or config to limit the cores for project until is this situation )
Or better implement feature like amicable number actually have
↑ You can set CPU core limits and fine-tune GPU here ↑ in your pref.
or BEst if you can quickly set - cores per project on every computer/host .. in "your" settings on project page . to avoid that waste of work



Well, it's not quite that bad. I'm currently running 4 different searches:

http://csgrid.org/csg/exact/overview.php

Two of those are using the MNIST dataset (which should use the same RAM as before), and there are two using the CIFAR-10 dataset (which will be 1+GB RAM). So only half the WUs being generated will be requiring more RAM.

What I can do is modify the memory requirements in the workunits, if that would help the BOINC scheduler schedule things better?

Jozef J
Send message
Joined: 19 Apr 14
Posts: 13
Combined Credit: 19,105,815
DNA@Home: 1,006,826
SubsetSum@Home: 115,384
Wildlife@Home: 17,983,605
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 3,230
Images Observed: 1

        
Message 6824 - Posted: 21 Mar 2017, 16:24:48 UTC

"I'm currently running 4 different searches:" but work unit only from one.. this morning only from exact2 0,19.
i am just pointing on incomming situation , no problem with big ram app, just we need prepare before :-)

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1791
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 710

              
Message 6825 - Posted: 21 Mar 2017, 16:32:07 UTC - in response to Message 6824.

"I'm currently running 4 different searches:" but work unit only from one.. this morning only from exact2 0,19.
i am just pointing on incomming situation , no problem with big ram app, just we need prepare before :-)


So workunit names look something like:

exact_genome_1487653029_21_46088_0

This is basically:

exact_genome____

Workunits with a search id of 9 or 10 are running the larger CIFAR-10 dataset, and workunits with a search id of 11 or 12 are running the MNIST dataset. The search ID matches up to the searches on the overview page:

http://csgrid.org/csg/exact/overview.php

Hope that makes sense!

Profile JumpinJohnny
Avatar
Send message
Joined: 24 Sep 13
Posts: 235
Combined Credit: 7,567,625
DNA@Home: 192,548
SubsetSum@Home: 201,740
Wildlife@Home: 7,173,338
Wildlife@Home Watched: 55,997,833s
Wildlife@Home Events: 15,584
Climate Tweets: 312
Images Observed: 351

              
Message 6826 - Posted: 22 Mar 2017, 1:46:45 UTC - in response to Message 6825.

Thanks for the explanation of the Work Units.
That helps us to understand the activity we are seeing on our computers.

Any description of the project(s) to inform us about the productive science being done is also welcome.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1791
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 710

              
Message 6827 - Posted: 22 Mar 2017, 2:51:36 UTC - in response to Message 6826.

Thanks for the explanation of the Work Units.
That helps us to understand the activity we are seeing on our computers.

Any description of the project(s) to inform us about the productive science being done is also welcome.


Here's the latest version of a paper I've been working on which describes some of the most recent results and the algorithm I'm using:

https://arxiv.org/abs/1703.05422

The newly updated code adds some improvements to how I'm doing backpropagation, and it also is evolving the hyperparameters used to train how backpropagation works.

I'm also evaluating things on the CIFAR-10 dataset (https://www.cs.toronto.edu/~kriz/cifar.html), which is more challenging than MNIST and also color images so if things work well on that -- we'll be well suited to start working on Wildlife@Home images.

If you have any questions about the paper (it may be too academic) just ask away!

Profile Steve Hawker*
Send message
Joined: 8 Apr 13
Posts: 134
Combined Credit: 829,896
DNA@Home: 11,932
SubsetSum@Home: 299,708
Wildlife@Home: 518,257
Wildlife@Home Watched: 5,541,577s
Wildlife@Home Events: 2,169
Climate Tweets: 8,579
Images Observed: 55

              
Message 6828 - Posted: 22 Mar 2017, 19:54:03 UTC - in response to Message 6825.

Workunits with a search id of 9 or 10 are running the larger CIFAR-10 dataset, and workunits with a search id of 11 or 12 are running the MNIST dataset.


Would be nice if we could select which ones to run. the larger ones slow my old Macbook to a crawl and running the GPU (for SETI) at the same time just locks it up totally.

I thank you.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1791
Combined Credit: 2,265,607
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 1,622,832
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 710

              
Message 6831 - Posted: 23 Mar 2017, 1:27:42 UTC - in response to Message 6828.

Workunits with a search id of 9 or 10 are running the larger CIFAR-10 dataset, and workunits with a search id of 11 or 12 are running the MNIST dataset.


Would be nice if we could select which ones to run. the larger ones slow my old Macbook to a crawl and running the GPU (for SETI) at the same time just locks it up totally.

I thank you.


I should be able to do that. Will work on it!


Post to thread

Message boards : News : [wildlife] EXACT2 v0.19