Advanced search

Message boards : News : longer DNA@home workunits

Author Message
Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3258 - Posted: 8 Sep 2014, 22:02:01 UTC

I've created a new set of work units from DNA@Home using a new data file (from human genome revision 19 -- hg19). These should run a bit longer than the other work units, so please let me know if the time estimates and credit is okay. They're actually not as long as I'd like them to be, so depending on how things are working work units after this could be 10x longer.

Profile Conan
Avatar
Send message
Joined: 13 Apr 12
Posts: 151
Combined Credit: 47,587,735
DNA@Home: 399,792
SubsetSum@Home: 1,448,876
Wildlife@Home: 45,739,067
Wildlife@Home Watched: 70,910s
Wildlife@Home Events: 0
Climate Tweets: 410
Images Observed: 0

          
Message 3259 - Posted: 8 Sep 2014, 22:19:58 UTC

Estimated run time on my computers is 10 to 14 minutes, so I will see how they run.

Conan

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3260 - Posted: 8 Sep 2014, 22:24:28 UTC - in response to Message 3259.
Last modified: 8 Sep 2014, 22:24:46 UTC

Estimated run time on my computers is 10 to 14 minutes, so I will see how they run.

Conan


Okay that's not too bad, if it's accurate than I can make them 10x longer without much of a problem. Didn't want these to run too long because I wanted to make sure they wouldn't crash and validate correctly.

Profile Charles Dennett
Avatar
Send message
Joined: 10 Aug 14
Posts: 39
Combined Credit: 354,259
DNA@Home: 219,783
SubsetSum@Home: 134,472
Wildlife@Home: 4
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 3261 - Posted: 8 Sep 2014, 23:08:50 UTC

On my fastest system they are estimated at 10 minutes or so. So far one has completed and it was done in 93 seconds cpu time. The next one is at 50% complete with about 43 minutes of cpu time. Quite a difference. So, right now the estimate is 90 minutes or so for this one. Not sure why it's taking so much longer. This is a 4 core system running Linux. There are three other tasks running and they appear to be in line with the one that's half done. Time will tell.
____________

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3262 - Posted: 8 Sep 2014, 23:14:14 UTC - in response to Message 3261.

On my fastest system they are estimated at 10 minutes or so. So far one has completed and it was done in 93 seconds cpu time. The next one is at 50% complete with about 43 minutes of cpu time. Quite a difference. So, right now the estimate is 90 minutes or so for this one. Not sure why it's taking so much longer. This is a 4 core system running Linux. There are three other tasks running and they appear to be in line with the one that's half done. Time will tell.


Strange, there's probably an error with the one that took 93 seconds. Was it a test_hg19 work unit or something else? 90 minutes sounds more right for the test_hg19 work units.

Profile Charles Dennett
Avatar
Send message
Joined: 10 Aug 14
Posts: 39
Combined Credit: 354,259
DNA@Home: 219,783
SubsetSum@Home: 134,472
Wildlife@Home: 4
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 3263 - Posted: 8 Sep 2014, 23:14:57 UTC
Last modified: 8 Sep 2014, 23:17:09 UTC

A double check of that fast one shows it was a leftover from the last batch of small ones. Must have been stuck. It was created earlier today. So, the time estimates for the new batch seem to be very low.
____________

Profile JumpinJohnny
Avatar
Send message
Joined: 24 Sep 13
Posts: 237
Combined Credit: 10,275,610
DNA@Home: 192,548
SubsetSum@Home: 201,740
Wildlife@Home: 9,881,323
Wildlife@Home Watched: 55,997,833s
Wildlife@Home Events: 15,584
Climate Tweets: 331
Images Observed: 351

              
Message 3264 - Posted: 8 Sep 2014, 23:15:46 UTC

They come to me with an "estimated completion" time of 9:25.
They are completing at around 50 minutes on this rig.
Ten times longer would be a lot...
I'll run them overnight and see how it goes.
** I'm using the single GPU for Collatz Conjecture "mini_collatz" WUs @ (0.88cpu + 1 ATI gpu)
____________

Profile nenym
Send message
Joined: 29 Apr 13
Posts: 4
Combined Credit: 18,838,604
DNA@Home: 1,118,166
SubsetSum@Home: 1,295,535
Wildlife@Home: 16,424,903
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 3265 - Posted: 9 Sep 2014, 0:54:30 UTC

Tasks seems to have a validate horror, any validator bug?

Profile JumpinJohnny
Avatar
Send message
Joined: 24 Sep 13
Posts: 237
Combined Credit: 10,275,610
DNA@Home: 192,548
SubsetSum@Home: 201,740
Wildlife@Home: 9,881,323
Wildlife@Home Watched: 55,997,833s
Wildlife@Home Events: 15,584
Climate Tweets: 331
Images Observed: 351

              
Message 3266 - Posted: 9 Sep 2014, 1:02:18 UTC
Last modified: 9 Sep 2014, 1:09:29 UTC

What is the check-pointing set at for these???
Seems to be at 50% ??? I thought check-pointing was done by time, not percentage?

I have validate errors on all completed tasks that are not pending.
gibbs_test_hg19_1_71_0_1
gibbs_test_hg19_1_72_0_1
etc. etc. etc.

I think I will abort all these and wait until further news.
I noticed a huge memory increase while running these... related?

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3267 - Posted: 9 Sep 2014, 1:31:36 UTC - in response to Message 3266.

What is the check-pointing set at for these???
Seems to be at 50% ??? I thought check-pointing was done by time, not percentage?

I have validate errors on all completed tasks that are not pending.
gibbs_test_hg19_1_71_0_1
gibbs_test_hg19_1_72_0_1
etc. etc. etc.

I think I will abort all these and wait until further news.
I noticed a huge memory increase while running these... related?


This will take a whole lot more memory as the input file is quite a bit larger. Let me check out the errors.

Checkpointing is done by the number of iterations the application does. Since I set it to do 10000 total iterations and it was previously checkpointing every 5000, 50% looks right. I should probably make it checkpoint more frequently, maybe every 5% or so.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3268 - Posted: 9 Sep 2014, 1:33:00 UTC - in response to Message 3267.

it's also really strange that they're erroring out. From the tasks themselves it looks like they're returning the same results...

Profile Conan
Avatar
Send message
Joined: 13 Apr 12
Posts: 151
Combined Credit: 47,587,735
DNA@Home: 399,792
SubsetSum@Home: 1,448,876
Wildlife@Home: 45,739,067
Wildlife@Home Watched: 70,910s
Wildlife@Home Events: 0
Climate Tweets: 410
Images Observed: 0

          
Message 3269 - Posted: 9 Sep 2014, 1:43:05 UTC - in response to Message 3260.

Estimated run time on my computers is 10 to 14 minutes, so I will see how they run.

Conan


Okay that's not too bad, if it's accurate than I can make them 10x longer without much of a problem. Didn't want these to run too long because I wanted to make sure they wouldn't crash and validate correctly.


They are taking just over an hour (between 3,994 and 4,248 seconds) to actually run.

Four so far returned with two pending and two with Validate Errors (along with my wingmen).

Conan

Profile nenym
Send message
Joined: 29 Apr 13
Posts: 4
Combined Credit: 18,838,604
DNA@Home: 1,118,166
SubsetSum@Home: 1,295,535
Wildlife@Home: 16,424,903
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 3272 - Posted: 9 Sep 2014, 2:46:59 UTC - in response to Message 3268.

it's also really strange that they're erroring out. From the tasks themselves it looks like they're returning the same results...
Stderr of invalid task (validate error) doesn't show any error, it seems to be validator bug.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 3273 - Posted: 9 Sep 2014, 2:50:39 UTC - in response to Message 3272.

it's also really strange that they're erroring out. From the tasks themselves it looks like they're returning the same results...
Stderr of invalid task (validate error) doesn't show any error, it seems to be validator bug.


Actually what's happening is the stderr is too long, so it's cutting off information the validator needs to validate things.


Post to thread

Message boards : News : longer DNA@home workunits