Please bring deadlines of sent tasks to order, server-wise

Message boards : Number crunching : Please bring deadlines of sent tasks to order, server-wise

To post messages, you must log in.

AuthorMessage
Profile Michael H.W. Weber
Avatar

Send message
Joined: 18 Sep 05
Posts: 13
Credit: 6,672,462
RAC: 0
Message 93955 - Posted: 9 Apr 2020, 9:51:58 UTC

What happens since years with Rosetta@home and what is absolutely unacceptable is that Rosetta servers ROUTINELY send out tasks with shorter deadlines than those already running on the same machines.

As a result, the tasks already being worked on are COLLECTIVELY put on hold and the new ones are loaded into RAM to be processed. This not only might lead to RAM issues but causes massive fails in meeting the deadlines. This is especially a problem since with COVID19 you have more and more tasks which require 1.5 days of full computation. But there are many machines which do not run 24/7 including laptops and then there is only a 3 days deadline as demonstrated in this example.

Please correct this problem ASAP - it is easy: Just never hand out tasks with a deadline shorter than that of the previously handed out WU and the problem is solved.

Michael.
President of Rechenkraft.net e.V.

http://www.rechenkraft.net - The world's first and largest distributed computing association. We make those things possible that supercomputers don't.
ID: 93955 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1673
Credit: 17,603,339
RAC: 22,064
Message 93959 - Posted: 9 Apr 2020, 10:36:00 UTC - in response to Message 93955.  

What happens since years with Rosetta@home and what is absolutely unacceptable is that Rosetta servers ROUTINELY send out tasks with shorter deadlines than those already running on the same machines.
When?
All the Tasks i could see had a deadline of 4 days.


This is especially a problem since with COVID19 you have more and more tasks which require 1.5 days of full computation.
Check your settings. The default is 8 hours.
All of my Tasks run for around 8 hours (some quite a bit less, some 5min longer). There were a batch of Rosetta Mini Tasks that ran for 16 hours about a week or so ago, but that didn't last for long at all before they went back to the default of 8


never hand out tasks with a deadline shorter than that of the previously handed out WU and the problem is solved.
Then the problem is solved.
I've not seen a Task with a deadline shorter than 4 days since i joined the project (which was only 2 week s ago).

There were some Tasks that errored out straight away, and some new applications released over the last few days. That will result in Estimated completion times to be shorter than the actual processing time, so that will result in overfull caches till those Estimates settle down again.
If running more than one project, it's best to run with a small cache (Say one day & 0.02 extra) . If running more than a couple of projects, it's best to run with a very small cache (say 0.5 days & 0.02 days extra).
Grant
Darwin NT
ID: 93959 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
CIA

Send message
Joined: 3 May 07
Posts: 100
Credit: 21,059,812
RAC: 0
Message 93972 - Posted: 9 Apr 2020, 13:11:56 UTC - in response to Message 93959.  


If running more than one project, it's best to run with a small cache (Say one day & 0.02 extra) . If running more than a couple of projects, it's best to run with a very small cache (say 0.5 days & 0.02 days extra).


It took me awhile, but I've finally come around to the "shortest buffer possible" camp. Yes there's been some downtime as the researchers scrambled with the "problem" of having too many new users and the increase in computing power, but they seem to have that sorted now. I have internet 24/7 so I just keep .5 days of work and no additional work. The project gets results faster, and I don't have to worry about too many tasks in my buffer with odd deadlines.
ID: 93972 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 93997 - Posted: 9 Apr 2020, 20:08:29 UTC - in response to Message 93959.  

Then the problem is solved.
I've not seen a Task with a deadline shorter than 4 days since i joined the project (which was only 2 week s ago).


The normal is (I'm forgetting) an 8 or 10 day deadline. So they are talking about having a cache of deadlines a week away and then getting new work with 3 day deadlines.

Any preempted tasks will get pushed to swap space though, so I don't see a memory contention issue.

Taking 1.5 days to run is a bit of a concern. ...unless you selected the new maximum runtime preference. Any time you bump up the runtime preference, BOINC Manager will request too much work to fill the cache defined. So that may be part of the concern here too. I've always suggested reducing cache preference and only bumping runtime preference a notch or two at a time to avoid the problem of getting too many WUs.
Rosetta Moderator: Mod.Sense
ID: 93997 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Please bring deadlines of sent tasks to order, server-wise



©2024 University of Washington
https://www.bakerlab.org