Waste of resource.

Message boards : Number crunching : Waste of resource.

To post messages, you must log in.

AuthorMessage
Profile adrianxw
Avatar

Send message
Joined: 18 Sep 05
Posts: 653
Credit: 11,840,739
RAC: 28
Message 106086 - Posted: 27 Apr 2022, 7:50:20 UTC
Last modified: 27 Apr 2022, 7:53:22 UTC

As I understand things, a work unit has "a problem", it generates a random number then uses the program to run the model with that random start point. If it finishes within the time limit set by the user, it generates a new random number and repeats.
Thinking about that, setting a long run time may not be a good idea.
Rational, the project has estimated the number of runs needed to produce the result they need. They estimate the number of work units required to achieve such a result, (they can estimate this from in house runs on their own machines). They sit back and wait.
By setting a long run time, I generate more results per work unit on both of my machines. Others are quite possibly doing the same thing. The number of runs required when the initial estimate was made is reached earlier. Are further work units then not sent, or are they left in the queue to, basically, waste the resource?
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 106086 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1685
Credit: 17,975,622
RAC: 23,468
Message 106087 - Posted: 27 Apr 2022, 8:44:43 UTC - in response to Message 106086.  

Are further work units then not sent, or are they left in the queue to, basically, waste the resource?
Each Task is different data, so it is worth doing all the other Tasks.
The default from the project is 8 hours as that is what they have determined gives them a useful result from the available hardware.

If running a Task for longer makes you happy, then no problem, the extra work is still of benefit. But it's not necessary.

However the fact that your system is over committed and it takes you over 16 hours to do 12 hours of actual processing really is just a waste of your computing time & power. 4 hours of computing time that's not used to actually compute anything is a waste of of resources IMHO.
Grant
Darwin NT
ID: 106087 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 393
Credit: 12,114,842
RAC: 3,702
Message 106088 - Posted: 27 Apr 2022, 9:25:17 UTC - in response to Message 106086.  

As I understand things, a work unit has "a problem", it generates a random number then uses the program to run the model with that random start point. If it finishes within the time limit set by the user, it generates a new random number and repeats.
Thinking about that, setting a long run time may not be a good idea.
Rational, the project has estimated the number of runs needed to produce the result they need. They estimate the number of work units required to achieve such a result, (they can estimate this from in house runs on their own machines). They sit back and wait.
By setting a long run time, I generate more results per work unit on both of my machines. Others are quite possibly doing the same thing. The number of runs required when the initial estimate was made is reached earlier. Are further work units then not sent, or are they left in the queue to, basically, waste the resource?


I agree, they set the default for a reason - we should not change it without a better reason.
ID: 106088 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 209
Credit: 26,087,497
RAC: 16,725
Message 106192 - Posted: 14 May 2022, 6:24:59 UTC - in response to Message 106086.  

This number of "runs" it really big, like from tens of thousands to hundreds of thousands runs per model, in some cases it can be more than a million. The fact that we increase the number of runs in one task by increasing the target computation time does not affect anything - the total number is regulated by the number of computed and validated results on server side.
When scientists decide that they have already received enough runs for this particular model/goal, they simply cancel all remaining tasks of the same series that are still in the queue. Including ones that have been already downloaded to volunteer PCs (probably you have already noticed this sometime - when a part of tasks from a series that worked without problems / errors is suddenly canceled by a command from the server - "server abort" in WUs status).


P.S
And Well, in any case, even if we generate some "excessive" result it will not be a complete loss of resources. Because with this pseudo-random search approach used in R@H, more runs/passes = better (more accurate) result become. The cut-off point (when enough is enough to say) is rather arbitrary. It is usually chosen when the improvement in accuracy is already quite insignificant and it is not practical to continue to spend large amounts of computing power on it. But insignificant doesn't mean they don't exist at all.
ID: 106192 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Waste of resource.



©2024 University of Washington
https://www.bakerlab.org