|
From: | Christian Meesters |
Subject: | Re: file permissions on joblog |
Date: | Thu, 28 Jul 2022 08:44:49 +0200 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 |
Hi,not quite. Under SLURM the jobstep starter (SLURM lingo) is "srun". You do not do ssh from job host to job host, but rather use "parallel" as a semaphore avoiding over subscription of job steps with "srun". I summarized this approach here:
https://mogonwiki.zdv.uni-mainz.de/dokuwiki/start:working_on_mogon:workflow_organization:node_local_scheduling#running_on_several_hosts (uh-oh - I need to clean up that site, many outdated sections there, but this one should still be ok)
One advantage: you can safely utilize the resources of both (or more) hosts - the master hosts and all secondaries. How much resources you require depends on your application and the work it does. Be sure to consider I/O (e.g. stage-in file to avoid random I/O with too many concurrent applications, etc.), if this is an issue for your application.
Cheers Christian On 7/28/22 08:24, Ole Tange wrote:
On Tue, Jul 26, 2022 at 2:49 AM Rob Sargent <robjsargent@gmail.com> wrote:I'm using GNU Parallel behind not one but two intermediaries. I submit to the first, the first wraps the jobs in a "slurm" script and submits the jobs to the second (the slurm host). The jobs run on the slurm host under the account of the first intermediary, username and group not accessible to me. I would like to be able to view the --joblog file but its permissions are 640.I take it that the problem is similar to: ssh otheruser@localhost "parallel --joblog foo ..." Try: mkdir /tmp/foo; touch /tmp/foo/log; chmod 666 /tmp/foo/log ssh otheruser@localhost "parallel --joblog /tmp/foo/log ..." If you have access to a database: parallel -Sotheruser@localhost --sqlandworker pg://user:pass@host/db/tbl 'id;echo' ::: 1 sql pg://user:pass@host/db/tbl "select * from tbl;" /Ole
[Prev in Thread] | Current Thread | [Next in Thread] |