Table Of Contents
Previous topic
gromacs.qsub – utilities for batch submission systems
Next topic
gromacs.run – Running simulations
gromacs.qsub – utilities for batch submission systems
gromacs.run – Running simulations
Warning
Experimental code, use at your own risk.
The Manager encapsulates knowledge of how to run a job on a remote system. This basically means the host name, where are the scratch directories, what queuing system is running, etc. It simply uses ssh to communicate with the remote system and thus it is necessary to set up ssh with public key authentication to make this work smoothly.
The remote directory name is constructed in the following way:
See Manager for how the values in the configuration file are used.
Example:
[DEFAULT]
name = leviathan
[local]
topdir = ~
[remote]
hostname = leviathan.petagrid.org
scratchdir = /scratch/username/Projects
[queuing_system]
name = PBS
qscript = leviathan.pbs
walltime = 24.0
start_cwd = True
All entries except walltime and start_cwd are required; walltime can be omitted or set to None.
name
identifier of the configuration; should also be the name of the configuration file, i.e. name.cfg
hostname
fully qualified domain name of the host; used for running ssh hostname or scp FILES hostname:DIR
scratchdir
top level directory on the remote host udner which the working directories are constructed; see below for how this is done
name
identifier for the queuing system (should be a valid python identifier)
qscript
default queuing system script template; store it in ~/.gromacswrapper/qscripts
walltime
maximum allowed run time on the system; job files are written in such a way that Gromacs stops run at 0.99 or walltime. If omitted then the job runs until it is done (provided the queuing system policy allows that)
start_cwd
Set to True means that the queuing system requires the queuing system script to cd into the job directory; this seems to be a bug in some versions of PBS, which we can work-around in Manager.qsub()
The configuration files are stored in the ~.gromacswrapper/manager directory. A file named “foo.cfg” corresponds to the manager named “foo”.
The Manager class must be customized for each system such as a cluster or a super computer through a cfg file (see Configuration file). It then allows submission and control of jobs remotely (using ssh).
Base class to launch simulations remotely on computers with queuing systems.
Basically, ssh into machine and run job.
The manager is configured through a cfg file “name.cfg”, whose format is described in Configuration file.
If a special job submission procedure is required then a class must be dreived that implements a specialized Manager.qsub() method.
ssh must be set up (via ~/.ssh/config) to allow access via a commandline such as
ssh <hostname> <command> ...
Typically you want something such as
host <hostname>
hostname <hostname>.fqdn.org
user <remote_user>
in ~/.ssh/config and also set up public-key authentication in order to avoid typing your password all the time.
Set up the manager.
| Arguments : |
|
|---|
alias for get_status()
alias for get_status()
Concatenate parts of a run in dirname.
Always uses gromacs.cbook.cat() with resolve_multi = ‘guess’.
Note
The default is to immediately delete the original files (cleanup = True).
| Keywords : |
|
|---|
scp -r dirname from host into targetdir
| Arguments : |
|
|---|---|
| Returns : | return code from scp |
Directory on the remote machine.
Check status of remote job by looking into the logfile.
Report on the status of the job and extracts the performance in ns/d if available (which is saved in Manager.performance).
| Arguments : |
|
|---|---|
| Returns : | True is job is done, False if still running None if no log file found to look at |
Note
Also returns False if the connection failed.
Warning
This is an important but somewhat fragile method. It needs to be improved to be more robust.
Find checkfile locally if possible.
If checkfile is not found in dirname then it is transferred from the remote host.
If needed, the trajectories are concatenated using Manager.cat().
| Returns : | local path of checkfile |
|---|
Regular expression used by Manager.get_status() to parse the logfile from mdrun.
Calculate how many dependent (chained) jobs are required.
Uses performance in ns/d (gathered from get_status()) and job max walltime (in hours) from the class unless provided as keywords.
n = ceil(runtime/(performance*0.99*walltime)
| Keywords : |
|
|---|---|
| Returns : | n or 1 if walltime is unlimited |
scp dirname to host.
| Arguments : | dirname to be transferred |
|---|---|
| Returns : | return code from scp |
scp filename to host in dirname.
| Arguments : | filename and dirname to be transferred to |
|---|---|
| Returns : | return code from scp |
Submit job remotely on host.
This is the most primitive implementation: it just runs the commands
cd remotedir && qsub qscript
on Manager._hostname. remotedir is dirname under Manager._scratchdir and qscript is the name of the queuing system script in remotedir.
| Arguments : |
|
|---|
Directory on the remote machine.
URI of the directory on the remote machine.
Set up production and transfer to host.
| Arguments : |
|
|---|
Set up position restraints run and transfer to host.
kwargs are passed to gromacs.setup.MD_restrained()
Wait until the job associated with dirname is done.
Super-primitive, uses a simple while ... sleep for seconds delay
| Arguments : |
|
|---|
The actual config file contents can be retrieved with get_manager_config().
Load manager configuration file from filename.
The following classes and functions are mainly documented for developers.
Find a configuration file for manager name.
Create a new section in the configuration.
Raise DuplicateSectionError if a section by the specified name already exists. Raise ValueError if name is DEFAULT or any of it’s case-insensitive variants.
Get an option value for a given section.
If `vars’ is provided, it must be a dictionary. The option is looked up in `vars’ (if provided), `section’, and in `defaults’ in that order.
All % interpolations are expanded in the return values, unless the optional argument `raw’ is true. Values for interpolation keys are looked up in the same manner as the option.
The section DEFAULT is special.
Return option as an expanded path.
Check for the existence of a given option in a given section.
Indicate whether the named section is present in the configuration.
The DEFAULT section is not acknowledged.
Return a list of tuples with (name, value) for each option in the section.
All % interpolations are expanded in the return values, based on the defaults passed into the constructor, unless the optional argument `raw’ is true. Additional substitutions may be provided using the `vars’ argument, which must be a dictionary whose contents overrides any pre-existing defaults.
The section DEFAULT is special.
Return a list of option names for the given section name.
Read and parse a filename or a list of filenames.
Files that cannot be opened are silently ignored; this is designed so that you can specify a list of potential configuration file locations (e.g. current directory, user’s home directory, systemwide directory), and all existing configuration files in the list will be read. A single filename may also be given.
Return list of successfully read files.
Like read() but the argument must be a file-like object.
The `fp’ argument must have a `readline’ method. Optional second argument is the `filename’, which if not given, is taken from fp.name. If fp has no `name’ attribute, `<???>’ is used.
Remove an option.
Remove a file section.
Return a list of section names, excluding [DEFAULT]
Set an option. Extend ConfigParser.set: check for string values.
Write an .ini-format representation of the configuration state.
Properties of a job.