Next: SGEPEJob() create Up: Parallel job support Previous: Parallel job support Contents Index
This creates a new Job object, used to keep track of multiple worker processes. It is initially empty, but acts just like an ordinary Python list, so you can add or remove Worker objects (see below) using ordinary list operations (e.g., append, del). Also, if you provide a list of suitable worker objects to Job(), they will automatically be added.
Each worker runs a MODELLER process. The system attempts to start this process in the same way as the MODELLER script used for the manager. If the manager is run using your machine's system Python, the worker is started by running 'python modlib/modeller/parallel/modworker.py', while if the manager was started using the 'mod10.3' script, the worker will be too. In some cases, it may get this command line wrong, in which case you can specify the command explicitly using the modeller_path variable. For example, set it to 'mod10.3' to force it to use the version of Python built in to MODELLER rather than the system Python.
Each worker, when started, tries to connect back over the network to the manager node. By default, they try to use the fully qualified domain name of the machine on which you create the Job object (the manager). If this name is incorrect (e.g., on multi-homed hosts) then specify the true hostname with the host parameter. You can also set host to 'localhost' if your machine does not have network connectivity and/or you are running only local workers.
Each worker will run in the same directory as the manager, so will probably fail if you do not have a shared filesystem on all nodes. The output from each worker is written to a logfile called '${JOB}.workerN' where '${JOB}' is info.jobname and 'N' is the number of the worker, starting from zero.
Job.worker_startup_commands is a Python list, initially empty, of Python commands that will be run on each worker when it is started up. You can add your own worker initialization by adding to this list.
Once you have created the job, to use the task interface, submit one or more tasks with Job.queue_task(), and then run the tasks with Job.run_all_tasks() or Job.yield_tasks_unordered().
To use the message-passing interface, first start all workers with Job.start(), and then use Communicator.send_data(), Communicator.get_data() and Worker.run_cmd() to pass messages and commands.
Example: See Job.start(), Job.run_all_tasks() command.