Parallel job support

The parallel module provides methods to parallelize MODELLER calculations over multiple processors. This may be employed to make use of more than one CPU on a multi-processor machine, or of several nodes in a cluster. Each remote process is termed a ‘worker’, while the process which starts and controls the parallel job is termed the ‘manager’.

Two methods for accessing the workers from the manager are provided; the first is a task-based interface (Job.queue_task(), Job.run_all_tasks() and Job.yield_tasks_unordered()) which is largely transparent in use and fault-tolerant, while the second is a lower-level worker-manager message-passing interface (Job.start(), Communicator.send_data(), Communicator.get_data() and Worker.run_cmd()), which is similar to MPI (although coarse-grained) and requires you to handle errors yourself. It is recommended that you use the task interface unless you require a large amount of message passing.

This module relies on several modules in the Python standard library. These modules are not provided with MODELLER, so you must additionally install Python if it is not already present on your system. It is also experimental, and may need modification to work with your own networking or parallel system.