Re: [modeller_usage] Multicore Jobs on QB3 Cluster Nodes

[Modeller Caretaker <>]

Albion Baucom wrote:
> How would I go about passing arguments to sge_qsub_job to get slave  
> node jobs to use multiple cores on the QB3 cluster?

That depends on how you reserved those cores. With SGE there are two
ways of doing this that we commonly use with Modeller:
1. Request a parallel environment (PE) in your SGE job script to reserve
several cores (-pe option). Then in your Modeller script (which runs on
the first of these cores) use the sge_pe_job class to create a slave
task on each of the cores.

2. Run a single-core job, then in your Modeller script use sge_qsub_job
to use qsub to start one or more *additional* array jobs (using the -t
option; not a PE job) each containing one or more tasks that talk to the
master.

The advantage of the first method is that you are guaranteed to get all
your cores at once, while with the second you can dynamically expand the
number of cores you need at runtime (subject to load on the cluster(s),
of course).

> I am currently using the following Modeller directive to submit  
> multiple jobs to the cluster
> 
> j = sge_qsub_job(maxslave=jobparas.maxslave,options  
> =jobparas.optionline,modeller_path=jobparas.modeller_path)
> 
> where
> 
> jobparas = jobparameters(maxslave=5, arch='opt64', queue='long.q',  
> name='htpg',
>                           modeller_path="/diva1/home/modeller/SVN/bin/ 
> modpy.sh \
>                                          /diva1/home/modeller/SVN/bin/ 
> modslave.py")
> 
> My question, is how do I pass the arguments "-pe smp 4" and "-R yes"  
> for instance? Looking at the Modeller documentation, its not clear how  
> those parameters could be passed as arguments to sge_qsub_job in this  
> instance.

If you are running a PE job, use the sge_pe_job class, not sge_qsub_job.
In this case Modeller does not need to start a second SGE job and thus
there is no need to pass SGE options to it. If you take the second
option and use sge_qsub_job to start tasks, the "options" argument is
where you specify SGE options (e.g. options="-l opt64 -q long.q"). I
can't tell you how to use jobparameters though because that
class/function is not ours and not part of Modeller - you should ask
whoever wrote it.

	Ben Webb, Modeller Caretaker
-- 
             http://www.salilab.org/modeller/
Modeller mail list: http://salilab.org/mailman/listinfo/modeller_usage