How do I use multiple cpu when doing loop which I have shown in a script below. I am trying to use several cpus to get a faster build of a model.
Knut J
rom modeller import * from modeller.automodel import * from modeller.parallel import *
env = environ()
j= job() j.append(local_slave()) #cpu 1 j.append(local_slave()) #cpu 2 j.append(local_slave()) #cpu 3 j.append(local_slave()) #cpu 3
j.start()
class Model(automodel): def select_atoms(self): return selection(self.residue_range('1:','180:'))
a=Model(env,alnfile='hNaa10p.ali', knowns='2OB0', sequence='hnaa10', assess_methods=assess.DOPE)
a.starting_model=1 a.ending_model=200 a.use_parallel_job(j) a.make()
# Get a list of all successfully built models from a.outputs ok_models = filter(lambda x: x['failure'] is None, a.outputs)
# Rank the models by DOPE score key = 'DOPE score' ok_models.sort(lambda a,b: cmp(a[key], b[key]))
# Get top model m = ok_models[0] print "Top model: %s (DOPE score %.3f)" % (m['name'], m[key])
On 01/20/2010 01:14 PM, Knut J Bjuland wrote: > How do I use multiple cpu when doing loop which I have shown in a script > below. I am trying to use several cpus to get a faster build of a model.
Your script looks fine to me, and should run in parallel, except that you're using the automodel class (not loopmodel) so you're doing regular homology modeling, not loop modeling.
Ben Webb, Modeller Caretaker
Hi I got this output when I ran the script. Is there any python library that I have not installed. I am using python 2.6 in ubuntu 2.6 with all numpy,scipy and scitools installed. I tried to use inheritance to get my class to also import parallel.
Knut J
MODELLER 9v7, 2009/06/12, r6923
PROTEIN STRUCTURE MODELLING BY SATISFACTION OF SPATIAL RESTRAINTS
Copyright(c) 1989-2009 Andrej Sali All Rights Reserved
Written by A. Sali with help from B. Webb, M.S. Madhusudhan, M-Y. Shen, M.A. Marti-Renom, N. Eswar, F. Alber, M. Topf, B. Oliva, A. Fiser, R. Sanchez, B. Yerkovich, A. Badretdinov, F. Melo, J.P. Overington, E. Feyfant University of California, San Francisco, USA Rockefeller University, New York, USA Harvard University, Cambridge, USA Imperial Cancer Research Fund, London, UK Birkbeck College, University of London, London, UK
Kind, OS, HostName, Kernel, Processor: 4, Linux Human-Naa50p 2.6.31-17-generic x86_64 Date and time of compilation : 2009/06/12 12:23:44 MODELLER executable type : x86_64-intel8 Job starting time (YY/MM/DD HH:MM:SS): 2010/01/21 09:51:09
read_to_681_> topology.submodel read from topology file: 3 automodel__W> Topology and/or parameter libraries already in memory. These will be used instead of the automodel defaults. If this is not what you want, clear them before creating the automodel object with env.libs.topology.clear() and env.libs.parameters.clear()
check_ali___> Checking the sequence-structure alignment.
Implied target CA(i)-CA(i+1) distances longer than 8.0 angstroms:
ALN_POS TMPL RID1 RID2 NAM1 NAM2 DIST ---------------------------------------------- END OF TABLE
getf_______W> RTF restraint not found in the atoms list: residue type, indices: 16 235 atom names : C +N atom indices : 1851 0
getf_______W> RTF restraint not found in the atoms list: residue type, indices: 16 235 atom names : C CA +N O atom indices : 1851 1848 0 1852 mdtrsr__446W> A potential that relies on one protein is used, yet you have at least one known structure available. MDT, not library, potential is used. 0 atoms in HETATM residues constrained to protein atoms within 2.30 angstroms and protein CA atoms within 10.00 angstroms 0 atoms in residues without defined topology constrained to be rigid bodies condens_443_> Restraints marked for deletion were removed. Total number of restraints before, now: 16251 14565 1468 (of 1853 total) atoms selected for optimization Running 20 tasks on 3 slaves /usr/bin/python2.6 /usr/lib/modeller9v7/bin/modslave.py -slave 127.0.1.1:17153:0:NVOBZUNU >& build_model_single.slave0 /usr/bin/python2.6 /usr/lib/modeller9v7/bin/modslave.py -slave 127.0.1.1:17153:1:MKBZURFK >& build_model_single.slave1 /usr/bin/python2.6 /usr/lib/modeller9v7/bin/modslave.py -slave 127.0.1.1:17153:2:TGSVVIND >& build_model_single.slave2 Identified slave <Slave on localhost> Identified slave <Slave on localhost> Identified slave <Slave on localhost> Traceback (most recent call last): File "/home/knut/workspace/natAhnaa10/src/build_model_single.py", line 38, in <module> a.make() File "/usr/lib/python2.6/dist-packages/modeller/automodel/automodel.py", line 109, in make self.multiple_models(atmsel) File "/usr/lib/python2.6/dist-packages/modeller/automodel/automodel.py", line 212, in multiple_models self.parallel_multiple_models(atmsel) File "/usr/lib/python2.6/dist-packages/modeller/automodel/automodel.py", line 232, in parallel_multiple_models self.outputs.extend(job.run_all_tasks()) File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 127, in run_all_tasks for task in self._finish_all_tasks(): File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 160, in _finish_all_tasks task = self._process_event(obj, s) File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 176, in _process_event task = obj.task_results() File "/usr/lib/python2.6/dist-packages/modeller/parallel/slave.py", line 61, in task_results r = self.get_data(allow_heartbeat=True) File "/usr/lib/python2.6/dist-packages/modeller/parallel/communicator.py", line 84, in get_data (cmdtype, obj) = self._recv() File "/usr/lib/python2.6/dist-packages/modeller/parallel/communicator.py", line 125, in _recv raise RemoteError(obj.exc, self) modeller.parallel.communicator.RemoteError: <type 'AttributeError'>: 'module' object has no attribute 'KJ' from <Slave on localhost>
> Date: Wed, 20 Jan 2010 14:26:01 -0800 > From: modeller-care@salilab.org > To: knutjbj@hotmail.com > CC: modeller_usage@salilab.org > Subject: Re: [modeller_usage] muliplay cpu > > On 01/20/2010 01:14 PM, Knut J Bjuland wrote: > > How do I use multiple cpu when doing loop which I have shown in a script > > below. I am trying to use several cpus to get a faster build of a model. > > Your script looks fine to me, and should run in parallel, except that > you're using the automodel class (not loopmodel) so you're doing regular > homology modeling, not loop modeling. > > Ben Webb, Modeller Caretaker > -- > modeller-care@salilab.org http://www.salilab.org/modeller/ > Modeller mail list: http://salilab.org/mailman/listinfo/modeller_usage _________________________________________________________________ Windows Live: Friends get your Flickr, Yelp, and Digg updates when they e-mail you. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/soc...
On 1/21/10 12:59 AM, Knut J Bjuland wrote: > Hi I got this output when I ran the script. ... > I tried to use inheritance to get my > class to also import parallel. ... > modeller.parallel.communicator.RemoteError: <type 'AttributeError'>: > 'module' object has no attribute 'KJ' from <Slave on localhost>
When you define your own classes, you must do so in a separate Python file from your main script (e.g. put them in another file called 'myclasses.py' and at the top of your main script say "from myclasses import *") This is a consequence of the way Python's "pickle" module works (which is used by Modeller's parallel module to send objects between different CPUs).
See also http://salilab.org/modeller/9v7/manual/node412.html
Ben Webb, Modeller Caretaker
participants (2)
-
Knut J Bjuland -
Modeller Caretaker