loop modeling parallel-queue
Hi,
Let's suppose we are sending a protein to refine four of its loops, eight repeats (32 loops).
If I spawn eight Slaves (job().append()), a single model is sent to each slave and each slave compute the four loops.
Now suppose we are sending the same protein, to refine four of its loops just one time (4 loop). Spawning the same eight Slaves, the model is sent to just one Slave, and that slave compute all of the four loops alone (i.e. modeller is not parallelizing).
My question is: is there any difference between a) sending the eight jobs from within the modeller script or b) sending one job after another to a third queue software initializing the enviroment with
env = environ(0 - random.randrange(2, 50000)) #A different random seed each time.
or is modeller taking into account previous results?
Thanks in advance,
Hi,
I had recently encountered a similar problem. There is also another issue, when running 8 jobs concurrently in the same directory each thread needs some input files (in my understanding) which are deleted or modified by some other threads and that leads to deadlock. The solution to this problem in to run each thread in a different directory. I am pasting my scripts to save you from trouble. The script will find how many CPUs are available on your machine and spawn a job to each one. To launch the loop modeling type "python loop_parallel.py". Of course you will need to modify the scripts according to your needs, and I think MODELLER libraries must be in your PYTHONPATH (read the installation instructions how to do this). I will make them neater and as from the MODELLER caretaker to put the in the script library.
Script name "loop_parallel.py":
#!/usr/bin/env python > > from modeller import * > from modeller.scripts import complete_pdb > from modeller.parallel import * > # Load in my task from mytask.py (note: needs to be in a separate Python > # module like this, in order for Python's pickle module to work correctly) > from build_model_class import Build_Model > import sys, re, os, shutil > import multiprocessing > > > INITIAL_CONFORMATION = "initial_loop1_conformation" > > > log.verbose() > env = environ() > env.io.hetatm = True > > CURRENT_DIR = os.getcwd() > # directories for input atom files > env.io.atom_files_directory = ['.'] > > j = job() > for cpu in range(0, multiprocessing.cpu_count()): > j.append(local_slave()) > if not os.path.exists(CURRENT_DIR + "/results_" + str(cpu+1)): > os.mkdir(CURRENT_DIR + "/results_" + str(cpu+1)) > shutil.copy(INITIAL_CONFORMATION+".pdb", CURRENT_DIR + "/results_" + > str(cpu+1)+"/") > start = 1 > offset = 374 ## the number of loop models -1 each thread will create > for n in range(1,9): > end = start+offset > print start, end > j.queue_task(Build_Model("initial_loop1_conformation", > "p110a_loop1_model", start, end, CURRENT_DIR + "/results_"+str(n))) > start = end+1 > results = j.run_all_tasks() >
Script name "Build_model_class.py". Put it in the same directory as loop_parallel.py:
#!/usr/bin/env python > from modeller import * > from modeller.automodel import * > from modeller.parallel import * > from modeller.scripts import complete_pdb > import os, sys, glob > import multiprocessing > > > > > class MyLoopModel(loopmodel): > # This routine picks the residues to be refined by loop modeling > def select_loop_atoms(self): > return selection(self.residue_range('148:', '165:')) > > def special_restraints(self, aln): > rsr = self.restraints > # Keep whole model but loop atoms rigid_bodies > wholeSel = selection(self) - self.select_loop_atoms() > r = rigid_body(wholeSel) > rsr.rigid_bodies.append(r) > > > > > class Build_Model(task): > """A task to build models with MODELLER""" > def run(self, initial_structure, output_model_name, start, end, > OUT_DIR): > os.chdir(OUT_DIR) > log.verbose() > env = environ() > env.io.hetatm = False > # scale up sheet hydrogen bond weights > #env.schedule_scale = physical.values(default=1.0, h_bond=100.0) > # directories for input atom files > env.io.atom_files_directory = ["."] > a = MyLoopModel(env, > inimodel=initial_structure, > sequence=output_model_name) # code of the target > a.loop.starting_model = start # First loop model > a.loop.ending_model = end # Last loop model > a.loop.md_level = refine.very_slow # Loop model refinement > level > a.make() > a.cluster(cluster_cut=1.5) ## cluster all of the output models, and > output an averaged cluster structure, both optimized (in the file > cluster.opt) and unoptimized (in cluster.ini) > ### remove redundant files to save disk space > #filepatterns = [".rsr", ".V*", ".D*", ".ini", ".sch", ".sch"] > #for filepattern in filepatterns: > # for filename in glob.glob(initial_structure+filepattern): > # os.remove(filename) > return >
Script name "assess_dope.py", to assess the created loop models with the DOPE method. To run it type "python assess_dope.py"
# -*- coding: utf-8 -*- > # Example for: selection.assess_dope() > > from modeller import * > from modeller.scripts import complete_pdb > import glob > > output_model_name="p110a_loop1_model" > > > env = environ() > env.libs.topology.read(file='$(LIB)/top_heav.lib') > env.libs.parameters.read(file='$(LIB)/par.lib') > > # Read a model previously generated by Modeller's automodel class > fout = open(output_model_name + "_dopehr_scores.txt", 'w') > for modelname in glob.glob("results_*/"+output_model_name+".BL*.pdb"): > print "Evaluating model ", modelname > mdl = complete_pdb(env, modelname) > # Select all atoms in the first chain > atmsel = selection(mdl.chains[0]) > score = atmsel.assess_dopehr() > print "DOPE-HR score for model ", modelname, " is " + str(score) > fout.write("DOPE-HR score for model " + modelname + " is " + str(score) + > "\n") >
good luck, Thomas
On 26 January 2011 12:53, David Rodríguez david.rguez.diaz@gmail.comwrote:
> Hi, > > Let's suppose we are sending a protein to refine four of its loops, eight > repeats (32 loops). > > If I spawn eight Slaves (job().append()), a single model is sent to each > slave and each slave compute the four loops. > > Now suppose we are sending the same protein, to refine four of its loops > just one time (4 loop). Spawning the same eight Slaves, the model is sent to > just one Slave, and that slave compute all of the four loops alone (i.e. > modeller is not parallelizing). > > My question is: is there any difference between a) sending the eight jobs > from within the modeller script or b) sending one job after another to a > third queue software initializing the enviroment with > > env = environ(0 - random.randrange(2, 50000)) #A different random seed each > time. > > or is modeller taking into account previous results? > > Thanks in advance, > > -- > > David Rodríguez Díaz, PhD Student > Fundación Pública Galega de Medicina Xenómica (SERGAS) > Santiago de Compostela (Spain) > http://webspersoais.usc.es/persoais/david.rodriguez.diaz > > http://webspersoais.usc.es/persoais/david.rodriguez.diaz > > _______________________________________________ > modeller_usage mailing list > modeller_usage@salilab.org > https://salilab.org/mailman/listinfo/modeller_usage > >
On 1/26/11 3:43 AM, Thomas Evangelidis wrote: > I had recently encountered a similar problem. There is also another > issue, when running 8 jobs concurrently in the same directory each > thread needs some input files (in my understanding) which are deleted or > modified by some other threads and that leads to deadlock.
I don't think that's the question David was asking. But what you say is not correct - 8 jobs will run concurrently in the same directory (in fact it is designed to work that way, since they will need the same initial conformation input file). The parallel loop modeling should not delete or modify any input files, so I don't know why you're seeing a deadlock - we have not seen such behavior here. Perhaps some problem with your network storage? The parallel framework also does not use threads - it uses independent processes - and that together with the master-slave design makes a deadlock (where A is waiting for B, but B is also waiting for A) rather unlikely.
That said, the scripts you attach will get rather confused if you run them in the same directory, because they are rather less efficient than the parallel loop modeling included with Modeller. You are asking each slave to build the set of restraints and initial model before building the loop models. Obviously if one slave overwrites this initial model while another slave is trying to read it, things will get confused (but a deadlock should still not occur - the slave will simply raise an exception). But since the initial model and restraints are always the same, this is not necessary. The extra computational effort (and the potential for confusion) can be simply avoided by building the restraints and initial model on the master node - then all the slaves do is read them in and generate one or more loop models. This is in fact what happens if you call loopmodel's "use_parallel_job" method before you call make().
Ben Webb, Modeller Caretaker
On 26 January 2011 18:14, Modeller Caretaker modeller-care@salilab.orgwrote:
> On 1/26/11 3:43 AM, Thomas Evangelidis wrote: > >> I had recently encountered a similar problem. There is also another >> issue, when running 8 jobs concurrently in the same directory each >> thread needs some input files (in my understanding) which are deleted or >> modified by some other threads and that leads to deadlock. >> > > I don't think that's the question David was asking. But what you say is not > correct - 8 jobs will run concurrently in the same directory (in fact it is > designed to work that way, since they will need the same initial > conformation input file). The parallel loop modeling should not delete or > modify any input files, so I don't know why you're seeing a deadlock - we > have not seen such behavior here. Perhaps some problem with your network > storage? The parallel framework also does not use threads - it uses > independent processes - and that together with the master-slave design makes > a deadlock (where A is waiting for B, but B is also waiting for A) rather > unlikely. > > That said, the scripts you attach will get rather confused if you run them > in the same directory, because they are rather less efficient than the > parallel loop modeling included with Modeller. You are asking each slave to > build the set of restraints and initial model before building the loop > models. Obviously if one slave overwrites this initial model while another > slave is trying to read it, things will get confused (but a deadlock should > still not occur - the slave will simply raise an exception). But since the > initial model and restraints are always the same, this is not necessary. The > extra computational effort (and the potential for confusion) can be simply > avoided by building the restraints and initial model on the master node - > then all the slaves do is read them in and generate one or more loop models. > This is in fact what happens if you call loopmodel's "use_parallel_job" > method before you call make().http://salilab.org/mailman/listinfo/modeller_usage
Yes the exception I was getting was that *.ini file couldn't be found. I tried initially "use_parallel_job" to model loops on a master node from an initial conformation, but MODELLER wasn't scaling when trying to create many loop models. I have 8 CPUs on my local machine and only 1 was running. Surprisingly it did scale when I tried to create just a few loop models, i.e. 16. Since I couldn't invest more that 1 day in debugging I decided to do it by creating one directory for every slave node.
I do have a collection of scripts that does homology modelling and loop modeling with the dopehr_loopmodel class using this "use_parallel_job" method, but I couldn't adapt it to do just loop modelling from an initial conformation...weird!
On 1/26/11 2:53 AM, David Rodríguez wrote: > My question is: is there any difference between a) sending the eight > jobs from within the modeller script or b) sending one job after another > to a third queue software initializing the enviroment with > > env = environ(0 - random.randrange(2, 50000)) #A different random seed > each time. > > or is modeller taking into account previous results?
Tasks in Modeller's parallel system are completely independent, so no, it shouldn't make any difference whether you run all 8 tasks on the same machine or you run them on 8 different machines. You can also manually break up the jobs as you describe - it shouldn't make any difference - that is essentially what the parallel loop modeling does already (run the same calculation but with a different random seed). I wouldn't recommend setting a random random seed though - that will make your results non-reproducable. Just set
env = environ(-2 - number_of_task)
so that each task gets a different (but known) random seed.
Ben Webb, Modeller Caretaker
participants (3)
-
David Rodríguez -
Modeller Caretaker -
Thomas Evangelidis