[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[modeller_usage] parallelization and the master/slaves' env.io.atom_files_directory?

i'm separating a script that has worked into separate slave and master modules as described in http://salilab.org/modeller/manual/node435.html. 

my master script looks like this:


j = job()
j.append(local_slave())
j.append(local_slave())

for afile in alignFiles:

    alnRunDir = '%s%s/' % (dataDir,aname)
    if not os.path.isdir(alnRunDir):
	print "doAllAlign: Creating directory",alnRunDir
	os.makedirs(alnRunDir)

    for runIdx in range(nruns):
	j.queue_task(SeedTask(argumentList))

for result in j.yield_tasks_unordered():
    print result


and my slave task like this:


class SeedTask(task):
    """A task to read in a PDB file on the slave, and return the resolution"""

    def run(self,dataDir,afile,aname,runIdx,randMethod,knownLbl,seqLbl,nmodPerRun,atomDeviation ):
      env = environ()
    	env.io.atom_files_directory = [dataDir]
    	arg4seed = - (int(runIdx) + 2)
    	env = environ(arg4seed)

    	runDir = '%s/%s/run_%02d/' % (dataDir,aname,runIdx)
    	if not os.path.isdir(runDir):
    		print "doAllAlign: Creating directory",runDir
    		os.makedirs(runDir)

    	os.chdir(runDir)

    	a = automodel(env, alnfile=afile,
    		  knowns=knownLbl, sequence=seqLbl,
    		  assess_methods=(assess.DOPE, assess.GA341))

    	a.starting_model = 1
    	# 2do: test task / run ratio?
    	a.ending_model = nmodPerRun

    	a.deviation = atomDeviation
    	if randMethod=='XYZ':
    		a.rand_method = randomize.xyz
    	elif randMethod=='DIHEDRAL':
    		a.rand_method = randomize.dihedrals

    	a.make()
i'm stepping thru with a debugger, and it gets to the yield_tasks_unordered() generator in the master. but as i attempt to 
make the next step it throws an exception that seems
to have to do with env.io.atom_files_directory now (in the
master/slave version) not getting set correctly?


Running 14 tasks on 2 slaves
/usr/bin/python2.6 /usr/lib/modeller9.10/bin/modslave.py -slave 127.0.1.1:59861:0:BGRESKQY >& seed_align_pp.slave0
/usr/bin/python2.6 /usr/lib/modeller9.10/bin/modslave.py -slave 127.0.1.1:59861:1:ZCQMRSNQ >& seed_align_pp.slave1
Identified slave <Slave on localhost>
Identified slave <Slave on localhost>
Traceback (most recent call last):
  File "/home/rik/Data/pkg/eclipse/plugins/org.python.pydev.debug_2.2.3.2011100616/pysrc/pydevd.py", line 1267, in <module>
    debugger.run(setup['file'], None, None)
  File "/home/rik/Data/pkg/eclipse/plugins/org.python.pydev.debug_2.2.3.2011100616/pysrc/pydevd.py", line 1020, in run
    pydev_imports.execfile(file, globals, locals) #execute the script
  File "/home/rik/Code/eclipse/modeller/src/seed_align_pp.py", line 98, in <module>
    doAllAlign()
  File "/home/rik/Code/eclipse/modeller/src/seed_align_pp.py", line 91, in doAllAlign
    for result in j.yield_tasks_unordered():
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 146, in yield_tasks_unordered
    for task in self._finish_all_tasks():
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 164, in _finish_all_tasks
    task = self._process_event(obj, s)
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/job.py", line 180, in _process_event
    task = obj.task_results()
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/slave.py", line 61, in task_results
    r = self.get_data(allow_heartbeat=True)
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/communicator.py", line 89, in get_data
    (cmdtype, obj) = self._recv()
  File "/usr/lib/python2.6/dist-packages/modeller/parallel/communicator.py", line 130, in _recv
    raise RemoteError(obj.exc, self)
modeller.parallel.communicator.RemoteError: <type 'IOError'>: pdbnam_____E> Filename for PDB code not found: 1A04A.pdb
              Directories:
              Extensions : :.atm:.pdb:.ent:.crd
              (Also tried prepending 'pdb', looking for .Z, .gz, .bz2, .7z, .xz,
              and trying PDB-style subdirectories - e.g. ab for pdb1abc.ent)
 from <Slave on localhost>


would you know what i'm doing wrong here?  thanks again for your help.

	Rik