- IMP-dev - salilab.org

RMF backward compatibility
by Daniel Russel 27 Jul '12

27 Jul '12

I removed some code from the RMF library that provided backward compatibility with certain rather old files (1 year+ I think). If you run into any files that don't open after you update IMP and they matter, I can put the code back (I rearranged some functions and would have had to update the code). Alternatively, if you know you have some old files, you can do rmf_slice old.rmf new.rmf -f -1 which will update the files into new.rmf (the -f -1 just means move every frame).

1 0

IMP documentation & tutorials by the people
by Barak Raveh 18 Jul '12

18 Jul '12

Hey guys, Here is a very tempting suggestion, which might benefit all of us in the long run, but will require some group effort :) As we all know, though lots of efforts have been made to document IMP, getting started on IMP development or usage is (quite naturally) still somewhat far from obvious (missing or outdated documentation or examples, no tutorials etc.). So - the idea is that if we team up, we can improve the documentation and write tutorials in relatively small effort, benefiting new users and developers, and surely also old ones. We can then have peer-reviewing of the resulting documentation (e.g., one team checking the documentation / tutorials written by another team). Here are some initial ideas Daniel and I had in mind, and feel free to add suggestions: * A team will make sure the Getting Started and Developer Guide are up to date, make them even friendlier and extend them further * Other teams will validate existing example codes, add missing example codes, and most important, write friendly tutorials on how to accomplish common IMP tasks (both development, e.g., "how to write new code for Monte-Carlo simulation with restraints", and usage e.g. "how to dock from EM") * Perhaps one team can also go over some key classes and improve their code documentation, especially in the core / kernel / base / atom modules. Ok, Who's in? :) We will try to keep the task relatively smalls, so that they can be done in 1-2 working days. Cheers, Barak

3 3

Re: [IMP-dev] Optimizer
by Daniel Russel 16 Jul '12

16 Jul '12

A couple of comments: - There is already a Quasi-Newton implementation in GSL (using the GSL library). - the simplest optimizer we have currently is probably core::SteepestDescent, so that might be a good one to look at for code to read to get started. core::ConjugateGradients is a bit better developed and structurally simple although the actual optimization function in it is awfully complicated to follow. We don't have an example optimizer at this point. - the IMP_OPTIMIZER() macro simply defines some basic functions and declares do_optimize and do_show (which you then need to implement in your .cpp file). For some reason it doesn't appear in the nightly docs. Not sure why as things look ok. You have to read modules/kernel/include/optimizer_macros.h, sorry. I added imp-dev as this topic seems of general interest. On Mon, Jul 16, 2012 at 9:56 AM, Pia Unverdorben <unverdor(a)biochem.mpg.de>wrote: > Dear Ben and Daniel, > > together with an intern I would like to build a quasi newton optimizer in > IMP. Do you have an example or "recipe" on how to setup such an optimizer > and how to use the IMP_OPTIMIZER() macro? > Thanks a lot! > > Cheers, > Pia > >

2 1

rmf file size
by Riccardo Pellarin 06 Jul '12

06 Jul '12

Hi all. I'm using the rmf file support to write coordinates of my optimization run. I set up the rmf file in this way rh = RMF.create_rmf_file("traj.rmf") IMP.rmf.add_hierarchy(rh, self.prot) and then at each step "i" I save the coordinates in the file for i in range(10000): mc.optimize(100) IMP.rmf.save_frame(rh,i) Now, the resulting rmf file for my system has a size of about 100 KB per frame, which is only slightly lower than the 120 KB per frame of an appended pdb file. (My optimization run saves 100000 frames, which makes about 10 GB for the rmf). How is it possible that a binary file is that bulky? For comparison, a dcd trajectory frame only occupies 18 KB, for the same system. Am I doing something wrong? cheers Riccardo

2 1

write the granularity of the logs
by Javier Velazquez 05 Jul '12

05 Jul '12

I have a simple proposal. write the granularity ad the beginning of each log message. Something like LOG:TERSE blah blah Then a couple of "greps" would make going through all the output less painful, wouldn't it?

6 8

Re: [IMP-dev] Is it possible to write decorators in python?
by Daniel Russel 02 Jul '12

02 Jul '12

Added imp-dev. First, a general comment. As you point out, swig doesn't wrap protected methods, which doesn't interact well with various C++ idioms that use them. Better supporting python would require either making those methods public or doing some trickiness where they are public to swig and the swig wrappers (and visual studio since it makes the protection part of the method signature), but protected when seen by the C++ side of IMP and doxygen. It is probably worth instituting one of these as standard practice in IMP. For example by adding an IMP_PROTECTED macro that resolves to the correct one of public/protected depending on the context. As for Decorator, the public constructor you see is, somewhat confusingly just there to illustrate what an actual decorator should look like (it only appears in doxygen). It probably makes for cleaner docs to simply link to the ExampleDecorator. I'll fix that presently. On Jun 28, 2012, at 4:23 PM, Javier Velazquez wrote: > Daniel, > > Running a program with the decorator below stubbornly says: > def __init__(self, *args, **kwargs): raise AttributeError("No constructor defined") > > According to SWIG's manual no constructor is wrapped if it is protected. It is indeed protected in Decorator.h, but you define a public constructor too. As a result of not accessing the public decorator, I can't access the member function get_particle(). There is a way around this problem using a self.particle member, but looks ugly, and I don't know if it is entirely correct. Is there a correct way to do it? > > > Thanks > > > class Price(IMP.Decorator): > price_key = IMP.IntKey("price") > > def __init__(self, p): > if not self.particle_is_instance(p): > self.setup_particle(p) > IMP.Decorator.__init__(self,p) > > def setup_particle(self, p): > p.add_attribute(self.price_key, 0) > return Price(p) > > def get_price(self): > return self.get_particle().get_value(self.price_key) > > def set_price(self, x): > self.get_particle().set_value(self.price_key, int(x)) > > def particle_is_instance(self, p): > return p.has_attribute(self.price_key)

2 1

Re: [IMP-dev] read_pdb in imp
by Daniel Russel 30 Jun '12

30 Jun '12

One gets warnings like that when atoms are read from the PDB whose atom type (the 3rd entry on an ATOM line in the PDB file) is something non-standard or incorrect. They aren't necessarily worth worrying about, depending on what you want to do with the file. We could, perhaps, provide the residue type and index in the error message which might make the message easier to deal with (Ben would know more). However, there is a tradeoff as the warnings in PDB reading are collapsed to only print out each warning once and files with an unknown atom type often have many of the same unknown atom type. So adding the residue index might results in lots and lots of warnings :-) On Jun 29, 2012, at 7:53 PM, Khuri, Natalia wrote: > Hi Ben and Daniel! > > Not sure who to direct this question, but I get the following warning when i try to read a myoglobin pdb file: > ========= > begin read_pdb: > WARNING Radius not found for type, default value is used "NP1" > ========== > Here is the Python code I am using: > > import IMP.atom > m= IMP.Model() > h= IMP.atom.read_pdb("1DWR.pdb",m) > > Is there a way to make the warning more descriptive? I do not know where to start looking for the atom in question. > Also, should I post the questions to the forum on-line instead of sending them to you directly? > > Thank you, > > Natalia Khuri > PhD Student, Graduate Group in Biophysics > Laboratory of Andrej Sali > UCSF MC 2552 > Byers Hall Room 503B > 1700 4th Street > San Francisco, CA 94158-2330, USA

3 2

Re: [IMP-dev] Interaction graph routine
by Daniel Russel 28 Jun '12

28 Jun '12

> And now I have another one: When creating a PairScore, the help recommends > to return all the particles that could be read if the particle in the > argument is scored. The argument is only one particle, and obviously in a > pair score 2 particles are read when scoring. The examples of creating a > PairScore return only the particle too. Why? > The short answer is that it is easier and faster to figure out what are all the possible particles that can be passed as part of pairs to a pair score than all the pairs that can be passed. Take when the pairs are generated from close pairs. The n input particles are known, a priori. To get inputs over all pairs, one would have to generate and pass all n^2 pairs to the pair score. That would be slow and you would then have a list of length n^2 with only n unique items, which would have to be further processed. I could not think of a case where it was that bad to just inspect particles on at a time for getting inputs. That said, you can certainly construct cases where more input particles will be returned when you pass the particles one at a time than if you were to pass all the pairs. And passing a full list of particles, rather than calling the function repeatedly may be noticeably more efficient.

1 0

Re: [IMP-dev] optimizer state question
by Daniel Russel 26 Jun '12

26 Jun '12

I forwarded this to imp-dev since I am not super familiar with the MD support int IMP. My guess is that the velocities are not being added before the OptimizerState is called. As for the other questions, the general error handler function is now IMP::base::handle_error (and is now documented). You can also do "catch throw" in gdb to break any time there is an exception, but this is a bit annoying when reading pdbs as exceptions are thrown over missing/unknown atoms. On Tue, Jun 26, 2012 at 12:15 PM, Dave Barkan <dbarkan(a)salilab.org> wrote: > ** > Hey Daniel, > > I am getting an IMP usage exception when I try to use a > VelocityScalingOptimizerState (output run from gdb): > > Traceback (most recent call last): > File "generateTrajectories.py", line 56, in <module> > p.runMolecularDynamics() > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/pepdock/peptideDocker.py", > line 107, in runMolecularDynamics > self.mdHandler.runMdStage(i) > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/pepdock/md.py", > line 59, in runMdStage > > self.mdOptimizer.optimize(self.schedule.getStepCountForMdStage(stageNumber)) > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/__init__.py", > line 3049, in optimize > return _IMP.Optimizer_optimize(self, *args) > _IMP.UsageException: Usage check failure: Can't get attribute that is not > there > Context: MD 0::optimize/simulate > > WARNING No frames were saved to file "HierarchySaveLink0" even though > objects were added. > WARNING Object "HierarchySaveLink0" was never used. See the IMP::Object > documentation for an explanation. > WARNING Object "lj_flex_flex" was never used. See the IMP::Object > documentation for an explanation. > [Inferior 1 (process 40694) exited with code 01] > > I can use other OptimizerStates without seeing this error. > VelocityScalingOptimizerState was working before but stopped after I > changed some of my code (no IMP updates) around so I am probably not > initializing something correctly. > > I tried running it with gdb but it appears to exit without allowing me to > see the backtrace (it says 'no stack' when I type backtrace). Previously I > was able to set a breakpoint when a usage exception was occurring by typing > > break IMP::internal::assert_fail > > before running gdb. But that seems to no longer work: > > (gdb) break IMP::internal::assert_fail > Function "IMP::internal::assert_fail" not defined. > Make breakpoint pending on future shared library load? (y or [n]) > > So my questions: > > 1. Does anything immediately stand out to you with the error? > 2. Can I get gdb to stop upon the Usage Exception with the stack in memory > so I can track down the error? > > Thanks, > Dave >

2 2

module/bin/X -> build/module_bin/module/x
by Daniel Russel 25 Jun '12

25 Jun '12

I moved where compiled/linked things from the module bin directories go. This addresses a frequent source of bugs and complications in the build scripts caused by SCons being sloppy about differentiating between the source and target directories when using a repository. Sorry. The new directories are still added to the path by imppy.sh should you can still call your module binaries by doing ./tools/imppy.sh mybinary arguments or .tools/imppy.sh myscript.py arguments In a change that is less likely to be noticed, there is a new benchmark folder in modules where you can put benchmarks to remove some annoying fiddling in the sconscripts.

1 0