June 2012 - IMP-dev - salilab.org

Re: [IMP-dev] Is it possible to write decorators in python?
by Daniel Russel 02 Jul '12

02 Jul '12

Added imp-dev. First, a general comment. As you point out, swig doesn't wrap protected methods, which doesn't interact well with various C++ idioms that use them. Better supporting python would require either making those methods public or doing some trickiness where they are public to swig and the swig wrappers (and visual studio since it makes the protection part of the method signature), but protected when seen by the C++ side of IMP and doxygen. It is probably worth instituting one of these as standard practice in IMP. For example by adding an IMP_PROTECTED macro that resolves to the correct one of public/protected depending on the context. As for Decorator, the public constructor you see is, somewhat confusingly just there to illustrate what an actual decorator should look like (it only appears in doxygen). It probably makes for cleaner docs to simply link to the ExampleDecorator. I'll fix that presently. On Jun 28, 2012, at 4:23 PM, Javier Velazquez wrote: > Daniel, > > Running a program with the decorator below stubbornly says: > def __init__(self, *args, **kwargs): raise AttributeError("No constructor defined") > > According to SWIG's manual no constructor is wrapped if it is protected. It is indeed protected in Decorator.h, but you define a public constructor too. As a result of not accessing the public decorator, I can't access the member function get_particle(). There is a way around this problem using a self.particle member, but looks ugly, and I don't know if it is entirely correct. Is there a correct way to do it? > > > Thanks > > > class Price(IMP.Decorator): > price_key = IMP.IntKey("price") > > def __init__(self, p): > if not self.particle_is_instance(p): > self.setup_particle(p) > IMP.Decorator.__init__(self,p) > > def setup_particle(self, p): > p.add_attribute(self.price_key, 0) > return Price(p) > > def get_price(self): > return self.get_particle().get_value(self.price_key) > > def set_price(self, x): > self.get_particle().set_value(self.price_key, int(x)) > > def particle_is_instance(self, p): > return p.has_attribute(self.price_key)

2 1

Re: [IMP-dev] read_pdb in imp
by Daniel Russel 30 Jun '12

30 Jun '12

One gets warnings like that when atoms are read from the PDB whose atom type (the 3rd entry on an ATOM line in the PDB file) is something non-standard or incorrect. They aren't necessarily worth worrying about, depending on what you want to do with the file. We could, perhaps, provide the residue type and index in the error message which might make the message easier to deal with (Ben would know more). However, there is a tradeoff as the warnings in PDB reading are collapsed to only print out each warning once and files with an unknown atom type often have many of the same unknown atom type. So adding the residue index might results in lots and lots of warnings :-) On Jun 29, 2012, at 7:53 PM, Khuri, Natalia wrote: > Hi Ben and Daniel! > > Not sure who to direct this question, but I get the following warning when i try to read a myoglobin pdb file: > ========= > begin read_pdb: > WARNING Radius not found for type, default value is used "NP1" > ========== > Here is the Python code I am using: > > import IMP.atom > m= IMP.Model() > h= IMP.atom.read_pdb("1DWR.pdb",m) > > Is there a way to make the warning more descriptive? I do not know where to start looking for the atom in question. > Also, should I post the questions to the forum on-line instead of sending them to you directly? > > Thank you, > > Natalia Khuri > PhD Student, Graduate Group in Biophysics > Laboratory of Andrej Sali > UCSF MC 2552 > Byers Hall Room 503B > 1700 4th Street > San Francisco, CA 94158-2330, USA

3 2

Re: [IMP-dev] Interaction graph routine
by Daniel Russel 28 Jun '12

28 Jun '12

> And now I have another one: When creating a PairScore, the help recommends > to return all the particles that could be read if the particle in the > argument is scored. The argument is only one particle, and obviously in a > pair score 2 particles are read when scoring. The examples of creating a > PairScore return only the particle too. Why? > The short answer is that it is easier and faster to figure out what are all the possible particles that can be passed as part of pairs to a pair score than all the pairs that can be passed. Take when the pairs are generated from close pairs. The n input particles are known, a priori. To get inputs over all pairs, one would have to generate and pass all n^2 pairs to the pair score. That would be slow and you would then have a list of length n^2 with only n unique items, which would have to be further processed. I could not think of a case where it was that bad to just inspect particles on at a time for getting inputs. That said, you can certainly construct cases where more input particles will be returned when you pass the particles one at a time than if you were to pass all the pairs. And passing a full list of particles, rather than calling the function repeatedly may be noticeably more efficient.

1 0

Re: [IMP-dev] optimizer state question
by Daniel Russel 26 Jun '12

26 Jun '12

I forwarded this to imp-dev since I am not super familiar with the MD support int IMP. My guess is that the velocities are not being added before the OptimizerState is called. As for the other questions, the general error handler function is now IMP::base::handle_error (and is now documented). You can also do "catch throw" in gdb to break any time there is an exception, but this is a bit annoying when reading pdbs as exceptions are thrown over missing/unknown atoms. On Tue, Jun 26, 2012 at 12:15 PM, Dave Barkan <dbarkan(a)salilab.org> wrote: > ** > Hey Daniel, > > I am getting an IMP usage exception when I try to use a > VelocityScalingOptimizerState (output run from gdb): > > Traceback (most recent call last): > File "generateTrajectories.py", line 56, in <module> > p.runMolecularDynamics() > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/pepdock/peptideDocker.py", > line 107, in runMolecularDynamics > self.mdHandler.runMdStage(i) > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/pepdock/md.py", > line 59, in runMdStage > > self.mdOptimizer.optimize(self.schedule.getStepCountForMdStage(stageNumber)) > File > "/trombone1/home/dbarkan/IMP/impBuildFour/imp/debug/build/lib/IMP/__init__.py", > line 3049, in optimize > return _IMP.Optimizer_optimize(self, *args) > _IMP.UsageException: Usage check failure: Can't get attribute that is not > there > Context: MD 0::optimize/simulate > > WARNING No frames were saved to file "HierarchySaveLink0" even though > objects were added. > WARNING Object "HierarchySaveLink0" was never used. See the IMP::Object > documentation for an explanation. > WARNING Object "lj_flex_flex" was never used. See the IMP::Object > documentation for an explanation. > [Inferior 1 (process 40694) exited with code 01] > > I can use other OptimizerStates without seeing this error. > VelocityScalingOptimizerState was working before but stopped after I > changed some of my code (no IMP updates) around so I am probably not > initializing something correctly. > > I tried running it with gdb but it appears to exit without allowing me to > see the backtrace (it says 'no stack' when I type backtrace). Previously I > was able to set a breakpoint when a usage exception was occurring by typing > > break IMP::internal::assert_fail > > before running gdb. But that seems to no longer work: > > (gdb) break IMP::internal::assert_fail > Function "IMP::internal::assert_fail" not defined. > Make breakpoint pending on future shared library load? (y or [n]) > > So my questions: > > 1. Does anything immediately stand out to you with the error? > 2. Can I get gdb to stop upon the Usage Exception with the stack in memory > so I can track down the error? > > Thanks, > Dave >

2 2

module/bin/X -> build/module_bin/module/x
by Daniel Russel 25 Jun '12

25 Jun '12

I moved where compiled/linked things from the module bin directories go. This addresses a frequent source of bugs and complications in the build scripts caused by SCons being sloppy about differentiating between the source and target directories when using a repository. Sorry. The new directories are still added to the path by imppy.sh should you can still call your module binaries by doing ./tools/imppy.sh mybinary arguments or .tools/imppy.sh myscript.py arguments In a change that is less likely to be noticed, there is a new benchmark folder in modules where you can put benchmarks to remove some annoying fiddling in the sconscripts.

1 0

git flow tutorial + quick start
by Barak Raveh 21 Jun '12

21 Jun '12

This is the git flow tutorial (ask Ben to install git flow for you on your machine). http://yakiloo.com/getting-started-git-flow/ Quick start: 1) Open a github account https://github.com/ 2) Get Daniel to invite you to the github project , 3) create and change to a folder of your choice 4) "git flow init" and enter-enter-enter-enter-enter 5) "git clone <address of git-hub repository provided by Daniel>" You can then start new features like Daniel showed yesterday, as described in the tutorial, etc. (e.g., "git flow feature start <feature-name>")

2 1

self-organizing maps
by Yannick Spill 18 Jun '12

18 Jun '12

Hi list, In the light of my recent discovery of the statistics module in imp, I was wondering if some of you would be interested in an implementation of self-organizing maps (SOM). They are a generalization of principal components analysis to nonlinear models. http://en.wikipedia.org/wiki/Self-organizing_map There's a shortcut for us to have it in IMP. It's called GPLVM, and is a probabilistic SOM based on gaussian processes. Is that something that would interest anyone? Yannick

4 4

clustering - code consolidation
by Barak Raveh 14 Jun '12

14 Jun '12

Now the full version... Daniel and I discussed a little bit consolidation of clustering things in statistics and kmeans modules. Please tell me if it is agreed that things will work with the following interface: * The Embedding family of classes (used to embed data in vector form) will remain as is * There will be a "Clustering" class from which all clustering algorithms will derive, with a constructor that takes either Embedding class, or EmbeddingAdaptor for implicit conversions from e.g., FloatsList * The Clustering classes will also have a *void ::execute()* method and a*::get_clustering results() *method, that will return the clustering results (using the exisiting PartitioningClustering class, perhaps we can change its name to PartitioningClusteringResults). So bottom line, if you will want to cluster some data, you will do something like *FloatsList data; // or create an Embedding object* *KMeansClustering kmeans(data, params);* *kmeans.execute();* *PartitioningClustering clustering_results = kmeans.get_clustering_results() * Makes sense? For backward compatibility, I will add a DEPRECATED warning to existing clustering methods, so they will be removed within a few months completely. Barak

2 4

clustering - consolidation
by Barak Raveh 14 Jun '12

14 Jun '12

Hi, Daniel and I discussed a little bit consolidation of clustering things in statistics. I am not touching anything out of the statistics / kmeans module for now, but please tell me if it is agreed that things will work as follows: * The Embedding family of classes (used to embed data in vector form) will remain as is * There will be a "Clustering" class from which all clustering algorithms will derive, with: * constructor that takes either Embedding class, or EmbeddingAdaptor for implicit conversions from e.g., vector form -- Barak

2 1

distance based scoring
by Daniel Russel 14 Jun '12

14 Jun '12

Following further discussion after the meeting yesterday, Dina and I have an alternative proposal for how to move forward with distance-based scoring. The changes would be to provide a class template <class DistanceScore, class DistanceScoreAndDerivatives> class DistancePairScore{ ... }; The DistanceScore argument would be a functor (or a function) with the following signature double DistanceScore(Model *, ParticleIndexPair, double distance) where the return value is the score for the pair of particles separated by distance. And the DistanceScoreAndDerivatives argument would be the same except that it would return a pair of doubles, on for the score and one for the derivative. We would then (gradually), provide functor versions of the existing distance-based pair scores and implement the corresponding PairScores in terms of the functor and this template class. For python users, we could provide a python implementation of DistancePairScore that would allow experimenting with such pair scores (but which would be rather slow). The advantages of this over the proposed changes are mostly that of flexibility, efficiency and reduced amount of code to right. On the flexibility side, doing things this way makes it easier to create modified versions of existing pair scores (eg by weighting them) as that just requires writing a functor that calls the other functions and adds up the values with the expected weights. On the efficiency side, such an approach computes the distances and then uses them immediately, rather than storing them in memory and passing over the stored values repeatedly. And on the reduced code side, it just requires adding one non-trivial class (whose implementation already exists) and we had wanted to move the scoring computations to functors anyway (and that can be done on-demand).

2 2