July 2009 - IMP-dev - salilab.org

Last week in IMP
by Daniel Russel 17 Jul '09

17 Jul '09

- Per Javi's suggestion, the conventions have been split into Coding conventions<file:///flute1/home/drussel/src/IMP/svn/doc/html/coding_conventions.html>and API <file:///flute1/home/drussel/src/IMP/svn/doc/html/conventions.html>conventions" and linked from the main page. - benchmarks got moved to their own module, IMP::benchmark<file:///flute1/home/drussel/src/IMP/svn/doc/html/namespaceIMP_1_1benchmark.html>and will now be tracked over time - There is a wiki page on optimization<https://salilab.org/imp/wiki/optimization_protocols/> - There is now a wiki page to list undocumented IMP-related functionality<https://salilab.org/imp/wiki/internal_functionality/>

1 0

Followup answer to Javi's question about restraints and hierarchy level of detail
by Daniel Russel 15 Jul '09

15 Jul '09

On some further thought, I think it is, in general, better that most scoring sorts of code takes a set of particles describing the (for example) protein, at the desired level of detail instead of a Hierachy from which they extract such a description. This allows the user to select the resolution used without having to maintain different hierarchies with leaves at different resolutions. More concretely things like the em::FitRestraint which takes a set of particles are doing the right thing. It allows the user to have a fully atomic model of a protein, instead to pass the residues to em::FitRestraint when it is a low resolution map. (That said, it should probably take XYZRs instead of particles :-) Likewise, the rmsd(XYZs, XYZs) function is much nicer than the reject rmsd(Hierarchy, Hierarchy) since you can choose to compute rmsd on a residue or domain level.

1 0

Some issues raised when changing the set of particles
by Daniel Russel 13 Jul '09

13 Jul '09

I've been doing some simulations where the set of particles involved changes quite frequently. This has raised some issues with how IMP should handle particles being created and destroyed 1) When can you change the set of particles being optimized? My current opinion is that OptimizerStates should be allowed to change the set of particles, but ScoreStates should not. With that rule, the space being optimized can change between optimizer steps, but not during one. I added a constructor to BrownianDynamics which allows you to explicitly provide the set of the particles to optimize (via a SingletonList), but I think simply rescanning the model might not be noticeably less efficient. 2) How to handle saving and restoring state? The model_io functions don't work when the number of particles changes. The reason for this is that there is no way to insert extra particles read into restraints and things. One possible solution to this is to have the List{Singleton,Pair}Container objects use a Particle as their underlying storage mechanism. Then their sets of particles can be easily saved and restored by the model_io functions (or anything else which looks at all the particles). Since restraints (should) use these containers as their underlying storage, then restraints could have their state restored too. Making such a change would result in much more data being stored as particle attributes, so we would have to be careful that it didn't result in a performance penalty. Anyone have any thoughts on these questions?

1 0

Last week in IMP
by Daniel Russel 10 Jul '09

10 Jul '09

Changes for the week: * FilteredListSingletonContainer and FilteredPairSingletonContainer went away as there was too much hard to factor code shared with ListSingletonContainer and PairSingletonContainer. The IMP::core::ClosePairsScoreState is now handled by a list of filters on the class itself. That is, do IMP::core::ClosePairsScoreState::add_close_pair_filter() to add a filter. * Hooks have been added to allow actions to be taken on an assertion or check failure. See IMP::HandleFailure and its descendents for more details. For exampe, one can write out the model to a file to view in chimera on each error. * IMP::display::CGOWriter has been much improved and now allows provides lots of display-time control and easy building of animations. * Generic optimizer states have been added (IMP::core::SingletonsOptimizerState and IMP::core::PairsOptimizerState). We have a choice for the IMP-dev meeting next week: (1) how to use the various error checking macros and functions in your own code (IMP_check, IMP_assert, failure handlers) (2) an overview of the tools available for manipulating molecules: What we have, what we need. Sends preferences to me. Since a few questions have come up about it, I thought I would say a few words about why we use reference counted pointers and why, in general, no classes should ever store raw pointers to other objects. All entities in a compute program have lifetimes, that is, the interval between when they are created to when they are destroyed. Once an object's lifetime is over, it is an error to access the object. In C++ programs, the lifetime of a local variable is that of the scope containing them (they are alive, for example, until the function in which they are defined exits). The lifetime of objects created using "new" lasts until "delete" is called on them. On important question in any computer system is how the lifetime of shared objects is managed. A shared object is one that is accessed via stored pointers or references from a variety of places. It is exacerbated in IMP, since any object that python sees is implicitly shared with python, and python will delete any object it thinks is no longer in use. There are several common conventions for dealing with shared objects: (1) the code which created the shared object is responsible for destroying it. This convention shifts all the load to the users since, the user needs to know when all other code is done with the shared object and explicitly keep it alive until then. As a result, this convention only works with small systems where someone can understand all the internal relationships between objects. (2) eliminate shared objects. Nice and simple, but quite limiting. (3) reference counting: With reference counting, shared objects are kept alive until everything is done with them. This is typically done by associating a reference count with each object. Each bit of code using the shared object increments the counter when it acquires a reference to the object and then decrements the counted when it is done. When the counter goes to zero, nothing is using the object and the object can go away. (4) garbage collection: a garbage collector scans the running program to determine when the object no longer in used and then deletes it. This is what java does, but is very hard to implement in C++ In IMP we chose (3), both because it is what Python does and is relatively straight foward. To make your life easier, objects in IMP are divided into little-o-objects which should never be shared and big-O objects which should be passed by pointers and can be shared. It also provides a class, IMP::Pointer to handle the incrementing and decrementing of the reference count for you. To make sure an object doesn't go away, just create an reference counted pointer with IMP::Pointer<ObjectType> my_object_ref_counted_pointer(my_object_ptr); This increments the reference count. When my_object_ref_counted_pointer is destroyed, the reference count will be decremented, relinquishing control of the object. The reference counts are shared with Python, so an object will not be deleted until python is done with it and all there are no reference counted pointers referring to it. It is also quite easy to define your own reference counted object: simply inherit from IMP::RefCounted. Collections of reference counted objects can be stored using an IMP::VectorOfRefCounted (Particles, Restraints etc use this mechanism).

1 0

Accelerated C++
by Daniel Russel 09 Jul '09

09 Jul '09

The book I mentioned on Tuesday, "Accelerated C++", has arrived. It looks good from first glance, getting to vectors in the first 30 pages and only describing pointers and arrays starting on page 170, which seems like a good order :-) And it eventually covers smart pointers (unfortunately not immediately following pointers though). Anyway, anyone who wants to borrow it is welcome to it. It is on my shelves. At only 300 small pages, it is pretty short for a C++ book :-)

1 0

rmsd function
by Keren Lasker 09 Jul '09

09 Jul '09

hi, are you all ok with me adding a function MPATOMEXPORT Float rmsd(const Hierarchy& m1 ,const Hierarchy& m2); IMPATOMEXPORT Float rmsd(const Hierarchies& mh1 ,const Hierarchies& mh2); to pdb.h or utility.h in IMP::atom ? thanks, Keren.

5 12