- IMP-dev - salilab.org

Last week in IMP
by Daniel Russel 04 Aug '09

04 Aug '09

Week of July 18 The atom::BondPairContainer has been renamed to atom::BondPairFilter since it wasn't a real container. A real atom::BondPairContainer has been added. Out of source builds mostly works. To do an out of source build, make a link to scons_tools and SConstruct from the source directory in the build directory and then add the line repository="../path_to_src_dir" in the config.py in the build directory. Then running scons in the build directory will put generated files there. This allows you to make both a release andfast build off of the same set of sources. Proper docs will be added soon. The SConscript setup for a module has been greatly simplified so it should be easy enough for users to maintain their own module and updates to the build system shouldn't involve changes to user SConscripts any more. This does mean that any non-svn modules need to be updated. Look at IMP.core or another module and copy what is there. Again, proper docs will be added soon.

1 0

Restraint::{get,set}_is_active
by Daniel Russel 03 Aug '09

03 Aug '09

Does anyone use these functions? One could do the same thing with RestraintSets. I think they are left over from the vague plans for handling inactive particles. If no one uses them I'll remove them.

1 0

Last week in IMP
by Daniel Russel 17 Jul '09

17 Jul '09

- Per Javi's suggestion, the conventions have been split into Coding conventions<file:///flute1/home/drussel/src/IMP/svn/doc/html/coding_conventions.html>and API <file:///flute1/home/drussel/src/IMP/svn/doc/html/conventions.html>conventions" and linked from the main page. - benchmarks got moved to their own module, IMP::benchmark<file:///flute1/home/drussel/src/IMP/svn/doc/html/namespaceIMP_1_1benchmark.html>and will now be tracked over time - There is a wiki page on optimization<https://salilab.org/imp/wiki/optimization_protocols/> - There is now a wiki page to list undocumented IMP-related functionality<https://salilab.org/imp/wiki/internal_functionality/>

1 0

Followup answer to Javi's question about restraints and hierarchy level of detail
by Daniel Russel 15 Jul '09

15 Jul '09

On some further thought, I think it is, in general, better that most scoring sorts of code takes a set of particles describing the (for example) protein, at the desired level of detail instead of a Hierachy from which they extract such a description. This allows the user to select the resolution used without having to maintain different hierarchies with leaves at different resolutions. More concretely things like the em::FitRestraint which takes a set of particles are doing the right thing. It allows the user to have a fully atomic model of a protein, instead to pass the residues to em::FitRestraint when it is a low resolution map. (That said, it should probably take XYZRs instead of particles :-) Likewise, the rmsd(XYZs, XYZs) function is much nicer than the reject rmsd(Hierarchy, Hierarchy) since you can choose to compute rmsd on a residue or domain level.

1 0

Some issues raised when changing the set of particles
by Daniel Russel 13 Jul '09

13 Jul '09

I've been doing some simulations where the set of particles involved changes quite frequently. This has raised some issues with how IMP should handle particles being created and destroyed 1) When can you change the set of particles being optimized? My current opinion is that OptimizerStates should be allowed to change the set of particles, but ScoreStates should not. With that rule, the space being optimized can change between optimizer steps, but not during one. I added a constructor to BrownianDynamics which allows you to explicitly provide the set of the particles to optimize (via a SingletonList), but I think simply rescanning the model might not be noticeably less efficient. 2) How to handle saving and restoring state? The model_io functions don't work when the number of particles changes. The reason for this is that there is no way to insert extra particles read into restraints and things. One possible solution to this is to have the List{Singleton,Pair}Container objects use a Particle as their underlying storage mechanism. Then their sets of particles can be easily saved and restored by the model_io functions (or anything else which looks at all the particles). Since restraints (should) use these containers as their underlying storage, then restraints could have their state restored too. Making such a change would result in much more data being stored as particle attributes, so we would have to be careful that it didn't result in a performance penalty. Anyone have any thoughts on these questions?

1 0

Last week in IMP
by Daniel Russel 10 Jul '09

10 Jul '09

Changes for the week: * FilteredListSingletonContainer and FilteredPairSingletonContainer went away as there was too much hard to factor code shared with ListSingletonContainer and PairSingletonContainer. The IMP::core::ClosePairsScoreState is now handled by a list of filters on the class itself. That is, do IMP::core::ClosePairsScoreState::add_close_pair_filter() to add a filter. * Hooks have been added to allow actions to be taken on an assertion or check failure. See IMP::HandleFailure and its descendents for more details. For exampe, one can write out the model to a file to view in chimera on each error. * IMP::display::CGOWriter has been much improved and now allows provides lots of display-time control and easy building of animations. * Generic optimizer states have been added (IMP::core::SingletonsOptimizerState and IMP::core::PairsOptimizerState). We have a choice for the IMP-dev meeting next week: (1) how to use the various error checking macros and functions in your own code (IMP_check, IMP_assert, failure handlers) (2) an overview of the tools available for manipulating molecules: What we have, what we need. Sends preferences to me. Since a few questions have come up about it, I thought I would say a few words about why we use reference counted pointers and why, in general, no classes should ever store raw pointers to other objects. All entities in a compute program have lifetimes, that is, the interval between when they are created to when they are destroyed. Once an object's lifetime is over, it is an error to access the object. In C++ programs, the lifetime of a local variable is that of the scope containing them (they are alive, for example, until the function in which they are defined exits). The lifetime of objects created using "new" lasts until "delete" is called on them. On important question in any computer system is how the lifetime of shared objects is managed. A shared object is one that is accessed via stored pointers or references from a variety of places. It is exacerbated in IMP, since any object that python sees is implicitly shared with python, and python will delete any object it thinks is no longer in use. There are several common conventions for dealing with shared objects: (1) the code which created the shared object is responsible for destroying it. This convention shifts all the load to the users since, the user needs to know when all other code is done with the shared object and explicitly keep it alive until then. As a result, this convention only works with small systems where someone can understand all the internal relationships between objects. (2) eliminate shared objects. Nice and simple, but quite limiting. (3) reference counting: With reference counting, shared objects are kept alive until everything is done with them. This is typically done by associating a reference count with each object. Each bit of code using the shared object increments the counter when it acquires a reference to the object and then decrements the counted when it is done. When the counter goes to zero, nothing is using the object and the object can go away. (4) garbage collection: a garbage collector scans the running program to determine when the object no longer in used and then deletes it. This is what java does, but is very hard to implement in C++ In IMP we chose (3), both because it is what Python does and is relatively straight foward. To make your life easier, objects in IMP are divided into little-o-objects which should never be shared and big-O objects which should be passed by pointers and can be shared. It also provides a class, IMP::Pointer to handle the incrementing and decrementing of the reference count for you. To make sure an object doesn't go away, just create an reference counted pointer with IMP::Pointer<ObjectType> my_object_ref_counted_pointer(my_object_ptr); This increments the reference count. When my_object_ref_counted_pointer is destroyed, the reference count will be decremented, relinquishing control of the object. The reference counts are shared with Python, so an object will not be deleted until python is done with it and all there are no reference counted pointers referring to it. It is also quite easy to define your own reference counted object: simply inherit from IMP::RefCounted. Collections of reference counted objects can be stored using an IMP::VectorOfRefCounted (Particles, Restraints etc use this mechanism).

1 0

Accelerated C++
by Daniel Russel 09 Jul '09

09 Jul '09

The book I mentioned on Tuesday, "Accelerated C++", has arrived. It looks good from first glance, getting to vectors in the first 30 pages and only describing pointers and arrays starting on page 170, which seems like a good order :-) And it eventually covers smart pointers (unfortunately not immediately following pointers though). Anyway, anyone who wants to borrow it is welcome to it. It is on my shelves. At only 300 small pages, it is pretty short for a C++ book :-)

1 0

rmsd function
by Keren Lasker 09 Jul '09

09 Jul '09

hi, are you all ok with me adding a function MPATOMEXPORT Float rmsd(const Hierarchy& m1 ,const Hierarchy& m2); IMPATOMEXPORT Float rmsd(const Hierarchies& mh1 ,const Hierarchies& mh2); to pdb.h or utility.h in IMP::atom ? thanks, Keren.

5 12

Next IMP developers meeting
by Daniel Russel 29 Jun '09

29 Jun '09

Can we have it Wednesday before group meeting instead? I have a workshop all Tuesday. Also, the topic will most likely be an introduction to STL and Boost and other C++ libraries that are good to know about.

4 7

Units in IMP
by Daniel Russel 26 Jun '09

26 Jun '09

I have been making extensive use of some C++ classes I wrote to do compile time unit analysis in IMP. I find them very useful and wouldn t want to have to write code without them and as a result thought I would ask if others are interested. They enable things like - automatically converting between different length measures (say angstroms and nanometers) so you don't have to worry which number is which - check that unit types match, so can can't assign an energy value to a force or daltons to kilograms - convert between SI units and the energy units used in IMP etc. You use them by writing things like Nanometer side(10); CubicAngstrom volume= side*side*side; Nanosecond time_step(10); simulation_parameters.set_maximum_time_step(time_step); // it otherwise would want femtoseconds As a said, I quite like using them and would like to make them generally available. There are a couple of holdups: - since it is a compile time library, it doesn't really translate to python, so we would need to continue to provide separate units and unit free interfaces for things (for example SimulationParameters has set_maximum_time_step and set_maximum_time_step_in_femtosecond, the latter just takes a double - Boost, has added its own units library (as of 1.36). It is based on the same principles as mine (they both grew out of a Boost.MPL example) but there are various differences so converting to the boost one would take a bit of work and we would need to provide some of our own code on top of the boost library. My thought had been to wait until IMP required a new enough boost and switch to that, but it is taking a while. Are people interested? The code is pretty stable, but could use a bit of documentation and cleanup if others are to use it.

2 2