Changes for the week:

* FilteredListSingletonContainer and FilteredPairSingletonContainer went away as there was too much hard to factor code shared with ListSingletonContainer and PairSingletonContainer. The IMP::core::ClosePairsScoreState is now handled by a list of filters on the class itself. That is, do IMP::core::ClosePairsScoreState::add_close_pair_filter() to add a filter.

* Hooks have been added to allow actions to be taken on an assertion or check failure. See IMP::HandleFailure and its descendents for more details. For exampe, one can write out the model to a file to view in chimera on each error.

* IMP::display::CGOWriter has been much improved and now allows provides lots of display-time control and easy building of animations.

* Generic optimizer states have been added (IMP::core::SingletonsOptimizerState and IMP::core::PairsOptimizerState).

We have a choice for the IMP-dev meeting next week: (1) how to use the various error checking macros and functions in your own code (IMP_check, IMP_assert, failure handlers) (2) an overview of the tools available for manipulating molecules: What we have, what we need. Sends preferences to me.

Since a few questions have come up about it, I thought I would say a few words about why we use reference counted pointers and why, in general, no classes should ever store raw pointers to other objects.

All entities in a compute program have lifetimes, that is, the interval between when they are created to when they are destroyed. Once an object's lifetime is over, it is an error to access the object. In C++ programs, the lifetime of a local variable is that of the scope containing them (they are alive, for example, until the function in which they are defined exits). The lifetime of objects created using "new" lasts until "delete" is called on them.

On important question in any computer system is how the lifetime of shared objects is managed. A shared object is one that is accessed via stored pointers or references from a variety of places. It is exacerbated in IMP, since any object that python sees is implicitly shared with python, and python will delete any object it thinks is no longer in use. There are several common conventions for dealing with shared objects:

(1) the code which created the shared object is responsible for destroying it. This convention shifts all the load to the users since, the user needs to know when all other code is done with the shared object and explicitly keep it alive until then. As a result, this convention only works with small systems where someone can understand all the internal relationships between objects.

(2) eliminate shared objects. Nice and simple, but quite limiting.

(3) reference counting: With reference counting, shared objects are kept alive until everything is done with them. This is typically done by associating a reference count with each object. Each bit of code using the shared object increments the counter when it acquires a reference to the object and then decrements the counted when it is done. When the counter goes to zero, nothing is using the object and the object can go away.

(4) garbage collection: a garbage collector scans the running program to determine when the object no longer in used and then deletes it. This is what java does, but is very hard to implement in C++

In IMP we chose (3), both because it is what Python does and is relatively straight foward. To make your life easier, objects in IMP are divided into little-o-objects which should never be shared and big-O objects which should be passed by pointers and can be shared. It also provides a class, IMP::Pointer to handle the incrementing and decrementing of the reference count for you. To make sure an object doesn't go away, just create an reference counted pointer with IMP::Pointer<ObjectType> my_object_ref_counted_pointer(my_object_ptr); This increments the reference count. When my_object_ref_counted_pointer is destroyed, the reference count will be decremented, relinquishing control of the object. The reference counts are shared with Python, so an object will not be deleted until python is done with it and all there are no reference counted pointers referring to it. It is also quite easy to define your own reference counted object: simply inherit from IMP::RefCounted. Collections of reference counted objects can be stored using an IMP::VectorOfRefCounted (Particles, Restraints etc use this mechanism).