- IMP-dev - salilab.org

rmf warning
by Daniel Russel 04 Aug '11

04 Aug '11

Unfortunately, I need to make changes to the RMF file format that are not backward compatible. Let me know if you have data saved in rmf that you need to access and I'll work on a work-around. Otherwise, backward compatibility will be lost on your next svn update. Sorry about that, the change is needed to make reading certain large rmf files tolerably fast. The changes will include various other minor changes to make the format more space efficient and support other architectures.

1 0

Adding a restraint to more than one RestraintSet
by Daniel Russel 04 Aug '11

04 Aug '11

Does anyone use this ability? Removing it would simplify various things.

3 2

Another proposed mover change
by Daniel Russel 30 Jul '11

30 Jul '11

I'd like to make another minor change to the mover, namely to change the propose_move method so that it returns a list of particles that were moved. Monte Carlo implementations can then use this list to only evaluate the restraints whose scores have been changed, replacing the functionality that was removed when the incremental evaluation support was removed. I will, of course, make the changes for movers in svn.

1 0

Proposed change to Movers
by Daniel Russel 27 Jul '11

27 Jul '11

I'd like to move the MoverBase-based movers off of SingletonContainers :-). This would simplify their implementation, make running with them slightly more efficient, as well as simplifying their usage as. It would also harmonize the various movers a bit. And, of course, it would break various existing code. I could provide a backward compatibility functions, but I am reluctant to do that as providing the functions is a bit complicated and the semantics have changed (that is the particles are no longer shared) and, while I don't expect the changed semantics will make any difference to anyone, I would rather people inspect their code just to be sure. So the new constructors for the BallMover and NormalMover would be BallMover(ParticlesTemp particles, FloatKeys dofs, double radius). or BallMover(ParticlesTemp particles, double radius). This will also remove some of the direct accesses to the contents of containers to move towards supporting filters on the all pairs containers and such things.

1 0

proposal to narrow container interfaces
by Daniel Russel 25 Jul '11

25 Jul '11

Right now, containers in IMP (ie SingletonContainer, PairContainer), support looking at the particles via (for a SingletonContainer) 1) get_particles() 2) iterate through the particles 3) a for loop on the indexes and using get_particle() 4) applying a SingletonModifier or SingletonScore 2 and 3 are quite limiting and effectively eliminate the ability to use containers where it is non-trivial to predict how many items the container will contain without generating them all. More concretely, it eliminates the possibility of using filters with the AllPairContainer and other similar containers (or at least makes it very difficult). I would like to remove 2 and 3. No python code is likely to be affected as it probably all uses the get_particles() equivalebts and I can fix all the code in IMP svn (which most likely uses 3). How much would such a change be likely to affect people? To get an idea, search for get_numer_of_particles() and get_number_of_particle_pairs() in your c++ code. Thanks.

1 0

proposed imp dev meeting on coarse grained models and pdb
by Daniel Russel 05 Jul '11

05 Jul '11

Lots of people have been bumping into the fact that it is kind of annoying to handle the following sort of work flow: - read a pdb - create a coarse grained rigid model - move the coarse grained rigid model around (ignoring the atoms as much as possible, for efficiency) - write an appropriately transformed version of the pdb The problems are: - doing this reliably requires you saving the initial transformation of the rigid body and then transforming the components of the initial pdb by the difference between the initial and final. There are lots of plausible seeming ways of doing the job that don't work reliably. - less troublesome, creating all the particles for a large pdb can take a bit of time I'm not sure of a way of providing a simple interface to this that is easy to implement reliably. And so I thought it might be worth having a brainstorming session/discussion on the topic to see if we can come up with a nice, canned solution. Interested? Some possible bits of solutions are: - IMP::atom::create_rigid_body can store the initial transform internally and provide a method to get how much the molecule moved - the hierarchy clone method could keep a link back to the hierarchy it was created from to help you in going back (if you want to keep the whole hierarchy in memory) - we could provide a single method that transforms a pdb file without creating particles from it

1 0

Accelerating restraint evaluation
by Daniel Russel 25 Jun '11

25 Jun '11

We have been discussing ideas for accelerating the evaluation of certain restraints, particularly excluded volume and connectivity-based ones, by taking advantage of the fact that you often don't care how bad things are once they are sufficiently bad. To be more concrete, the idea is to add a function Model::evaluate_if_ok(Restraints, Weights) which - returns the score if the score is less than Model::get_maximum_score and each restraint score is less than its maximum - otherwise returns a very large score (something greater than Model::get_maximum_score() The idea is that restraint evaluation in general, and individual restraints in particular can cease their computations if the can prove the score is too high. The usage scenarios envisioned are: - domino: since we reject any state which scores badly, we can use this method directly - monte carlo: any state that score sufficiently badly is vanishingly unlikely to be accepted, and so we could, in principle, save some work in evaluation by dynamically varying the maximum score to be, say, 5kT times the current score. I'm not sure how useful this is in practice. Restraints that could benefit: - excluded volume: there is no need to evaluation all of the pairs once we have found several collisions. How to best combine this optimization with the optimization currently used of only updating the non-bonded list periodically is non-obvious, especially given the often incoherent evaluations performed by domino. - connectivity restraints: any large edge that is required means we don't have to check anything else - pairsrestraint: we can skip evaluating later pairs of the earlier ones score too badly Further directions: - in domino we often only care if each restraint score is below its maximum (assuming there is no model maximum score). In this case, many restraints can do a cheaper computation to simply determine if things are OK. The context of using this is much narrower and the benifit smaller, so I'm not sure we want to implement this now. The impact on existing code: - no changes need to be made to any restraints/pair scores - restraint/pair scores can be updated to take advantage of this functionality by overloading the Restraint::do_evaluate_if_ok() method (a default implementation is provided) My questions are: - are there other areas where these ideas might be applied which should be kept in mind when implementing this functionality to make it more generally useful? - are there other restraints which could easily benefit?

2 2

RMF warning
by Daniel Russel 23 Jun '11

23 Jun '11

I had to make a chance to the rmf file format that is not backwards compatible. If there is someone who has lots of data saved for whom going through another format is a problem, let me know and we can work on a solution. Otherwise, be aware that updating IMP today will render your old files unreadable. In detail, the problem is that I was using an hdf5 attribute to store the list of keys. Apparently there is a maximum size on how much data can be stored in attributes, and I was running in to that limit. As a result, the list of keys has been moved to a data set, which is something that can grow arbitrarily large.

1 0

removing incremental evaluation
by Daniel Russel 20 Jun '11

20 Jun '11

I'd like to remove the support for incremental evaluation from IMP. My reasons for proposing it are: - it complicates various code in the model and Particle - there seems to be a reasonable runtime cost even when it is turned off - we have an alternative mechanism based on decomposing restraints and analyzing the dependency graph which is more efficient and simpler. This method is not super automatic, but can be made so if needed. - I don't think anyone is using it.

3 2

IMP::display::CMMWriter
by Davide Baù 14 Jun '11

14 Jun '11

Hi, I have some problem in understanding how the new writers work. I'm using the CMMwriter like, writing out only the final model (frame): > # Setup a CMM writer > writer = IMP.display.CMMWriter("model.%1%.cmm") > log = IMP.display.WriteOptimizerState(writer) > g = IMP.display.XYZRsGeometry(ps) > log.add_geometry(g) > log.update() This generates the model file with the coordinates (each line is a chimera marker) and a file called "model.%1%.cmm", which only contains the header line: <marker_set name="model.0.cmm"> This header should be in the model file, but since it is missing it causes a problem with chimera. Is there a way to get rid of the model.%1%.cmm file so that to have just one file containing all the informations? I've tried without the %1% string: > writer = IMP.display.CMMWriter("model.1.cmm") but that rises the error: > e = o.do_optimize(steps) > File "/usr/src/imp/trunk/build/lib/IMP/core/__init__.py", line 4338, in do_optimize > return _IMP_core.MonteCarlo_do_optimize(self, *args) > _IMP.InternalException: Cannot set frame on writer without file name pattern. Thanks, Davide -- Davide Baù Structural Genomics Laboratory Bioinformatics & Genomics Department, Prince Felipe Research Center Avda. Autopista del Saler 16, 46012 Valencia, Spain Tel: +34 96 328 96 80 (ext. 1004) Fax: +34 96 328 97 01 email: dbau(a)cipf.es web: http://bioinfo.cipf.es/dbau/ *** http://www.saveaswwf.com ***

3 3