May 2011 - IMP-dev - salilab.org

change of optimizer performance from v7018 to v9277
by Pia Unverdorben 25 May '11

25 May '11

Dear all, is it possible, that something fundamental changed in the optimizers, steepest descent as well as conjugate gradients? I try to fit a protein to an em density, simulated from another conformational state using the em restraint and plenty of harmonic restraints. With an older version (revision 7018) it moves towards the density, whereas with the revision 9277 the score and the positions barely change, irrespective of the number of steps, threshold, stepsize and different weights of restraints. More importantly, i did compare the optimization results of v 7018 and 9277 of conjugate gradient optimization and steepest descent from identical starting positions with identical restraints and optimizer settings. the initial scores are identical, but then the problem starts: For example, with v7018 my model improves from a score of 1835 to 1101 after 100 steps of steepest descent optimization. However, with v9277 my score increases (!) to 1836. behavior of conjugate gradients similarly deteriorates for v9277, albeit i did not observe increases of scores. nevertheless, improvements in score were disappointingly low. Does someone know, why the optimizers behave so differently, for my purposes worse? Thanks a lot! Regards, Pia

3 3

imp-dev meeting Wednesday
by Daniel Russel 24 May '11

24 May '11

On running IMP in parallel, particularly Domino. It will mostly be unstructured discussion about planning for the future, rather than recipes for how to do things. 11am. Room to be decided. --Daniel

1 0

Re: [IMP-dev] change of optimizer performance from v7018 to v9277
by Daniel Russel 18 May '11

18 May '11

On May 18, 2011, at 9:05 AM, Friedrich Foerster wrote: > I remember painfully writing a test case for the em restraint and the derivatives when I implemented the em score. Is that test not functional anymore? They are still there, but I'm not sure if they pass. It also isn't clear to me what exactly they test as they use some modeller functionality that I am not familiar with to do the actual checking. (Reason I'm not sure about their status: I broke the build last night on all platforms except windows, and for some reason the windows test run didn't terminate. And the em tests have been failing in general for rather a while, and it is not something I've looked in to. And I don't have modeller installed on any of my machines, so that is not a test I run myself). Keren, do you have more sense of the status of those tests and, if they are broken, when they broke? Anyway, there is a test I added a bit back to test that the magnitudes of the derivatives make sense, since they didn't (eg that they roughly match the number one gets for a numeric approximation of the derivative). So there are definitely problems that need to be fixed :-) > > Cheers > > Friedrich > > Dr. Friedrich Foerster > Max-Planck Institute of Biochemistry > Am Klopferspitz 18 > D-81252 Martinsried > > www.tomotronic.org > > > On 18.05.2011, at 16:55, Keren Lasker <kerenl(a)salilab.org> wrote: > >> Major changes to the code from my part were in verion 7212 in class CoarseCC, which mainly dealt with rigid bodies support. >> I am not sure if that is the reason for Pia's problem, but as she needs to use EM based optimization relatively soon, i will take a look again. >> In any case - Pia - it would be useful if you can write a test case that I can check things against. >> keren. >> On May 18, 2011, at 7:43 AM, Daniel Russel wrote: >> >>> I suspect it is an issue with em::FitRestraint as the derivatives returned by that restraint are currently broken (the direction is fine, but the magnitude off by quite a bit). As a result derivative-using optimizers don't perform well with it. >>> >>> Keren, Ben, is there a plan for fixing it? >>> --Daniel >>> >>> On May 18, 2011, at 6:31 AM, Pia Unverdorben wrote: >>> >>>> Dear all, >>>> >>>> is it possible, that something fundamental changed in the optimizers, steepest descent as well as conjugate gradients? >>>> I try to fit a protein to an em density, simulated from another conformational state using the em restraint and plenty of harmonic restraints. With an older version (revision 7018) it moves towards the density, whereas with the revision 9277 the score and the positions barely change, irrespective of the number of steps, threshold, stepsize and different weights of restraints. >>>> More importantly, i did compare the optimization results of v 7018 and 9277 of conjugate gradient optimization and steepest descent from identical starting positions with identical restraints and optimizer settings. the initial scores are identical, but then the problem starts: >>>> For example, with v7018 my model improves from a score of 1835 to 1101 after 100 steps of steepest descent optimization. However, with v9277 my score increases (!) to 1836. behavior of conjugate gradients similarly deteriorates for v9277, albeit i did not observe increases of scores. nevertheless, improvements in score were disappointingly low. >>>> Does someone know, why the optimizers behave so differently, for my purposes worse? >>>> >>>> Thanks a lot! >>>> >>>> Regards, >>>> >>>> Pia >>>> _______________________________________________ >>>> IMP-dev mailing list >>>> IMP-dev(a)salilab.org >>>> https://salilab.org/mailman/listinfo/imp-dev >>> >> >> >> _______________________________________________ >> IMP-dev mailing list >> IMP-dev(a)salilab.org >> https://salilab.org/mailman/listinfo/imp-dev >>

3 5

recently in IMP
by Daniel Russel 14 May '11

14 May '11

- An interactive mode has been added to IMP::domino::DominoSampler. The interactive mode is useful for understanding domino sampling runs and distributing domino comptutations across a cluster. - Functions IMP::domino::set_assignments() and IMP::domino::get_assignments() were added which support writing subset assignments to data fields in hdf5 files. These can be used to helping to distribute domino on a cluster as well as debugging the sampling process. - a python function IMP.show_graphviz() has been added that attempts to render a graph with \c dot and then open the pdf. - The python functions IMP.show_altgraph() and IMP.get_networkx_graph() have been added to provide alternatives for quick viewing of \imp graphs. Unfortunately, they still pale compared to graphviz. - IMP.atom.create_cover() was added to facilitate creating a summary particle for part of a hierarchy in order to accelerate restraints. - IMP.hdf5 was renamed to IMP.rmf, the proposed name of the file format that it supports. That also better separates the hdf5 support from the file format support. - The old I/O functions have been marked as deprecated in favor of IMP.hdf5. You may need to compile with deprecated="yes" if you use them. - The python graph interface now has a remove_vertex method.

1 0

atom::setup_as_rigid_body
by Daniel Russel 11 May '11

11 May '11

There is some bug but with setup_as_rigid_body which makes is not work correctly with Selections (the selection will only choose the root particle under certain circumstances when it should choose the leaves as some of the internal particles are given a radius of 0). Since the method is not documented or supported and has other areas where it doesn't work right, I'm not terribly motivated to fix it and would rather simply remove it. Thoughts?

2 2

Residue number vs residue index
by Daniel Russel 03 May '11

03 May '11

It was pointed out that the thing we call a "residue index" in imp appears to be more commonly referred to as a "residue number" (that is, the number of the residue as stored in the pdb file). Given that there are a number of classes involved and it may involve a lot of outside code, it is unclear how to best deal with it. The things affected include - IMP::atom::Residue::get_index() and set_index() - IMP::atom::get_residue_index() - IMP::atom::Selection (the residue_indexes and residue_index arguments and functions) - IMP::atom::Fragment, various functions - IMP::atom::Domain, various functions Possible actions are: 1) improve the documentation to say residue index= residue number as we tend to use the name index for such things in IMP 2) provide new functions which use number instead of index in their names and hide the old ones from the docs (but support them indefinitely, as they are all very simple) 3) provide new functions and deprecated the old ones 4) remove all such functions :-) Thoughts?

2 3