September 2008 - IMP-dev

Formatting suggestion
by Daniel Russel 30 Sep '08

30 Sep '08

A number of projects I have been involved in replace the "namespace IMP {" and "} // namesapce IMP" sorts of lines with "IMP_BEGIN_NAMESPACE" and "IMP_END_NAMESPACE" or "IMP_DOMINO_BEGIN_NAMESPACE" "IMP_DOMINO_END_NAMESPACE". Such a change has several advantages: - the close braces are automatically labelled - you aren't constantly fighting the editor's desire to indent because you are in a nested block - IMP_DOMINO is once conceptual unit, and having it on two separate lines just conceals that - we can have macros for internal namespaces so we have standard names for that - it is a pretty simple matter to write a sed script which changes most of the existing code to use these macros Just a suggestion.

2 1

Directory structure
by Daniel Russel 26 Sep '08

26 Sep '08

Given the shift to modules, it is not clear that the original include directory structure is the best any more (with IMP/restraints holding restraints, for example). I would propose that all public classes/ functions in the modules API be in headers in the module main include directory. Structuring things this way means that the include paths are the same as the namespace nesting which is the same as the python module structure, which is nice. And the base classes are all together in the include/IMP directory already.

2 6

links with relative paths
by Daniel Russel 25 Sep '08

25 Sep '08

Can IMP use links with relative paths rather than absolute paths for the headers? Now copying the build directory or moving it makes all the links invalid. Or does something break if you use relative paths?

2 1

scons standards
by Daniel Russel 25 Sep '08

25 Sep '08

For some reason or another, the _wrap.h files aren't up to standards and the _wrap.py have odd indentation :-)

2 1

doc cleanups
by Daniel Russel 25 Sep '08

25 Sep '08

In preparation for the presentation, I made various improvements to the documentation. I added a new group exp_restraints for experimental restraints. The only things with non-obvious reasons are my removing of the brief descriptions of many of the Particle methods and the making of the internal state private. They are both to make the web page cleaner as it has many methods and the removed brief descriptions simply restated the functions name. Their effect was to make the methods list much longer and harder to read without providing any additional information. Some of the changes touch other modules, either adding or fixing doxygen groups. It also adds doc/internal to the doxygen search path as a place to put things like the introduction I have written (not submitted as it is still not stable). And it makes the list have paths on different lines as was discussed in a previous email.

1 0

doc dependence on lib
by Daniel Russel 24 Sep '08

24 Sep '08

doing "scons doc" builds the kernel libraries and python wrapper which it would be nice to avoid as those can take a long while on a laptop. I loose the dependency trail at build/include :-)

2 5

IMP reorg
by Ben Webb 24 Sep '08

24 Sep '08

OK, so given Daniel's and Keren's suggestions, here's what I'll do: I will create three new modules: core, exp, and modeller. All developers will have commit privileges to the core and exp modules. Modifications to the kernel will still need to go through me. The other specific modules will remain as they currently are - each with a given owner (Keren for em and domino, me for modeller). The core module will contain most of the code that most users and other projects use. Thus, most of the concrete classes currently in the kernel (headers in subdirectories, not the top-level directory - e.g. ConnectivityRestraint, RestraintSet, Daniel's nonbonded lists) will move into the core module. API changes, deletions from, or additions of new restraints, etc. to core (but not simply new methods to existing classes, or cleanups of internals, etc.) should be announced on imp-dev *before* the code is committed. (A general outline of new features is fine - I just don't want to wake up one day and find 20,000 lines of code in some entirely new set of classes.) Code in the core module can be at various levels of maturity. The only requirement is that the test cases pass "most of the time". (The general rule of thumb should be that the nightly builds do not break for more than 3 consecutive days.) As Daniel suggested, immature code should be marked in a Doxygen group so that users can be aware. The exp or experimental module is for code that is (scientifically) experimental in nature. I will put particle refiners, cover bonds, tunnel singleton score, and wormlike chain in there. This code should still have test cases, of course, and be marked if it is immature just like core is, but won't be built or tested as part of the nightly builds, so there is less requirement that the tests pass all the time. If a bunch of classes are envisaged for a given project (e.g. NPC dynamics, a new optimizer, etc.) then probably the best thing is a distinct module for that, where you'd have more flexibility, or even something entirely external (e.g. Assembler). Finally, the modeller module will contain what's currently in modeller.py, plus anything else that requires Modeller, so that the kernel and core can be built and tested without needing Modeller around. (It will also get some additional Modeller-specific restraints which I'll need to entice users away from Modeller.) I will decide what goes in each module. Movements between modules should be exceedingly rare. It should certainly *not* be the case that somebody uses exp simply to write dodgy code that doesn't work, and then moves it to core when the bugs are fixed. If stuff really needs to be moved between modules, *I* will do it. I will have the nightly build system email imp-dev if the nightly builds fail. The culprit will then hopefully fix the problem ;) (with my assistance if necessary). I will create a Bugzilla instance to keep track of IMP bugs and wishlist items. Bug fixes should, of course, come with a testcase to ensure the bug stays fixed, and to some extent document the nature of said bug. I will also work with Keren to put Assembler into the automatic build system. I am reluctant to set too many rules, but there is an obvious requirement that things rely reasonably stable, so that developers can concentrate on the science and not on updating their code to work with the most recent changes. It is my responsibility to keep things that way, so I will revert patches (or users ;) if they cause extreme breakage. I'll be relying on everyone to take care with their code and its testcases. So while I cannot make everybody happy, I can do the next best thing and make everybody equally unhappy (hopefully only moderately so ;). Obviously there is a bunch of stuff to do, and I am not going to spend my holiday weekend working on it. Plus, I will be in Chicago for most of next week. So probably things will be ready to move forward at the start of the following week. If anybody can point out an obvious problem with this solution, I may have to think about (and delay) things some more. Otherwise, have a relaxing and sunny Labor Day. ;) Ben -- ben(a)salilab.org http://salilab.org/~ben/ "It is a capital mistake to theorize before one has data." - Sir Arthur Conan Doyle

2 4

Re: [IMP-dev] Build system failure, 09/24/2008
by Ben Webb 24 Sep '08

24 Sep '08

build-system(a)salilab.org wrote: > Build system failed on 09/24/2008. > Failed components: MODELLER, IMP, MDT, OD-DOPE. An update of the Intel Fortran compiler for Windows broke Modeller and dependent projects; now fixed. I also fixed a minor problem with the IMP modeller module tests in r730. Ben -- ben(a)salilab.org http://salilab.org/~ben/ "It is a capital mistake to theorize before one has data." - Sir Arthur Conan Doyle

1 0

Other very minor comments
by Daniel Russel 23 Sep '08

23 Sep '08

"scons test" and "install" and things should by default install everything that they know how to build/install/test as most libraries behave that way and doing everything individually requires a lot of typing. At least on unix systems lib is more popular than libs (regarding build/libs). Perhaps the name of doc/internal should be changed to something more descriptive like API since it is actually what anyone who uses imp should read (as opposed to just building IMP). The boost version should probably be moved up to at least 1.35 as it doesn't guarantee backwards compatibility and I don't think we any idea about old versions.

2 3

examples
by Daniel Russel 23 Sep '08

23 Sep '08

First, the example directory should probably be publicly writable (if it is not already) as that is one of the hardest sort of things to get people to do. The easier anyone can add to examples, the better. Second, it would be good to put the examples.passed file elsewhere (and the state*.vrml) since that is a directory that people will be browsing and those things just clutter it up. It might make sense to have a directory under build which can be used as a tmp dir to keep cruft out of the way.

1 0