Daniel Russel wrote: > I have written some code to use BALL to > read PDB files and turn them in to particles and a hierarchy (and to > evaluate MD energy for such a hierarchy). Keren wanted to use this too. > Do you have a preferred SVN structure/location for this sort of thing? > Something like trunk/IMP_BALL with include and src directories? And > pyext too I guess.
Sure, my policy is, roughly speaking: if it doesn't break the core unit tests, put it in. Better to have it in SVN where we can poke holes in it than it to live on your laptop for ever.
But this may be an opportune point to discuss the layout of the IMP repository. Right now it looks basically like:
imp libsaxs src doc mdt bin src doc pyext test new_imp bin IMP tests impEM rsr doc pyext od_dope src doc pyext test tnt python test tools
The top-level directories are independent modules which are built separately but use a shared set of build scripts (in the tools directory). libsaxs is Frido's SAXS module, which plugs in to Modeller; mdt uses the Modeller C and Python interfaces; od_dope uses the Modeller C interface; tnt and new_imp use the Modeller Python interface. new_imp is essentially what Bret turned in before he left, and which Daniel, Keren and I are currently working on.
The projects have similar sub-directories: bin for generated binaries; src for C/C++ source code; doc for documentation; pyext for Python extensions; python for pure Python code; test for testcases. However, new_imp uses 'IMP' as its source code directory, and the tests live under that. new_imp also contains some sub-projects, such as Keren's impEM interface and Bret's RSR (Restrainer web interface).
As Daniel pointed out, new_imp's naming is a little inconsistent (e.g. imp/new_imp/IMP to get to the sourcecode) so what about renaming the IMP subdirectory to src, and the new_imp top-level directory to kernel (or perhaps base) ? Dependent projects such as impEM and rsr would then become top-level directories:
imp libsaxs mdt kernel bin src test doc pyext rsr impEM od_dope tnt tools
This scheme would, however, require you to manually handle dependencies between the projects (e.g. impEM and rsr would be independent projects) but that's not a huge hurdle - just run 'scons' in the kernel directory before running 'scons' in the impEM directory. A slightly more radical rearrangement could look like
imp libsaxs mdt bin kernel src test doc pyext rsr impEM od_dope tnt tools
and use a top-level build system which would track dependencies between projects, staging binaries and libraries for all IMP projects to the bin directory (but the downside is the targets are a bit more wordy - e.g. 'scons kernel', 'scons kernel-test', 'scons 'impEM', scons 'impEM-test' rather than just 'scons' and 'scons test' in the kernel and impEM directories).
Thoughts?
There is, of course, also the issue over whether libsaxs, mdt, od_dope and tnt belong here or in their own repository. My understanding is that the Grand Plan is that they'll become part of IMP eventually, which is why they're there. But if we want to distribute them under a non-free license for any reason, it might make sense to put them in a separate repository in the future.
Ben
My suggestion would be to have each thing (including IMP, which would be called kernel) look something like
kernel src: various .cpps include IMP: public include files which define things in the IMP namespace tests doc lib: where built libs go pyext bin python
The include/src separation makes it clearer where external (but within lab) users should look for headers used by the project. We really should have all are headers be places than everyone can write #include <IMP/foo.h" to include so build won't break when things get installed. All the include paths can presumably be added to the scons config file for everything so cross-project dependencies just work.
On Oct 16, 2007, at 6:38 PM, Ben Webb wrote:
> Daniel Russel wrote: >> I have written some code to use BALL to read PDB files and turn >> them in to particles and a hierarchy (and to evaluate MD energy >> for such a hierarchy). Keren wanted to use this too. Do you have a >> preferred SVN structure/location for this sort of thing? Something >> like trunk/IMP_BALL with include and src directories? And pyext >> too I guess. > > Sure, my policy is, roughly speaking: if it doesn't break the core > unit tests, put it in. Better to have it in SVN where we can poke > holes in it than it to live on your laptop for ever. > > But this may be an opportune point to discuss the layout of the IMP > repository. Right now it looks basically like: > > imp > libsaxs > src > doc > mdt > bin > src > doc > pyext > test > new_imp > bin > IMP > tests > impEM > rsr > doc > pyext > od_dope > src > doc > pyext > test > tnt > python > test > tools > > The top-level directories are independent modules which are built > separately but use a shared set of build scripts (in the tools > directory). libsaxs is Frido's SAXS module, which plugs in to > Modeller; mdt uses the Modeller C and Python interfaces; od_dope > uses the Modeller C interface; tnt and new_imp use the Modeller > Python interface. new_imp is essentially what Bret turned in before > he left, and which Daniel, Keren and I are currently working on. > > The projects have similar sub-directories: bin for generated > binaries; src for C/C++ source code; doc for documentation; pyext > for Python extensions; python for pure Python code; test for > testcases. However, new_imp uses 'IMP' as its source code > directory, and the tests live under that. new_imp also contains > some sub-projects, such as Keren's impEM interface and Bret's RSR > (Restrainer web interface). > > As Daniel pointed out, new_imp's naming is a little inconsistent > (e.g. imp/new_imp/IMP to get to the sourcecode) so what about > renaming the IMP subdirectory to src, and the new_imp top-level > directory to kernel (or perhaps base) ? Dependent projects such as > impEM and rsr would then become top-level directories: > > imp > libsaxs > mdt > kernel > bin > src > test > doc > pyext > rsr > impEM > od_dope > tnt > tools > > This scheme would, however, require you to manually handle > dependencies between the projects (e.g. impEM and rsr would be > independent projects) but that's not a huge hurdle - just run > 'scons' in the kernel directory before running 'scons' in the impEM > directory. A slightly more radical rearrangement could look like > > imp > libsaxs > mdt > bin > kernel > src > test > doc > pyext > rsr > impEM > od_dope > tnt > tools > > > > Thoughts? > > There is, of course, also the issue over whether libsaxs, mdt, > od_dope and tnt belong here or in their own repository. My > understanding is that the Grand Plan is that they'll become part of > IMP eventually, which is why they're there. But if we want to > distribute them under a non-free license for any reason, it might > make sense to put them in a separate repository in the future. > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle
A couple more comments: I think we need to think about what happens when IMP gets installed by an end user. It is nice if they can install the headers (in /usr/ include for example) and then build projects that look more or less like ours. This requires all headers which add things to the IMP namespace sitting in directories called IMP among other things. I don't know what the right thing to do with python stuff is.
I would also suggest that each project: kernel, rsr etc have a separate root level scons file so that it really is separate and I don't have to keep a copy of everyone else's code around. It can refer to ../tools for example to get shared defs.
On Oct 16, 2007, at 6:38 PM, Ben Webb wrote:
> Daniel Russel wrote: >> I have written some code to use BALL to read PDB files and turn >> them in to particles and a hierarchy (and to evaluate MD energy >> for such a hierarchy). Keren wanted to use this too. Do you have a >> preferred SVN structure/location for this sort of thing? Something >> like trunk/IMP_BALL with include and src directories? And pyext >> too I guess. > > Sure, my policy is, roughly speaking: if it doesn't break the core > unit tests, put it in. Better to have it in SVN where we can poke > holes in it than it to live on your laptop for ever. > > But this may be an opportune point to discuss the layout of the IMP > repository. Right now it looks basically like: > > imp > libsaxs > src > doc > mdt > bin > src > doc > pyext > test > new_imp > bin > IMP > tests > impEM > rsr > doc > pyext > od_dope > src > doc > pyext > test > tnt > python > test > tools > > The top-level directories are independent modules which are built > separately but use a shared set of build scripts (in the tools > directory). libsaxs is Frido's SAXS module, which plugs in to > Modeller; mdt uses the Modeller C and Python interfaces; od_dope > uses the Modeller C interface; tnt and new_imp use the Modeller > Python interface. new_imp is essentially what Bret turned in before > he left, and which Daniel, Keren and I are currently working on. > > The projects have similar sub-directories: bin for generated > binaries; src for C/C++ source code; doc for documentation; pyext > for Python extensions; python for pure Python code; test for > testcases. However, new_imp uses 'IMP' as its source code > directory, and the tests live under that. new_imp also contains > some sub-projects, such as Keren's impEM interface and Bret's RSR > (Restrainer web interface). > > As Daniel pointed out, new_imp's naming is a little inconsistent > (e.g. imp/new_imp/IMP to get to the sourcecode) so what about > renaming the IMP subdirectory to src, and the new_imp top-level > directory to kernel (or perhaps base) ? Dependent projects such as > impEM and rsr would then become top-level directories: > > imp > libsaxs > mdt > kernel > bin > src > test > doc > pyext > rsr > impEM > od_dope > tnt > tools > > This scheme would, however, require you to manually handle > dependencies between the projects (e.g. impEM and rsr would be > independent projects) but that's not a huge hurdle - just run > 'scons' in the kernel directory before running 'scons' in the impEM > directory. A slightly more radical rearrangement could look like > > imp > libsaxs > mdt > bin > kernel > src > test > doc > pyext > rsr > impEM > od_dope > tnt > tools > > and use a top-level build system which would track dependencies > between projects, staging binaries and libraries for all IMP > projects to the bin directory (but the downside is the targets are > a bit more wordy - e.g. 'scons kernel', 'scons kernel-test', 'scons > 'impEM', scons 'impEM-test' rather than just 'scons' and 'scons > test' in the kernel and impEM directories). > > Thoughts? > > There is, of course, also the issue over whether libsaxs, mdt, > od_dope and tnt belong here or in their own repository. My > understanding is that the Grand Plan is that they'll become part of > IMP eventually, which is why they're there. But if we want to > distribute them under a non-free license for any reason, it might > make sense to put them in a separate repository in the future. > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle
Daniel Russel wrote: > A couple more comments: > I think we need to think about what happens when IMP gets installed by > an end user. It is nice if they can install the headers (in /usr/include > for example) and then build projects that look more or less like ours. > This requires all headers which add things to the IMP namespace sitting > in directories called IMP among other things. I don't know what the > right thing to do with python stuff is.
Well, this is already the case with the exception of the headers. The shared libraries and Python extensions already should get put in sensible places by 'scons install' (try it - you may find prefix= useful though). I didn't install the headers because nobody has asked for them yet. But it makes sense to split internal from external headers (currently the distinction is simply that internal headers aren't installed).
> I would also suggest that each project: kernel, rsr etc have a separate > root level scons file so that it really is separate and I don't have to > keep a copy of everyone else's code around. It can refer to ../tools for > example to get shared defs.
True - this would also simplify the build system. But we may need to revisit it if we build a very complex set of interdependent modules.
Ben
participants (2)
-
Ben Webb -
Daniel Russel