- IMP-dev - salilab.org

Re: [IMP-dev] ISD module compile error on Mac?
by Daniel Russel 20 May '12

20 May '12

I doubt things have ever been looked into for doing static compilation of isd with cholmod. One needs to figure out which libraries cholmod depends on and add them to the add_dependency call (when doing dynamic linking they are brought in automatically). To help us get started, you can do ldd on the cholmod dynamic lib (somehting like /usr/lib/libcholmod.so) and send the results (if you are in a mac, you need to run "otool -L " on the .dylib instead). That shows what libraries the dynamic lib brings in on your platform. If you don't need isd, just disbable it as I'm not sure we want to get this working unless really needed as it could be painful. On May 19, 2012, at 7:41 PM, Seung Joong Kim wrote: > Hi Yannick, > > Suddenly I have a problem in compiling the ISD module as follows. Do you have any idea what happens? I recently upgraded to OSX 10.7 Lion, hopefully that is not an issue.. Thanks a lot! > > Compiling shared build/src/imp_isd_config.cpp > Compiling modules/isd/bin/benchmark_gaussianprocess.cpp > Install file: "build/swig/IMP_isd.i" as "build/data/swig/IMP_isd.i" > Running swig on file build/swig/IMP_isd.i > /opt/local/bin/swig -castmode -interface _IMP_isd -DPySwigIterator=IMP_ISD_PySwigIterator -DSwigPyIterator=IMP_ISD_SwigPyIterator -python -c++ -naturalvar -fvirtual -Wextra -o /Users/sjkim/imp/trunk/build/src/IMP_isd_wrap.cpp-in -oh /Users/sjkim/imp/trunk/build/src/IMP_isd_wrap.h-in -Ibuild/swig -I/opt/local/include -I/usr/include -I/usr/local/include -I/Users/sjkim/imp/trunk/build/include -I/Users/sjkim/imp/trunk -DIMP_SWIG /Users/sjkim/imp/trunk/build/swig/IMP_isd.i > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > /Users/sjkim/imp/trunk/build/include/IMP/isd/bivariate_functions.h:129: Warning 314: 'lambda' is a python keyword, renaming to '_lambda' > Install file: "build/src/IMP.isd.py" as "build/lib/IMP/isd/__init__.py" > Patching swig file build/src/IMP_isd_wrap.h > Patching swig file build/src/IMP_isd_wrap.cpp > Compiling shared build/src/IMP_isd_wrap.cpp > Linking Shared Library build/lib/libimp_isd.dylib > Undefined symbols for architecture x86_64: > "_dpotrf_", referenced from: > _r_cholmod_super_numeric in libcholmod.a(cholmod_super_numeric.o) > "_zpotrf_", referenced from: > _c_cholmod_super_numeric in libcholmod.a(cholmod_super_numeric.o) > _z_cholmod_super_numeric in libcholmod.a(cholmod_super_numeric.o) > ld: symbol(s) not found for architecture x86_64 > collect2: ld returned 1 exit status > scons: *** [build/lib/libimp_isd.dylib] Error 1 > scons: building terminated because of errors. > > - SJ > > -- > Seung Joong Kim, Ph.D. > > Postdoctoral Scholar, Andrej Sali group > Department of Bioengineering and Therapeutic Sciences > University of California at San Francisco (UCSF) > 1-217-649-2147 >

3 6

sampler vs. optimizer
by Dina Schneidman 19 May '12

19 May '12

Hi, I am trying to figure out the difference between sampler and optimizer. When each one should be used/developed? What is the relationship between them? How each one works with restraints and scoring functions? Dina

4 5

add_attribute vs. add_cache_attribute
by Dina Schneidman 18 May '12

18 May '12

Hi, What is the difference between the two functions for adding Particle attributes? Dina

2 1

git and svn
by Daniel Russel 11 May '12

11 May '12

Since there have been a couple of questions about using git with imp svn recently, I finished a cheat sheet I had started. See < http://salilab.org/imp/wiki/git> and please add more helpful commands.

1 0

Recently in IMP
by Daniel Russel 30 Mar '12

30 Mar '12

There have been a lot of minor changes since I last sent out such an email, I'll leave those to the release notes and instead and detail the more interesting ones. Scoring of models is now organized around ScoringFunction objects. The look kind of like how the scoring part of a the model used to look (with evaluate calls), but are independent objects. You can create different types and instances of them (most commonly, a IMP::core::RestraintsScoringFunction from a list of restraints). They can then be used to specify how a particular optimizer or sampler scores it models (making it easier to use different scoring functions for different things). And different scoring methods can be used (eg an IMP::core::IncrementalScoringFunction when only moving a small number of particles at a time). This change resulted in a vast simplification on the code used to evaluate restraints and resulted in the fixing of many bugs. Moving forwards, it makes it easier to support multithreaded scoring (which can be done by implementing a new ScoringFunction subclass). The changes are more or less backwards compatible as basic scoring through Model::evaluate and Restraint::evaluate is still supported, so little code should need to be changed. There is a new base class IMP::ModelObject for all objects which introduce dependencies in the model. IMP::Particle, IMP::ScoreState, IMP::Restraint, IMP::ScoringFunction and IMP::core::Mover all inherit from it. Such objects, when they are registered with the Model (either via a set_model() call or passing the Model to the constructor, or, for backwards compatibility, adding a restraint to the Model or ScoringFunction or adding a ScoreState to the Model) are considered part of the Model (until they are unregistered or destroyed). They can take action when the set of dependencies changes, so that relationships can be correctly updated when, eg, rigid bodies are destroyed or added (most classes don't do this yet). This also opens the way for IMP::core::Movers to have more intelligence, so they can, for example, weight the move probabilities based on how much a given particle contributes to a score (and properly maintain this weighting as things change). Also, decomposing restraints into their simplest parts (eg a non-bonded list into n^2 pairwise terms), works in the obvious manner and so contexts that want to use such simple restraints (such as incremental scoring) can do so directly. Scoring in Domino has been simplified around a single IMP::domino::RestraintCache object that provides a centralized cache for all restraints used. This provides a single point to control memory usage (by bounding the total number of cache entries), serialize and unserialize. In addition, coupled with the previous change, it removes the need for the IMP::domino::OptimizeRestraints objects as the restraint cache does that operation internally. Many unnecessary #include statements in IMP headers have been removed. As a result, it may be necessary to add #includes to code not in the IMP repository to get it to compile. As a general rule, you should #include the header defining any symbol you use in a given .h or .cpp file (and not depend on it being indirectly included through another header). Restraints now store (and can produce) the score returned in their last evaluate call. I made efforts to provide backwards compatibility throughout the changes. But I did remove various methods that I though were unlikely to be used by anyone and so not worth the trouble to provide in the new internal architecture. If there is some method that I removed that you really need, email me and we can figure out a work around or can provide a backwards compatibility interface.

1 0

Upcoming changes in IMP
by Daniel Russel 02 Mar '12

02 Mar '12

There were various inconsistencies in how IMP handled maximum scores (eg who was responsible for reporting that scores were bad, how maxima were handled when decomposing restraints etc). This led to various bugs in domino and other optimizers when used with maximum scores. In order to address this and make cache and make cache management easier in Domino, I'd like to introduce the following changes: - introduce IMP::domino::RestraintCache which provides a centralized cache for all restraint evaluations within a domino run. It has an interface that can be used to extract scores and will, eventually, support reading from and writing to disk. - IMP::domino::RestraintScoreSubsetFilterTable and IMP::domino::MinimumRestrainScoreSubsetFilterTable will move to using this. That change will require code changes if you do something beyond just use them for filtering (eg extracting scores). - reorganize general restraint evaluation around the concept of an IMP::ScoringFunction. This will bundle a list of restraints as well as cached information to aid in speedy evaluation. Since it is an object independent from the model, it is easier to manage having multiple scoring functions around (as is done in a number of Optimizers, Domino, etc). The model will have a default ScoringFunction, so once again, basic usage (Model::evaluate(), Restraint::evaluate()) will not change, but more involved evaluates, such as explicitly passing restraints to the model to evaluate or using Model::evaluate_if_good()) will change by requiring that you create a ScoringFunction first. These provide a much cleaner and simpler set of functionality to manage restraint evaluation, and so should make further experiments less likely to introduce bugs as has happened recently. Do these make sense? Any issues? More questions? As usual, code in the IMP repository will be updated as needed. I more or less have the changes made (along with numerous bug fixes that came up in the process), but will be away for the weekend, so will wait until at least Monday.

1 0

Re: [IMP-dev] Compiling IMP on Red Hat
by Ben Webb 29 Feb '12

29 Feb '12

On 02/21/2012 08:36 AM, Davide Baù wrote: > On Feb 21, 2012, at 5:23 PM, Ben Webb wrote: >> Is there a reason this machine is years out of date? (The latest >> update for RHEL 5 is 5.7.) It seems entirely possible that a bug in >> one of the system headers was fixed in the interim. > Actually I don't know. I will contact the informatics department to > ask if it is possible to update the system (unfortunately we can > just "use" the cluster.) You can point them to http://en.wikipedia.org/wiki/RHEL#Version_history 5.3 has been obsolete since 5.4 was released in September 2009. > I got a error on the algebra module: Errors building: > build/src/IMP.algebra.py. It's curious that there's a red box on the > algebra module on Mac while it compiled with no errors on my Mac. It's a different Mac (32-bit code is built on a 10.4 - Tiger - machine). Chances are your machine is more similar to our Snow Leopard 64-bit build machine. I've added more descriptive labels to this page, which should show up tomorrow. ... > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:243: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:244: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:245: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:246: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:247: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:248: > Error: Template 'GridD' undefined. > /project/devel/dbau/imp/imp-r12787/build/swig/IMP_algebra.i:249: Odd. The SWIG file should be parsing algebra's GridD.h header, which defines that template. But Daniel did make some changes there recently - are you sure you don't have a mismatch of versions somewhere (perhaps an older version of GridD.h paired with a newer SWIG interface)? I'd suggest a fresh checkout to be sure. Ben -- ben(a)salilab.org http://salilab.org/~ben/ "It is a capital mistake to theorize before one has data." - Sir Arthur Conan Doyle

3 4

Compiling IMP on Red Hat
by Davide Baù 21 Feb '12

21 Feb '12

Hi, I'm trying to compile IMP on a Red Hat machine: > $ cat /etc/issue > Red Hat Enterprise Linux Server release 5.3 (Tikanga) > Kernel \r on an \m > > $ uname -a > Linux n9 2.6.18-128.1.6.el5.Bull.3 #1 SMP Thu Nov 5 11:31:54 CET 2009 > x86_64 x86_64 x86_64 GNU/Linux I first tried with the latest release (r12787), which compiled on a Mac and on Ubuntu, but not on Red Hat. Since there was a notification of IMP build failure of this release, I tried with the latest successful build revision (r11171), but I get a build/src/IMP_base_wrap.os compilation error (I have reported the details below). The only IMP version I was able to compile on Red Hat was the r12645, but had to disable domino. Can anyone help me out with this issue? Thanks, Davide > g++ -o build/src/IMP_base_wrap.os -c -isystem /apps/IMP/include -isystem /apps/BOOST/1_48_0/include -isystem /apps/CGAL/3.9/include -Wno-deprecated -Winit-self -Wcast-align -fno-operator-names -Wundef -Woverloaded-virtual -fexpensive-optimizations -ffast-math -ftree-vectorize -ffinite-math-only -fstrict-aliasing -fno-trapping-math -fno-signaling-nans -fno-float-store -Wno-unused -funsafe-loop-optimizations -isystem /apps/PYTHON/2.6.5/include/python2.6 -fPIC -DBOOST_ALL_NO_LIB -DIMP_SWIG_WRAPPER -Ibuild/include -Ibuild/src build/src/IMP_base_wrap.cpp > build/include/IMP/base/log.h: In function 'PyObject* _wrap_push_log_context(PyObject*, PyObject*)': > build/include/IMP/base/log.h:84: error: too many arguments to function 'void IMP::base::push_log_context(std::string)' > build/src/IMP_base_wrap.cpp:9533: error: at this point in file > build/src/IMP_base_wrap.cpp: In function 'PyObject* _wrap_new_CreateLogContext__SWIG_1(PyObject*, PyObject*)': > build/src/IMP_base_wrap.cpp:10099: error: no matching function for call to 'IMP::base::CreateLogContext::CreateLogContext(const char*, const void*)' > build/include/IMP/base/log.h:351: note: candidates are: IMP::base::CreateLogContext::CreateLogContext(const IMP::base::CreateLogContext&) > build/include/IMP/base/log.h:351: note: IMP::base::CreateLogContext::CreateLogContext(std::string) > build/include/IMP/base/log.h:351: note: IMP::base::CreateLogContext::CreateLogContext() > build/src/IMP_base_wrap.cpp: In function 'PyObject* _wrap_CreateLogContext_set__SWIG_0(PyObject*, PyObject*)': > build/src/IMP_base_wrap.cpp:10281: error: no matching function for call to 'IMP::base::CreateLogContext::set(const char*, const void*)' > build/include/IMP/base/log.h:351: note: candidates are: void IMP::base::CreateLogContext::set(std::string) > build/src/IMP_base_wrap.cpp: In function 'PyObject* _wrap_Object_set_check_level(PyObject*, PyObject*)': > build/src/IMP_base_wrap.cpp:10943: error: 'class IMP::base::Object' has no member named 'set_check_level' > build/src/IMP_base_wrap.cpp:10945: error: 'class IMP::base::Object' has no member named 'set_check_level' > build/src/IMP_base_wrap.cpp: In function 'PyObject* _wrap_Object_get_check_level(PyObject*, PyObject*)': > build/src/IMP_base_wrap.cpp:11015: error: 'const class IMP::base::Object' has no member named 'get_check_level' > scons: *** [build/src/IMP_base_wrap.os] Error 1 > scons: building terminated because of errors. > > > IMP configuration: > boost ='yes' > boostlibs ='' > boostversion ='104800' > boostlibsuffix ='' > endian ='little' > boost_filesystem ='yes' > boost_filesystemlibs='boost_filesystem' > cgal ='no' > ann ='no' > hdf5 ='no' > boost_thread ='yes' > boost_threadlibs ='boost_thread' > gsl ='no' > fftw3 ='no' > boost_programoptions='no' > opencv21 ='no' > opencv22 ='no' > bullet ='no' > build ='fast' > prefix ='/usr' > local ='True' > pythonpath ='/project/devel/dbau/imp/imp-r11171' > libpath ='/apps/BOOST/1_48_0/lib' > ldlibpath ='/project/devel/dbau/imp/imp-r11171' > swigpath ='/apps/SWIG/2.0.4/bin' > datapath ='/project/devel/dbau/imp/imp-r11171' > cxxcompiler ='g++' > cxxflags='-isystem /apps/IMP/include -isystem /apps/BOOST/1_48_0/include -isystem /apps/CGAL/3.9/include -Wall -Wextra -Wno-deprecated -Winit-self -Wstrict-aliasing=2 -Wcast-align -fno-operator-names -Wundef -Woverloaded-virtual -O3 -fexpensive-optimizations -ffast-math -ftree-vectorize -ffinite-math-only -fstrict-aliasing -fno-trapping-math -fno-signaling-nans -fno-float-store -Wno-unused -funsafe-loop-optimizations' > pythoncxxflags ='-isystem /apps/IMP/include -isystem /apps/BOOST/1_48_0/include -isystem /apps/CGAL/3.9/include -Wno-deprecated -Winit-self -Wcast-align -fno-operator-names -Wundef -Woverloaded-virtual -fexpensive-optimizations -ffast-math -ftree-vectorize -ffinite-math-only -fstrict-aliasing -fno-trapping-math -fno-signaling-nans -fno-float-store -Wno-unused -funsafe-loop-optimizations' > bincxxflags='-isystem /apps/IMP/include -isystem /apps/BOOST/1_48_0/include -isystem /apps/CGAL/3.9/include -Wall -Wextra -Wno-deprecated -Winit-self -Wstrict-aliasing=2 -Wcast-align -fno-operator-names -Wundef -Woverloaded-virtual -O3 -fexpensive-optimizations -ffast-math -ftree-vectorize -ffinite-math-only -fstrict-aliasing -fno-trapping-math -fno-signaling-nans -fno-float-store -Wno-unused -funsafe-loop-optimizations' > shliblinkflags =' -shared' > arliblinkflags ='' > binlinkflags ='' > pythonlinkflags =' -shared' > rpath ='True' > static ='False' > pythonsosuffix ='.so' > pkgconfig ='True' > path ="/apps/GCC/4.6.2/bin:/apps/PYTHON/2.6.5/bin:/apps/modules//wrappers/fortran:/apps/modules//wrappers/c++:/apps/modules//wrappers/c:/opt/mpi/mpibull2-1.3.9-18.s/bin:/opt/intel/Compiler/11.0/069/bin/intel64:/usr/kerberos/bin:/opt/cuda//bin:/usr/local/bin:/bin:/usr/bin:/home/devel/dbau/bin" > python ="/apps/PYTHON/2.6.5/bin/python" > > > Enabled dependencies: Boost.FileSystem, Boost.Thread, Boost > Disabled dependencies: Bullet, HDF5, OpenCV21, CGAL, ANN, Boost.ProgramOptions, modeller, FFTW3, GSL, OpenCV22 > > Enabled modules: em, kernel, scratch, domino, misc, fivec, statistics, container, saxs, system, parallel, example, benchmark, core, algebra, restrainer, example_system_local, base, atom, compatibility, display > Disabled modules: rmf, RMF, cgal, multifit, bullet, gsl, modeller, em2d > > Enabled applications: em_tools > Disabled applications: rmf_tools, ligand_score, saxs_tools, nup84, multifit, example_system, rnapii, foxs > > Enabled systems: Groel > > Errors building: > build/src/IMP_base_wrap.os

2 1

New 'IMP-build' mailing list
by Ben Webb 27 Jan '12

27 Jan '12

Emails about IMP build failures now go to the IMP-build mailing list, rather than IMP-dev. This makes the IMP-dev mailing list archives a little tidier (no longer cluttered with those emails). If you were subscribed to IMP-dev, you are also subscribed to IMP-build. IMP-build contains topic filters, which you can see if go to the IMP-build info page (https://salilab.org/mailman/listinfo/imp-build) and click on 'Unsubscribe or edit options', then log in and scroll down the page. You can use these to choose to get only a subset of build failure messages. For example, if you only work on SAXS, you can select the saxs and saxs_merge filters and then only get an email if the IMP.saxs module or the saxs_merge application fails. (If you select no filters, you get any email if any part of IMP fails, same as before.) If you want to discuss IMP development but don't actually work on any IMP code, you can unsubscribe from IMP-build if you so choose. The build system results page (http://salilab.org/imp/nightly/tests.html) is still public, of course. Ben -- ben(a)salilab.org http://salilab.org/~ben/ "It is a capital mistake to theorize before one has data." - Sir Arthur Conan Doyle

1 0

show-changes
by Daniel Russel 25 Jan '12

25 Jan '12

I added a little script to IMP tools called show-changes that shows changes in IMP (as announced in the history file) that have occurred since you last updated. The idea is to help people get an idea if there are any changes that would require updating their code before doing svn update so you don't have to spend time updating your code when you are in the middle of something else. And I hope to encourage people to read the change log so they have a sense of what is happening in IMP. Do ./tools/show-changes from you svn checkout to see what it says. Let me know if there are any problems.

2 1