July 2010 - IMP-users

problems with a sample script (basic IMP optimization)
by Benjamin SCHWARZ 12 Aug '10

12 Aug '10

Hi list, As a first step towards a serious usage of IMP, I tried to write a sample code using rigid bodies, restraints and optimizers. With MonteCarlo optimizer, at the end of optimization the model seems to remains unchanged. With ConjugateGradients or SteepestDescent, when using too many steps in the optimizer (under 100 though !?) the script generally crashes with one of these error messages : File "/usr/local/lib/python2.6/site-packages/IMP/core/__init__.py", line 642, in optimize def optimize(self, *args): return _IMP_core.ConjugateGradients_optimize(self, *args) _IMP.UsageException: Attempting to construct a rotation from a non-quaternion value. The coefficient vector must have a length of 1. Got: 0.165858 -0.916284 0.55176 0.327904 gives 1.27905 File "/usr/local/lib/python2.6/site-packages/IMP/core/__init__.py", line 642, in optimize def optimize(self, *args): return _IMP_core.ConjugateGradients_optimize(self, *args) _IMP.UsageException: Attempting to construct a rotation from a non-quaternion value. The coefficient vector must have a length of 1. Got: 0.389486 -0.600546 0.553302 0.99031 gives 1.79921 Moreover when the script finishes, its outputs seem not relevant at all, though the score is better (smaller). maybe it is normal though, due to the choice of my restraints or to the lack of optimization steps… I think my understanding of IMP usage is still very far from perfect, will you help me sorting that out by having a look at my code ? You will fin the script attached to this message, its outlet is really basic : - a structure is loaded ( I choose 1RDT, a complex comprising two protein domains and two peptides) - a map is sampled at 10A for that structure - chains are extracted from the structure, rigidified and randomly moved in space - two restraints are added to the model in order to tale into account - the fit of all atoms to the sampled map - no steric clashes between rigid bodies - optimization thank you very much Dr. Benjamin SCHWARZ Biocomputing group Email : schwarz(a)igbmc.fr Voice : +33 (0)3 68 85 47 30 FAX : +33 (0)3 68 85 47 18 Structural Biology & Genomics Dept. - IGBMC 1 rue Laurent Fries BP 10142 F - 67404 Illkirch CEDEX

4 14

basic map handling
by Benjamin SCHWARZ 03 Aug '10

03 Aug '10

Hi list, I could not find how to Generate a density map from a structure with a *fixed* dimension : When using IMP.em.SampledDensityMap(ps,resolution,apix) with a fixed size for apix, the grid size depends on the resolution. I would like to write a function that would allow me to generate maps with the same number of vertices (and the same space coverage) for different resolutions. I thus tried : def pdbFile2dmap (pdbStream,densityFile,resolution,apix,sel,bboxCoverage=1.2): ''' pdbStream : where to read the structure densityFile : name of the file where to drop the density map resolution : resolution of the density map apix : voxel size sel : selector for reading structure bboxCoverage : approximate bbox coverage of the density map ''' m= IMP.Model() # read protein mh = IMP.atom.read_pdb(pdbStream,m,sel) # add radius info to each atom IMP.atom.add_radii(mh) # compute bbox, and map size in voxels bbox = IMP.atom.get_bounding_box(mh) diag = bbox.get_corner(0) - bbox.get_corner(1) nx = int(bboxCoverage * diag[0] / apix) ny = int(bboxCoverage * diag[1] / apix) nz = int(bboxCoverage * diag[2] / apix) # compute origin origin = bbox.get_corner(0) + (1-bboxCoverage)/2 * diag dmap = IMP.em.SampledDensityMap() # dmap.CreateVoidMap(nx,ny,nz) ps = IMP.Particles(IMP.core.get_leaves(mh)) dmap.set_particles(ps) dmap.set_origin(origin) dmap.get_header_writable().set_number_of_voxels(nx,ny,nz) dmap.get_header_writable().set_resolution(resolution) dmap.update_voxel_size(apix) dmap.resample() dmap.calcRMS() # computes statistic stuff about the map and insert it in the header print dmap.get_header().show(),"\n" IMP.em.write_map(dmap,densityFile,IMP.em.MRCReaderWriter()) that resulted in : python(10802) malloc: *** mmap(size=18446744073709314048) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Traceback (most recent call last): File "./pdb2density.py", line 181, in <module> pdbFile2dmap(pdbStream,densityFile,resolution,apix,sel,1.2) File "./pdb2density.py", line 75, in pdbFile2dmap dmap.update_voxel_size(apix) File "/usr/local/lib/python2.6/site-packages/IMP/em/__init__.py", line 428, in update_voxel_size def update_voxel_size(self, *args): return _IMP_em.DensityMap_update_voxel_size(self, *args) MemoryError: std::bad_alloc Can you tell me if my approach is OK, and how to solve my allocation problem ? I would also like to know if there are easy means in IMP to 1.) Strip a density map : out of an existing map, create a new map or restrict the existing one, so as to conserve only a central subset of voxels 2.) Re-sample a map : Conserve the density map "box" size, and change the number of voxels that cover it --Ben

3 7

A remark concerning software design and the get_leaves() function
by Benjamin SCHWARZ 31 Jul '10

31 Jul '10

It is probably a question of taste, but every time I need the leaves of a hierarchy, my intuitive guess is to write mh.get_leaves() instead of IMP.core.get_leaves(mh). So I wonder if there are gory reasons for this function to be outside the class… --Ben

3 2

Questions about example script local_fitting.py
by Benjamin SCHWARZ 21 Jul '10

21 Jul '10

hello list, I started my IMP apprenticeship by trying to run and understand the example script local_fitting.py script. I spent some time rewritting it and parsing the documentation, and I come to you now with remaining questions. I am pretty new to the CryoEM field, so I will probably ask naïve questions. If some of these questions can be answered by reading a manual, please tell me which manual and I will be glad to read it. 1.) As a first remark concerning this script, the module IMP.algebra is used throughout the script, though never imported. Funnily, it puzzles my PyDev plugin but does not appear to lead to execution problems. 2.) I have problems wit regard to the resolution parameter and its apparent absence in density maps : at line 30 the header of a loaded density map is modified. dmap.get_header_writable().set_resolution(resolution) I though, the idea was to fix a possibly distinct resolution value for the resampled map; so I commented the line and replaced it by sampled_input_density.get_header_writable().set_resolution(resolution) at line 36… Which resulted in an error during the local fitting process. I thus checked wether the map header contained a resolution or not : >>> dmap.get_header().get_has_resolution() False >>> dmap.get_header().get_resolution() nan Does the resolution never appear it map files ? What is the incidence/relevance of "arbitrarily" setting this parameter such as it is apparently done in this example ? 3.) I don't fully understand the way objects share elements, and what happens when these elements are moved. For instance, the three objects : line 19 : mh=IMP.atom.read_pdb(IMP.em.get_example_path("input.pdb"),m,sel) line 24 : ps= IMP.Particles(IMP.core.get_leaves(mh)) line 58 : prot_rb = IMP.atom.setup_as_rigid_body(mh) seem to share atoms since when moving : line 60 : IMP.core.transform(prot_rb,local_trans) it looks like every atoms in mh and in ps have also been moved. I thus have the feeling that all three objects share references on the same atoms. If this is true, I don't understand what happens when iteratively applying transformations to prot_rb at the end of the script : line 100 : prot_rb.set_transformation(fitting_sols.get_transformation(i)) line 102 : rmsd=IMP.atom.get_rmsd(IMP.core.XYZs(ps),IMP.core.XYZs(IMP.core.get_leaves(mh_ref))) line 103 : IMP.atom.write_pdb(mh,"temp_"+str(i)+".pdb") Since the transformations are applied iteratively, I would expect them to be composed; hence, parsing from 1 to fitting_sols.get_number_of_solutions() in this order or in reverse order would not lead to the same result. I checked this behavior, and the solutions are (happily) the same whatever the order in which they are parsed... and I can't guess why… 4.) Is there a difference between prot_rb.set_transformation and IMP.core.transform ? 5.) The signature is not the same for function IMP::em::local_rigid_fitting in the documentation and in the script. In the documentation, it has a ( FittingSolutions & fr) argument whereas, according to the script, this in fact seems to be the return type 6.) I could not find the documentation for the class FittingSolutions Thanks for any hint --Ben

5 6

pdb parser issue in IMP.atom.read_pdb
by Benjamin SCHWARZ 20 Jul '10

20 Jul '10

You probably know it, but some atom names would simply result in read_pdb to fail. This behavior seems a bit drastic too me. Maybe it would be possible to build a hierarchy containing atoms that could be recognized, and provide some warnings, or give access to the number of encountered errors or ignored atoms when building the hierarchy; leaving the final choice to the user to play with its structure or to reject it. As an example : pdb code 1E1Q mh = IMP.atom.read_pdb(pdbFilePath,m) _IMP.ValueException: Illegal character in atom name, only A-Z, 0-9, " "allowed. Got "HET: O5'"

2 1

IMP.atom.read_pdb confusing error message when not finding file
by Benjamin SCHWARZ 20 Jul '10

20 Jul '10

When trying to read a pdb file that does not exist on my disk I get a puzzling message : mh = IMP.atom.read_pdb(pdbFilePath,m,sel) NotImplementedError: Wrong number of arguments for overloaded function 'read_pdb'. Possible C/C++ prototypes are: IMP::atom::read_pdb(IMP::TextInput,IMP::Model *) IMP::atom::read_pdb(IMP::TextInput,IMP::Model *,IMP::atom::PDBSelector const &,bool) IMP::atom::read_pdb(IMP::TextInput,IMP::Model *,IMP::atom::PDBSelector const &)

2 1

Binary installation of IMP 1.0 on Macbook Pro OSX 10.6.3 snowleopard
by Benjamin SCHWARZ 19 Jul '10

19 Jul '10

>> By the way, would you be interested in any testimony with regard to IMP installation ? If so, I could open a thread here just for that. > Of course, please do. So be it. The installation in itself was OK, though I found the "Installation successful" messages a bit disconcerting. Maybe you should consider an installation log file or a longer message to tell what exactly was installed, in which directory, and suggest some starting points for beginners. Concerning the utilization now : I have two python 2.6 installed on my computer though I do not remember nor why nor how. The default python (the one that runs when typing "python" on the command line) links to /Library/Frameworks/Python.framework/Versions/2.6/bin/python (it claims to be a v2.6.4 compiled with gcc 4.0.1 ) My second python dwells in /usr/bin and claims to be a v2.6.1 compiled with gcc 4.2.1 With the first interpreter, IMP is not automatically found and I have to modify the PYTHONPATH (therefore the localization of IMP in the installation message could be of interest.). When this is done, the package is found but any attempt to use it results in an error : Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.6/site-packages/IMP/__init__.py", line 25, in <module> _IMP = swig_import_helper() File "/usr/local/lib/python2.6/site-packages/IMP/__init__.py", line 17, in swig_import_helper import _IMP ImportError: dlopen(/usr/local/lib/python2.6/site-packages/_IMP.so, 2): no suitable image found. Did find: /usr/local/lib/python2.6/site-packages/_IMP.so: mach-o, but wrong architecture With /usr/bin/python the IMP package is automatically detected, and everything runs fine. In order to explain the problem observed with the default interpreter, I firstly suspected it had something to do with the gcc version, but now I rather think it is a 32bits/64bits conflict : > file /usr/local/lib/python2.6/site-packages/_IMP.so /usr/local/lib/python2.6/site-packages/_IMP.so: Mach-O 64-bit bundle x86_64 > file /Library/Frameworks/Python.framework/Versions/2.6/bin/python /Library/Frameworks/Python.framework/Versions/2.6/bin/python: Mach-O universal binary with 2 architectures /Library/Frameworks/Python.framework/Versions/2.6/bin/python (for architecture ppc): Mach-O executable ppc /Library/Frameworks/Python.framework/Versions/2.6/bin/python (for architecture i386): Mach-O executable i386 > file /usr/bin/python /usr/bin/python: Mach-O universal binary with 3 architectures /usr/bin/python (for architecture x86_64): Mach-O 64-bit executable x86_64 /usr/bin/python (for architecture i386): Mach-O executable i386 /usr/bin/python (for architecture ppc7400): Mach-O executable ppc

2 1

combine IMP.atom.Selectors ?
by Benjamin SCHWARZ 15 Jul '10

15 Jul '10

Hello list, I am very new to IMP, thus probably coming with a bunch of fairly naive question. Today I am wondering if it is possible to combine Selectors in order, for instance to select only ATOMS of chain A from a pdb file. If this is not directly possible what would you suggest to achieve the task ? thanks a lot for any time spent to answer me --Ben

2 3