I've never used Instruments but I'm giving it a try now. I will also try to reduce the number of restraints using lists if possible. On a side note, I've updated to the latest trunk version (r9632). While it compiles fine on both linux and mac, I'm not able to compile local modules. with the scons option "local=True", I get the following error:
> Checking for ExampleLib with variables...(cached) no > Checking for ExampleLib with pkg-config...no > The lib argument must be given as a list. It was not for ExampleLib
On IMP version r7392 local modules compile with no errors. Davide
On Jun 7, 2011, at 6:12 PM, Daniel Russel wrote: > I don't think we have ever tried nearly that many restraints, so there may be some inefficiency that we haven't noticed. One possibility is building the dependency graph which is used to figure out the relationship between restraints and particles and scores states. Another is the non-bonded list or lists if you have many packed particles. It can be a bit difficult to control memory usage and in fact there was a bug a bit back with their memory usage increasing uncontrollably. > > For tracking down problems, if you have a mac where you can run things, there is a wonderful program there called "Instruments" which can track all memory allocations and deallocations and report where they were done. I would also suggest first updating to a more recent version of IMP to see if it is the non-bonded bug. > > On Jun 7, 2011, at 7:52 AM, Davide Baù wrote: > >> Hi everyone, >> >> I'm having a problem with the memory usage of the IMP::Model::evaluate(bool) method. >> >> I'm trying to optimize a set of particles as done for the alpha-globin domain (I'm doing some testing on a larger chromatin domain) using a large set of restraints (over 10^6 restraints). >> The memory usage is OK until all the restraints are generated (~2GB), and then increase to 100% of memory and swap (16GB + 16GB) when I first call m.evaluate(False) (to check the initial score) causing the script to stop. I understand that a large number of restraints are being evaluated, but I was wondering if this is an expected behavior (i.e. if there is a sort of limitation in the number of restraints that can be implemented per memory GB) or if there is a workaround, before trying to run the script on a (shared) machine with much more memory. >> >> I'm using an old version of IMP (r7392) with the following optimizer: >>> # Set up optimizer >>> o = IMP.core.MonteCarlo() >>> o.set_return_best(True) >>> o.set_model(m) >>> fk = IMP.core.XYZ.get_xyz_keys() >>> mov = IMP.core.NormalMover(ps, fk, 0.25) >>> o.add_mover(mov) >>> lo = IMP.core.ConjugateGradients() >>> o.set_local_steps(lsteps) >>> o.set_local_optimizer(lo) >> >> I've tried without the ConjugateGradients after reading this https://salilab.org/imp/bugs/show_bug.cgi?id=106, but I guess it does not depend on CG (I don't have ClosePairContainer in the current code). >> >> Thanks and best regards, >> Davide >> >> >> -- >> Davide Baù >> Structural Genomics Laboratory >> Bioinformatics & Genomics Department, Prince Felipe Research Center >> Avda. Autopista del Saler 16, 46012 Valencia, Spain >> Tel: +34 96 328 96 80 (ext. 1004) Fax: +34 96 328 97 01 >> email: dbau@cipf.es web: http://bioinfo.cipf.es/dbau/ >> >> *** http://www.saveaswwf.com *** >> >> _______________________________________________ >> IMP-dev mailing list >> IMP-dev@salilab.org >> https://salilab.org/mailman/listinfo/imp-dev > > _______________________________________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/listinfo/imp-dev