Ben,
I noticed that during the build time we are running not only unit tests but also examples. This causes problems with paths to input data files and forces each example to use complicated logic to calculate paths. I believe it would be nice if examples were as simple as possible, since they are meant to be read by the users. Would it be a good idea to modify the build process to run examples from their respective directories? This way all examples can use the relative paths and if a user wishes to run an example, he/she can just cd to the example directory and run it directly from there.
Thx, Elina
On Wed, Feb 10, 2010 at 8:37 AM, Notification of IMP commits < imp-commits@salilab.org> wrote:
> Author: ben@SALILAB.ORG > Date: 2010-02-10 08:37:04 -0800 (Wed, 10 Feb 2010) > New Revision: 4751 > > Modified: > trunk/modules/restrainer/examples/basic_setup.py > Log: > Use get_example_path() to find inputs. > > > Modified: trunk/modules/restrainer/examples/basic_setup.py > =================================================================== > --- trunk/modules/restrainer/examples/basic_setup.py 2010-02-10 16:07:22 > UTC (rev 4750) > +++ trunk/modules/restrainer/examples/basic_setup.py 2010-02-10 16:37:04 > UTC (rev 4751) > @@ -4,11 +4,13 @@ > import IMP.restrainer > > # Load molecular hierarchy definition > -RepParser = > IMP.restrainer.XMLRepresentation('input/eg1_representation.xml') > +RepParser = IMP.restrainer.XMLRepresentation( > + > IMP.restrainer.get_example_path('input/eg1_representation.xml')) > representation = RepParser.run() > > # Load the restraint set > -RestraintParser = IMP.restrainer.XMLRestraint('input/eg1_restraint.xml') > +RestraintParser = IMP.restrainer.XMLRestraint( > + > IMP.restrainer.get_example_path('input/eg1_restraint.xml')) > restraint = RestraintParser.run() > > # Place representation into IMP model > > _______________________________________________ > IMP-commits mailing list > IMP-commits@salilab.org > https://salilab.org/mailman/listinfo/imp-commits >
On 02/10/2010 11:13 AM, Elina Tjioe wrote: > I noticed that during the build time we are running not only unit tests > but also examples.
Indeed, since we need to test the examples to make sure they work.
> This causes problems with paths to input data files > and forces each example to use complicated logic to calculate paths.
True, particularly in Restrainer's case where you have paths embedded in your XML files - you can't use get_example_path() there.
> Would it be a good idea to > modify the build process to run examples from their respective > directories? This way all examples can use the relative paths and if a > user wishes to run an example, he/she can just cd to the example > directory and run it directly from there.
Yes, I agree - for running examples during the build, it should be straightforward enough to have the build system cd into build/doc/examples to run the example. That also ensures that any outputs generated by the example don't end up littering the source tree.
If a user wants to run an example manually, he/she can cd into the directory containing it.
Ben
> >> This causes problems with paths to input data files >> and forces each example to use complicated logic to calculate paths. > > True, particularly in Restrainer's case where you have paths embedded in your XML files - you can't use get_example_path() there. Can't you use relative paths there? eg the .xml file contains "./foo.dat" (or just "foo.dat") which should search for the data file relative to the input xml. Otherwise, either you can't relocate you setup or have to always run it from the same directory relative to the .xml file, both of which would be very annoying. Or perhaps I'm confused.
> >> Would it be a good idea to >> modify the build process to run examples from their respective >> directories? This way all examples can use the relative paths and if a >> user wishes to run an example, he/she can just cd to the example >> directory and run it directly from there. > > Yes, I agree - for running examples during the build, it should be straightforward enough to have the build system cd into build/doc/examples to run the example. That also ensures that any outputs generated by the example don't end up littering the source tree. The directory containing the examples isn't necessarily writable (say once they are installed), so we have to be careful about any files they write. I think it is reasonable to insist that the working directory is writeable to produce output, but we could also have the samples write to a tmp dir.
On 02/10/2010 11:33 AM, Daniel Russel wrote: >> True, particularly in Restrainer's case where you have paths embedded >> in your XML files - you can't use get_example_path() there. > Can't you use relative paths there? eg the .xml file contains > "./foo.dat" (or just "foo.dat") which should search for the data file > relative to the input xml. Otherwise, either you can't relocate you > setup or have to always run it from the same directory relative to the > .xml file, both of which would be very annoying. Or perhaps I'm confused.
I think you are confused - Elina is already using relative paths (e.g. 'input/foo.xml') but they don't work since the examples are run from the top-level directory - they need to be either 'modules/restrainer/examples/input/foo.xml' or 'build/doc/examples/restrainer/input/foo.xml' either of which would, if hard-coded in the XML, obviously preclude a user from running an installed example. Relative paths aren't relative to other files, of course, but relative to the current working directory (it is possible for Restrainer to have relative paths in the XML files be relative to the directory containing that XML though, I suppose).
> The directory containing the examples isn't necessarily writable (say > once they are installed), so we have to be careful about any files they > write. I think it is reasonable to insist that the working directory is > writeable to produce output, but we could also have the samples write to > a tmp dir.
You have to balance the confusion that will ensue if users can't find the example outputs against trying to write output into an unwriteable directory. Since the latter yields a pretty clear error message, it seems preferable to me.
Ben
On Feb 10, 2010, at 11:43 AM, Ben Webb wrote:
> On 02/10/2010 11:33 AM, Daniel Russel wrote: >>> True, particularly in Restrainer's case where you have paths embedded >>> in your XML files - you can't use get_example_path() there. >> Can't you use relative paths there? eg the .xml file contains >> "./foo.dat" (or just "foo.dat") which should search for the data file >> relative to the input xml. Otherwise, either you can't relocate you >> setup or have to always run it from the same directory relative to the >> .xml file, both of which would be very annoying. Or perhaps I'm confused. > > I think you are confused - Elina is already using relative paths (e.g. 'input/foo.xml') but they don't work since the examples are run from the top-level directory - they need to be either 'modules/restrainer/examples/input/foo.xml' or 'build/doc/examples/restrainer/input/foo.xml' either of which would, if hard-coded in the XML, obviously preclude a user from running an installed example. Relative paths aren't relative to other files, of course, but relative to the current working directory (it is possible for Restrainer to have relative paths in the XML files be relative to the directory containing that XML though, I suppose). Indeed, that is what it should be (and is done in other similar files, eg the XCode xml config file). If they are, then we don't have a problem since the script can use the get_example_path to find the xml file and all the data files will be in the same dir as the xml file. The context for finding data files in the xml file is the xml file, not whatever directory the user happens to call things from.
> >> The directory containing the examples isn't necessarily writable (say >> once they are installed), so we have to be careful about any files they >> write. I think it is reasonable to insist that the working directory is >> writeable to produce output, but we could also have the samples write to >> a tmp dir. > > You have to balance the confusion that will ensue if users can't find the example outputs against trying to write output into an unwriteable directory. Since the latter yields a pretty clear error message, it seems preferable to me. Agreed. I don't like the writing to a temp dir, which is why I think being able to run examples from anywhere is a good thing.
On 02/10/2010 11:58 AM, Daniel Russel wrote: > Indeed, that is what it should be (and is done in other similar > files, eg the XCode xml config file). If they are, then we don't have > a problem since the script can use the get_example_path to find the > xml file and all the data files will be in the same dir as the xml > file. The context for finding data files in the xml file is the xml > file, not whatever directory the user happens to call things from.
That pronouncement makes it sound like such behavior is some kind of standard, which it of course is not, but I agree that it makes sense for IMP. It should be straightforward for Elina to modify the Restrainer code to do that.
Ben
On Feb 10, 2010, at 12:22 PM, Ben Webb wrote:
> On 02/10/2010 11:58 AM, Daniel Russel wrote: >> Indeed, that is what it should be (and is done in other similar >> files, eg the XCode xml config file). If they are, then we don't have >> a problem since the script can use the get_example_path to find the >> xml file and all the data files will be in the same dir as the xml >> file. The context for finding data files in the xml file is the xml >> file, not whatever directory the user happens to call things from. > > That pronouncement makes it sound like such behavior is some kind of standard, which it of course is not, I would say it is standard behavior in well designed software, but not some kind of standard :-)
-
Ben Webb -
Daniel Russel -
Elina Tjioe