Assuming there are no hiccups with today's doc changes tonight, I'd propose we declare the current state of IMP to be the release (see the develop branch or the release/2.1 branch). Anyone who hasn't run their code on IMP from the last week or so (it hasn't changed significantly in that period) is highly encouraged to do so to make sure some recent fixup hasn't broken anything.
On 10/21/13 11:00 AM, Daniel Russel wrote: > Assuming there are no hiccups with today's doc changes tonight, I'd > propose we declare the current state of IMP to be the release (see the > develop branch or the release/2.1 branch).
See also http://integrativemodeling.org/nightly/results/?branch=release/2.1
If one of the tests that's failing in there is in your code, now's the time to fix it if you don't want that breakage to be in the release... Notable failures are a segfault in IMP.display, something broken in weighted SAXS profiles, and a bunch of failing tests in IMP.isd.
Ben
weighted SAXS profiles are still experimental, so it is ok to release without this fix.
On Mon, Oct 21, 2013 at 11:05 AM, Ben Webb ben@salilab.org wrote:
> On 10/21/13 11:00 AM, Daniel Russel wrote: > >> Assuming there are no hiccups with today's doc changes tonight, I'd >> propose we declare the current state of IMP to be the release (see the >> develop branch or the release/2.1 branch). >> > > See also > http://integrativemodeling.**org/nightly/results/?branch=**release/2.1http://integrativemodeling.org/nightly/results/?branch=release/2.1 > > If one of the tests that's failing in there is in your code, now's the > time to fix it if you don't want that breakage to be in the release... > Notable failures are a segfault in IMP.display, something broken in > weighted SAXS profiles, and a bunch of failing tests in IMP.isd. > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle > ______________________________**_________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >
perhaps remove those SAXS failed tests from the release branch? It is nice if users get a fail free release (so they can know if they have real installation problems)
On Mon, Oct 21, 2013 at 11:07 AM, Dina Schneidman duhovka@gmail.com wrote:
> weighted SAXS profiles are still experimental, so it is ok to release > without this fix. > > > On Mon, Oct 21, 2013 at 11:05 AM, Ben Webb ben@salilab.org wrote: > >> On 10/21/13 11:00 AM, Daniel Russel wrote: >> >>> Assuming there are no hiccups with today's doc changes tonight, I'd >>> propose we declare the current state of IMP to be the release (see the >>> develop branch or the release/2.1 branch). >>> >> >> See also >> http://integrativemodeling.**org/nightly/results/?branch=**release/2.1http://integrativemodeling.org/nightly/results/?branch=release/2.1 >> >> If one of the tests that's failing in there is in your code, now's the >> time to fix it if you don't want that breakage to be in the release... >> Notable failures are a segfault in IMP.display, something broken in >> weighted SAXS profiles, and a bunch of failing tests in IMP.isd. >> >> Ben >> -- >> ben@salilab.org http://salilab.org/~ben/ >> "It is a capital mistake to theorize before one has data." >> - Sir Arthur Conan Doyle >> ______________________________**_________________ >> IMP-dev mailing list >> IMP-dev@salilab.org >> https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >> > > > _______________________________________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/listinfo/imp-dev > >
On 10/21/13 11:16 AM, Barak Raveh wrote: > perhaps remove those SAXS failed tests from the release branch? It is > nice if users get a fail free release (so they can know if they have > real installation problems)
I don't see what that buys you - the non-working code is still there. If someone manages to get as far as running the tests while still having "real installation problems" most likely all of them will fail anyway - that should be obvious.
Ben
the code is working, the test fails because different platforms give slightly different results, floating point issues. I can release the threshold in the test, but prefer to find better solution.
On Mon, Oct 21, 2013 at 11:21 AM, Ben Webb ben@salilab.org wrote:
> On 10/21/13 11:16 AM, Barak Raveh wrote: > >> perhaps remove those SAXS failed tests from the release branch? It is >> nice if users get a fail free release (so they can know if they have >> real installation problems) >> > > I don't see what that buys you - the non-working code is still there. If > someone manages to get as far as running the tests while still having "real > installation problems" most likely all of them will fail anyway - that > should be obvious. > > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle > ______________________________**_________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >
If they just run "ctest" right after the installation (which is a very good idea), I assume the SAXS tests will run by default, and they will get a lot of failures and be confused. Of course if they can be easily fixed that's also a good solution :) But otherwise, it is nice if after you download the release and run cmake, you will get a completely clean result, assuming that all went well with the installation.
On Mon, Oct 21, 2013 at 11:21 AM, Ben Webb ben@salilab.org wrote:
> On 10/21/13 11:16 AM, Barak Raveh wrote: > >> perhaps remove those SAXS failed tests from the release branch? It is >> nice if users get a fail free release (so they can know if they have >> real installation problems) >> > > I don't see what that buys you - the non-working code is still there. If > someone manages to get as far as running the tests while still having "real > installation problems" most likely all of them will fail anyway - that > should be obvious. > > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle > ______________________________**_________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >
we can simply disable all the failing tests for the release to get a clean ctest run :)
On Mon, Oct 21, 2013 at 11:51 AM, Barak Raveh barak.raveh@gmail.com wrote:
> If they just run "ctest" right after the installation (which is a very > good idea), I assume the SAXS tests will run by default, and they will get > a lot of failures and be confused. Of course if they can be easily fixed > that's also a good solution :) But otherwise, it is nice if after you > download the release and run cmake, you will get a completely clean result, > assuming that all went well with the installation. > > > On Mon, Oct 21, 2013 at 11:21 AM, Ben Webb ben@salilab.org wrote: > >> On 10/21/13 11:16 AM, Barak Raveh wrote: >> >>> perhaps remove those SAXS failed tests from the release branch? It is >>> nice if users get a fail free release (so they can know if they have >>> real installation problems) >>> >> >> I don't see what that buys you - the non-working code is still there. If >> someone manages to get as far as running the tests while still having "real >> installation problems" most likely all of them will fail anyway - that >> should be obvious. >> >> >> Ben >> -- >> ben@salilab.org http://salilab.org/~ben/ >> "It is a capital mistake to theorize before one has data." >> - Sir Arthur Conan Doyle >> ______________________________**_________________ >> IMP-dev mailing list >> IMP-dev@salilab.org >> https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >> > > > > -- > Barak > > _______________________________________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/listinfo/imp-dev > >
Exactly :) We can fix known failed tests in future minor or major releases, but I think the users should get a failure only if the installation itself is bad or if they themselves broke something. Barak
On Mon, Oct 21, 2013 at 11:57 AM, Dina Schneidman duhovka@gmail.com wrote:
> we can simply disable all the failing tests for the release to get a clean > ctest run :) > > > On Mon, Oct 21, 2013 at 11:51 AM, Barak Raveh barak.raveh@gmail.comwrote: > >> If they just run "ctest" right after the installation (which is a very >> good idea), I assume the SAXS tests will run by default, and they will get >> a lot of failures and be confused. Of course if they can be easily fixed >> that's also a good solution :) But otherwise, it is nice if after you >> download the release and run cmake, you will get a completely clean result, >> assuming that all went well with the installation. >> >> >> On Mon, Oct 21, 2013 at 11:21 AM, Ben Webb ben@salilab.org wrote: >> >>> On 10/21/13 11:16 AM, Barak Raveh wrote: >>> >>>> perhaps remove those SAXS failed tests from the release branch? It is >>>> nice if users get a fail free release (so they can know if they have >>>> real installation problems) >>>> >>> >>> I don't see what that buys you - the non-working code is still there. If >>> someone manages to get as far as running the tests while still having "real >>> installation problems" most likely all of them will fail anyway - that >>> should be obvious. >>> >>> >>> Ben >>> -- >>> ben@salilab.org http://salilab.org/~ben/ >>> "It is a capital mistake to theorize before one has data." >>> - Sir Arthur Conan Doyle >>> ______________________________**_________________ >>> IMP-dev mailing list >>> IMP-dev@salilab.org >>> https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >>> >> >> >> >> -- >> Barak >> >> _______________________________________________ >> IMP-dev mailing list >> IMP-dev@salilab.org >> https://salilab.org/mailman/listinfo/imp-dev >> >> > > _______________________________________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/listinfo/imp-dev > >
On 10/21/13 11:51 AM, Barak Raveh wrote: > If they just run "ctest" right after the installation (which is a > very good idea), I assume the SAXS tests will run by default, and > they will get a lot of failures and be confused.
Actually they will get precisely one failure.
> But otherwise, it is nice if after you download the release and run > cmake, you will get a completely clean result
It would also be nice if when you sent your kids to school, the school didn't tell you that all the teachers were deranged lunatics. But if they were, wouldn't you want to know? ;)
Anyhow, if you want an artificially "clean" result, there are two perfectly good decorators to skip a test or to mark the failure as expected. Look for @unittest.skip or @unittest.expectedFailure in the existing set of tests.
Ben
My personal opinion?
If the users got to the point of downloading and installing IMP and all dependencies without any trouble ... then that is the kind of people that don't get scared by a couple of failing tests :)
On Mon, Oct 21, 2013 at 12:15 PM, Ben Webb ben@salilab.org wrote:
> On 10/21/13 11:51 AM, Barak Raveh wrote: > >> If they just run "ctest" right after the installation (which is a >> very good idea), I assume the SAXS tests will run by default, and >> they will get a lot of failures and be confused. >> > > Actually they will get precisely one failure. > > > But otherwise, it is nice if after you download the release and run >> cmake, you will get a completely clean result >> > > It would also be nice if when you sent your kids to school, the school > didn't tell you that all the teachers were deranged lunatics. But if they > were, wouldn't you want to know? ;) > > Anyhow, if you want an artificially "clean" result, there are two > perfectly good decorators to skip a test or to mark the failure as > expected. Look for @unittest.skip or @unittest.expectedFailure in the > existing set of tests. > > > Ben > -- > ben@salilab.org http://salilab.org/~ben/ > "It is a capital mistake to theorize before one has data." > - Sir Arthur Conan Doyle > ______________________________**_________________ > IMP-dev mailing list > IMP-dev@salilab.org > https://salilab.org/mailman/**listinfo/imp-devhttps://salilab.org/mailman/listinfo/imp-dev >
-
Barak Raveh -
Ben Webb -
Daniel Russel -
Dina Schneidman -
Riccardo Pellarin