- Bayesian validation

Bayesian Validation Workshop Dec 6-8
by Jared Sagendorf 21 Aug '23

21 Aug '23

Hi everyone, I'm not sure who has or hasn't responded to Jeff Hoch's inquiry, but could you let me know if you are available to attend a tentativly planned workshop in-person on those dates? The meeting would be held in Connecticut. In order to get the ball rolling, we need a rough headcount of who would be attending. Thanks! - Jared

2 1

possible dates for Bayesian retreat
by Hoch,Jeffrey 18 Aug '23

18 Aug '23

Hi Everyone: The venue in northwest CT (Interlaken Inn) is available October 17/18/19. Would that work for everyone (start at noon on the 17th, end at noon on the 19th)? Yours, Jeff

6 8

summary of today's meeting
by Jared Sagendorf 21 Jul '23

21 Jul '23

Thanks everyone for the discussion today! Here's a summary of the meeting: - Today I presented a high-level overview of model representation in the IHM-Dictionary. The slides I showed can be found here: https://docs.google.com/presentation/d/1oQYKRt4-F7HRLWpUIDr9yiuCxffZIjQ1/ed… - Jeff has initiated the process of getting a Slack channel set up - Joe Courtney will act as coordinator on the UConn side, I'll coordinate on the UCSF/Rutgers side - Hamid will tentatively present some material on Bayesian analysis next meeting, which hopefully can be related to future directions for IHM-Dictionary/PDB-Dev and the PDB proper! - Follow up on discussion for an in-person meeting, potentially later this year. Our next joint meeting is 08/18. See you all then! - Jared

1 0

new PDBe tool on protein conformational states
by Jared Sagendorf 19 Jul '23

19 Jul '23

Hi all, check out this announcement from PDBe on protein conformation comparison: https://www.ebi.ac.uk/pdbe/news/illuminating-protein-conformational-landsca… This is relevant to our current discussion on terminology, which in my opinion is about more than just words, since terminology is a reflection of ones underlying conceptual framework. For our joint meeting with Jeff Hoch's group this Friday, I'll present a brief summary of their work, and I'd also like to present a summary of the current IHM-Dictionary definitions. It'd be good to come prepared to contribute! - Jared

1 0

Re: Next meeting agenda
by Jared Sagendorf 12 Jul '23

12 Jul '23

Sure Jeff, pushing it forward two hours to 12:00 PST should work for most of us. Hopefully that makes it easier for you! I'll update the calendar event on our side. Congrats on the R24! See you Friday, - Jared On Wed, Jun 21, 2023 at 12:20 PM Hoch,Jeffrey <hoch(a)uchc.edu> wrote: > Hi Jared – met too! I added a term to the dictionary . Yulia, Michael > and I will be driving back from a Gordon Conference in Vermont Friday > morning, and it might be tight for us to make the schedule time. I know > there are many schedules to check, but would it be possible to bump the > time back a couple of hours to take some of the stress out? -Jeff > > > > PS – good news is that we heard BMRB will be funded via an R24 mechanism. > Don’t know the budget yet, though. > > > > *From: *Jared Sagendorf <jared.sagendorf(a)rcsb.org> > *Date: *Wednesday, June 21, 2023 at 2:08 PM > *To: *"Hoch,Jeffrey" <hoch(a)uchc.edu> > *Subject: *Next meeting agenda > > > > *** Attention: This is an external email. Use caution responding, opening > attachments or clicking on links. *** > > Hi Jeff, looking forward to the next meeting! > > For the agenda this week, I was thinking we could start with having a look > at some of the terminology, now that we have a dictionary in place. I'll > put together a few slides that I hope will explain the way the Sali group > speaks about modeling, and we can identify if there's any inconsistencies > between the two groups. > > Aside from that, I thought it would be good to begin a discussion on what > the current hurdles are to actually implementing a fully Bayesian > validation pipeline for the PDB. What are the unsolved problems or > unimplemented steps that are missing to actually push this forward, and > could we work together to start solving those issues? > > Also happy to send out any items you would like to bring up! > > > > - Jared >

3 4

next meeting moved to 10:00 AM PST, July 5th
by Jared Sagendorf 29 Jun '23

29 Jun '23

Hi all, I had to re-schedule our next subgroup meeting. Based on people's replies, Wednesday July 5th at 10:00 AM PST seems to work. Thanks to those who can attend, see you then! - Jared

1 0

Fwd: Next meeting agenda
by Jared Sagendorf 21 Jun '23

21 Jun '23

Hey everyone, would moving the subgroup meeting forward by two hours (12:00 PST) work for you, so we can accommodate Jeff's request? Let me know. - Jared ---------- Forwarded message --------- From: Hoch,Jeffrey <hoch(a)uchc.edu> Date: Wed, Jun 21, 2023 at 12:20 PM Subject: Re: Next meeting agenda To: Jared Sagendorf <jared.sagendorf(a)rcsb.org> Cc: Pustovalova,Yulia <ypustovalova(a)uchc.edu>, Eghbalnia,Hamid R. < heghbalnia(a)uchc.edu>, Pozhidaeva,Alexandra <pozhidaeva(a)uchc.edu>, Courtney,Joseph M. <jcourtney(a)uchc.edu>, Gryk,Michael R. <gryk(a)uchc.edu> Hi Jared – met too! I added a term to the dictionary . Yulia, Michael and I will be driving back from a Gordon Conference in Vermont Friday morning, and it might be tight for us to make the schedule time. I know there are many schedules to check, but would it be possible to bump the time back a couple of hours to take some of the stress out? -Jeff PS – good news is that we heard BMRB will be funded via an R24 mechanism. Don’t know the budget yet, though. *From: *Jared Sagendorf <jared.sagendorf(a)rcsb.org> *Date: *Wednesday, June 21, 2023 at 2:08 PM *To: *"Hoch,Jeffrey" <hoch(a)uchc.edu> *Subject: *Next meeting agenda *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. *** Hi Jeff, looking forward to the next meeting! For the agenda this week, I was thinking we could start with having a look at some of the terminology, now that we have a dictionary in place. I'll put together a few slides that I hope will explain the way the Sali group speaks about modeling, and we can identify if there's any inconsistencies between the two groups. Aside from that, I thought it would be good to begin a discussion on what the current hurdles are to actually implementing a fully Bayesian validation pipeline for the PDB. What are the unsolved problems or unimplemented steps that are missing to actually push this forward, and could we work together to start solving those issues? Also happy to send out any items you would like to bring up! - Jared

3 2

Terminology Discussion
by Jared Sagendorf 09 Jun '23

09 Jun '23

Hi all, thanks for the insightful comments this morning. Upon reflection, I think a major limitation in the vocabulary is the overlapping meanings of the term "state" and "model", which is the real origin of the confusion about what an "ensemble" is. I would like to propose the following resolution: * Don't use "model" to refer to a particular choice of the model variables/parameters. The model is like a class definition, which can have many instantiations, which would be called "objects" in a programming analogy. You would never confuse these two in a programming interview. * Don't use the term "state". Instead, refer to either a "conformational state" or a "model state", defined below * Conformational State: A set of spatial coordinates for the modeled system that represent a biological entity. A model representation may include multiple conformational states simultaneously which together make up a model state<https://docs.google.com/document/d/1Pwucs7mGSxRhP5RqPDf-GQ8kU58g6ioLZHxWMn7…>. * Model State: A particular instantiation of all the spatial/temporal coordinates defined in the model representation. A model state may encompass multiple conformational states<https://docs.google.com/document/d/1Pwucs7mGSxRhP5RqPDf-GQ8kU58g6ioLZHxWMn7…>, and if so we refer to it as a multi-state model. By these definitions, we should then refer to either "conformational ensembles" to mean a distribution of plausible conformational states we might observe our biological object of study in, or "model ensembles" to mean an ensemble over model states, for example an ensemble over pathways (e.g. a sequence of conformational states) a protein might take from an unfolded to a folded conformational state. Further clarity could be achieved by replacing "multi-state model" by "multi-conformation model", since "state" is ambiguous. If it's still confusing, I'll come up with some visual aids for the next meeting. Let me know if you have any comments/thoughts! - Jared

1 0

modeling terminology dictionary
by Jared Sagendorf 02 Jun '23

02 Jun '23

On our end, you all already have access because the google doc is in the shared drive: https://docs.google.com/document/d/1Pwucs7mGSxRhP5RqPDf-GQ8kU58g6ioLZHxWMn7… - Jared

1 0

Re: First meeting on Bayesian model validation (05/26)
by Jared Sagendorf 02 Jun '23

02 Jun '23

Hi everyone, great first meeting! I put together a rough draft of a dictionary (with a little input from Andrej), which you should have received an invite for. Please feel free to contribute, as I think it will take some time to arrive at something which is both internally consistent and consistent with the vocabulary already established in the literature (in so far as there is consistency). We initially discussed a github repo, but after some thought I felt a google doc was more appropriate, especially considering there are some nice reference tools. In order to avoid too many collisions, it may be preferable to make edit suggestions rather than edit directly. We also may want to come up with a list of discouraged terms for terms which are inherently vague and unhelpful. In addition, if anyone would like to join the more regular Sali Lab subgroup meetings, or be on our internal mailing list, let me know and I'd be happy to send invites. Have a great weekend. - Jared On Thu, Jun 1, 2023 at 5:54 AM Hoch,Jeffrey <hoch(a)uchc.edu> wrote: > Hi Jared – > > > > I love your “few things”! Those are a mouthful . I’ll introduce our > team members, and it would be great if you could summarize the discussions > that you had in your initial meeting. We can offer an NMR perspective on > the topics. As this will be our first all-hands meeting, I might share a > small slide deck that I used to make the case to the wwPDB PIs that there > is a need and an opportunity to make structure validation “more Bayesian”. > I think I shared some of those slides with you all when I visited, but the > deck is short and would be a good way of getting us all on the same > wavelength (although it’s pretty clear were very close, if not already > there). > > > > I’m attaching a manuscript that’s currently undergoing final revisions. > Please share it among your group, but not outside. A bit of context – when > I took over as head of BMRB, there was already a validation task force > working on revamping the validation pipeline for NMR structures. > Unfortunately, although the effort is/was very well-intentioned, it retains > much ad hoc and archaic language, e.g. “violations” of NMR “restraints” are > dealt with in a way that simply isn’t consistent or applicable in a broader > sense to any other type of empirical data. Rather than move the goalposts > on the task force, I suggested they complete their work to achieve the > original goal, and we would start a Bayesian initiative afresh. It serves > to highlight some of the issues we will have to deal with – such as how do > you convert hard upper and lower distance bounds into something that can > yield a realistic distribution of errors/structures? This will be necessary > for retrospective analysis of NMR structures in the PDB because distance > bounds are in most cases all that was supplied by depositors. Going > forward, BMRB will need to require peak tables with intensities of NOESY > cross-peaks, or perhaps even raw time-domain data for NOESY experiments. > > > > I’m cc’ing the rest of our team so you can capture their email addresses > if you haven’t already. They are > > > > Kumaran Baskaran – BMRB liaison to wwPDB and BMRB representative of the > NMR VTF > > Michael Gryk – associate director of BMRB and our bona fide data > scientist > > Hamid Eghbalnia – lead of the analytics technology development component > of the NMRbox P41 grant, and our bona fide statistician/Bayesian > > Yulia Pustovalova and Sasha Pozhidaeva, NMR spectroscopists par > excellence, who have been utilizing AlphaFold in their workflows and trying > out ways to validate computed structures based on prior knowledge. > > Joseph Courtney – NMR spectroscopist and developer of the COMPASS package > from Chad Rienstra’s group – COMPASS used MODELER, chemical shift > prediction, and integrated some other software packages to determine > protein structures from unassigned carbon-carbon correlation spectra > (solid-state NMR). The use of forward-modeling of chemical shifts and > unassigned peak lists to drive/constrain the structure determination was > ahead of its time and very pertinent to Bayesian validation. > > > > Looking forward to seeing you tomorrow. > > > > Yours, Jeff > > > > *From: *Jared Sagendorf <jared.sagendorf(a)rcsb.org> > *Date: *Wednesday, May 31, 2023 at 2:01 PM > *To: *"Hoch,Jeffrey" <hoch(a)uchc.edu> > *Subject: *Re: First meeting on Bayesian model validation (05/26) > > > > *** Attention: This is an external email. Use caution responding, opening > attachments or clicking on links. *** > > Hi Jeff, just looping back to this! A few things that came up during the > initial meeting with Andrej: > > [image: Image removed by sender.]- Development of improved metrics for > model quality > > - Standardization of priors, forward functions, and likelihoods > > - Establishment of a standardized vocabulary > > - How to get different communities involved in all of the above decisions > > In addition to any of the above, I'd be keen to learn more about what your > group has been working in w.r.t. model validation, Bayesian or otherwise! > > Let me know if you have any thoughts! > > > > - Jared > > > > On Thu, May 18, 2023 at 11:49 AM Hoch,Jeffrey <hoch(a)uchc.edu> wrote: > > Hi Jared – thanks for touching base. Let me work on assembling some > background information. First topic would be to introduce you all to the > UConn team. More shortly... J > > > > *From: *Jared Sagendorf <jared.sagendorf(a)rcsb.org> > *Date: *Thursday, May 18, 2023 at 2:34 PM > *To: *"Hoch,Jeffrey" <hoch(a)uchc.edu> > *Subject: *First meeting on Bayesian model validation (05/26) > > > > *** Attention: This is an external email. Use caution responding, opening > attachments or clicking on links. *** > > Hi Jeff, this is Jared Sagendorf from Andrej's lab (sorry for the change > in e-mail, still transitioning from my previous institution...). I'm > looking forward to the discussion next Friday! I was wondering if you would > like to set an agenda for the meeting, or keep the format relatively open? > > > > I am also interested to know your feelings on what the scope of these > discussions should be – a more broad discussion on Bayesian methods for > validation, or a more narrow discussion on what is practical specifically > in the context of the PDB/PDB-Dev. > > Finally, if there's any papers from your group or otherwise you'd like to > frame the discussion around feel free to let me know and I'll share with > the others on our end. > > > > Thanks! > > > > - Jared > > > > > >

1 0

2024

2023

Bayesian validation ----- 2024 ----- May 2024 April 2024 March 2024 February 2024 January 2024 ----- 2023 ----- December 2023 November 2023 October 2023 September 2023 August 2023 July 2023 June 2023 May 2023

Bayesian validation