I'm currently trying to model a protein that has a thioester linkage
between two residues. I've searched archives and Google and there's no
mention on how to include such a linkage in the model. So the question
is: is it possible? If so, how? If not, how can I overcome this?
As Joel points out, the way to do this is to create a custom patch
residue. See http://salilab.org/modeller/FAQ.html#9 and
http://www.charmm.org for more details on CHARMM patch residues. The
link you posted describes how to create a new residue type, not how to
make a link between two existing residues.
The example in the FAQ is for a disulfide, but I imagine your linkage
would be similar. As you already discovered, this patch residue should
go in modlib/top_heav.lib.