Defining new residue types is generally one of the more painful areas in developing and using a molecular modeling program. MODELLER has two quick-and-dirty solutions described in the next two sections that are often sufficient for comparative modeling involving new residue types. On the other hand, if you are willing to spend some time and define a new entry or complete an incomplete entry in the residue topology or parameter libraries, see the FAQ Section 1.9, Question 17.
The parameter library is used by the MAKE_RESTRAINTS command to construct bond, angle, dihedral angle, improper dihedral angle, and non-bonded Lennard-Jones restraints. If some parameters for these restraints are missing, they are guessed on the fly from the current Cartesian coordinates of the MODEL. Thus, when there are missing parameters, the MODEL coordinates must be defined before calling MAKE_RESTRAINTS. The coordinates can be defined by the BUILD_MODEL command (from the IC entries in the residue topology library), by the READ_MODEL command (from an existing coordinate file for MODEL), or by the TRANSFER_XYZ command (from template coordinate files aligned with MODEL). The bonds, angles, and improper dihedral angles are restrained by a harmonic potential with the mean equal to the value in the current structure and a force constant typical for chemical bonds, angles, and improper dihedral angles, respectively. The dihedral angles are restrained by a tri-modal cosine term with the mean equal to the angle in the current structure. A message detailing MODELLER's improvization is written to the log file.
The second relatively easy way of dealing with missing entries in the residue topology and/or parameters libraries is to use a ``block'' residue. These residues are restrained more or less as rigid bodies to the conformation of the equivalent residue(s) in the template(s). No chemical information is used. The template residues can themselves be defined as block residues. The symbol for the block residues is `BLK' in the four- and three-letter codes and `.' in the single-letter code. The atoms in a BLK residue include all uniquely named atoms from the equivalent residues in all the templates. The atom type of all BLK atoms is the CHARMM type `undf'. The IUPAC atom names (as opposed to the atom types) are the same as in the templates. The `undf' atom type for all BLK atoms facilitates using the PICK_ATOMS command for generating restraints on the `BLK' residues.
The `undf' atoms are treated differently from the other atoms during preparation of dynamic restraints: No pairs of intra-BLK atoms are put on the dynamic non-bonded list. Only the ``inter-BLK'' atom pairs and ``BLK-other'' atom pairs are considered for the dynamic non-bonded restraints. The radius of all block atoms is obtained from the $RADII_LIB library using the block atom names (as written out to a PDB file), not the `undf' atom type. All intra-BLK and inter-residue BLK restraints other than the non-bonded restraints have to be derived separately and explicitly by MAKE_RESTRAINTS command using RESTRAINT_TYPE = 'distance'. See script scripts/__homcsr.top for the routine that makes block restraints for comparative modeling by the `model' script. Lennard-Jones, electrostatic, and general non-bonded spline terms involving `undf' atoms are ignored by MODELLER.
For an example of how to use block residues, see the FAQ Section 1.9, Question 16.