homelette.evaluation

The homelette.evaluation submodule contains different classes for evaluating homology models.

It is possible to implement custom Evaluation building blocks and use them in the homelette framework.

Tutorials

Working with model evaluations in homelette is discussed in detail in Tutorial 3. Implementing custom evaluation metrics is discussed in Tutorial 4. Assembling custom pipelines is discussed in Tutorial 7.

Classes

The following evaluation metrics are implemented:

Evaluation_dope Evaluation_soap_protein Evaluation_soap_pp Evaluation_qmean4 Evaluation_qmean6 Evaluation_qmeandisco Evaluation_mol_probity

---

class homelette.evaluation.Evaluation_dope(model: Type[Model], quiet: bool = False)

Class for evaluating a model with DOPE score.

Will dump the following entries to the model.evaluation dictionary:

• dope

• dope_z_score

Parameters:

• model (Model) – The model object to evaluate

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

Notes

DOPE is a staticial potential for the evaluation of homology models [1]. For further information, please check the modeller documentation or the associated publication.

References

[1]

Shen, M., & Sali, A. (2006). Statistical potential for assessment and prediction of protein structures. Protein Science, 15(11), 2507–2524. https://doi.org/10.1110/ps.062416606

evaluate() → None

Run DOPE evaluation. Automatically called on object initialization

Return type:

None

class homelette.evaluation.Evaluation_soap_protein(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the SOAP protein protential.

Will dump the following entries to the model.evaluation dictionary:

• soap_protein

Parameters:

• model (Model) – The model object to evaluate

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

Notes

SOAP is a statistical potential for evaluating homology models [2]. For more information, please check the modeller and SOAP documentations or the associated publication.

References

[2]

Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560

evaluate() → None

Run SOAP protein evaluation. Automatically called on object initialization

Return type:

None

class homelette.evaluation.Evaluation_soap_pp(model: Type[Model], quiet: bool = False)

Class for evaluating a model with SOAP interaction potentials. This is used for the evaluation of models of protein complexes.

Will dump the following entries to the model.evaluation dictionary:

• soap_pp_all

• soap_pp_atom

• soap_pp_pair

Parameters:

• model (Model) – The model object to evaluate

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

Notes

SOAP is a statistical potential for evaluating homology models [3]. For more information, please check the modeller and SOAP documentations or the associated publication.

References

[3]

Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560

evaluate() → None

Run SOAP interaction evaluation. Automatically called on object initialization

Return type:

None

class homelette.evaluation.Evaluation_qmean4(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN4 potential.

Will dump the following entries to the model.evaluation dictionary:

• qmean4

• qmean4_z_score

Parameters:

• model (Model) – The model object to evaluate.

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

See also

Evaluation_qmean6, Evaluation_qmeandisco

Notes

QMEAN is a statistical potential for evaluating homology models [4] [5].

Briefly, QMEAN is a combination of different components. Four compoenents (interaction, cbeta, packing and torsion) form the qmean4 score.

For more information, please check the QMEAN documentation or the associated publications.

References

[4]

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

[5]

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

evaluate() → None

Run QMEAN4 protein evaluation. Automatically called on object initialization :rtype: None

class homelette.evaluation.Evaluation_qmean6(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN6 potential.

Will dump the following entries to the model.evaluation dictionary:

• qmean6

• qmean6_disco

Requires the following valid entries in the model.info dictionary:

• accpro_file (.acc file)

• psipred_file (.horiz file)

Parameters:

• model (Model) – The model object to evaluate.

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

See also

Evaluation_qmean4, Evaluation_qmeandisco

Notes

QMEAN is a statistical potential for evaluating homology models [6] [7].

QMEAN6 is a combination of six different components (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement). It is an extension to the QMEAN4 score, which additionally evaluates the agreement of the model to secondary structur predictions from PSIPRED [8] and solvent accessiblity predictions from ACCpro [9].

For more information, please check the QMEAN documentation or the associated publications.

References

[6]

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

[7]

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

[8]

Jones, D. T. (1999). Protein secondary structure prediction based on position-specific scoring matrices. Journal of Molecular Biology, 292(2), 195–202. https://doi.org/10.1006/JMBI.1999.3091

[9]

Magnan, C. N., & Baldi, P. (2014). SSpro/ACCpro 5: almost perfect prediction of protein secondary structure and relative solvent accessibility using profiles, machine learning and structural similarity. Bioinformatics, 30(18), 2592–2597. https://doi.org/10.1093/BIOINFORMATICS/BTU352

evaluate() → None

Run QMEAN6 protein evaluation. Automatically called on object initialization

Return type:

None

class homelette.evaluation.Evaluation_qmeandisco(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN DisCo potential.

Will dump the following entries to the model.evaluation dictionary:

• qmean6

• qmean6_z_score

• qmean_local_scores_avg

• qmean_local_scores_err

Requires the following valid entries in the model.info dictionary:

• accpro_file (.acc file)

• psipred_file (.horiz file)

• disco_file (generated by qmean.DisCoContainer.Save)

Parameters:

• model (Model) – The model object to evaluate.

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Raises:

ImportError – Unable to import dependencies

See also

Evaluation_qmean4, Evaluation_qmean6

Notes

QMEAN is a statistical potential for evaluating homology models [10] [11].

QMEAN DisCo is an extension of QMEAN by the inclusion of homology derived DIStance COnstraints [12]. These distance contraints do not influence the six component of the QMEAN6 score (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement), but only the local scores.

The distance contraints for the target have to be generated before and saved to a file.

For more information, please check the QMEAN documentation or the associated publications.

References

[10]

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

[11]

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

[12]

Studer, G., Rempfer, C., Waterhouse, A. M., Gumienny, R., Haas, J., & Schwede, T. (2020). QMEANDisCo-distance constraints applied on model quality estimation. Bioinformatics, 36(6), 1765–1771. https://doi.org/10.1093/bioinformatics/btz828

evaluate() → None

Run QMEAN DisCo protein evaluation. Automatically called on object initialization

Return type:

None

class homelette.evaluation.Evaluation_mol_probity(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the MolProbity validation service.

Will dump the following entries to the model.evaluation dictionary:

• mp_score

Parameters:

• model (Model) – The model object to evaluate

• quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables:

• model (Model) – The model object to evaluate

• output (dict) – Dictionary that all outputs will be dumped into

Notes

Molprobity is a program that evaluates the quality of 3D structures of proteins based on structural features [13] [14] [15]. For more information, please check the MolProbity webpage or the associated publications.

References

[13]

Davis, I. W., Leaver-Fay, A., Chen, V. B., Block, J. N., Kapral, G. J., Wang, X., Murray, L. W., Arendall, W. B., Snoeyink, J., Richardson, J. S., & Richardson, D. C. (2007). MolProbity: all-atom contacts and structure validation for proteins and nucleic acids. Nucleic Acids Research, 35(suppl_2), W375–W383. https://doi.org/10.1093/NAR/GKM216

[14]

Chen, V. B., Arendall, W. B., Headd, J. J., Keedy, D. A., Immormino, R. M., Kapral, G. J., Murray, L. W., Richardson, J. S., & Richardson, D. C. (2010). MolProbity: All-atom structure validation for macromolecular crystallography. Acta Crystallographica Section D: Biological Crystallography, 66(1), 12–21. https://doi.org/10.1107/S0907444909042073

[15]

Williams, C. J., Headd, J. J., Moriarty, N. W., Prisant, M. G., Videau, L. L., Deis, L. N., Verma, V., Keedy, D. A., Hintze, B. J., Chen, V. B., Jain, S., Lewis, S. M., Arendall, W. B., Snoeyink, J., Adams, P. D., Lovell, S. C., Richardson, J. S., & Richardson, D. C. (2018). MolProbity: More and better reference data for improved all-atom structure validation. Protein Science, 27(1), 293–315. https://doi.org/10.1002/pro.3330

evaluate() → None

Run MolProbity evaluation. Automatically called on object initialization

Return type:

None