ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning

Abstract Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models taken from NLP. These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-enco... View More