Learning biology from 250 million sequences
Rives and colleagues trained a language model on 86 billion amino acids from 250 million proteins; its learned representations encoded structure and enabled state-of-the-art prediction of mutation effects.
As reported in: Rives, Meier, Sercu et al. (2021). Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences. doi:10.1073/pnas.2016239118