Jakob Uszkoreit
Jakob Uszkoreit is a computer scientist and one of eight equal-contribution authors of Attention Is All You Need, the 2017 paper that introduced the Transformer architecture and quietly rewired the future of large language models. Where earlier sequence models leaned on recurrence or Convolutions to process language one step at a time, Uszkoreit pushed for something bolder: throw both away and let Self-attention do all the work.
According to the paper's own credits, Jakob proposed replacing RNNs with self-attention and started the effort to evaluate this idea. It was a provocative bet — dispensing with the sequential machinery that had defined Neural machine translation and Sequence to sequence learning for years — but it paid off spectacularly. Even the architecture's name carries his fingerprint: the name "Transformer" was picked because Jakob Uszkoreit, one of the paper's authors, liked the sound of that word.
Before that breakthrough, in March 2008, Uszkoreit joined the Google Brain team in Berlin, where he worked on early versions of Google Translate and contributed to the development of Google Assistant. He spent over a decade there working alongside co-authors Ashish Vaswani, Noam Shazeer, and Niki Parmar, among others, on large-scale Attention-based systems for language understanding.
In 2021, like several of his fellow Transformer authors, he departed the company. In July 2021, Uszkoreit left Google Brain and co-founded Inceptive, a biotech company that incorporates AI into RNA biology to develop novel therapeutics and biotechnologies — trading word embeddings for molecular ones, but keeping the same conviction that the right architecture can unlock an entire field.
His paper's descendants are everywhere: BERT, GPT, GPT-3, T5, and the Vision transformer all trace their lineage to the self-attention mechanism he helped champion, alongside later refinements like FlashAttention and Mixture of experts.