Keiran Paster

prof_pic.jpg

I am a fifth-year PhD candidate (on leave) in the Department of Computer Science at the University of Toronto and an early employee at xAI.

Highlights from my time at xAI include:

  • Grok 2 and Grok 2 mini, where I did the web data filtering, ablation, and data mixing efforts in the data team. I also contributed to the post-training of the “sus-column-r” model, xAI’s first model on Chatbot Arena.
  • Grok 3 and Grok 3 mini, where I did the web data filtering, ablation, and data mixing. Additionally, I was an initial member of the reasoning team and a core contributor to the reasoning models, where I did the training and data recipes.
  • Grok 4, where I led the RL data and ablation efforts as we scaled up our RL training by over 10x.
  • Grok 4 Fast, Grok 4.1, Grok 4.1 Fast, and Grok 4.20, where I led the Grok Next team, overseeing the reasoning and post-training teams. Some highlights during this time are our advancements in post-training, search, mathematical proofs, forecasting, and multi-agent.

selected publications

  1. llemma.png
    Llemma: An Open Language Model For Mathematics
    In International Conference on Learning Representations, 2024
  2. owm-color.png
    OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
    Keiran Paster*Marco Dos Santos*Zhangir Azerbayev, and Jimmy Ba
    In International Conference on Learning Representations, 2024
  3. steve-1.gif
    STEVE-1: A Generative Model for Text-to-Behavior in Minecraft
    In Advances in Neural Information Processing Systems, 2023
    Spotlight
  4. ape-algo.gif
    Large Language Models are Human-Level Prompt Engineers
    In International Conference on Learning Representations, 2023
  5. esper.gif
    You Can’t Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments
    Keiran PasterSheila A. McIlraith, and Jimmy Ba
    In Advances in Neural Information Processing Systems, 2022
  6. glamor.gif
    Planning from Pixels using Inverse Dynamics Models
    Keiran PasterSheila A. McIlraith, and Jimmy Ba
    In International Conference on Learning Representations, 2021