Keiran Paster


I am a fifth-year PhD student in the Department of Computer Science at the University of Toronto. I am fortunate to be advised by Jimmy Ba and Sheila McIlraith. I am also affiliated with the Vector Institute.

In 2023, I was a student researcher at Google working on data efficiency for LLMs with the Gemini and Blueshift teams.

My research interests lie in AI reasoning. Research highlights include:

  • Automatic prompt engineering (resulting in a popular prompting trick that gives insight into the psychology of GPT).
  • OpenWebMath, a dataset of all the mathematical documents on Common Crawl, and Llemma, the best open language model for mathematical reasoning.
  • STEVE-1, a Minecraft agent that can follow visual and text instructions using only keyboard and mouse commands.

selected publications

  1. llemma.png
    Llemma: An Open Language Model For Mathematics
  2. owm-color.png
    OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
    Keiran Paster*Marco Dos Santos*Zhangir Azerbayev, and Jimmy Ba
  3. steve-1.gif
    STEVE-1: A Generative Model for Text-to-Behavior in Minecraft
    In Advances in Neural Information Processing Systems, 2023
  4. ape-algo.gif
    Large Language Models are Human-Level Prompt Engineers
    In The Eleventh International Conference on Learning Representations , 2023
  5. esper.gif
    You Can’t Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments
    Keiran PasterSheila A. McIlraith, and Jimmy Ba
    In Advances in Neural Information Processing Systems, 2022
  6. glamor.gif
    Planning from Pixels using Inverse Dynamics Models
    Keiran PasterSheila A. McIlraith, and Jimmy Ba
    In International Conference on Learning Representations, 2021