Selim Jerad

PhD Student at the Toyota Technological Institute at Chicago

My research focuses on the theoretical foundations of language modeling architectures and establishing which problems they can(not) solve.

I am advised by William Merrill at TTIC and Ryan Cotterell at ETH Zürich. Prior to my doctoral studies, I completed a BSc in Communication Systems at EPFL and a MSc in Computer Science at ETH Zürich, with visiting student positions at the NYU Center for Data Science and the Okinawa Institute of Science and Technology.

sjerad@ttic.edu

Research Interests

I am broadly interested in studying language models as formal models of computation to delineate which problems they can solve, and how compactly they do so.

Transformer Theory. Expressivity and succinctness characterizations of transformers under different architectural assumptions (precision, attention, position encodings) from the lens of descriptive complexity (syntactic monoids, linear temporal logic, first-order logic, automata theory).
Transformer Variants. Developing a theory of novel architectural paradigms such as RNN-transformer hybrids and looped-padded transformers, with connections to circuit complexity.
Computational Complexity. Characterizations of context-free languages and subregular classes, bounds and tightness results for parsing, separations or collapses in circuit complexity.

Publications

Disentangling the Expressivity of RoPE
Selim Jerad, Anej Svete, Jiaoda Li and Ryan Cotterell
COLM 2026
[arXiv]

Context-Free Recognition with Transformers
Selim Jerad, Anej Svete, Sophie Hao, Ryan Cotterell, and William Merrill
ICML 2026
[arXiv]

Unique Hard Attention: A Tale of Two Sides
Selim Jerad, Anej Svete, Jiaoda Li, and Ryan Cotterell
ACL 2025 (Oral)
[arXiv]

Selim Jerad

PhD Student at the Toyota Technological Institute at Chicago

Research Interests

Publications

Invited talks