Training Cucafera: A Catalan Language Model from Scratch
A project story about training a 244M-parameter model on approximately 5.5B Catalan tokens with a single A100.
AI research · Barcelona / Banyoles
Pau Hidalgo Pujol
I’m an AI student and machine-learning researcher working on geometric deep learning for biology, language models, and interpretable systems.
Currently an AI Research Intern at Nostrum Biodiscovery and completing a BSc in Artificial Intelligence at UPC, after an exchange at ETH Zürich.
Writing
Latest writing
Research
Current and recent work
GRaM @ ICLR 2026Accepted
Fewer Edges, Faster Protein Graph Learning
Graph construction and efficient geometric message passing for protein representation learning.
Research note →Bachelor’s thesis2026
Human Attention for Large Language Model Training
Eye-tracking signals for training, evaluating, and interpreting language models.
Thesis summary →Projects
Selected projects
Background
My interests span mathematical foundations, geometric ML for biology, and language technology. I also hold a professional music diploma in cello.
Biography, experience, and CV →