Human Attention for Large Language Model Training
Bachelor’s thesis developed with Telefónica I+D and UPC, awarded 10/10 and proposed for honours.
Tagged
Language Models
5 items
→
LaIA
A web-search and video creation assistant designed to make public information more accessible to young people.
Training Cucafera: A Catalan Language Model from Scratch
A project story about training a 244M-parameter model on approximately 5.5B Catalan tokens with a single A100.
Cucafera
244M parameters, 5.5B training tokens, one A100 and a transparent record of how it was made.
Patufet
Pretraining text, educational material, stories, code, prompts, conversations, and instruction data.