Musings by Chris Hayduk

The Foundation Model Trap

Understanding DeepSeek Part II: DeepSeek-V2

A Primer on Multi-Head Causal Self-Attention

Understanding DeepSeek Part I: DeepSeekMoE

Understanding Protein Language Models Part III: Structure Prediction without Multiple Sequence Alignment in ESMFold

Understanding Protein Language Models Part II: Encoder-only Transformers as Continuous Fuzzy String Matching

OpenAI o3 and the Rise of the Intelligence Allocator

On Algorithmic Moats and the Path to AGI

Rejuvenating the Political System

ESM3 and the Future of Protein Language Models

A Perspective on the Limitations of Language Modeling

A Case Study in Finetuning Open Source LLMs: Training LLaMA 2 for the Text-to-SQL Task