MuXinCG's Blog

Back

Roadmap

Computer Science Learning Map

Course notes, algorithm reviews, and problem writeups arranged into practical reading tracks.

10

Published posts

2

Series

13

Topic tags

CS229 4 CS336 6

CS229

Machine Learning Foundations

Regression, classification, generalized linear models, and the mathematical base for later model work.

View collection →
4 notes Updated March 2, 2026 machine learning cs229 notes
  1. 01 CS229 Machine Learning CS229 Machine Learning - Introduction and Overview CS229 的监督、无监督与强化学习三大板块，所需的线性代数、概率论与微积分基础，以及本系列的整体规划。 machine learning cs229 notes
  2. 02 CS229 Machine Learning CS229 Machine Learning - Linear Regression 把误差项假设为高斯噪声后，最大似然估计恰好等价于最小二乘，再用梯度下降迭代求解，并以局部加权回归拟合非线性。 machine learning cs229 notes
  3. 03 CS229 Machine Learning CS229 Machine Learning - Logistic Regression 分类问题的目标变量只取离散值，用 sigmoid 把线性输出压到 (0,1) 区间，即得到逻辑回归模型。 machine learning cs229 notes
  4. 04 CS229 Machine Learning CS229 Machine Learning - Generalized Linear Models 指数族的统一形式 p(y;η)=b(y)exp(ηᵀT(y)−a(η))，由它构造广义线性模型可把最小二乘与逻辑回归纳入同一框架。 machine learning cs229 notes

CS336

Language Modeling From Scratch

Tokenizer, Transformer, systems, scaling laws, data processing, alignment, and reasoning RL.

View collection →
6 notes Updated June 5, 2026 language models cs336 notes transformer tokenization
  1. 01 CS336 Language Modeling from Scratch CS336 Language Modeling from Scratch - Introduction CS336 几乎不提供脚手架代码，要求从 tokenizer、Transformer 一路手写到分布式训练与对齐，本系列按 5 个 Assignment 组织。 language models cs336 notes
  2. 02 CS336 Language Modeling from Scratch CS336 - Assignment 1: Tokenizer & Transformer Basics 只用 PyTorch primitives 手写 BPE 分词器、RMSNorm/RoPE/SwiGLU 与 AdamW，拼出可在 TinyStories 上训练收敛的 decoder-only 模型。 language models cs336 transformer tokenization
  3. 03 CS336 Language Modeling from Scratch CS336 - Assignment 2: Systems (Triton & Distributed) 先 profile 定位瓶颈，用 Triton 手写 FlashAttention-2 以分块加 online softmax 避开 N×N 注意力矩阵，再实现 DDP 与优化器状态分片。 language models cs336 triton distributed training
  4. 04 CS336 Language Modeling from Scratch CS336 - Assignment 3: Scaling Laws 训练一批小模型拟合幂律再外推，用 IsoFLOP 方法在固定算力预算下寻找 loss 最低点，预测 compute-optimal 的模型规模与数据量。 language models cs336 scaling laws notes
  5. 05 CS336 Language Modeling from Scratch CS336 - Assignment 4: Data Processing & Filtering 把原始 Common Crawl 变成预训练语料：WARC 抽正文、fastText 识别语言、Gopher 规则与分类器过滤、PII 脱敏与去重。 language models cs336 data notes
  6. 06 CS336 Language Modeling from Scratch CS336 - Assignment 5: Alignment & Reasoning RL SFT 只在 response 上计算 loss 做指令微调，再用 expert iteration 与 GRPO 强化学习提升 MATH 等数学推理基准表现。 language models cs336 alignment reinforcement learning