CS229
Machine Learning Foundations
Regression, classification, generalized linear models, and the mathematical base for later model work.
- 01 CS229 Machine Learning CS229 Machine Learning - Introduction and Overview CS229 的监督、无监督与强化学习三大板块，所需的线性代数、概率论与微积分基础，以及本系列的整体规划。 machine learning cs229 notes
- 02 CS229 Machine Learning CS229 Machine Learning - Linear Regression 把误差项假设为高斯噪声后，最大似然估计恰好等价于最小二乘，再用梯度下降迭代求解，并以局部加权回归拟合非线性。 machine learning cs229 notes
- 03 CS229 Machine Learning CS229 Machine Learning - Logistic Regression 分类问题的目标变量只取离散值，用 sigmoid 把线性输出压到 (0,1) 区间，即得到逻辑回归模型。 machine learning cs229 notes
- 04 CS229 Machine Learning CS229 Machine Learning - Generalized Linear Models 指数族的统一形式 p(y;η)=b(y)exp(ηᵀT(y)−a(η))，由它构造广义线性模型可把最小二乘与逻辑回归纳入同一框架。 machine learning cs229 notes
CS336
Language Modeling From Scratch
Tokenizer, Transformer, systems, scaling laws, data processing, alignment, and reasoning RL.
- 01 CS336 Language Modeling from Scratch CS336 Language Modeling from Scratch - Introduction CS336 几乎不提供脚手架代码，要求从 tokenizer、Transformer 一路手写到分布式训练与对齐，本系列按 5 个 Assignment 组织。 language models cs336 notes
- 02 CS336 Language Modeling from Scratch CS336 - Assignment 1: Tokenizer & Transformer Basics 只用 PyTorch primitives 手写 BPE 分词器、RMSNorm/RoPE/SwiGLU 与 AdamW，拼出可在 TinyStories 上训练收敛的 decoder-only 模型。 language models cs336 transformer tokenization
- 03 CS336 Language Modeling from Scratch CS336 - Assignment 2: Systems (Triton & Distributed) 先 profile 定位瓶颈，用 Triton 手写 FlashAttention-2 以分块加 online softmax 避开 N×N 注意力矩阵，再实现 DDP 与优化器状态分片。 language models cs336 triton distributed training
- 04 CS336 Language Modeling from Scratch CS336 - Assignment 3: Scaling Laws 训练一批小模型拟合幂律再外推，用 IsoFLOP 方法在固定算力预算下寻找 loss 最低点，预测 compute-optimal 的模型规模与数据量。 language models cs336 scaling laws notes
- 05 CS336 Language Modeling from Scratch CS336 - Assignment 4: Data Processing & Filtering 把原始 Common Crawl 变成预训练语料：WARC 抽正文、fastText 识别语言、Gopher 规则与分类器过滤、PII 脱敏与去重。 language models cs336 data notes
- 06 CS336 Language Modeling from Scratch CS336 - Assignment 5: Alignment & Reasoning RL SFT 只在 response 上计算 loss 做指令微调，再用 expert iteration 与 GRPO 强化学习提升 MATH 等数学推理基准表现。 language models cs336 alignment reinforcement learning
Algorithms
Algorithm System
Core techniques, data structures, graph theory, strings, math, geometry, and review notes.
- 01 Algorithm Miscellany Algorithm Miscellany - Basics 用 bitset 压位加速集合运算，树上前缀和与差分借 LCA 处理路径统计，倍增法压缩大状态空间的递推。 algorithm review
- 02 Algorithm Miscellany Algorithm Miscellany - Prefix Sum and Difference 前缀和把区间求和降到 O(1)，差分把区间修改降到 O(1)，二维情形用容斥原理查询任意子矩阵。 algorithm data structure review
- 03 Algorithm Miscellany Algorithm Miscellany - Search A* 靠不超过真实距离的启发函数保证最优性，迭代加深与 IDA* 分别用深度和代价上限控制 DFS，配合记忆化与可行性剪枝。 algorithm search review
- 04 Algorithm Miscellany Algorithm Miscellany - Dynamic Programming 逐维求和把 k 维前缀和做到 O(kN)，据此得到 O(n·2ⁿ) 的子集和 DP，并用状态合法性判定重构最长不下降子序列。 algorithm dynamic programming review
- 05 Algorithm Miscellany Algorithm Miscellany - String 字符串哈希由哈希值不等推出原串不等，Trie 按字符转移建树，KMP 用前缀函数记录最长相等真前后缀以避免回溯。 algorithm string review
- 06 Algorithm Miscellany Algorithm Miscellany - Mathematics 数论中整除与约数的基本性质，以及多项式与生成函数的基本概念。 algorithm math review
- 07 Algorithm Miscellany Algorithm Miscellany - Data Structures 并查集用路径压缩加按大小合并维护集合归属，再以分块、ST 表与线段树支撑区间查询与修改。 algorithm data structure review
- 08 Algorithm Miscellany Algorithm Miscellany - Graph Theory 树的中心使最长链最短、重心使最大连通块不超过一半，重链剖分把任意路径拆成 O(log n) 条连续链交给线段树维护。 algorithm graph theory tree review
- 09 Algorithm Miscellany Algorithm Miscellany - Computational Geometry Algorithm Miscellany series: computational geometry-related algorithms. algorithm computational geometry review
- 10 Algorithm Miscellany Algorithm Miscellany - Miscellaneous 离散化用排序去重加二分把值域压到 O(n)，CDQ 分治则把动态问题转化为静态的点对问题。 algorithm review
OJ Practice
Problem Solving
Selected Codeforces and Luogu writeups focused on construction, bit tricks, trees, and implementation details.
- 01 Codeforces Solutions XOR Convenience - Codeforces Solution 把所有异或结果都导向最后一个位置，构造出每个值与其后某个值异或后等于其下标的排列。 algorithm codeforces construction bit manipulation
- 02 Luogu Solutions P1131 [ZJOI2007] Time Synchronization - Luogu Solution 树形 DP 求每棵子树内根到叶的最长路径，再在各子节点分支上补齐差值，只增加边权即可让所有叶子到达时间一致。 algorithm luogu tree dp tree
- 03 Codeforces Solutions MEX Reordering - Codeforces Solution 通过分析 0 与 1 的出现次数构造重排，使任意前缀与后缀的 MEX 都不相等，单次遍历 O(n) 完成。 algorithm codeforces construction
- 04 Codeforces Solutions The Curse of the Frog - Codeforces Solution 青蛙在数轴上跳跃且每种跳法有周期性罚时，贪心优先使用无罚时的跳法，O(n) 求出到达目标的最少罚时次数。 algorithm codeforces greedy
- 05 Codeforces Solutions XOR Array - Codeforces Solution 用前缀异或配合唯一重复值构造正整数数组，使恰好指定的子数组异或和为零，边算边输出，空间 O(1)。 algorithm codeforces construction bit manipulation