Summer 2026 — days 1–57, to · set out in full
foundations before the quarter starts · free, ~70 hours a week · 57 days
Day 1 —
Limited direct execution, and the paper everything else is measured against
- Paper (AI): Attention Is All You Need — Vaswani et al., 2017. The architecture everything else in the two years is built on or measured against. Read it first, and again on day 29 when you are implementing it.
- Systems chapter: OSTEP, read in chapter order — Ch 6 · Mechanism: limited direct execution. Leaves: Write the timeline of one timer interrupt from user code to handler and back, naming at every step which register set is live and who is responsible for saving it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 2 · Linear algebra, §2.4–2.5 linear dependence, span, norms. Leaves: Write one 3x3 system out in both index and matrix notation, invert it by hand, then find the b for which it has no solution and justify that from the span of the columns rather than from the arithmetic.
- System (atomic — whole in the day): Implement a user-level context switch between two stacks with a hand-written save and restore of the callee-saved registers and the stack pointer, plus a trap into a handler that returns to the exact instruction it left. Processes and fork/exec you have already read; this is the mechanism underneath them. Measure: Every callee-saved register and the stack pointer are identical across the round trip, asserted rather than eyeballed, and the switches per second over a million alternations are recorded.
- Open source — logged, not scheduled: nothing logged for this day
Day 2 —
ferrodb starts: the page and the log record
- Paper (systems): ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging — Mohan et al., 1992. The write-ahead log ferrodb is about to implement, including why redo and undo are separate passes. Read it the day you design the record format, not after.
- Systems chapter: OSTEP, read in chapter order — Ch 7 · Scheduling: introduction. Leaves: Hand-compute turnaround and response for one three-job workload under FIFO, SJF and STCF, and name the arrival pattern that makes FIFO worst.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 2 · §2.6–2.8 special matrices and vectors, eigendecomposition, the singular value decomposition. Leaves: Eigendecompose a 2x2 symmetric matrix by hand, then take the SVD of a 2x3 matrix that has no eigendecomposition at all, and write the one line on which of the two always exists.
- System (step 1 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Define the on-disk page format and the WAL record format; write the encoder/decoder and round-trip tests. Measure: A property test that encode(decode(x)) == x for a thousand random records, and the byte size of the page header stated.
- Open source — logged, not scheduled: nothing logged for this day
Day 3 —
Reverse-mode autodiff, written by hand
- Paper (AI): Automatic Differentiation in Machine Learning: a Survey — Baydin, Pearlmutter, Radul, Siskind, 2018. Names what reverse mode actually is before you write it, and why the numerical gradient check is the test that matters.
- Systems chapter: OSTEP, read in chapter order — Ch 8 · The multi-level feedback queue. Leaves: Construct the interactive job that starves under MLFQ without the priority boost — arrivals and burst lengths written out — then show on paper what the boost does to it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 2 · §2.9–2.12 the Moore-Penrose pseudoinverse, the trace operator, the determinant, and PCA. Leaves: Derive PCA the way §2.12 does — minimise reconstruction error, arrive at the top eigenvectors — and verify it numerically on a three-dimensional cloud you generate. Then write the one line linking §2.9's pseudoinverse to least squares; it is the same argument.
- System (atomic — whole in the day): Implement a scalar autodiff engine: a Value type with a backward closure per operation, topological ordering, and gradient accumulation. Measure: A finite-difference gradient check agreeing to 1e-6 on a composed expression with at least four operations.
- Open source — logged, not scheduled: nothing logged for this day
Day 4 —
A scheduler that is a probability distribution, and one that is not
- Paper (systems): A Fast File System for UNIX — McKusick, Joy, Leffler, Fabry, 1984. Inodes, cylinder groups and the locality argument. Read it against the page layout you froze on day 2 — FFS is the same decision made for a disk that rotates.
- Systems chapter: OSTEP, read in chapter order — Ch 9 · Scheduling: proportional share. Leaves: Work out the ticket allocation that gives three jobs a 50/30/20 share, then predict how far a hundred-quantum run will drift from it before you run one.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · Probability and information theory, §3.1–3.3 why probability, random variables, probability distributions. Leaves: Write today's lottery scheduler out as a probability distribution — sample space, PMF, what the ticket count is doing to it — and then say in one line why stride scheduling is not a distribution at all, which is exactly why it does not drift.
- System (atomic — whole in the day): Implement lottery scheduling and stride scheduling over the same ticket allocation, and run both for a hundred quanta and for ten thousand. Measure: Unfairness as the ratio of completion times for two equal-ticket jobs, reported at both run lengths, and stride shown to give the identical schedule across three seeds where lottery does not.
- Open source — logged, not scheduled: nothing logged for this day
Day 5 —
ferrodb: the write-ahead log
- Paper (AI): Adam: A Method for Stochastic Optimization — Kingma, Ba, 2015. The other half of this week's SGD-versus-Adam comparison; read the bias-correction argument, not just the update rule.
- Systems chapter: OSTEP, read in chapter order — Ch 10 · Multiprocessor scheduling. Leaves: Predict what one cache-cold migration costs in cycles from your machine's cache sizes, then measure it by pinning a thread and moving it, and keep both numbers.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · §3.4–3.6 marginal probability, conditional probability, the chain rule. Leaves: Factor one three-variable joint two different ways with the chain rule and show the two agree numerically; then write down the conditional that is undefined, and what the code should do when it reaches it.
- System (step 2 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement the write-ahead log: append, fsync policy, and a replay that is idempotent. Measure: Replay applied twice leaves byte-identical state; the fsync count per commit is measured and stated.
- Open source — logged, not scheduled: nothing logged for this day
Day 6 — · catch-up
Catch-up — close what the last five days left open
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (systems): Ethernet: Distributed Packet Switching for Local Computer Networks — Metcalfe, Boggs, 1976. Carrier sense, collision detection, exponential backoff — a complete system argued in fifteen pages. A catch-up day is the right day for a paper with nothing to implement.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 7 —
Linear and logistic regression, and the optimizer comparison
- Paper (systems): The Design and Implementation of a Log-Structured File System — Rosenblum, Ousterhout, 1992. The counterargument to FFS, and the one that makes crash recovery cheap. Compare it against the WAL you finished yesterday: both turn a random write into a sequential one.
- Systems chapter: OSTEP, read in chapter order — Ch 13 · The abstraction: address spaces. Leaves: Print your own program's code, stack and heap addresses, draw the address space they imply, and mark the two places that picture is a lie.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · §3.7–3.9 independence, expectation, variance and covariance, and the common distributions. Leaves: Construct two variables with zero covariance that are plainly dependent, then compute the mean and variance of the Bernoulli and the Gaussian by hand — the two distributions today's logistic and linear regressions are each assuming without saying so.
- System (atomic — whole in the day): Implement linear and logistic regression on NumPy with your own gradients; train the same model with SGD and with Adam. Measure: A loss curve for both optimizers on the same seed, and a sentence on which converged faster and at what cost.
- Open source — logged, not scheduled: nothing logged for this day
Day 8 —
The memory API, and the three ways it goes wrong
- Paper (systems): The Working Set Model for Program Behavior — Denning, 1968. Why page replacement is a locality question. The eviction policy ferrodb's buffer pool gets in three days is an answer to this paper.
- Systems chapter: OSTEP, read in chapter order — Ch 14 · Interlude: memory API. Leaves: Write the leak, the dangling free and the off-by-one as three five-line programs, run each under a memory checker, and record the exact message each one produces.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · §3.10–3.11 useful properties of common functions, and Bayes' rule. Leaves: Prove the four sigmoid and softplus identities in §3.10 by hand, then rewrite yesterday's logistic loss in terms of softplus and show on paper which of the two forms overflows first. Today is a day about bugs that return a number instead of crashing; this is one.
- System (atomic — whole in the day): Write an interposing malloc/free shim that records every allocation site and reports leaks, double frees and writes past the end using guard bytes; run it against the three bugs from this morning. Measure: All three are reported with the allocating line, and the shim's added cost per allocation is measured against the real allocator.
- Open source — logged, not scheduled: nothing logged for this day
Day 9 —
ferrodb: the pager and the buffer pool
- Paper (AI): Learning Representations by Back-Propagating Errors — Rumelhart, Hinton, Williams, 1986. Four pages. Read it the day after you wrote backprop by hand, not before — it is much better when you already know what it costs.
- Systems chapter: OSTEP, read in chapter order — Ch 15 · Mechanism: address translation. Leaves: Translate one virtual address to physical by hand under base-and-bounds, then find the two addresses that must trap and say which comparison catches each.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · §3.12 the technical details of continuous variables, and §3.13 self-information and entropy. Leaves: Compute the entropy of three distributions by hand — a fair coin, a biased coin, a point mass — and plot Bernoulli entropy against p; mark where the derivative blows up and say why that matters at the edges of a predicted probability.
- System (step 3 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Build the pager: fixed-size pages, a free list, and a buffer pool with pin/unpin. Measure: A test that a pinned page is never evicted, and the pool's memory footprint at 1000 pages.
- Open source — logged, not scheduled: nothing logged for this day
Day 10 —
A two-layer MLP, twice
- Paper (AI): Multilayer Feedforward Networks are Universal Approximators — Hornik, Stinchcombe, White, 1989. What a two-layer MLP can represent in principle — and why that says nothing about what it will learn. Read it the day you train one.
- Systems chapter: OSTEP, read in chapter order — Ch 16 · Segmentation. Leaves: Draw the three-segment layout with the stack's growth bit set, compute the physical address of one negative offset by hand, and say exactly what happens when that bit is wrong.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 3 · §3.13 continued: KL divergence and cross-entropy, and §3.14 structured probabilistic models. Leaves: Derive the cross-entropy your MLP minimises today as the KL from the empirical distribution to the model, showing the constant that drops out — then confirm on one real batch that your loss and the KL differ by exactly that constant.
- System (atomic — whole in the day): Implement a two-layer MLP on your own autodiff, train it on a small classification set, then reproduce it in PyTorch. Measure: The two agree to 1e-4 on the loss after one epoch with the same seed and initialization.
- Open source — logged, not scheduled: nothing logged for this day
Day 11 —
Free space, and the fragmentation nobody measures
- Paper (systems): End-to-End Arguments in System Design — Saltzer, Reed, Clark, 1984. The design principle that decides which layer of ferrodb owns the checksum. Answer that question in the note, in the paper's terms.
- Systems chapter: OSTEP, read in chapter order — Ch 17 · Free-space management. Leaves: Run best-fit, worst-fit and first-fit by hand over one ten-request trace, writing the free list out after every request; keep the trace, it is today's fixture.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 4 · Numerical computation, §4.1 overflow and underflow. Leaves: Make the naive softmax overflow and then underflow on inputs you choose deliberately, implement the shifted version and log-sum-exp, and show all three agreeing on the range where the naive one still works.
- System (atomic — whole in the day): Implement a free-list allocator with splitting and coalescing and the three placement policies behind one switch; drive all three from the same request trace. Measure: External fragmentation as a percentage of the arena after the trace, per policy, and the first request that fails under each.
- Open source — logged, not scheduled: nothing logged for this day
Day 12 — · catch-up
Catch-up — rerun the experiment that surprised you
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Dropout: A Simple Way to Prevent Neural Networks from Overfitting — Srivastava, Hinton, Krizhevsky, Sutskever, Salakhutdinov, 2014. The ensemble interpretation is the part worth arguing with. If your day-14 ablation disagreed with it, say so in the card.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 13 —
ferrodb: eviction that respects the log
- Paper (AI): Understanding the Difficulty of Training Deep Feedforward Neural Networks — Glorot, Bengio, 2010. Where initialization ablations come from. Reproduce one of its activation-variance plots on your own classifier this week.
- Systems chapter: OSTEP, read in chapter order — Ch 18 · Paging: introduction. Leaves: Walk one virtual address through a linear page table by hand, then compute what that table costs for a 32-bit space with 4 KB pages and say why nobody ships it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 4 · §4.2 poor conditioning. Leaves: Compute a 2x2 matrix's condition number from its eigenvalues by hand, then build the least-squares problem that loses four decimal digits to it and show them being lost. If §4.2 is too terse to convince you, Prince ch 6 plots the same surface before arguing about it — look once, then come back and write the eigenvalues down.
- System (step 4 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Add LRU eviction to the buffer pool; assert no dirty page is evicted without a WAL flush. Measure: A test that fails if the assertion is removed — write the failing version first.
- Open source — logged, not scheduled: nothing logged for this day
Day 14 —
Initialization, optimizers, and the curvature underneath both
- Paper (AI): Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification — He, Zhang, Ren, Sun, 2015. The correction to Glorot for ReLU nets — a factor of two that decides whether deep stacks train at all. Run both initializations today.
- Systems chapter: OSTEP, read in chapter order — Ch 19 · Translation lookaside buffers. Leaves: Predict the array stride that destroys TLB hit rate from your page size, then find the cliff with a stride benchmark and report the measured stride against the predicted one.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 4 · §4.3–4.5 gradient-based optimization, the Jacobian and Hessian, constrained optimization, linear least squares. Leaves: Take the Hessian of today's loss at both initializations, get its condition number from the eigenvalues, and say from that number alone which of your two ablations — Glorot against He, SGD against Adam — is fixing curvature and which is only fixing scale. The results table has to agree with the eigenvalues; if it does not, one of the two is wrong and you find out which.
- System (atomic — whole in the day): Run controlled initialization and optimizer ablations on one small classifier: Glorot versus He, SGD versus momentum versus Adam, three seeds each. Measure: A results table with mean and spread, and one sentence naming the effect that survives the seed noise.
- Open source — logged, not scheduled: nothing logged for this day
Day 15 —
Page tables that fit, and what the second walk costs
- Paper (systems): Experience with Processes and Monitors in Mesa — Lampson, Redell, 1980. Why a condition variable wakes into a loop and not into a guarantee. Read it before ferrodb has concurrent readers, not after they deadlock.
- Systems chapter: OSTEP, read in chapter order — Ch 20 · Paging: smaller tables. Leaves: Compute the memory a two-level table saves over a linear one for an address space with three sparse regions, and the extra memory reference per translation you paid for it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 5 · Machine learning basics, §5.1–5.2 learning algorithms, capacity, overfitting and underfitting. Leaves: Write out the task, the experience and the performance measure for three things you have already built this summer, in §5.1's terms, then draw the capacity-against-error curve you expect for one of them and keep the drawing to check against later.
- System (atomic — whole in the day): Build a two-level page table with valid and protection bits at both levels and a walker that faults at either, plus a linear table in the same file as the reference; run one address trace through both. Measure: Every address translates identically in the two, and the table memory of each is reported for an address space with three sparse regions.
- Open source — logged, not scheduled: nothing logged for this day
Day 16 —
ferrodb: the B-tree, leaves first
- Paper (AI): Gradient-Based Learning Applied to Document Recognition — LeCun, Bottou, Bengio, Haffner, 1998. LeNet, and the case for learned features over engineered ones. Long — read sections I–III today and stop there.
- Systems chapter: OSTEP, read in chapter order — Ch 21 · Beyond physical memory: mechanisms. Leaves: Add a present bit and a swap slot to the day-15 walker, then trace one fault end to end — evict, write back, read in, restart — counting the memory and disk accesses that single instruction cost.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 5 · §5.3–5.4 hyperparameters and validation sets, estimators, bias and variance. Leaves: Define the train/validation/test protocol you will use for the next 700 days in writing — split sizes, how seeds are drawn, what is never touched — and derive the bias of the variance estimator you are going to report alongside it.
- System (step 5 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement a B-tree: search and insert on leaf pages only, with a page-split test. Measure: The split test asserts key order across the two resulting pages and that no key was lost.
- Open source — logged, not scheduled: nothing logged for this day
Day 17 —
im2col, and a CNN that trains
- Paper (AI): ImageNet Classification with Deep Convolutional Neural Networks — Krizhevsky, Sutskever, Hinton, 2012. The same idea as LeNet fourteen years later with GPUs behind it. Note what changed: scale and engineering, not the concept.
- Systems chapter: OSTEP, read in chapter order — Ch 22 · Beyond physical memory: policies. Leaves: Run FIFO, LRU, clock and random over one reference string by hand, then construct the string that makes FIFO show Belady's anomaly and show it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 5 · §5.5–5.6 maximum likelihood estimation, and Bayesian statistics. Leaves: Derive least squares and cross-entropy from maximum likelihood, naming the distributional assumption each one is, then work the same problem as MAP under a Gaussian prior and watch weight decay fall out of it.
- System (atomic — whole in the day): Implement convolution via im2col on your own autodiff, then train a compact CNN with a clean harness and fixed seeds. Measure: Your im2col output matches a reference conv2d to 1e-5, and the training run is reproducible from the seed alone.
- Open source — logged, not scheduled: nothing logged for this day
Day 18 — · catch-up
Catch-up — write the note you skipped
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift — Ioffe, Szegedy, 2015. The stated mechanism has since been disputed. Say in your card whether your own evidence supports it — you have run the ablation.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 19 —
Virtual memory, assembled end to end
- Paper (systems): Congestion Avoidance and Control — Jacobson, Karels, 1988. Slow start, RTT estimation, self-clocking. Read it as a control-loop paper — it is the same shape as the group-commit decision ferrodb makes later.
- Systems chapter: OSTEP, read in chapter order — Ch 23 · Complete virtual memory systems. Leaves: Draw the complete path — translation, TLB, two-level walk, fault, eviction, restart — as one diagram carrying your own function names, and mark the three places VAX/VMS put a policy where you put a constant.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 5 · §5.7–5.8 supervised learning algorithms, and unsupervised learning algorithms. Leaves: Implement one algorithm from each section on the same data — logistic regression and PCA — and write the one sentence about what each is estimating that makes them different problems rather than different code.
- System (atomic — whole in the day): Join the day-15 walker, a small TLB in front of it and the swap path from day 16 into a single access() entry point, and push a real program's address trace through it. Measure: TLB hit rate, fault count and swap bytes for the trace, every reloaded page byte-identical, and a second run on the same trace producing the same three numbers.
- Open source — logged, not scheduled: nothing logged for this day
Day 20 —
ferrodb: internal nodes and root splits
- Paper (AI): Long Short-Term Memory — Hochreiter, Schmidhuber, 1997. The constant error carousel, stated as a fix to a specific failure. Read it before you train the RNN that diverges, and note what the gate is protecting.
- Systems chapter: OSTEP, read in chapter order — Ch 26 · Concurrency: an introduction. Leaves: Disassemble the counter increment, mark the two instruction boundaries the race lives between, then run the two-thread version until it loses an update and record how many runs that took.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 3 · N-gram language models. Leaves: Compute perplexity by hand on a five-word held-out set, then in code, and get the same number.
- System (step 6 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Add internal nodes and root splits; property-test that every key is still findable. Measure: Ten thousand random inserts, then every key found; tree height reported against key count.
- Open source — logged, not scheduled: nothing logged for this day
Day 21 —
A character-level RNN, and why it fails
- Paper (AI): On the Difficulty of Training Recurrent Neural Networks — Pascanu, Mikolov, Bengio, 2013. Names the exploding and vanishing gradient precisely enough to measure. Gives you clipping, and the reason for it.
- Systems chapter: OSTEP, read in chapter order — Ch 27 · Interlude: thread API. Leaves: Write create, join and detach as three small programs, including the one whose detached thread outlives main, and record what each does to the exit status.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 5 · §5.9–5.11 stochastic gradient descent, building a machine learning algorithm, and the challenges motivating deep learning. Leaves: Write today's character RNN out as §5.10's four pieces — dataset, cost, model, optimization procedure — and then name which of §5.11's challenges the gradient-norm histogram you measured is evidence of, and which it rules out.
- System (atomic — whole in the day): Train a character-level RNN on a small corpus; log gradient norm per step, then add clipping and rerun. Measure: The gradient-norm histogram before and after clipping, with the clip threshold justified from it.
- Open source — logged, not scheduled: nothing logged for this day
Day 22 —
Locks, and what they cost under contention
- Paper (systems): Development of the Domain Name System — Mockapetris, Dunlap, 1988. A retrospective on a live system: what the design got right, and what caching cost. Read it for the retrospective form — you will write these.
- Systems chapter: OSTEP, read in chapter order — Ch 28 · Locks. Leaves: Write test-and-set, test-and-test-and-set and a ticket lock as three short loops over the atomic they need, and predict which one starves a thread before you measure it.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 6 · Vector semantics and embeddings. Leaves: Build a term-document matrix on a small corpus and compute PPMI by hand for three word pairs.
- System (atomic — whole in the day): Implement a test-and-set lock, a test-and-test-and-set lock and a ticket lock over the same critical section, and benchmark all three from 1 to 16 threads. Measure: Throughput against thread count for each, and acquisition fairness as the ratio of most to fewest acquisitions per thread — which is where the ticket lock earns its extra cost.
- Open source — logged, not scheduled: nothing logged for this day
Day 23 —
ferrodb: delete, merge, and a fuzz loop
- Paper (AI): A Neural Probabilistic Language Model — Bengio, Ducharme, Vincent, Jauvin, 2003. Distributed word representations and the curse of dimensionality, a decade before it was obvious. This is the baseline word2vec is beating.
- Systems chapter: OSTEP, read in chapter order — Ch 29 · Lock-based concurrent data structures. Leaves: State the hand-over-hand list's protocol as an invariant — which two locks are held at the moment a node is unlinked — and mark the line of the coarse-grained version it replaces.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 6 · continued: word2vec and skip-gram. Leaves: Derive the skip-gram objective and write why negative sampling is not the same objective.
- System (step 7 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement delete and merge; check the tree invariant after every operation in a fuzz loop. Measure: A million random operations with the invariant checked after each, and the seed of any failure recorded.
- Open source — logged, not scheduled: nothing logged for this day
Day 24 — · catch-up
Catch-up — land what the last stretch left open
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (systems): Cramming More Components onto Integrated Circuits — Moore, 1965. Three pages, an economic argument rather than a physical law. Worth reading once, properly, instead of quoting.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 25 —
word2vec, implemented
- Paper (AI): Efficient Estimation of Word Representations in Vector Space — Mikolov, Chen, Corrado, Dean, 2013. word2vec. What was thrown away to make it fast is the interesting part, not the analogy arithmetic.
- Systems chapter: OSTEP, read in chapter order — Ch 30 · Condition variables. Leaves: Produce the missed wakeup with one condition variable over a bounded buffer, then write the two-variable fix and the reason the wait sits inside a while loop and not an if.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 6 · continued: evaluating embeddings. Leaves: Pick one intrinsic and one extrinsic evaluation, run both, and say where they disagree.
- System (atomic — whole in the day): Train skip-gram with negative sampling on a small corpus, on your own autodiff. Measure: Nearest neighbours for ten hand-chosen words, and one pair where the embedding is clearly wrong.
- Open source — logged, not scheduled: nothing logged for this day
Day 26 —
Semaphores, built out of what you already have
- Paper (systems): The Case for the Reduced Instruction Set Computer — Patterson, Ditzel, 1980. Eight pages, and a design argument rather than a specification. Read it now; riscv-vm implements the descendant of this argument in December.
- Systems chapter: OSTEP, read in chapter order — Ch 31 · Semaphores. Leaves: Derive the two semaphore operations from a mutex and a condition variable on paper, then write the initial value each of the three uses needs — lock, ordering, bounded buffer — and why.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 6 · Deep feedforward networks, §6.1–6.2 the XOR example, and gradient-based learning. Leaves: Derive algebraically why XOR needs a hidden layer, not by appeal to a picture, then work §6.2's output-unit table: for each output unit, the distribution it parameterises and the loss that follows from it. If §6.1 will not open, Prince ch 3 draws the piecewise-linear function first — look, then come back and write the algebra.
- System (atomic — whole in the day): Implement a semaphore from a mutex and a condition variable, then build a bounded buffer and a reader-writer lock on top of it, both under a thread sanitizer. Measure: The bounded buffer moves a million items with no element lost or duplicated, and writer starvation in the reader-writer lock is measured as a wait time rather than assumed away.
- Open source — logged, not scheduled: nothing logged for this day
Day 27 —
ferrodb: every mutation logged before it is applied
- Paper (AI): Neural Machine Translation by Jointly Learning to Align and Translate — Bahdanau, Cho, Bengio, 2015. Attention before transformers, invented to fix a fixed-length bottleneck. Implement this form before the scaled dot-product one.
- Systems chapter: OSTEP, read in chapter order — Ch 32 · Common concurrency problems. Leaves: Write the two-lock deadlock and an atomicity violation as the two smallest programs that can show them, then give the deadlock a lock-ordering fix and name which of the four conditions it removes.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 9 · RNNs and LSTMs. Leaves: Draw the LSTM cell from memory, then check it against the figure and mark what you got wrong.
- System (step 8 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Wire the B-tree through the pager so every mutation is logged before it is applied. Measure: An assertion that no page is written without a preceding log record, checked by the harness rather than by review.
- Open source — logged, not scheduled: nothing logged for this day
Day 28 —
Attention, from the matrix operations up
- Paper (AI): Effective Approaches to Attention-based Neural Machine Translation — Luong, Pham, Manning, 2015. The controlled comparison of scoring functions — the ablation you would otherwise have to run yourself.
- Systems chapter: OSTEP, read in chapter order — Ch 33 · Event-based concurrency. Leaves: Write the same worker twice, as threads and as an event loop over a readiness API, and say which one ferrodb wants in terms of what actually blocks in its write path.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 6 · §6.3 hidden units. Leaves: Write the derivative of ReLU, leaky ReLU, maxout and tanh at zero and at saturation, and for each say which of your own trained models it would have stalled — you have four of them now and the gradient logs to check against.
- System (atomic — whole in the day): Implement scaled dot-product attention and multi-head attention from matrix operations on your own autodiff, with shape tests. Measure: Attention weights sum to one along the right axis, checked as a test; one head visualised on a toy sequence.
- Open source — logged, not scheduled: nothing logged for this day
Day 29 —
What a path actually resolves to
- Paper (systems): An Efficient Algorithm for Exploiting Multiple Arithmetic Units — Tomasulo, 1967. Register renaming and reservation stations, 1967. Read it as the first out-of-order design; the pipeline you write in riscv-vm stalls where this one does not.
- Systems chapter: OSTEP, read in chapter order — Ch 39 · Interlude: files and directories. Leaves: Resolve one absolute path by hand with stat and readdir, naming every inode touched, then make a hard link and a symlink to the same file and write down what each did to the link count.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 6 · §6.4 architecture design: depth, width, and the universal approximation theorem. Leaves: State in two sentences what universal approximation does and does not promise, then count the linear regions of a 2-layer and a 4-layer network at the same parameter count and say which of the two facts the theorem is silent about.
- System (atomic — whole in the day): Write a resolver that walks a path component by component with openat and readdir, and a tool that prints inode number, link count and size for every entry it touches; run both over a tree you build with hard links, symlinks and one deliberate symlink loop. Measure: The resolver agrees with realpath on a hundred paths in that tree, and the loop is reported as a loop rather than hanging or recursing to the open-file limit.
- Open source — logged, not scheduled: nothing logged for this day
Day 30 — · catch-up
Catch-up — the ferrodb failure classes, written up
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Deep Residual Learning for Image Recognition — He, Zhang, Ren, Sun, 2016. The degradation problem is an optimization failure, not an overfitting one. That distinction is the one your own training notes keep getting wrong.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 31 —
ferrodb: crash recovery
- Paper (AI): Language Models are Unsupervised Multitask Learners — Radford, Wu, Child, Luan, Amodei, Sutskever, 2019. GPT-2: the decoder-only stack you implement tomorrow, and the zero-shot claim that made scale the story.
- Systems chapter: OSTEP, read in chapter order — Ch 40 · File system implementation. Leaves: Work out block by block what open, write and read cost in the vsfs layout — inode bitmap, data bitmap, inode, data — and total the I/Os for creating a five-block file from empty.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 10 · Transformers and large language models. Leaves: Write the causal mask as a matrix and show what changes if you get the off-by-one wrong.
- System (step 9 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement crash recovery: replay the WAL to the last checkpoint, then verify the tree. Measure: Recovery from a log of 10k records timed, and the tree invariant checked after every recovery.
- Open source — logged, not scheduled: nothing logged for this day
Day 32 —
A causal decoder block that generates
- Paper (AI): On Layer Normalization in the Transformer Architecture — Xiong et al., 2020. Pre-LN versus post-LN, with the gradient analysis. Decides whether the block you write today needs a warmup schedule.
- Systems chapter: CS:APP 3e, by section — Ch 1 · A tour of computer systems. Leaves: Follow one hello-world from source to process, naming every artifact the toolchain produced along the way.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 6 · §6.5–6.6 back-propagation and other differentiation algorithms, and the historical notes. Leaves: Draw the computational graph of your day-3 autodiff engine the way §6.5 draws it, mark every place the book's algorithm and your code differ, and say which of those differences is a bug rather than a choice. Twenty-seven days late is the right time to read this: you already know what it cost.
- System (atomic — whole in the day): Implement a causal decoder block with mask tests and a KV cache for incremental generation. Measure: Generation with and without the cache produces identical tokens, and the per-token time of each is measured.
- Open source — logged, not scheduled: nothing logged for this day
Day 33 —
What a number actually is on this machine
- Paper (systems): First Draft of a Report on the EDVAC — von Neumann, 1945. Where the stored-program machine is argued from logic elements up. Read the first twenty pages; you are about to spend a day on how it represents a number.
- Systems chapter: CS:APP 3e, by section — Ch 2 · §2.1–2.3 information storage and integer arithmetic. Leaves: Write the program that demonstrates signed overflow being undefined and unsigned wrapping being defined.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 2 · Regular expressions, text normalization, edit distance. Leaves: Implement minimum edit distance with the backtrace, not just the score.
- System (atomic — whole in the day): Write a small suite that demonstrates each integer trap: overflow, sign extension, shift beyond width, and comparison between signed and unsigned. Measure: Every case produces the documented value on your machine, and the one that differs from the standard is named.
- Open source — logged, not scheduled: nothing logged for this day
Day 34 —
ferrodb: crash injection
- Paper (AI): Neural Machine Translation of Rare Words with Subword Units — Sennrich, Haddow, Birch, 2016. BPE, and the open-vocabulary problem it was invented for. This is the tokenizer you implement tomorrow.
- Systems chapter: CS:APP 3e, by section — Ch 2 · §2.4–2.5 floating point. Leaves: Find the smallest float where x + 1 == x, predict it from the format first, then measure.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 2 · continued: byte-pair encoding. Leaves: Run BPE by hand on a ten-word corpus for five merges, and write the merge list out.
- System (step 10 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Build a crash-injection harness that kills the process at a random fsync and re-opens. Measure: The harness kills at a uniformly random fsync, and every reopen either recovers or reports a named failure class.
- Open source — logged, not scheduled: nothing logged for this day
Day 35 —
A tokenizer you wrote
- Paper (AI): SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing — Kudo, Richardson, 2018. What breaks when tokenization assumes whitespace. The reversibility requirement is the design constraint to steal.
- Systems chapter: CS:APP 3e, by section — Ch 3 · §3.1–3.4 machine-level representation, data movement. Leaves: Compile one function at -O0 and -O2 and diff the assembly; explain three differences.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 2 · continued: normalization and detokenization. Leaves: Write the round-trip test: detokenize(tokenize(s)) == s over a corpus with punctuation and Unicode.
- System (atomic — whole in the day): Implement a simplified BPE tokenizer: training the merge list, encoding, and exact detokenization. Measure: Round-trip is exact on a corpus with Unicode; vocabulary size versus mean tokens-per-word is plotted.
- Open source — logged, not scheduled: nothing logged for this day
Day 36 — · catch-up
Catch-up — rerun the last surprising experiment
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Mixed Precision Training — Micikevicius et al., 2018. Loss scaling and the master-weights copy. Read it before you profile the transformer in a few days — it explains the memory numbers you are about to see.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 37 —
Control flow, as the machine sees it
- Paper (systems): The UNIX Time-Sharing System — Ritchie, Thompson, 1974. Everything is a file, including the device. Read it for how much of an operating system fits in eleven pages.
- Systems chapter: CS:APP 3e, by section — Ch 3 · §3.5–3.6 arithmetic and control. Leaves: Take one loop, write the assembly you expect, then compile and compare — including which comparison the compiler chose.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 7 · Regularization, §7.1–7.3 parameter norm penalties, norm penalties as constrained optimization, under-constrained problems. Leaves: Show L2 regularization rescaling the parameters along the eigenvectors of the Hessian, in the notation you set up on day 14, and then say what L1 does instead that makes it sparse where L2 is not.
- System (atomic — whole in the day): Write three small functions in assembly — a loop, a branch, a switch — and verify each against a C reference on random inputs. Measure: All three agree with the C version on ten thousand random inputs.
- Open source — logged, not scheduled: nothing logged for this day
Day 38 —
ferrodb: a thousand crashes
- Paper (AI): BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding — Devlin, Chang, Lee, Toutanova, 2019. The encoder you fine-tune shortly, and the masked-LM objective that makes bidirectionality possible.
- Systems chapter: CS:APP 3e, by section — Ch 3 · §3.7 procedures and the stack. Leaves: Draw the stack frame for a three-deep call by hand, then verify it in a debugger.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 11 · Fine-tuning and masked language models. Leaves: Write why the [MASK] token creates a train/test mismatch, and what BERT does about it.
- System (step 11 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Run the crash harness a thousand times; fix whatever it finds and write down the failure class. Measure: A table of failure classes with counts, and every one either fixed or written down as a known limitation.
- Open source — logged, not scheduled: nothing logged for this day
Day 39 —
Train the small transformer, and profile it
- Paper (AI): RoBERTa: A Robustly Optimized BERT Pretraining Approach — Liu et al., 2019. A replication that found the original was undertrained. The clearest example in NLP of why you rerun the baseline yourself.
- Systems chapter: CS:APP 3e, by section — Ch 3 · §3.8–3.9 arrays and structures. Leaves: Work out the address arithmetic for a 2-D array access by hand, then check it against the generated code.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 7 · §7.4–7.8 dataset augmentation, noise robustness, semi-supervised and multi-task learning, early stopping. Leaves: Follow §7.8's derivation that early stopping is L2 regularization for a quadratic loss, then find the step at which today's training run would have stopped and state the weight decay that step count is equivalent to.
- System (atomic — whole in the day): Train the decoder-only model on a small corpus end to end; log loss, gradient norm, memory and tokens per second. Measure: A profile naming the single largest time cost, and one sentence on whether it is compute or memory bound.
- Open source — logged, not scheduled: nothing logged for this day
Day 40 —
What the optimizer can and cannot do for you
- Paper (systems): LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation — Lattner, Adve, 2004. Why the IR is the interface. Read it while you are looking at what -O2 did to your loop yesterday.
- Systems chapter: CS:APP 3e, by section — Ch 5 · §5.1–5.7 optimizing program performance. Leaves: Take one hot loop through code motion, reduction in strength and loop unrolling, timing after each.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 11 · continued: fine-tuning in practice. Leaves: Write the exact split protocol for tomorrow's fine-tune, including the seed and what is held out.
- System (atomic — whole in the day): Optimize one hot loop by hand through four versions, timing each, then compare against what the compiler does at -O3. Measure: A table of the four timings, and an explanation of the one the compiler beat you on.
- Open source — logged, not scheduled: nothing logged for this day
Day 41 —
ferrodb: checkpointing
- Paper (AI): GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding — Wang et al., 2019. What a benchmark is trying to be, and the diagnostic set most people ignore. Your evaluation script's error buckets start here.
- Systems chapter: CS:APP 3e, by section — Ch 6 · §6.1–6.4 the memory hierarchy and cache organization. Leaves: Compute the cache parameters of your own machine from the numbers, then confirm them with a stride benchmark.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 4 · Naive Bayes and sentiment classification. Leaves: Train the Naive Bayes baseline for tomorrow's task. If the encoder cannot beat it, that is the result.
- System (step 12 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Add checkpointing and WAL truncation; measure recovery time against log length. Measure: Recovery time plotted against log length before and after checkpointing, on the same hardware.
- Open source — logged, not scheduled: nothing logged for this day
Day 42 — · catch-up
Catch-up — the summer's open branches, closed
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (systems): Dynamo: Amazon's Highly Available Key-value Store — DeCandia et al., 2007. The availability argument for giving up consistency, made concretely and with the operational consequences admitted. Read it before autumn's distributed-systems term.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 43 —
Fine-tune an encoder, and report it honestly
- Paper (AI): Show Your Work: Improved Reporting of Experimental Results — Dodge et al., 2019. Why a single number with a single seed is not a result. This is where your reporting standard comes from, today, not later.
- Systems chapter: CS:APP 3e, by section — Ch 6 · §6.5–6.7 cache-friendly code. Leaves: Write the two matrix-multiply orderings and measure the cache-miss difference with a profiler.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 4 · continued: evaluation, precision, recall, F1. Leaves: Compute precision, recall and F1 by hand from a confusion matrix before your script prints them.
- System (atomic — whole in the day): Fine-tune a small encoder on a classification task with fixed splits, three seeds, and the Naive Bayes baseline alongside. Measure: Mean and spread over three seeds, against the baseline, with the seeds recorded in the README.
- Open source — logged, not scheduled: nothing logged for this day
Day 44 —
What the linker is doing to your program
- Paper (systems): How To Write Shared Libraries — Drepper, 2011. ELF segments, relocation, the GOT and PLT, from the person who maintained the loader. Sections 1–2 today.
- Systems chapter: CS:APP 3e, by section — Ch 7 · Linking, §7.1–7.7. Leaves: Produce a duplicate-symbol error and a missing-symbol error deliberately, and read what the linker actually said.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 14 · Question answering and information retrieval. Leaves: Implement BM25 and score five queries by hand against your own toy corpus.
- System (atomic — whole in the day): Build the same program statically and dynamically; inspect both with a symbol dumper and trace the dynamic loader. Measure: Binary sizes, symbol counts, and the first library call's resolution observed in the trace.
- Open source — logged, not scheduled: nothing logged for this day
Day 45 —
ferrodb: iterators and snapshots
- Paper (AI): Sequence to Sequence Learning with Neural Networks — Sutskever, Vinyals, Le, 2014. Encoder-decoder in its first form, fixed-length bottleneck and all. The thing the attention paper on day 27 was fixing.
- Systems chapter: OSTEP, read in chapter order — Ch 42 · Crash consistency: FSCK and journaling. Leaves: Write the three journal writes in order with the crash point between each pair, and say which of them leaves a state fsck can only repair by guessing.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 14 · continued: dense retrieval. Leaves: State what a dual encoder cannot express that a cross encoder can, with an example query.
- System (step 13 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement iterators: range scan, reverse scan, and a snapshot that does not see later writes. Measure: A concurrent test where the snapshot is opened, writes land, and the scan still returns the old view.
- Open source — logged, not scheduled: nothing logged for this day
Day 46 —
A retrieval pipeline, with the errors separated
- Paper (AI): Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer — Raffel et al., 2020. T5, and a systematic ablation across a whole design space. Read it as an experiment design, not as an architecture.
- Systems chapter: OSTEP, read in chapter order — Ch 43 · Log-structured file systems. Leaves: Compute the cleaner's cost for a segment that is 70 percent live, then set it beside ferrodb's WAL truncation and name the one job LFS has that ferrodb does not.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 14 · continued: evaluating retrieval. Leaves: Compute recall@k and MRR by hand for one query set before your script does.
- System (atomic — whole in the day): Build a dense-retrieval pipeline over a small corpus: encode, index, retrieve, then a reader on top. Measure: Retrieval error and reader error attributed separately, with the counts of each.
- Open source — logged, not scheduled: nothing logged for this day
Day 47 —
Flash, and the write amplification underneath everything
- Paper (systems): A Unified Theory of Garbage Collection — Bacon, Cheng, Rajan, 2004. Tracing and reference counting are duals. Read it as an argument that two things you thought were different are one thing.
- Systems chapter: OSTEP, read in chapter order — Ch 44 · Flash-based SSDs. Leaves: Compute the write amplification of a page-mapped FTL for a random 4 KB workload on a 256 KB erase block, and say what changes in ferrodb's fsync argument on a device with no seek time.
- AI chapter: Jurafsky & Martin, Speech and Language Processing 3e — Ch 14 · continued: retrieval-augmented generation. Leaves: Write the three failure modes a RAG system has that a generator alone does not.
- System (atomic — whole in the day): Simulate a flash device with erase blocks, a page-mapped FTL and a garbage collector, then drive it with a sequential and a random workload. Measure: Write amplification for both workloads, and the over-provisioning fraction at which the random number stops getting worse.
- Open source — logged, not scheduled: nothing logged for this day
Day 48 — · catch-up
Catch-up — the two repositories, made presentable
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Scaling Laws for Neural Language Models — Kaplan et al., 2020. The empirical claim that made the last five years look inevitable. Read the fit methodology as carefully as the conclusion.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 49 —
ferrodb: MVCC
- Paper (AI): Dense Passage Retrieval for Open-Domain Question Answering — Karpukhin et al., 2020. The dual encoder your pipeline is a version of, plus the in-batch negatives trick that makes it trainable.
- Systems chapter: OSTEP, read in chapter order — Ch 45 · Data integrity and protection. Leaves: Add a checksum to ferrodb's page header and decide in writing which layer verifies it — the end-to-end argument from day 11, applied — then produce a misdirected write and show the checksum catching it.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 7 · §7.9–7.11 parameter tying and sharing, sparse representations, bagging and other ensembles. Leaves: Write a convolution out as a dense layer with tied, sparse weights — §7.9's argument, made concrete — as an explicit matrix for one small input, and count the parameters the tying removed.
- System (step 14 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Add MVCC: per-key version chains and a read timestamp; test with concurrent readers. Measure: A test with readers at three timestamps against a live writer, each seeing exactly its own snapshot.
- Open source — logged, not scheduled: nothing logged for this day
Day 50 —
RAG, audited
- Paper (AI): Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020. Where retrieval error and generation error separate — the audit you did on day 46, done first here.
- Systems chapter: OSTEP, read in chapter order — Ch 48 · Distributed systems: an introduction. Leaves: Write ferrodb's failure model down — what can fail, what you assume cannot, and what the client is told in each case — and mark the assumption a network makes false.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 7 · §7.12–7.14 dropout, adversarial training, and tangent propagation. Leaves: Write why dropout is not exactly an ensemble, using the weight-scaling inference rule §7.12 gives, and construct one small model where that approximation is visibly wrong.
- System (atomic — whole in the day): Take your day-46 pipeline and label fifty failures by cause: retrieval miss, retrieval rank, reader hallucination, or ambiguity. Measure: A count per bucket, and the single change that would fix the largest one.
- Open source — logged, not scheduled: nothing logged for this day
Day 51 —
Attention is memory-bound
- Paper (systems): Scalable Parallel Programming with CUDA — Nickolls, Buck, Garland, Skadron, 2008. The execution model — threads, blocks, warps, shared memory — as its designers explain it. Read it before the first kernel, which is next summer.
- Systems chapter: CS:APP 3e, by section — Ch 6 · §6.6 revisited: the memory mountain. Leaves: Reproduce the memory mountain on your own machine and mark where each cache level ends.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 8 · Optimization for training deep models, §8.1–8.2 how learning differs from pure optimization, and the challenges. Leaves: List §8.2's failure modes — ill-conditioning, local minima, plateaus and saddle points, cliffs, vanishing and exploding gradients, inexact gradients — and write the one diagnostic that tells you it is that one. This is the checklist you use for the next two years, so it has to be yours.
- System (atomic — whole in the day): Measure memory and time against sequence length for your decoder, then implement one fused operation and rerun. Measure: The measured scaling exponent against the predicted one, and the fusion's effect on both.
- Open source — logged, not scheduled: nothing logged for this day
Day 52 —
ferrodb: the lock manager
- Paper (AI): FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Dao, Fu, Ermon, Rudra, Ré, 2022. The systems thread and the AI thread meeting: attention is memory-bound, and the fix is tiling, not approximation. You measured this yesterday.
- Systems chapter: OSTEP, read in chapter order — Ch 49 · Sun's Network File System. Leaves: Trace one client write through a server crash and back, and name the property of each request that made the blind retry safe.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 8 · §8.3 basic algorithms: SGD, momentum, and Nesterov momentum. Leaves: Derive momentum's effective learning rate as a function of the decay coefficient, then run plain SGD, momentum and Nesterov on the same quadratic and check the measured speedup against what the derivation predicted.
- System (step 15 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Implement a simple lock manager and a write path that detects write-write conflicts. Measure: Two concurrent writers to one key: exactly one commits, and the loser is told why.
- Open source — logged, not scheduled: nothing logged for this day
Day 53 —
The same result from the algebra instead of the hardware
- Paper (AI): Self-attention Does Not Need O(n²) Memory — Rabe, Staats, 2021. The same result as yesterday's paper argued from the algebra rather than the hardware. Say in the card which framing you find more useful, and why.
- Systems chapter: OSTEP, read in chapter order — Ch 50 · The Andrew File System. Leaves: Compare AFS's whole-file caching and callbacks against NFS's per-block protocol on one workload of many small reads, and say which of the two ferrodb's buffer pool resembles.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 8 · §8.4 parameter initialization strategies. Leaves: Re-derive the Glorot and He variances from §8.4 rather than quoting them, mark the assumption each one makes about the nonlinearity, and say which of the two your day-14 ablation actually supported once the seed noise was taken out.
- System (atomic — whole in the day): Implement the streaming (online) softmax and use it for a memory-bounded attention pass. Measure: Output matches the naive implementation to 1e-5, at measured constant memory in sequence length.
- Open source — logged, not scheduled: nothing logged for this day
Day 54 — · catch-up
Catch-up — the first lab emails, sent
No new chapter, no new system, no new increment. One paper, and whatever slipped: finish the build that ran long, re-run the experiment that surprised you, or write the note you skipped.
- Paper (AI): Training Compute-Optimal Large Language Models — Hoffmann et al., 2022. Chinchilla, which corrected the paper you read yesterday by rerunning it properly. Two papers, one lesson about baselines.
- Systems chapter: nothing new scheduled on a catch-up day
- AI chapter: nothing new scheduled on a catch-up day
- System: nothing new scheduled on a catch-up day
- Open source — logged, not scheduled: nothing logged for this day
Day 55 —
Group commit, and the last measurement
- Paper (systems): The Google File System — Ghemawat, Gobioff, Leung, 2003. A file system designed around the failure rate of its hardware and the shape of one workload. The clearest example of assumptions stated up front.
- Systems chapter: OSTEP, read in chapter order — Ch 53 · Introduction to security. Leaves: Write ferrodb's threat model in one paragraph — who the attacker is, what they can reach, what you are not defending — and mark the one place the code currently trusts bytes it did not write.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 8 · §8.5–8.6 algorithms with adaptive learning rates, and approximate second-order methods. Leaves: Write Adam out as a diagonal preconditioner on the Hessian you computed on day 14, and say what that framing predicts about the case where it loses to plain SGD — then find that case in your own logs or admit you have not hit it.
- System (step 16 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Add group commit; measure the throughput change and explain it from the fsync count. Measure: Throughput against batch size, with the fsync count per commit, and the explanation tied to the two numbers.
- Open source — logged, not scheduled: nothing logged for this day
Day 56 —
The benchmark that goes in the README
- Paper (AI): Distilling the Knowledge in a Neural Network — Hinton, Vinyals, Dean, 2015. Soft targets carry more than the label does. Read it as the first serious answer to "make it smaller without making it worse".
- Systems chapter: OSTEP, read in chapter order — Ch 54 · Authentication. Leaves: Work out why a password file stores a salted hash rather than a hash, with the cost of the offline attack on each written as a number for a ten-million-password list.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 8 · §8.7 optimization strategies and meta-algorithms: batch normalization, coordinate descent, Polyak averaging, curriculum. Leaves: Write the two competing explanations §8.7 gives for why batch normalization helps, then say which one this summer's ablations support and which two of them your setup simply cannot distinguish between.
- System (step 17 of 20 on ferrodb — crash-recoverable storage engine (Rust)): Benchmark: random and sequential read/write, varying value size; publish a table. Measure: The table, with the machine, the fsync policy and the value sizes stated next to it.
- Open source — logged, not scheduled: nothing logged for this day
Day 57 —
Summer closes: two repositories someone else could run
- Paper (AI): LoRA: Low-Rank Adaptation of Large Language Models — Hu et al., 2021. The parameter-efficient method you will actually use during term, when the compute budget is a laptop and an hour.
- Systems chapter: CS:APP 3e, by section — Ch 8 · §8.1–8.3 exceptional control flow. Leaves: Write the signal-handling program that demonstrates a race with the handler, then fix it correctly.
- AI chapter: Goodfellow, Bengio & Courville, Deep Learning — Ch 9 · Convolutional networks, §9.1–9.2 the convolution operation, and the motivation. Leaves: Name the three properties §9.2 says convolution buys — sparse interactions, parameter sharing, equivariant representations — and give one case for each where it is the wrong assumption about the data.
- System (atomic — whole in the day): Write the README for ferrodb and for the transformer repo: interface, invariants, one-command run, benchmark, known limits, and your own contribution stated. Measure: A clean clone on a second machine runs both from the README alone, with nothing added.
- Open source — logged, not scheduled: nothing logged for this day