# DDIA Ch. 12: one stream, one "5-minute" width, four different groupings ## Concept The book's two example events arrive at **10:03:39** and **10:08:12** — 4 minutes 33 seconds apart, so strictly *less than* 5 minutes. Feed that same pair through four operators that all call themselves "5-minute windows" and they are grouped **four different ways**: - a **sliding** window (anchored at each event, looking back 5 min) groups them **TOGETHER** — one window holds both, count 2; - a **tumbling** window (fixed buckets on a :00/:05/:10 grid) **SPLITS** them — 10:03:39 lands in `[10:00, 10:05)`, 10:08:12 in `[10:05, 10:10)`, count 1 + 1; - a **hopping** window (5-min size, advancing by a 1-min hop) puts each event in its own set of 5 overlapping windows, and the two sets **share none**; - a **session** window (30-min inactivity gap) groups them **TOGETHER** by activity — same burst, not same clock bucket. Same stream, same 5-minute width, different count and different grouping every time. "5 minutes" fixes the window's *size* but not its *boundaries*, its *overlap rule*, or whether boundaries exist at all. The grouping belongs to the operator, not the duration. ## Provenance *Designing Data-Intensive Applications*, 2nd ed. (Kleppmann & Riccomini, 2025), Chapter 12 — *Stream Processing*, §"Types of windows". The book walks through tumbling, hopping, sliding, and session windows and uses two events at 10:03:39 and 10:08:12 to illustrate how a tumbling window can put nearby events in different buckets while a sliding window keeps them together: > A tumbling window has a fixed length, and every event belongs to exactly one > window. […] A sliding window […] contains all the events that occur within > some interval of each other. This exercise runs those exact two timestamps through all four window operators and prints the grouping each one produces. ## Prerequisites The surprise is that a window's *type*, not its *size*, decides the grouping. These help; each line notes where it shows up. 1. **Event-time windows** — a window groups events by *when they happened* (their event-time timestamp), not when they were processed. It shows up as every operator here bucketing the fixed `EVENTS` list by its timestamps. 2. **Tumbling vs. hopping vs. sliding vs. session** — four ways to draw window boundaries: non-overlapping grid buckets; overlapping grid buckets; one window per event; or "whatever a burst of activity spans." Each is a different `run_*` function over the same list. 3. **Window size vs. hop vs. gap** — *size* is the width (5 min for all four); *hop* is how far a hopping window advances (1 min); *gap* is the silence that closes a session (30 min). It shows up as the `SIZE`, `HOP`, and `GAP` constants. ### Where to learn the prerequisites - **Window types (#1–#2):** DDIA §"Types of windows"; the Apache Flink docs on [Windows](https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/datastream/operators/windows/) are the clearest free reference — tumbling, sliding, and session windows with pictures. - **Size vs. hop vs. gap (#3):** the same Flink page names them (`size`/`slide`/`gap`); Google's Dataflow model paper (Akidau et al., 2015, "The Dataflow Model") is the free primary source for windows as a first-class concept. **The one that carries the result: #2.** "Window TYPE, not just size" is the whole exercise — the same 5-minute size produces four different answers purely because the four operators draw boundaries differently. ## Environment Deterministic, so the groupings reproduce exactly: - macOS 26.5.2 (Darwin 25.5.0), arm64 - Python 3.15.0a8, standard library only — no Docker - Fixed event list, deterministic — the counts are the same every run ## Mental model Four operators, same stream, same 5-minute size. Each answers a *different question* about the stream, so each draws boundaries differently. | Operator | Boundaries | A window is… | The question it answers | | --- | --- | --- | --- | | **Tumbling** | fixed grid, no overlap (`…:00, :05, :10`) | one bucket per event | "how many events in *this* clock bucket?" | | **Hopping** | fixed grid, advances by a small **hop** (overlap) | `size / hop` overlapping buckets per event | "how many events in *any* 5-min span, sampled every minute?" | | **Sliding** | relative to each event, looking back `size` | one window per event | "how many events happened within 5 min *of this one*?" | | **Session** | no grid — a **gap** of inactivity closes it | whatever a burst spans | "how many events in *this burst of activity*?" | Tumbling and hopping snap to a global clock grid; sliding is relative to each event; session ignores the clock entirely and follows the data. The whole thing is one script, `code/window_types.py`, running all four over the identical list. ## Setup ``` cd study/ddia/ch12/code python3 window_types.py ``` Do **not** read the output yet — make each prediction first. The two events under the microscope are `e1` @ 10:03:39 and `e2` @ 10:08:12 (4m33s apart). Events `e3`–`e5` add a burst-then-pause tail so the session window has something to group. ## Steps ### Step 1 — sliding: a window per event, looking back **Predict.** A sliding window anchors a 5-minute window at each event and looks back. The window anchored at `e2` (10:08:12) covers the previous 5 minutes. Does `e1` (10:03:39) fall inside it — are the two events grouped together? **Observe.** ``` SLIDING -- a 5-min window anchored at each event, looking back window for e1 = (09:58:39, 10:03:39]: e1 (count 1) window for e2 = (10:03:12, 10:08:12]: e1, e2 (count 2) window for e3 = (10:05:00, 10:10:00]: e2, e3 (count 2) window for e4 = (10:40:00, 10:45:00]: e4 (count 1) window for e5 = (10:42:30, 10:47:30]: e4, e5 (count 2) window anchored at e2 (10:08:12) looks back to 10:03:12 e1 @ 10:03:39 falls inside it -> e1 and e2 TOGETHER (count 2). ``` **TOGETHER, count 2.** The window ending at 10:08:12 reaches back to 10:03:12, and 10:03:39 is after that — so both events co-occur. Because the window is *relative to the event*, any two events within 5 minutes of each other land together. ### Step 2 — tumbling: fixed buckets on a clock grid **Predict.** A tumbling window chops time into fixed non-overlapping 5-minute buckets aligned to `…:00, :05, :10`. Same two events — same 4m33s gap. Which bucket does each land in? Together or split? **Observe.** ``` TUMBLING -- fixed 5-min buckets on a :00 / :05 / :10 grid (no overlap) e1 @ 10:03:39 -> bucket [10:00, 10:05) e2 @ 10:08:12 -> bucket [10:05, 10:10) e3 @ 10:10:00 -> bucket [10:10, 10:15) e4 @ 10:45:00 -> bucket [10:45, 10:50) e5 @ 10:47:30 -> bucket [10:45, 10:50) bucket [10:00, 10:05): e1 (count 1) bucket [10:05, 10:10): e2 (count 1) bucket [10:10, 10:15): e3 (count 1) bucket [10:45, 10:50): e4, e5 (count 2) e1 in [10:00, 10:05) e2 in [10:05, 10:10) -> e1 and e2 are SPLIT into DIFFERENT buckets (count 1 + count 1). ``` **SPLIT — count 1 + 1.** The 10:05 grid line falls *between* the two events, so they straddle a boundary and land in different buckets. The gap between them (4m33s) is smaller than the window (5 min), yet the fixed grid still separates them — because alignment, not distance, decides a tumbling bucket. ### Step 3 — hopping: overlapping buckets on the grid **Predict.** A hopping window is 5 minutes wide but starts a new window every 1 minute, so windows overlap and each event belongs to five of them at once. The two events are less than 5 minutes apart — so *some* overlapping 5-minute window must contain both, right? **Observe.** ``` HOPPING -- 5-min windows, new one every 1 min (they OVERLAP) e1 @ 10:03:39 -> 5 windows: [09:59, 10:04), [10:00, 10:05), [10:01, 10:06), [10:02, 10:07), [10:03, 10:08) e2 @ 10:08:12 -> 5 windows: [10:04, 10:09), [10:05, 10:10), [10:06, 10:11), [10:07, 10:12), [10:08, 10:13) shared windows: NONE -> e1 and e2 land in overlapping-but-DIFFERENT sets of windows; no single 5-min window on the hop grid straddles the pair. ``` **Disjoint — they share no window.** For a single 5-min window to hold both, its start would have to fall in `(10:03:12, 10:03:39]` — but the hop grid only offers starts on the minute (…10:03:00, 10:04:00…), and none land in that 27-second slot. So even overlapping windows, snapped to a grid, separate the pair. The last window holding `e1` is `[10:03, 10:08)` (ends *just before* 10:08:12); the first holding `e2` is `[10:04, 10:09)` (starts *just after* 10:03:39). ### Step 4 — session: grouped by a gap, not a clock **Predict.** A session window has no grid at all: it keeps grouping events until a gap of more than 30 minutes of silence, then starts a new session. The stream is a burst (10:03–10:10), then 35 minutes of silence, then a second burst (10:45–10:47). How many sessions, and where do `e1` and `e2` land? **Observe.** ``` SESSION -- grouped by a 30-min inactivity gap (clock ignored) e1 @ 10:03:39 e2 @ 10:08:12 (gap 273s = <= 30m: same session) e3 @ 10:10:00 (gap 108s = <= 30m: same session) e4 @ 10:45:00 (gap 2100s = > 30m: NEW session) e5 @ 10:47:30 (gap 150s = <= 30m: same session) session 1 [10:03:39-10:10:00]: e1, e2, e3 (count 3) session 2 [10:45:00-10:47:30]: e4, e5 (count 2) -> e1 and e2 are TOGETHER in session 1 -- grouped by activity, not by any 5-minute clock boundary. ``` **Two sessions; `e1` and `e2` TOGETHER in session 1.** No 5-minute boundary appears anywhere — the only thing that closes a window is the 35-minute silence (2100s > 30 min) before `e4`. The grouping tracks the *shape of the activity*, so a session can be far longer (here 6m21s and 2m30s) or shorter than 5 minutes. ## What you should see The same pair, `e1` @ 10:03:39 and `e2` @ 10:08:12, grouped four ways: - **Sliding:** TOGETHER — the window at `e2` holds both, count 2. - **Tumbling:** SPLIT — `[10:00, 10:05)` vs. `[10:05, 10:10)`, count 1 + 1. - **Hopping:** disjoint — each in 5 overlapping windows, sharing none. - **Session:** TOGETHER — both in session 1 (count 3), closed only by the 30-min gap. ## Why "5 minutes" is a length, and a length alone does not group anything. To turn a duration into a window you have to answer three more questions the word "5 minutes" leaves open: *where do the boundaries go, do windows overlap, and are boundaries drawn on the clock or on the data?* Each operator answers those differently, and the answer — not the 5 minutes — is what decides whether two events share a window. Tumbling and hopping both snap boundaries to a **global grid**. A grid line is placed without any reference to the events; it just falls where `…:05` falls. Two events 4m33s apart are closer than the 5-minute width, but if a grid line lands *between* them they are separated anyway. That is exactly what happens: the 10:05 line sits between 10:03:39 and 10:08:12, so tumbling splits them, and for hopping, no minute-aligned 5-minute window happens to start in the 27-second slot `(10:03:12, 10:03:39]` that would be needed to cover both. The separation is a property of *where the grid falls relative to the events*, which the size never controls. Sliding removes the grid: it anchors the window **at each event** and looks back `size`. Now "within 5 minutes of each other" is the literal test, so any two events closer than the width co-occur — 10:03:39 is inside `(10:03:12, 10:08:12]`, so `e1` and `e2` are together, guaranteed, regardless of where a clock grid would have fallen. Session goes further and ignores the clock *and* the size entirely: it grows a window for as long as events keep arriving within `gap` of each other, so the boundary is drawn by the data's own silence. Same two events, same 4m33s, four operators — and the grouping is whatever the operator's boundary rule says, never what "5 minutes" says. ### The boundary — where the effect vanishes Events **farther apart than the window size** are never grouped by tumbling, hopping, or sliding — no 5-minute window can hold two events more than 5 minutes apart, so for those three the size *is* a hard ceiling on togetherness. The divergence in this exercise lives entirely in the band *within* `size`: events closer than 5 minutes **may or may not** share a window, and *which* depends only on boundary alignment — the grid position for tumbling/hopping, the anchor for sliding. Push the pair past 5 minutes apart and tumbling, hopping, and sliding all agree ("apart"); only session can still group them, because its ceiling is the 30-minute gap, not the 5-minute size. The whole surprise is that alignment, not duration, is the free variable. ### Go deeper 1. **Watermarks and triggers per window type.** A window has to decide *when it is done* and can emit a result. Add late events (event-time before the current watermark) and see why tumbling can fire once per bucket at the watermark, but a session window can't fire until it's sure no event will extend the burst. 2. **Why session windows need a timeout.** A session has no predetermined end — only the `gap` closes it. Shrink `GAP` toward the inter-event spacing and watch session 1 fracture into more, smaller sessions; grow it past the 35-min silence and the two bursts merge into one. Unlike the grid windows, a session can't be emitted until `gap` of silence has actually elapsed. 3. **Move the grid.** Offset the tumbling/hopping grid by 2m30s (`GRID = t( "10:02:30")`) and watch `e1` and `e2` suddenly land in the *same* tumbling bucket — proof that alignment, not size, carried the split.