A glossary of tiny lectures, part 7

Week 7: applications

By Tim Menzies, published 2026-10-26, updated 2026-10-26

Summary: Textbooks give a chapter each to prediction, anomaly detection, Bayes classifiers and clustering. This week is the claim that those chapters are synonyms: given distance and column summaries, each one is a dozen lines. Count the new lines per chapter; that falling count is the whole argument of this course, and of the undergraduate project. Demos in ezr-eg7.lua; machinery verbatim from ezr-apps.lua.

One substrate, four chapters

New acronyms: knn, NB (naive Bayes), kpp (k-means++).

knn: The Fortune Teller. To guess a row's goals, sort everything by distance to it and average the goals of the nearest the.k. Nothing is fitted; the neighbours ARE the model. It is also the cheapest possible regression — and on auto93 it beats guessing the global mean by about four times.
```
function TBL.knn(i,row,k,    t)
  t = i:around(row)
  return adds(map(slice(t, 1, k or the.k), i:Y())).mu end
```
anomaly detection: The Bouncer. Loneliness is the whole test: how far is the nearest OTHER row? Score every row that way, normalize to 0..1, and 1 is the loneliest thing in the data. Wrap this around any learner and you have the certification envelope: the guess, plus a red light when the question is unlike anything the model was built from. The same code, pointed at a reference population instead of your own history, is BENCHMARKING — is this car yard near the good cluster, or far from every cluster? (synonyms, again.)
```
function TBL.anomaly(i,    dn,gap)
  gap = function(r,    lo,d)
    lo = 1e32
    for _,z in ipairs(i.rows) do
      if z ~= r then
        d = i:distx(r, z); if d < lo then lo = d end end end
    return lo end
  dn = Num()
  for _,r in ipairs(i.rows) do dn:add(gap(r)) end
  return function(r) return dn:norm(gap(r)) end end
```
naive bayes: The ER Nurse: sort arrivals into classes, fast, one row at a time. Each class keeps its own column summaries; the score for a row is the log of the class prior plus the logs of P(value | column), summed over the x columns. "Naive" means the columns are assumed independent — plainly false, and it wins anyway, because a wrong-but-consistent bias still ranks the classes correctly. Note the protocol at work: symbols get an m-estimate, numbers get a gaussian pdf, and likes never asks which is which.
```
function like(col,v,prior,    z)
  if not col.mu then
    return ((col.has[v] or 0) + the.L*prior)
           / (col.n + the.L) end
  z = 2 * col:div()^2 + 1e-32
  return exp(-(v - col.mu)^2 / z) / (pi * z)^0.5 end
```
The rig is test-then-train: every row is GUESSED before it is learned, so the accuracy needs no held-out split and the model is always as current as the last row seen.
```
function TBL.classify(i,wait,    at,h,seen,nh,want)
  wait, at = wait or the.wait, i.cols.klass.at
  h, seen, nh = {}, {}, 0
  for j,row in ipairs(i.rows) do
    want = row[at]
    if j >= wait and nh > 0 then
      push(seen, {mostlikes(h, row, #i.rows, nh), want}) end
    if not h[want] then h[want] = i:clone(); nh = nh + 1 end
    h[want]:add(row) end
  return seen end
```
kmeans: The Curator, textbook edition: pick k centroids, assign every row to its nearest, move each centroid to the middle of what it caught, repeat. Compare with week 3's node: kmeans needs k, needs a full pass per iteration, and needs a distance to every centroid — while node splits on two rows and gives you an index for free. Both cluster; they do not cost the same.
```
function TBL.kmeans(i,k,iter,    cents)
  cents = some(i.rows, k or the.kluster)
  for _ = 1, iter or the.iter do
    cents = recentre(assign(i, cents)) end
  return assign(i, cents) end
```
kpp: k-means++: start the centroids far apart, or kmeans starts in a corner and stays there. Each new centre is drawn with a chance proportional to its SQUARED distance from the centres chosen so far — random, but biased toward the empty regions. Week 3's poles made the same bet with two rows and no randomness.
```
function TBL.kpp(i,k,    cents,pool,ws)
  cents = {some(i.rows, 1)[1]}
  while #cents < (k or the.kluster) do
    pool = some(i.rows, min(the.few, #i.rows))
    ws   = map(pool, function(r) return d2(i, cents, r) end)
    push(cents, pool[wpick(ws)]) end
  return cents end
```
Aha, Kibler & Albert: Instance-based learning algorithms. David W. Aha, Dennis Kibler, Marc K. Albert. Machine Learning 6, 1991, 37-66.

Domingos & Pazzani: On the optimality of the simple Bayesian classifier under zero-one loss. Pedro Domingos, Michael Pazzani. Machine Learning 29, 1997, 103-130.

Arthur & Vassilvitskii: k-means++: The advantages of careful seeding. David Arthur, Sergei Vassilvitskii. SODA 2007, 1027-1035.