“Adaptive” is one of those words that flatters ambition. A thermostat is adaptive. A recommender system is adaptive. A system that revises its own objective based on human feedback is adaptive. Calling all three by the same word hides the fact that they pose different engineering problems and carry different stakes.
Three senses, ordered by what the system is actually allowed to change.
1. Parameter-level adaptivity
The system adjusts numerical parameters inside a fixed functional form. Control gains. Learning rates. Filter coefficients. Recommendation weights. The temperature of a softmax.
The behaviour class is fixed. The operating point moves.
Canonical territory: adaptive control, adaptive filtering, online learning. The engineering discipline here is stability and convergence under drift.
This is the most common — and the most conservative — sense.
2. Behavioural adaptivity
The system selects among qualitatively different behaviours based on context. Mode switching. Policy selection. Regime-dependent strategies.
The system changes what kind of thing it is doing, not only how.
Canonical territory: hybrid systems, hierarchical reinforcement learning, mixture-of-experts, bandit algorithms that switch arms.
The engineering problem shifts. The question is no longer “does the parameter converge” but “is the mode choice legible, recoverable, and appropriate”.
3. Goal-level adaptivity
The system revises its own objective, based on feedback about whether the prior objective was worth pursuing.
Value learning. Preference revision. Goal inference. Assistance games.
This is the rarest sense in deployed systems and the most epistemically loaded. A parameter-level adaptive system asks, am I tracking the target. A goal-level adaptive system asks, is the target the right one.
Why the distinction matters
Much of the ML and HCI literature uses “adaptive” without naming which sense is in play. A paper describing a parameter-tuning pipeline reaches for the same word as a paper describing a system that revises what it is trying to do. The design decisions, the evaluation criteria, and the risks do not move in parallel across the three senses. The word does.
A modest claim: before evaluating an adaptive system, it is worth asking which of the three it occupies. The sense determines the appropriate evaluation and the appropriate standard of care.
What this concept does not cover
- It is not a taxonomy of all adaptive systems. Other cross-cuts — temporal horizon, observability, online versus offline, single versus multi-agent — are orthogonal to this one.
- It is not a claim that one sense is better than another.
- It is not about affect-adaptive systems specifically.
- It is not about whether current systems should be doing goal-level adaptation. That is a separate question.
Sources
- Karl J. Åström and Björn Wittenmark. Adaptive Control. 2nd ed., Addison-Wesley, 1995. ISBN 978-0-201-55866-1. Google Books.
- Bernard Widrow and Samuel D. Stearns. Adaptive Signal Processing. Prentice-Hall, 1985. ISBN 978-0-13-004029-9. Google Books.
- Simon Haykin. Adaptive Filter Theory. 5th ed., Pearson, 2014. ISBN 978-0-13-267145-3. Publisher page.
- Andrew G. Barto and Sridhar Mahadevan. “Recent Advances in Hierarchical Reinforcement Learning.” Discrete Event Dynamic Systems 13(4):341–379, October 2003. DOI: 10.1023/A:1025696116075.
- Stuart Russell. Human Compatible: Artificial Intelligence and the Problem of Control. Viking, 2019. ISBN 978-0-525-55861-3. Publisher page.
- Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell. “Cooperative Inverse Reinforcement Learning.” NeurIPS, 2016. NeurIPS proceedings; arXiv:1606.03137.