How to Think About Autonomous Systems: A Practitioner's Guide
How to Think About Autonomous Systems: A Practitioner’s Guide
The Problem With “Autonomous”
The word “autonomous” gets used loosely. In AI discourse, it means anything from a chatbot that generates text without human intervention to a self-driving car making real-time decisions about whether to brake. In creative practice, it might mean a generative art system that produces novel outputs, or a publishing pipeline that distributes content without manual approval. The word covers too much ground to be useful without qualification.
Here’s a more useful framing: an autonomous system is one where the coordination logic is encoded, not improvised. A human still designs the system, sets its constraints, and reviews its outputs. But the system decides when to act, what to act on, and how to route work between components — without a human making those decisions in real time.
This is the distinction between playing an instrument and composing for an orchestra. The instrumentalist makes decisions note by note. The composer encodes decisions into a score, and the orchestra executes them. The composer is still the author; the autonomy is in the execution layer, not the creative intent.
The eight-organ system is an autonomous system in this specific sense. It has 97 repositories across 8 organizations, connected by dependency edges, governed by promotion criteria, monitored by automated audits, and coordinated by orchestration workflows. No human manually triggers the weekly audit or decides which repos to evaluate for promotion. The system does that. But a human designed every rule it follows.
Five Mental Models
After five years of building and operating this kind of system, I’ve arrived at five mental models that I use repeatedly. They’re not theoretical — they emerged from debugging real failures and designing real solutions.
1. The Dependency Graph Is the Architecture
When you have more than a dozen interacting components, the dependency graph is the system’s architecture. Not the org chart, not the README hierarchy, not the directory structure — the graph of what depends on what.
In the eight-organ system, the dependency graph has 31 validated edges. ORGAN-I (Theory) feeds ORGAN-II (Art). ORGAN-II feeds ORGAN-III (Commerce). ORGAN-IV (Orchestration) observes all organs. ORGAN-V (Public Process) documents all organs. These aren’t suggestions — they’re enforced constraints.
The critical rule: no back-edges. ORGAN-III cannot depend on ORGAN-II. ORGAN-II cannot depend on ORGAN-III. Information flows in one direction. This prevents circular dependencies, which in autonomous systems produce oscillation: A triggers B, B triggers A, infinite loop.
If you’re building an autonomous system, draw the dependency graph first. Then ask: are there cycles? If yes, break them. A directed acyclic graph (DAG) is not a theoretical nicety — it’s a prerequisite for reliable automation. Every CI/CD pipeline, every build system, every package manager enforces this constraint because the alternative is non-termination.
2. State Machines Over Ad Hoc Decisions
Every entity in an autonomous system should have an explicit state, and every transition between states should have explicit criteria.
The eight-organ system uses two state machines:
- Implementation status: DESIGN_ONLY → SKELETON → PROTOTYPE → ACTIVE (→ ARCHIVED)
- Promotion status: LOCAL → CANDIDATE → PUBLIC_PROCESS → GRADUATED → ARCHIVED
Each transition has documented criteria. A repo can’t move from SKELETON to PROTOTYPE without tests. A repo can’t move from LOCAL to CANDIDATE without documentation. These criteria are encoded in governance-rules.json and enforced by the promote-repo.yml workflow.
The alternative — making promotion decisions ad hoc — works when you have 5 repos. It breaks at 20. At 97, it’s impossible. The state machine scales because the rules don’t change with the number of entities. Adding the 98th repo doesn’t require rethinking the governance model; it just adds another entity to the state machine.
Practical advice: If you find yourself making the same kind of decision repeatedly about different entities (“Is this repo ready? Is that one?”), you need a state machine. Define the states, define the transitions, define the criteria. Then let the machine decide.
3. Constraints Generate, They Don’t Restrict
This is counterintuitive but essential: in autonomous systems, constraints are generative. The more precisely you define what the system cannot do, the more reliably it does what it should do.
The eight-organ system has exactly three types of constraints:
- Structural constraints: dependency edges (what can flow where)
- Quality constraints: promotion criteria (what must be true before a transition)
- Governance constraints: rules about who can change what (CODEOWNERS, branch protection)
None of these prevent creative work. They channel it. A repo can do anything within its organ’s domain. ORGAN-II repos can be generative art, interactive theater, music composition, game design — the constraint is that they must be art, not commerce. This boundary actually helps: you don’t waste time wondering whether a game should be monetized (that’s ORGAN-III’s problem) or whether a composition needs a theoretical framework (that’s ORGAN-I’s job).
In multi-agent AI systems, the same principle applies. An agent with unbounded capability and no constraints doesn’t produce better output — it produces incoherent output. Define the agent’s domain, its tools, its stopping conditions, and its output format. The constraints are what make the agent useful.
4. Observability Is Not Optional
An autonomous system you can’t observe is an autonomous system you can’t trust. And a system you can’t trust is one you’ll eventually override manually, which defeats the purpose.
The eight-organ system has four observability layers:
- Registry:
registry-v2.jsonrecords the state of every entity - Audit workflows: automated weekly checks that detect drift (missing files, broken CI, stale deps)
- Metrics pipeline:
calculate-metrics.py→system-metrics.jsoncomputes system-wide metrics from source data - Essay pipeline: ORGAN-V essays document decisions, rationale, and lessons learned — human-readable observability
The key insight is that observability must be automated. A dashboard that requires someone to run a script and read the output is observability theater. The audit workflow runs every Monday at 06:30 UTC whether anyone remembers to check or not. The metrics pipeline recomputes from source data, so the numbers can’t drift from reality.
Practical advice: For every automated action in your system, there should be an automated check that verifies the action happened correctly. And the check should run on a schedule, not on demand. If it only runs when someone remembers, it won’t run when it matters most.
5. The Human Is the Appellate Court, Not the Trial Court
In legal systems, trial courts hear cases first. Appellate courts only hear appeals — cases where the trial court’s decision is contested. This is the right model for human oversight of autonomous systems.
The system makes the routine decisions: which repos need attention, which meet promotion criteria, which have failing CI. The human reviews the system’s decisions and intervenes only when the automated judgment is wrong or when the situation is genuinely novel.
This is different from the common model where the human approves every action. Approval-based governance doesn’t scale. If you have 97 repos and each one needs a human approval for each status transition, you’ve just created a bottleneck that eliminates the benefit of automation.
The eight-organ system implements this through the orchestrator-agent workflow. The orchestrator runs weekly, builds the system graph, identifies repos that need attention, and generates recommendations. The human reviews the recommendations, not the individual repo states. If the orchestrator recommends promoting a repo and the human disagrees, the human overrides. But the human doesn’t proactively scan all 97 repos looking for promotion candidates — that’s the system’s job.
Practical advice: Design your system so that human attention is the scarce resource to conserve, not the cheap resource to spend. Every decision that can be made by encoded criteria should be. Reserve human judgment for the cases that actually need it.
Common Failure Modes
These are the failures I’ve encountered or narrowly avoided:
Premature automation. Automating a process you don’t yet understand well enough to encode correctly. The fix: run the process manually 3-5 times, document the decision criteria you’re actually using, then automate.
Constraint-free agents. Giving an autonomous component maximum flexibility and hoping it figures out the right behavior. It won’t. Constraints are design decisions. Omitting them isn’t freedom — it’s abdication.
Observability debt. Building the automation but not the monitoring. You’ll discover the system has been doing the wrong thing for weeks when something visibly breaks. The fix: build the audit before the automation.
Circular dependencies. Allowing bidirectional information flow between components. It always seems harmless (“ORGAN-III just needs one small input from ORGAN-II”), and it always produces coupling that makes the system unpredictable. Enforce the DAG.
Human-in-the-loop theater. Adding human approval steps that the human rubber-stamps because they don’t have time or context to evaluate. Either the approval is meaningful (invest in giving the human the context to make a real decision) or it’s not (remove it and rely on automated checks).
Where This Leads
Autonomous systems thinking is increasingly relevant beyond infrastructure engineering. LLM agent frameworks (LangChain, CrewAI, AutoGen) are autonomous systems with the same design challenges: dependency management, state tracking, constraint encoding, observability. The mental models above apply directly.
The eight-organ system was designed before multi-agent AI frameworks existed. But the design patterns converge because the underlying problem is the same: how do you coordinate multiple semi-independent components into coherent output without a human micromanaging every step?
The answer, in every domain, is the same: clear structure, explicit state, enforced constraints, automated observation, and human oversight at the appellate level. The specific implementation varies — GitHub Actions vs. agent orchestrators, JSON registries vs. vector databases, promotion workflows vs. tool-use routing. But the architecture is the same.
That’s how to think about autonomous systems: not as “things that work without humans” but as “things where the coordination logic is clear enough to encode.” The human is still the architect. The system is the orchestra.
This essay is part of the ORGAN-V Public Process — building in public, documenting everything.
| *Related repos: orchestration-start-here | agentic-titan | recursive-engine–generative-entity* |