Every instructor at this academy knows the feeling of a student’s first solo: the groundwork is long, the checklist is fixed, and the actual moment is calm precisely because nothing in it is improvised. We kept that shape in mind when a small group of us on the operations side decided to bring the academy’s scattered website and booking plumbing under GitOps — the practice of keeping a written, versioned description of how our systems should be set up, and letting an automated helper continuously pull that description and correct the live machines when they wander. This diary is the step-by-step version of our first month, written while the details were still fresh, in the hope that the next school to try this wastes less time than we did.
Why We Chose GitOps at All
Honesty first: nothing was on fire. We simply kept rediscovering the same category of small disaster. A plugin update applied by hand on a Friday. A configuration file fixed on the live server but never recorded anywhere. Two machines that were supposed to be identical and were not, in ways nobody could explain. Each incident was small; together they described an operation run on memory, and memory does not survive staff changes.
The pitch that convinced the sceptics among us was not efficiency. It was the same argument that grounds our flying: written plans, reviewed before execution, with a record kept afterwards. GitOps offered that for the machines behind the website, and it promised — correctly, as it turned out — that most of the value would arrive before any clever tooling did. We built the month on the practices collected in the Ops Atlas GitOps reference, and leaned on it hardest while writing the override procedure in step seven.
The Eight Steps, As We Actually Ran Them
Step 1: Write down what runs, before touching anything
One afternoon, one shared document: the public site, the booking subdomain, the two databases behind them, the mail relay, the nightly job that nobody remembered approving. The document was uncomfortable in the way an honest weight-and-balance sheet is uncomfortable. Several things existed that no current staff member could explain, and one or two things we believed existed turned out to have been retired months earlier. Writing the list cost us an afternoon and repaid it within the week, because the two or three myths it killed would each have cost a day of confused debugging later.
Step 2: Turn the honest list into files in a repository
We described each system’s intended configuration in files and put them under version control. Nothing was automated yet; the point was to have one place where the plan lived. The rule from day one was blunt: anything absent from the repository does not count as part of the official setup, whoever installed it. That single rule caught three undocumented changes in its first week.
Step 3: Install the agent, and let it run itself
Next came the reconciler — the program that reads the repository and keeps reality aligned with it. We installed it and then did the thing that still delights every engineer who sees it for the first time: we put its own configuration into the repository it watches. The tool now upgrades and repairs itself the same way it handles everything else. There is a word for that kind of arrangement in aviation, and the word is elegant.
Step 4: Hand it one harmless corner of the estate
The marketing blog got the honour — low traffic, low risk, replaceable content. We deliberately started with destructive cleanup switched off, so the agent could apply what we declared but would not delete anything it found lying around. One small territory, fully observed, beats half the estate loosely watched.
Step 5: Try to fool it
The most educational hour of the month. A colleague edited the live blog’s configuration by hand, deliberately, without telling the repository. Twelve minutes later the change was gone, undone without ceremony, and an event recorded who had done it and when. The sceptic on our team became the practice’s loudest advocate in that moment — not because the automation was clever, but because the rollback was boring.
Step 6: Turn on the alarms, not just the repair
Automatic correction handles the ordinary. For the interesting cases we added alerts: whenever reality and the written plan disagreed for more than a few minutes, a message went to a channel humans actually read. The alerts caught a second, sneakier category of problem — differences we had decided to accept, such as a scheduler adjusting capacity on its own. Naming those differences deliberately, resource by resource, turned vague unease into explicit policy.
Step 7: Write the override before we needed it
Any system that automatically undoes changes will, one night, undo a change the on-call person is making on purpose to keep things breathing. So before extending the territory, we wrote the break-glass procedure: who may pause the agent, how the pause is announced, how long it may last, and how the pause itself gets recorded. We rehearsed it once in daylight. The rehearsal took eleven minutes and was worth more than any document we wrote all month.
Step 8: Extend slowly, prune last
Only after a fortnight of quiet operation on the blog did we move the next system across — again one territory at a time, again with deletion still switched off. Deleting is the last permission we grant anywhere, and it stays off until the declared state has run cleanly long enough that we trust our own files. Prune is where every horror story we read began; we saw no reason to add ours.
What GitOps Changed After One Month
Three effects were visible by the end of the month. Changes got smaller, because every change now faced a reviewer and nobody writes an epic for a one-line fix. The 2 a.m. server login all but disappeared, because a hand-edit no longer accomplishes anything — the agent quietly reverses it. And new staff stopped needing folklore; the repository plus the change history told them what was true, which is more than any handover doc ever managed.
If you want the month judged on behaviour rather than feeling, these were the observable signs:
- Every production change traced to a reviewed merge with a named author and a time.
- Hand-made changes stopped persisting, so the running systems matched the written plan without anyone polishing them into agreement.
- The alert channel carried maybe two messages on an ordinary day — few enough to read, enough to matter.
- Newest team member made a safe production change in week two, guided entirely by the files and the history.
It is the same satisfaction our students describe after their first solo above the practice area: nothing dramatic happened, and that was entirely the point. The fleet news — a new simulator joining the lineup, like the FRASCA 142 we added for IFR procedure training — now reaches the site along the identical reviewed route that every other change takes.
What We Would Do Differently
Two things, plainly. We would write the override procedure before step four rather than after it, because the middle of a migration is exactly when the first disagreement arrives. And we would take the alarms more seriously from day one; our first week of alerts was tuned so broadly that the channel earned a reputation for crying wolf, which took another week to repair. Neither mistake was expensive, but both were avoidable with the advice above, which is why this diary exists in this much detail rather than as a triumphant one-paragraph summary.
The wider lesson is the one this academy keeps teaching in every classroom: the ALSIM simulator chosen for IFR, IPC and upset training earns its keep not because it flies, but because it can be reset, repeated and audited — and so can a well-run repository. If your organisation’s systems cannot be reset and repeated, GitOps is the cheapest route we know to making them so. And if the whole idea still sounds abstract, look at how we announce anything that matters, from new aircraft to an award earned against hundreds of other schools: the announcement is written, reviewed and published in an orderly way. We simply taught our servers to expect the same courtesy.