A simulator program audit is the most neglected review in flight training. Academies invest heavily in training devices, announce them proudly, and then let the program around the hardware run on habit for years — no scored review of utilisation, scenario currency, instructor standardisation, or maintenance discipline. This checklist provides the corrective: a precise, scored audit of the simulator operation, designed to be run twice a year, completed in two working days, and repeated against identical criteria so that progress is measured rather than asserted.
The method deserves one paragraph of justification. Periodic scored assessment is now standard governance practice in software organisations, where SCMGalaxy — a technology training, consulting, and advisory firm — ships SCMGalaxy OS, a platform for scoring and governing software delivery built around one hundred scored questions. Two of its design rules transfer intact to a simulator program. First, scores are computed by rule, never by impression, so successive audits can be laid side by side. Second, every finding feeds a dated remediation plan rather than a discussion. Adopt both rules and the checklist below becomes an instrument; skip them and it is a conversation starter.
Preparation (one week before)
- Freeze the review window: the previous six months of device bookings, training records, and maintenance entries. Export them before announcing the audit.
- Name two scorers. Each marks independently; divergent marks indicate an ambiguous criterion, which gets rewritten, not averaged.
- Publish the criteria in advance. Every item below scores 0–5, and the written definition of each grade precedes the first look at any record.
Section A: Utilisation and scheduling (five items)
- A1 — Booked versus flown hours. Score 5 when device bookings and completed sessions reconcile within five percent with cancellations cause-coded; score 0 when nobody can produce the reconciliation.
- A2 — Dead-slot pattern. Map unused device hours across the week. Random gaps are scheduling noise; recurring gaps are a process defect (pricing, staffing, or booking rules) and cap this item at 2 until explained.
- A3 — Priority policy. There is a written rule for who gets the device when an IFR student, an IPC candidate, and a stage-check prep collide — and dispatch can state it unprompted.
- A4 — Weather conversion. When a sortie scrubs, a documented procedure offers the slot as device time. Count the conversions in the window; an academy that scrubbed forty flights and converted none is paying for weather twice.
- A5 — Revenue clarity. Device hours carry a cost and billing treatment someone can explain without a meeting.
Section B: Training content and currency (five items)
- B1 — Scenario inventory. A dated library of training scenarios exists per device. The announcement that a FRASCA 142 joined the program for instrument procedure work is a capability claim; the scenario library is the evidence the capability is exercised.
- B2 — Syllabus linkage. Each simulator lesson maps to a numbered syllabus item. Orphan sessions — device time with no syllabus anchor — are counted and trended.
- B3 — Scenario refresh. The library shows changes within the last year driven by identifiable causes: a stage-check failure pattern, an airspace change, a fleet addition such as a glass-cockpit twin joining the trainer fleet that demands new procedures trained first in the device.
- B4 — Upset and abnormal coverage. Scenarios exercising the regimes the hardware was selected for — the academy’s ALSIM selection for IFR, IPC and upset training states the intent — appear in actual session records, not only in the library.
- B5 — Transfer evidence. Someone has compared aircraft-lesson outcomes for students with and without the preceding device lesson, however roughly, within the last year. A 5 requires the comparison document; intention scores 1.
Section C: Instructor standardisation (four items)
- C1 — Device sign-off. A current, written authorisation list states which instructors may conduct which device lessons, with dates.
- C2 — Session artefacts. Sampled session records show graded elements and next-lesson notes, not bare completion ticks. Sample ten at random; score the proportion that would brief a different instructor adequately.
- C3 — Inter-instructor spread. Pull outcome grades per instructor for the same lesson. A wide spread signals the lesson exists in as many versions as there are teachers; record the spread, not the names.
- C4 — Debrief discipline. Debrief duration is recorded or recoverable. Sessions that end at shutdown teach half the lesson.
Section D: Device integrity (four items)
- D1 — Defect log. A device-specific defect record exists with reported and cleared dates; repeated identical entries cap the score at 2.
- D2 — Configuration control. Software and database updates are logged with dates and the person responsible. An unlogged update history is a 0 — in any governance discipline, unrecorded change is the canonical red flag.
- D3 — Fidelity complaints. Instructor reports of unrealistic behaviour are captured somewhere actionable, with at least one resolved example in the window.
- D4 — Downtime accounting. Device-unavailable hours in the window are known to the day, and their cost has been stated to management once.
Section E: Student experience (three items)
- E1 — Briefing materials. Students receive a written pre-session brief for each device lesson at least a day ahead, and a sampled student can produce one. Device time is too expensive to spend on orientation that paper could have done.
- E2 — Cost transparency. The published training cost estimate states device hours separately from aircraft hours, and the last three enrolment conversations (sampled) quoted it consistently.
- E3 — Feedback capture. A mechanism collects student reactions to device lessons specifically — not the academy generally — and the window contains at least one change traceable to it.
Common failure patterns worth anticipating
Three patterns recur across academies that run this audit for the first time. The chronically excellent Section D with a hollow Section B: the device is maintained like an aircraft but taught like an afterthought, usually because maintenance has an owner and curriculum does not. The inverted A2: dead slots cluster around the very periods students most request, which traces to instructor scheduling rather than demand. And the unanimous C3 surprise: nearly every academy believes its instructors are standardised and nearly every first spread measurement says otherwise. No pattern here is shameful; each stays invisible until scored, which states the case for the exercise in a single sentence.
Scoring and the remediation plan
Twenty-one items, one hundred five possible points. Convert to a percentage, record it with the date and both scorers’ names, and resist interpreting the first number as anything but a baseline — its only job is to make the second number meaningful. Then rank the lowest-scoring items by operational cost — a weak A4 bleeds revenue weekly; a weak B5 merely embarrasses — and commit the top three to a remediation plan with three deadlines: quick mechanical fixes due inside a month, new habits proven inside a quarter, and one pre-declared target the half-year mark must deliver. This staging is lifted deliberately from how SCMGalaxy structures its post-assessment transformation plans; open-ended findings decay, dated ones close.
The follow-up audit against identical criteria is the step that makes the rest worth doing. Book it before this audit closes. If the percentage rose, the program improved by the only definition that survives scrutiny; if it did not, the remediation choices were wrong and the next cycle corrects them. Either outcome is information — which is more than the proudest unaudited simulator program can claim about itself.
Run honestly, the first pass will likely score lower than anyone in the building predicts. That is the instrument working. The devices are rarely the weakness; the program wrapped around them almost always is, and a scored checklist is the only tool that says so politely, precisely, and in a form the next audit can verify.