TL;DR
- What was at stake
- The Air Force was about to commit to one enterprise IT contract. Without a definition of good service that users would recognize, the contract would only pay for uptime.
- What changed
- My five personas and their Experience Level Agreements went into the Wave 1 and Wave 2 solicitation requirements for a $5.7B contract.
- My role
- Research lead and advisor to Air Force program leadership, leading a research team of 4 within a larger Isobar advisory team of about 10
- Timeline
- About 15 months
- Recognition
A challenge coin from the U.S. Air Force Chief Experience Officer, for leading the move to a user-centered service model
IT satisfaction out of 100 at the furthest-along pilot base, against comparable bases that hadn’t transitioned. It gave leadership the evidence to scale.
Faster startup at the pilot sites, with 59% faster data transfer and 92% less security workload.
Five personas and their Experience Level Agreements, in the Wave 1 and Wave 2 solicitation requirements for a $5.7B contract.
Vendors did the migration. My part was setting the user requirements it was measured against.
A base could meet every service level and still have Airmen waiting an hour to log in.
Read the full case study
Four recommendations
Each persona got its own measures of success, written as a layer alongside the existing service levels rather than in place of them. Keeping the service levels gave procurement something familiar to build on.
Stop buying the same equipment for everyone. Garrison Users get reliable, lower-cost devices. Power Users and Flightline Users get what their jobs require.
Route routine requests like password resets to self-service, measured by first contact resolution, and send complex issues to specialists with their own repair-time commitments. I helped set up this model.
Separating the service into specialized commercial pieces keeps the Air Force from being locked into one vendor. Isobar helped shape this strategy, and it drew on the persona-based service design I led.
EITaaS was meant to replace base-by-base IT with one enterprise service. Splitting the front end across vendors cut against that, and it kept the Air Force from being locked into one company.
The program measured whether the systems worked
If the systems met their service levels, the IT experience was good.
Even after the network upgrades at Buckley, 88% of users still reported at least one issue that hurt their work.
One standard setup and support model can’t serve people working in very different conditions.
What would have changed my mindIf people at all nine installations had named the same top problems, one standard setup would have held up.
Interviews at all nine installations sorted people into five groups with different needs, from flightline crews working offline to analysts who needed heavy compute.
Every base ran its own IT
Things were bad enough that an Air Force officer’s open letter, “Fix our computers,” went viral. Airmen and Guardians recognized the hour-long logins and ten forced restarts in a day he described, and the Air Force’s CIO answered him publicly.
More than 230 organizations could make changes on the network, and more than 80 separate authority-to-operate boundaries split it up. Every base bought its own hardware, ran its own helpdesk, and decided for itself what “working IT” meant. EITaaS, Enterprise IT as a Service, was meant to replace that with one commercial enterprise service. A pilot at nine installations tested the model before the Air Force committed to a $5.7B contract.
Isobar wasn’t part of the delivery team. We advised Air Force program leadership under the Architecture and Integration Directorate, so any influence had to come from the quality of the work. I joined after the pilot was underway, replacing the previous research lead.
Research that had to hold up in a contract
Describing the current experience wasn’t enough. The future state had to be specific enough for vendors to be measured against, so every finding needed to end in something countable.
- Interviews across all nine installations, with Airmen, Guardians, and IT staff
- Pre and post surveys, with a base outside the program as a control, weighted by status, role, and career family
- Moderated usability testing of the enterprise service desk
- Journey mapping from the moment something breaks to when it’s fixed
Picked up the work after the previous lead left, including surveys already in motion.
Covered all nine installations.
Working sessions with Air Force program leadership to agree on thresholds.
Part time, alongside Advana.
The XLAs couldn’t be written until the personas were created, so interviews and personas took most of the first seven months. A bid protest paused the work for a few months in early 2023, and from month 10 Advana took more of my time.
Five kinds of users, five definitions of working IT
A desk-bound administrator and a flightline maintenance tech need different things from IT, work in different conditions, and mean different things when they say it works. Treating everyone as one user type was behind both the cost problem and the experience problem. The interviews showed what each group needed to do its job, and I turned those needs into thresholds the vendors could be held to.
The Flightline XLA was written against the pilot’s startup and data transfer benchmarks.
Reading the survey results carefully
Satisfaction after a migration is easy to over-read. After the first network migration at Offutt, the share of users satisfied with the network fell from 28% to 15%. Buckley barely moved, and the control base held flat. Without the control and the weighting, that drop could have been blamed on survey timing or on who happened to answer. With them, it pointed at the migration. After the base network and Wi-Fi upgrades, Offutt recovered 14 points and Buckley rose 24.
| Base and measure | Before | After |
|---|---|---|
| Buckley, slow speeds at least daily | 55% | 25% |
| Offutt, slow speeds at least daily | 65% | 38% |
Surveys only caught part of it. Responses were linked to service desk tickets and endpoint monitoring, which showed what the scores couldn’t. In earlier program data, 87% of incidents were resolved, but 52% took seven days or more.
“Outlook spins constantly and it takes 10-15 minutes to logon and load Outlook in the morning.”
“There are more websites that are blocked from my work computer, but it’s so common and I expect it so I don’t memorize the sites. I just email the information to my home email and work from there.”
That second answer is a user experience problem and a security risk at once.
Impact
In EITaaS, my findings had to be specific enough to inform contract requirements without losing what people on the ground said. For program leadership and engineering, the personas gave everyone the same picture of who they were building for, so provisioning decisions were based on what people at the bases needed.
Hindsight
I’d document the chain from specific interview and survey findings to each threshold. That would have made the framework easier to defend and to hand to the next phase of the program.When vendors pushed back on a threshold, the program could point to the finding behind it instead of re-arguing it.
The XLAs came from one point in time. A standing feedback loop would show whether a threshold was still the right thing to measure, and not only whether vendors were meeting it.The XLAs could change as bases transitioned, instead of locking in one year’s expectations for a multi-year contract.
Our influence depended on trust with program leadership and on the Air Force survey office, which controlled when and how surveys went out. Earlier effort on both would have cut weeks of waiting.
With two more weeks
Check whether hitting a threshold, like startup time for Flightline Users, actually tracks with that group’s satisfaction.
The survey report flagged DNS and the security layer as a likely bottleneck that still needed analysis.
Find out how common it is for people to email work home to get around blocked sites. Satisfaction scores don’t measure that risk.