In this blog post I Wrote the Book on Azure OpenAI Here’s What’s Changed Since Then we will walk through the most meaningful shifts I’ve seen since my book was published, and what those changes mean for leaders and builders. I Wrote the Book on Azure OpenAI Here’s What’s Changed Since Then is not a victory lap—it’s a field note from the last mile of real deployments.

One thing I didn’t fully appreciate while writing is just how quickly “using a model” would evolve into “operating a capability.” Azure OpenAI isn’t just about prompts anymore. It’s about governance, throughput engineering, multimodal experiences, and building systems that stay safe and predictable under pressure.

I’m a published author, but I’m also an enterprise architect at heart. I’ve spent 20+ years building and hardening platforms across Azure and Microsoft 365, working with organisations in Melbourne and across Australia (and a few internationally). The patterns below come directly from that practical work—anonymised, but very real.

High-level what Azure OpenAI actually is (and why it keeps changing)

At a high level, Azure OpenAI is Microsoft’s enterprise-grade way to use frontier AI models inside Azure. You get the “brains” (large language models and now reasoning models), but you also get Azure-native controls: identity, networking, logging, regional choices, and guardrails that matter to CIOs.

The technology behind it is straightforward in concept. You send text (and increasingly images and audio) to a model endpoint. The model predicts the best next tokens to generate a response, guided by instructions you provide. Where it gets interesting is the enterprise layer: controlling where data flows, how requests scale, how outputs are constrained, and how risk is managed.

What’s changed since publication (the short version)

• Model choice is now a strategy decision, not just “GPT-4 vs GPT-3.5.”
• Throughput and cost engineering has become its own discipline (Standard vs Provisioned, plus batch patterns).
• Reasoning models changed expectations for accuracy and problem solving, but also changed latency and governance conversations.
• Multimodal is no longer a demo feature; it’s landing in real workflows.
• Enterprise guardrails matured—and leaders now expect them upfront (not “we’ll add it later”).

1) The model landscape moved from “one best model” to “a portfolio”

When I wrote the book, a lot of teams wanted a single default model for everything: chat, summarisation, search, and basic automation. In my experience, that’s now the exception.

Today, I see successful programs treat models like a portfolio:

• A general model for everyday drafting, summarisation, and Q&A.
• A smaller fast model for high-volume tasks (triage, classification, routing, metadata generation).
• A reasoning model for complex decision support, technical analysis, or multi-step planning.

The practical outcome is simple: cost goes down, user experience goes up, and your “critical path” workflows get a model chosen for the job—not a model chosen by habit.

2) Provisioned throughput became a first-class architecture choice

One pattern I keep running into is teams piloting successfully, then hitting a wall the moment the use case becomes popular. Not because the model is bad—because the platform wasn’t designed for predictable throughput.

Azure OpenAI has matured its throughput options. In plain language:

• Standard: great for experimentation and uneven traffic, but performance and rate limits can fluctuate.
• Provisioned: you reserve capacity for steadier performance and more predictable cost at scale.

I now treat “How will we handle demand spikes?” as a day-one question. If your organisation is rolling out an AI assistant to thousands of staff, you’re not really building a chatbot—you’re building a digital utility.

A lesson from the field

In one anonymised program, the pilot ran beautifully for 200 users. The enterprise rollout to 8,000 users turned into a queueing problem overnight. We ended up separating workloads into tiers: fast low-cost classification on a small model, and “expensive thinking” only when the workflow truly needed it.

The business result wasn’t just lower cost. It was trust. People stop using systems that feel unreliable, even if they’re brilliant on a quiet Tuesday.

3) Reasoning models changed what “good” looks like

Earlier generations of deployments often assumed the model would be “smart, but occasionally wrong,” and you’d manage that with disclaimers and human review. That still matters—but I’ve seen expectations shift.

Reasoning models introduced a different profile:

• Better at multi-step tasks like root cause analysis, architecture trade-offs, and structured planning.
• More sensitive to latency and workload design; you can’t treat every query like a deep analysis job.
• More value in constraints (schemas, structured outputs, and tight instructions) to keep results operational.

My practical takeaway: don’t just ask “Which model is smartest?” Ask “Which model is smart enough, fast enough, and governable enough for this workflow?”

4) Multimodal moved from novelty to real utility

For a long time, leaders heard “multimodal” and assumed marketing fluff. What I’m seeing now is more pragmatic: teams using images and documents as everyday inputs.

Examples I’ve seen work well:

• Interpreting screenshots of system errors and mapping them to known incident playbooks.
• Extracting structured data from messy documents to reduce admin load.
• Assisting frontline teams with visual checks (where policy allows it) rather than relying on long written descriptions.

This is also where risk increases. More input types means more places sensitive information can hide. Which brings me to the next point.

5) Security and governance matured—and the bar is higher in Australia

In Australian organisations, I’ve found the conversation inevitably lands on Essential Eight alignment, data residency expectations, and privacy obligations. Even when a use case starts small, leaders want confidence that it won’t turn into an uncontrolled shadow-IT channel.

What’s changed since publication is that governance is no longer “phase two.” Mature programs now design for:

• Identity-first access (who can use what, and for which data classes).
• Network boundaries (private access patterns where required, not just public endpoints).
• Logging and auditability that stands up in real incident response, not just in a slide deck.
• Data handling rules that reflect Australian privacy expectations and internal classification schemes.

My opinion: the fastest way to kill an AI program is to treat governance as bureaucracy. The best way to accelerate adoption is to make guardrails invisible to end users but very real to auditors.

6) “On your data” is still the goal, but teams are more honest about it now

The dream hasn’t changed: ask a question and get an accurate answer grounded in internal knowledge. What has changed is realism about what it takes.

In practice, successful implementations usually require:

• Search and retrieval design (what content, how chunked, how refreshed, how ranked).
• Content hygiene (outdated policies and conflicting docs produce confident nonsense).
• Clear boundaries between “summarise what we know” and “advise what we should do.”

I’ve seen the best results when teams treat it like building a product: iterate the knowledge base, measure answer quality, and continuously improve the retrieval layer.

A concrete scenario I’d update in the book today

Here’s an anonymised scenario that mirrors what I’m seeing across many environments.

An IT division launches an internal assistant for Microsoft 365 and Azure operational questions. Week one is great: staff ask how to request access, how to interpret alerts, how to follow change processes.

Then it spreads. People start pasting incident details, screenshots, and customer-related snippets. The assistant becomes a magnet for sensitive data, and leadership suddenly needs answers to uncomfortable questions: Who’s using it? What’s being shared? Can we prove controls align with our policies?

The fix isn’t to shut it down. The fix is to design the assistant like a controlled system:

• Restrict access by role and data classification.
• Separate “general help” from “incident analysis” workloads.
• Use structured outputs for operational actions (tickets, summaries, post-incident reports).
• Measure throughput, latency, and cost so the program remains predictable.

Practical steps I recommend now (even for small pilots)

• Pick a model per task, not per program. Default to cheaper/faster models for high-volume work.
• Decide early how you’ll scale. If success means thousands of users, design for throughput from day one.
• Constrain outputs when the response becomes an input to a system (JSON schemas, strict formats).
• Build a feedback loop. Treat hallucinations and bad answers as defects you can reduce, not “just AI.”
• Make governance a product feature. If it’s hard to use safely, people will work around it.

A small code example that reflects the “new normal”

I’m not going to drown this post in code, but one shift is worth showing: using structured outputs so responses are operational, not just conversational.

// Pseudocode example: ask for a change summary in a strict JSON shape
// (Exact SDK syntax varies by language and API version.)

const schema = {
 type: "object",
 properties: {
 changeTitle: { type: "string" },
 riskLevel: { type: "string", enum: ["low", "medium", "high"] },
 customerImpact: { type: "string" },
 backoutPlan: { type: "string" },
 approvalsNeeded: { type: "array", items: { type: "string" } }
 },
 required: ["changeTitle", "riskLevel", "customerImpact", "backoutPlan", "approvalsNeeded"]
};

const result = await model.generate({
 input: "Summarise this proposed firewall change for CAB review...",
 outputFormat: { jsonSchema: schema }
});

return JSON.parse;

The business benefit is that your downstream systems can trust the shape of the output. You’re no longer hoping the model “formats it nicely.” You’re specifying what “done” looks like.

What hasn’t changed (and I’d keep it exactly as written)

Despite all the platform evolution, the fundamentals are steady.

• Clear use cases beat clever demos.
• Data quality is the multiplier.
• Architecture decisions show up later as adoption or failure.
• Human trust is earned through reliability, not through impressive one-off answers.

Closing reflection

Azure OpenAI is moving quickly, but the direction is consistent: more model options, more enterprise controls, and more expectation that AI systems behave like production platforms—not experiments.

If I had to summarise what changed since publication in one line, it’s this: we’ve shifted from “Can we do this?” to “Can we operate this safely, predictably, and at scale?”

Looking ahead, I suspect the next wave of maturity won’t be about bigger models. It’ll be about better system design—especially how we combine reasoning, retrieval, and governance into something leaders can confidently bet on. What part of your AI stack feels least “operational” today?