Agent Studio Office Hours – How the Reasoner Actually Thinks (And How to Stop Confusing It)
We had our highest attendance yet, 40 people on the call and the questions did not let up! Ryan was meeting with a customer and Steve was still waking up on the West Coast, so I ran this one solo — and honestly, it turned into one of the most teaching-heavy sessions we've done. I spent the first chunk breaking down how LLM reasoners works under the hood, then we got into live debugging, scoping gotchas, and a sneak peek at the observation interpreter.
If you've ever wondered _why_ your plugin works sometimes and falls apart other times, this session was for you. Let's get into it.
- How the Reasoner Works: Context, Attention & Why Less Is More
This was the big one. I walked through how the LLM actually processes everything in Agent Studio — your plugin config, slot descriptions, conversation messages, action outputs — all of it gets packed into a context window as tokens. The reasoner uses attention to figure out what matters, but here's the thing: attention drops off hard in the middle of a long context. It's called the "lost in the middle" problem, and it means stuffing more content into context can actually _hurt_ your plugin's performance. Less really is more.
- The Golden Rule: Never Chain Actions Without a Slot
If you're chaining action activities in your conversation process, every API response from every action gets dumped into context. That's a fast way to confuse the reasoner. The fix: use compound actions with return expressions so you only pass back the fields you actually need. I walked through a room booking example — calendar lookup, room list, capacity check, reservation — all handled in one compound action with a clean return shape.
- Observation Layer: Why Structured Output Mappers Matter
This is a newer topic and docs are coming soon. There's a layer between your output mapper and the reasoner called the observation interpreter. When your output mapper returns structured data (JSON, YAML objects), the observation layer can parse it, sample a couple of objects, and send a compact representation to the reasoner. But if you return a plain text string? It can't parse anything — the whole blob goes straight into context. Always use structured data in your output mappers. And on a related note: content activities are in a weird spot right now. Their LLM doesn't share context with action activities, so don't use them for steering or formatting. Use structured output mappers instead.
- Plugin Scoping: Case Sensitivity & Group-Based Access
Someone hit a classic gotcha — `user.email in [list]` rules are case-sensitive. If your list has `John@company.com` but the user record has `john@company.com`, scoping fails silently. The fix is wrapping with `$LOWERCASE()` and making sure your list entries are all lowercase. For group-based scoping, the move right now is user tags via ingestion (sync your AD groups into tags).
- Live Debug: Notify Recipient IDs & the Compound Action Test Button
We did a live debug of a contractor expiry notification. The bug: using `lookup_id` instead of `record_id` as the notify recipient. Notify needs the actual `record_id` — not email, not lookupId. Also, notify uses Markdown (not HTML) for formatting. I also flagged the compound action test button — some orgs are missing it, and I'm pushing to get it enabled everywhere.
- Quick Hits
→ MCP support is on the roadmap for Q2 2026, with A2A in the future
→ For proactive notifications, the pattern is: ServiceNow Flow Designer (or any alerting tool) fires a webhook into an ambient agent
→ The [DSL Mapper Playground](https://help.moveworks.com/docs/dsl-and-mapper-playground) is your best friend for testing output mapper expressions
→ Community roadmap is live: [community.moveworks.com/p/roadmap](https://community.moveworks.com/p/roadmap)
Here's the recording to revisit any part of the session or share it with your team: Recording link
Resources & Next Steps
→ [DSL & Mapper Playground](https://help.moveworks.com/docs/dsl-and-mapper-playground) — test your output mapper expressions live
→ [Community Roadmap](https://community.moveworks.com/p/roadmap) — see what's shipping next
→ Register for upcoming sessions: [Developer Office Hours](https://www.moveworks.com/us/en/resources/webinars/developer-office-hours)
If nothing else sticks from this session, remember: the reasoner sees _everything_ you put in context. Treat every field, every description, every action output like it costs attention — because it does. Build lean. 🤖