---
title: Agents on Azure that look before they
answer
subtitle: Autonomous agents on Azure AI
Foundry, agentic RAG over Microsoft 365,
and LLMOps with Azure PromptFlow
slug: azure-foundry-agents-agentic-rag
hook: Digital employees that call tools,
remember context and answer from
company knowledge. When the first search
comes back thin, they look again
instead of guessing.
role: "Agentic AI Engineer (contract): design,
build and deployment, end to end"
employer: MindHYVE.ai (contract, remote)
client: MindHYVE.ai's own agent platform
period: May 2025 – Jan 2026
tracks:
- agentic-genai
- mlops
- full-stack-fde
tags:
- Azure AI Foundry
- agentic RAG
- corrective RAG
- self-RAG
- LangGraph
- LLMOps
- Azure PromptFlow
- multi-agent systems
metrics:
- value: an estimated ~37%
label: Fewer errant LLM outputs in
production tests, after prompt
optimization
and automated deployment with Azure
PromptFlow
when: May 2025 – Jan 2026
- value: up to 15–20%
label: Simulated ROI improvement in
backtests, from an LLM multi-agent
trading
system. Backtests only.
when: May 2025 – Jan 2026
---
## Context
MindHYVE.ai builds domain-specific AI agents.
It calls them digital employees, for areas
like finance, healthcare, education, law
and marketing. From May 2025 to January
2026 I worked there as a contract agentic
AI engineer, alongside my Turing contract,
and owned agent work end to end: design,
build and deployment, on Azure.
## The problem
An agent that reasons well but answers from
memory is a confident intern. Enterprise
users need agents that use the company's
own knowledge, point to it, and admit when
it isn't there. They also need agents that
behave on Tuesday the way they did in the
demo.
## Constraints
- **Azure-native.** The platform ran on Azure,
so the agents, tools and pipelines did
too.
- **Company knowledge lives in Microsoft
365.** Answers had to come from there, not
from the model's training data.
- **Real time.** Personalization and decision
automation had to run as live inference,
not overnight batches.
- **Prompt changes are deployments.** A "small
wording tweak" can change behavior
everywhere, so prompts needed testing and
controlled rollout like code.
## What I built
**1. Agents on Azure AI Foundry.** Domain
agents built on Azure AI Foundry with
Azure Machine Learning, Azure Functions
and Azure AI Services, plus Azure-native
inference pipelines for real-time
personalization and decision automation.
Multi-step workflows ran in LangGraph and
LangChain-style frameworks, covering
reasoning, memory and tool use. I also
used CrewAI, AutoGen and Semantic Kernel
where they fit the job better than a
graph.
**2. Agentic RAG over Microsoft 365.**
LangGraph agents that search Microsoft 365
content with semantic and vector search,
decide when to look again, and answer from
company knowledge instead of guessing. The
"look again" part is corrective and
self-RAG: the agent grades what it
retrieved, rewrites the query and searches
again when the passages are thin or
off-topic, and says so when the answer
still isn't there.
**3. LLMOps with Azure PromptFlow.** Prompt
optimization and automated deployment
through Azure PromptFlow, with tracing and
evals in LangSmith and Langfuse. In
production tests, errant outputs fell by
an estimated ~37%.
**4. Also on the bench: an LLM multi-agent
trading system.** Orchestrated with
LangGraph and LangChain and evaluated in
backtests: up to 15–20% simulated ROI
improvement. Backtests only. The market
was not consulted.
## How it fits together
Three layers.
**Knowledge.** Microsoft 365 content is pulled
in through a connector and indexed for
semantic and vector search.
**Reasoning.** A LangGraph agent on Azure AI
Foundry plans each step: search, call a
tool (Azure Functions), or answer.
Retrieved passages get graded for
relevance and grounding. Grounded: answer
with sources. Thin or off-topic: rewrite
the query and retrieve again. Still not
grounded: say so.
**Operations.** Prompts and flows are
versioned, evaluated and deployed through
Azure PromptFlow. Every run is traced.
Azure ML inference pipelines feed
real-time personalization and decision
automation into the agents.
## Stack
- **Azure:** Azure AI Foundry (now Microsoft
Foundry), Azure Machine Learning, Azure
Functions, Azure AI Services, Azure
PromptFlow, Azure AI Search.
- **Microsoft 365:** Microsoft Graph for
content access.
- **Agents:** LangGraph, LangChain, CrewAI,
AutoGen, Semantic Kernel. Tool calling,
memory, multi-agent workflows.
- **Retrieval:** agentic RAG, corrective RAG,
self-RAG; semantic and vector search;
embeddings.
- **LLMOps:** prompt optimization, LangSmith,
Langfuse, automated deployment.
- **Engineering:** Python, Docker, Cursor,
Claude Code.
## Results
- **An estimated ~37% fewer errant outputs**
in production tests, after prompt
optimization and automated deployment with
Azure PromptFlow.
- **Up to 15–20% simulated ROI improvement in
backtests** from the trading agents.
Simulated, in backtests, and labelled that
way every time.
- **For the Microsoft 365 agentic RAG, I don't
have a number I can stand behind, so there
isn't one here.** What I can show is the
design, and the reasons for it.
## What I'd do differently
- **Build the retrieval eval set before the
agent.** Fifty real questions with known
source documents, including ten whose
answer isn't in Microsoft 365 at all. The
"look again" loop is only as good as its
grader, and the grader needs ground truth.
- **Test permissions explicitly.** Run every
eval question as two users with different
access. A retrieval system that leaks
across permissions is worse than one that
doesn't work.
- **Budget the loops.** Looking again is great
for accuracy and terrible for cost if
nothing caps it. I'd set a per-question
token and latency budget on day one and
trace it.
- **Use the 2026 managed pieces.** Today I'd
start from Foundry Agent Service with
Foundry IQ (Azure AI Search) for
knowledge, and serve tools over MCP so
every agent reuses the same ones.
## What I left out
Not included, on purpose:
- MindHYVE.ai's customers, tenants and data;
- which Microsoft 365 sources were indexed,
and the index design;
- prompts, agent definitions and anything
specific to the company's products;
- model choices and internal metrics beyond
the two figures above.
## How to verify
- **LinkedIn projects under MindHYVE.ai** (May
2025 – Jan 2026): "Agentic RAG over
Microsoft 365", "Autonomous AI agents on
Azure AI Foundry", "LLMOps: prompt
optimization with Azure PromptFlow" (the
estimated ~37%) and "Autonomous LLM
multi-agent trading system" (up to 15–20%
simulated ROI in backtests).
- **LinkedIn experience:** MindHYVE.ai,
Agentic AI Engineer, contract, May 2025 –
Jan 2026.
- **Employment check:** MindHYVE.ai, same
title and dates.
- **In an interview:** a walk-through of the
method, within NDA limits.
- **Code:** walkthroughs available on request.
Case study 06 · MindHYVE.ai (contract, remote) · May 2025 – Jan 2026
Agents on Azure that look before they answer
Autonomous agents on Azure AI Foundry, agentic RAG over Microsoft 365, and LLMOps with Azure PromptFlow
Digital employees that call tools, remember context and answer from company knowledge. When the first search comes back thin, they look again instead of guessing.
✓ Compiled from this page's own markdown.
Context
MindHYVE.ai builds domain-specific AI agents. It calls them digital employees, for areas like finance, healthcare, education, law and marketing. From May 2025 to January 2026 I worked there as a contract agentic AI engineer, alongside my Turing contract, and owned agent work end to end: design, build and deployment, on Azure.
The problem
An agent that reasons well but answers from memory is a confident intern. Enterprise users need agents that use the company's own knowledge, point to it, and admit when it isn't there. They also need agents that behave on Tuesday the way they did in the demo.
Constraints
- Azure-native. The platform ran on Azure, so the agents, tools and pipelines did too.
- Company knowledge lives in Microsoft 365. Answers had to come from there, not from the model's training data.
- Real time. Personalization and decision automation had to run as live inference, not overnight batches.
- Prompt changes are deployments. A "small wording tweak" can change behavior everywhere, so prompts needed testing and controlled rollout like code.
What I built
1. Agents on Azure AI Foundry. Domain agents built on Azure AI Foundry with Azure Machine Learning, Azure Functions and Azure AI Services, plus Azure-native inference pipelines for real-time personalization and decision automation. Multi-step workflows ran in LangGraph and LangChain-style frameworks, covering reasoning, memory and tool use. I also used CrewAI, AutoGen and Semantic Kernel where they fit the job better than a graph.
2. Agentic RAG over Microsoft 365. LangGraph agents that search Microsoft 365 content with semantic and vector search, decide when to look again, and answer from company knowledge instead of guessing. The "look again" part is corrective and self-RAG: the agent grades what it retrieved, rewrites the query and searches again when the passages are thin or off-topic, and says so when the answer still isn't there.
3. LLMOps with Azure PromptFlow. Prompt optimization and automated deployment through Azure PromptFlow, with tracing and evals in LangSmith and Langfuse. In production tests, errant outputs fell by an estimated ~37%.
4. Also on the bench: an LLM multi-agent trading system. Orchestrated with LangGraph and LangChain and evaluated in backtests: up to 15–20% simulated ROI improvement. Backtests only. The market was not consulted.
How it fits together
Three layers.
Knowledge. Microsoft 365 content is pulled in through a connector and indexed for semantic and vector search.
Reasoning. A LangGraph agent on Azure AI Foundry plans each step: search, call a tool (Azure Functions), or answer. Retrieved passages get graded for relevance and grounding. Grounded: answer with sources. Thin or off-topic: rewrite the query and retrieve again. Still not grounded: say so.
Operations. Prompts and flows are versioned, evaluated and deployed through Azure PromptFlow. Every run is traced. Azure ML inference pipelines feed real-time personalization and decision automation into the agents.
Stack
- Azure: Azure AI Foundry (now Microsoft Foundry), Azure Machine Learning, Azure Functions, Azure AI Services, Azure PromptFlow, Azure AI Search.
- Microsoft 365: Microsoft Graph for content access.
- Agents: LangGraph, LangChain, CrewAI, AutoGen, Semantic Kernel. Tool calling, memory, multi-agent workflows.
- Retrieval: agentic RAG, corrective RAG, self-RAG; semantic and vector search; embeddings.
- LLMOps: prompt optimization, LangSmith, Langfuse, automated deployment.
- Engineering: Python, Docker, Cursor, Claude Code.
Results
- An estimated ~37% fewer errant outputs in production tests, after prompt optimization and automated deployment with Azure PromptFlow.
- Up to 15–20% simulated ROI improvement in backtests from the trading agents. Simulated, in backtests, and labelled that way every time.
- For the Microsoft 365 agentic RAG, I don't have a number I can stand behind, so there isn't one here. What I can show is the design, and the reasons for it.
What I'd do differently
- Build the retrieval eval set before the agent. Fifty real questions with known source documents, including ten whose answer isn't in Microsoft 365 at all. The "look again" loop is only as good as its grader, and the grader needs ground truth.
- Test permissions explicitly. Run every eval question as two users with different access. A retrieval system that leaks across permissions is worse than one that doesn't work.
- Budget the loops. Looking again is great for accuracy and terrible for cost if nothing caps it. I'd set a per-question token and latency budget on day one and trace it.
- Use the 2026 managed pieces. Today I'd start from Foundry Agent Service with Foundry IQ (Azure AI Search) for knowledge, and serve tools over MCP so every agent reuses the same ones.
What I left out
Not included, on purpose:
- MindHYVE.ai's customers, tenants and data;
- which Microsoft 365 sources were indexed, and the index design;
- prompts, agent definitions and anything specific to the company's products;
- model choices and internal metrics beyond the two figures above.
How to verify
- LinkedIn projects under MindHYVE.ai (May 2025 – Jan 2026): "Agentic RAG over Microsoft 365", "Autonomous AI agents on Azure AI Foundry", "LLMOps: prompt optimization with Azure PromptFlow" (the estimated ~37%) and "Autonomous LLM multi-agent trading system" (up to 15–20% simulated ROI in backtests).
- LinkedIn experience: MindHYVE.ai, Agentic AI Engineer, contract, May 2025 – Jan 2026.
- Employment check: MindHYVE.ai, same title and dates.
- In an interview: a walk-through of the method, within NDA limits.
- Code: walkthroughs available on request.