You Don't Need the Cloud to Run an AI Agent (And That Changes Everything)

Most people hear AI agent and immediately picture the cloud.

External APIs.
Data leaving the network.
Security teams reaching for the big red "nope" button.

In a lot of companies, that mental image alone is enough to end the conversation. ChatGPT is blocked. External LLM APIs are off limits. Air-gapped or regulated environments don't even pretend this is an option.

So the conclusion usually sounds like this:

"AI agents sound cool, but we can't use them here."

Which really means more scripts, more manual troubleshooting, and more humans doing repetitive tasks that computers are actually very good at.

But here's the part most people miss.

You don't need the cloud to run AI agents.

Video Demo

Below are the timestamps:

▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬

• 00:00 Introduction
• 00:26 Introducing kagent: AI on Kubernetes
• 00:49 Exploring kagent's Website and GitHub
• 01:17 Understanding K Agent's Features and Projects
• 03:09 Sponsor Message: Up Cloud Special Deal
• 04:09 Setting Up K Agent: Step-by-Step Guide
• 06:22 Deploying and Testing AI Agents
• 09:14 Navigating the K Agent Dashboard
• 11:10 Customizing and Running AI Agents
• 15:45 Using K Agent CLI for AI Operations
• 18:12 Building and Managing AI Agents with YAML
• 21:55 Uninstalling K Agent and Final Thoughts

Understanding the Role of AI Agents in Today's Technology

At a basic level, AI agents are autonomous agents built using artificial intelligence to perform tasks on their own. Modern agent technology relies heavily on large language models (LLMs) and natural language processing to understand inputs, reason about problems, and make decisions.

The key word here is autonomous.

An AI agent isn't just responding to a single prompt from human users. It can analyze data, identify patterns, choose between actions, interact with external tools or external systems, and learn from past interactions using short-term memory and long-term memory.

Most AI agents work by following an observe–plan–act loop. They collect information from their environment — sometimes even sensor data — process it through a reasoning process powered by generative AI, and then execute tasks to complete well-defined tasks efficiently.

That's why intelligent agents show up everywhere today, from conversational agents in customer service to workflow agents automating business processes. But most examples assume cloud-based AI systems.

Platform engineering lives in a very different world.

Why Platform Engineers Hit a Wall with AI Agents

In theory, autonomous AI agents are perfect for platform work. In practice, most teams can't deploy AI agents safely.

Why?

Because traditional software and many pre-built AI agents assume:

• Always-on internet access
• External APIs are acceptable
• Data can freely leave the cluster

In finance, healthcare, government, and other real-world applications, those assumptions collapse quickly. Human oversight, auditability, and minimizing human error matter more than novelty.

That's where running AI agents on-prem, inside Kubernetes, starts to make sense.

How AI Agents Work (Without the Marketing)

Under the hood, AI agents analyze information, make decisions, and take action. Large language models help with reasoning, but the real value comes from an agent's ability to use external tools.

That might mean querying databases, inspecting infrastructure, interacting with management systems, or reading documentation. Some agents operate as simple reflex agents, reacting immediately to inputs. Others are model-based reflex agents, maintaining internal state and context.

More advanced AI agents include goal-based agents, utility-based agents, and learning agents that adapt over time. These sophisticated AI agents rely on feedback, memory, and sometimes human intervention to operate safely in dynamic environments.

In complex scenarios, multiple AI agents work together in multi-agent systems, where multiple specialized agents coordinate with other AI agents and even human agents to complete complex tasks.

At this point, this starts to look very familiar to anyone building distributed systems.

Types of AI Agents and Why Control Matters

There are many agent types, each suited to different workloads.

Some AI agent types handle routine tasks. Others are goal-based agents that plan ahead. Learning agents improve over time by learning from experience and maintaining context across past interactions.

Many modern AI solutions rely on multiple agents working together in multi agent systems. These compound AI systems can automate complex workflows and deliver significant cost savings — but only if they are controlled.

For platform engineering, the challenge isn't capability. It's control.

You don't want autonomous agents that can do everything.
You want intelligent systems that operate independently within clear boundaries.

That's the philosophy behind kagent.

AI Assistants vs AI Agents (Why the Difference Matters)

AI assistants are designed to support human employees by responding to prompts and helping human users perform tasks. They're useful, but they usually depend on direct human input.

AI agents go further.

Advanced AI agents can make decisions, interact with external systems, automate complex tasks, and execute tasks without constant human oversight. Instead of assisting with individual actions, an AI agent performs workflows end to end.

For platform teams, this distinction matters. AI assistants answer questions. AI agents own processes.

kagent: AI Agents That Belong Inside Kubernetes

kagent is an open-source framework built specifically for DevOps and platform engineers who want to build, deploy, and run AI agents without violating security policies.

With kagent, you can deploy AI agents entirely inside Kubernetes:

• On-prem
• Offline
• Fully under your control

Agents, models, tools, and memory are Kubernetes resources. You define them declaratively, version them with Git, and manage them like any other infrastructure component. You can use hosted models or an internal model, depending on your environment.

This isn't AI bolted onto Kubernetes.
It's agentic AI designed for Kubernetes.

According to Gartner, by 2026, 80% of engineering organizations will operate platform teams as internal service providers. AI agents that can't live inside the platform won't survive that shift.

Demo: Running AI Agents Fully Inside Kubernetes

In the video, I demonstrate running AI agents end to end using kagent.

I start with a local Kind cluster, but the same setup works with K3s or an on-prem Kubernetes cluster. After installing kagent with Helm, the controller, dashboard, and CRDs are deployed.

From there, the demo focuses on three things.

First, I configure a model. kagent abstracts the model layer, so you can switch between external providers or an internal model without changing how agents are defined.

Second, I interact with a Kubernetes AI agent. I ask real operational questions about namespaces, pod health, and available APIs. The agent analyzes live cluster data using Kubernetes-native tools, not guesses, and explains the results clearly.

Finally, I show building AI agents declaratively using YAML. I define what the agent can do, which tools it can use, and how it should behave. I also add an MCP tool that lets an agent fetch documentation on demand.

Everything runs inside the cluster. Nothing leaves unless you explicitly allow it.

That's the difference between a demo and a platform-ready AI system.

Learning Agents and Complex Workflows

Learning agents improve over time by storing context in short-term and long-term memory. In platform environments, that means agents that learn common failure patterns, improve troubleshooting accuracy, and reduce repeated human intervention.

With multiple agents, you can automate complex workflows like incident response, documentation discovery, and system analysis. Each agent performs a focused role, while the system as a whole behaves like an intelligent platform component.

kagent keeps these systems observable and auditable by design.

Deploying AI Agents Without Losing Sleep

Deploying AI agents is both a technical and operational decision.

You need to know where the agent runs, what data it can access, what actions it can take, and how it behaves when something goes wrong. kagent supports cloud, hybrid, and on-prem deployments while keeping behavior consistent.

That consistency builds trust, and trust is non-negotiable when automating critical systems.

Final Thoughts

AI agents don't need the cloud.

They need clear boundaries, strong defaults, and platforms that respect reality.

kagent doesn't chase hype. It gives platform engineers a way to use AI agents: multiple agents, multi-agent systems, and intelligent agents, inside the environments they already trust.

Inside Kubernetes.
Inside policy.
Inside the real world.

Once you see that working, it's very hard to go back.

Code