“Implemented AI in DevOps” can sound like a résumé phrase with more hype than substance. It does not have to be.

A useful first implementation does not require an enterprise AIOps platform, a large budget, or months of integration work. A small team can connect existing observability data, source-control context, an affordable language model, and the chat tool people already use.

The goal is simple: a developer asks an infrastructure question in plain language, and the system gathers evidence before answering.

For example:

  • What caused the latency spike last night?
  • Which merge request could have affected this service?
  • Why did this pipeline fail after the dependency update?
  • Which deployment happened before the error rate increased?

Instead of opening several dashboards and repositories, the engineer starts with one question.

The minimal working architecture

The first version needs five parts.

1. Observability as a data source

Connect Grafana, Datadog, or another observability platform through an MCP server or a narrowly scoped API integration. The model needs access to the metrics, logs, alerts, and traces relevant to the question.

The integration should be read-only. Start with a small number of approved queries and data sources. The purpose is investigation, not autonomous production control.

A good first scope might include:

  • service error rate and latency;
  • recent alert history;
  • logs for a selected namespace or service;
  • deployment annotations;
  • resource saturation for CPU, memory, disk, and GPU workloads.

The model does not need every metric on the first day. It needs enough context to connect symptoms with events.

2. Source control and delivery context

Connect GitLab or GitHub so the system can inspect repositories, merge requests, pipeline results, issues, and deployment history.

Observability tells you what the system did. Source control helps explain what changed.

When these two sources are available together, the assistant can compare a spike in errors with a deployment, inspect the related change, and identify whether the evidence supports a connection. It should present that as a hypothesis, not as certainty.

Use a dedicated read-only account or token with access only to the required repositories. Do not give the first prototype permission to merge code, restart workloads, change infrastructure, or modify alerts.

3. An affordable language model

Start with a cost-efficient model rather than the largest available option. Claude Haiku, DeepSeek, or a suitable self-hosted model can handle log summarization, code context, and structured investigation surprisingly well.

The model is not the source of truth. Metrics, logs, traces, repositories, and pipeline records are the evidence. The model’s role is to decide which evidence to request, connect information across sources, and explain the result.

This distinction matters because fluent answers can still be wrong. Every response should show the observations that support its conclusion and clearly label uncertainty.

Inference costs money, but a controlled workflow can cost far less than a dedicated enterprise tool or the hidden cost of repeatedly interrupting senior engineers.

4. A system prompt that describes the environment

A generic assistant does not understand your infrastructure. Give it a map.

The system prompt should describe:

  • environments and naming conventions;
  • where metrics, logs, and traces live;
  • how services map to repositories;
  • how deployments are identified;
  • which dashboards and queries are authoritative;
  • what the assistant may and may not do;
  • how it should cite evidence and express uncertainty.

This is not a replacement for documentation. It is a compact operational contract that teaches the assistant how to navigate the available tools.

Keep the prompt in version control. Review changes. Treat it like code because it influences production investigations.

5. Slack or Discord as the interface

Use a Slack or Discord bot or webhook as the first interface. The team already works there, so adoption does not require another dashboard.

The flow becomes:

  1. An engineer asks a question in chat.
  2. The assistant identifies the services and time range involved.
  3. It queries observability and source control through approved tools.
  4. It processes the returned evidence.
  5. It answers in plain language with links to the relevant dashboard, pipeline, or merge request.

The best interface is often the one that does not ask people to change where they work.

What the result should look like

A weak answer says: “Latency increased because of a deployment.”

A useful answer says:

API latency increased from 220 ms to 1.8 s between 01:12 and 01:29 UTC. Deployment payments-api-847 completed at 01:08 UTC from merge request 1421. The change introduced an additional database lookup in the checkout path. Database connection usage reached 94% during the same interval. This is a correlation, not proof; compare the query profile before and after the deployment.

The second answer is valuable because it exposes the evidence, links related events, and states its confidence honestly.

Why this changes team behavior

The first benefit is not full automation. It is fewer interruptions.

On-call engineers receive an explanation of anomalies instead of only a raw alert. Developers can investigate failed pipelines without immediately messaging the platform team. New teammates can ask how services and deployments relate while they are still learning the environment.

In a well-scoped implementation, a large share of recurring questions can be answered automatically or at least prepared with the right evidence before a human joins the investigation.

That protects focus. It also makes infrastructure knowledge more accessible to the whole engineering team.

Guardrails for the first version

Keep the first implementation deliberately limited:

  • read-only tools;
  • explicit repository and observability scopes;
  • no credentials in prompts or model-visible logs;
  • evidence links in every answer;
  • clear uncertainty labels;
  • request and response logging with sensitive-data filtering;
  • rate limits and cost limits;
  • a documented escalation path to a human engineer.

Do not begin by letting an LLM restart production workloads or modify infrastructure. The fastest path to trust is a system that investigates well and knows the boundaries of its authority.

Two hours can be enough for the first useful prototype

The initial prototype can be small: one observability source, one repository, a low-cost model, a focused system prompt, and a chat integration.

That is enough to demonstrate a permanent shift in how the team works with infrastructure. The assistant becomes a shared investigation layer between operational data and engineering context.

The résumé line then becomes concrete. You did not merely “use AI.” You designed a controlled workflow, connected real production evidence, reduced interruptions, and made incident context accessible in plain language.

Start narrow. Keep it read-only. Make every answer traceable to evidence. Then expand only when the team trusts the results.