What Does an AI Engineer Do? Role, Skills, Salary and Path

Three years ago “AI engineer” mostly meant a machine learning researcher with a different business card. Today it’s a distinct job with its own skills, its own tooling and its own interview loop, and it’s the fastest-growing engineering title in most job markets. It’s also badly explained, because the role is new enough that half the listings were written by people who aren’t sure what they’re hiring for. This page is what the job actually is, how it differs from the roles next to it, what it pays, and how to get into it.

The short answer: An AI engineer builds products on top of foundation models. Rather than training models, they take a model through an API or an open-weights checkpoint and engineer everything around it:

• Prompts and structured outputs.
• Retrieval, so the model can use company data.
• Tool use and agents, so it can take actions.
• Evaluations, so quality can be measured.
• Production plumbing for latency, cost, safety and observability.

It’s product engineering where one component behaves probabilistically. It’s distinct from a machine learning engineer (who trains and serves models) and a data scientist (who analyses data and builds predictive models for decisions), pays a premium over both at most companies, and is reachable from a general software engineering background without a research degree.

Where the role came from

Until 2022, building an “AI feature” meant collecting labelled data, training a model, serving it, and monitoring it: a machine learning project, measured in quarters, done by ML engineers and data scientists. Large language models changed the economics. A general-purpose model behind an API could do classification, extraction, summarisation and generation without training, and a product engineer could ship a first version in a week.

That created a new kind of work. The model was no longer the hard part; the hard parts were getting the right context in front of it, making its output reliable enough to ship, measuring whether a change made things better or worse, and running it at a cost and latency the business could live with. Those problems look like software engineering, not research, and the people solving them needed a name. “AI engineer” stuck.

What an AI engineer actually does

A typical week at a product company includes most of these:

• Prompt and output design. Writing and versioning the instructions a model gets, defining structured output schemas so the result can be used by code, and designing tool schemas so the model can call functions reliably. This is less “prompt engineering” as a trick and more API design with a probabilistic component.
• Retrieval. Building the pipeline that gives the model the right information: chunking documents, generating embeddings, storing them in a vector or hybrid index, retrieving and reranking at query time, and deciding what goes into the context window. Retrieval-augmented generation (RAG) is the core of most enterprise AI features, and most of the quality problems live here.
• Agents and tool use (what is an AI agent). Building systems where the model decides which tools to call and in what order: search, database queries, internal APIs, code execution. The engineering is in the loop around the model: state, retries, guardrails, limits, and knowing when a plain pipeline is better than an agent.
• Evaluation. Writing evals: sets of real inputs with expected outputs or grading rubrics, run automatically on every change. This is the part that separates people who ship reliable AI features from people who ship demos. It includes choosing metrics, building golden datasets, using models as graders, and catching regressions.
• Production engineering. Latency budgets, streaming, caching, batching, rate limits, fallbacks when a provider is down, cost per request, logging and tracing of model calls, and privacy (what goes to which provider, what’s retained).
• Safety and guardrails. Input and output filtering, prompt injection defences for anything that reads untrusted content, refusal handling, and human-in-the-loop for high-stakes actions.
• Model selection and fine-tuning. Comparing models on the team’s evals, deciding between API models and open weights, and occasionally fine-tuning a smaller model when the task is narrow and volume is high. Most AI engineers fine-tune rarely; the ones at AI-native companies do it more.
• Working with product. Deciding what the feature should do when the model is uncertain, designing the UX around latency and errors, and being honest about what the model can’t do yet.

What they usually don’t do: train foundation models, do research, build the data platform, or run the company’s classical ML models for fraud, ranking or forecasting. Those belong to research, data engineering and ML engineering.

AI engineer vs. ML engineer vs. data scientist vs. software engineer

Role	Core question	Works with	Typical output
AI engineer	How do we build this into a product with a model?	Model APIs, retrieval, agents, evals, product code	A shipped feature that uses a model reliably
ML engineer	How do we train and run this model at scale?	Training pipelines, feature stores, serving, monitoring	A model in production with drift monitoring
Data scientist	What does the data say we should do?	Notebooks, statistics, experiments, dashboards	An analysis, a prediction model, a recommendation
Software engineer	How do we build this product?	Application code, APIs, databases, infrastructure	Features and systems

The lines blur at small companies, where one person does two of these. At larger companies they’re separate teams, and the AI engineer is usually closest to the product team.

Why the role is growing so fast

• Every product is adding a model. Search, support, coding tools, document workflows, sales tooling, internal knowledge. Each one needs someone who can make it work reliably, and that’s a different skill from calling an API once.
• The demo-to-production gap is wide. Getting a model to do something impressive takes an afternoon. Getting it to do that thing correctly 99 percent of the time, at acceptable cost, without leaking data, takes an engineer. Companies learned this the hard way in 2024 and 2025.
• The tooling keeps changing. New models, new context lengths, new agent frameworks, new evaluation approaches every quarter. Companies need people whose job is to keep up.
• It pays for itself. An AI feature that works is visible to customers immediately. Few engineering roles have that clear a line from work to revenue.

Who hires AI engineers

• AI-native startups building products where the model is the core: coding assistants, agents for specific industries, search, voice. The most demanding roles and usually the best paid.
• Product companies adding AI features. SaaS, fintech, healthcare, legal tech, e-commerce. The bulk of listings, and the ones most reachable from a general engineering background.
• Model providers and infrastructure companies, for developer relations, solutions engineering and applied work. Often titled “applied AI engineer” or “forward deployed engineer”; what is a forward deployed engineer covers that variant.
• Enterprises and consultancies building internal tools and customer-facing assistants, usually RAG-heavy work over document collections.
• Agencies that build AI features for clients, which is a good entry point because you see many problems fast.

Titles that mean roughly the same job: AI engineer, applied AI engineer, LLM engineer, generative AI engineer, AI product engineer, AI software engineer, member of technical staff (at AI labs, which covers everything).

Skills and background that get you hired

Technical, in the order most listings weight them

• Python, properly: async, typing, packaging, testing. TypeScript is increasingly required for product-side work.
• Model APIs: streaming, structured outputs, tool calling, handling context limits, retries and fallbacks across providers.
• Retrieval: chunking strategies, embedding models, vector and hybrid search (Postgres with pgvector, Elasticsearch, a managed vector database), reranking, and how to evaluate retrieval separately from generation.
• Evaluation: building a golden set, choosing metrics, model-graded evals, regression testing on every change. This is the skill that most distinguishes strong candidates.
• Agent patterns: tool loops, planning, state management, and the judgement to use a plain pipeline when an agent isn’t needed.
• Production: observability for model calls (tracing, token accounting), caching, cost control, latency budgets, and security including prompt injection.
• Data handling: enough SQL and data engineering to build the pipelines that feed retrieval.
• One framework, deeply. LangChain or LlamaIndex or the provider’s own SDKs, plus the honesty to say when not to use a framework.

Judgement

• Knowing what models are good and bad at, and designing around the failures rather than hoping they won’t happen.
• Being able to say “this shouldn’t be an AI feature” when it shouldn’t.
• Reading a model’s output critically instead of being impressed by it.

Background that works

• Backend or full-stack engineers who built one real AI feature. The most common path.
• ML engineers moving toward product work.
• Data scientists who can write production code.
• Fresh graduates with a strong portfolio: a RAG system over a real corpus with evals, an agent that does a real task, and a write-up of what went wrong and what you changed. Portfolios matter more in this role than in most.

What AI engineers earn

• United States: roughly $110,000 to $130,000 at entry level, $145,000 to $180,000 mid-level, and $180,000 to $250,000 and above for senior engineers, with total compensation substantially higher at AI-native companies and labs. Engineers focused on LLM application work earn a premium of roughly a quarter to a third over generalist ML engineering at the same level.
• Europe and UK: roughly £60,000 to £130,000 base, with London and remote-US roles at the top.
• India: listings range from around 12 lakhs for junior roles to 40 lakhs and above for senior engineers at funded startups, with AI-native companies and US-funded startups paying well above that.

Pay goes up with demonstrated production experience: a feature you shipped, with numbers on quality, cost and latency. It goes down for candidates whose experience is prompt-and-demo only.

Is it the right job for you?

It suits you if:

• You like product work and want a short loop from code to something users see.
• You’re comfortable with systems that are probabilistic, where “it works” is a measurement, not a boolean.
• You enjoy a field where best practices are still being written and you’ll have to read papers, changelogs and other people’s failures.
• You can be sceptical about your own output.

It doesn’t suit you if:

• You want to do research or train models; that’s ML engineering or a research role.
• You want stable tooling and a settled way of doing things.
• You dislike evaluation work; it’s a large part of the job and it’s not glamorous.
• You want to be far from users and product decisions.

How to get in

1. Build one real thing end to end. A retrieval system over a real document set (your company’s docs, a public corpus), with an eval set of 50 real questions, a measured baseline, and at least two improvements you can quantify. Or an agent that completes a real multi-step task with tools, with failure handling. Write up what broke.
2. Learn evaluation before frameworks. Anyone can wire a chain together. Being able to say “this change improved accuracy from 71% to 84% on our eval set and cut cost by 30%” is what gets offers.
3. Get production experience however you can. Ship an AI feature at your current job, even a small one. If you can’t, contribute to an open-source AI tool, or take a contract through an agency.
4. Read the model providers’ documentation and cookbooks properly. They’re the closest thing to a textbook, and most candidates skim them.
5. Prepare for the interview loop: a coding round (usually normal), a system design round for an AI feature (retrieval, evals, cost, failure modes), and a practical (build or debug something with a model). Be ready to explain a failure in detail.
6. Tailor the resume to each listing. AI engineer listings vary widely: some are RAG-and-enterprise, some are agents-and-product, some are ML-adjacent with fine-tuning. Lead with what each one names; Tailr does this from the listing you’re viewing.

How to tailor your resume for an AI engineer role

Read the listing for its centre of gravity. If it says RAG, retrieval, documents and enterprise, lead with retrieval work and evals. If it says agents, tools and workflows, lead with an agent you built and how you kept it reliable. If it mentions fine-tuning, inference or open weights, lead with model work and infrastructure.

Quantify every AI project in three dimensions: quality (on what eval, from what to what), cost (per request or per month), and latency. “Built a RAG assistant” is a hobby project on a resume; “built a RAG assistant over 40,000 support articles, lifted answer accuracy from 62% to 86% on a 200-question eval set, at $0.03 per query and p95 under two seconds” is a hire.

Tailr does this from the job listing itself: it tailors your resume to the specific AI engineer role you’re viewing, so the retrieval, agent or model work that listing cares about is what your resume leads with, and generates a matching cover letter. How to tailor your resume to a job description walks through the method, and best AI coding tools for developers covers the tooling most AI engineering teams use day to day. Try Tailr on the next AI engineer listing you open.

Related guides

• How to Get Started With AI Engineering: A Beginner’s Roadmap
• What Is AI Engineering? A Plain-English Explainer
• Top 30 AI Engineer Interview Questions and Answers
• What Is an LLM? Large Language Models Explained Simply

Conclusion

An AI engineer is a product engineer whose main component is a model: they build the retrieval, the tool use, the evaluation and the production plumbing that turn an impressive demo into a feature that works. The role is distinct from machine learning engineering and data science, it’s growing faster than any other engineering title, it pays a premium, and it’s reachable from a general software background if you build one real system with real evals and can talk honestly about what broke. Learn evaluation first, ship something to production, and tailor the resume to what each listing actually wants.