No sensible company hires someone without a job ad, an interview and a probation period. You don’t hand a new person the keys to the whole office, the company bank login and a to-do list titled “sort everything out” on day one.
With AI agents, that is exactly what companies do. They buy a licence, plug it into the inbox and wait for a miracle.
Then comes the disappointment. It’s not just my impression. MIT NANDA’s August 2025 report found that about 95% of corporate generative AI pilots delivered no measurable return. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, mostly because of rising costs and unclear business value.
I have been running IT projects for more than ten years and I know this pattern. The technology is rarely the problem. The problem is that nobody wrote down what the “new hire” is actually for. So I treat AI agents like recruitment. Five steps, no magic.
1. The job ad: one job, not “AI for everything”
A good job ad describes a specific role. “Specialist in everything” is a red flag for people and for agents. Instead of “we will bring AI into customer service”, write: “the agent answers questions about order status, using data from the sales system, when the office is closed”.
A simple test: if you can’t describe the job in two sentences, the agent won’t understand it either.
2. The interview: test it on real cases
You don’t judge a candidate by the CV, you judge them by how they solve a task. Take 20–30 real cases from the last few weeks together with the answers a person gave. Give them to the agent and compare. You’ll see where it shines and where it makes things up with a straight face.
It takes one afternoon and saves months of wishful thinking. It also answers an important question: do you even have the data the agent needs?
3. Onboarding: only the keys it really needs
A new person gets procedures, examples of good work and access to two or three systems, not to everything. Same with an agent. If it checks order status, it doesn’t need permission to issue invoices or email your whole customer list.
The best instructions for an agent look like good instructions for an intern: short, specific, with examples and a clear “if you’re not sure, ask a human”.
4. A manager: a named person who owns the result
Every new hire has a manager. So does an agent. Not “the IT team”, but a named person who reviews its work, fixes the instructions and decides when it gets more independence.
At first the agent drafts and a human approves. When corrections stay rare for a few weeks, you can let it handle the simpler cases on its own. Trust grows with data, not with post-demo excitement.
5. The 90-day review: three numbers and one decision
After three months you sit down for a review. You don’t ask “is this AI cool?”. You look at three numbers:
- how many tasks the agent completed per month,
- what one task costs, counting licences, tokens and the owner’s time,
- what share of its work a person had to correct.
Then you make one of three decisions, just like with a person after probation: extend the contract, change the scope or part ways. Each of them is fine. The only failure is the fourth option: the agent keeps running because nobody checked whether it pays off.
The best agent is a boring agent
The agents that really pay for themselves rarely impress anyone in a demo. They sort documents, update the CRM, draft replies, check that an invoice matches the order. One job, every day, no holidays, no complaints. That is exactly why they pay off.
If you’re wondering where to start, start with step one. Take a sheet of paper and describe, in two sentences, one task that eats a few hours of someone’s week. We can go through the other four steps together on a 20-minute call.
