How I Build AI Agents and Automations: The Parts of My Process That Never Change
My automation process gets rebuilt every few months and the tools turn over completely. These are the moves that survive every rewrite, from killing the idea early to letting feedback do the redesign.
My process keeps changing. These parts have not.
The first thing I ever automated took me hours and saved me about twenty minutes a week. I was proud of it for a day. Then I ran it a second time, the input arrived in a slightly different shape, and it produced garbage so confidently that I nearly used the output before I noticed.
That was the moment the lesson landed. I had automated the steps I could see and none of the thinking I was actually doing.
I have built multiple marketing automation gems, workflows and skills, for work and for my personal projects, and the tool is never the hard part. Knowing exactly what job you are handing over is the hard part, and most of us are much worse at describing our own work than we think.
The way I build has been rewritten several times since then, and the tools have turned over completely. What is left underneath is a handful of moves I make every time, mostly before anything gets built. Those are what this post is about.
I describe it by hand before I hand it over
I open a text file and write the task out the way I would explain it to someone joining next week. Every step, in the order I actually do them rather than the order that sounds logical.
If you have ever handed a task to someone new and watched them follow your instructions exactly and still get it wrong, you know what this finds. The instructions were never the job. The job included a pile of small judgments you stopped noticing you make.
When I wrote up how I pick prompts to track in a GEO tool, my first draft said “choose the ones with real buying intent.” That sentence hides about a dozen decisions. What counts as intent. How branded language gets treated against category language. When a prompt is too broad to deserve a slot. None of it existed anywhere but my head.
You cannot hand over a decision you cannot describe. Most failed automations are really failed descriptions.
I spend an hour trying to kill it
This is the one I would keep if I had to drop everything else. Before building, I argue against my own idea in writing.
Ten reasons this will not work. Not three, ten. The first three are the objections you already half know, and the interesting ones show up around number seven, once you have run out of easy answers. Then: what is wrong with the idea as I have described it, how the output could come out technically correct and still be useless, and what will block me that I have not accounted for.
That last one catches the most. Access I do not have. A rate limit. A field I assumed was in the export. If you have built one of these, you know the moment where the logic works beautifully and then you find out you cannot get at the one source it all depends on.
When I was scoping Let The Bots In, my site scanner, the objection that survived was about the output. Telling someone their site scores 62 is worth nothing on its own. It only earns its existence if it names the handful of things to change, which meant writing every check as a fix rather than as a score I would explain later.
I put the same questions to a model, and the framing is the whole trick: do not help me build this, give me ten reasons it fails. Left alone, an LLM will happily help you build whatever you propose, well enough that you walk away feeling validated. It argues the other side just as capably, but only when asked.
What this hour catches most often is an idea where the judgment is the entire job. Automate one of those and the work moves from doing to checking, which looks like leverage from the outside and feels like nothing from the inside.
I decide who is on the other end before I decide what it does
Output first, as a shape rather than a pipeline. A tab in a sheet. An email with three numbers in it. A score with five things to fix. Working backward from that artifact tells me which fields I need, which tells me which steps are real and which ones I invented along the way.
Then the question people skip: what does that person do in the minute after they open it? If the honest answer is that they copy three numbers into another tool, then that other tool is the real destination and half the build just disappeared.
Close behind is what they are technically comfortable with. The scanner forced me into this one. Whoever uses it wants to know whether AI crawlers can read their site, and a crawl log would be a perfectly accurate answer and a completely useless one. Same data, wrong format, dead product.
When the only user is me, the bar changes and I leave the rough edges in on purpose.
I sort the steps into mechanical and judgment
Now I go back through the write-up and mark every step as one of two things. Mechanical means there is one correct answer a machine can reach. Fetching a page, parsing it, checking whether a file exists, formatting, writing the output somewhere. Judgment means someone has to make a call a rule cannot fully capture, like whether a page actually answers the question it claims to.
Mechanical goes to code, which is cheap, predictable, and does not get creative at three in the morning. Judgment goes to a model, and the prompt for that step is mostly the paragraph I already wrote by hand. The description does double duty. It was documentation, and now it is the instruction.
Watch for the steps that feel like judgment until you write the rule down and find it is three conditions. Those move to the mechanical pile and the whole thing gets cheaper and steadier.
Whatever judgment I keep for myself becomes a review point later.
I build the version that can be wrong cheaply
The ugly version gets built the same day the notes are done. One page, ten checks, one input. If the idea cannot survive at that size, no amount of orchestration saves it at scale. Often I will have a model generate the finished artifact with fake data first, because looking at a fake output is the fastest way to notice you asked for the wrong three columns while changing your mind is still free.
This is also where the form settles, and it is smaller than I expect almost every time. Same path on every run is a script. Same path with one call in the middle is a script with a model in it, which is what most people mean when they say agent. A real agent earns its place when the next step depends on what the last step found.
Reaching higher than you need has a price. When a script breaks you read a line number. When an agent misbehaves you are debugging a decision, and anyone who has watched one loop politely into nonsense knows that particular afternoon.
I let the feedback do the redesign
The plan gets me to version one. Everything after that comes from use.
The prompt bot I built for myself is on its fourth or fifth version, and none of those changes came from planning. It kept answering before asking me anything, so I reordered the instructions to clarify first and build second. It refused things it had no business refusing, so that got rewritten. Every fix arrived through using it and being mildly annoyed.
When other people are involved I put the rough version in front of them and treat what comes back as the spec I could not write alone. This column is useless. I need it Monday, not Wednesday. Can it just tell me what changed. None of that is available at the planning stage, because nobody can describe what they want from something that does not exist yet.
One thing about asking. Do not ask whether it is useful, because people are polite and the answer is always yes. Ask what they did with it the last time it ran. The gap between those two answers is the next version.
Silence counts as feedback too. If nobody complains and nobody uses it, that is the loudest result you are going to get.
I make it tell me when it breaks
Two questions I used to skip, which is why a good number of my early automations are quietly dead.
The first is what a wrong output costs. Ten bad rows in a sheet I skim on Monday is nothing. Ten bad rows sitting live somewhere, shaping decisions for a quarter, is a different situation. The cost of a mistake is never spread evenly, so the review goes where the damage is.
The second is how I find out it stopped. I run my own stack on a small server and things there fail in the most boring ways available. A certificate lapses. A container restarts and one service does not come back. A credential expires. Nothing announces itself, and you find out weeks later when you go looking for an output and the newest file is from the twelfth.
So the last thing I add is the part that tells me it failed. A message when a run does not complete, or an output where empty is obvious at a glance. Ten minutes of work.
An automation that fails quietly is worse than no automation, because you were counting on it.
The notes file is what I actually keep
By the time something runs, the file that started as “here is how I do this by hand” has become the spec, the prompt and the documentation at once. Three months later it is the only reason the thing is repeatable. Code reads back fine. The reasoning behind it is gone unless you wrote it down while you still had it.
If you want to start somewhere, take the task you keep repeating and write down how you do it. Not for the automation. Just write it down. Half the time you find a step you can delete instead.
Every tool named in my notes from two years ago has been replaced. The pages themselves still hold up, and that is the only reason I trust any of this.