Most AI tools sell you the same promise. Set it and forget it.
A week ago I actually tested it. I gave an AI agent a real job with real rules, put it on a schedule, then closed my laptop and went and lived my life. I didn't open it again until the following week.
The work was done and waiting. Here's what it did, and just as importantly, the one place it stopped cold.
Up front: I'm partnered with Doba. The tests and the verdict here are my own.
The job I gave it
The agent is Doba Pilot, an AI agent for dropshipping. The skill I wanted to test is scheduled tasks, because the whole point of automation is that it runs whether you're paying attention or not.
I wrote the task the way I'd write it for a person: find new pet supplies on Doba that match my conditions, under $30, at least 25% profit margin, and in stock. Push the matching products to me.
That's it. Plain English, no prompt engineering.
I have pet supply stores because I've got a husky and I know what people will spend on their animals. I already did the product research once. But trends move, suppliers change, and margins change. Research you did once and never touched again goes stale. So I set it to run every Monday at 9 a.m. and let the list refresh itself.
Set it up on a Friday. Walked away.
What was waiting
Monday at 9 a.m. it ran. By the time I went and looked, it had nine products lined up.
The useful part wasn't the count. It was that every product came with the research already attached.
First it showed me why the product cleared my rules at all: right category, right price range, in stock, high profit margin. Then it showed me the part I actually care about, which is the supplier behind it.
The three supplier numbers I read first
If you're new to this, these are worth stealing.
- How long the supplier has been open. Three months is a little risky. I don't know anything about that supplier yet.
- Fulfillment rate. Anything over 95% I think is pretty good. One supplier came back at 100% with 3.6 years open, which is pretty incredible. That's one I'd really be interested in.
- Refund rate. This is the one beginners skip. One came in at 1.22%, which is higher than I normally want to see. Another had been open a year and a half at 0%, which is better. A third was at 3.45%, and that's a little concerning, even though I liked their fulfillment rate.
You can pull all of that yourself. It just takes a long time, per product, every week, forever. That's the part I handed off.
The "failure" that wasn't
I re-ran the task live so I could show it working, and it came back marked Failure.
I clicked in, and the reason was that there were no new results. I'd run it a couple of days earlier, so nothing had changed since. Not a real failure. Just nothing new to report.
Worth knowing before you see that word and assume something broke.
Where it stopped
I sell a fair bit on eBay, so I figured I'd have the agent connect my eBay store. It can't. It told me how to connect it instead, which means that job goes back on my list and I go do it myself.
That's the honest limit on every agent right now, not just this one. They don't have access to your stuff.
When you ask an agent to do something involving your accounts, there's a step where you have to go do it yourself. I'd rather know that up front than find it out on a deadline. An agent that hands you accurate instructions is still saving you time. It just isn't doing the clicking.
What's actually live
Doba Pilot now has a skills page, which is the whole product written out as a list of jobs you can hand over. 22 skills in four groups: connecting to a store, finding products, writing the listings, and running the day to day.
Live right now: Shopify, Wix, BigCommerce, eBay, Walmart, TikTok Shop. Amazon and Temu are listed as coming soon.
Two of the recommended scheduled tasks are where I'd point a beginner before anything clever. A shipment watch that monitors shipping exceptions so you know your orders actually shipped correctly. And a canceled order guard, so an unshipped Doba order gets canceled when your customer cancels theirs. Both are boring. Both are the kind of thing you forget about until it costs you.
One note on cost: re-running that report was one credit, so it isn't unlimited. The trial is 99 cents for 7 days on a new account, which is cheap enough to just go find out for yourself.
Try Doba Pilot: https://www.doba.com?mtic=MT-GNRUCN
7-day trial for $0.99 (new accounts). Code FIKbVPDgqYvv gets you 10% off a paid plan. It doesn't apply to the $0.99 trial.
The takeaway
Give an agent a job you'd be comfortable giving a person. Write the rules the way you'd say them out loud. Then go check what it actually did, including where it stopped.
That last part is the whole skill.
If you want the first test, where I pointed the same agent at a niche and it told me not to enter it, that one is here: I Asked an AI Dropshipping Agent to Research a Niche. It Told Me No.
I'm partnered with Doba and I make videos for them regularly. The tests, the results and the verdict here are my own, including the parts that aren't flattering.
