Internal AI · Lifecycle
Agent-operated push channel
The problem
We pay for a marketing automation tool that has to be operated by a person, by hand, and it is expensive. As a result, sends went out when someone felt it was time, to an audience that seemed right that day. There was no real cap on how often the same person heard from us. Whether a send had worked was decided afterwards by looking at that week's numbers and telling a story about them.
What I built
An agent runs the channel. I built it because I want to replace the marketing automation tool we pay for. It can learn from what happened, and personalise further than a person setting up campaigns by hand. Who gets a send is defined in SQL, so it is a query someone else can read and disagree with, and frequency caps are enforced in the send path. Every send holds out 10% of the audience, and lift is measured against that control group.
If this were a guideline, I would drop the cap or the holdout the first time I wanted a send out fast.
What changed
We can say whether a send did anything. A campaign that doesn't beat its holdout doesn't get to claim it worked, which has killed a few ideas I was quite fond of (I still miss a couple of them).
What I'd do differently
I would make the holdout impossible to turn off. In hindsight, it is the first thing someone drops when they are in a hurry to get a send out, and then you are back to reading the week's numbers and telling a story.