Why do executives sometimes reject winning test variants in favor of AI messaging?
Answered at 19:00 in How to Stop Guessing and Start Testing with Casey Hill, DoWhatWorks · The Content Cocktail Hour
Why do executives sometimes reject winning test variants in favor of AI messaging?
Executives and investors often push for AI messaging for optics during funding rounds, even when data shows non-AI variants convert better. The venture market has shifted significant investment toward AI companies, so leaders want their brand associated with AI regardless of test results. Casey recommends involving stakeholders before running tests and securing approval to implement data-driven changes rather than fighting decisions after results are in.
Full answer and detailsRead the whole answer, key facts and every other question from this episode
In short
Executives and investors often push for AI messaging for optics during funding rounds, even when data shows non-AI variants convert better.
Casey Hill, Chief Marketing Officer at DoWhatWorks · How to Stop Guessing and Start Testing with Casey Hill, DoWhatWorks · The Content Cocktail Hour · answer at 19:00
Key takeaways
- Executives and investors often push for AI messaging for optics during funding rounds, even when data shows non-AI variants convert better.
Quote to remember
“Avoid generic AI language and then also try to make sure that you tie in the outcome.”
— Casey Hill
The full answer · about 2 min
Executives and investors often push for AI messaging for optics during funding rounds, even when data shows non-AI variants convert better. The venture market has shifted significant investment toward AI companies, so leaders want their brand associated with AI regardless of test results. Casey recommends involving stakeholders before running tests and securing approval to implement data-driven changes rather than fighting decisions after results are in.
Key facts
- Only 11% of split tests beat their controls according to DoWhatWorks study
- Eight out of 10 times customer logos lose when tested head-to-head against alternatives
- Two CTAs consistently win over one CTA in hero section testing
- Approximately 70% of top 100 SaaS companies default to annual pricing displayed monthly
- DoWhatWorks database contains tens of thousands of tests across multiple industries
People, companies and topics mentioned
More answers from this episode
- 2:00What percentage of split tests actually beat their control according to DoWhatWorks data?
- 7:00Why do customer logos consistently lose in A-B tests on B2B SaaS websites?
- 10:00What is the positioning logic for customer logos on a B2B SaaS website?
- 12:00Should B2B SaaS websites display one or two CTAs in the hero section?
- 17:00Is generic AI messaging effective in converting website visitors?