My Idea Guy My Idea Guy My Idea Guy My Idea Guy My Idea Guy My Idea Guy My Idea Guy

Computer vs. Copywriter in A/B Split Testing Situation

Split Testing SoftwareThis headline got me thinking…

“CAN A COMPUTER BEAT A COPYWRITER? “Impossible!” Said One of the World’s Top Copywriters Until He Got His Butt Whipped By An Uncanny New Software Program Called “Glyphius!”

That was the headline for a new piece of software that came highly recommended to me from Michel Fortin (arguably the world’s best online copywriter).

So I decided to give it a try with a real world experiment.

Split testing your headlines, sub headlines, guarantees, bullets and a variety of other elements are crucial if you are looking to get the maximum performance from your marketing materials.The problem is, most of us don’t have the patience for it.

That’s why I was intrigued by this piece of software because it proclaimed it could predict the winning outcome BEFORE you even send any traffic to your site!

You can see the entire salesletter for this software at www.TestYourAds.com

Initially I would have been very skeptical but like I said, it came highly recommended from Michel and he shared with me that he has run a bunch of tests to see if the software was correct and apparently it came very close if not exactly right on.

So the question popped in my head - you mean instead of having to go through the painful process of split testing I can just plug my different ads or headlines into this software and it will tell me which ones will do the best?

Yup, that’s exactly what it proclaims to do.

Here’s a screen shot:

split testing software

However, through my own tests, I found something that doesn’t quite jive.

I’ve been running a naming contest for my upcoming membership site and we’ve had hundreds of people vote on the name they like the best.

But here’s the problem…

The names that are winning didn’t receive the highest score in the split testing software.

Here’s a look at the score that each name recieved from the software:

Leading Edge Web Tools - Score: 81
Best Web Tools Explained - Score: 56
Your Research Team - Score: 54
Web Tech Explained - Score: 44
Business Tools Revealed - Score: 23
Business Tool Trainer - Score: 21
Secret Business Tools - Score: 14
My Web Tool Guy - Score: 11
Monthly Business Tools - Score: -33
My Idea Tool Box - Score: -102

Therefore this split testing software projected that the top name would be “Leading Edge Web Tools” with a score of 81.

Here’s what’s ironic…

As I write this, “My Idea Tool Box” is the number one vote getter in the contest but it is scored the lowest according to the software with a score of -102!

So what does all this mean?

Well there could be a number of factors contributing to this and the major one is the fact that there is a contest for the name of my new membership site. This could obviously skew the results because the people involved in the finals could be getting everyone they know to vote for them regardless of whether they actually like the name or not (I’m not suggesting that is happening but it’s a definite possibility).

Another factor could easily be the fact that the results from the poll are shown on the blog and as Cheryl mentioned in the comments under the poll, this could skew the results as well because people are influenced by the current standings versus making a judgment based on their true opinion.

Lastly, I personally think that this split testing software works much better for longer headlines or ads and not neccessarily for product names.

Here’s how the software works (taken from the site at www.TestYourAds.com)

“…contains a built-in database of over a hundred thousand profitable and unprofitable ads. It allows you to score your own ad copy against this database for profitability.

It’s really simple. You just copy/paste a headline, sentence, order link, Adwords ad, resource box or any other copy that is meant to sell into the edit box in Glyphius. Once you do that, you click the “Score” button. In milliseconds, thousands of words and punctuation are compared to the internal database and scored. As far as you can tell… INSTANTLY, you see the score right under where you typed.”

So if you are testing product names I think TestYourAds is a start but there are probably some other factors you may want to include.

With that said, I did find the software extremely useful when I was generating headlines for another salesletter I was creating just to get a quick glance on the scores that each would receive.

It does an excellent job of pointing you in the right direction by helping you narrow down your top 3-5 that you want to put through a real world split test.

However, ultimately, you’ll never know the true result of a split test unless you actually conduct the split test :)

Let me know your thoughts in the comments below.


***********************************

About The Author
***********************************
Stu McLaren is a sought after marketing consultant for small business owners. He specializes in helping entrepreneurs leverage the internet in their marketing activities and is well known for his expertise in building and managing highly profitable affiliate programs.

***********************************
Other Websites of Interest:
***********************************
IdeaSeminar.com
MarketingWebGraphics.com
GetStartedWithInternetMarketing.com
GetStartedWithInfoMarketing.com
BasicListBuilding.com
Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • bodytext
  • del.icio.us
  • blinkbits
  • BlinkList
  • blogmarks
  • Bumpzee
  • Netscape
  • NewsVine
  • Reddit
  • Slashdot
  • Spurl
  • StumbleUpon
  • Technorati
  • YahooMyWeb

RSS feed | Trackback URI

9 Comments »

Comment by mrsurftheweb
2007-05-25 12:57:40

Well I think the top computer picked name is the most descriptive for sure. For the bottom computer pick I would make it “My Idea WEB Tool Box”

 
Comment by Gavin
2007-05-25 13:23:07

Your not really comparing apples with apples.

A headline I think is a different animal to a product name, maybe that’s why the results aren’t as expected.

Gavin
http://www.OutsourceSuccess.com

 
Comment by Gas Calculator
2007-05-25 15:46:46

I think the software might be right. Dan Kennedy says the product name should spell out the benefits of the product if possible. Everyone talking about your product will use the product name, if it’s also a great sales headline people will market your product when they talk about it.

 
Comment by tileart4you
2007-05-25 18:19:54

I think the software did pick the most descriptive name. I like a combination of the first and last place names, like Leading Edge Web Tool Box.

 
Comment by kenny
2007-05-26 00:48:33

Hi Stu,
Interesting post…
I think the one difference is that the software doesn’t know that you have ‘branded’ yourself as ‘my idea guy’.
How does that score in the software?
So, when you ask your audience what will suit your member site - they think that a name that matches your branding will suit best.
In this experiment, I would choose your subscribers voting - not the software.
My thoughts…
Kenny

 
Comment by John
2007-05-26 09:30:57

I agree completely with Gavin’s comments.

What you need isn’t a successfull ads database, but a branding engine, because that’s what you’re doing.

One of the best ways I’ve seen this done is with an ‘ask campaign’ that elicits the customers ‘preception’ of what your product will be.

For example, “When you think of Web 2.0, what images come to mind?”
or, “What do you picture in your mind when you think of a website that provides new tools for use in your business?”

If you were to put these types of questiosn out there, I think that would explain why the ‘worst scoring’ name actually resonates best with yoru customers.

 
Comment by Stu
2007-05-27 21:04:43

Thanks everyone for your thoughts - very insightful.

Kenny - that’s a really good point and I know for sure that the software does not take those kinds of things into consideration.

John - I really like the two questions you gave because it’s not asking the customer to come up with any “ideas” but rather asking them to describe how they would describe the product.

I can see how that would be extremely effective for finding the language they use when talking about the product - brilliant!

Thanks everyone - I’m really enjoying this discussion :)

 
Comment by fastcart
2007-05-31 22:17:23

LOL…”vote getter”…from what i remember, the glyphius data is based upon profitability. And you are judging your results based upon what people claim to like. What they say they enjoy and what they will whip out their credit cards for are far different things.

Just think, how many people do you know who claim to buy porn? Probably none. Yet, somebody out there is spending $3.5Billion on it online last year. That guy musn’t get out of the house much…

 
Name
E-mail
URI
Subscribe to comments via email
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> in your comment.

Trackback responses to this post