Multivariate Testing Success Stories

Ever wish you could find the perfect combination of landing page elelments that boosts your conversion rate? Well, we’ve got good news for you. Multivariate testing is a great way to test several page elements like different headlines, images and call-to-actions in hundreds of different random combinations on one landing page.

Here are three case studies that we hope will help convince you to start using multivariate testing. Click on the company name to read the whole case study. 

1. American Greetings

American Greetings creates innovative social expression products and they were looking to both decrease their cost-per-acquisition and increase their conversion rates. Through testing they accomplished both of their goals and more. On Mother’s Day alone, a real-time change in traffic based on test results resulted in over $45,000 in incremental revenue. That’s revenue that would have been lost without testing and real-time analytics! 

2. TEKsystems

TEKsystems is an IT recruiting agency and they needed a lower cost-per-conversion on their PPC ads. Anvil Media used LiveBall’s MVT and A/B testing featues to help decrease TEKsystems cost-per-conversion by more than 50%! 

3. Closed Loop Marketing

Closed Loop Marketing, an online marketing agency, uses LiveBall to create and test landing pages for their clients. For their client, Blurb, they launched an aggressive testing plan that included both A/B and MVT and nearly doubled Blurb’s conversion rate. 

We love seeing the great conversion lifts our customers get from using the code-free multivariate testing tools that are part of the LiveBall platform! If you are a LiveBall customer and aren’t using multivariate testing, give us a call and we’ll help you get started.

What kind of tester are you?

It may sound silly, but it’s good to know what kind of tester you are — kind of like your testing personality type. Just as investors have varying profiles and risk tolerance, so too do online marketers in testing roles. Organizations also tend to have testing profiles — predispositions or cultural norms that affect planning, documentation and reporting.

We’ve found that marketers exhibit characteristics that ultimately push them towards being rock stars, pragmatists or purists — and often combinations of all three. 

Are you a testing Rock Star, Pragmatist, Purist or a combination?

The History of A/B and Multivariate Testing

We all know that A/B and multivariate testing is important for successful landing pages, but did you know that these testing methods started back in the 1700’s with the answer for scurvy?

It’s time for a history lesson to explore how other types of testing have influenced the landing page testing methods we use and love today. 

The History of A/B and Multivariate Testing

It was 1747 and scurvy was a major problem for that time. A British Royal Navy ship surgeon, Mr. James Lind, started what would be the beginning of A/B and multivariate testing. By giving specific crew members different solutions, then testing the results of those variations over time, he came to the conclusion that citrus fruits were the cure to scurvy. 

Although many didn’t believe him at first, he went on to prove this and pioneer what we know in our testing methods, today.  When we use modern multivariate testing and A/B testing for landing pages, we should thank Mr. James Lind for our conversion rate increases.

Hypothesis Testing

If you’ve taken an introductory statistics course, this term may ring a bell for you. It’s statistical hypothesis testing, and it also is a forefounder to the A/B testing and multivariate testing methods we use for landing pages. 

Hypothesis testing is basically the product of three statisticians: Ronald Fisher, an agricultural statistician, and the mathematician/statistician team of Jerzy Neyman and Egon Pearson. Two differing methods created by these men in the early 20^th century have actually combined into a hybrid to become modern hypothesis testing.

According to Wikipedia, this testing method is:

A method of making decisions using data, whether from a controlled experiment or an observational study (not controlled). In statistics, a result is called statistically significant if it is unlikely to have occurred by chance alone, according to a pre-determined threshold probability, the significance level.

Just as in multivariate testing, this type of testing uses data gathered from a control and an uncontrolled subject. Depending on elements introduced, different results will occur. Again, for us this relates to how we can add elements, including change in headline, colors, CTA and others to improve our landing page conversion rates.

Likelihood Ratio Testing

Another creation of the Neyman/Pearson is likelihood ratio testing. The comparison of two models, “null” (control) and alternative, are the basis of this type of testing. A better description is quoted here, from Wikipedia:

In statistics, a likelihood ratio test is a statistical test used to compare the fit of two models, one of which (the null model) is a special case of the other (the alternative model). The test is based on the likelihood ratio, which expresses how many times more likely the data are under one model than the other. This likelihood ratio…can then be used to compute a p-value, or compared to a critical value to decide whether to reject the null model in favour of the alternative model.

Again, another method directly related to A/B testing and multivariate testing. Whether it’s multi or singular elements, we can learn how this method influences our own online marketing testing.

A/B and Multivariate Testing Today

Going through this little history lesson about the roots of multivariate testing and A/B testing can better help us understand the importance and methods to improve our online marketing efforts. Whether you’re curing scurvy, a poorly performing conversion rate or really any type of  complex problem solving, use these testing methods to your advantage.

If you’re looking for help with A/B and multivariate testing, contact our landing page experts.

Relationship Between Life's Tests & Landing Page Tests

You have been tested your whole life. From the moment you were born life has presented tests for you. Some tests have been very structured and taken with a #2 pencil, but most weren’t as obvious. For example when your significant other asks, “Do I look good in this outfit?” Look out! It’s a test! The response you choose could alter your entire day, week, month, or life!

Most life lessons are learned through experience, and really it’s the same thing in online marketing.  You’ll never know for sure if a new landing page layout could work better unless you try it. Ah yes, the segway into landing pages, but seriously, why not test? Testing allows you to improve your conversion rates and sometimes it does so by proving our assumptions wrong. 

You probably hear a lot about multivariate testing and A/B testing. Is the information your hearing correct? A/B Testing or “split testing” is a method by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response or conversion rates. Multivariate or “MVT” testing can theoretically test the effectiveness of limitless combinations. The only limit on the number of combinations and the number of variables in a multivariate test is the amount of time it will take to reach statistical signifigance.

But how do you choose the most valuable testing for your business? It’s a trick question. Both methods are effective at improving conversion rates; it just depends on what you are trying to test and what you want to learn from your tests. 

Here is a side-by-side comparsion for you to consider:

Whether you choose MVT or A/B testing (or both), make sure you wait to draw any conclusions until your tests reach statistical significance! If you don’t let your test reach this low margin of error then you’re just guessing and really defeating the whole purpose of the test. Let your test prove what works for your business, don’t guess!

Oh, almost forgot…the answer to “Do I look good in this outfit?” is always “Yes, you look great!”

Happy testing!

How do you know if you need landing page management software?

There are many ways to make landing pages.  You can hand code static pages yourself, outsource the work to contractors or an agency, or you can even use certain tools like a website CMS, blogging platforms or modules of a marketing automation system to make pages.  So, how do you know when it’s time for a landing page management platform? 

It will feel like you’ve hit a wall.  It will seem as if there is just never enough time or resources to launch a new campaign, start testing, or even to just implement a content change.  Each of the methods mentioned above will certainly help you make pages, but they aren’t designed to help you manage, grow and optimize a large, sophisticated landing page program.

Here are a couple red flags signaling that it’s time to change the way you manage your landing page program:

1. Your pages are disorganized

Do your landing pages ‘live’ in different areas?  Perhaps some are in your website CMS, others are handled by an agency, and then more still are handled by a different regional team? When your content isn’t centralized and organized it’s almost impossible to scale your efforts, because you’ll constantly find yourself recreating the wheel. Landing page management platforms centralize and organize all of the content that goes into creating landing pages so that you can, for example, make a lead generation form once and then reuse it on several different pages.

2. Your analytics are opaque

Can you easily and confidently compare how well campaigns did compared to each other?  Opaque analytics go hand-in-hand with disorganized pages.  If you don’t have one central location for where your pages are stored then trying to analyze and compare metrics across campaigns is going to be very difficult. If your landing page and test analytics are managed seperately or by someone who isn’t part of the team guiding your online marketing strategy, then that’s also a sign that you should consider landing page management software. If your decision makers don’t have complete, easy access to real-time data, how will they be able to act in time to take advantage of what your tests and analytics are telling you?  

3. Your branding is inconsistent

Are your latest branding guidelines accurately reflected on every landing page or do some show old messaging or use an outdated look? If it’s difficult to update all of your pages with new branding guidelines or if you’re not even sure whether they are all up to date, then it’s time to change the way you manage your program.  By centralizing all of your content within one platform, a landing page management software will make it easier to implement mass changes and to enforce brand standards.

4. Changes are inefficient

Does a relatively simple content update take days to implement or does testing just seem like it would be impossible? If your current landing page creation process involves a lot of hand-off points from, for example, someone on your marketing staff to a writer then an agency or developer (or something along these lines), then your answer is probably yes. Since landing page management platforms eliminate the need for code or help from developers, you can let marketing take direct control of the strategy, implementation and management of your landing page program.  When it takes fewer resources and less time to both launch and test landing pages, all of the sudden the idea of launching a new campaign or localizing pages doesn’t seem the least bit daunting.

5. Testing is rare

When all of your resources are tied up into getting pages live, there’s often very little time left to start testing.  If you’re only running a few tests or not any running tests at all, then this is a major sign that it’s time to consider landing page management software.  Most platforms, like LiveBall, will help you create, launch and track several – if not hundreds– of A/B and multivariate tests without code or help from developers.  If you’re not testing, you’ll have no way of knowing if your landing pages could be doing better and if you could be getting higher conversion rates or improving your CPL.

If any of these scenarios sound too familiar, then check out our free 10 Point Landing Page Management Buying Guide.  Although we obviously hope that you choose LiveBall, this guide is completely objective – it will just walk your team through the key points you should consider before starting your search.