Visibility

If you’re like most online marketers, you are doing everything you can to optimize your website. One of the best ways to optimize your site is through multivariate testing. In a multivariate test, variations of your site’s content are presented to visitors, and as visitors navigate through the site their behavior is tracked to determine how each variation impacts marketing goals like conversion, average order value, retention, and so on.

Marketers unfamiliar with experimental methods like multivariate testing are often unclear about where to start or how to ensure success.

What is Multivariate Testing?
Controlled testing is a method for determining how variations of your web site impact user behavior. A/B testing enables you to test two things at once; for example, a new version of a landing page headline versus the existing, or “control” version. Multivariate testing builds upon A/B testing, and enables you to create and test many versions of your web site simultaneously. In a multivariate test, you’re able to determine which combination of variations maximizes the desired user behavior, such as making a purchase, registering for an account, opting-in to newsletter campaigns, and so on. In effect, your web site becomes a test lab where your users essentially tell you what works and what doesn’t.

Multivariate testing shows you not only which new versions of an offer or product image generate more sales or pull more leads, but it also reveals exactly which elements act as behavioral levers for the site. For example, did the product image influence user behavior more, less, or the same as the call-to-action?  Understanding how each site element causes users to interact with a site is the essence of a “test-learn-repeat” process that marketers can use to continually improve their site.

What Resources are Required for Effective MVT?
Optimizing a web site is hard work and typically requires not just Marketing, but also IT and QA (quality assurance) to fine-tune the systems and ensure that they're optimizing towards the marketers goals and not away from them. Unfortunately, if you are trying to run lots of tests, this can be a difficult process to manage. Tests that require significant effort to implement act as a deterrent to ongoing testing and can mean the difference between success and failure. In order to understand the total cost of your testing efforts, make sure that you consider the resources required for both the initial and ongoing implementation of the testing solution.

What Do I do with Test Results?
The process of testing reveals not only what works and should be implemented, but also what doesn't work and should be avoided.  Every web site idea, whether content, functionality or campaign related, should be tested with actual end users to determine if it helps or hinders the user experience and, by extension, your marketing goals.  While some new ideas lift conversions, others fail -- sometimes significantly.  The ability to test a new idea and ‘look before you leap’ is an unmistakable competitive advantage, particularly when you have made testing a part of your marketing process and not just a one-time activity.

How can Multivariate Testing Be Used To Optimize my Web Marketing?
Multivariate testing can yield some spectacular results in enhancing online effectiveness. For example, one of our customers, an online auction house, ran a series of multivariate test campaigns to understand which elements were most influential in bidding conversion. The team tested variations in elements such as catalog page layout and messaging, individual item landing pages, and calls-to-action. They made test variations in critical elements on their site, such as creative elements, event promotions, image sizes, copy, navigation and page layouts, without the need to make a single change to the underlying catalog auction system. These changes resulted in:

•    429% increase in bidding activity
•    83% increase in catalog browsing activity
•    166% increase in individual item views
•    590% increase in opt-in registrations


How to Prepare Your Website for Multivariate Testing: Top Three Tips to Ensure Your Success
While multivariate testing is an effective and proven technique for optimizing a site, some sites can be made easier to test than others. Technical factors such as the structure of the HTML come into play in determining how much prep work you will need to create a new multivariate test.  But since testing is so valuable in terms of giving you a feedback loop into your visitors’ preferences and behaviors, it’s well worth considering how to make your site more readily testable. Doing so will yield a competitive advantage that makes it possible to out-test, out-learn, and out-optimize competitors.

There are a few times in your website’s lifecycle that are ideal for considering how to make your site more testable. For example, building a brand-new site or redesigning your current one gives you the perfect opportunity to apply some best practices and make your site’s elements easier to test.  Another good time to prepare your site for multivariate testing is when you are adding major new content or functionality, where those new elements can be carefully introduced on the site so as to maximize their effectiveness without hindering existing site goals.

Here are three top tips to prepare your site for multivariate testing success, which represent the fundamentals of making content testable:

1)    Use Cascading Style Sheets (CSS)
CSS is a popular choice for web design consistency and standardization, and that’s great news for multivariate testing. Among other things, CSS centralizes site-wide styles such as text and headline specifications (font, style, color, size), page layout, and positioning of elements. So, when you make a CSS change, you change it in one place and the style is updated everywhere on the site. This makes it much easier to test multiple variations of site elements since you don’t have to wrestle with changes in multiple places, which you would have to if you weren’t using cascading style sheets.

For example, a recent test for a B2B site revealed that by increasing the size of body text by 0.2 “em,” the time-per-visit increased by 21% and pageviews-per-visit increased by 18%. This test was highly fruitful, and because only one line of CSS was changed, it was very easy to design and deploy.

2)    Employ Text-Oriented Navigation
In the past, web designers often relied on images or Flash to navigation text elements, particularly for sites that were built before 2004-2005, since browser technology was such that CSS couldn’t be relied on for precise positioning or stylization of text and labels.  Unfortunately, using images or Flash for navigation has the side effect of making it difficult to test alternate text labels – for example, to test “My Preferences” vs. “My Profile” vs. “My Settings.”   Over the last few years, however, browsers have become much better at supporting most CSS standards, so there’s little reason anymore to rely on images or Flash for navigation.  Text links are much easier to modify and control, and thus are much easier to test and optimize.

But why bother testing navigation?  Because they’re the milestones by which visitors step through your site to accomplish their tasks and reach your marketing goals.  The best product, price, or promotion is no good if the visitor can’t grasp which link to click on to find the information they need, or to get to the next step in your site’s conversion process.

3)    Apply Fluid Design
Another important tip to keep in mind when creating or overhauling a site’s design is to use fluid design wherever possible.  Fluid design is a general approach where a site’s elements can be added, removed, or rearranged without interrupting or breaking the presentation of the surrounding content. Besides being a good design principle that makes the site more accessible across varying screen sizes, fluid design also makes multivariate testing much easier because you can determine which layout is most effective.

The truth about layout is that different types of visitors scan content in different ways.  For example, older visitors who grew up without computers may view a page’s sections in a different order than younger visitors who have been “wired” since birth.  Similarly, behavioral differences exist based on gender, education level, and language spoken.  By making it easy to adjust page layout through the use of fluid design, you’ll be able to test and optimize for the audience that is unique to your site.

For example, online jewelry retailer ShopNBC.com wanted to determine how cross-sell items should be displayed relative to customer product reviews.  Because the site had a fluid design, a multivariate test was easily constructed that optimized the arrangement of elements within the page, leading to a 16% increase in average order value.

And remember that sometimes “less is more,” meaning that visitors may prefer less cluttered pages as they browser your site.  Fluid design makes it possible to remove elements on your pages in order to determine the optimal amount of information that should be shown.

Below is an example of how SiteSpect used all three design principles in redesigning its website to make it easier to test various elements.

Overall, you’ll want your website to be modular, flexible, and easily testable.  If you are looking to optimize your web marketing, multivariate testing ought to be part of your toolbox.