Landing page testing can result in dramatic and lasting improvements to your online marketing program profits. However, setting up a testing program involves many moving parts.
A camel is a horse designed by a committee.
—Sir Alec Issigonis (designer of the Mini car)
Your testing program can look like it’s moving forward but still fail. If it does, it will likely happen as a result of a “death by a thousand cuts.” Your original test ideas will be reviewed, tweaked, overruled, and co-opted by many others along the way. It is like the children’s game of “telephone” in which all of the participants line up and an original message is repeated and whispered to the next person in line. By the time it reaches the other end, you end up with something completely unrecognizable. But you have no choice because you need the cooperation of many people. You have to bear the ultimate responsibility for wrangling and herding them roughly in the desired direction. So your skills as a diplomat and persuader will be very important.
To help guide you through this process, I have put together a list of the “usual suspects.” These are roles that are commonly needed for a successful landing page optimization program. Depending on the scope of your program, not all of them may be applicable to you or your organization. If your company or department is small, several of these roles may be assigned to a single person. In fact, many of them may currently be assigned to you. If that is the case, use extra caution to take yourself and your predispositions into account as much as possible.
Each role in this section has an associated scope of responsibilities or expertise. I have included a section on typical skills and training required. Each role also has a specific overlap with your landing page optimization program. I will also bring up common issues that repeatedly come up in our tests involving each role (with suggestions about how to address them whenever possible).
Remember to pay particular attention when you get to the sections describing your own role or roles. Read them with the additional perspective of understanding how others may view your responsibilities, biases, and working style. Don’t become a stumbling block to your own program by ignoring your role in it.
Product managers are typically responsible for all aspects of developing and marketing a particular product or service. They may also have profit and loss (P&L) responsibility (i.e., they are held accountable for the profit and loss performance of the product group as a whole). Product managers have operational day-to-day responsibility for the product team, although they may draw on staff members from other departments as needed.
Skills and training
Product managers can come from a wide variety of backgrounds, including technical, management, and finance. Their main roles are to manage and facilitate. They are usually effective at putting together cross-disciplinary teams of people. Product managers make sure that the product features are responding to competitive marketplace needs, and that the marketing is effective.
At a minimum, product managers will have to approve any landing page project related to their product. Often they will be the person leading or managing the effort.
They may be needed to secure the help of any project-based people outside of your normal team. Product managers also help push through any approvals required from other departments.
If your product manager is already behind your landing page optimization project, then you have a valuable ally. If that’s not the case, you may have a hard time getting enough of your product manager’s sustained attention to make it happen. Product managers typically wear too many hats and are overextended. Since their role is to herd others, they are also commonly interrupt-driven and easily distracted.
Product managers are also often in the “if it ain’t broke, don’t fix it” mode. If the landing page in question already drives significant revenue or value, you may have a difficult time convincing the product manager that there is a need for landing page testing. He or she may be preoccupied with other problems, and may not understand the financial opportunity presented by improving the conversion rate. This may especially be the case if the landing page testing effort requires the product manager to take on significant extra work to coordinate the project.
You must usually go through the product manager to get budget approval for the project. Often, this individual is not the final financial decision maker, and must in turn get approval from his or her boss. Depending on the scale of your company, you may have to wait until the next budget cycle to get your project authorized.
Your webmasters are responsible for the care and maintenance of your website. They maintain the content, control the site organization, and administer the file-naming conventions (e.g., for page names and graphical images). The webmasters make sure that the site does not have any broken links, missing content, browser compatibility issues, or improper form handling. They also create and review the Web analytics reports for the site.
Skills and training
Webmasters commonly come from a programming or technical background (operational focus), or from a creative or copywriting background (content focus).
The webmasters will most likely need to approve (or at least be aware of) the changes that you propose to make to the site. They will need to provide you with the original page elements, and help you upload the new test versions of all pages and graphics.
They may also be involved in the cutover from your quality assurance environment to the live data collection. Webmasters may also track and monitor your data collection through Web analytics software.
Some webmasters are very territorial. They like to control exactly how and when changes happen on their site. This mind-set can often get in the way of testing, because you are perceived as violating their turf and creating extra work for them. In extreme situations, this can result in stonewalling or even outright denial of access to certain parts of the site.
You will need the help of your webmaster to download and upload content to the staging portion of your Web server (i.e., one where you can test changes without affecting the live website). Ideally you should have the ability to transfer files to and from the staging server yourself. But many webmasters will refuse access (ostensibly for security reasons) and insist on doing it themselves. Unfortunately, during quality assurance testing numerous uploads may be required to get rid of all known bugs and problems with your test implementation. If your webmaster is uncooperative or busy, this can drag out your implementation schedule considerably.
Webmasters are also in charge of policing the style guides and naming conventions for the site. They can be very particular about file-naming conventions and HTML coding standards. This can hold up testing if not addressed ahead of time.
Because webmasters are often involved in the testing (and may even be in charge of it), it is always a good idea to get their active cooperation. The best way to do this is to arm yourself with information ahead of time, follow procedures, and communicate clearly throughout. Ask for design guidelines, style sheets, procedures, and Web coding conventions. Address exactly what content changes will be tested in your written test plan. This should include the names and locations of all files and content involved.
The graphics designer is responsible for creating all graphical elements of your landing page.
Skills and training
Typically, graphics designers will have an artistic background in the visual arts, including drawing, painting, film, animation, and photography. Some will have had additional training in production graphic design for business. They may also have had additional training specifically in Web design and related graphics, photo editing, animation, video authoring, and Web design software packages.
It is likely that you will need the involvement of a graphics designer for your landing page testing. They can create individual graphical elements such as pictures, buttons, navigation menus, and rollover images. Some of these changes are simple (e.g., changing button text or background colors). Other changes may involve a complete redesign of your landing page and its layout.
Graphics designers may also be needed if your test plan calls for special interactive content (such as comparison shopping guides, product demonstrations, and software wizards).
Graphics designers usually have an artistic bent. This can mean that they are more concerned with self-expression than with the goals of the business. In their need to keep themselves amused, they often try to work on fun projects or turn routine assignments into artistic outlets.
Unfortunately, this tendency can be at odds with the goals of landing page testing.
Internet visitors often respond best to stark landing pages on which visual distractions are kept to a minimum. You must keep tight control over your graphic designers. Use the yardstick of “is it absolutely necessary?” to determine if design elements should be included or emphasized. In fact, it is often a good idea to test the complete removal of existing graphical elements.
The same applies to the use of more subtle color treatments. Bold ones may look more “interesting” to you, but plain ones are often less distracting to the visitor.
Anything that draws visual attention on the page (other than your primary call-to-action) should be questioned. Instruct your graphics designers to unclutter the landing page. In other words, emphasis on key elements should be achieved by toning down the surrounding page, rather than by making them even bolder to compete with other graphical elements.
The same applies to the use of animation, sound, video, or fancy interactive demonstrations.
To get your approval for inclusion in the test plan, graphics designers must show you that their proposed visual solution is the only acceptable one that accomplishes the business objectives and supports the desired outcome. This is a high hurdle to surmount.
An alternative approach is to allow one “artistic” option for some test plan variables.
Who knows, it may even end up being part of the best performing recipe. This, after all, is the whole point of testing.
Copywriters are responsible for the text content of your website.
Skills and training
Copywriters come from a variety of backgrounds. Some have a formal education in writing and composition. Others were drafted into the role of technical writer. Still others are experts in their subject matter area and have been asked to write specific content or articles for the website.
Many of the elements that you may want to tune involve copywriting: headlines, body text, sales copy, the call-to-action, the naming of navigation links, button text, figure captions, and text embedded in graphics.
Copywriters are often generalists. They may need the support of your marketing staff, subject matter experts, or product manager to make sure that your main messaging points and technical features are being presented effectively from the perspective of your intended audience.
Copywriters write for a variety of corporate materials, including informational articles, technical writing, and print collateral. When writing for the Web, they tend to carry over the same editorial style. In fact, effective landing page copy often goes against the very grain of traditional expositional writing and grammar. Copywriters need to be closely monitored to make sure that their Web writing conforms to the format guidelines for web writing. This includes use of the inverted pyramid style, factual objective information, and the use of concise sentence fragments or bullet lists.
The marketing managers are responsible for the marketing of your product or service. They formulate the marketing plan and oversee its execution.
Skills and training
Marketing managers often come from an advertising, communications, finance, or creative background. Depending on the size of your company, they may be generalists who are responsible for everything, or specialists who are only in charge of Web promotion. If your advertising mix includes offline components, they may be responsible for making sure that a consistent view of your product exists across all channels.
As part of your test plan development, marketing managers can often provide a lot of excellent background information. This includes marketing intelligence about competitors and your product’s positioning in your industry. They can communicate with the copywriter to ensure that there is compliance with the specific marketing messages that need to be included in your sales copy and headlines. Marketing managers can also help you understand larger business goals and the potential impact of your landing page test on your business partners, customer service, and offline channels.
Marketing managers understand the full scope of your promotional and traffic acquisition activities. They have a good handle on the expectations of people landing on the site from different online marketing campaigns and what happens upstream of the landing page. It is important to communicate to them the key requirements for your test traffic. Your ideal traffic sources for testing should be recurring, controllable, and stable. The marketing manager can help you to identify the largest subset of your traffic that matches these criteria.
Marketing managers often control the marketing plan and are in contact with public relations and other parts of your company. They can tell you about the timing of external events (such as tradeshows and industry-specific seasonal changes). In addition, marketing managers can tell you in advance about any major public relations or product announcements. You must work closely with them to make sure that your proposed data collection period does not fall during these periods; otherwise, your data may be skewed or irreparably tainted.
User experience (commonly abbreviated as UX) is an interdisciplinary field that examines how users interact with a particular system, object, or device. This includes how they view it, learn about its capabilities, and use it in the real world.
Skills and training
In the case of landing page optimization, you are mainly concerned with the Web browser and the content and function of your landing pages, so the UX professionals will likely come from the specific disciplines of human factors, user-centered design, human computer interaction, information architecture, interface design, visual design, or usability testing. Most of these are pretty specialized skills and are not commonly available in most online marketing departments. However, UX people may be working within your software development team. If no one at the company has this background, it is possible to hire consultants on a project basis. With their broad experience, such people can often provide critical insights into which elements to test.
UX specialists are usually given wide scope to improve the overall usability of a system. Because usability is a huge factor in conversion rate improvement, they are usually heavily involved in development of the written test plan (helping to determine the elements to be tested, along with the specific alternatives for each element to be considered). UX experts can also readily construct the roles, tasks, and AIDA decision process steps for your business, and help you identify gaps. They are also experienced at matching important business objectives to the needs of users.
UX practitioners are generalists. They may have been involved in the design of many websites on a variety of topics. For this reason, it is important to team them with a subject matter expert (SME). Without the support of someone knowledgeable in your industry or business, UX practitioners may miss important aspects of your conversion process or business goals.
UX people are usually good at the functional and architectural aspects of your design (i.e., common usability issues that are likely to affect all of your visitors). They are usually weaker on the content issues, such as text copy, marketing message, and graphical design.
UX practitioners are also believers in the notion that through good design procedures and small-scale usability testing (involving a few representative test subjects) they can come up with a high-performance and coherent design. However, they need to understand that you are not asking them to come up with a perfect design. Their ideas may be used piecemeal as part of your test. It is often tremendously liberating for them to be allowed to come up with ideas without having to be sure they are all good.
The best landing page version will be found statistically (by watching the behavior of thousands of people), not through qualitative or small-scale usability testing. UX practitioners are also often in favor of testing, but they often have problems with statistically based testing such as is described in this book. Because of this, their involvement should usually be confined to helping decide what to test, and not the subsequent data collection or analysis.
Programmers are responsible for the functional (as opposed to the strictly visual) aspects of your website.
Skills and training
The background of programmers is diverse. There are few acknowledged accreditations in the industry. Many excellent programmers are self-taught. The speed of technology changes requires programmers to become lifelong learners or face the prospect of skill obsolescence. Some programmers are focused on the presentation of information to the end user and are adept at scripting languages that make up the front end (i.e., the visual portion of the software application with which the visitor interacts).
Others concentrate on the representation, storage, and manipulation of the underlying data that make up the back end (i.e., they focus on databases and algorithms).
Any functional changes to your landing page or website may potentially require programming support:
• Mouse rollover behavior
• Reconfiguration of form elements based on visitor actions
• Capturing of additional information (changes to the database)
• Business rules and logic
• Changes in the flow through your pages
• Reorganization of the area where you collect data (and the order in which you collect it)
• Processing any new Web-based forms
Programmers tend to be poor user interface designers, graphics designers, and copywriters. They think only in functional terms. If a certain capability is technically possible, they will not usually try to optimize or improve the user experience. The result is that landing page changes touched by programmers are often very unappealing to your visitors (with the consequence of lower conversion rates). So you must be specific in your quality control and testing about the details surrounding any changes that the programmers make. This includes background colors, fonts and font sizes, form field order and layout, text labels, and error messages.
The best way to deal with this problem is to have detailed specifications for the required functional changes. Include screenshot mock-ups of the proposed designs. You should also spend some time sensitizing the programmers to the subtleties of good design and emphasizing its importance.
On the other hand, programmers are often very receptive to empirical real-world data. If you can show them that a design option actually performs better than an alternative, they are likely to be very enthusiastic about finding more options like the successful one.
Your system administrators keep your server network running and operational. They are responsible for Internet connectivity, the load and demand on your Web servers, keeping software up to date, backing up your data, and computer security.
Skills and training
System administrators usually come from very technical backgrounds. They are often detail oriented and keep track of a large number of operational details and procedures related to their job duties.
System administrators will be involved in the following ways in your test:
• Moving from the staging to the live environment
• Rerouting traffic for the test
• Reviewing the proposed testing technology and implementation requirements
• Certifying that personal or private customer data is not disclosed during testing
• Ensuring that network security is not compromised by the testing
• Assuring that server loads and Web page loading times are not significantly affected by the test
Since choosing a particular tuning technology or testing company partner involves technical elements, system administrators will typically want to get involved. They will vet the underlying technology and project implementation procedures in a lot of detail to understand the impact that it will have on their domain. In some cases, they have veto power over choosing certain kinds of tuning technology approaches.
Part of the system administrator’s concern has to do with control over the hosting and presentation of the alternative tuning elements during the test. Some testing technologies rely on outside hosting of site elements on the Web servers of the testing company.
This is often strongly resisted by system administrators, because they cannot guarantee the security or response times of another company’s Web servers. Other technical approaches allow all new content for the test to reside within the current technical environment of the landing page. In other words, alternative tuning elements are also hosted on the company’s Web servers. This is much more likely to put the system administrator’s mind at ease.
Additional concerns of system administrators center on security and data integrity. They want to make sure that your testing method does not introduce any new vulnerability. This includes inadvertent disclosure of private customer data (such as e-commerce credit card information or personal contact information).
These issues are usually easily addressed if you review the proposed technical approach in detail with system administrators, or arrange for them to talk directly with the technical staff of the proposed testing technology company that you are considering using.
Quality Assurance Tester
The quality assurance (QA) tester ensures that all proposed changes to your website function properly before being released as part of your live site.
Skills and training
Most QA positions are not full-time. They are typically project based (e.g., parts of a complete website redesign process). Consequently, QA staffs have a variety of backgrounds and may spend the majority of their time in other roles, such as webmaster, graphics designer, copywriter, or marketing assistant.
QA should always be involved in the tuning process after the test plan has been implemented (and before the changes are moved to your live site prior to the commencement of data collection). Once problems are uncovered, they are sent back to the implementation team for rework.
It is important that the person assigned to perform QA is not the same person who oversaw the implementation of the test. Otherwise, there is a clear conflict of interest and a tendency to shortchange the QA testing process.
It goes without saying that your landing page optimization should be based on a formal written test plan document that defines the specific elements and values to be tested. As soon as the test plan is completed, you should independently create a QA plan to go with it. The QA plan should note all important design and technical constraints for the proposed test. The QA tester should use this plan to make sure that all variable values are independently tested and that all key combinations of variables are also considered.
QA testers are supposed to be detail-oriented. In fact, this is a requirement for the role. However, some people take things a bit too far. They refuse to sign off on any deviations from the original test plan that are even a little bit out of compliance. At this point you must often make a judgment call about whether the discrepancies are likely to significantly affect the outcome of the test. You may have to overrule the QA tester and accept the current state of the implementation.
As I mentioned earlier, not all landing page tests will require the full cast of characters above. Depending on the scope of your test and the size of your company you may be able to start small and build on your initial successes. As you involve more people, you just need to think about their training, motivations, and concerns. I wish you good luck and higher conversion rates!
This article is the second in a continuing series on landing page optimization.