The Surprisingly Important Element of Successful Testing


I’ve had a lot of positions at my current company (ClickEquations, formerly Commerce360), but one of the ones I most enjoyed was a multivariate testing project I did with a large entertainment provider.

In preparing for the project, I read a ton about testing, what to test, past tests that worked, etc. There is no shortage of information about what to test or how to test it (see., for example).

What surprised me about the experience was that very little time was spent talking about the most important elements of a successful test: people.

Specifically, the people you need to help you with a test.


CRO & Big Companies

If you manage your own website, launching a test happens in an afternoon and you’re making the calls. If you’re heading up a conversion rate project for a website with hundreds of thousands of visitors a day, it quickly becomes a project convincing people to help you test with some conversion rate optimization attached.

That was the case for the company I worked with, as it was one of their first forays into testing. Getting momentum required buy-in from several key stakeholders:

  1. Champion – We had an awesome internal advocate who prioritized the project and rallied the troops. He served as the central point of contact to coral everything on the client side while my coworker (the account manager) and I got to focus on the testing.
  2. Executive Sponsorship – The client had to lay out money for both software (we used SiteSpect) and our consulting fees. Working on our project also meant diverting hours for contributing departments (see below) from other projects to testing.
  3. Development Buy-In – Every testing project involves some changes to your site in one way or another: DNS changes, tags in footers, Mboxes, etc. The bigger the site, the more cautious the IT and development team. You need to get in their sprint cycle. I recommend you choose a technology that allows you to ask for help from development once and then permits changes on the fly afterward.
  4. Creative – Someone has to create all of those test variations and in bigger companies, they generally have to be sensitive to brand guidelines or other restrictions. More importantly, you need an ally on the creative team who is will to let you butcher their work and possibly test something that, from a design perspective, they may not necessarily agree with.
  5. Analytics – Chances are, there is already someone in the company doing analytics or, as was the case for my client, statistics. You’re going to need access to analytics data to design an effective test and they might even review it with you.


Converstion Team Optimization

Testing can fundamentally change how a business operates and where decision making power lies. It also involves new and different business processes. That is to say, testing means change. People, especially bigger companies, are often slow to change.

In my experience, a great Champion and Executive Sponsor set the tone for the rest of the team. They approve budgets, timelines and the allocation of resources. A great process and communication help win over the rest of the team, but that’s for another blog post.

3 High ROI Landing Page Books

I don’t know about you, but I haven’t been reading books as much as I would like.  Mostly, I get snippets on blogs or bits in articles.  Case in point: I’ve been in the middle of Always Be Testing for months now (ack, sorry Bryan!).

Well, I’m trying to finish more of the things I’ve started and focus on only the things that matter.  In the world of optimization, landing page testing falls high on that list.

I wager that’s probably on your list too (at least, I hope it is), so I put together a quick list of three landing page books:

  1. Marketing Sherpa Landing Page Book
  2. Always Be Testing: The Complete Guide to Google Website Optimizer
  3. Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions

1. Marketing Sherpa Landing Page Book

Marketing Sherpa puts out a killer newsletter and regularly releases compilations of their own primary and secondary research.  Here’s a brief snippet:

8 Design tips for landing pages with links to other pages:

  1. Ruthlessly eliminate click links that are irrelevant pages or advertisers, and minimize the typeface of those to privacy and legal information.
  2. Make sure links change color after they are clicked by each visitor.
  3. Make the area around each link clickable (even if the link itself only has a word or two underlined, or a small click button) so the visitor doesn’t have to hit the spot right on with their mouse for it to work.
  4. Carefully copywrite your links so someone reading the first three words or so will understand what they’ll get from the click. People skimming a list of links rarely read more than a few words per line. Unclear, boring, or duplicative-sounding links won’t get clicks.
  5. Make your hero shot clickable, with a separate window of information opening so the visitor is not taken away from the main landing page. A surprising number of folks will click on your hero shot.
  6. Don’t make visitors click to a conversion form if possible. Clicks should be for more information, not for additional conversion steps. Include your form or the first step of the form on this page. Make this conversion step obviously bigger and graphically different from all other click links on the page.
  7. If your page has to appeal to multiple audiences and there’s no way you can d a separate landing page for each, then the page should focus on the primary audience. Create a big fat link for the secondary audience to click on to go to a page specifically designed for them. Example: a page with info for kids with a fat link saying “Parents, click here.”
  8. If linked information is critical to conversion, then consider including several different links to it on the same page in different formats. Some people will click on underlined text, others on graphics, and others on search boxes. You need to be sure all three surfing types are able to arrive at the same place for the next step in the conversion process. Don’t worry about duplicative linking. It’s reassuring rather than annoying.

The Marketing Sherpa Landing Page Book is 30% off until December 31st.

2. Always Be Testing: The Complete Guide to Google Website Optimizer

My guess is that most business who dip their toes in testing use Google Website Optimizer.  Bryan Eisenberg of FutureNow and GrokDotCom wrote his third book all about the software as well as the advice on persuasion and website testing.  Amazon has the book as well as reviews.  You can also get an overview at Testing Toolbox, a central location for everything related to the book.

3. Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions

Tim Ash’s book is also pretty fresh off the presses, though he’s been around for a while.  His landing page book focuses entirely on the best and worst practices of landing page design and testing.  It piggybacks off of his writing at Search Engine Watch and the SiteTuner’s blog

Available at Amazon for the regular folk.  Die hard fans willing to throw the recession to the wind might want an autographed copy of the book, which you can purchase here :-)

Which landing page books do you recommend?

6 Methods to Personalize Your Site Experience By Referrer

referrer logs

Visitors are not anonymous.  Yet, most websites present the same experience to everyone.  When you treat everyone the same you leave conversions, and money, on the table.

Referrers (aka referring sites or referring domains) can often give you hints about who your visitors are and what might help them more.  Your web analytics tool or server logs gather referrer information.  Here’s 6 ideas of how you can use this information to personalize the site experience.

  1. New Visitors – When I was at X Change, Cisco noted that they include a site section on their homepage to customize content based on referrers.  Their “New to Cisco?” personalized box for new visitors converts at more than double the average rate!  Recognize newbies with orientation messaging.
  2. Competitive Comparison – Let’s pretend you’re Dell.  Someone comes to your product page after visiting a review on CNet or Wired.  This is a great opportunity to piggyback off the tone and content of that article to address criticisms or play up noted features.
  3. Coupon Sites – Sites like cater to a budget (recession?) conscious crowd.  You probably have sale sections, newsletters with coupons or other bargain shopper friendly options.  Try to capture these shoppers with targeted promotions.
  4. YouTube - Video lovers may be more responsive to demos, tips, tutorials or user generated content.  If you’ve got it, try targeting these videos with relevant media.
  5. Help Forums – Current customers might be looking for support and prospects could be scoping out reviews.  You might be able to steer them to your officially sanctioned support
  6. Press Releases – Press releases are likely to pull in reporters and bloggers.  As you see people visit from known press release distributors and major media outlets, offer them additional news content or point to your press room to keep up the momentum.

Have you used referring information to personalize?  What software do you use?  Which segments are best?  Share your questions and tips in the comments section.

5.5 Ways to Boost Conversion Rate!

Looking for ideas to improve your website’s conversion rate? Today, I’m speaking at Search Camp Philly on Conversion 101, aka 5.5 Ways to Boost Conversion Rate. It’s at 3:00.

I hate boring and long presentations that you can’t use. So, mine is almost entirely pictures with examples of ideas to improve ecommerce websites. My advice is to:

  1. Slim Down Your Checkout
  2. Love Your Landing Pages
  3. Juice Your Shipping and Return Policy
  4. Answer Questions with Photos
  5. Succeed with Site Search

My bonus “half” recommendation, number 5.5, is to test!

If you can’t make it, no worries, my presentation is below. Feel free to download, share and repurpose (just link back). It has a lot of pictures, so I recommend you look at it in full screen mode (that little icon in the lower right that looks like a screen).

Full presentation after the link!

Continue reading

Which Pages Are People Testing?

A/B and multivariate testing are staring to get their due.  This whitepaper from ZeroDash1 is a good state of the union for the MVT market.  I thought this graphic of which page types people test was illustrative:

homepage testing

Most of the traction in multivariate testing is on landing pages, with the rest evenly split between home pages, conversion/purchase pages and product pages.

I’m not terribly surprised because:

  1. People who are testing are more likely to have landing pages, which are generally more advanced than a small webmaster has.  Plus, those people are more likely driving acquisition channels to these pages, where conversion makes a big difference in profitability.
  2. Testing, like analytics, is best used first where you can follow the money.  Campaign landing pages are often the leakiest part of the bucket.

69% of people think testing is worth it and 22% can’t live without it, which is pretty compelling data.  Of course, you need to translate the value into dollars and cents to your boss…

There’s a good graph in there with ideas of things to test, so download the whitepaper.

16 Ways to Harness the Power of Pictures – Ecommerce Tip #9

I’m back! After a crazy 6 weeks at work (6 new clients!), I finally have some time to get back into writing. In case you missed it, I was blogging elsewhere about blog conversion and the importance of search queries.

Last time I wrote, I mentioned 4 ways to profit from the buying cycle. Let’s get right into the meat and potatoes of closing the sale on your ecommerce site: pictures.

If you want to sell online, then you have to close the tangibility gap–the inability of a customer to touch, feel and assess your product. Great copy goes a long way, but you’ll need compelling pictures to fill in the gaps. Here are 16 ways to sell more with better pictures:

1. Provide Zoom In – The #1 issue with pictures (in my opinion) is that they’re too small. It can be hard to distinguish features and detail. This is doubly true if you sell clothes and people want to inspect patterns, stitching and other nuances.

2. Offer Multiple Angles – Front, back, sides, top, bottom. Make it easy for people to see your products from every side. It also helps reduce return rates, because there are no unexpected surprises.

3. Scan and Review Details – Think of scanning as an advanced zoom. It lets you get very close to the product and pull the zoom up and down to review the product as if you were holding it. The finer the detail, and the bigger the price, the more people will want to examine closely.

4. Packaging – If your product features an especially interesting package, as is often the case for jewelry or any well designed product, consider showing the packaging as well. It sells the whole purchase experience.

5. Product in Use – Demonstrate how the product works in picture. Help people visualize themselves using it.

6. Product in Settings (usage ideas, aspirational settings) – Furniture retailers have this one down perfectly. I love Room and Board for all of their design ideas. They understand the aesthetic of their

7. 3-D Rotation – Sometimes static pictures don’t tell the hole story of a product, especially detailed ones like collectibles.

8. Paired with Other Products (cross-sell/up-sell) – Show the ideal pairing – a great wine with a wine glass, the perfect shirt with a pair of pants, or a Vespa in a great Italian city.

Continue reading

Ecommerce Tip #8 – 4 Ways to Profit From the Buying Cycle

The problem with a lot of ecommerce sites is that they assume you want to buy right now. Of course, that’s only true of a portion of your visitors.

Some visitors are browsing, some researching and others comparison shopping. Your site and marketing can (and should) speak to each of these potential buyers. Here are 4 ideas to reach them:

1. Email


Email is routinely cited as one of the highest ROI tactics for online marketers. It’s great for customer retention and building a relationship.

Give sneak previews to your email list to find out which products generate the most interest. Test out creative concepts with an A/B split to email recipients.

Offers are the obvious route for email, but I don’t think they’re always necessary. In fact, I think relevance is much more important. Design Public, my favorite modern store, rarely provides offers. Instead, they have a simple email that highlights their interesting modern goods. I open everyone, because I love the products. Value is king (or queen):

When asked why recipients stopped subscribing to opt-in emails, more than one-half said the content was no longer relevant, and 40% said they were getting too many offers. – JupiterResearch (2007)

By the way, that image is a really great example of an email signup box from the New York Times. The iconography is easy to understand. The call-to-action and offer are compelling. Any doubts about value are easy with a link to a sample email and the privacy policy.

2. Blog


Blogs are so mainstream that it seems silly to mention them. Or, it would if businesses were using them better.

I think the best blogs reflect your target audience and connect with them in a meaningful way. It’s not another news or PR outlet. Those conversations are so inauthentic that they’re not worth the energy it takes to produce them.

Let’s look at a good example: Urban Outfitters.

Putting aside the rather unfriendly horizontal scrolling, the content is surprisingly well targeted. They have geographic specific content for local events (no doubt near their stores). Music, a critical component of the store experience and target market, has its own section. The Projects section opens up the design studios for enthusiasts.

Where’s the conversion? A small link to shopping and some product mentions.

The value for Urban is that they connect with their customers and stay top of mind, not to mention the potential link value from a worthy presence in the blogosphere.

Continue reading

Ecommerce Tip #7 – Speed Up Your Website

In spite of ever rising broadband connections, site speed remains an issue for many users. Jakob Nielsen noted this in 1997 and it holds true 11 years later. In fact, Google incorporates it as a factor in Quality Score.

As you plan your website redesign, don’t forget…

Ecommerce Tip #7 – Speed Up Your Website

Images, javascript and heavy Flash can all way down your site performance in terms of speed and, sometimes, conversion. Before you start slashing and burning features, first assess the problem.

Here’s where I’d start:

  1. Any of your entry pages are obvious candidates
  2. The homepage
  3. Pages with a high bounce rate
  4. Pages with a high exit rate
  5. Content/code heavy pages

Next, run the pages through a few tools:

  1. Site Report Card – Free
  2. Web Page Analyzer – Free
  3. UI Test – Free
  4. Net Mechanic – Pay

Read up on some of the thoughts about site speed

    1. High Performance Web Sites: Essential Knowledge for Front-End Engineers
    2. How To Improve Site Conversion, Minimize Google Ad Cost, And Reduce Your Carbon Footprint
    3. Google article
    4. Check your current standing in AdWords

    Finally, it’s time to optimize.  If you’re cutting out invisible code, go for it.  Trimming image sizes?  Fine.  But if you’re planning on cutting out elements, don’t forget to test!

    Tune in next week for another tip.

    Ecommerce Tip #5 – Write a Less Sucky Return Policy

    Let’s assume you have a great offer and someone who’s interested in your product.  Why might they not buy?  Oftentimes, the answer is anxiety.

    Prospects might be anxious about a lot of things, such as:

    • Privacy
    • Security
    • Spam
    • Value

    Satisfaction can also cause a lot of anxiety.  What do I do if it doesn’t fit?  What if I don’t like it?  What if this cable doesn’t work with my TV?

    That leads me to…

    Ecommerce Tip #5 – Write a Less Sucky Return Policy 

    I bucket the elements of a conversion inducing return policy into 3 areas:

    1. Friendly
    2. Findable
    3. Understandable

    1. Friendly

    It goes without saying, that if you have an aggressively unfriendly return policy that you’ll turn some people off.  What’s unfriendly?

    1. A fee that’s disproportionate to the purchase
    2. Unreasonable time to try out the product
    3. Unfair credits/refunds
    4. No return policy

    There has to be a balance between a profitable policy with reasonable limits and one that’s more consumer friendly.  Don’t forget to check out your competitors too, because your prospects are.
    2. Findable

    Continue reading