Let me guess.. the reason you visit this blog is because your e-commerce store is not generating enough sales and you are struggling in finding ways to volume up your sales.

‍

Conversion rate optimization (CRO) is a data-driven process that allows you to deeply understand your customers and shape their experience in order to bring you maximum business results. With data-driven CRO, every conclusion is proven by numbers and data. It is a systematic process of proposing what and how to run your tests.

‍

CRO is also an endless learning process since you can test almost everything – the more you test, the better you become. Each time, you will draw your conclusions and learn valuable lessons for the next more informed test.

‍

If your site generates less than 500 conversions per month (reason being that, with equal split, you can achieve 250 conversions per variation), then instead of A/B testing, focus on answering the following questions:

‍

‍‍There is no reason to optimize before you have validated your business model and what you offer. Optimization will not fix bad businesses. Especially if you are a new player, focus more on your business and what value you are offering your customers.‍

‍

‍Although you might not have enough data to test, you could have enough to draw some useful conclusions from. If you have been selling for a while, you very well might have enough data to analyze. Whenever you draw segments from a specific period, make sure you haven’t done any big changes during that period because this would lead to you draw conclusions based on your past site using data from your present site.

‍

The first thing to remember is that: it’s not your traffic but your conversions that matter. The higher the rate of conversions you are getting, the shorter the time the test would take to give you results. Your site needs a certain number of conversions for the test to be valid.

‍

Heuristics analysis is a method of inspecting potential problem points on the website. The evaluator examines the interface and how it complies with usability principles. This is a widely applied practice and taught in e-commerce where UIs are designed within a short period of time and low budget which limits usability testing.

‍

The analysis starts with the expert sniffing around for quick wins in the battle to optimize your site. The site would be inspected for the most obvious issues with your user interface (UI). This generally applies to the most visited pages according to Google Analytics such as the homepage, collection page, product page, cart page, etc.

‍

What does a heuristic analysis focus on?

1.  Simplicity and clarity of the website and the offer. Users don’t like confusion – your system should give them clear navigational cues and instructions as to where they are and what next step they need to take.

‍

2.  Ease of information handling. The language your UI uses should match the customer’s, and the message you are sending should flow naturally into their mind. Take some time to do customers research and see what language they are using.

‍

3.  User freedom. This is one of the most common mistakes of modern websites. Users want a certain amount of control over navigation, and a constantly sliding header with no way to control it often violates this freedom and leaves customers anxious or frustrated.

‍

4.  Consistency. Use a consistent language, image, and style across your site so that your customers will easily remember you. This also helps to standardize your value proposition.

‍

5.  Error messages. Users will eventually make mistakes. In these cases, deliver a sophisticated message to avoid frustrating or irritating them.

‍

6.  Visible retrieval. Users often forget what they have read, so reminders should be visible to facilitate. In practice, a sticky header or changing the color of an already-clicked button are two smart examples of this.

‍

7.  Aesthetic aspects of the page. An attractive page encourages the user to spend longer on the page, but remember that minimalism is ideal when delivering your message to customers.

‍

Certainly, there are many other factors besides those examples we mentioned above, so our suggestion is that you consider hiring an expert to conduct the analysis for you. Since you’re the one who developed the site, you might have become too familiar with the content, leading you to assume everything is clear and well-reasoned.

‍

Hence, you need someone with expertise and a fresh pair of eyes to evaluate the pages and determine what can be quickly improved. An expert may only need a day or two to conduct a thorough and comprehensive analysis.

‍

Working with an expert in quality heuristics analysis will give you a bunch of low-cost optimization opportunities, allowing you to take a great leap ahead of other companies whose A/B testing choices are based on random guesswork.

‍

Technical testing is what should be done next to ensure your site is functioning properly. As boring as it may sound, by following the points below, you might just discover that your site suffers from terrible technical bugs.

‍

Don’t overlook this steps unless you’re prepared to skip a lot of optimization opportunities. If you don’t have the basic technical knowledge, don’t hesitate to ask a developer for assistance.

‍

The following are things you should be checking for during your technical analysis (remember, you’ll want to test your site on a variety of different devices and browsers):

‍

1. Indexing. More indexed pages means more people will visit your site – which means more money – so when checking your index, consider how these factors are affecting your index: .txt file; the presence of meta tags on the pages; lack of iframe, JS, the presence of Flash or other navigate technologies that prevent indexing; sitemap format speeds up-indexing and reindexing the pages; invalid URL structures; and incorrectly configured server response codes.  

‍

2. Load speed. The higher your page load speed, the more satisfied your users will be; this also increases your search ranking. You can utilize tools like Firebug and Google’s PageSpeed Insights. Remember, errors often appear on small devices so don’t skip response speed testing on them.

‍

3. Bad links and 404 pages. Bad links lead to broken pages, non-existent images, etc. In these cases, users will end up on a 404 page. Too many error pages like this will cause your site ranking to be downgraded. Hence, check for pages that have been removed, whether intentionally or by accident. There are several ways to do it: Yandex.Webmaster Panel, Google Search Console, Xenu Link Sleuth, Screaming Frog Seo Spider, and so on.

‍

To technically analyze a website, you will need to build a to-do list of necessary tasks and always have a developer to assist you if you’re not one yourself. Note, errors often appear on mobile devices, so, pay attention when you do cross-browser testing.

‍

You can watch this video to learn about the four elements to conversion optimization research:

A web analytics analysis will give you an initial understanding of your clients. Where are they spending their time looking on your website, where the biggest drop-offs are, etc.? With this analysis, you are looking to qualify or to disqualify assumptions that were made in the heuristic test, and discover potential problem areas for A/B test ideas.

‍

One popular approach is to focus on the highest value pages such as, for example, identifying the most visited page that also has low conversions. T

‍

his page should be your optimization program’s next focus. Investigate if the customers find what they need on this page and whether or not they were encouraged to take action. If not, why did they stop before turning into a conversion? You should ask yourself if you would act the same if you were in your customers’ shoes. Finally, identify any technical errors that might exist.

‍

The following are the most common questions you should answer during a web analytics analysis:

• Are the buttons or functions of the page actually being used by users? What are the most/least commonly used ones?  
• How often is the search used?
• What are the reactions of the most valuable clients?
• How often are items removed from the cart, and at what point are they removed?

‍

Steps to take when analyzing a page:

• Browse through the site.
• Take notes on which aspect(s) could be improved.
• Develop a list of questions. Always approach analysis with a problem in your mind and questions you want to answer so you know exactly what data you want to analyze and what you want to accomplish.

‍

Google Analytics health check: Use your Google Analytics report to gather data for the following issues:

• Cross-browser and cross-device reports detailing the contribution of transactions made and the bounce rate.
• A number of people landing on a broken link – this, of course, not only affects SEO but also the user experience.
• Loading times – again, compare these to support your technical analysis.
• Use Google data to spot leaks in sales funnels – the focus should be on your “add to cart,” “payment,” and “review your cart” pages to determine how leaks on these pages are impacting the number of final sales.
• Search history – look for the most commonly searched items, sections, and information on your website, how their search differs among different demographics, if the search results are satisfying or not, etc.
  Lastly, you can always put together a custom report to help you understand your users, as this is often more useful than focusing only on Google’s default reports.

‍

Watch the video below to know how you can learn about your best customers by using data from Google Analytics:

‍

It’s easy to focus on the numbers and ignore the people making those numbers happen. Hence, it’s useful to gather user feedback via a customer survey. I suggest a 30-second, on-site survey to see what language clients are using, what would possibly hinder them from purchasing, what they value, what friction points exist, possible objections, etc.

‍

Besides the customer survey route, asking quick questions via chat logs or conducting interviews can also reveal valuable customer insights.

‍

The customer survey will give you:

• Accurate answers about what your most important customers value, their individual behaviors and traits, and what their objections are. This valuable information enables you to make more targeted changes to your site (calls to action, design, products, and service).
• The ability to generate appropriate copy through understanding your customers’ value proposition.
• An informed direction regarding decisions about UX or the implementation of meaningful changes.

‍

Basically, user testing collects information about a user’s experience while engaging with the site. User testing allows you to see your design, copy, service or product through your customers’ eyes so you can generate insightful ideas for A/B tests.

‍

Some factors you should be investigating: the degree of intuitiveness of your site’s navigation system; the clarity, concision, and comprehension of the value proposition; the simplicity of your lead funnel; and any existing friction points.

‍

Different ways of user testing:

• Obtaining user traffic recordings. There are multiple ways to get customer recordings, whether via Hotjar (an intuitive research and optimization tools for web businesses). You can also conduct an exit survey or on-site survey to understand the reasons site visitors customers leave.
• Asking 10 – 15 people to perform a task on your website and ask them to report their feedback. Tools such as TryMyIU, UsabilityTools,  or other similar services can also help to gather this type of data.
• Implementing a chat log on your site to better understand and support your customers.
• And, of course, you can always conduct personal interviews directly with your customers.

‍

If you have at least 100 views per day, run the following analyses:

• Heatmaps to see which areas catch the most mouse hovering;
• Click maps to see what users click on (this can be recorded by Google Analytics);
• Scroll maps to see how far people are willing to scroll down the page. You can then use this to optimize your page length;
• Session replays on the page to visualize traffic patterns and see how your customers react to your site.
• Some tools offer an attention map that reveals the most viewed areas of the page.

‍

Mouse tracking analysis eliminates guesswork and allows you to draw quick and more precise conclusions about user preferences, and in turn generating more reasonable ideas for A/B testing, as this type of analysis converts behaviors into an easy-to-read format that allows you to see what works and what doesn’t.

‍

This CRO research discussion is getting pretty in-depth, isn’t it? As you can see, it is a vital process in which each step relies in some way on the other, meaning that you cannot simply skip one of them if you’re looking for the most accurate results. Thorough CRO research helps you develop the best hypotheses, ensuring maximum data sufficiency and accuracy, which will save you both time and money at the end of the day.

‍

You can always seek help to a professional in this area which will ultimately save you money and time.

Launching a test without a hypothesis is like starting a journey without knowing where you’re heading. It’s likely you won’t get what you want out of all your hard effort, time, and money. If you have zero ideas about what your hypothesis should be, our suggestion is to ponder the CRO research topics in Part 1 again and take a closer look at collected data.

‍

Just like your journey, the success of the test depends strongly on how viable your hypothesis is. Then, how can you develop a strong hypothesis? That’s exactly the question we’ll answer in this article.

‍

A hypothesis is an educated guess or prediction, a tentative assumption you make before running a test.

‍
It is important that your hypothesis states clearly what could be changed, the result you’re expecting, and your reasoning. A hypothesis follows this simple formula:

‍

If X, then Y, because of/due to Z.

In CRO, you would follow this syntax:

If A is changed to ….. , (conversion metric) will be improved/harmed because….

‍

Characteristics of a winning hypothesis:

• It aims to trigger people’s reactions to the on-site changes, either negatively or positively, to see how people perceive your brand. For example, a title informing the event of a sale will create urgency and hypothetically increase conversion.
• It should be easily tested.
• It is insightful and provides learning.

‍

Please watch the video below to understand the four different types of buyers and how to sell to them.

‍

A verified hypothesis will determine whether your assumptions are correct. It allows you to make informed decisions about any intended site changes.

‍

Proponents of a hypothesis

A hypothesis includes 3 main parts: the variable, the result, and the reasoning.

‍

To find out the appropriate variable, you could consider the following common factors:

• CTA and CTA button.
• Product or service information.
• Value communication.
• Page copy.

‍

Try to isolate a single variable for A/B testing by studying data collected from Part 1. You choose from the highest valued variables ( for example the most visited page, most viewed items, etc.).

‍

This can be decided by using the current performance data; you have to predict what you expect your experience result to be. Obviously, the ultimate purpose of the CRO process is to increase conversion, but a hypothesis and changing one element can be useful for identifying ways to influence a specific factor.

‍

Please watch this video the get an idea of the right conversion rate optimization mindset.

‍

‍

You need to first be clear on your conversion goal, what CTA you are expecting, and what’s going wrong that prevents people from reacting to your site in a way that helps you realize your goal.

‍

You can then form a problem that you are looking to solve through testing. After you have a problem in mind, you can start to form a hypothesis. Common problems can be:

• CTA which is not clear and not visible CTA button.
• Lack of product or service descriptions.
• Miscommunicated product/ service value.
• Page copy which doesn’t speak the clients’ language.

‍

Based on your research, you should be able to come up with at least one solution about what you should change (reason can be based on customer interviews, user testing, heat map analyses, etc):

• Choose one isolated variable from the problem-stating step.
• Brainstorm several changes that can be made with that variable and analyze ways to create the best outcome.
• Double-check if your research data supports those solutions or not.

‍

Consider how the proposed change might impact your problem. This should take into account what you want to test and how it will affect your conversion problem.

‍

A solid hypothesis does not necessarily guarantee a win, but it will guarantee a lesson learned about your clients, no matter what the outcome. Here’s one more learning tip: categorize your results by device, browser, traffic source, and visitor type, so that you’ll have the best chance of determining a winning combination. This also supports your learning process.

Now that we have introduced you to CRO research in part 1 and formed a proper hypothesis & outlined the needed treatment in part 2 it is time to cover test implementation, measuring tests, and ongoing learning.

‍

We will start by zooming out from the individual test level covered in the last part and looking at the CRO efforts as a whole.

‍

There is one more step to do before you jump into testing, you need to have your tests prioritized. To prioritize the tests we recommend using a test prioritization framework where you calculate a score for each implementation based on expected impact & ease of implementation. The higher the total score, the higher the priority for testing that assumption:

‍

After you have prioritized your tests, you’ll need to create a testing plan. It is very important to form a testing plan before you move on with executing to ensure that what you are testing is clear in timelines, measurements and expectations. A spreadsheet is a good choice for this.

‍

The testing plan should include:

• Test page URL
• Hypothesis
• KPIs measured
• Estimated test duration
• Test start date
• The test conclusion date (Filled post-test)
• Results (Filled post-test)
• Confidence level (Filled post-test)
• Link to further test info (if any).

‍

Your testing plan instantly tells everyone reading it what are the expected outcomes, timelines, and what we expected to find. It also ensures you’ll be able to change your testing personnel and don’t run any tests twice!

‍

Keep this document for you to review in the future and track as you move along. Because any single test can often take a long time, you might forget the original drivers behind the test. Also, over the years, it is common that your testing personnel or CRO agency to change. So, this document can also be helpful if there are any such changes taking place.

‍

The fewer conversions you have, the bigger of a lift you will need to have to have a statistically significant test. This means that the smaller sites require more radical changes for them to be statistically detectable. This is a good to keep in mind when creating the testing plan.

‍

One of the common worst-case scenarios is being too timid and ending up with an endless stream of undecided results.

‍

You also don’t want your testing to take a long time since, over time, cookies start to expire and seasonal changes come more and more into play. In case you’re not sure what a cookie is, you can read more about it here. Generally, it is recommended that you aim for tests that you can conclude within four full weeks.

‍

This gives you a representative sample size without too much data pollution (cookies expiring, seasonal changes, and monthly behavioral changes).

‍

You will find this tool handy to calculate your required sample size in terms of visitors you need. The required total sample size should not be higher than your total monthly visitors. That’s because if the sample size required is greater than your total monthly visitor count, then naturally, your test will take more than a month.

‍

Your testing plan instantly tells everyone reading it what are the expected outcomes, timelines, and what we expected to find. It also ensures you’ll be able to change your testing personnel and don’t run any tests twice!

‍

Keep this document for you to review in the future and track as you move along. Because any single test can often take a long time, you might forget the original drivers behind the test. Also, over the years, it is common that your testing personnel or CRO agency to change. So, this document can also be helpful if there are any such changes taking place.

‍

The fewer conversions you have, the bigger of a lift you will need to have to have a statistically significant test. This means that the smaller sites require more radical changes for them to be statistically detectable. This is a good to keep in mind when creating the testing plan. One of the common worst-case scenarios is being too timid and ending up with an endless stream of undecided results.

‍

You also don’t want your testing to take a long time since, over time, cookies start to expire and seasonal changes come more and more into play. In case you’re not sure what a cookie is, you can read more about it here. Generally, it is recommended that you aim for tests that you can conclude within four full weeks. This gives you a representative sample size without too much data pollution (cookies expiring, seasonal changes, and monthly behavioral changes).

‍

Also, you can use tools like visual website optimizer, and A/B calculators to calculate the A/B test duration or have a trusted agency run all of this for you.

‍

Now, you are ready to launch the tests!

‍

Now that you are executing the testing plan and are ready to run your tests, you need to make sure you write, design and develop your landing/testing page appropriately. It is good to expect 2-3 rounds of copy editing, design iterations and quality assurance until you get the exact page you’re ready to test.

‍

At this point, your test pages should be bug-free and work perfectly. Even one silly mistake in the testing implementation can screw up all of the hard work you have done so far. This means that you need to make sure your testing variations are thoroughly tested before you launch them.

‍

Moreover, a common mistake is not testing the pages for bugs AFTER you launch them. We’ve heard of numerous stories where pages worked perfectly before launch, and then 2-3 days into a test, the page stopped functioning as expected.

‍

Take the necessary measures to make sure this isn’t you. Otherwise, it’ll most likely lead to 2-3 weeks of lost testing time (and a lot more in lost revenue!).

‍

What you can do in this case, is set up reminders for yourself 2 days, 5 days, and 10 days into your testing period. The reminders are to double-check that test pages are working exactly as expected throughout the testing period. You can also peek into the data to see that conversions are tracked properly, but don’t draw premature conclusions based on very little evidence.

‍

The last step, which is also the most important step long-term is learning. Now that you have run your test, you got your result, and now there is an opportunity for tangible learning that you should not just overlook. When done right CRO is not just optimization.

‍

It is also a powerful method to learn more about the users and user behavior. In many cases, startups have changed their entire business focus based on learnings gathered from A/B testing and CRO research.

‍

Remember: you are running the tests based on assumptions, which means that ideas and opinions don’t matter. We have to be able to validate or invalidate them based on actual data. And this is the time for you to validate or invalidate the assumptions!

‍

By analyzing and learning the changes, you will have a clear picture of your users, what works for them and what doesn’t, and then utilize these findings throughout the site. This is a great example of the Build-Measure-Learn loop, where you utilize what you have learned.

‍

There are two resources to check when you are analyzing the variant and trying to learn from it: Google Analytics (impact on the whole) and heat mapping tool (behavior on that variant page).

‍

With Google Analytics, you can discover the different testing groups and what kind of actions they took on the page and how they moved through the website. This can help you identify possible bottlenecks and patterns amongst multiple “winning” variations or “losing” variations.

‍

With your heat mapping tool, you can review heatmaps and session recordings of winning and losing variations of your test, which can help you understand more about your users and help guide you for future tests that you’ll be implementing shortly.

‍

That’s it!

‍

The steps outlined above and in parts 1 & 2 of this 3-part series are the steps of a perfect conversion rate optimization process. The keyword to emphasize here is process.

‍

On the surface running a couple of tests can appear quite simple but there are a lot of elements to consider while building and implementing an A/B test from start to finish.

‍

We hope that we’ve outlined the tools and methods to help you run A/B tests on your own. However, if you’d like some help optimizing your eCommerce store & want to take it off your own plate (handing it over to experts), you can apply to work with us. Since 2015, we are doing the exact things for 100’s of clients.

‍

You can apply for our support here: www.crogurus.com/apply