Too many companies…

And Too many “experts”…

Are doing the same critical mistake

They are trying to optimize using split testing in the wrong way!

In many cases,

When companies hire my service to help them optimize their products & funnels

They are in the following situation:

  1. Chose and implemented a split testing tool
  2. Identified using analytics the problematic places in the funnel with the biggest drop or the place in the product that has problems.
  3. Started doing split testing
  4. With the split testing they sometimes succeed, sometimes fail
  5. But overall, they seem to come to a point where they are stuck
  6. They invest time & money in creating tests but no real progress is really achieved.

The sad thing,

Is that they don’t realize the real problem

Most of them believe that if they can succeed if only they:

  • Had more time…
  • Had more people on it
  • Could do more tests…
  • There was more traffic they could reach results faster

They don’t realize they have become gamblers.

Constantly hoping that in the next round ( test ) they will get the grand prize.

I show them that they have been using split testing in the wrong way,

We make the necessary changes

And in a very short time they start seeing constant uplifts

Not only in conversion rate & on-boarding

But also on the business KPI’s

Let me explain in detail, How to do this in the right way to get remarkable uplifts.

Why it’s working in the beginning ?

In the beginning the page is usually NOT optimized At all and contains a lot of friction

so almost every change that is being done according to best practices will result in an improvement

So they follow this process:

  1. Use Analytics to know that you have a problem in that page…
  2. Use heat-maps to see How users are interacting with that page
  3. Suggest Hypotheses
  4. Create relevant Treatments that are based on Optimization/CRO best practices

and in the beginning it is working…


but than they reach the point that the page is relatively friction less…

They already picked the LowHanging Fruits

and it is now becoming more and more difficult…

The Challenge

the maze of optimization possibilities

They reached a point that nothing is simple any more

there are simply too many parameters !!!

Let’s take for example that the goal is to optimize the Home Page:

  • There are many elements on that page ( hero image, CTA, secondary CTA, testimonials,..)
  • There is a big dependency between the elements – they support each other – for example – if you change the main CTA from “book a room” to “Search” – can you really keep everything else the same ?
  • There are many possible arrangements of those elements on the page
  • For each element there are many possible “treatments”
  • The traffic is composed of many prospect “types” – any change you do have a positive impact on some of them and a negative impact on others.
  • MOST of the prospects need more then 1 session to be converted – so it shouldn’t be All or Nothing for this session.
  • So on…

The number of possible combinations is simply too high…

This is why people in those companies start to feel like gamblers

Each split testing is becoming closer and closer to a bet

The problem is with the way of thinking

most companies are trying to solve questions like:

  • What are the elements that I should put
  • What is the best version of each element
  • What is the best arrangement of elements
  • And so on…

But those are NOT the questions they should be asking…

In this stage, in order to advanced beyond that point

it is a MUST to really understand your customers


Only when you understand why people behave in a certain way and What is impacting their decision making

Only when you understand Why, You can start optimizing in a structured way and achieve real progress.

The real questions you should be asking

Optimizing based on customer understanding

The keys to unleash your full potential, to uncover the ENORMOUS OPPORTUNITIES UNDER THE SEA LEVEL

Is to forget now about the specific pages/places you have a problem

And Understand exactly what is going on in the mind of your prospects.

Only then we can go back and optimize the funnel

(in many cases, during the process of understanding…we realize that the funnel itself is not optimized)
The questions you SHOULD be asking are totally different

They should be focused around at 4 areas :

1. Why People are considering to buy a product/service like yours?

  • What is the “Job” they are really trying to accomplish?
  • What is the unique Context that they have?

2. What are their Game-Changing-Success-Parameters?

  • How do they measure and compare available solutions ( like your product/service)?
  • What are the parameters that are most important for them (Consider parameters in 3 dimensions : Functional, Emotional and Social)?

3. What are the forces around the current solution they use?( they always use a solution, and from time to time they “switch” to a new solution):

  • Problems & Limitations of the existing solutions – pushing them to switch to a new solution
  • Existing habits & Switching costs – stopping them from making the change ( notice that “switching costs” are not only real costs but also emotional & Social.

4. What are the forces around switching to your product?

  • Benefits of your solution ( measured as perceived change in Game-Changing-Success-Parameters)
  • Concerns about the new solution ( functional, emotional and social)

You get the answers for those questions by implementing various flows into your company:

  • Automatic tools – Polls, Surveys
  • Interviews
  • Question asked by support teams / Customer success teams
  • And many more

This is a very BIG subject,

If you’re interested that I’ll write about this please comment below or contact me (

In either case, make sure you’re subscribed to get updates:

Creating Hypotheses

Once you really understand your prospects & Customers,

You can go to the next phase which is to create Hypotheses & Treatments
Example 1:

Assuming you’ve learned that prospects are very concerned that they will not be able to import their data from the current solution. You are able to tailor specific treatments to minimize that concern and build a Hypothesis that introducing that treatment will cause that concern to be minimized and will be resulted in an uplift in conversion or other goal.
Example 2:

Assuming you’re a restaurant chain that learned that her most loyal customers with the highest LifeTime Value are actually guys who use tinder to date ( an Actual real case !!!) you might suggest a treatment to change the ad copy from something like : “Saving money on dinner” to something like : “Relieving the pain of going on expensive dates”.

The Real Role of Split Testing

Only now we have reached to the actual role of split testing !!!!

Which is to…

Approve/Disapprove Hypothesis about how certain treatments will impact a certain KPI’s.

PLEASE…Do not underestimate this role!

When making changes in critical places along your funnels and product,

You MUST conduct a relevant split testing,

There are always surprises,

  • The basic understanding of the customers is wrong
  • The treatments are not good
  • The treatments had a negative impact on just a few customers, but they are the most important customers
  • And so on…

So…always verify your important changes.

Split testing role is to verify hypothesis about how certain treatments will impact KPI’s Click To Tweet

The Right way (Summary)

  1. Understand your customers in the 4 key areas
  2. Review the entire funnel/customer journey
  3. Create Hypothesis to answer the right questions
  4. Identify places with the biggest potential
  5. Develop treatments
  6. Verify with Split testing
  7. Repeat..over and over


You’re invited to speak your mind…Write a comment!

Learn more on How I’m helping companies like yours boost their business KPI’s >>