Web Analytics and Statistical Bias

by John D. Cook, PhD

Sample Bias

The people who block ads and cookies differ from those who don’t by more than just their aversion to ads and cookies.

Around 30% of users actively block ads. Sometimes this also means blocking the JavaScript that drives analytics.

Technologically savvy, privacy-conscious users are more likely to use VPNs, ad blockers, etc. This group may skew younger and higher income. Therefore the missing 30% will not be a random sample from the overall population.

This becomes most important when these tech-informed users are the target audience, both in terms of ad targets and analytics. This could impact analytics to the point where Google Analytics is not helpful in measuring sales, even though may still provide some useful metrics in terms of inbound traffic.

On the other hand, sometimes the target audience is less likely than usual to block ads or cookies, such as large governmental, educational, or corporate organizations that enforce the use of a browser like MS Edge, which does not block third party cookies by default. Another example would be ads aimed at older or lower income users who are less likely to be blocking.

Cookie Acceptance Statistics

Studies on the percentage of people blocking first or third party cookies deliver wildly variable results, which suggests true figures are not well understood.

This study suggests that around 40%-60% of users accept all cookies, 20% reject all the cookies they can, and the remainder reject some cookies and not others.

In the majority of cases using standard cookie walls, first party cookies would be treated as essential and therefore no opt-out needs to be given. The exception would be sites using first party cookies for analytics, which are a small minority.

Firefox, Safari, and Brave block third party cookies by default. That’s probably somewhere around 21% of users. No major browsers block first party cookies by default because that can significantly degrade functionality.

 

Happier Users & More Traffic

We’re passionate about delivering value through design, data and development. Let’s talk!