e-Commerce Advent Calendar

At the moment I work in Akamai on mPulse (a web-performance real user monitoring tool) and I have not worked on e-commerce codebase for about 5 years but I still love to visit some of the Magento related events and spend quality time with folks from the Magento community that I can honestly call friends.

The motivation

Few months ago I visited the Magento UnConference in Untrecht, Netherlands and I wanted to share some ideas and to talk about what I was mastering the last few years. I proposed a talk where I wanted to talk about Core Web Vitals monitoring and some easy things that folks could do to get some monitoring for free.

One of the things that was covered was the CrUX dashboard which is a web page where one can just write a hostname, click a button and get a monthly breakdown of the Core Web Vitals for a given hostname.

I wanted to share something useful so people could go to the office the day after, open the CrUX dashboard and understand where their Magento shops stand in terms of Core Web Vitals.

I asked the audience if someone could share the domain name of a Magento 2 shop that they are working on. The audience was a bit shy but finally someone suggested using the Adobe Marketplace domain. We looked at the Core Web Vitals and we had a discussion but something was missing … I really wanted to make it relatable and to look at the Magento 2 shop that someone from the audience was working on.

I was really puzzled after my session and I started asking myself "What is actually the situation with the Core Web Vitals & Magento 2?".

I had a few discussions with friends in the evening and I got the following idea: "Let’s collect the Core Web Vitals information for the whole Magento 2 ecosystem and visualise it.".

The research

TLDR;

I collected interesting data that makes sense for me because I would like to get a good level of detail like having dimensions about what theme is used, what third-party Mangeto 2 modules are used and more. However, after I made the first results public Barry Pollard pointed to something interesting that gives a very good high level view about the Magento 2 web-performance state.

It's event more interesting because we can compare different platforms:

Check out the Core Web Vitals Technology Report (cwvtech.report) here: Shopify vs Magento 2

---

My research

In a few days, I acquired a list of over ~43 000 of Magento 2 domains. I also wrote a script that was calling the PageSpeed Insights API and was extracting the Core Web Vitals information.

It's important to say that the I collected was for combined for Desktop and Mobile together. What Barry suggested for Core Web Vitals Technology Report has a breakdown option for Desktop and Mobile.

The rest of the work was already routine. I set up ClickHouse DB, imported the data in ClickHouse DB and visualised it with Grafana.

I collected data from the end of January 2023 till the middle of December 2023 … let’s call it roughly 11 months.

Also not all domains had CrUX data. I am aware of 2 cases when this could happen: 1) a webmaster opted out a domain from CrUX or 2) a website has a very low number of visits.

Also it took about 4 hours to scan all the ~43 000 … the PageSpeed Insights API is fast but has a rate-limiting maximum number of requests per minute. I had to implement something that introduces a retry logic with a delay when I was getting rate-limited.

Between 20 000 and 25 000 domains had CLS, FCP, LCP and INP.

Let’s look at the collected data.

CLS

The CLS has the following thresholds:

• between 0 and 0.1 - Good.
• between 0.1 and 0.25 - Needs improvement.
• over 0.25 - Poor

This is what we got for CLS for 2023.

Also to make it more clear I charted only how the good levels of CLS progressed over the year.

FCP

The FCP has the following thresholds:

• between 0 and 1.8 sec - Good.
• between 1.8 and 3 sec - Needs improvement.
• over 3 sec - Poor

This is what we got for FCP for 2023.

Also to make it more clear I charted only how the good levels of FCP progressed over the year.

LCP

The LCP has the following thresholds:

• between 0 and 2.5 sec - Good.
• between 2.5 and 4 sec - Needs improvement.
• over 4 sec - Poor

This is what we got for FCP for 2023.

Also to make it more clear I charted only how the good levels of FCP progressed over the year.

INP

The INP has the following thresholds:

• between 0 and 0.2 sec - Good.
• between 0.2 and 0.5 sec - Needs improvement.
• over 0.5 sec - Poor

This is what we got for INP for 2023.

Also to make it more clear I charted only how the good levels of INP progressed over the year.

In closing

I would love to see the Magento 2 ecosystem improving more on CLS, FCP and LCP. I consider FCP and LCP as low hanging fruits and often image optimization, using a CDN, improving Time to First Byte or reducing the number of render blocking resources solves the problem.

The harder metrics to optimize according to me are INP and CLS. INP for example often requires rewriting/simplifying your JavaScript, identifying a badly written third-party script, identifying a CSS animation that consumes a lot of resources and all the other crazy things that could make a frontend janky.

I am pleasantly surprised that Magento 2 has more than 80% of the origins with good INP values.

Also an evolution of this research would be to have a way to be able to add more details like, what Magento theme is used, country, industry, does the Magento shop use a CDN ... There are so many things that can be added to the equation and developing a simple crawler should do the trick.

Let's see what will happen in 2024.

---

Interested in submiting an article?

Please check our contribute page in case you are interest to submit an article.