Magento 2 and 1 Million Products

Can Magento 2 handle 1 million products?

I came across that question many times. It got me interested and I started searching the web. I didn’t find an answer so I decided to set up my own experiment.

In this article I will install the latest Magento 2.2.2 (M2), load it with 1 million products and look at the performance. I will describe the procedure in details and will provide specifics so anybody could replicate and prove my findings.

Preconditions

Here is what I will be running:

1. Magento Open Source 2.2.2.

2. Debian 8 server with Linode 12G plan: 12G RAM, 6 CPU cores, 196Gb SSD.

I will be using Magento 2 performance toolkit to generate data for testing. It comes with M2 and it helps with testing the impact of various 3rd-party extensions and customizations.

Load 1M products

M2 performance toolkit has different profiles:

1. Small profile: 800 simple products, 30 categories, 200 customers, 80 orders.

2. Medium profile: 24k simple products. 300 categories, 2000 customers, 50000 orders.

3. Large profile: 300k simple products, 3000 categories, 5000 customers, 100k orders.

4. Extra large profile: 600k simple products, 6000 categories, 10k customers, 150k orders.

Profiles are defined as XML files and are stored in setup/performance-toolkit/profiles/ce/ folder inside Magento root. Here is an example of medium profile:

Medium Profi

We would edit extra large profile XML to set the maximum number of simple products to 1 million:

-- <simple_products>600000</simple_products>

++ <simple_products>1000000</simple_products>

Then we would run an import process:

php bin/magento setup:perf:generate-fixtures  setup/performance-toolkit/profiles/ce/extra_large.xml

It would take a while to complete. I had to run it two times because the first time it crashed MySQL server:

MySQL cr

The second time it run out of memory:

Out of memory

I then manually reindexed it:

php bin/magento indexer:reindex

Then again it failed on Catalog Rule Product reindex:

Reindex failed

It seemed the server just could not handle the testing data that big.

I decided to cut down the number of catalog price rules from 20 to 2, categories from 6000 to 60, customers from 10000 to 100 and orders from 16000 to 1500. I made certain edits to extra_large profile and restarted setup:perf:generate-fixtures.

This time it went OK and now I have 1,400,000 products in Magento 2:

1 million SKUs

Speed Measurements

Now it is time to feel 1 million products. I have to say I put store in production mode and enabled all cache options.

We will be measuring server response time or time to first byte. Full page load time depends on network capacity and can be different from computer to computer.

Admin Panel

Backend seems fast. TTFB of Dashboard page is 1.37 sec:

Dashboard-TTF

Catalog > Products takes 2.12 sec:

Catalog-TTFB

Catalog > Categories takes a bit more, 3.43 sec:

Catalog-Category-TTFB

Frontend Performance

Storefront is not so fast. Of course Full Page Cache helps but for the sake of an experiment I will turn it off:

FPC is off

We are using the default Magento 2 Luma theme.

Homepage responds in 846 ms:

Homepage-TTFB

Category page responds in 10,03 sec:

Category-TTFB

Adding a configurable product to a cart takes 1.34 sec which is alright:

Add to cart TTFB

Takeaway

Magento 2 can easily handle more than 1 million products. Frontend speed is bearable and backend seems fast enough.

Reindex process does take time, in my case it was around an hour for all indexers to finish.

PS: Check out the article where I loaded WooCommerce with 1 million products and compared its performance to Magento 2.

If you find this post interesting do not hesitate to sign up for our newsletter and join the 1472 people who receive Magento news, tips and tricks regularly.

Thank You!