E-commerce is growing globally, forecasts for 2020 promise it to reach over 4 trillion USD and over 4.2 billion USD for retail e-commerce. We are approaching a very special season for online retail business owners: Black Friday, Cyber Monday and preparing for the holiday season. Every team behind an e-commerce business is planning big for this time. Every online store will try to attract as much traffic as possible. Are you planning to do the same? Let’s assume your efforts will be successful, and at the big sale time there will be a huge number of visitors at your website. Are you ready for such success? Is your website ready? Even a single additional second to load a page can make a visitor leave, so you have to make sure everything operates as smoothly as possible. So what should you do to prepare for your success? Load test in advance!
Load testing goals
Your website works perfectly for the usual amount of visitors you have, it operates smoothly and load times are fine. But chances are high that holiday season is a busy time of year for you, and you’re expecting more web traffic than usual. What happens if suddenly five times more visitors appear? How about ten times more? The great marketing efforts along with the budget spent to attract all that audience can create decent revenue only if the website works as expected. If load times grow, or even worse, the website becomes unavailable, you lose money instead of generating income. That is why experienced e-commerce players load test their services to prepare for such traffic increases. Finding out how many concurrent users your website can handle appropriately is a necessary step in this preparation.
In this blog post we will describe some practices of preparation for large sales by load testing that helped a customer of Loadero to get ready for last year’s sale season. One of the main targets of the load test was to test the page load balancer. It should distribute the incoming HTTP requests among available servers, or add more servers to the ensemble, if the load is getting too high for the current amount of servers. So to test the balancer we have to simulate a load of concurrent visitors.
It starts with a test script
A performance or a load test has to simulate the same actions a real user does, when visiting an online store with an intention to purchase. If your users usually login before shopping, testing this process and your login page is also very important. Make sure to check out our post on testing login process, if that is your case. So in the case we are looking at today, the customer has a huge online store, and when the sale is on, visitors will come to the website, choose products, add them to a cart and make the purchase. That is exactly what a test script should do.
To have a constant load from the user, the script contained a cycle of navigation through pages and adding products to the cart multiple times before going to the checkout page. Loadero uses Selenium for browser automation and the script can be written in Java or JavaScript, if you are new to this our beginner’s guide to test automation with Javascript can be very helpful for you.
It may be reasonable for you to test other features that must be working under high load. Common e-commerce load testing practices also include having a huge number of registered users logging in during a short period of time, checking if order confirmation emails are sent swiftly and correctly, making sure stock tracking does its job, when multiple users order the same product. When creating a test script, keep in mind that your e-commerce website third party integrations can also be impacted by thousands of load test participants. Make sure that your test doesn’t create any unwanted requests for stock fulfillment, logistics, on-demand production or any other kind of integration you might use since it might generate unexpected revenue loss.
A load test with thousands of concurrent participants
Once the test script is ready, it can be scaled in a matter of minutes in Loadero, we have described this process in detail in one of our previous blog posts, make sure to read it if you are doing the same.
Whatever your target number of participants in the planned load test is, it’s always a good idea to start small and gradually scale up. Our customers’ testing target was to check the e-commerce website performance with 50 000+ users, yes – 50K. The starting point was a successfully executed test with 500 concurrent participants, once it ran smoothly the number of participants was increased by 10 000 at a time. For each test created there was a different number of participants ready to be launched from different locations around the world. If the test run was successful and the page could hold the load, another 10 000 increase in participant count was made and the test was executed again.
For load tests it is very important to choose an appropriate start interval and increment strategy. Start interval describes the total amount of time in which all participants start their tests and access the service. Having a large start interval may cause some participants to finish the test before other participants have joined at all. Increment strategy specifies participant distribution over start interval when beginning test execution. This is useful because 10k participants joining at the same time would create a DDoS attack instead of load test and, for this reason, we advise using incremental load. Loadero allows four increment strategies for different scenarios. You can read more about them in our wiki.
Getting the maximum insights from a load test
Having a detailed results report of your test run is very handy, but getting a clear understanding of visitor experience is no less valuable. If you are running performance tests in Loadero, consider switching to the session recording mode for more insights on what is happening during the test. It enables you to review a test run for debugging or seeing the operation of a web service yourself. Session recording is not available for large scale load tests. But you also can visit the website while it holds the load yourself. Every time you run a load test, there also must be human participants, to check and evaluate the usability of the service under the high load.
One of the most important parameters that was constantly checked was the page load time. The customer has determined the limit for that. So if the page failed to load during this time, it was considered unusable for the visitors. Finding the threshold, where this happens, was one of the load testing targets.
Load testing goals achieved. What’s next?
So once the number of concurrent users that can visit the website without experiencing too long load times is known, how can you proceed? What if the marketing team can attract more visitors than that number? There can be different approaches to this, our customer decided to have a queue. Once the number of maximum concurrent visitors without the load delays is reached, new visitors are added to a queue with a counter. When a user has finished shopping or has left the website for some other reason, the first one waiting in the queue is let to the website to start shopping.
It may sound very easy, but this functionality has to be tested as well. To do so the script should be executed with the number of participants higher than required to start the queue. The main target of the tests was to make sure the queue is working correctly and as expected . Once this is confirmed, it is safe to say that the service is ready for the big sale and a huge number of visitors can be attracted to the website.
Steps to load test your e-commerce website
Is your web service ready for the sale season as well? We sincerely hope it is, but you have to make sure before spending your marketing budget on advertising your online store. Here are the steps you should take to load test it in advance:
- Sign up for the Loadero free trial to prepare for the load testing free of charge, no payment information required.
- Create a project, a test and configure it. Here is an explanation of this with examples.
- Run your free tests to validate that your script has no bugs and works as expected.
- Upgrade your account and scale the test in five easy steps for load testing.
- Run a load test and analyze your test run results.
If the analysis of the results report indicates there are problems in your service, make sure to fix them before advertising big for the sale season. Doing that in advance can save you a lot of money and also give peace of mind.
If questions about using Loadero appear along the way, you are always welcome to contact our support team via web chat in Loadero interface or email. We would like to help you achieve every testing goal and have a bugless, smoothly operating service in the end.
