We are currently building at AFRICASYS a mobile platform application to solve problem around drugs accessibility for population and one of the major requirements at performance level was expressed as:
« The platform MUST be able to handle 100.000 users requests per minute ». Now that the development as reached the 1st milestone, the team has to prove that the application can actually take that load…. Here comes
The best definition is probably what Apache Foundation said about the tool :
The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions http://jmeter.apache.org/
JMeter is one of the major open source peace of software out there used for Load Testing. It’s popularity forces major proprietary load testing platforms to develop plugin to be compatible with it. So i decided that the team must also go with it.
Fully featured, JMeter gives ability to load and performance test many different applications/server/protocol types:
- Web – HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, …)
- SOAP / REST Webservices
- Database via JDBC
- Message-oriented middleware (MOM) via JMS
- Mail – SMTP(S), POP3(S) and IMAP(S)
- Native commands or shell scripts
- Java Objects
Highly customizable even you have a very particular application & protocols you can easily get the job done through beanshell sampler feature that i really appreciate.
Now that we know the best (at least for our needs) load testing tool, what else has to be done to insure you will get the best from your application at performance standpoint ?
Cloud Native & Scale Out Architecture
To insure that you can get the best out of your application : you must design it right from the start to be able to start small and grow big, very big if necessary. Yes Cloud permits to scale (at a reasonable cost today) but any workload is not made up for Cloud. You cannot just take any app from on premise and move it to Cloud to be able to say you are Cloud Native: it takes for than than.
The essential pillars for me are:
-be able to auto-provision your code to the infrastructure. A process that takes your codes and get them inside your runtime (Virtual Machine or Container)
-be able to auto-scale (out & in) : you application must support the ability to grow horizontally with all the consequences (shared states, temporary data, distributed caching, stateless authentication & request: round robin routing to any nodes for any request , auto-discovery and clustering tools…) and finally the ability to shoot at node at any time without any consequence on your business
–Highly Availability or Auto-redundancy : having at least 2 nodes in 2 different datacenter to insure you service has the less possible chance to go completely off
Our application stack is made of:
JWT authentication, Hazelcast for distributed caching layer, No local/in memory states but Redis Cluster or AWS Elasticache (or equivalent), transactional backend services, Centralized logging with ELK, AWS ELB (or HA Proxy if you are running on AWS), Multi-AZ (datacenter) designed, HA Database layer (MySQL Cluster for our case), Artifactory & Jenkins Pipeline based deployment with SSH connector to push binary (codes packaged) at runtime. You can also build ready to use images (AMI or snapshot or template according to your Cloud provider)
root@devops:/home/africasys# java -version
openjdk version « 1.8.0_151 »
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
Server version: 5.7.20-0ubuntu0.16.04.1 (Ubuntu)
OVH VPS Cloud 2 deployed in RBX datacenter (I’m running the JMeter over internet from my laptop at Morangis – Paris Area)
Do the JMeter JMX test file, run the test, do the analysis on results and recognize the limit of your application
Here is my methodology for load testing. Again i’m not JMeter but i’ve read enough to understand how to analyze report and what are the meaning of JMeter items :
- start with a thread group setting low (we started at 100 concurrent users with a ramp-up period of 10s (10s to charge the workload of those 100 concurrent users)
- observe Error Rate, Average and Standard Deviation: if any error it means your are already above your capability. if not consider Standard Deviation which is supposed to be less the Average/2. If above it’s mean some of your users are getting a degraded experience so you are above your capability else you can increase the concurrent users count in the Thead Group setting.
- iterate to the point that there is still NO error and your Std Deviation get worst (more than Avg/2) : you’ve reached your application limit then take the previous numbers.
We have successfully reached 15.000 users per minute on a small sized application running on 256 mb of JVM Hype
2.500 users / 10s
41 requests / s
15.000 users / minutes
Max : 1.1 s to be served
Avg : 75 ms
Max Std Deviation accepted : 75.2 = 37.5: Value = 34.53 => WE ARE GOOD
Our target application server sizing is 4 Gb on 2 nodes with a Load Balancer in front , the team has not made yet the test on it so we made an optimistic linear extrapolation :
Optimistic (linear) extrapolation expected on Production on 2 nodes of Xms4096m
15K * 4 * 4 * 2 = 480.000 users / minutes
PS: this still has to be proved : maybe DB server is the bottleneck, maybe scalability is not linear at JVM side. This is why i call it optimistic + linear