Just a Simple Load Test of REST APIs Written in Python, JavaScript, Java, Golang and Rust
This is more or less of a follow up of Dexter Darwich’s article in 2020. I recently got a job as a technical writer, and my company is using Rust for back-end services. I’ve used Java in my first job and I’ve learned Python, modern JavaScript and Golang in the previous job (editing IT books). I did try to learn Rust but haven’t got very far. But I’m curious how would they compare to each other after two years.
Python and Java are still very popular, at least in my country, for implementing production services. I see them in job openings all the time. Are they really that practical compared to newcomers like Go and Rust? Is using JavaScript as back-end language also a good idea?
Of course, like the other article, this is not meant to be a precise, serious and credible test, and the result may vary a lot on different machines with different testing parameters. You are welcome to test them yourself, preferably on larger scales than I did.
The REST API
I wrote the same “Hello World” API in five languages. It would extract a name parameter from the GET request, for example,
http://localhost:3000?name=Beeblebroxand returns
{
"message": "Hello Beeblebrox!"
}If the name parameter is not provided, it would return “Hello World!” in the message field instead.
I also added some requirements:
- Explicitly returns HTTP status code 200 (OK). There will be times that you need to send code 500 or 405 to indicate something went wrong.
- For Java, Go and Rust, a struct or class is used to define JSON fields. This is the proper way to do in real REST APIs — I tried not to go the “convenient” way by using JSON strings. But I still use a dictionary in Python and an object literal in JavaScript, since this is how the script language are meant to be used.
Contestants
Here’s the list of the languages and tools I’ve used:
- Flask 2.1.2 running in Python 3.10.4 with or without uwsgi as server (200 workers)
- Express 4.18.1 (JavaScript) running in Node.js 18.1.0
- Opine 2.2.0 (TypeScript) running in Deno 1.21.3
- Spring Boot 2.6.7 (Java, project generated by Spring Initializr and compiled to executable jar by Apache Maven 3.8.5) running in OpenJDK 18
- Golang 1.18.2 with Gin 1.7.7 (compiled to executable binary)
- Rust 1.60 with Axum 0.5.5 (compiled to executable binary with --release flag)
All of these frameworks support simultaneous or parallel requests with various capacities.
Like Darwich, I deploy all these to Docker containers (I used Docker Desktop 4.8.1 on Windows 11) so they can be more properly compared. All code use local host with port 3000. All executable exes or jar are also compiled in the container.
All the code, including Dockerfile and how to run them on your machine (although I don’t have a Linux environment to test them) as well as the detail test results, can be found in this repo.
Docker Image Size
I used the following Docker images to make them as smallest as possible:
- python:3.10.4-alpine3.15
- node:16.15.0-alpine3.15
- denoland/deno:alpine
- maven:3.8.5-openjdk-18-slim (builder) / openjdk:18-alpine3.15 (deploy)
- golang:1.18.1-alpine3.15 (builder) / alpine:3.15 (deploy)
- rust:1.60-alpine3.15 (builder) / alpine:3.15 (deploy)
REPOSITORY SIZE
py-api-docker 354 MB
ts-deno-api-docker 118 MB
java-api-docker 346 MB
go-api-docker 20.4 MB
rust-api-docker 10.9 MB
js-api-docker 174 MB
py-no-wsgi-api-docker 59 MBI did not use multistage Dockerfile for Python because uwsgi is installed globally (the “traditional” way is to install packages in a virtual environment in the builder stage image then copy it into the deploy stage image, but this will not do for uwsgi).
Idle CPU/Memory Usage
Load Test Tools
My laptop has a Intel i5-1135G7 (2.4G, 4 cores) processor with 16 GB RAM installed. To avoid problems, I ran one container at a time during testing.
I use K6 as the testing tool. The script (script.js) is like this:
import http from 'k6/http';
import { sleep } from 'k6';export const options = {
vus: 100, // simulate 100 users
iterations: 10000, // total 10000 requests
};// k6 main function
export default function () {
const port = 3000
http.get(`http://localhost:${port}?name=${makeid()}`);
sleep(1); // wait 1 sec between requests
}// generate random names
function makeid() {
let text = "";
const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; for (let i = 0; i < 5; i++)
text += possible.charAt(Math.floor(Math.random() * possible.length)); return text;
}
This JavaScript script would have K6 simulate 100 users to send total 10,000 requests. Each user would wait 1 second between requests. For each request a random-generated name with 5 letters would be attached into the URL.
To run the test, execute
k6 run script.jsThe output result would be like this (this is Go’s first result):
running (01m41.2s), 000/100 VUs, 10000 complete and 0 interrupted iterations
default ✓ [ 100% ] 100 VUs 01m41.2s/10m0s 10000/10000 shared itersdata_received..................: 1.5 MB 15 kB/
data_sent......................: 910 kB 9.0 kB/s
http_req_blocked...............: avg=11.03µs min=0s med=0s max=3.95ms p(90)=0s p(95)=0s
http_req_connecting............: avg=3.73µs min=0s med=0s max=2.09ms p(90)=0s p(95)=0s
http_req_duration..............: avg=6.24ms min=839.2µs med=4.79ms max=122.68ms p(90)=7.54ms p(95)=8.52ms
{ expected_response:true }...: avg=6.24ms min=839.2µs med=4.79ms max=122.68ms p(90)=7.54ms p(95)=8.52ms
http_req_failed................: 0.00% ✓ 0
✗ 10000
http_req_receiving.............: avg=33.31µs min=0s med=0s max=4.7ms p(90)=0s p(95)=351.9µs
http_req_sending...............: avg=18.97µs min=0s med=0s max=3.92ms p(90)=0s p(95)=0s
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=6.19ms min=839.2µs med=4.74ms max=122.52ms p(90)=7.42ms p(95)=8.43ms
http_reqs......................: 10000 98.78779/s
iteration_duration.............: avg=1.01s min=1s med=1.01s max=1.12s p(90)=1.01s p(95)=1.01s
iterations.....................: 10000 98.78779/s
vus............................: 100 min=100
max=100
vus_max........................: 100 min=100 max=100
I also monitors the CPU and memory load using docker stats command and record the “normal” value as best as I could.
Test 1#: One Second Request Delay
The first test is 100 users sending 10,000 requests with 1 second delay:
CPU usage:
- Python: ~10–20%
- Python/no uwsgi: ~20%
- JS: ~3–7%
- TS/Deno: ~1.5–2.5%
- Java: ~5–30%
- Go: ~0.5–1.5%
- Rust: ~0.5–1.5%
Test 2#: 0.1 Second Request Delay (High Loading)
This is a more unrealistic test, also 100 users with 10,000 requests but reduce the delay to 0.1 seconds. This would increase the load and make the difference more significant.
I restarts the containers before testing. Some languages might perform better after a while.
CPU usage:
- Python: ~150–200%
- Python/no uwsgi: ~125–135%
- JS: ~30–40%
- TS/Deno: ~10–20%
- Java: ~50–250%
- Go: ~8–10%
- Rust: ~5–10%
We can remove the super-high duration value of Python without uwsgi:
We can perhaps deduct the following observations:
- Go and Rust are the fastest — in the second test Go is actually a tiny bit faster, but in my previous tests they performs practically the same at higher loading.
- Rust’s memory usage is the lowest.
- Python with uwsgi actually performs pretty well but the memory usage is staggering.
- JS (on Node.js) and TS on Deno also have very similar performance (not far behind Go and Rust) except the latter needs more memory.
From the Coding Perspective
Of course, performance may not be everything. Sometimes you simply need to write code fast.
The Rust version of code is perhaps the most complicated and hardest to read (further more, the Axum doc does not give you a straight answer either):
use axum::{extract::Query, http::StatusCode, routing::get, Json, Router};
use serde::Serialize;
use std::collections::HashMap;#[derive(Serialize)]
struct Msg {
message: String,
}#[tokio::main]
async fn main() {
let app = Router::new().route("/", get(handler)); println!("Rust REST service started...");
axum::Server::bind(&"0.0.0.0:3000".parse().unwrap())
.serve(app.into_make_service())
.await
.unwrap();
}async fn handler(Query(params): Query<HashMap<String, String>>) -> (StatusCode, Json<Msg>) {
let name = match params.get("name") {
Some(value) => value,
None => "World",
};
let msg = Msg {
message: format!("Hello {}!", name.trim()),
}; (StatusCode::OK, Json(msg))
}
You also need to define dependencies correctly in Cargo.toml:
[package]
name = "rust-api"
version = "0.1.0"
edition = "2021"[dependencies]
axum = "0.5.5"
serde = {version = "1.0.137", features = ["derive"]}
tokio = { version = "1.18.2", features = ["full"] }
Golang/Gin by comparison is more concise with pretty straight forward functions:
package mainimport (
"fmt"
"net/http"
"strings" "github.com/gin-gonic/gin"
)type Msg struct {
Message string `json:"message"`
}func main() {
gin.SetMode(gin.ReleaseMode) router := gin.Default()
router.GET("/", handler) fmt.Println("Go REST service started...")
router.Run("0.0.0.0:3000")
}func handler(ctx *gin.Context) {
name := ctx.DefaultQuery("name", "World")
msg := Msg{
Message: fmt.Sprintf("Hello %s!", strings.TrimSpace(name)),
} ctx.JSON(http.StatusOK, msg)
}
Java is a bit lengthy (actually one the longest if we count those import lines) but still easy to understand enough:
package com.example.java_api;import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;class Msg {
public final String message; public Msg(String message) {
this.message = message;
}
}@SpringBootApplication
@RestController
public class JavaApiApplication { public static void main(String[] args) {
SpringApplication.run(JavaApiApplication.class, args);
} @GetMapping("/")
public ResponseEntity<Msg> handler(@RequestParam(value = "name", defaultValue = "World") String name) {
String message = String.format("Hello %s!", name);
Msg msg = new Msg(message.trim());
return new ResponseEntity<>(msg, HttpStatus.OK);
}}
By the way, the Spring Boot project’s default ip/port can be set in /src/main/resources/application.properties:
server.address=0.0.0.0
server.port=3000However, the efforts to set up a Java environment (I did it in VS Code; you can find instructions here) is way more complicated than any other languages. It was already like this a decade ago with Eclipse, except that I don’t need an external Apache Tomcat server now.
JavaScript (Express), TypeScript on Deno and Python (Flask) are very similar and actually looks extremely simple:
# Pythonfrom flask import Flask, request, jsonifyapp = Flask(__name__)@app.route("/", methods=['GET'])
def handler():
name = request.args.to_dict().get('name', 'World')
msg = {
"message": f'Hello {name.strip()}!'
}
return (jsonify(msg), 200)if __name__ == "__main__":
app.run(host='0.0.0.0', port=3000)
There is also a wsgi.py that is the entry point for the uwsgi server. See the repo for more details.
And
// JavaScriptimport express from "express"const app = express();app.get("/", async function (req, res) {
const name = req.query.name ?? "World";
const msg = {
message: `Hello ${name.trim()}!`
};
res.status(200).json(msg);
});app.listen(
3000,
"0.0.0.0",
console.log("JS REST service started...")
);
I set the JavaScript project to use ES Module instead of CommonJs.
Then:
// TypeScript on Denoimport { opine } from "https://deno.land/x/opine@2.2.0/mod.ts";const app = opine();app.get("/", async function (req, res) {
const name = req.query.name ?? "World";
const msg = {
message: `Hello ${name.trim()}!`
}; res.setStatus(200).json(msg);
});app.listen(
"0.0.0.0:3000",
() => console.log("TS/Deno REST service started..."),
);
Unlike Node.js, Deno projects does not need package.json and supports TypeScript by default. Opine’s API is also very Express-like with only some minor differences.
Ending Thoughts
Both Go and Rust are very fast — of course, we tested them with 3rd party packages, this doesn't necessarily apply to other usages, and many experiments online already proved that Rust can usually be faster — but Rust is clearly extremely memory efficient. Lower memory usage means you can run more containers on the cloud for more service capacity. Rust programs is also known for being very stable; you just have to pay the price of steep learning curve.
On the other hand, Go is a lot easier to learn and code if you have experiences in C-like languages. The garbage collection usually makes it a bit slower than Rust, but sometimes being able to rapid-develop features might be important too.
Flask is still popular probably for its potential for machine learning projects and for Python being a “beginner’s language”. Pure Flask don’t need much memory and is easy to code. With uwsgi it can actually handle requests pretty well, but the memory trade-off is too significant to ignore.
JavaScript is always one of the most popular language for its front-end origin. With environments like Node.js and Deno they can be used for back-end developments as well. For some people, using full-stack JS might be just what they want, and it is easy to learn as well. And JavaScript (TypeScript would be compiled to JS too after all) also runs pretty fast, thanks to the V8 JavaScript engine.
As for Java…I haven’t write Java for a long time and never missed it. Apparently there are lots of legacy systems around, many people still use it, and I guess you can still find ways to speed it up (using GraalVM for example). Otherwise not so recommended in my opinion.
Let’s all for today, thanks for reading!
