Comparing PHP-FPM, NGINX Unit, and Laravel Octane

This article compares the performance of several different web servers for a Laravel-based application. What follows is a lot of graphs, configuration settings, and my personal conclusions which do not pretend to represent universal truth in any way.

I myself have been working with NGINX Unit (+ Lumen) for a while, but I still see PHP-FPM being used in new projects quite often. When I suggest switching to NGINX Unit, a question naturally ensues, "How exactly is it better?". A search for relevant articles on the Internet yields few results: mostly quite old and uninformative articles on obscure websites and a 2019 article on Habr that does a similar comparison of several web servers for a Symfony-based application. But, first, PHP and NGINX Unit have already gone far ahead since its publication; second, I was interested in performance for Laravel and Lumen. So, I cobbled together a simple test bench with several web servers and did some load testing, the results of which I now share.

About the testing bench

Test bench stats:

• CPU: AMD Ryzen 9 5900X 12-Core

• RAM: DDR4 4000 MHz 32GiB x2

• SSD: Samsung SSD 980 PRO 500GB nvme

• OS: xubuntu 20.04

Testing applications:

• Laravel 8.69.0

  Laravel is one of the most popular frameworks for PHP. Out of the box, it contains lots of functionality that solves almost any business needs in a reasonable time. Meanwhile, it is flexible enough: almost any task can be implemented in many ways, starting with designing the application itself (although there is a minus here: some ways of solving a task can cause lots of problems ranging from the need to support abysmal code to performance declines and memory leaks).

• Lumen 8.3.1

  A lightweight version of Laravel, intended mainly for API implementation. In terms of functionality, it kept the most important components of Laravel, trimming what's not critically important (for example, facades). At the same time, it performs better than Laravel but doesn't solve all tasks equally well. As it usually happens, there is a tool of choice for each specific task. In some tasks, Laravel does better, in others, it's Lumen, while for some PHP is not an option at all.

Web servers:

• Php-fpm

  Standard PHP process manager. Each request requires framework initialization.

• NGINX Unit

  An application web server developed by the nginx team. Each request requires framework initialization.

• Laravel Octane

  Strictly speaking, this is not a web server but an application management package from the Laravel team. Under the hood, it uses the Swoole or the RoadRunner web server. The framework is initialized with the first request, then stored in memory and does not reinitialize at subsequent requests.

PHP version in all builds: 8.0.12

Testing tools:

• Yandex.Tank for load testing. The most recent Docker image available is used.

• Telegraf (included in the Yandex.Tank toolset) to collect resource usage statistics for applications.

• Overload (also from the Yandex.Tank kit) for plotting.

Configuration

Applications

All applications have a standard configuration. Error logging is disabled. The cache is collected at the first access only for Laravel/Lumen components (such as the router, etc.), and there is no cache for controllers. The application has a single endpoint that processes the request when accessed to return a built-in JSON response containing a random number, a configuration setting, a static string, and request headers.

Example service response:

{
    "app_env": "prod",
    "type": "unit-lumen",
    "number": 1527706674,
    "headers": {
        "accept-language": [
            "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7"
        ],
        "accept-encoding": [
            "gzip, deflate"
        ],
        "accept": [
            "text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/avif,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3;q=0.9"
        ],
        "user-agent": [
            "Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.99 Safari\/537.36"
        ],
        "upgrade-insecure-requests": [
            "1"
        ],
        "connection": [
            "keep-alive"
        ],
        "host": [
            "10.100.9.3"
        ],
        "content-length": [
            ""
        ],
        "content-type": [
            ""
        ]
    }
}

All applications run in Docker containers. All applications use a custom php.ini with the following settings:

upload_max_filesize = 50M
post_max_size = 50M
opcache.enable=1
opcache.interned_strings_buffer=64
opcache.max_accelerated_files=100000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=0
opcache.validate_timestamps=0
opcache.max_wasted_percentage=10
apc.enable_cli=1
memory_limit=256M

Individual changes, if any, are noted for each application.

To test PHP-FPM, an additional nginx container is used since PHP-FPM works via the FastCGI protocol. I used the official nginx:1.19.6-alpine image.

user  nginx;
worker_processes  auto;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
worker_connections  2048;
}
http {
include       /etc/nginx/mime.types;
default_type  application/octet-stream;
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

access_log  /var/log/nginx/access.log  main;

sendfile       on;
tcp_nopush     on;
resolver_timeout 10s;
server_tokens off;
keepalive_timeout 3;
reset_timedout_connection on;
client_body_timeout 2;
send_timeout 1;
server_names_hash_bucket_size 128;
client_max_body_size 32m;
proxy_buffers 4 512k;
proxy_buffer_size 256k;
proxy_busy_buffers_size 512k;

gzip on;
gzip_comp_level 9;
gzip_disable "msie6";
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml;

include /etc/nginx/conf.d/*.conf;

}

server {
listen       80;
server_name  localhost;
error_page   500 502 503 504  /50x.html;
location = /50x.html {
root   /usr/share/nginx/html;
}
location / {
    try_files $uri /index.php$is_args$args;
}

location ~ ^/index\.php(/|$) {
    fastcgi_pass 10.100.9.100:9000;
    fastcgi_split_path_info ^(.+\.php)(/.*)$;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME /var/www/public/$fastcgi_script_name;
}

}

Guinea pigs a.k.a. containers I've built

1. nginx + PHP-FPM + Laravel

memory_limit=512M
pm = static
pm.max_children = 100
pm.max_requests = 1000

2. nginx + PHP-FPM + Lumen

The configuration is completely identical to nginx + PHP-FPM + Laravel

3. NGINX Unit + Laravel

It so happened that a little earlier I had already built a container with NGINX Unit for PHP8 since there were no ready-made containers that met my production tasks. Without much hesitation, I used the same container as the basis for the test, slightly modifying it. You can see that the build is based on ubuntu:hirsute; admittedly, I am not on such friendly terms with apline, and there was no time to deal with it then. I needed a build here and now, and I had no plans for it to reach production in its original form. Although, given the test results, it may now happen.

{
  "listeners": {
    "*:80": {
      "pass": "routes"
    }
  },
  "routes": [
    {
      "match": {
        "uri": [
          "*.manifest",
          "*.appcache",
          "*.html",
          "*.json",
          "*.rss",
          "*.atom",
          "*.jpg",
          "*.jpeg",
          "*.gif",
          "*.png",
          "*.ico",
          "*.cur",
          "*.gz",
          "*.svg",
          "*.svgz",
          "*.mp4",
          "*.ogg",
          "*.ogv",
          "*.webm",
          "*.htc",
          "*.css",
          "*.js",
          "*.ttf",
          "*.ttc",
          "*.otf",
          "*.eot",
          "*.woff",
          "*.woff2",
          "/robot.txt"
        ]
      },
      "action": {
        "share": "/var/www/public"
      }
    },
    {
      "action": {
        "pass": "applications/php"
      }
    }
  ],
  "applications": {
    "php": {
      "type": "php 8.0",
      "limits": {
        "requests": 1000,
        "timeout": 60
      },
      "processes": {
        "max": 16,
        "spare": 16,
        "idle_timeout": 30
      },
      "user": "www-data",
      "group": "www-data",
      "working_directory": "/var/www/",
      "root": "/var/www/public",
      "script": "index.php",
      "index": "index.php"
    }
  },
  "access_log": "/dev/stdout"
}

4. NGINX Unit + Lumen

The configuration is completely identical to NGINX Unit + Laravel

5. Laravel Octane (Swoole) + Laravel

I was prompted to add Octane to the application test suite by a message from Taylor (creator of Laravel) on Twitter. I was confused by his statements of Octane being faster than Lumen and a slight increase in speed. In this regard, I decided to conjure a container with Swoole-based Laravel Octane for the tests.

Besides Swoole, Laravel Octane supports RoadRunner; initially, I planned to test it as well. But strangely enough, I could not get the latter to work normally even with crutches inside Docker (to be precise, I succeeded, but it couldn't make it through auto-deploy, only starting manually), so I axed it. However, Swoole was enough for my purposes. In addition, Laravel Octane + Swoole adds additional tools that may be useful in solving a number of my tasks.

OCTANE_SERVER=swoole

php artisan octane:start --port=80 --workers=16 --max-requests=1000 --host=host=host.docker.internal

Yandex.Tank

For testing, I decided to use two different load profiles reflecting two situations: "standard" for daily frequent loads, "stress" for understanding the limits of system performance.

phantom:
  address: 10.100.9.101
  uris:
    - /
  load_profile:
    load_type: rps
    schedule: step(5, 100, 5, 10s) const(100, 2m30s)
  timeout: 2s
console:
  enabled: true
telegraf:
  config: 'monitoring-nginx-unit-laravel.xml'
  enabled: true
  kill_old: false
  package: yandextank.plugins.Telegraf
  ssh_timeout: 30s
overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  token_file: 'overload_token.txt'

phantom:
  address: 10.100.9.101
  uris:
    - /
  load_profile:
    load_type: rps # schedule load by defining requests per second
    schedule: line(1, 1000, 10m)
  timeout: 2s
console:
  enabled: true
telegraf:
  config: 'monitoring-nginx-unit-laravel.xml'
  enabled: true
  kill_old: false
  package: yandextank.plugins.Telegraf
  ssh_timeout: 30s
overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  token_file: 'overload_token.txt'

The timeout of 2 seconds was not chosen randomly. First, in API applications, the endpoint taking too long to respond is most often fatal for the application's performance (I regularly use an API gateway within the microservice architecture, always rigidly setting the request timeouts: depending on the task in hand, from 1 to 3 seconds, but no more. But at the same time, if the endpoint implements really heavyweight logic, I always extract such things into jobs). Second, when testing PHP-FPM with a default timeout of 11 seconds, Yandex.Tank didn't have enough load capacity because it waited too long for a response (more on this later).

Telegraf

Everything is simple here; I created a minimal configuration to collect data on resource consumption.

<Monitoring>
  <Host address="10.100.9.101" interval="1" username="root">
    <CPU/> <Kernel/> <Net/> <System/> <Memory/> <Disk/> <Netstat /> <Nstat/>
  </Host>
</Monitoring>

Load testing and results

For testing, all participants were strictly limited by the number of logical processor cores:

• Container with the application: 16 cores

• Nginx (for PHP-FPM tests): 2 cores

• Yandex.Tank: 4 cores (2 cores weren't enough to run with PHP-FPM even with a reduced timeout)

• The extremely important and valuable process of forwarding /dev/zero to /dev/null (questions are unnecessary, it's my personal crutch): strictly 1 fully loaded kernel

• All OS processes: the remaining 1 core, actually free, in reserve

All applications were cold-started, i.e. the endpoint was not pre-heated and the cache was not collected before the test was started.

Next, the testing graphs. If there is no error graph, there were no errors.

I did not add the memory load graph to avoid bloating the post indefinitely. In synthetic tests, memory consumption is almost zero, and the difference is minimal. For approximately the same reasons, the containers had no restrictions on memory usage, since there is always enough of it.

"Standard" load profile

1. nginx + PHP-FPM + Laravel

Detailed report

2. nginx + PHP-FPM + Lumen

Detailed report

3. NGINX Unit + Laravel

Detailed report

4. NGINX Unit + Lumen

Detailed report

5. Octane (Swoole) + Laravel

Detailed report

Response time percentiles (ms)

"Stress" load profile

1. nginx + PHP-FPM + Laravel

Detailed report

2. nginx + PHP-FPM + Lumen

Detailed report

3. NGINX Unit + Laravel

Detailed report

4. NGINX Unit + Lumen

Detailed report

5. Octane (Swoole) + Laravel

Detailed report

Response time percentiles (ms)

* An asterisk in the table will mark PHP-FPM builds that could not bear the load at all. They will indicate the rps cutoff where they began to collapse. Respectively, the percentiles are given under this cutoff. I decided to add them to the table anyway, even in this form.

The Laravel Octane build is listed twice. It has full stats indicating where the server could not withstand the load, and further data up to the point of collapse. As with PHP-FPM, an asterisk indicates the threshold rps.

Conclusions

The results of the testing, I must admit, quite surprised me. Of course, I expected PHP-FPM to be slower than NGINX Unit, but not to such a degree: the difference in response generation time is almost 10 times. I was also surprised to see PHP-FPM being the only one to choke completely during the stress test. Perhaps this is due to the fact that I am not an expert in nginx-PHP configuration, but the rest of the servers running on the same config passed the stress test, albeit with losses. As a result, I concluded that my switch to NGINX Unit a long time ago was for the better.

As for the performance comparison: if we put aside PHP-FPM, NGINX Unit is the only server that passed the stress test without losses, although I had high hopes for Octane. Consequently, I will not take PHP-FPM into account any further.

Bursts in execution time for the 100% percentile can be observed in both NGINX Unit and Octane + Swoole. In both cases, they aren't catastrophic, and the reason can be that the testing was done in an environment that, putting it mildly, wasn't up to this task. Again, all the tests were purely synthetic; in real conditions, when the application will, among other things, run business logic, access the database and churn the cache, the situation will be different.

Technically, Octane will be faster in real conditions due to application state and all connections being preserved. In addition, Swoole has its own high-performance cache implementations and some other features to speed up heavy business logic. But there is also a price: first, everything rests upon the fact that Octane needs more memory (I understand that the price of RAM in data centers is not high for large businesses, but it's worth considering); second, what's more importantly, Octane is extremely unfriendly to bad code; it simply doesn't forgive mistakes. There is even a section on the official website with implementation examples that work on other servers but cause the application to fail on Octane.

On the other hand, NGINX Unit proved to be more stable under high loads. Meanwhile, Unit in conjunction with Lumen yields almost no ground to Octane in synthetic tests, so I doubted Taylor's words for a reason. Lumen will still live and continue to surprise (as for me, I did not expect it to be so fast compared to Laravel).

As a result, I concluded for myself that NGINX Unit is the go-to choice for an extremely stable service that will go on even where others can't survive. And for it to run really well, it still makes sense to use Lumen, API-wise. However, if you need a superfast service to run complex business logic on the fly, the choice is Octane - if resources allow (factoring in both the memory considerations and the fact that when high loads are expected, the load balancer and horizontal scaling should be taken care of well in advance). In this case, Octane is the proverbial "weapon of mass destruction" that is not to be used indiscriminately, in which Taylor is absolutely right. For each task, its own tool.

That's it; I hope this will help you choose a tool that solves your problems.

Thanks for translate @aliensstolemycow and the rest of the NGINX Unit team