Weblog

Performance testing while under attack

28 February 2014, 20:30

Chris Wadge has used the slowhttptest tool to see how well several (untuned) webservers are handling the slowloris attack. The results are quite interesting. I'll let them speak for themselves.

Test parameters
Test type	SLOW HEADERS
Number of connections	4096
Verb	GET
Content-Length header value	4096
Extra data max length	52
Interval between follow up data	10 seconds
Connections per seconds	128
Timeout for probe connection	3
Target test duration	240 seconds
Using proxy	no proxy

It's all about the green line and the required time to deal with the bad requests. It shows that Hiawatha stays available for other clients while under attack from one and deals with the attack more quickly than the other webservers. In other words, if you want sleep well at night knowing that your websites are online even while under attack, go for Hiawatha!

Cherokee crash

The Cherokee webserver was also tested. But because it crashed out of the box during the test, it didn't meet the tester's 'untuned' criteria, which was used for all of the other webservers featured.

Ali

28 February 2014, 21:22

You need a reddit button Hugo

Hugo Leisink

28 February 2014, 21:23

I don't know reddit. Tell me what link I need and I'll add it.

Zoltan

28 February 2014, 21:23

Congrats!!!
Well done, I've always known you are the best. Simply the best!

Ali

28 February 2014, 21:34

Hope this helps: http://www.reddit.com/buttons/

Alex

8 March 2014, 00:49

I've been waiting for this test a long time. Bravo Hugo!
I never doubted that Hiawatha is the best

René

11 March 2014, 19:09

Ligt dat nou aan mij? Dat als je langer naar het Hiawatha screenshotje kijkt, de kleurtjes steeds mooier worden.

PICCORO Lenz McKAY

14 March 2014, 16:43

i think we need a test agains lighttpd.. apache are too heavy to compare.. and nginx are too bullisious..

the really confrontation are hiawatha vs lighttpd..

good work

Hugo Leisink

14 March 2014, 20:54

I've added the graph for lighttpd. Again, many thanks to Chris Wadge!

Chris Wadge

15 March 2014, 04:34

Hey all! In the interest of full disclosure, a few more details to note about these tests (hopefully I'm using the right variation of BBCode here):

The test machine for all targets ran on the following: VirtualBox 4.3.8, 4 dedicated vcores (AMD FX-8350), 4GB RAM (DDR3 1866), Debian Sid/Jessie, Linux 3.13, layer 2 via gigabit ethernet (no NAT or intermediate)
None of the target webservers were tuned outside of their defaults, except to point to the same webroot (Debian's tiny, stock "it works!" page) for consistency.
The reason I didn't tune any of the webservers before testing was to keep the most level playing field possible. In other words, one can't be accused of giving one webserver an advantage (accidentally or otherwise) simply by being more skilled at tuning one than another. For a truly scientific test, we'd need double-blind panels of sysadmins to tune *all* of them, take the best results from each, etc. etc. All I wanted was a simple litmus to satisfy my curiosity! Make of them what you will.
I did tune the underlying Debian system, especially the filesystem and network stack, to be as well-tuned as it could be and give each webserver the best chance possible. I did not roll a custom kernel however, which is usually a treatment I give my own servers. The reason is that certain kernel options, most crucially timing, can have a pretty massive impact on webserver performance and behavior. Instead I used the default kernel which shipped with Sid at the time.
I also tested Cherokee [pub.dotbalm.org], which actually would have received "2nd place" were we ranking these, behind only Hiawatha. Unfortunately it's apparently mostly unmaintained these days, and it just crashed out of the box without me tampering with it. Thus it didn't meet my "untuned" criteria, which was used for all of the other webservers featured.
Before anybody accuses me of being anti-Apache, yes, I know all too well you'd have to be a bit of an idiot not to tune Apache before deploying. It can perform a lot better than it did here... In fact, I've tuned Apache for a few of the biggest sites in Silicon Valley, and got pretty impressive results out of it by any standard. But not without a lot of elbow grease, additional caching, modules, testing, etc. Based on my experience with both webservers, I'm very confident I can get better, more secure results with Hiawatha, and with a lot less effort. That's a big win in my book.

James L

22 March 2014, 06:50

Please add OpenLiteSpeed to the test.

Chris Wadge

22 March 2014, 09:29

OK James, here's OpenLiteSpeed [pub.dotbalm.org] on the same testbed with the same criteria.

Chris Wadge

22 March 2014, 10:30

Hiawatha performed well. Though it wasn't within the criteria for the test, you may be interested to note that it also used less CPU cycles and less memory than any other webserver tested to clear all of the connections. And of course, it did it the quickest, too. It's frustrating that more people don't know about your handiwork, Hugo; Hiawatha really is a great webserver.

James L

22 March 2014, 12:06

The link doesn't show OpenLiteSpeed graph.

James L

22 March 2014, 12:12

Good ones! A little strange to see LiteSpeed Enterprise is currently use for shared hosting, is there a difference between OpenLiteSpeed and Enterprise version that handling attacks?

James L

22 March 2014, 12:17

On the other hand, some claims G-Wan web server is the world fastest server, I believe it work well only for static page. Yet, I'm interested if you can benchmark G-Wan as well. It will be my last request

Chris Wadge

22 March 2014, 13:53

Hey James, I'd never heard of G-WAN, so it was an interesting experiment for me. Here [pub.dotbalm.org] are the results.

Also, this not at all within the test parameters as I did some basic tuning, but for fun I cranked up both Hiawatha and the torture test to 11, so to speak. Gave Hiawatha a ConnectionsTotal and ConnectionsPerIP that was essentially limitless, and hit it with a 15-minute, high ramp-up SlowHTTPTest. In short, the results for the much smaller test seem to scale linearly [pub.dotbalm.org].

James L

22 March 2014, 14:10

That's awesome to see Chris! Will consider using Hiawatha for home web server. Would love to see what tuning brings.

Chris Wadge

22 March 2014, 14:54

@James L: Regarding LiteSpeed, it does actually have some anti-DoS features built in which are available in the FOSS version as well, which is cool. But as testing the untuned webserver was the basis of the experiment, I didn't enable any of them. Thus in production, with a tuned OpenLiteSpeed instance, the SlowHTTPTest would fail, but the server would remain healthy. Of course, the same could be said of Hiawatha, since I didn't enable any of its optional security features either. In other words, the test results are unfair, but they're equally unfair across the board.

park

22 December 2017, 19:22

Can you Please add H2O and Caddy to the test???

James L

28 January 2018, 14:36

Didn't know I had comment this back then, now I'm sticking to H2O which has anti DDos, HTTP2 Push Server and all the good stuff.