The Second Counter Slashdot Experience

On February 23, 1999, around 15:43 European Time, the Linux Counter was listed on Slashdot, causing a breakdown of services.

On Thursday, February 25, 1999, at 11:07 their time, they did it again.

This is a chronicle of what happened.

The Linux Counter Setup

The Linux Counter is a project that has been running since 1993, with the chief aim of letting people tell the world "I use Linux".

It is currently running on a 90 MHz Pentium machine, which was upgraded to 48 Mbytes of RAM.
It is located in Norway, and its connection to the outside world is through a 256 Kbit/second leased line. Its timezone is European (MET, +0100), six hours ahead of the US East Coast, nine hours ahead of California.

This time, the counter stayed up.

I won't say that it handled the load effortlessly - a modern Linux box with the load in the high 10s is not a pretty sight.
I guess many users were turned off by the long delays and didn't get registered; still, many did get through, and got registered.

Things are back to normal now; see the running stats for what the current situation looks like.

Number of visitors per hour

This image shows a well-configured (but underpowered) machine's response to slashdot:

1. Valiant shouldering of the load
2. Trouble from an unexpected source
3. Slow return to normality

In the first minute after the article went up, people arrived.

Soon, the 16 available processes were all busy running my too-heavy Perl scripts, and the new clients were sending SYN packets and waiting. And they kept on doing it.

In fact, so many were doing this that the kernel wondered if there were SYN flood attacks going on. Go figure...

Shortly after midnight, the /var partition, where the HTTP logfiles go, filled up. Apparently something else required access to that partition too - at least registrations weren't successful either.
Luckily, I was online at the time, deleted a couple of files, and watched the counter come back to normal.
After that, it was plain sailing.

Number of operations per hour

This one graphs successful registrations. It went to 300 and stayed there for many hours. You can see the disk-full event shortly after midnight.
The peak to the left is the tail end of the previous Slashdot experience; you can tell how much lower than this one it is.

Number of Web accesses per hour, last 3 days

I didn't know the little box could do that much....

Load on the access line

This time, I was able to capture the day graph before the start of slashdot rolled off the lefthand side. A 90 MHz pentium IS able to serve 200 kbits/second of web pages - but only when the script processors aren't hogging the CPU.

Note: This graph isn't aligned with the others!

The week embedding both slashdot events.

Load on the server

If you want to see a Linux box in pain, look no further :-)
The red line corresponds to a load of "1"; until the disk filled, the load had been stable around 12 since the article was posted.

Who were all these people?

About 8000 people (different IP numbers) visited the site on the 25th; about 1600 people a day managed to execute one or more scripts.

Since the counter lets people tell which country they live in, this actually tells us something about Slashdot readers.

Here are some comparative numbers for Days Before Slashdot, Days During Slashdot, and Days After Slashdot.

Date	US	Canada	Germany	Norway
Feb 13	24	4	8	0
Feb 26	582	98	35	8
Feb 27	563	80	53	21
Mar 6	42	7	6	4

In words: The North Americans increased by roughly a factor of 20, fallling back to a number double their previous registration rate; the Europeans multipled by perhaps five times, and fell back to roughly their previous registration rate.
(Of course, such an "analysis" would get me flunked on a statistics exam - however, I think the tendency is about right; if 563 out of the US' 266 million people registered on the 27th, and 53 out of Germany's 83 million did, doesn't that indicate that there are four times as many Slashdot readers per capita in the US as in Germany?)

Lessons learned

• Slashdotting is still good for you, as an exercise in work under fire.
  The counter's configuration has proved that it stays up and running (as long as there is disk space around).
• A well-handled Slashdot load lasts longer and has more people in it than the load left after the server's been down for a while.
• The adrenaline kick of a slashdotting still feels real good!
• On the 25th and 26th together, 2670 people registered with the counter.

Enjoy!