Concurrency models still matter. I really believe that, and I hope you do too. In fact, my very first InfoWorld article in 2012 compared the concurrency models for the most popular runtimes (Java, Python, and PHP). If I ever write a book on software architecture, it might be a chapter. The editor cut the hell out of it to read better on the web and turned it into “How to make PHP apps scale.”
If you don’t care about concurrency models, here’s why you should: Developers are caught up in a lot of different software stacks. However, many of these stacks are associated with particular runtimes, and those runtimes have concurrency models associated with them. The trade-offs imposed by these different runtimes are really important, as the following sections explain.
Multiprocess runtimes
The original Apache HTTPD WebServer was multiprocess. (You can still use this model but it isn’t what most people do these days.) PostgreSQL is also multiprocess.
Basically when you run ls -l or dir from the command prompt, a process is started that is managed by your operating system and runs on your processor. In modern operating systems, it has its own memory space and can be preempted (suspended from using resources) so other processes can run. In server software, you end up with a process per thread of execution—that is, each process has only one thread. There is a master process that manages the whole thing (or answers the HTTP requests, for example).
For the original Apache Web Server, that meant each web request that came in had one process.
The advantage of multiprocess is that it is very stable. If a process dies, it doesn’t have to affect anything else. The disadvantages are that starting a new process is pretty expensive, and communicating between subprocesses is cumbersome. There are ways, but honestly there aren’t good ways to communicate between these processes. You have awful ideas like shared memory, operating-system-specific interprocess communication methods, and actual remote procedure calls (RPC). Frankly, having a server open up socket connection connections to itself is gross.
No modern software should be written this way. There was a time that Linux and other platforms didn’t actually support native threads and had to fork off a subprocess for concurrency. So supporting multiple platforms meant you had to do these torturous things. But in the modern era, there are better ways.
Multithreaded runtimes
Modern operating systems support having a process manage multiple threads of execution. Threads can be dedicated to run on a processor. The main difference with multiprocessing is that the threads in multithreaded runtimes share one memory space. The downside is that if a thread corrupts that memory space or does other very bad things, it will crash the whole thing.
Most Java Virtual Machine (JVM)-based software is multithreaded, as are many modern database platforms. You know how when you’re using Jetty or Tomcat and something sucks down all the memory and the whole thing crashes? That’s the trade-off of multithreading. And that out-of-memory exception is the “nice” version of multithreading, with managed (garbage-collected) memory. If the application code is written in C and you manually dimension memory, you can have other nasty buffer underrun and overrun errors to crash that process out!
The upside, such as it is, to multithreading is that the threads don’t have to create a whole new memory space each time. Also, you can pool one memory space. From both a performance and scalability standpoint, this is a clear win. Additionally, communication between threads is much easier in a multithreaded model than in a multiprocess model. You do have to worry about things like thread safety and synchronization, but there are weird permutations of those things with subprocesses as well in a multiprocess runtime.
Mixed multiprocess and multithreaded runtimes
This hybrid runtime mixes subprocesses with threads. It gets most of the scalability advantage of the multithreaded model but if, say, one process drives then threads then if one thread blows up its memory space it blows up just its ten requests when it core-dumps. That’s much better than blowing up the whole thing.
This hybrid model is available in modern Apache Web Server as the worker module. Other server software uses this threading model as well.
Nonblocking/event-driven runtimes
OK, so this isn’t actually a thread model. It is a technique for dealing with I/O and is orthogonal to the actual thread model.
In all the multithreaded techniques, threads exist in a pool. One way to attack a server is to open a bunch of connections and exhaust the thread pool. (If there is no limit on the number of threads, you can just take down the whole server by eating up the all the CPU to the point that the admin has to hit the Off switch because the server is too slow to log in to.)
In an event model, a thread is returned to the pool as soon as it does any I/O (read a file, make an RPC call, call the database) and is then available to handle new requests. So fewer threads are needed to handle more requests.
Modern Apache Web Server offers this approach as the event module. Node.js also does this, but its thread pool size is 1. That’s not ideal because a thread
can be saturated and so you’ll end up manually managing the multiple processes.
Still, the implication of the nonblocking/event-driven model is dramatic. As you move to higher-scale software, you’ll want an event-driven model despite the increased complexity of coding this way. You’ll also want threads, by the way.
Understanding the limitations of the runtime models on concurrency and how to navigate them is important. Not only for development but because one day it will all come crashing down and you’ll need to understand this to know how, why, and where to debug! (You’re welcome.)