Threading, as a software method, is bad.

The idea is good in itself: Make the program execution split up in several threads, which all run independently; creating an illusion that the program is doing several things simultaneously – and, if you have multiple cores, processors or hyperthreading, the program actually does several things at once.

It is a wonderful way to scale – letting ten users work with the same program, or hundred users, is all a matter of how many threads you create.

But from experience, we’ve realized that threading is very hard to pull off without glitches. Why?

• It has proven very difficult to isolate threads from each other: They all access the same data, so you need data isolation through locks. But sometimes it’s hard to predict – especially in a GUI environment – which data that require locks or not. We’ve discovered instances where deeply embedded components were not “thread-safe” but actually used the same data source.
• When threads collide, they do so unpredictably, and can be a nightmare to find. A program can run 10.000 queries without problem, and query 10.001 will crash the whole program.
• It is very difficult to debug, and exponentially more so to observe thread collisions in a debugger.
• Threading problems are more likely to occur in production systems that development systems.

.NET and Java do mitigate these problems somewhat, but still doesn’t solve the fundamental problem – that threaded programming is hard, error-prone, and can lead to very serious failure in production environments.

Erlang, the language developed by Ericsson, solves this in a wonderfully simplistic way. The main difference between a process and a thread in a conventional system is that threads share global data between them, whereas processes all reside in different address spaces and thus share no global data. Erlang doesn’t have any global data, and thus, the distinction disappears. As they say,

“If you think Java is stable, Erlang is rock steady.”

So, is there a way to mitigate this situation? There just might be.

What if there was a way to split the process into separate “services” that share no data between them? What if it was possible to create a message-based system of requests and responses, where the system itself would be able to create services as needed? And if the only way to communicate between these services would be through some kind of message system which is guaranteed to be stable? (WM_COPYDATA comes to mind.)

One might argue that copying data back and forth in the system is slower, more memory-consuming, and less efficient. But if we are indeed approaching a paradigm shift in programming, where the limits of a single CPU has been reached, it only makes sense to adopt a different methodology to work in a multi-CPU environment. More threads, more concurrency, more CPU’s running more threads simultaneously – then the problem of increasing speed can simply be dealt with by, basically, adding more CPU’s. Instead of tuning each thread at the expense of computational stability, you tune the available services and the overall flow of the system, which is one of the cornerstones in highly parallel environments. (Think of a brain.)

Of course it would pay off to fine-tune individual components/threads/services as well; the advantage of the service-based setup is that it would be remarkably easy to extract one service and test it, simulating a virtual testing environment around it.

Such a system might very well be implemented in Delphi, in a similar way as Erlang, by building services that run as threads and operate on Windows message queues. And since, ideally, no data would be shared between them, it would make no difference whether the individual components were in-process or out-of-process. Building on the Serialization module I built earlier, this might be quite easy to accomplish.

Likely candidates would be, for instance, our internal web server than I’ve never been able to make as stable as I want it.