If you ever wrote production multi-threaded server in Java you know how it is difficult to implement load balancing between worker threads. You need to fight many issues to have good load balancer:
- You need to limit somehow the number of worker threads, because with unlimited thread pool you can have memory exhausted.
- You need to implement sophisticated procedure for clean worker shutdown.
- If you are using Executors you might now that they are not provide out of the box solution. You need to run into some tricks to have Executors do the load balancing job right.
- And for multi-threaded code you have to use those painful synchronized and locks that make your application lock up and/or degrade performance. And the debugging lock issues is a real pain.
ZeroMQ might tremendously help you with solving all these problems and much more. ZeroMQ is a high-performance asynchronous messaging library. It is native, but it has bindings for many languages and to be fair it is worth the hassle of dealing with native code.