Recently I had a task to develop the application which will have large work queue and which need to survive the restarts. The application need to be lightweight. After trying several different persistent engines for Java I''ve chosen to stick with Berkley DB Java edition. This persistent engine is pretty lightweight it is fast, optimized for multi-threaded usage and have no problems with reclaiming free space.
As I needed the fast persistent queue at a cost of possible data loss on system crash I've chosen non-transactional API for Berkley DB. With non-transactional API the great speed can be achieved for persistent queue at a price of loss of some data at system crash. The more data you allow to be lost the greater speed of the queue you will have. Though you can opt to sync to disk each operation on the queue and in that case your data loss will be minimal.
There is some twist to use more specific data storage engines than RDBMS. The mongodb is a modern feature-rich non-RDBMS database. It can be used with grails quite well, though some special care should be taken. In this post we will create base secure grails application powered by mongodb.
If you are going to use mongodb for production grails application you might stuck with the issue that replica set is really not supported by current version 1.0.0.M1 of mongodb plugin.
It is quite surprising because mongo java driver supports replica set configuration quite fine.
The problem is that GORM plugin is built on top of version 0.5.1 of gmongo library which doesn't support replica set. Only current development version 0.7 of gmongo library supports this feature.
Log4j doesn't allow to catch stdout and stderr messages out of the box. However, you can still intercept them with a custom output stream, which is especially useful when you have to log data that third-party libraries write to the standard streams.
This has already been done by Jim Moore (have a look at the
LoggingOutputStream in the log4j source code). The issue is that this
org.apache.log4j.Priority which are now partially deprecated.
If you ever wrote production multi-threaded server in Java you know how it is difficult to implement load balancing between worker threads. You need to fight many issues to have good load balancer:
ZeroMQ might tremendously help you with solving all these problems and much more. ZeroMQ is a high-performance asynchronous messaging library. It is native, but it has bindings for many languages and to be fair it is worth the hassle of dealing with native code.