The database query log could be a crucial part of a database restoring process required after data corruption caused by either an illegal update operation or a damage of database files. The main task here is to find the right format for storing the queries.
The format should be targeted at programmatic data retrieving, and be minimalistic to keep the size of the audit collection, that holds the query log, to a minimum. Also, it must contain additional information such as the time of modification, or the collection name. You could imagine the following set of properties:
- collection - the collection name
- operation - the update operation name
- timestamp - the time of modification
- custom fields - to keep the data
Custom fields would depend on the update operation. For instance, for the 'insert' operation, only one custom field must be stored - the 'object' field which contains an object that is being inserted into a collection, this way the operation:
|
will result in the following log record:
|
Let's have a look at the custom fields required to log all the available Java driver operations:
Operation | Object | 'Criteria' object | 'Sort' object | 'Update' object | 'Insert' flag | 'Multi' flag |
Add | + | |||||
Remove | + | |||||
Update | + | + | + | + | ||
FetchAndRemove | + | + | ||||
FindAndModify | + | + | + | + |
The number of optional fields is very limited and a lot of operations use the same objects and flags, so a simple plain old object can be used to represent a log record in your application:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Of course, to persist log data to a MongoDB audit collection, instead of a POJO, you should use the wrapper
for the BasicDBObject
document:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
|
Records stored in this way can be queried and read by humans, which gives sufficient debugging capabilities and, also, can be converted to update queries during an automatic restoring process.
This post is written for the MongoDB Java driver version 2.4
Scala / Groovy Developer & Technical Lead
Looking to hire a software developer?
Don't hesitate to contact us.
Comments