I was recently working on a project where the client wanted a single zip file to run the application. Yes, a single zip file. It’s probably not what you are thinking. The main issue was with the database, because RDBMS is something different/isolated from the application. I saw this as an opportunity for growth and explored some in-memory databases, because IMDBs can be embedded inside the application and require no separate database file or connection. It worked well, which made me recognize how useful IMDBs can be within a multitude of organizations. This post is so I can demonstrate the working and architecture of IMDBs, and hopefully help them to become more pervasive in the technological world.
An in-memory database system is purely what we say: it requires no disk Input/Output whatsoever, while including cache management, file management and others. For this reason, an in-memory database is also faster than a conventional database that is either fully-cached or stored on a RAM-disk.
In-memory databases support formal data definition language (DDL) and database schemas. They also give support for relational, transaction logging, database indexes, client/server architectures, security features, etc.
Database scientist Jim Gray, working for IBM, conceptualized this technology 30 years ago when he built one of the first in-memory engines (IMS/VS FastPath) way back in 1978.
The first significant commercial IMDB offering was TimesTen (later acquired by Oracle). Since then, almost all top DB vendors boast an IMDB product, such as IBM’s SolidDB or Sybase’s ASE.
|In-Memory Databases||On-Disk Databases|
|All data stored in main memory, no need to perform disk I/O to query or update data.||All data stored on disk, disk I/O needed to move data into main memory when needed.|
|Data is persistent or volatile depending on the in-memory database product.||Data is always persisted to disk.|
|Specialized data structures and index structures assume data is always in main memory.||Traditional data structures like B-Trees designed to store tables and indices efficiently on disk.|
|Database size limited by the amount of main memory.||Virtually unlimited database size.|
|Optimized for specialized workloads, i.e. communications industry-specific HLR/HSS workloads.||Support very broad set of workloads, i.e. OLTP, data warehousing, mixed workloads, etc.|
The above diagram is explained as follows:
Placing the entire on-disk database on a RAM disk will speed up both database reads and writes. These databases are embedded within the application itself. However, the database is still hard-wired for disk storage, and processes in the database to facilitate disk storage, such as caching and file I/O, will continue to operate even though they are now redundant.
IMDB usually runs with the application server in which it is embedded. Additionally, data in an on-disk database system must be transferred to numerous locations as it is used. In the above diagram, the handoffs required for an application to read a piece of data from an on-disk database modify it and write that record back to the database. These steps require time and CPU cycles and cannot be avoided in a traditional database, but when the data resides in the RAM, minimal CPU cycle and null disk I/O is required.
Connection String <property name="driverClassName" value="org.hsqldb.jdbcDriver" /> <property name="url" value="jdbc:hsqldb:file:/home/vikask/elmo/db/elmo;" /> <property name="username" value="sa" /> <property name="password" value="" />
In this simple Java project to demonstrate Hibernate, HSQL and Maven using Java Annotations, the HSQL database is used to make the project simple. This is because we can use in-memory database and we would only need a JAR file to be included in our project.
In-memory databases are among the top ten technologies impacting the IT sector. The speed in which application infrastructure technologies are maturing brings us faster and more affordable semi-conductors, meaning that mainstream use of in-memory computing (IMC) is more likely in the future.
At present, the IMDB system has greater market share and the in-memory application platform is projected to increase significantly. According to Research and Markets, in-memory data management is expected to reach $1 Billion by 2016.
This forecast is based off of a sharp rise in transactional and analytical requirements for voluminous big-data. IMDB solutions are quickly finding homes in almost every database warehousing and analytics application. Today, database management and analysis technologies are ubiquitous. From private sector organizations to public/government sector, data management and analysis technologies play a vital role in achieving operational efficiency. Current organizations are finding a need for technologies that can store more data in less space while analyzing at higher speeds.