Advances in In-Memory Data Management (IMDM) technologies have made it possible to analyze more data in less time resulting in rapid adoption. These technologies rely on RAM to store large datasets in-memory and with increasing data volumes, the RAM costs can become a significant budget item. Several customers that do not need micro-second latency or are burdened with increasing costs, would accept a slightly lower application latency if it comes with reduced cost.
A new innovation in the IMDM space allows SSDs to be used along with RAM for in-memory data management without impacting the application using the in-memory data. This capability enables an application to scale it’s in-memory data needs simply by introducing additional SSDs in the RAID configuration. Application latency requirements can also be managed a lot better as the in-memory data volume changes by adjusting the ratio of available capacity within RAM and SSDs. And, most importantly, the total cost of ownership becomes more manageable as SSDs are significantly cheaper than RAM.
Terracotta’s BigMemory 4.1 introduces this new & innovative feature called BigMemory Hybrid that seamlessly uses SSDs as an extended in-memory tier along with the available RAM without requiring any application changes or programming. This feature treats the full in-memory store (RAM + SSDs) as a single entity and can search across the full data set to find the right key/value pair(s). Additionally, this feature supports ‘Fast Restart’ backup and recovery to restore BigMemory servers in case of a crash regardless of the data being stored in RAM or on SSDs. To provide additional failover, BigMemory Hybrid is designed to work with WAN replication to support disaster recovery across multiple data centers. All industry leading SSD vendors are supported like Intel, Fusion-IO, etc. as part of this new feature.
The image below shows how BigMemory Hybrid works across three different memory tiers with the associated TPS and data volumes.
Figure 1. BigMemory Hybrid allows you to expand BigMemory in Terracotta servers, keeping more data closer to your application for increased transactions per second (TPS)
If you are interested in learning more about this exciting new feature, please leave a comment below or contact me.