Why Use an Embedded Key-Value Store?

When building software, you want your application to be as maintainable and as fast as possible. An embedded key-value store, or ‘KVS’, can help you build a wide variety of applications with minimal setup.

By the end of this article, you’ll understand why it’s worth considering an embedded key-value store for your next application.

What is a key-value store?

According to Wikipedia:

A key-value database, or key-value store, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, and a data structure more commonly known today as a dictionary or hash table.

What makes a Key-Value Store ‘Embedded’?

A KVS is considered ‘embedded’ when used as a library linked to your service and NOT run as a separate service. So, running an instance of MongoDB alongside your microservice would not be considered an embedded key-value store.

Let’s look at another example: SQLite is run as a library that is embedded at the application layer, and therefore this is an example of an embedded database. SQLite, however, is not of key-value structure.

Advantages of being ‘Embedded’

Extremely Fast Data Reads

In the diagram below, you can see that a standard DB server requires a network roundtrip for each query. One advantage of using an embedded library is that your queries don’t need to leave the application server — saving valuable resources and speeding up your application.

Low Latency Writes

Because there’s no network latency and the embedded DB is used mostly within the same machine or VM, it provides extremely fast writes for applications.

Easy Testing

Since your embedded KVS stores your data in memory, all you need to do is delete the relevant files, and you can move on to your next test suite. This makes writing tests so much easier and a lot faster.

Small Footprint

Some embedded databases are smaller than 1 MB, which makes them particularly suitable for mobile and IoT devices with limited memory.

A Classic Example of an Embedded Database

Most browsers, including Chrome, Firefox, and Safari use ‘IndexedDB’ to store browser data. If you head over to the dev tools, you’ll actually see it there under the storage tab.

If Chrome had to access a remote SQL database for each request, it wouldn’t be very efficient, as each request would take longer to handle.

Instead, your browser is confined and stores data such as the URLs you have accessed or browser history, your cached pages, and credentials in an embedded DB — this makes the browser a perfect example to use an embedded KVS.

Another Example of a KVS — The Embedded Storage Engine

The storage engine communicates between the Database Management System (DBMS) and the file system as seen in the diagram below:

The DBMS consists of multiple integrated components for creating, accessing, and modifying data in databases.

The storage engine is a component inside the DBMS used to store data. It does so by leveraging a file system at the operating system level.

A storage engine, such as Speedb or RocksDB, is a great use case of an embedded KVS, as it’s important to optimize for enormous amounts of reads and writes in order to store data. Since the storage engine is a separate component, it can easily be changed out without affecting the rest of the stack.

Key-Value Stores are really, really fast.

Key-value stores use simple operational commands such as get, put, and delete, which makes them extremely efficient at read and write operations. The KVS paths are shorter and allow for more operations per second compared to other database structures.

Key-Value Stores are Great for Plenty of Use Cases

KVSs are commonly used for in-memory data caching to speed up applications by minimizing reads and writes to slower disk-based databases. KVSs are fast, efficient, and simple to use. These properties allow for a wide range of general use cases, including:

• Real-time recommendations — the KVS can quickly access and present new recommendations as the viewer searches for a movie.
• Session management — web applications store user session details and preferences for quick retrieval using the user’s key.

Furthermore, key-value stores can accelerate all kinds of applications including microservices, stream processing, edge/IoT, metadata stores, and graph databases.

Example Key-Value Stores you can use today

Some popular key-value stores include:

• Redis: an external in-memory key-value store.
• Speedb: an embedded KVS, a fork of RocksDB optimized for performance.
• BerkeleyDB: key-value store with ACID: replication and locking, etc.

Summary

A KVS offers speed, flexibility, and is easy to maintain for quick development. Furthermore, a KVS is considered embedded when it’s used as an internal library compared to an external database.

There are plenty of use cases for embedded KVSs, which include: web browsers, movie recommendations, and storage engines.

Because the KVS is so powerful and flexible, you should definitely consider it for your next application.