FlashDB embedded database

What is FlashDB

Let's take a look at the information on the official website first
FlashDB is an ultra lightweight embedded database, which focuses on providing data storage solutions for embedded products. Different from the traditional database based on file system, FlashDB combines the characteristics of Flash and has strong performance and reliability. And extend the service life of Flash as much as possible on the premise of ensuring very low resource occupation.

FlashDB provides two database modes:

Key value database: a non relational database that stores data as a set of key value pairs, where the key is used as a unique identifier. KVDB has simple operation and strong scalability.
Time Series Database: Time Series Database (TSDB) stores data in chronological order. TSDB data has time stamp, large data storage capacity and high insertion and query performance.
Usage scenario
Nowadays, there are more and more types of Internet of things products, and the types and total amount of data generated during operation are also increasing. FlashDB provides a variety of data storage schemes, which not only occupies small resources, but also has large storage capacity. It is very suitable for Internet of things products. The following are the main application scenarios:

Key database:
Product parameter storage
User configuration information storage
Small file management
Time series database:
Store dynamically generated structured data, such as environmental monitoring information collected by temperature and humidity sensors, human health information recorded in real time by intelligent bracelets, etc
Record operation log: store the operation log of product history, abnormal alarm record, etc
Main characteristics
The resource occupation is very low, and the memory occupation is almost 0;
Support multiple partitions and instances. When the amount of data is large, the partition can be refined to reduce the retrieval time;
Support wear balance and prolong Flash life;
Support power failure protection function with high reliability;
Support string and blob types to facilitate user operation;
KV incremental upgrade is supported. After product firmware upgrade, KVDB content also supports automatic upgrade;
Support modifying the status of each TSDB record to facilitate user management;

See if there is a general understanding here. I have also checked some other embedded databases, but they are basically used on Linux. There are still certain requirements for resources. It is unrealistic to use them at the level of single chip microcomputer.

Open source links from top and bottom authors: https://gitee.com/Armink/FlashDB.
Worship the great God of armink.
Personal understanding: in fact, the database is basically add, delete, change and query. There are also rich API interfaces, such as traversing the entire database and retrieving the database by timestamp; As well as modifying and deleting the database, it also increases the balance and smoothing of sectors. Basically, all functions are included. When I was working on a project, I also wanted to access it, but I still didn't solve the functions such as fast retrieval. Later, I plan to look at the bottom layer and write the processing logic of this piece.

transplant

  • The same first source code

  • There are multiple projects under the demos directory
  • Personal use is SPI flash, so choose stm32f405rg_spi_flash this project
  • After adding the source code to the project

It's like this when it's finished

  • Then modify the configuration file fdb_cfg.h file
#ifndef _FAL_CFG_H_
#define _FAL_CFG_H_

#define FAL_DEBUG 1 		// Start printing
#define FAL_PART_HAS_TABLE_CFG 	// Startup equipment list
#define FAL_USING_SFUD_PORT 	// Using sfud universal serial flash

/* ===================== Flash device Configuration ========================= */
extern struct fal_flash_dev nor_flash0;

/* flash device table */
#define FAL_FLASH_DEV_TABLE                                          \
{                                                                    \
    &nor_flash0,                                                     \
}

/* ====================== Partition Configuration ========================== */
#ifdef FAL_PART_HAS_TABLE_CFG
/* partition table */
#define FAL_PART_TABLE                                                                 \
{                                                                                      \
    {FAL_PART_MAGIC_WORD,  "fdb_tsdb1",       "norflash0",           0, 1024*1024, 0}, \
    {FAL_PART_MAGIC_WORD,  "fdb_kvdb1",       "norflash0",   1024*1024, 1024*1024, 0}, \
}
#endif /* FAL_PART_HAS_TABLE_CFG */

#endif /* _FAL_CFG_H_ */

Run it and check the log


The middle is too long. Omit it

It can be found that flash will be initialized for the first run. Some marks should be written in the sector header
You can also find that the last two pieces of data are written to the database

Then reset once

This time, the log is much simpler. Without the initialization process, the first time is really slow. I almost thought it was in a dead cycle

[FlashDB][sample][kvdb][basic] ==================== kvdb_basic_sample ====================
[FlashDB][sample][kvdb][basic] get the 'boot_count' value is 1
[FlashDB][sample][kvdb][basic] set the 'boot_count' value to 2

The number of starts is recorded here. It has been started twice in total

[FlashDB][sample][tsdb] ==================== tsdb_sample ====================
[FlashDB][sample][tsdb] append the new status.temp (36) and status.humi (85)
[FlashDB][sample][tsdb] append the new status.temp (38) and status.humi (90)
[FlashDB][sample][tsdb] [query_cb] queried a TSL: time: 1, temp: 36, humi: 85
[FlashDB][sample][tsdb] [query_cb] queried a TSL: time: 2, temp: 38, humi: 90
[FlashDB][sample][tsdb] [query_cb] queried a TSL: time: 3, temp: 36, humi: 85
[FlashDB][sample][tsdb] [query_cb] queried a TSL: time: 4, temp: 38, humi: 90
[FlashDB][sample][tsdb] [query_by_time_cb] queried a TSL: time: 1, temp: 36, humi: 85
[FlashDB][sample][tsdb] [query_by_time_cb] queried a TSL: time: 2, temp: 38, humi: 90
[FlashDB][sample][tsdb] [query_by_time_cb] queried a TSL: time: 3, temp: 36, humi: 85
[FlashDB][sample][tsdb] [query_by_time_cb] queried a TSL: time: 4, temp: 38, humi: 90
[FlashDB][sample][tsdb] query count is: 2
[FlashDB][sample][tsdb] set the TSL (time 1) status from 3 to 3
[FlashDB][sample][tsdb] set the TSL (time 2) status from 3 to 3
[FlashDB][sample][tsdb] set the TSL (time 3) status from 2 to 3
[FlashDB][sample][tsdb] set the TSL (time 4) status from 2 to 3
[FlashDB][sample][tsdb] ===========================================================

A total of 4 pieces of data are written, written twice, and printed after traversal.

summary

The migration is over, and I sorted it out with the log. API functions are relatively complete and easy to use. After reading the author's introduction, this library has a history of several years and has been updated in five major versions. I'm going to take a look at the bottom, have an in-depth understanding, and then share with you.

Tags: Database Big Data

Posted on Mon, 13 Sep 2021 16:03:22 -0400 by dajawu