Popular explanation: introduction to cache, cache algorithm and cache framework

introduction

We've all heard of cache. When you ask them what cache is, they will give you a perfect answer, but they don't know how cache is built, or they don't tell you what criteria should be adopted to select the cache framework. In this article, we will discuss caching, caching algorithms, caching framework and which caching framework is better.

interview

"Cache is a temporary place to store data (frequently used data). Because the cost of getting the original data is too high, I can get it faster."

This is the answer of programmer one (programmer one is an interviewer) in the interview (a month ago, he submitted his resume to the company and wanted to apply for a java development position with rich experience in caching, caching framework and large-scale data operation).

programmer one implements his own cache through the hash table, but all he knows is his cache and his hash table storing 150 records, which is what he thinks of as large-scale data (CACHE = hashtable, just search in the hash table). Therefore, let's take a look at the interview process.

Interviewer: what criteria did you choose for the caching scheme?

programmer one: Well, (thought for 5 minutes) well, based on, based on, based on data (cough...)

Interviewer: excese me! Can you repeat that?

programmer one: data?!

Interviewer: OK. Talk about several cache algorithms and their functions

programmer one: (staring at the interviewer, his face showed a very strange expression. No one knew that humans could make such an expression)

Interviewer: Well, let me put it another way. What will I do when the cache reaches its capacity?

Programmer one: capacity? HMM. (when thinking about... There is no limit to the capacity of hash table. I can add any entries, and it will automatically expand the capacity) (this is the idea of programmer one, but he didn't say it)

The interviewer thanked the programmer one (the interview lasted 10 minutes). Then a woman came up and said: Thank you for your time. We'll call you. I wish you a good mood. This was the worst interview for programmer one (he didn't see the recruitment requirements for candidates with rich experience and background. In fact, he only saw a good reward).

Do what you say

After programmer one left, he wanted to know the interviewer's questions and answers, so he went online to check. Programmer one knew nothing about caching, except that when I needed caching, I would use hash table.

After he used his favorite search engine to search, he found a good article about caching and began to read

Why do we need caching?

A long time ago, when there was no cache... Users often requested an object, and the object was retrieved from the database. Then, the object became larger and larger, and the user's request time each time became longer and longer, which also made the database very painful. He worked all the time. Therefore, this event makes users and databases very angry, and then the following two things may happen:

1. Users are annoyed, complaining and even don't use the application (this happens in most cases)

2. The database is packed home and leaves the application. Then, there is a big trouble (there is no place to store data) (in rare cases)

God sent me

A few years later, researchers at IBM (in the 1960s) introduced a new concept called "cache".

What is caching?

As mentioned at the beginning, cache is "a temporary place to store data (frequently used data). Because the cost of getting the original data is too high, I can get it faster."

The cache can be regarded as a pool of data. These data are copied from the real data in the database and labeled (key ID) in order not to retrieve them. fantastic

programmer one already knows this, but he doesn't know the following cache terms.

Hit:

When the customer initiates a request (we say he wants to view a product information), our application accepts the request, and if it is the first time to check the cache, it needs to read the product information from the database.

If an entry is found in the cache through a tag, the entry will be used. We call it cache hit. Therefore, the hit rate is not difficult to understand.

Cache Miss:

However, two points need to be noted:

1. If there is still cache space, the missed objects will be stored in the cache.

2. If the cache is full and does not hit the cache, the old objects in the cache will be kicked out and the new objects will be added to the cache pool according to a certain strategy. These strategies are collectively referred to as alternative strategies (caching algorithms), which determine which objects should be proposed.

Storage costs:

When there is no hit, we will take the data from the database and put it into the cache. The time and space required to put this data into the cache is the storage cost.

Index cost:

Similar to storage costs.

invalid:

When the data in the cache needs to be updated, it means that the data in the cache is invalid.

Alternative strategy:

When the cache misses and the cache capacity is full, you need to kick out an old entry and add a new entry in the cache. What entry should be kicked out is determined by the substitution strategy.

Optimal alternative strategy:

The best alternative strategy is to kick out the most useless items in the cache, but the future can not be predicted, so this strategy is impossible to achieve. But there are many strategies that are working towards this current situation.

Java Street nightmare:

While programmer one was reading this article, he fell asleep and had a nightmare (everyone has nightmares).

programmer one: nihahha, I'm going to disable you! (crazy state)

Cache object: No, no, let me live. They still need me. I have children.

programmer one: every cached object will say that before it expires. Since when did you have children? Don't worry, disappear forever now!

Hahaha... Programmer one smiled horribly, but the siren broke the silence. The police arrested programmer one and accused him of killing (invalidating) a cache object that still needs to be used. He was taken to prison.

programmer one suddenly woke up. He was frightened and sweating all over. He began to look around and found that it was really a dream. Then he hurried to continue reading this article and tried to eliminate his panic.

After programmer one woke up, he began to read the article again.

Cache algorithm

No one can say which caching algorithm is better than other caching algorithms

Least Frequently Used(LFU):

Hello, I'm LFU. I'll calculate how often they are used for each cache object. I'll kick out the least commonly used cache objects.

Least Recently User(LRU):

I'm an LRU caching algorithm. I kick away the least recently used cache objects.

I always need to know when and which cache object is used. It's very difficult for someone to understand why I always kick out the least recently used objects.

The browser uses me (LRU) as the caching algorithm. New objects will be placed at the top of the cache. When the cache reaches the capacity limit, I will kick away the objects at the bottom. The trick is: I will put the newly accessed cache objects at the top of the cache pool.

Therefore, cache objects that are often read will always stay in the cache pool. There are two ways to implement me, array or linked list.

My speed is very fast, and I can also be adapted by the data access mode. I have a big family. They can improve me and even do better than me (I do envy sometimes, but it doesn't matter). Some members of my family include LRU2 and 2Q. They exist to improve LRU.

Least Recently Used 2(LRU2):

I'm Least Recently Used 2. Someone told me to use twice at least recently. I prefer this name. I will put the objects accessed twice into the cache pool. When the cache pool is full, I will kick away the cache objects that are used least twice. Because the object needs to be tracked twice, the access load will increase with the increase of cache pool. If I use it in a large cache pool, there will be a problem. In addition, I also need to track objects that are not cached because they have not been read for the second time. I'm better than LRU, and I'm in the optional to access mode.

Two Queues(2Q):

I'm Two Queues; I put the accessed data into the LRU cache. If this object is accessed again, I will transfer it to the second and larger LRU cache.

I kick away cache objects to keep the first cache pool 1 / 3 of the second cache pool. When the cache access load is fixed, replacing LRU with LRU2 is better than increasing the cache capacity. This mechanism makes me better than LRU2. I am also a member of the LRU family, and I am in the optional to access mode.

Adaptive Replacement Cache(ARC):

I'm ARC. Some people say I'm between LRU and LFU. In order to improve the effect, I'm composed of two LRUs. The first, L1, contains items that have only been used once recently, while the second LRU, L2, contains items that have been used twice recently. Therefore, L1 places new objects and L2 places common objects. That's why people think I'm between LRU and LFU, but it doesn't matter. I don't mind.

I am considered to be one of the best caching algorithms, self-tuning and low load. I also save historical objects so that I can remember those removed objects. At the same time, I can see whether the removed objects can stay and kick other objects away instead. My memory is poor, but I am fast and applicable.

Most Recently Used(MRU):

I am MRU, which corresponds to LRU. I will remove the most recently used objects, and you will ask me why. Well, let me tell you, when an access comes, some things are unpredictable, and finding the least recently used objects in the cache system is a very time complex operation. That's why I'm the best choice.

I am how common it is in database memory cache! Whenever a cache record is used, I put it at the top of the stack. When the stack is full, guess what? I will replace the object at the top of the stack with a new object!

First in First out(FIFO):

I'm a first in, first out algorithm. I'm a low load algorithm and don't have high requirements for cache object management. I track all cache objects through a queue. The most commonly used cache objects are placed in the back, while the earlier cache objects are placed in the front. When the cache capacity is full, the cache objects in the front will be kicked away, and then the new cache objects will be added. I'm fast, but I don't apply.

Second Chance:

Hello, I'm second chance. I'm modified from FIFO. It's called second chance cache algorithm. What's better than FIFO is that I improve the cost of FIFO. Like FIFO, I also observe the front end of the queue, but it is very different from FIFO immediately. I will check whether the object to be kicked out has a flag previously used (1 a bit). If it has not been used, I will kick it out; Otherwise, I will clear the flag bit and add the cache object to the queue as a new cache object. You can imagine that it's like a ring queue. When I met this object at the head of the team again, because he didn't have this flag, I kicked him away immediately. I am faster than FIFO in speed.

CLock:

I'm Clock. A better FIFO is better than second chance. Because I won't put the marked cache object at the end of the queue like second chance, but I can also achieve the effect of second chance.

I hold a circular list of cached objects, and the header pointer points to the oldest cached object in the list. When a cache miss occurs and there is no new cache space, I will ask the flag bit of the cache object pointed to by the pointer to decide what I should do. If the flag is 0, I will directly replace this cache object with a new cache object; If the flag bit is 1, I will increment the header pointer, and then repeat the process until the new cache object can be placed. I'm faster than second chance.

Simple time-based:

I am a simple time-based caching algorithm. I invalidate those cached objects through absolute time cycles. For new objects, I will save them for a specific time. I'm fast, but I don't apply.

Extended time-based expiration:

I am an extended time-based expiration caching algorithm. I invalidate cached objects through relative time; For new cache objects, I will save them for a specific time, such as every 5 minutes, 12:00 every day.

Sliding time-based expiration:

I'm sliding time-based expiration. The difference is that the life starting point of the cache object I manage is calculated from the last access time of the cache. I'm fast, but I'm not very applicable.

Other caching algorithms also consider the following points:

Cost: if cached objects have different costs, those objects that are difficult to obtain should be saved.

Capacity: if the cache objects have different sizes, those large cache objects should be cleared, so that more small cache objects can come in.

Time: some caches also store the expiration time of the cache. Computers will fail because they have expired.

Depending on the size of the cache object, regardless of other caching algorithms may be necessary.

E-mail!

After reading this article, programmer one thought for a while, and then decided to send an email to the author. He felt where the author's name was, but he couldn't remember it. Anyway, he sent the email, and he asked the author how caching works in a distributed environment.

The author of the article received an email. Ironically, the author is the person who interviewed programmer one. The author replied

In this section, let's see how to implement these famous caching algorithms. The following code is just an example. If you want to implement the caching algorithm yourself, you may have to add some extra work.

LeftOver mechanism

After programmer one read the article, he then read the comments of the article. One of the comments mentioned the leftover mechanism - random cache.

Random Cache

I'm a random cache. I replace cache entities at will. No one dares to complain. You can say that the replaced entity is unlucky. Through these behaviors, I go to cache entities at will. I am better than FIFO mechanism. In some cases, I am even better than LRU, but usually LRU is better than me.

It's comment time

When programmer one continued to read the comment, he found a comment very interesting. This comment implemented some caching algorithms. It should be said that this comment made a link to the reviewer's website. Programmer one followed the link to that website and then read it.

Look at cache elements (CACHE entities)

publicclassCacheElement
{
     privateObjectobjectValue;
     privateObjectobjectKey;
     privateintindex;
     privateinthitCount;// getters and setters
}
//This cache entity has cached key s and value s. The data structure of this entity will be used by all the following cache algorithms.
//Common code for caching algorithms
publicfinalsynchronizedvoidaddElement(Objectkey,Objectvalue)
{
     intindex;
     Objectobj;
// get the entry from the table
     obj = table.get(key);
// If we have the entry already in our table
// then get it and replace only its value.
     obj = table.get(key);
if(obj != null)
     {
         CacheElement element;
         element = (CacheElement)obj;
         element.setObjectValue(value);
         element.setObjectKey(key);
return;
     }
}

The above code will be used by all cache algorithm implementations. This code is used to check whether the cache element is in the cache. If so, we will replace it. However, if we can't find the cache corresponding to this key, what will we do? Let's take a closer look at what will happen!

Site visit

Today's topic is very special because we have special guests. In fact, they are the participants we want to listen to. But first, let's introduce our guests: Random Cache and FIFO Cache. Let's start with Random Cache.

Look at the implementation of random caching

publicfinalsynchronizedvoidaddElement(Objectkey,Objectvalue)
{
     intindex;
     Objectobj;
     obj = table.get(key);
if(obj != null)
     {
         CacheElement element;// Just replace the value.
         element = (CacheElement)obj;
         element.setObjectValue(value);
         element.setObjectKey(key);
return;
     }// If we haven't filled the cache yet, put it at the end.
if(!isFull())
     {
         index = numEntries;
         ++numEntries;
     }
else{// Otherwise, replace a random entry.
         index = (int)(cache.length * random.nextFloat());
         table.remove(cache[index].getObjectKey());
     }
     cache[index].setObjectValue(value);
     cache[index].setObjectKey(key);
     table.put(key,cache[index]);
}
//Look at the implementation of FIFO buffer algorithm
publicfinalsynchronizedvoidaddElement(Objectkey,Objectvalue)
{
     intindex;
     Objectobj;
     obj = table.get(key);
if(obj != null)
     {
         CacheElement element;// Just replace the value.
         element = (CacheElement)obj;
         element.setObjectValue(value);
         element.setObjectKey(key);
return;
     }
// If we haven't filled the cache yet, put it at the end.
if(!isFull())
     {
         index = numEntries;
         ++numEntries;
     }
else{// Otherwise, replace the current pointer,
// entry with the new one.
         index = current;
// in order to make Circular FIFO
if(++current >= cache.length)
             current = 0;
         table.remove(cache[index].getObjectKey());
     }
     cache[index].setObjectValue(value);
     cache[index].setObjectKey(key);
     table.put(key,cache[index]);
}

Look at the implementation of LFU caching algorithm

publicsynchronizedObjectgetElement(Objectkey)
{
     Objectobj;
     obj = table.get(key);
if(obj != null)
     {
         CacheElement element = (CacheElement)obj;
         element.setHitCount(element.getHitCount() + 1);
         returnelement.getObjectValue();
     }
     returnnull;
}
publicfinalsynchronizedvoidaddElement(Objectkey,Objectvalue)
{
     Objectobj;
     obj = table.get(key);
if(obj != null)
     {
         CacheElement element;// Just replace the value.
         element = (CacheElement)obj;
         element.setObjectValue(value);
         element.setObjectKey(key);
return;
     }
if(!isFull())
     {
         index = numEntries;
         ++numEntries;
     }
else
     {
         CacheElement element = removeLfuElement();
         index = element.getIndex();
         table.remove(element.getObjectKey());
     }
     cache[index].setObjectValue(value);
     cache[index].setObjectKey(key);
     cache[index].setIndex(index);
     table.put(key,cache[index]);
}
publicCacheElement removeLfuElement()
{
     CacheElement[]elements = getElementsFromTable();
     CacheElement leastElement = leastHit(elements);
     returnleastElement;
}
publicstaticCacheElement leastHit(CacheElement[]elements)
{
     CacheElement lowestElement = null;
for(inti = 0;i < elements.length;i++)
     {
         CacheElement element = elements[i];
if(lowestElement == null)
         {
             lowestElement = element;
         }
else{
if(element.getHitCount() < lowestElement.getHitCount())
             {
                 lowestElement = element;
             }
         }
     }
     returnlowestElement;
}

Today's topic is very special because we have special guests. In fact, they are the participants we want to listen to. But first, let's introduce our guests: Random Cache and FIFO cache. Let's start with Random Cache.

The most important code should be the leastHit method. This code is to

Find the element with the lowest hitCount, delete it, and leave a location for the new cache element.

Look at the implementation of LRU cache algorithm

privatevoidmoveToFront(intindex)
{
     intnextIndex,prevIndex;
if(head != index)
     {
         nextIndex = next[index];
         prevIndex = prev[index];
// Only the head has a prev entry that is an invalid index
// so we don't check.
         next[prevIndex] = nextIndex;
// Make sure index is valid. If it isn't, we're at the tail
// and don't set prev[next].
if(nextIndex >= 0)
             prev[nextIndex] = prevIndex;
else
             tail = prevIndex;
         prev[index] = -1;
         next[index] = head;
         prev[head] = index;
         head = index;
     }
}
publicfinalsynchronizedvoidaddElement(Objectkey,Objectvalue)
{
     intindex;Objectobj;
     obj = table.get(key);
if(obj != null)
     {
         CacheElement entry;
// Just replace the value, but move it to the front.
         entry = (CacheElement)obj;
         entry.setObjectValue(value);
         entry.setObjectKey(key);
         moveToFront(entry.getIndex());
return;
     }
// If we haven't filled the cache yet, place in next available
// spot and move to front.
if(!isFull())
     {
if(_numEntries > 0)
         {
             prev[_numEntries] = tail;
             next[_numEntries] = -1;
             moveToFront(numEntries);
         }
         ++numEntries;
     }
else{// We replace the tail of the list.
         table.remove(cache[tail].getObjectKey());
         moveToFront(tail);
     }
     cache[head].setObjectValue(value);
     cache[head].setObjectKey(key);
     table.put(key,cache[head]);
}

The logic of this code is the same as the description of LRU algorithm. The cache used again is extracted to the front, and the last element is deleted every time.

conclusion

We have seen the implementation of LFU cache algorithm and LRU cache algorithm. As for how to implement it, it is up to you to decide whether to use array or LinkedHashMap. If it is not enough, I usually use array for small cache capacity and LinkedHashMap for large cache capacity.

Posted on Sun, 05 Dec 2021 08:14:42 -0500 by jaco