Catalogue of series articles

preface

Many storage systems (such as MySQL) provide slow query logs to help developers and operation and maintenance personnel locate the slow operation of the system. The so-called slow query log is that the system calculates the execution time of each command before and after the command is executed. When it exceeds the preset threshold, it records the relevant information of the command (such as occurrence time, time consumption and command details). Redis also provides similar functions.

1, Slow query log

Slow query only records command execution time, not including command queuing and network transmission time. Therefore, the time for the client to execute the command will be greater than the actual execution time of the command. Because of the command execution queuing mechanism, slow query will lead to cascading blocking of other commands. Therefore, when the client requests timeout, it is necessary to check whether there is a corresponding slow query at this time point, so as to analyze whether it is the command cascading blocking caused by slow query.

2, Configuration parameters of slow query

Redis provides slowlog log lower than and slowlog Max len configurations to solve these two problems.

1. It can be seen from the literal meaning that slowlog log slow than is the preset threshold. Its unit is microseconds (1 second = 1000 milliseconds = 1000000 microseconds). The default value is 10000. If a "very slow" command (such as keys *) is executed, if its execution time exceeds 10000 microseconds, it will be recorded in the slow query log. (Note: if slowlog log - Log - lower than = 0, all commands will be recorded. If slowlog - Log - lower than < 0, no commands will be recorded.)
2. Redis uses a list to store slow query logs. Slowlog Max len is the maximum length of the list. A new command is inserted into the list when it meets the slow query conditions. When the slow query log list is at its maximum length, the first command inserted will be removed from the list. For example, slowlog Max len is set to 5. When the 6th slow query is inserted, the first data in the queue head will be listed and the 6th slow query will be listed.

3, How to configure slow query log

3.1 configure it directly in the redis.conf configuration file

################################## SLOW LOG ###################################

# The Redis Slow Log is a system to log queries that exceeded a specified
# execution time. The execution time does not include the I/O operations
# like talking with the client, sending the reply and so forth,
# but just the time needed to actually execute the command (this is the only
# stage of command execution where the thread is blocked and can not serve
# other requests in the meantime).
#
# You can configure the slow log with two parameters: one tells Redis
# what is the execution time, in microseconds, to exceed in order for the
# command to get logged, and the other parameter is the length of the
# slow log. When a new command is logged the oldest one is removed from the
# queue of logged commands.

# The following time is expressed in microseconds, so 1000000 is equivalent
# to one second. Note that a negative number disables the slow log, while
# a value of zero forces the logging of every command.
slowlog-log-slower-than 10000

# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 128

3.2 use the command config set for dynamic modification

config set slowlog-log-slower-than 20000 
config set slowlog-max-len 1000 
config rewrite

Tip: config rewrite is used to persist the configuration to the local configuration file.

4, Slow query log query

4.1 get slow query log

You can query slow query logs through the command slowlog get [n] (the number of logs can be specified in parameter n)

127.0.0.1:6379> slowlog get 
1) 1) (integer) 666 
   2) (integer) 1456786500 
   3) (integer) 11615 
   4)1) "BGREWRITEAOF"
2) 
  1) (integer) 665 
  2) (integer) 1456718400 
  3) (integer) 12006 
  4) 1) "SETEX" 
     2) "video_info_200" 
     3) "300" 
     4) "2"

You can see that each slow query log consists of four attributes, namely, the identification id, occurrence timestamp, command time, execution command and parameters of the slow query log.

4.2 get the current length of the slow query log list

The current length of the slow query log list can be obtained through the command slowlog len

127.0.0.1:6379> slowlog len 
(integer) 45

4.3 slow query log reset

The slow query log can be reset through the command slowlog reset. In fact, the log list is cleaned up

127.0.0.1:6379> slowlog len 
(integer) 45
127.0.0.1:6379> slowlog reset 
OK
127.0.0.1:6379> slowlog len 
(integer) 0

summary

Development suggestions:

1. Slowlog Max len configuration suggestion: it is recommended to increase the slow query list online. When recording slow queries, Redis will truncate long commands and will not occupy a lot of memory. Increasing the slow query list can slow down the possibility that slow queries can be eliminated. For example, online queries can be set to more than 1000.
2. Suggestions for slowlog log slow than configuration: if the default value exceeds 10ms, it is judged as a slow query, which needs to be adjusted according to the Redis concurrency. Because Redis adopts single thread to respond to commands, for high traffic scenarios, if the command execution time is more than 1 ms, Redis can support less than 1000 OPS at most. Therefore, Redis for high OPS scenarios is recommended to be set to 1 ms.
3. ·Because the slow query log is a first in first out queue, that is, if there are many slow queries, some slow query commands may be lost. To prevent this, you can periodically execute the slow get command to persist the slow query log to other storage (such as MySQL), and then create a visual interface for query,