Loguru: a more elegant and concise Python log management module



When it comes to logging in Python development, we may usually think of the built-in standard library - logging. Although the logging library adopts a modular design and can be combined with different handler s, it is cumbersome in configuration. At the same time, in the scenario of multithreading or multi process, if no special processing is carried out, the logging will be abnormal.

This article will introduce a very elegant and concise logging third-party library - loguru. We can call it directly by importing the instance of its encapsulated logger class.

install

Use pip to install. The Python 3 version is installed as follows:

pip3 install loguru

Basic use

We directly import the instantiated object of the logger class encapsulated by loguru. Instead of manually creating a logger, we directly call different levels of log output methods. Let's start with an example:

from loguru import logger

logger.debug('This is debug information')
logger.info('This is info information')
logger.warning('This is warn information')
logger.error('This is error information')

When the IDE or terminal is running, it will be found that loguru outputs different levels of information with different colors, making the results more intuitive, including time, level, module name, line number and log information.


The corresponding relationship between different log levels and logging methods in loguru is as follows:

loguru configuration log file

By default, logger uses sys.stderr standard error output to output the log to the console. If you want to output the log to other locations at the same time, such as log files, we only need to use one line of code.

For example, output the log information to the 2021-3-28.log file, which can be written as follows:

from loguru import logger

logger.add("E:/PythonCode/MOC/log_2021-3-28.log",rotation="500MB", encoding="utf-8", enqueue=True, retention="10 days")

logger.info('This is info information')

As described above, loguru completes the configuration of log files directly through the add() method.

String formatting of log content

When outputting logs, loguru also provides a very flexible string format function for outputting logs, as follows:

import platform
from loguru import logger

rounded_value = round(0.345, 2)

trace= logger.add('2021-3-28.log')

logger.info('If you are using Python {version}, prefer {feature} of course!', version=platform.python_version(), feature='f-strings')

# Execute the above code and the output result is
2021-03-28 13:43:26.232 | INFO     | __main__:<module>:9 - If you are using Python 3.7.6, prefer f-strings of course!

Configuration and analysis of loguru log common parameters

sink: you can pass in a file like object, a str string, a pathlib.Path object, a method (coroutine function), or a handler of the logging module (logging.Handler).

level (int or str, optional): the lowest severity level at which the logged message should be sent to the receiver.

format (str or callable, optional): the format module uses a template to format the recorded message before sending it to the receiver.

Filter (scalable, STR or dict, optional): used to determine whether each recorded message should be sent to the receiver.

colorize (bool, optional) – whether the color mark contained in the formatted message should be converted into Ansi code for terminal coloring, or stripped in other ways. If None, the selection is made automatically according to whether the sink is TTY.

serialize (bool, optional): whether the recorded message and its record should be converted into JSON string before being sent to the receiver.

backtrace (bool, optional): whether the formatted exception trace should extend upward beyond the capture point to display the full stack trace that generated the error.

diagnose (bool, optional): whether the exception trace should display variable values to simplify debugging. In production, this should be set to "False" to avoid leaking sensitive data.

enqueue (bool, optional): whether the message to be recorded should first pass through the multi process security queue before reaching the receiver. This is useful when logging to a file through multiple processes. This also has the advantage of making log calls non blocking.

catch (bool, optional): whether to automatically catch the errors that occur when the receiver processes log messages. If True, the exception message sys.stderr is displayed on the. However, exceptions are not propagated to the caller, preventing the application from crashing.

If the sink (sink) is a file path (pathlib.Path), the following parameters can be applied, and add() returns the identifier associated with the added sink:

rotation: separate log files, conditions for when to close the current log file and start a new file,; For example, "500 MB", "0.5 GB", "1 month 2 weeks", "10h", "monthly", "18:00", "sunday", "monday at 18:00", "06:15"

retention (str, int, datetime.timedelta or callable, optional) to configure the maximum retention time of old logs, for example, "1 week, 3 days" and "2 months"

Compression (STR or scalable, optional): the compression or archive format that the log file should be converted to when closed, for example, "gz", "bz2", "xz", "lzma", "tar", "tar.gz", "tar.bz2", "tar.xz" and "zip"

delay (bool, optional): whether the file should be created immediately after the receiver is configured, or delayed until the first recorded message. The default is' False '.

mode (str, optional): the same open mode as the built-in open() function. The default is' a '(open the file in attach mode).

buffering (int, optional): the buffering strategy of the built-in open() function, which defaults to 1 (line buffer file).

encoding (str, optional): the file encoding is the same as the built-in 'open()' function. If 'None', it defaults to 'locale.getpreferredencoding().

loguru log common methods

Stop logging to file

After adding sink to the add method, we can also delete it. When deleting, we can delete it according to the id just returned by the add method and restore it to the standard output. As follows:

from loguru import logger

trace= logger.add('2021-3-28.log')
logger.error('This is error information')

logger.remove(trace)
logger.warning('This is warn information')

The console output is as follows:

2021-03-28 13:38:22.995 | ERROR    | __main__:<module>:7 - This is error information

2021-03-28 13:38:22.996 | WARNING  | __main__:<module>:11 - This is warn information

The contents of the log file 2021-3-28.log are as follows:

2021-03-28 13:38:22.995 | ERROR    | __main__:<module>:7 - This is error information

After the sink object is removed, the contents after that are no longer output to the log file.

Output to text only, not in console

Delete the previously added handler through logger.remove(handler_id=None) and stop sending logs to its receiver. Then add the output log file through add to output only the text, not the console, as follows:

from loguru import logger
# Clear previous settings
logger.remove(handler_id=None) 

trace= logger.add('2021-3-28.log')

logger.error('This is error information')
logger.warning('This is warn information')

filter configure log filtering rules

As follows, we implement the custom method error_only, judge the log level. When the log level is ERROR, it returns TRUE. When we set the filter parameter in the add method, it is set to error_only to filter out all logs except ERROR.

from loguru import logger

def error_only(record):
    """
    error Log judgment 
    Args:
        record: 

    Returns: If the log level is ERROR, output TRUE

    """
    return record["level"].name == "ERROR"

# Logs other than ERROR are filtered out
logger.add("2021-3-28.log", filter=error_only)

logger.error('This is error information')
logger.warning('This is warn information')

In the 2021-3-28.log, we can see that only the ERROR level log is recorded.

2021-03-28 17:01:33.267 | ERROR    | __main__:<module>:11 - This is error information

Format configuration logging format template

from loguru import logger

def format_log():
    """

    Returns:

    """
    trace = logger.add('2021-3-28.log', format="{time:YYYY-MM-DD HH:mm:ss} {level} From {module}.{function} : {message}")

    logger.warning('This is warn information')

if __name__ == '__main__':
    format_log()

As follows, we can see that in the 2021-3-28.log file, such as "{time: yyyy-mm-dd HH: mm: SS} {level} from {module}. {function}: {message}" format template, records are made:

# 2021-3-28.log
2021-03-28 14:46:25 WARNING From 2021-3-28.format_log : This is warn information

Other formatting template properties are as follows:


Add additional attributes through extra bind() to provide more attribute information for structured logs, as follows:

from loguru import logger

def format_log():
    """

    Returns:

    """
    trace = logger.add('2021-3-28.log', format="{time:YYYY-MM-DD HH:mm:ss} {extra[ip]}  {extra[username]} {level} From {module}.{function} : {message}")
    
    extra_logger = logger.bind(ip="192.168.0.1", username="Zhang San")
    extra_logger.info('This is info information')
    extra_logger.bind(username="Li Si").error("This is error information")

    extra_logger.warning('This is warn information')

if __name__ == '__main__':
    format_log()

As follows, we can see that in the 2021-3-28.log file, the logs are recorded according to the above template, as follows:

2021-03-28 16:27:11 192.168.0.1  Zhang San INFO From 2021-3-28.format_log : This is info information
2021-03-28 16:27:11 192.168.0.1  Li Si ERROR From 2021-3-28.format_log : This is error information
2021-03-28 16:27:11 192.168.0.1  Zhang San WARNING From 2021-3-28.format_log : This is warn information

Level configures the minimum log level of the log

from loguru import logger

trace = logger.add('2021-3-29.log', level='ERROR')

rotation configures the mechanism of log rolling recording

We want to create log files periodically or automatically separate log files according to the file size. We can directly use the rotation parameter of the add method for configuration.

For example, create a log file every 200MB to avoid each log file being too large, as follows:

from loguru import logger

trace = logger.add('2021-3-28.log', rotation="200 MB")

For example, create a log file at 6:00 every day, as follows:

from loguru import logger

trace = logger.add('2021-3-28.log', rotation='06:00')

For example, create a log file every 2 weeks, as follows:

from loguru import logger

trace = logger.add('2021-3-28.log', rotation='2 week')

Retention configure log retention mechanism

Usually, some remote log files need to be cleared periodically to avoid log accumulation and waste storage space. We can configure the maximum retention time of the log through the retention parameter of the add method.

For example, set the maximum retention time of log files to 7 days, as follows:

from loguru import logger

trace = logger.add('2021-3-28.log', retention='7 days')

Compression configure log compression format

loguru can also configure the compression format of the file, such as saving in zip file format. The example is as follows:

from loguru import logger

trace = logger.add('2021-3-28.log', compression='zip')

serialize log serialization

If we want to output a structured log similar to the json line format, we can use the serialize parameter to write the serialized json format of the log information into the log file. Finally, we can import the log file similar to MongoDB and ElasticSearch for subsequent log analysis. The code example is as follows:

from loguru import logger
import platform

rounded_value = round(0.345, 2)

trace= logger.add('2021-3-28.log', serialize=True)

logger.info('If you are using Python {version}, prefer {feature} of course!', version=platform.python_version(), feature='f-strings')

In the 2021-3-28.log file, we can see that each log information is serialized and stored in the log file, as follows:

{
    "text": "2021-03-28 13:44:17.104 | INFO     | __main__:<module>:9 - If you are using Python 3.7.6, prefer f-strings of course!\n",
    "record": {
        "elapsed": {
            "repr": "0:00:00.010911",
            "seconds": 0.010911
        },
        "exception": null,
        "extra": {
            "version": "3.7.6",
            "feature": "f-strings"
        },
        "file": {
            "name": "2021-3-28.py",
            "path": "F:/code/MOC/2021-3-28.py"
        },
        "function": "<module>",
        "level": {
            "icon": "\u2139\ufe0f",
            "name": "INFO",
            "no": 20
        },
        "line": 9,
        "message": "If you are using Python 3.7.6, prefer f-strings of course!",
        "module": "2021-3-28",
        "name": "__main__",
        "process": {
            "id": 22604,
            "name": "MainProcess"
        },
        "thread": {
            "id": 25696,
            "name": "MainThread"
        },
        "time": {
            "repr": "2021-03-28 13:44:17.104522+08:00",
            "timestamp": 1616910257.104522
        }
    }
}

Traceback record (exception tracing)

loguru integrates a called better_ The exceptions library can not only record exceptions and errors, but also trace exceptions. As follows, we delete list elements during traversing the list to trigger IndexError exceptions,

Exception capture is realized through the catch decorator. The code example is as follows:

from loguru import logger

trace= logger.add('2021-3-28.log')

@logger.catch
def index_error(custom_list: list):

    for index in range(len(custom_list)):
        index_value = custom_list[index]
        if custom_list[index] < 2 :
            custom_list.remove(index_value)

        print(index_value)

if __name__ == '__main__':
    index_error([1,2,3])

Running the above code, we can find the Traceback log information output by loguru. The current variable values are also output in the Traceback log information, as follows:

In the 2021-3-28.log file, the exception tracing information in the above format is also output, as follows.

2021-03-28 13:57:13.852 | ERROR    | __main__:<module>:16 - An error has been caught in function '<module>', process 'MainProcess' (7080), thread 'MainThread' (32280):
Traceback (most recent call last):

> File "F:/code/MOC/2021-3-28.py", line 16, in <module>
    index_error([1,2,3])
    └ <function index_error at 0x000001FEB84D0EE8>

  File "F:/code/MOC/2021-3-28.py", line 9, in index_error
    index_value = custom_list[index]
                  │           └ 2
                  └ [2, 3]

IndexError: list index out of range

At the same time, the code examples of class methods and static methods in the class are attached for reference

from loguru import logger

trace = logger.add('2021-3-28.log')

class Demo:
    @logger.catch
    def index_error(self, custom_list: list):
        for index in range(len(custom_list)):
            index_value = custom_list[index]
            if custom_list[index] < 2:
                custom_list.remove(index_value)

    @staticmethod
    @logger.catch
    def index_error_static(custom_list: list):
        for index in range(len(custom_list)):
            index_value = custom_list[index]
            if custom_list[index] < 2:
                custom_list.remove(index_value)


if __name__ == '__main__':
    # Demo().index_error([1, 2, 3])
    Demo.index_error_static([1, 2, 3])

You can also capture and record exceptions through the logger.exception method:

from loguru import logger

trace = logger.add('2021-3-28.log')

def index_error(custom_list: list):
    for index in range(len(custom_list)):
        try:
            index_value = custom_list[index]
        except IndexError as  err:
            logger.exception(err)
            break

        if custom_list[index] < 2:
            custom_list.remove(index_value)


if __name__ == '__main__':
    index_error([1, 2, 3])

Running the above code, we can find the Traceback log information output by loguru. The current variable values are also output in the Traceback log information, as follows:

The following is the supporting information. For the friends doing [software testing], it should be the most comprehensive and complete war preparation warehouse. This warehouse has also accompanied me through the most difficult journey. I hope it can also help you!

Finally, it can be in the official account: the sad spicy bar! Get a 216 page interview document of Software Test Engineer for free. And the corresponding video learning tutorials for free!, It includes basic knowledge, Linux essentials, Shell, Internet program principles, Mysql database, special topics of packet capture tools, interface test tools, test advanced Python programming, Web automation test, APP automation test, interface automation test, advanced continuous test integration, test architecture, development test framework, performance test, security test, etc.

Don't fight alone in learning. It's best to stay together and grow together. The effect of mass effect is very powerful. If we study together and punch in together, we will have more motivation to learn and stick to it. You can join our testing technology exchange group: 914172719 (there are various software testing resources and technical discussions)

Friends who like software testing, if my blog is helpful to you and if you like my blog content, please click "like", "comment" and "collect" for three times!

Haowen recommendation

Job transfer interview, job hopping interview, these interview skills that software testers must know!

Interview experience: move bricks in first tier cities! Another software testing post, 5000 will be satisfied

Interviewer: I've worked for three years and have a preliminary test? I'm afraid your title of software test engineer should be enclosed in double quotation marks

What kind of person is suitable for software testing?

The man who left work on time was promoted before me

The test post changed jobs repeatedly and disappeared

Tags: Python Back-end Programmer software testing IT

Posted on Thu, 11 Nov 2021 19:43:56 -0500 by magicmoose