golang common libraries: logging library logrus

Introducing logrus

It is a structured and plug-in logging library. Fully compatible with the log module in the golang standard library. It also has built-in two log output formats, JSONFormatter and TextFormatter, to define the output log format.

github address: https://github.com/sirupsen/logrus

logrus usage

Version used: logrus v1.8.1

1. Start using

package main

import (
	log "github.com/sirupsen/logrus"
)

func main() {
	log.WithFields(log.Fields{
		"animal": "walrus",
	}).Info("a walrus appears")
}

Run output:

time="2021-11-11T17:41:48+08:00" level=info msg="a walrus appears" animal=walrus

2. Set log format, log level and output mode

Format log

1) Built in log format

log Formatter , there are two types of built-in formatter in logrus, logrus.TextFormatter and logrus.JSONFormatter

  • logrus.JSONFormatter {}, set to json format. All setting options are in logrus.JSONFormatter

    log.SetFormatter(&log.JSONFormatter{
        TimestampFormat: "2006-01-02 15:04:05", // Set the date output format in json
    })
    
    log.SetFormatter(&log.JSONFormatter{}) // Set to json format
    
  • logrus.TextFormatter {}, set to text format. All setting options are in logrus.TextFormatter

    log.SetFormatter(&log.TextFormatter{
        TimestampFormat: "2006-01-02 15:04:05",
        ForceColors:  true,
        EnvironmentOverrideColors: true,
        // FullTimestamp:true,
        // DisableLevelTruncation:true,
    })
    

2) Custom log format

Can be based on Formatter The interface defines the log Format. There is a Format method in which there is a struct type data *Entry , Entry.Data is a collection of all fields, Fields The type is map[string]interface {}.

For example: entry.Data["msg"], entry.Data["time"]`. The timestamp

package main

import (
	"fmt"

	jsoniter "github.com/json-iterator/go"
	log "github.com/sirupsen/logrus"
)

type MyJSONFormatter struct {
	JSONPrefix string
	Otherdata  string
}

func (my *MyJSONFormatter) Format(entry *log.Entry) ([]byte, error) {
	// fmt.Println(entry.Data["msg"])

	entry.Data["msg"] = fmt.Sprintf("%s%s", my.JSONPrefix, my.Otherdata)
	json, err := jsoniter.Marshal(&entry.Data)
	if err != nil {
		return nil, fmt.Errorf("failed to marshal fields to JSON , %w", err)
	}
	return append(json, '\n'), nil

}

func main() {
	formatter := &MyJSONFormatter{
		JSONPrefix: "jsonprefix-",
		Otherdata:  ":otherdata:",
	}

	log.SetFormatter(formatter)
	log.Info("this is customered formatter")
}

3) Setting the log format with a third-party custom formatter

wait

Set log level

logrus logs have seven levels, from high to low: panic, fatal, error, warn, info, debug, trace

  • log.SetLevel(log.WarnLevel) / / set the output warning level

Set log output mode

  • log.SetOutput(os.Stdout) / / input to stdout and output to Stderr by default
  • logfile, _ := os.OpenFile("./logrus.log", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0644)
    logrus.SetOutput(logfile) / / output to a file

example:

package main

import (
	log "github.com/sirupsen/logrus"
    "os"
)

func init() {
	log.SetFormatter(&log.JSONFormatter{}) // Set format json
	log.SetLevel(log.WarnLevel) // Set output warning level
    // Output to stdout instead of the default stderr
    log.SetOutput(os.Stdout)
}

func main() {
	log.WithFields(log.Fields{
		"animal": "dog",
		"size":   10,
	}).Info("a group of dog emerges from the zoon")

	log.WithFields(log.Fields{
		"omg":    true,
		"number": 12,
	}).Warn("the group's number increased")

	log.WithFields(log.Fields{
		"omg":    true,
		"number": 100,
	}).Fatal("th ice breaks")

    // the logrus.Entry returned from WithFields()
	contextLogger := log.WithFields(log.Fields{
		"common": "this is a common filed",
		"other":  "i also should be logged always",
	})
	// Common field output
	contextLogger.Info("I'll be logged with common and other field")
	contextLogger.Info("Me too")
}

Run output:

{"level":"warning","msg":"the group's number increased","number":12,"omg":true,"time":"2021-11-11T18:00:55+08:00"}
{"level":"fatal","msg":"th ice breaks","number":100,"omg":true,"time":"2021-11-11T18:00:55+08:00"}

From the output results, we can see that the log information at Info level is not output.

Mask code for setting log level

func init() {
	log.SetFormatter(&log.JSONFormatter{}) // Set format json
	// log.SetLevel(log.WarnLevel) / / set the output warning level
}

Output during operation:

{"animal":"dog","level":"info","msg":"a group of dog emerges from the zoon","size":10,"time":"2021-11-11T18:26:45+08:00"}
{"level":"warning","msg":"the group's number increased","number":12,"omg":true,"time":"2021-11-11T18:26:45+08:00"}
{"level":"fatal","msg":"th ice breaks","number":100,"omg":true,"time":"2021-11-11T18:26:45+08:00"}
exit status 1

From the output log information, the log info information of contextLogger is not output. The log information is not output. Why is the log not output?

Mask the above Fatal output log:

// log.WithFields(log.Fields{
	// 	"omg":    true,
	// 	"number": 100,
	// }).Fatal("th ice breaks")

Output during operation:

{"animal":"dog","level":"info","msg":"a group of dog emerges from the zoon","size":10,"time":"2021-11-11T18:28:56+08:00"}
{"level":"warning","msg":"the group's number increased","number":12,"omg":true,"time":"2021-11-11T18:28:56+08:00"}
{"common":"this is a common filed","level":"info","msg":"I'll be logged with common and other field","other":"i also should be logged always","time":"2021-11-11T18:28:56+08:00"}
{"common":"this is a common filed","level":"info","msg":"Me too","other":"i also should be logged always","time":"2021-11-11T18:28:56+08:00"}

At this time, the contextLogger log information can be output.

3. Fatal processing of logrus

The above example defines that after the Fatal log is output, the subsequent logs cannot be output. Why? There is a message exit status 1 after the log,

Because os.Exit(1) will be executed after the Fatal output of logrus. What if there are some necessary programs to deal with behind the program?

logrus provides RegisterExitHandler Method to handle some problems in case of fatal exception.

package main

import (
	"fmt"
	log "github.com/sirupsen/logrus"
)

func main() {
	log.SetFormatter(&log.TextFormatter{
		TimestampFormat: "2006-01-02 15:04:05",
	})

	log.RegisterExitHandler(func() {
		fmt.Println("It happened fatal Exception, perform some necessary processing work")
	})

	log.Warn("warn")
	log.Fatal("fatal")
	log.Info("info") //Will not execute
}

Run output:

time="2021-11-11 21:48:25" level=warning msg=warn
time="2021-11-11 21:48:25" level=fatal msg=fatal
 It happened fatal Exception, perform some necessary processing work
exit status 1

4. Split log files

If the log file is too large, you want to cut it into small files, but logrus does not provide this function.

One is to use the logrotate command of linux system to segment the log files generated by logrus.

The other is to use the hook function of logrus as a plug-in for log segmentation. eureka file-rotatelogs , but the library status

It is already in the archived state. The library author does not accept any modifications and will not continue to maintain it. So use it carefully.

stay logrus issue Found this in the https://github.com/natefinch/lumberjack Library of cutting files.

example:

package main

import (
	log "github.com/sirupsen/logrus"

	"gopkg.in/natefinch/lumberjack.v2"
)

func main() {
	logger := &lumberjack.Logger{
		Filename:   "./testlogrus.log",
		MaxSize:    500,  // Log file size in MB
		MaxBackups: 3,    // Maximum number of expired logs retained
		MaxAge:     28,   // Maximum time to retain expired files, in days
		Compress:   true, // Whether to compress the log. The default is not to compress. Set it to true here to compress the log
	}

	log.SetOutput(logger) // logrus sets how logs are output

}

5. Set logrus instance

If an application uses logs in multiple places, a logrus can be instantiated separately as a global log instance.

package main

import (
	"os"

	"github.com/sirupsen/logrus"
)

var log = logrus.New()

func main() {
	log.Out = os.Stdout // Set the output log location. You can set the log to file

	log.WithFields(logrus.Fields{
		"fruit": "apple",
		"size":  20,
	}).Info(" a lot of apples on the tree")
}

Output:

time="2021-11-11T18:39:15+08:00" level=info msg=" a lot of apples on the tree" fruit=apple size=20

6. fields

When using logrus, log.WithFields(log.Fields{}).Fatal() is encouraged to replace og. Fatalf ("failed to send event% s to topic% s with key% d"), that is, instead of formatting with% s,% D, the variable event is directly passed in and topic is given to log.Fields, which makes structured log output very user-friendly and beautiful.

log.WithFields(log.Fields{
  "event": event,
  "topic": topic,
  "key": key,
}).Fatal("Failed to send event")

7. Set default fields

For example, in link tracking, there will be an rquest_id ,trace_id, etc. I think the log always has these two fields. How to set logrus?

You can use log.WithFields(log.Fields{"request_id": request_id, "trace_id": trace_id})

requestLogger := log.WithFields(log.Fields{"request_id": request_id, "trace_id": trace_id})
requestLogger.Info("something happened on that request")
requestLogger.Warn("something not great happened")

example:

package main

import (
	"github.com/google/uuid"
	log "github.com/sirupsen/logrus"
)

func main() {
	uid := uuid.New()
	request_id := uid
	trace_id := uid
	requestLogger := log.WithFields(log.Fields{"request_id": request_id, "trace_id": trace_id})
	requestLogger.Info("something happened on that request")
	requestLogger.Warn("something not great happened")
}

8. hook hook - extend logrus function

hook provides logrus with powerful and extensible functions

Users can write hook plug-ins for logrus and write hooks according to their log requirements.

logrus also has some built-in plug-ins hooks.

A hook written by a third party to logrus, Third party hook list.

Official syslog hook example:

package main

import (
	"log/syslog"

	"github.com/sirupsen/logrus"
	lSyslog "github.com/sirupsen/logrus/hooks/syslog"
)

func main() {
	log := logrus.New()
	hook, err := lSyslog.NewSyslogHook("", "", syslog.LOG_INFO, "")
	if err != nil {
		log.Hooks.Add(hook)
	}
}

reference resources

Tags: Go

Posted on Thu, 11 Nov 2021 19:47:06 -0500 by Concat