Preface

Why logs are needed

• Debugging Development
• Program Run Log
• User Behavior Log

Different purposes determine the format and frequency of log output. As a developer, the goal of printing logs during debugging development is to output as much information as possible (such as context, variable values,...) to aid in development testing, so the log format should be readable and printed more frequently. While the program runs, the log format tends to be structured (easy to analyze and search) and print less frequently for performance and focus on critical information, such as error s.

Go Standard Library Log

Use

We often use Go log as a set of three functions:

• Print/Printf/Println: Print log information
• Panic/Panicf/Panicln: After printing the log information, call Panic with the assembled string as the parameter
• Fatal/Fatalf/Fatalln: os.Exit(1) exits the program after printing log information

Formatted output with f suffix and line breaks with ln suffix are added, but a line break is automatically added in the print log scenario, where the ln suffix is not very different. The difference between Panic and Fatal is that Panic can be captured.

Examples are as follows:

package main

import "log"

func main() {
	log.Println("Log Information 1")
	log.Print("Log Information 2")
	log.Panicln("Log Information 3")
	log.Fatalln("Log Information 4") // Unable to run
}

2021/11/09 15:41:34 Log Information 1
2021/11/09 15:41:34 Log Information 2
2021/11/09 15:41:34 Log Information 3
panic: Log Information 3
goroutine 1 [running]:
log.Panicln({0xc0000bdf60, 0x0, 0x1015ba5})
        /usr/local/opt/go/libexec/src/log/log.go:368 +0x65
main.main()
        /Users/bytedance/go/src/github.com/he2121/log_test/main.go:8 +0xa8

You can see that the default log format is time + msg. Why is this so? Can I change it?

Principles and customizations

func Println(v ...interface{}) {
   std.Output(2, fmt.Sprintln(v...))
}

You can see that the previous methods all call the Output method in the std object.

var std = New(os.Stderr, "", LstdFlags)

func New(out io.Writer, prefix string, flag int) *Logger {
	return &Logger{out: out, prefix: prefix, flag: flag}
}

type Logger struct {
	mu     sync.Mutex // ensures atomic writes; protects the following fields
	prefix string     // prefix on each line to identify the logger (but see Lmsgprefix)
	flag   int        // properties
	out    io.Writer  // destination for output
	buf    []byte     // for accumulating text to write
}

std is constructed using log.New with three parameters

• io.Writer: Any structure that implements the Writer interface, the log will be written here, and the output location of std is os.Stderr
• Prefix: Custom prefix string, std passed in empty string
• flag: Some flags, prefix customized, std is LstdFlags, from which time comes in the default log format

const (
	Ldate         = 1 << iota     // the date in the local time zone: 2009/01/23
	Ltime                         // the time in the local time zone: 01:23:23
	Lmicroseconds                 // microsecond resolution: 01:23:23.123123.  assumes Ltime.
	Llongfile                     // full file name and line number: /a/b/c/d.go:23
	Lshortfile                    // final file name element and line number: d.go:23. overrides Llongfile
	LUTC                          // if Ldate or Ltime is set, use UTC rather than the local time zone
	Lmsgprefix                    // move the "prefix" from the beginning of the line to before the message
	LstdFlags     = Ldate | Ltime // initial values for the standard logger
)

Log output location, custom prefix string, flag can all be customized

Example (stdlogger modified directly here, should own a New logger in the actual project):

func main() {
   var buf bytes.Buffer
   log.SetOutput(&buf)
   log.SetPrefix("[info: ]")
   log.SetFlags(log.LstdFlags | log.Lshortfile)
   log.Println("Custom Log Format")
   //fmt.Print(&buf)
}

The example above places the log output in bytes.Buffer and can be changed to file, stdout, network...

Prefixed with [info:] string

Lshortfile is added to flag: the name and location of the file where the log statement will be printed.

Executing the above statement, the console found no output.

Because the log output location is no longer stderr.Remove the comment on the last line, print buff, output as:

[info: ]2021/11/09 16:49:27 main.go:14: Custom Log Format

Advantage

• go built-in, easy to use

shortcoming

• There are several common log levels missing, Info/debug/error...
• Missing log structured output capabilities
• performance

Go3rd party log Library

logrus

logrus : Logrus is a structured logger for Go (golang), which is fully compatible with the standard library log API. (19.1 k stars)

Why logrus ?

1. Compare full log levels

PanicLevel: Logging, panic
FatalLevel: Logging, program exit
ErrorLevel: Error Level Log
WarnLevel:  Warning Level Log
InfoLevel:  Key Information Level Logs
DebugLevel: Debug Level
TraceLevel: Tracking Level

1. Support structured logging

2. Usability

• hook
• WithField
• ...

Simple use

package main

import "github.com/sirupsen/logrus"

func main() {
	logrus.SetLevel(logrus.TraceLevel)	// Set a low level in the test environment.
	//logrus.SetLevel(logrus.InfoLevel) 	//  Performance needs to be considered in production environments, key information needs to be considered, level set a little higher
	//logrus.SetReportCaller(true) 			//  Caller file name and location
	//logrus.SetFormatter(new(logrus.JSONFormatter)) 	//  Log format set to json
	logrus.Traceln("trace Journal")
	logrus.Debugln("debug Journal")
	logrus.Infoln("Info Journal")
	logrus.Warnln("warn Journal")
	logrus.Errorln("error msg")
	logrus.Fatalf("fatal Journal")
	logrus.Panicln("panic Journal")
}

For ease of use, libraries generally create an object with default values, and the outermost method of a package is generally to manipulate the default object, as is the case with logrus and logs.

Before structuring:

INFO[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:12 main.main() Info Journal                                      
WARN[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:13 main.main() warn Journal                                      
ERRO[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:14 main.main() error msg                                    
FATA[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:15 main.main() fatal Journal

After structuring:

{"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:12","func":"main.main","level":"info","msg":"Info Journal","time":"2021-11-09T17:37:27+08:00"}
{"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:13","func":"main.main","level":"warning","msg":"warn Journal","time":"2021-11-09T17:37:27+08:00"}
{"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:14","func":"main.main","level":"error","msg":"error msg","time":"2021-11-09T17:37:27+08:00"}
{"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:15","func":"main.main","level":"fatal","msg":"fatal Journal","time":"2021-11-09T17:37:27+08:00"}

zap

zap : a fast, structured, hierarchical logger (14k stars).

why zap?

1. Compare full log levels

2. Support structured logging

3. performance
   

Simple use

Zap provides two types of loggers - Sugared Logger and Logger

Sugared Logger emphasizes performance and ease of use and supports structured and printf-style logging.

Logger places a strong emphasis on performance and does not provide a printf-style api (which reduces interface {} and reflective performance loss), as in the following examples:

package main

import "go.uber.org/zap"

func main() {
   // sugared
   sugar := zap.NewExample().Sugar()
   sugar.Infof("hello! name:%s,age:%d", "xiaomin", 20)	// Pritf style, ease of use
   // logger
   logger := zap.NewExample()
   logger.Info("hello!", zap.String("name", "xiaomin"), zap.Int("age", 20)) // Emphasize performance
}

// output 
{"level":"info","msg":"hello! name:xiaomin,age:20"}
{"level":"info","msg":"hello!","name":"xiaomin","age":20}

zap has three default configurations that create a logger, example, development, production, example:

package main

import "go.uber.org/zap"

func main() {
   // example
   logger := zap.NewExample()
   logger.Info("example")
   // Development
   logger,_ = zap.NewDevelopment()
   logger.Info("Development")
   // Production
   logger,_ = zap.NewProduction()
   logger.Info("Production")
}

// output
{"level":"info","msg":"example"}
2021-11-09T21:02:07.181+0800    INFO    log_test/main.go:11     Development	
{"level":"info","ts":1636462927.182271,"caller":"log_test/main.go:14","msg":"Production"}

You can see that there are differences in log level, log output format, and default fields.

You can also customize the logger, such as the following example

package main

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"os"
)

func main() {
	encoder := getEncoder()
	sync := getWriteSync()
	core := zapcore.NewCore(encoder, sync, zapcore.InfoLevel)
	logger := zap.New(core)
	
	logger.Info("info Journal",zap.Int("line", 1))
	logger.Error("info Journal", zap.Int("line", 2))
}

// Responsible for formatting encoding logs
func getEncoder() zapcore.Encoder {
	return zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig())
}

// Location responsible for log writing
func getWriteSync() zapcore.WriteSyncer {
	file, _ := os.OpenFile("./log.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, os.ModePerm)
	syncFile := zapcore.AddSync(file)
	syncConsole := zapcore.AddSync(os.Stderr)
	return zapcore.NewMultiWriteSyncer(syncConsole, syncFile)
}
// output
// Create log.txt, append log
// console print log
//{level":"info","ts":1636471657.16419,"msg":"info log","line":1}
//{level":"error","ts":1636471657.1643898,"msg":"info log","line":2}

Starting with New (core zapcore.Core, options... Option) *Logger, zapcore.Core needs to be constructed

1. Through the NewCore(enc Encoder, ws WriteSyncer, enab LevelEnabler) Core method, three more parameters are passed in

   • Encoder: Responsible for setting the log format of encoding, you can set the JSON or text structure, or you can customize the key value in json, time format...

   • ws WriteSyncer: Location responsible for log writing. The example above is written to both file and console, where the network can also be written.

   • LevelEnabler: Set logging level

2. options is an interface that implements the apply(*Logger) method and extends many functions

   • Override core Configuration
   • Add hook
   • Increase key value information
   • Eror log output separately

summary

This paper first introduces the use and customization of the Go Standard Library log, summarizes the advantages and disadvantages of the log, and introduces the topic of third-party libraries logrus and zap. Both libraries are excellent and powerful, and are highly recommended for use.