Preface
Why logs are needed
- Debugging Development
- Program Run Log
- User Behavior Log
Different purposes determine the format and frequency of log output. As a developer, the goal of printing logs during debugging development is to output as much information as possible (such as context, variable values,...) to aid in development testing, so the log format should be readable and printed more frequently. While the program runs, the log format tends to be structured (easy to analyze and search) and print less frequently for performance and focus on critical information, such as error s.
Go Standard Library Log
Use
We often use Go log as a set of three functions:
- Print/Printf/Println: Print log information
- Panic/Panicf/Panicln: After printing the log information, call Panic with the assembled string as the parameter
- Fatal/Fatalf/Fatalln: os.Exit(1) exits the program after printing log information
Formatted output with f suffix and line breaks with ln suffix are added, but a line break is automatically added in the print log scenario, where the ln suffix is not very different. The difference between Panic and Fatal is that Panic can be captured.
Examples are as follows:
package main import "log" func main() { log.Println("Log Information 1") log.Print("Log Information 2") log.Panicln("Log Information 3") log.Fatalln("Log Information 4") // Unable to run }
2021/11/09 15:41:34 Log Information 1 2021/11/09 15:41:34 Log Information 2 2021/11/09 15:41:34 Log Information 3 panic: Log Information 3 goroutine 1 [running]: log.Panicln({0xc0000bdf60, 0x0, 0x1015ba5}) /usr/local/opt/go/libexec/src/log/log.go:368 +0x65 main.main() /Users/bytedance/go/src/github.com/he2121/log_test/main.go:8 +0xa8
You can see that the default log format is time + msg. Why is this so? Can I change it?
Principles and customizations
func Println(v ...interface{}) { std.Output(2, fmt.Sprintln(v...)) }
You can see that the previous methods all call the Output method in the std object.
var std = New(os.Stderr, "", LstdFlags) func New(out io.Writer, prefix string, flag int) *Logger { return &Logger{out: out, prefix: prefix, flag: flag} } type Logger struct { mu sync.Mutex // ensures atomic writes; protects the following fields prefix string // prefix on each line to identify the logger (but see Lmsgprefix) flag int // properties out io.Writer // destination for output buf []byte // for accumulating text to write }
std is constructed using log.New with three parameters
- io.Writer: Any structure that implements the Writer interface, the log will be written here, and the output location of std is os.Stderr
- Prefix: Custom prefix string, std passed in empty string
- flag: Some flags, prefix customized, std is LstdFlags, from which time comes in the default log format
const ( Ldate = 1 << iota // the date in the local time zone: 2009/01/23 Ltime // the time in the local time zone: 01:23:23 Lmicroseconds // microsecond resolution: 01:23:23.123123. assumes Ltime. Llongfile // full file name and line number: /a/b/c/d.go:23 Lshortfile // final file name element and line number: d.go:23. overrides Llongfile LUTC // if Ldate or Ltime is set, use UTC rather than the local time zone Lmsgprefix // move the "prefix" from the beginning of the line to before the message LstdFlags = Ldate | Ltime // initial values for the standard logger )
Log output location, custom prefix string, flag can all be customized
Example (stdlogger modified directly here, should own a New logger in the actual project):
func main() { var buf bytes.Buffer log.SetOutput(&buf) log.SetPrefix("[info: ]") log.SetFlags(log.LstdFlags | log.Lshortfile) log.Println("Custom Log Format") //fmt.Print(&buf) }
The example above places the log output in bytes.Buffer and can be changed to file, stdout, network...
Prefixed with [info:] string
Lshortfile is added to flag: the name and location of the file where the log statement will be printed.
Executing the above statement, the console found no output.
Because the log output location is no longer stderr.Remove the comment on the last line, print buff, output as:
[info: ]2021/11/09 16:49:27 main.go:14: Custom Log Format
Advantage
- go built-in, easy to use
shortcoming
- There are several common log levels missing, Info/debug/error...
- Missing log structured output capabilities
- performance
Go3rd party log Library
logrus
logrus : Logrus is a structured logger for Go (golang), which is fully compatible with the standard library log API. (19.1 k stars)
Why logrus ?
- Compare full log levels
PanicLevel: Logging, panic FatalLevel: Logging, program exit ErrorLevel: Error Level Log WarnLevel: Warning Level Log InfoLevel: Key Information Level Logs DebugLevel: Debug Level TraceLevel: Tracking Level
-
Support structured logging
-
Usability
- hook
- WithField
- ...
Simple use
package main import "github.com/sirupsen/logrus" func main() { logrus.SetLevel(logrus.TraceLevel) // Set a low level in the test environment. //logrus.SetLevel(logrus.InfoLevel) // Performance needs to be considered in production environments, key information needs to be considered, level set a little higher //logrus.SetReportCaller(true) // Caller file name and location //logrus.SetFormatter(new(logrus.JSONFormatter)) // Log format set to json logrus.Traceln("trace Journal") logrus.Debugln("debug Journal") logrus.Infoln("Info Journal") logrus.Warnln("warn Journal") logrus.Errorln("error msg") logrus.Fatalf("fatal Journal") logrus.Panicln("panic Journal") }
For ease of use, libraries generally create an object with default values, and the outermost method of a package is generally to manipulate the default object, as is the case with logrus and logs.
Before structuring:
INFO[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:12 main.main() Info Journal WARN[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:13 main.main() warn Journal ERRO[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:14 main.main() error msg FATA[0000]/Users/bytedance/go/src/github.com/he2121/log_test/main.go:15 main.main() fatal Journal
After structuring:
{"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:12","func":"main.main","level":"info","msg":"Info Journal","time":"2021-11-09T17:37:27+08:00"} {"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:13","func":"main.main","level":"warning","msg":"warn Journal","time":"2021-11-09T17:37:27+08:00"} {"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:14","func":"main.main","level":"error","msg":"error msg","time":"2021-11-09T17:37:27+08:00"} {"file":"/Users/bytedance/go/src/github.com/he2121/log_test/main.go:15","func":"main.main","level":"fatal","msg":"fatal Journal","time":"2021-11-09T17:37:27+08:00"}
zap
zap : a fast, structured, hierarchical logger (14k stars).
why zap?
-
Compare full log levels
-
Support structured logging
-
performance
Simple use
Zap provides two types of loggers - Sugared Logger and Logger
Sugared Logger emphasizes performance and ease of use and supports structured and printf-style logging.
Logger places a strong emphasis on performance and does not provide a printf-style api (which reduces interface {} and reflective performance loss), as in the following examples:
package main import "go.uber.org/zap" func main() { // sugared sugar := zap.NewExample().Sugar() sugar.Infof("hello! name:%s,age:%d", "xiaomin", 20) // Pritf style, ease of use // logger logger := zap.NewExample() logger.Info("hello!", zap.String("name", "xiaomin"), zap.Int("age", 20)) // Emphasize performance }
// output {"level":"info","msg":"hello! name:xiaomin,age:20"} {"level":"info","msg":"hello!","name":"xiaomin","age":20}
zap has three default configurations that create a logger, example, development, production, example:
package main import "go.uber.org/zap" func main() { // example logger := zap.NewExample() logger.Info("example") // Development logger,_ = zap.NewDevelopment() logger.Info("Development") // Production logger,_ = zap.NewProduction() logger.Info("Production") }
// output {"level":"info","msg":"example"} 2021-11-09T21:02:07.181+0800 INFO log_test/main.go:11 Development {"level":"info","ts":1636462927.182271,"caller":"log_test/main.go:14","msg":"Production"}
You can see that there are differences in log level, log output format, and default fields.
You can also customize the logger, such as the following example
package main import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" "os" ) func main() { encoder := getEncoder() sync := getWriteSync() core := zapcore.NewCore(encoder, sync, zapcore.InfoLevel) logger := zap.New(core) logger.Info("info Journal",zap.Int("line", 1)) logger.Error("info Journal", zap.Int("line", 2)) } // Responsible for formatting encoding logs func getEncoder() zapcore.Encoder { return zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()) } // Location responsible for log writing func getWriteSync() zapcore.WriteSyncer { file, _ := os.OpenFile("./log.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, os.ModePerm) syncFile := zapcore.AddSync(file) syncConsole := zapcore.AddSync(os.Stderr) return zapcore.NewMultiWriteSyncer(syncConsole, syncFile) } // output // Create log.txt, append log // console print log //{level":"info","ts":1636471657.16419,"msg":"info log","line":1} //{level":"error","ts":1636471657.1643898,"msg":"info log","line":2}
Starting with New (core zapcore.Core, options... Option) *Logger, zapcore.Core needs to be constructed
-
Through the NewCore(enc Encoder, ws WriteSyncer, enab LevelEnabler) Core method, three more parameters are passed in
-
Encoder: Responsible for setting the log format of encoding, you can set the JSON or text structure, or you can customize the key value in json, time format...
-
ws WriteSyncer: Location responsible for log writing. The example above is written to both file and console, where the network can also be written.
-
LevelEnabler: Set logging level
-
-
options is an interface that implements the apply(*Logger) method and extends many functions
- Override core Configuration
- Add hook
- Increase key value information
- Eror log output separately
summary
This paper first introduces the use and customization of the Go Standard Library log, summarizes the advantages and disadvantages of the log, and introduces the topic of third-party libraries logrus and zap. Both libraries are excellent and powerful, and are highly recommended for use.