This paper mainly records the use of ELK to complete the business system log collection, mainly using log4net for data entry and docking.
Related connections:
ELK docker img: https://hub.docker.com/r/sebp/elk
Documentation: https://elk-docker.readthedocs.io/#usage
Docker version installation: docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it -d --restart=always --name elk sebp/elk
Error message encountered:
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Repair method:
Permanent solution: in / etc/sysctl.conf Add a line at the end of the file: vm.max_map_count=262144 restart virtual machine
Interim resolution: sysctl -w vm.max_map_count=262144
The visualization port of Kinaba is: 5601
Two nuget packages, Log4net and log4stash, need to be introduced into the project.
For log4stash source code and configuration documents, please refer to the following address: https://github.com/urielha/log4stash
log4 net.config configuration file
<?xml version="1.0" encoding="utf-8" ?> <configuration> <configSections> <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/> </configSections> <log4net> <appender name="ElasticSearchAppender" type="log4stash.ElasticSearchAppender, log4stash"> <Server>localhost</Server> <Port>9200</Port> <IndexName>log_test_%{+yyyy-MM-dd-mm}</IndexName> <IndexType>LogEvent</IndexType> <Bulksize>2000</Bulksize> <BulkIdleTimeout>10000</BulkIdleTimeout> <IndexAsync>True</IndexAsync> </appender> <logger name="log"> <!--(high) OFF > FATAL > ERROR > WARN > INFO > DEBUG > ALL (low) --> <level value="INFO" /> <appender-ref ref="ElasticSearchAppender"/> </logger> </log4net> </configuration>
Here is how I use it in my ABP framework:
docker_ log4 net.config This configuration file is used to replace the dynamic variable values of the real ELK. We will replace these values in the code. After docker deployment, you only need to change the configuration of ELK. log4 net.config Is the replaced value of the file content. log4 Source.config This file is the configuration of writing local files using log4net.
In the framework of ABP, log4net is used by default, so you only need to introduce log4stash.
stay Startup.cs Under the file, add the following changes under the ConfigureServices method.
This code implements the logic: judge whether to use the configuration elkServer environment variable. If not, use the original method directly. If it exists, it will read the data of the template, then replace the variable and write it into the configuration file.
//Replace environment variable value var isElkServer = !string.IsNullOrEmpty(_appConfiguration["ELKServer"]); if (isElkServer) { using var reader = new StreamReader("docker_log4net.config"); var LogConfig = reader.ReadToEnd().Replace("@IndexName", "node_process_data_" + _hostingEnvironment.EnvironmentName). Replace("@Server", _appConfiguration["ELKServer"].Split(":")[0]). Replace("@Port", _appConfiguration["ELKServer"].Split(":")[1]); reader.Dispose(); using var writer = new StreamWriter("log4net.config"); writer.Write(LogConfig); writer.Flush(); } var logConfig = isElkServer ? "log4net.config" : "log4Source.config"; //Configure Abp and Dependency Injection return services.AddAbp<NodeProcessServiceWebModule>(options => { options.IocManager.IocContainer.AddFacility<LoggingFacility>( f => f.LogUsing(new Log4NetLoggerFactory(logConfig, reloadOnChange: true)) ); });
This makes it easy to write to elk's platform.
Add custom properties:
log4net.LogicalThreadContext.Properties["Camera_Code"] = folderName; log4net.LogicalThreadContext.Properties["Object_Name"] = name; _logger.Info(message); log4net.LogicalThreadContext.Properties.Clear();
ELK displays on the Dashboard according to the log analysis: