Netdata Embedded Configuration
background
Netdata can indeed be called a very excellent project. At present, there are not many monitoring systems that can operate stably on embedded devices, and even fewer that really have strong commercial value. For example, Baota Qinglong also provides functions similar to netdata, but it is a little Pediatrics if it is really used as a monitoring means for operation and maintenance personnel, More like personal amateur development and use.
Netdata can be called the most perfect embedded monitoring scheme at present. Even so, it still has many defects in embedded devices, most of which come from the embedded devices themselves; Although netdata can run on embedded devices, it is more aimed at servers or even server clusters; In other words, netdata targets more server operation and maintenance personnel than embedded operation and maintenance personnel.
Although the requirements for embedded applications are not as harsh as the server, they are also much more complex than the client program. An excellent embedded monitoring program should run stably for 7 * 24 hours and reduce the system overhead (CPU, disk, network and memory) as much as possible. To do this, personalized customization of Netdata is an indispensable process.
This section will first explain the configuration rules of Netdata configuration file, then give a configuration practice, and finally give the effect diagram; One of the principles of embedded devices is to save as much as possible without unintentional overhead, but it can not lack necessary functions. In short, it is more fast and better to save, so how to configure Netdata needs to be realistic, and there is no absolute answer.
Netdata configuration rules
Netdata is a highly configurable monitoring system. Most of its configuration files exist in / etc/netdata, and most of the configuration files begin with *. conf. it is worth mentioning that netdata follows the principle that the Convention is better than the configuration. Therefore, if there is no configuration file, it will start the default configuration at compile time.
The main configurations of Netdata are apps+groups.conf, charts.d.conf and health_alarm_notify.conf and netdata.conf, of which the key is netdata.conf, which is the main configuration file of Netdata.
Netdata.conf is the main configuration file of netdata. By modifying it, nearly 90% of netdata can be personalized. Therefore, understanding how to configure netdata.conf is a very important and necessary skill. If you access netdata configuration through IP:Port/netdata.conf, you will find that there are many configuration items, and everyone is different, There are more than 4000 lines here, as shown in the following figure:
Modifying the configuration of Netdata depends on skills, not mussels; The configuration of Netdata adopts ini format, and its configuration principle is similar to the form of multi-level domain name. For example:
[plugins] -> [plugin:proc] -> [plugin:proc:diskspace] -> [plugin:proc:diskspace:/mnt/disk/disk1]
This is the multi-level domain name system. Blocking anywhere on the link will cut off the whole link. For example, if I disable [plugins], then [plugin:proc] is useless, so [plugin:proc:diskspace] is also invalid, let alone [plugin:proc:diskspace:/mnt/disk/disk1].
Generally speaking, you don't need to set it to such a deep level. Generally, it's like the above example. It's often enough to set it to [plugin:proc:diskspace].
netdata main configuration modification
For embedded devices, some main modifications are given here, that is, the first several levels of Netdata configuration:
# netdata configuration # global netdata configuration [global] run as user = root hostname = unknown update every = 10 config directory = /var/netdata/etc/netdata stock config directory = /var/netdata/lib/netdata/conf.d log directory = /var/netdata/var/log/netdata web files directory = /var/netdata/share/netdata/web cache directory = /var/netdata/var/cache/netdata lib directory = /var/netdata/var/lib/netdata home directory = /var/netdata/var/lib/netdata lock directory = /var/netdata/var/lib/netdata/lock plugins directory = "/var/netdata/libexec/netdata/plugins.d" "/var/netdata/etc/netdata/custom-plugins.d" memory mode = dbengine page cache size = 4 dbengine disk space = 128 timezone = Asia/Shanghai OOM score = 0 process scheduling policy = nice pthread stack size = 1048576 [web] default port = 25000 enable gzip compression = no [plugins] PATH environment variable = /usr/bin:/usr/sbin:/bin:/sbin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin proc = yes diskspace = yes timex = no cgroups = no tc = no idlejitter = no enable running new plugins = yes check for new plugins every = 60 slabinfo = no apps = yes fping = no charts.d = yes ioping = no perf = no node.d = no python.d = no [health] enabled = no [backend] enabled = no [statsd] enable = no [plugin:proc] netdata server resources = no /proc/pagetypeinfo = no /proc/stat = yes /proc/uptime = yes /proc/loadavg = yes /proc/sys/kernel/random/entropy_avail = yes /proc/pressure = yes /proc/interrupts = yes /proc/softirqs = yes /proc/vmstat = yes /proc/meminfo = yes /sys/kernel/mm/ksm = yes /sys/block/zram = yes /sys/devices/system/edac/mc = yes /sys/devices/system/node = yes /proc/net/dev = yes /proc/net/wireless = no /proc/net/sockstat = no /proc/net/sockstat6 = no /proc/net/netstat = yes /proc/net/snmp = no /proc/net/snmp6 = no /proc/net/sctp/snmp = no /proc/net/softnet_stat = no /proc/net/ip_vs/stats = no /sys/class/infiniband = no /proc/net/stat/conntrack = no /proc/net/stat/synproxy = no /proc/diskstats = yes /proc/mdstat = no /proc/net/rpc/nfsd = no /proc/net/rpc/nfs = no /proc/spl/kstat/zfs/arcstats = no /proc/spl/kstat/zfs/pool/state = no /sys/fs/btrfs = no ipc = yes /sys/class/power_supply = no [plugin:apps] command options = without-users without-groups
There are several important core modifications:
- update every: set to 10, i.e. collect data every 10 seconds. Reduce netdata CPU utilization and disk storage by reducing the data collection frequency of netdata; Reducing disk storage can greatly reduce the IO operation, memory read-write frequency and bandwidth overhead caused by netdata querying previous data.
- dbengine disk space: set the disk storage space to 128MB to prevent it from occupying too much storage space. Under this configuration, this size can store more than a week of data.
- OOM score: set it to 0 to prevent the automatic destruction of netdata due to the shortage of system resources.
- process scheduling policy: modify the thread scheduling policy so that netdata can better collect data and prevent data from not being collected when the system resource is tight because the scheduling policy is idel (after all, this is an important time for monitoring).
- pthread stack size: modify the stack space of the thread to 1MB.
- Compression: enable gzip compression and netdata.compression_ratio is all disabled to no, that is, the stored data is not compressed, which reduces the burden of CPU during compression and decompression.
- plugin:apps: application programs disable grouping. Generally, there is only one root user on the embedded system. You don't need to know the information of users and groups, just one person.
- plugin:proc: tailor some collection indicators to reduce the amount of data collection.
Through the above modifications, netdata will be more suitable for the operation of embedded devices.
The complete configuration file is given below:
# netdata configuration # global netdata configuration [global] run as user = root hostname = unknown update every = 10 config directory = /var/netdata/etc/netdata stock config directory = /var/netdata/lib/netdata/conf.d log directory = /var/netdata/var/log/netdata web files directory = /var/netdata/share/netdata/web cache directory = /var/netdata/var/cache/netdata lib directory = /var/netdata/var/lib/netdata home directory = /var/netdata/var/lib/netdata lock directory = /var/netdata/var/lib/netdata/lock plugins directory = "/var/netdata/libexec/netdata/plugins.d" "/var/netdata/etc/netdata/custom-plugins.d" memory mode = dbengine page cache size = 8 dbengine disk space = 128 timezone = Asia/Shanghai OOM score = 0 process scheduling policy = nice pthread stack size = 1048576 [web] default port = 25000 enable gzip compression = no [plugins] PATH environment variable = /usr/bin:/usr/sbin:/bin:/sbin:/sbin:/usr /sbin:/usr/local/bin:/usr/local/sbin proc = yes diskspace = yes timex = no cgroups = no tc = no idlejitter = no enable running new plugins = yes check for new plugins every = 60 slabinfo = no apps = yes fping = no charts.d = yes ioping = no perf = no node.d = no python.d = no [health] enabled = no [backend] enabled = no [statsd] enable = no [netdata.compression_ratio] enabled = no [plugin:proc] netdata server resources = no /proc/pagetypeinfo = no /proc/stat = yes /proc/uptime = yes /proc/loadavg = yes /proc/sys/kernel/random/entropy_avail = yes /proc/pressure = yes /proc/interrupts = yes /proc/softirqs = yes /proc/vmstat = yes /proc/meminfo = yes /sys/kernel/mm/ksm = yes /sys/block/zram = yes /sys/devices/system/edac/mc = yes /sys/devices/system/node = yes /proc/net/dev = yes /proc/net/wireless = no /proc/net/sockstat = no /proc/net/sockstat6 = no /proc/net/netstat = yes /proc/net/snmp = no /proc/net/snmp6 = no /proc/net/sctp/snmp = no /proc/net/softnet_stat = no /proc/net/ip_vs/stats = no /sys/class/infiniband = no /proc/net/stat/conntrack = no /proc/net/stat/synproxy = no /proc/diskstats = yes /proc/mdstat = no /proc/net/rpc/nfsd = no /proc/net/rpc/nfs = no /proc/spl/kstat/zfs/arcstats = no /proc/spl/kstat/zfs/pool/state = no /sys/fs/btrfs = no ipc = yes /sys/class/power_supply = no [plugin:apps] command options = without-users without-groups [plugin:proc:/proc/diskstats:mtdblock0] enable = no [plugin:proc:/proc/diskstats:mtdblock2] enable = no [plugin:proc:/proc/diskstats:mtdblock3] enable = no [plugin:proc:/proc/diskstats:mtdblock4] enable = no [plugin:proc:/proc/diskstats:mtdblock5] enable = no [plugin:proc:/proc/diskstats:mtdblock1] enable = no [plugin:proc:/proc/net/dev:ip6gre0] enabled = no [plugin:proc:/proc/net/dev:sit0] enabled = no [plugin:proc:/proc/net/dev:lo] enabled = no [plugin:proc:/proc/net/dev:ip6tnl0] enabled = no [plugin:proc:/proc/net/netstat] bandwidth = auto input errors = auto multicast bandwidth = auto broadcast bandwidth = auto multicast packets = auto broadcast packets = auto ECN packets = no TCP reorders = no TCP SYN cookies = no TCP out-of-order queue = auto TCP connection aborts = auto TCP memory pressures = auto TCP SYN queue = no TCP accept queue = no filename to monitor = /proc/net/netstat
Netdata modification rendering
The running status of netdata after modification is given in the order of modification.
The storage space is calculated in one week, i.e. 7 days.
When reaching the final rendering, its occupation of the system is basically as follows:
The occupation of the system before modification is basically as follows: