Network traffic analysis tool - visualization Network - netflow [6] - production network traffic monitoring architecture design

Powerful tool for network traffic analysis - visualization Network - netflow [1] - Basic Principles
Network traffic analysis tool - visualization Network - netflow [2] - Introduction and configuration of Cisco NetFlow working principle
Network traffic analysis tool - visualization Network - netflow [3] - differences between version 5 and version 9 of netflow
Network traffic analysis tool - visualization Network - netflow [4] - Introduction to nfdump receiver
Powerful tool of network traffic analysis visual network data collector fprobe under netflow [5] - linux
Network traffic analysis tool - visualization Network - netflow [6] - production network traffic monitoring architecture design
fprobe parameter - e
fprobe parameter - n -k

Topological graph

Monitoring point selection

The selection of monitoring point mainly depends on the data flow to be monitored. If the monitoring point is placed on the public network interface, the captured addresses are all public network addresses. If the monitoring point is deployed on the load balancing internal network interface, the captured data can be divided into public network address and internal network address.
Select the public network port. At this time, it should be noted that if you want to use fprobe for 1:1 sampling, you need to mirror the traffic to a linux physical server. The virtual machine should not be allowed, because the virtual opportunity discards the packets of the non target address and cannot capture the mirrored packets. If you use multi line bgp to connect with the operators, please check the switch documents first and then mirror them Like configuration, to avoid the impact of image port congestion on the packets in and out of the service port.
Choose the internal network port of load balancing. If the load balancing is a linux server, fprobe can be directly deployed on the linux server. If fprobe cannot be used, image data needs to be sent to a linux physical server, and then processed by fprobe.
Select the switch gateway, and use the netflow configuration of the switch directly, but pay attention to the sampling ratio to prevent excessive cpu memory consumption and production impact.

Configuration based on switch acquisition

For detailed configuration, please refer to the netflow documents of each switch
Basic knowledge and ideas: Network traffic analysis tool - visualization Network - netflow [2] - Introduction and configuration of Cisco NetFlow working principle

Configuration example

flow record try
description test
match ipv4 source address
match ipv4 destination address
match ipv4 protocol
match transport sourceport
match transport destinationport
collect counter bytes
collect counter packets long
collect timestamp sysuptime first
collect timestamp sysuptime last

flow exporter try_exporter
description test_ex
destination 10.136.76.117
source Loopback1
transport udp 9999
template data timeout 30

flow monitor try
description test
exporter try_exporter
cache type immediate
record try

sampler test1
mode deterministic 1 outof 2

interface GigabitEthernet0/0/4
ip flow monitor try sampler test1 input
ip flow monitor try sampler test1 output

Configuration and self start monitoring based on fprobe acquisition

Please check the installation method Powerful tool of network traffic analysis visual network data collector fprobe under netflow [5] - linux

Configuration example

fprobe -i eth0 -e 10 10.2.82.60:9999

Self starting script:

In order to prevent fprobe failure caused by process exception or restart, use the following script to generate crontab job for regular check. The script determines the start command of fprobe by looking up port 9999. Make sure the fprobe process is started.

## Check the basic configuration and generate the ABCD NetFlow folder
ll /jian_netflow/
ps -ef | grep fpro | grep -v grep
crontab  -l|grep netflow
mkdir /jian_netflow
cd /jian_netflow 

## Generate the do \netflow.sh script
res=$(ps -ef | grep fpr|grep -v grep|grep -o fpro.*999)
echo 'res_ps=$(ps -ef | grep fprobe | grep -v grep)' > do_netflow.sh 
echo 'if [ -z "$res_ps" ];then' >> do_netflow.sh 
echo '/usr/sbin/'$res >> do_netflow.sh 
echo 'fi' >> do_netflow.sh 
cat do_netflow.sh 
chmod 755 do_netflow.sh 
echo '*/1 * * * * cd /jian_netflow/&&/bin/bash do_netflow.sh'>>/var/spool/cron/root ##Configure crontab jobs

## Check build results
ll /jian_netflow/do_netflow.sh  ## Check the existence of do_netflow.sh
tt=$?
ps -ef | grep fpro | grep -v grep ## Check if fprobe process starts
ff=$?
crontab  -l|grep netflow ## Check if crontab job exists
nn=$?
echo result:$tt'|'$ff'|'$nn ##Return the processing result. If it is all 0, it is correct
ls  ##To view the file under / Jian NetFlow, you should see the script do netflow.sh

nfdump configuration and high availability

Please check the installation method Network traffic analysis tool - visualization Network - netflow [4] - Introduction to nfdump receiver
nfdump basic configuration (listening for 9999 port):

nfcapd -w 1 -t 60 -D -p 9999 -T all -S 0 -l /data1/netflow/netflow_neiwang -P nfcapd_neiwang.pid

High availability:

Here, we use the keepalived software to generate Vip. In fact, we use vrrp protocol. Please check [connect (I haven't written yet)]. Two nfdump receivers use the keepalived software at the same time. The effect is that one of the servers carries Vip and the other is in the hot standby state. Once the main device goes down, the standby device immediately carries Vip.
Install and start:

yum -y install keepalived 
service keepalived start

Determine the Vip and configure it (please use the same network segment and conflicting address for Vip)

## A device
## cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
}
vrrp_instance GLOBAL_10_20.81.183 {
    state BACKUP
    interface eth0
    virtual_router_id 212
    priority 190
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.20.81.183
    }
}

## B device
## cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
}
vrrp_instance GLOBAL_10_202.81.183 {
    state BACKUP
    interface eth0
    virtual_router_id 212
    priority 190
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.202.81.183
    }
}

## After two devices are configured, both devices need to be restarted at the same time
service keepalived restart

## keepalived self startup
chkconfig keepalived on

keepalived is configured well. If the firewall policy is not enabled, there will be two devices carrying Vip at the same time. This situation is very bad, because Vip will occupy each other, resulting in data can not be received continuously. Execute the following command to add the policy of releasing vrrp protocol

iptables -A INPUT -p vrrp -j ACCEPT
service iptables save

Check: use ip a to check two devices, and only one side should carry Vip.

Sorting and timing compression script

Effect:

Organize script: move the generated nfcapd file and record the log. Please run the script after the nfcapd file is generated normally.

Scheduled compression script: in the early morning of the next day, compress the nfcapd file generated the previous day, and delete the expired compressed data. Here, it is set to delete the compressed file 45 days ago.

Storage location:

The folder specified by nfcapd. The storage location in this instance is: / data1/netflow/netflow_neiwang

Execute command:

## If the grooming script is exchange.sh, execute
nohup sh exchange_neiwang.sh &
## If the scheduled compression script is yasuo.sh, execute
nohup sh yasuo.sh &

Collation script

#!/bin/bash
#Start time:
# date --date=@1506651600 +%Y%m%d%H%M
#201709291020
#date -d "2014-12-05 19:45:44" +%s
#1417779944

###############Variable parameters
time_i=$(date -d "+1 minute"  "+%Y-%m-%d %H:%M")
timestamp=$(date -d "$time_i" +%s)
#Starting time
###############################################

###############Fixed parameter
save_route1=../nfdump_imfo
#Store one day compressed package tgz path
####################################

while [ 1 ]
    do
        #ee is the specified date including minutes, and the folder date specified by eedir is only days
        ee=$(date --date=@$timestamp +%Y%m%d%H%M )
        eedir=$(date --date=@$timestamp +%Y%m%d )
        file=nfcapd.$ee
        folder_nfdump_infoname=nfdump_info_$eedir

        #If there is no specified file plus, generate one for tgz storage, save by date
        #If there is no specified file plus, generate one for nfcapd storage, save by date
        #If the specified file is not added, a will be generated for log storage and saved by date
        if [ ! -d "$save_route1" ];then
            mkdir $save_route1
        fi
        if [ ! -d "$folder_nfdump_infoname" ];then
            mkdir $folder_nfdump_infoname
        fi
        if [ ! -d "log_netflow" ];then
            mkdir log_netflow
        fi
        #Organize the generated nfcapd files into the $folder? Nfdump? Infoname folder
        #And keep a log
        #File has sleep 5s, no sleep 60s
        if [ -f "$file" ];then
            echo read $file  >> log_netflow/log.txt_$eedir
            mv $file $folder_nfdump_infoname/$file      
            timestamp=$[$timestamp+60]
            sleep 5
        else
            echo Non-existent $file >> log_netflow/log.log_$eedir
            sleep  60
        fi
done

Timed compression script

###############Fixed parameter
save_route1=../nfdump_imfo
####################################

while [ 1 ]
    do
        #The folder date specified by eedir is only days
        eedir=$(date  +%Y%m%d )
        folder_nfdump_infoname=nfdump_info_$eedir
            if [ -f  ${folder_nfdump_infoname}/nfcapd.${eedir}0100 ];then
                time_i_yasuo=$(date -d "-1 day"  "+%Y%m%d")
                folder_nfdump_infoname=nfdump_info_$time_i_yasuo
                if [ -d $folder_nfdump_infoname ];then
                tar -zcvf "$folder_nfdump_infoname".tgz $folder_nfdump_infoname  >> log_netflow/log.tgz_$time_i_yasuo
                echo $(date) generate"$folder_nfdump_infoname".tgz >> log_netflow/log.txt_$time_i_yasuo
                mv "$folder_nfdump_infoname".tgz $save_route1
                echo $(date) move"$folder_nfdump_infoname".tgz >> log_netflow/log.txt_$time_i_yasuo
                rm -rf "$folder_nfdump_infoname"
                echo $(date) delete"$folder_nfdump_infoname"Folder >> log_netflow/log.txt_$time_i_yasuo
                            fi  
            fi
    sleep  60  

    time_i_delete=$(date -d "-45 day"  "+%Y%m%d")
    folder_nfdump_infoname_delete=nfdump_info_$time_i_delete
                #tgz filename to delete 9 days ago   
    rm -rf "$save_route1"/"$folder_nfdump_infoname_delete".tgz
                #Delete tgz files 45 days ago
done

Traffic query

Extension (incomplete)

Get one minute packet traffic, which can be converted to bandwidth, and filter specific addresses. Carry out traffic cost accounting of Intranet address.

Storage (incomplete)

The data is stored in a specific format in the database.

Presentation (incomplete)

Show through grafana

Tags: Operation & Maintenance network Linux crontab iptables

Posted on Thu, 12 Mar 2020 03:28:37 -0400 by Highland3r