Dr. Niall McMahon
2022-11-22
If you print these slides, think about using two pages per sheet although don't worry too much about it!
-
Netflow was introduced on Cisco routers in the mid-1990s.
-
Netflow Version 5 is the most popular; enabling it is straightforward.
-
Version 5 is supported across the Cisco router range.
-
Version 5 uses bandwidth the most efficiently of all Netflow versions.
-
Netflow-enabled routers (and switches) can collect IP traffic statistics on all Netflow-enabled interfaces.
-
Routers and switches then forward the Netflow data to a Netflow collector.
-
The collector is usually a server running analysis code.
-
A router is an infrastructure device that moves data packets between computer networks.
-
Routers direct data traffic on the internet.
-
This data is sent in the form of packets from router to router.
-
A router is normally connected to at least two IP networks, i.e. two data lines.
-
The packet header contains the packet destination.
-
Using information in its routing table or routing policy, the router forwards the packet towards its destination, usually via intermediate routers.
You can find a nice description of how routers work here.
-
Home and small office internet.
-
Enterprise and internet grade machines.
Data traffic enters and exits devices via interfaces. An interface can be physical - a hardware port - or logical, defined in software. A router is used to connect networks and so needs an interface into each. This is defined by an IP address. Common interfaces include:
-
Ethernet (often,
eth0
)
-
Wireless (often,
wlan0
)
-
The loopback interface is a virtual - in software - interface that is not tied to a particular port or hardware. It has two main functions: (i) to allow network programs on the same computer to communicate and (ii) to provide a stable, always-on and hardware independent, interface. This is useful for important connections or for device management access.
-
A network packet is a unit of data carried by on a packet-switched network.
-
A packet consists of: (i) control information, (ii) user data.
-
The user data is the payload.
-
The control information contains the information required to deliver the payload, e.g. the source and destination network addresses.
-
Control information resides in packet headers and packet trailers.
-
Packet switched networks share the line bandwidth between multiple communications. Similar to multitasking on a CPU. See our notes on sockets.
-
Circuit switched networks devote a line to a communication for its duration.
-
A packet strictly refers to a data unit at Layer 3, the Network Layer. The Network Layer is responsible for directing the flow of packets across a network, i.e. the instructions a packet's control information. This includes the IP addresses of the sender and receiver
-
A Layer 2 unit is is a frame. Layer 2 is the Data Link layer, responsible for connecting devices across the Physical (hardware) Layer. It takes packets and produces a frame.
-
A Layer 4 unit is a segment or a datagram. Layer 4 is the Transport Layer, responsible for moving data to and from a particular process. The process prepares its data and passes it to the Transport Layer.
-
For TCP/IP over Ethernet, a TCP segment is carried by IP packets, which are each carried in one or more Ethernet frames.
A flow is a unidirectional sequence of packets that have the same values for seven key fields, taken together called a tuple; these seven values define a unique key for the flow. they are:
-
Ingress interface (SNMP ifIndex).
-
Source IP address.
-
Destination IP address.
-
IP protocol.
-
Source port for UDP or TCP. (0 for other protocols.)
-
Destination port for UDP or TCP. (Type and code for ICMP and 0 for other protocols.)
-
IP Type of Service. (This is the second byte of the IPV4 header.)
Printing a Netflow summary will look like:
Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows
2022-11-22 00:00:00.500 0.000 UDP 127.0.0.1:24920 -> 192.168.0.1:22126 1 46 1
2022-11-22 00:00:00.600 0.000 UDP 192.168.0.1:22126 -> 127.0.0.1:24920 1 80 1
-
There is a tuple for every flow on the internet.
-
If you know that tuple, you can fast switch every packet on the internet.
-
Fast switching uses meta information about packets to optimise switching.
-
Netflow-enabled routers will output a flow record when the flow is finished.
-
When the router sees new traffic for an existing flow it resets the aging counter.
-
TCP session termination in a TCP flow will also expire the flow.
-
Routers can be configured to create a flow record at a fixed intervals.
-
NetFlow records are exported using User Datagram Protocol (UDP) to the Netflow collector.
-
The IP address of the NetFlow collector and the destination UDP port are configured on the router.
-
This is often UDP port 2055.
-
If flow records are lost or corrupted on the way to the collector, they cannot be retrieved.
-
The UDP protocol does not inform the router of the loss.
-
Some modern implementations of NetFlow use the Stream Control Transmission Protocol (SCTP) - to help reduce the risk of loss - but this introduces performance issues.
All Netflow packets begin with version-dependent header. The structure is:
-
Bytes 0 - 1:
version
. Netflow export format version, e.g. v5.
-
Bytes 2 - 3:
count
. Number of flows exported in this packet (1 - 30).
-
Bytes 4 - 7:
sys_uptime
. Time in milliseconds since device boot.
-
Bytes 8 - 11:
unix_secs
. Unix time. At moment of export.
-
Bytes 12 - 15:
unix_nsecs
. Residual Unix nanoseconds.
-
Bytes 16 - 19:
flow_sequence
. Sequence counter of total flows seen.
-
Byte 20:
engine
. Flow-switching engine type.
-
Byte 21:
engine_id
. Slot number on flow-switching engine.
-
Bytes 22 - 23:
sampling_interval
. Sampling mode and value.
The NetFlow v5 contains:
-
Input interface index used by SNMP (ifIndex in IF-MIB).
-
Output interface index or zero if the packet is dropped.
-
Timestamps for the flow start and finish time, in milliseconds since the last boot.
-
Number of bytes and packets observed in the flow
-
Layer 3 headers:
-
Source & destination IP addresses
-
ICMP Type and Code.
-
IP protocol
-
Type of Service (ToS) value
-
Source and destination port numbers for TCP, UDP, SCTP.
-
For TCP flows, the union of all TCP flags observed over the life of the flow.
-
Layer 3 Routing information:
-
IP address of the immediate next-hop (not the BGP nexthop) along the route to the destination
-
Source & destination IP masks (prefix lengths in the CIDR notation)
-
Bytes 0-3:
srcaddr
. Source IP address.
-
Bytes 4-7:
dstaddr
. Destination IP address.
-
Bytes 8-11:
nexthop
. IP address of next hop router.
-
Bytes 12-13:
input
. SNMP index of input interface.
-
Bytes 14-15:
output
. SNMP index of output interface.
-
Bytes 16-19:
dPkts
. Packets in the flow.
-
Bytes 20-23:
dOctets
. Total number of Layer 3 bytes in the packets of the flow.
-
Bytes 24-27:
first
. SysUptime at start of flow.
-
Bytes 28-31:
last
. SysUptime at the time the last packet of the flow was received.
-
Bytes 32-33:
srcport
. TCP/UDP source port number or equivalent.
-
Bytes 34-35:
dstport
. TCP/UDP destination port number or equivalent.
-
Bytes 36:
pad1
. Unused (zero) bytes.
-
Bytes 37:
tcp_flags
. Cumulative OR of TCP flags.
-
Bytes 38:
prot
. IP protocol type (TCP = 6; UDP = 17).
-
Bytes 39:
tos
. IP type of service (ToS).
-
Bytes 40-41:
src_as
. Autonomous system number of the source, either origin or peer.
-
Bytes 42-43:
dst_as
. Autonomous system number of the destination, either origin or peer.
-
Bytes 44:
src_mask
. Source address prefix mask bits.
-
Bytes 45:
dst_mask
. Destination address prefix mask bits.
-
Bytes 46-47:
pad2
. Unused (zero) bytes.
See Example Cisco NetFlow Config - Standard version 5 from SolarWinds.
First enable the global set of commands:
ip flow-export destination ip_address port
ip flow-export source interface
ip flow-export version 5 [peer-as | origin-as]
ip flow-cache timeout active 1
ip flow-cache timeout inactive 15
snmp-server ifindex persist
Then enable Netflow on all Layer-3 interfaces - on an interface by interface basis - you want to watch, using:
ip route-cache flow
Newer devices use:
ip flow ingress
There is also the ip flow egress
command for special circumstances.
Many other manufacturers have Netflow equivalent systems, including Juniper, 3Com/Hp, Huawei, Nokia, Ericsson, Citrix and many more.