Flow masking and ranging aggregation

aggregation AGGREGATIONFLAG SRCPORTHW SRCPORTLW DSTPORTHW DSTPORTLW SUBNET_ON

Flow masking and ranging aggregation, WTF?

Imagine you are interested in flow based statistics of traffic between networks or between certain port ranges, or you like to get rid of VLANs or protocols? You are not? Go to some place else and make yourself useful. If yes, keep on reading.

Note, I’m not talking about the different L3/4 operational modes already listed in The basics tutorial. They are discussed in the L2/3 flow aggregation modes tutorial.

In this tutorial we explore the masking and ranging flexibility to redefine flows. So the option to remove one of these parameters or aggregate several IPs in one flow, e.g. all /24 or according to country and organization.

Preparation

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:

t2build -e -y

Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Then compile the core (tranalyzer2) and the following plugins:

t2build tranalyzer2 basicFlow basicStats connStat txtSink

...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:

mkdir ~/data ~/results

The sample PCAPs used in this tutorial can be downloaded here:

Please save them in your ~/data folder.

Now you are all set!

T2 flow aggregation scheme

The constants controlling the flow aggregation are residing in tranalyzer.h. Open the file and search for // Aggregation modes as shown below:

tranalyzer2

vi src/tranalyzer.h

...
#define SUBNET_ON       1 // Core control of subnet function for plugins

/* -------------------------------------------------------------------------- */
/* -------------------- DO NOT EDIT THE FOLLOWING BLOCKS -------------------- */
/* -------------------------------------------------------------------------- */

// Aggregation modes
#define L4PROT  0x01
#define DSTPORT 0x02
#define SRCPORT 0x04
#define DSTIP   0x08
#define SRCIP   0x10
#define VLANID  0x20
#define SUBNET  0x80

// SUBNET mode: IP flow aggregation network masks
#define CNTRY_MSK 0xff800000
#define TOR_MSK   0x00400000
#define ORG_MSK   0x003fffff

#define NETIDMSK  (CNTRY_MSK | ORG_MSK) // netID mask

/* -------------------------------------------------------------------------- */
/* -------------------------------------------------------------------------- */
/* -------------------------------------------------------------------------- */

// Flow Aggregation
#define AGGREGATIONFLAG 0x00 // each bit: 1: aggregation activated, see aggregation modes defines above

#define SRCIP4CMSK 24 // src IPv4 aggregation CIDR mask
#define DSTIP4CMSK 24 // dst IPv4 aggregation CIDR mask

#define SRCIP6CMSK 120 // src IPv6 aggregation CIDR mask
#define DSTIP6CMSK 120 // dst IPv6 aggregation CIDR mask

#define SRCPORTLW 1    // src port lower bound
#define SRCPORTHW 1024 // src port upper bound

#define DSTPORTLW 1    // dst port lower bound
#define DSTPORTHW 1024 // dst port upper bound
...

The aggregation modes define a specific bit in the eight bit AGGREGATIONFLAG. Default is 0x00, so normal six tuple aggregation. From L4PROT to VLANID the aggregation operates directly on the 6-tuple hash resulting in accumulation of packets from different 6-tuple flows. If SUBNET is set the subnet tables are loaded as if SUBNET_ON is activated. Both are mutual exclusive, so SUBNET_ON can be 0 while the AGGREGATIONFLAG = SUBNET. In the following the activation of each mode is discussed. Let’s start simple with the network IP aggregation.

Network flow aggregation

Imagine you are interested in traffic only flowing between networks. e.g. 10.4.1.0/24 to 10.5.2.0/24 or from 10.4.5.0/24 to all outside networks. Or even better aggregate all traffic between universities of China and a specific corporation in USA, would that be neat, right? Since version 0.9 you can do it. Goto chapter Subnet aggregation` below.

Switch now SRCIP and DSTIP on, it does not make any sense to switch only one on, as we cannot know before the packets is sorted into a masked flow which network mask is to apply. Try the default /24 mask.

The following configs are equivalent:

t2conf tranalyzer2 -D AGGREGATIONFLAG="(SRCIP | DSTIP)" && t2build -R

or for the HEX friends

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x18 && t2build -R && t2build -R

and run t2:

t2 -r ~/data/annoloc2.pcap -w ~/results

================================================================================
Tranalyzer 0.9.4 (Anteater), Cobra. PID: 43547, Prio: 0, SID: 666
================================================================================
Date: 1751989854.000140670 sec (Tue 08 Jul 2025 17:50:54 CEST)
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.4
    02: basicStats, 0.9.4
    03: connStat, 0.9.4
    04: txtSink, 0.9.4
[INF] IPv4 Ver: 6, Rev: 02072025, Range Mode: 0, subnet ranges loaded: 7237971 (7.24 M)
[INF] IPv6 Ver: 6, Rev: 02072025, Range Mode: 0, subnet ranges loaded: 1419101 (1.42 M)
Processing file: /home/user/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 66
Dump start: 1022171701.691172000 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
Dump stop : 1022171726.640398000 sec (Thu 23 May 2002 16:35:26 GMT)
Total dump duration: 24.949226000 sec
Finished processing. Elapsed time: 0.645772134 sec
Finished unloading flow memory. Time: 0.860106555 sec
Percentage completed: 100.00%
Number of processed packets: 1219015 (1.22 M)
Number of processed bytes: 64082726 (64.08 M)
Number of raw bytes: 844642686 (844.64 M)
Number of pad bytes: 1758405 (1.76 M)
Number of pcap bytes: 83586990 (83.59 M)
Number of L2 packets: 247 [0.02%]
Number of IPv4 packets: 1218588 (1.22 M) [99.96%]
Number of IPv6 packets: 180 [0.01%]
Number of A packets: 564213 (564.21 K) [46.28%]
Number of B packets: 654802 (654.80 K) [53.72%]
Number of A bytes: 29447146 (29.45 M) [45.95%]
Number of B bytes: 34635580 (34.64 M) [54.05%]
<A packet load>: 52.19
<B packet load>: 52.89
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 1480 (1.48 K)
basicStats: Flow max(b/s), pkts: 19015999488 (19.02 Gb/s), 2
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 2622 (2.62 K) [0.00%] bytes
basicStats: Biggest L3 flow talker: 138.212.189.0 (JP): 23601 (23.60 K) [1.94%] packets
basicStats: Biggest L3 flow talker: 138.212.189.0 (JP): 33731054 (33.73 M) [52.64%] bytes
connStat: Number of unique source IPs: 2498 (2.50 K)
connStat: Number of unique destination IPs: 1816 (1.82 K)
connStat: Number of unique source/destination IPs connections: 186
connStat: Max unique number of source IP / destination port connections: 407
connStat: IP connF=connSipDprt/connSip: 0.162930
connStat: IP connG=connSipDprt/connSipDip: 2.188172
connStat: Source IP with max connections: 138.212.189.0 (JP): 499 connections
connStat: Destination IP with max connections: 138.212.187.0 (JP): 690 connections
connStat: Biggest L3 talker: 138.212.187.0 (JP): 141530 (141.53 K) [11.61%] packets
connStat: Biggest L3 talker: 138.212.187.0 (JP): 169510114 (169.51 M) [264.52%] bytes
--------------------------------------------------------------------------------
Headers count: min: 2, max: 5, avg: 3.01
Number of ARP packets: 247 [0.02%]
Number of GRE packets: 20 [0.00%]
Number of IGMP packets: 12 [0.00%]
Number of ICMP packets: 3059 (3.06 K) [0.25%]
Number of ICMPv6 packets: 11 [0.00%]
Number of TCP packets: 948743 (948.74 K) [77.83%]
Number of TCP bytes: 52643546 (52.64 M) [82.15%]
Number of UDP packets: 266900 (266.90 K) [21.89%]
Number of UDP bytes: 11234272 (11.23 M) [17.53%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 16374 (16.37 K)
Number of processed L2   flows: 99 [0.60%]
Number of processed IPv4 flows: 16213 (16.21 K) [99.02%]
Number of processed IPv6 flows: 62 [0.38%]
Number of processed A    flows: 9149 (9.15 K) [55.88%]
Number of processed B    flows: 7225 (7.22 K) [44.12%]
Number of request        flows: 9127 (9.13 K) [55.74%]
Number of reply          flows: 7247 (7.25 K) [44.26%]
Total   A/B    flow asymmetry: 0.12
Total req/rply flow asymmetry: 0.11
Number of processed A+B packets/A+B flows: 74.45
Number of processed A   packets/A   flows: 61.67
Number of processed   B packets/  B flows: 90.63
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B   packets/s: 48859.83 (48.86 K)
Number of processed A     packets/s: 22614.45 (22.61 K)
Number of processed   B   packets/s: 26245.38 (26.25 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<Number of processed flows/s>: 656.29
<Bandwidth>: 270268555 b/s (270.27 Mb/s)
<Snapped bandwidth>: 20548205 b/s (20.55 Mb/s)
<Raw bandwidth>: 270835716 b/s (270.84 Mb/s)
Max number of flows in memory: 16374 (16.37 K) [6.25%]
Memory usage: 0.15 GB [0.22%]
Aggregated flowStat=0x0c0018fa0222d044
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
[INF] Stop dissecting: Clipped packet, unhandled protocol or subsequent fragment
[INF] Layer 2 flows
[INF] IPv4 flows
[INF] IPv6 flows
[INF] ARP
[INF] IPv4/6 fragmentation
[INF] IPv4/6 in IPv4/6
[INF] GRE encapsulation
[INF] GTP tunnel
[INF] SSDP/UPnP

As you can see, basicStats and connStat report now /24 networks. The resulting flow file is a bit shorter, because several IPv4/6 flows are aggregated into /24 or /120 flows respectively.

tail -n 13 ~/results/annoloc2_flows.txt | tcol

B  129   0x0400000200004001  1022171701.710399000  1022171726.638720000  24.928321000  1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800    19.67.132.0    us  "!Ford Motor Company"               27005  138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    27025  17  1782  1208  0      65571     153934    28    61    36.7963   6.098478  0  0.071746  0.01398895   0.009389925  71.48495  2630.382  0.1919732    -0.4025558  1  1  1  1  1    1  0     4198402
A  1109  0x0400000200004000  1022171701.919937000  1022171726.638720000  24.718783000  1  3  eth:ipv4:icmp  00:d0:02:6d:78:00  00:00:1c:b6:1a:53  0x0800    193.107.159.0  at  "AT-WESTNET"                        0      138.212.184.0  jp  "!ASAHI KASEI CORPORATION"    0      1   104   0     0      2912      0         28    28    28        0         0  0.249289  0.2376806    0.03044914   4.207327  117.8052  1            1           1  1  1  1  1    1  104   2912
A  324   0x0400000200004000  1022171701.712093000  1022171726.638722000  24.926629000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:bf:08:44:81  0x0800    19.59.134.0    us  "!Ford Motor Company"               65230  138.212.187.0  jp  "!ASAHI KASEI CORPORATION"    58290  6   9459  5223  0      13696632  0         1448  1448  1448      0         0  0.067445  0.00263523   0.006631293  379.4737  549477.9  0.2885166    1           1  2  2  2  2    1  9459  13696632
B  324   0x0400000000004001  1022171701.713111000  1022171726.639230000  24.926119000  1  3  eth:ipv4:tcp   00:50:bf:08:44:81  00:d0:02:6d:78:00  0x0800    138.212.187.0  jp  "!ASAHI KASEI CORPORATION"          58290  19.59.134.0    us  "!Ford Motor Company"         65230  6   5223  9459  0      0         13696632  0     0     0         0         0  0.066113  0.004772384  0.008416669  209.5392  0         -0.2885166   -1          2  1  1  1  0.5  1  0     4198402
A  473   0x0400000000004000  1022171701.723484000  1022171726.638724000  24.915240000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:da:68:7f:84  0x0800    216.83.48.0    us  "IPXO"                              13600  138.212.188.0  jp  "!ASAHI KASEI CORPORATION"    20     6   864   1295  5184   0         1767728   0     0     0         0         0  0.068622  0.02883708   0.01269573   34.67757  0         -0.1996295   -1          1  1  2  2  2    1  864   0
B  473   0x0400000200004001  1022171701.724443000  1022171726.640395000  24.915952000  1  3  eth:ipv4:tcp   00:50:da:68:7f:84  00:d0:02:6d:78:00  0x0800    138.212.188.0  jp  "!ASAHI KASEI CORPORATION"          20     216.83.48.0    us  "IPXO"                        13600  6   1295  864   0      1767728   0         1176  1460  1365.041  134.3699  0  0.112955  0.01924014   0.02731735   51.97474  70947.64  0.1996295    1           1  1  1  1  1    1  0     4198402
A  703   0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:10:a7:02:4d:33  0x0800    70.98.46.0     us  "ABUL-14-7385"                      6699   138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    2603   6   32    28    0      33280     96        666   1414  1040      401.9105  0  2.251048  0.777589     0.9006743    1.286026  1337.468  0.06666667   0.9942474   1  2  2  2  2    1  32    33280
B  703   0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  1  3  eth:ipv4:tcp   00:10:a7:02:4d:33  00:d0:02:6d:78:00  0x0800    138.212.185.0  jp  "!ASAHI KASEI CORPORATION"          2603   70.98.46.0     us  "ABUL-14-7385"                6699   6   28    32    96     96        33280     0     8     3.428571  4.210036  0  2.292653  0.88349      0.8520955    1.131875  3.880713  -0.06666667  -0.9942474  2  1  1  1  0.5  1  0     4198402
A  9149  0x0400000200004000  1022171726.638730000  1022171726.638730000  0.000000000   1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:04:75:85:fd:a2  0x0800    192.50.41.0    jp  "!Toyohashi University of Technol"  64251  138.212.190.0  jp  "!ASAHI KASEI CORPORATION"    1112   17  1     0     12     6         0         6     6     6         0         0  0         0            0            0         0         1            1           1  1  1  1  1    1  6     36
A  21    0x0400000200004000  1022171701.691707000  1022171726.638737000  24.947030000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:00:e8:87:02:d1  0x0800    138.212.226.0  jp  "!ASAHI KASEI CORPORATION"          1103   138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    445    6   543   985   2868   1701      1407845   0     63    3.132597  13.76817  0  0.589     0.04594295   0.03578168   21.76612  68.18447  -0.289267    -0.9975864  1  1  2  2  2    1  5762  20820
B  21    0x0400000200004001  1022171701.692758000  1022171726.640391000  24.947633000  1  3  eth:ipv4:tcp   00:00:e8:87:02:d1  00:d0:02:6d:78:00  0x0800    138.212.185.0  jp  "!ASAHI KASEI CORPORATION"          445    138.212.226.0  jp  "!ASAHI KASEI CORPORATION"    1103   6   985   543   0      1407845   1701      183   1460  1429.284  191.0024  0  0.889418  0.02532757   0.0522222    39.4827   56432     0.289267     0.9975864   1  1  1  1  1    1  0     4198402
A  1035  0x0400000000004000  1022171701.876636000  1022171726.639226000  24.762590000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:01:02:b4:36:56  0x0800    133.26.84.0    jp  "MEIJI-NET Meiji University"        4766   138.212.187.0  jp  "!ASAHI KASEI CORPORATION"    80     6   1692  2729  10152  0         3970812   0     0     0         0         0  0.614191  0.01463511   0.04707645   68.32888  0         -0.2345623   -1          1  1  2  2  2    1  8168  0
B  1035  0x0400000200004001  1022171701.877349000  1022171726.639232000  24.761883000  1  3  eth:ipv4:tcp   00:01:02:b4:36:56  00:d0:02:6d:78:00  0x0800    138.212.187.0  jp  "!ASAHI KASEI CORPORATION"          80     133.26.84.0    jp  "MEIJI-NET Meiji University"  4766   6   2729  1692  12     3970812   0         0     1460  1455.043  70.74226  0  0.480045  0.009073598  0.03994242   110.2097  160359.9  0.2345623    1           1  1  1  1  1    1  0     4198402

Note the IP which is masked with SRCIP4CMSK and DSTIP4CMSK. If a list of IP contained in a flow is desired set the constant BFO_SUBNET_IPLIST in basicFlow.h to 1. The number of IPs displayed is defined by the constant BFO_MAX_IP, by default 5.

basicFlow

vi src/basicFlow.h

...
#define BFO_SUBNET_IPLIST      0 // 0: Display only the IP masked by SRCIP[46]CMSK and DSTIP[46]CMSK
                                 // 1: Display a list of IP aggregated
...

Set BFO_SUBNET_IPLIST=1, recompile basicFlow and rerun T2 on the same pcap.

t2conf basicFlow -D BFO_SUBNET_IPLIST=1 && t2build basicFlow

t2 -r ~/data/annoloc2.pcap -w ~/results/

tail -n 13 ~/results/annoloc2_flows.txt | tcol

B  129   0x0400000200004001  1022171701.710399000  1022171726.638720000  24.928321000  1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800    19.67.132.0    us  "!Ford Motor Company"               27005  138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    27025  17  1782  1208  0      65571     153934    28    61    36.7963   6.098478  0  0.071746  0.01398895   0.009389925  71.48495  2630.382  0.1919732    -0.4025558  1  1  1  1  1    1  0     4198402
A  1109  0x0400000200004000  1022171701.919937000  1022171726.638720000  24.718783000  1  3  eth:ipv4:icmp  00:d0:02:6d:78:00  00:00:1c:b6:1a:53  0x0800    193.107.159.0  at  "AT-WESTNET"                        0      138.212.184.0  jp  "!ASAHI KASEI CORPORATION"    0      1   104   0     0      2912      0         28    28    28        0         0  0.249289  0.2376806    0.03044914   4.207327  117.8052  1            1           1  1  1  1  1    1  104   2912
A  324   0x0400000200004000  1022171701.712093000  1022171726.638722000  24.926629000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:bf:08:44:81  0x0800    19.59.134.0    us  "!Ford Motor Company"               65230  138.212.187.0  jp  "!ASAHI KASEI CORPORATION"    58290  6   9459  5223  0      13696632  0         1448  1448  1448      0         0  0.067445  0.00263523   0.006631293  379.4737  549477.9  0.2885166    1           1  2  2  2  2    1  9459  13696632
B  324   0x0400000000004001  1022171701.713111000  1022171726.639230000  24.926119000  1  3  eth:ipv4:tcp   00:50:bf:08:44:81  00:d0:02:6d:78:00  0x0800    138.212.187.0  jp  "!ASAHI KASEI CORPORATION"          58290  19.59.134.0    us  "!Ford Motor Company"         65230  6   5223  9459  0      0         13696632  0     0     0         0         0  0.066113  0.004772384  0.008416669  209.5392  0         -0.2885166   -1          2  1  1  1  0.5  1  0     4198402
A  473   0x0400000000004000  1022171701.723484000  1022171726.638724000  24.915240000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:da:68:7f:84  0x0800    216.83.48.0    us  "IPXO"                              13600  138.212.188.0  jp  "!ASAHI KASEI CORPORATION"    20     6   864   1295  5184   0         1767728   0     0     0         0         0  0.068622  0.02883708   0.01269573   34.67757  0         -0.1996295   -1          1  1  2  2  2    1  864   0
B  473   0x0400000200004001  1022171701.724443000  1022171726.640395000  24.915952000  1  3  eth:ipv4:tcp   00:50:da:68:7f:84  00:d0:02:6d:78:00  0x0800    138.212.188.0  jp  "!ASAHI KASEI CORPORATION"          20     216.83.48.0    us  "IPXO"                        13600  6   1295  864   0      1767728   0         1176  1460  1365.041  134.3699  0  0.112955  0.01924014   0.02731735   51.97474  70947.64  0.1996295    1           1  1  1  1  1    1  0     4198402
A  703   0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:10:a7:02:4d:33  0x0800    70.98.46.0     us  "ABUL-14-7385"                      6699   138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    2603   6   32    28    0      33280     96        666   1414  1040      401.9105  0  2.251048  0.777589     0.9006743    1.286026  1337.468  0.06666667   0.9942474   1  2  2  2  2    1  32    33280
B  703   0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  1  3  eth:ipv4:tcp   00:10:a7:02:4d:33  00:d0:02:6d:78:00  0x0800    138.212.185.0  jp  "!ASAHI KASEI CORPORATION"          2603   70.98.46.0     us  "ABUL-14-7385"                6699   6   28    32    96     96        33280     0     8     3.428571  4.210036  0  2.292653  0.88349      0.8520955    1.131875  3.880713  -0.06666667  -0.9942474  2  1  1  1  0.5  1  0     4198402
A  9149  0x0400000200004000  1022171726.638730000  1022171726.638730000  0.000000000   1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:04:75:85:fd:a2  0x0800    192.50.41.0    jp  "!Toyohashi University of Technol"  64251  138.212.190.0  jp  "!ASAHI KASEI CORPORATION"    1112   17  1     0     12     6         0         6     6     6         0         0  0         0            0            0         0         1            1           1  1  1  1  1    1  6     36
A  21    0x0400000200004000  1022171701.691707000  1022171726.638737000  24.947030000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:00:e8:87:02:d1  0x0800    138.212.226.0  jp  "!ASAHI KASEI CORPORATION"          1103   138.212.185.0  jp  "!ASAHI KASEI CORPORATION"    445    6   543   985   2868   1701      1407845   0     63    3.132597  13.76817  0  0.589     0.04594295   0.03578168   21.76612  68.18447  -0.289267    -0.9975864  1  1  2  2  2    1  5762  20820
B  21    0x0400000200004001  1022171701.692758000  1022171726.640391000  24.947633000  1  3  eth:ipv4:tcp   00:00:e8:87:02:d1  00:d0:02:6d:78:00  0x0800    138.212.185.0  jp  "!ASAHI KASEI CORPORATION"          445    138.212.226.0  jp  "!ASAHI KASEI CORPORATION"    1103   6   985   543   0      1407845   1701      183   1460  1429.284  191.0024  0  0.889418  0.02532757   0.0522222    39.4827   56432     0.289267     0.9975864   1  1  1  1  1    1  0     4198402
A  1035  0x0400000000004000  1022171701.876636000  1022171726.639226000  24.762590000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:01:02:b4:36:56  0x0800    133.26.84.0    jp  "MEIJI-NET Meiji University"        4766   138.212.187.0  jp  "!ASAHI KASEI CORPORATION"    80     6   1692  2729  10152  0         3970812   0     0     0         0         0  0.614191  0.01463511   0.04707645   68.32888  0         -0.2345623   -1          1  1  2  2  2    1  8168  0
B  1035  0x0400000200004001  1022171701.877349000  1022171726.639232000  24.761883000  1  3  eth:ipv4:tcp   00:01:02:b4:36:56  00:d0:02:6d:78:00  0x0800    138.212.187.0  jp  "!ASAHI KASEI CORPORATION"          80     133.26.84.0    jp  "MEIJI-NET Meiji University"  4766   6   2729  1692  12     3970812   0         0     1460  1455.043  70.74226  0  0.480045  0.009073598  0.03994242   110.2097  160359.9  0.2345623    1           1  1  1  1  1    1  0     4198402
[stefan@wurble results]$ tail -n 13 ~/results/annoloc2_flows.txt | tcol | xclip -i
[stefan@wurble results]$ xclip -o
B  129   0x0400000200004001  1022171701.710399000  1022171726.638720000  24.928321000  1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800    19.67.132.240;19.67.132.229  us  "!Ford Motor Company"               27005  138.212.185.98   jp  "!ASAHI KASEI CORPORATION"    27025  17  1782  1208  0      65571     153934    28    61    36.7963   6.098478  0  0.071746  0.01398895   0.009389925  71.48495  2630.382  0.1919732    -0.4025558  1  1  1  1  1    1  0     4198402
A  1109  0x0400000200004000  1022171701.919937000  1022171726.638720000  24.718783000  1  3  eth:ipv4:icmp  00:d0:02:6d:78:00  00:00:1c:b6:1a:53  0x0800    193.107.159.17               at  "AT-WESTNET"                        0      138.212.184.165  jp  "!ASAHI KASEI CORPORATION"    0      1   104   0     0      2912      0         28    28    28        0         0  0.249289  0.2376806    0.03044914   4.207327  117.8052  1            1           1  1  1  1  1    1  104   2912
A  324   0x0400000200004000  1022171701.712093000  1022171726.638722000  24.926629000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:bf:08:44:81  0x0800    19.59.134.250                us  "!Ford Motor Company"               65230  138.212.187.240  jp  "!ASAHI KASEI CORPORATION"    58290  6   9459  5223  0      13696632  0         1448  1448  1448      0         0  0.067445  0.00263523   0.006631293  379.4737  549477.9  0.2885166    1           1  2  2  2  2    1  9459  13696632
B  324   0x0400000000004001  1022171701.713111000  1022171726.639230000  24.926119000  1  3  eth:ipv4:tcp   00:50:bf:08:44:81  00:d0:02:6d:78:00  0x0800    138.212.187.240              jp  "!ASAHI KASEI CORPORATION"          58290  19.59.134.250    us  "!Ford Motor Company"         65230  6   5223  9459  0      0         13696632  0     0     0         0         0  0.066113  0.004772384  0.008416669  209.5392  0         -0.2885166   -1          2  1  1  1  0.5  1  0     4198402
A  473   0x0400000000004000  1022171701.723484000  1022171726.638724000  24.915240000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:50:da:68:7f:84  0x0800    216.83.48.236                us  "IPXO"                              13600  138.212.188.139  jp  "!ASAHI KASEI CORPORATION"    20     6   864   1295  5184   0         1767728   0     0     0         0         0  0.068622  0.02883708   0.01269573   34.67757  0         -0.1996295   -1          1  1  2  2  2    1  864   0
B  473   0x0400000200004001  1022171701.724443000  1022171726.640395000  24.915952000  1  3  eth:ipv4:tcp   00:50:da:68:7f:84  00:d0:02:6d:78:00  0x0800    138.212.188.139              jp  "!ASAHI KASEI CORPORATION"          20     216.83.48.236    us  "IPXO"                        13600  6   1295  864   0      1767728   0         1176  1460  1365.041  134.3699  0  0.112955  0.01924014   0.02731735   51.97474  70947.64  0.1996295    1           1  1  1  1  1    1  0     4198402
A  703   0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:10:a7:02:4d:33  0x0800    70.98.46.15                  us  "ABUL-14-7385"                      6699   138.212.185.188  jp  "!ASAHI KASEI CORPORATION"    2603   6   32    28    0      33280     96        666   1414  1040      401.9105  0  2.251048  0.777589     0.9006743    1.286026  1337.468  0.06666667   0.9942474   1  2  2  2  2    1  32    33280
B  703   0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  1  3  eth:ipv4:tcp   00:10:a7:02:4d:33  00:d0:02:6d:78:00  0x0800    138.212.185.188              jp  "!ASAHI KASEI CORPORATION"          2603   70.98.46.15      us  "ABUL-14-7385"                6699   6   28    32    96     96        33280     0     8     3.428571  4.210036  0  2.292653  0.88349      0.8520955    1.131875  3.880713  -0.06666667  -0.9942474  2  1  1  1  0.5  1  0     4198402
A  9149  0x0400000200004000  1022171726.638730000  1022171726.638730000  0.000000000   1  3  eth:ipv4:udp   00:d0:02:6d:78:00  00:04:75:85:fd:a2  0x0800    192.50.41.136                jp  "!Toyohashi University of Technol"  64251  138.212.190.218  jp  "!ASAHI KASEI CORPORATION"    1112   17  1     0     12     6         0         6     6     6         0         0  0         0            0            0         0         1            1           1  1  1  1  1    1  6     36
A  21    0x0400000200004000  1022171701.691707000  1022171726.638737000  24.947030000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:00:e8:87:02:d1  0x0800    138.212.226.85               jp  "!ASAHI KASEI CORPORATION"          1103   138.212.185.72   jp  "!ASAHI KASEI CORPORATION"    445    6   543   985   2868   1701      1407845   0     63    3.132597  13.76817  0  0.589     0.04594295   0.03578168   21.76612  68.18447  -0.289267    -0.9975864  1  1  2  2  2    1  5762  20820
B  21    0x0400000200004001  1022171701.692758000  1022171726.640391000  24.947633000  1  3  eth:ipv4:tcp   00:00:e8:87:02:d1  00:d0:02:6d:78:00  0x0800    138.212.185.72               jp  "!ASAHI KASEI CORPORATION"          445    138.212.226.85   jp  "!ASAHI KASEI CORPORATION"    1103   6   985   543   0      1407845   1701      183   1460  1429.284  191.0024  0  0.889418  0.02532757   0.0522222    39.4827   56432     0.289267     0.9975864   1  1  1  1  1    1  0     4198402
A  1035  0x0400000000004000  1022171701.876636000  1022171726.639226000  24.762590000  1  3  eth:ipv4:tcp   00:d0:02:6d:78:00  00:01:02:b4:36:56  0x0800    133.26.84.187                jp  "MEIJI-NET Meiji University"        4766   138.212.187.109  jp  "!ASAHI KASEI CORPORATION"    80     6   1692  2729  10152  0         3970812   0     0     0         0         0  0.614191  0.01463511   0.04707645   68.32888  0         -0.2345623   -1          1  1  2  2  2    1  8168  0
B  1035  0x0400000200004001  1022171701.877349000  1022171726.639232000  24.761883000  1  3  eth:ipv4:tcp   00:01:02:b4:36:56  00:d0:02:6d:78:00  0x0800    138.212.187.109              jp  "!ASAHI KASEI CORPORATION"          80     133.26.84.187    jp  "MEIJI-NET Meiji University"  4766   6   2729  1692  12     3970812   0         0     1460  1455.043  70.74226  0  0.480045  0.009073598  0.03994242   110.2097  160359.9  0.2345623    1           1  1  1  1  1    1  0     4198402

The first record contains two IP addresses aggregated into one flow. Now try to set the aggregation mask for SRCIP4CMSK and DSTIP4CMSK to 8 using t2conf. Recompile and rerun t2 as home work and see how many IP addresses are now aggregated in one flow.

Port flow aggregation

It serves a good purpose if you are interested in flow reduction with a specific statistically questions on your mind and not being interested in the actual flow content. So don’t load any L7 plugins in carving mode as the state machines get corrupted, as different packets from different six-tuple flows are aggregated.

It can be useful for all plugins which are NOT initiated by ports, all statistical plugins, httpSniffer.

In order to enable the port aggregation mode the following configs are equivalent:

t2conf tranalyzer2 -D AGGREGATIONFLAG="(SRCPORT | DSTPORT)"

or in HEX

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x06

Then the constants listed below define the range of ports which be aggregated into port class 1, the rest is shoved into port class 0. The default class 1 is defined by the standardized ports. But do not invoke the commands yet, lets look into the tranalyzer.h file first.

tranalyzer2

vi src/tranalyzer.h

...
/* -------------------------------------------------------------------------- */
/* -------------------- DO NOT EDIT THE FOLLOWING Aggregation Modes --------- */
/* -------------------------------------------------------------------------- */

// Aggregation modes
#define L4PROT  0x01
#define DSTPORT 0x02
#define SRCPORT 0x04
#define DSTIP   0x08
#define SRCIP   0x10
#define VLANID  0x20
#define SUBNET  0x80

// SUBNET mode: IP flow aggregation network masks
#define CNTRY_MSK 0xff800000
#define TOR_MSK   0x00400000
#define ORG_MSK   0x003fffff

#define NETIDMSK  (CNTRY_MSK | ORG_MSK) // netID mask


/* -------------------------------------------------------------------------- */
/* -------------------------------------------------------------------------- */
/* -------------------------------------------------------------------------- */

// Flow Aggregation
#define AGGREGATIONFLAG 0x18 // each bit: 1: aggregation activated
                             // (see aggregation modes defined above)
...
#define SRCPORTLW 1     // src port lower bound
#define SRCPORTHW 1024  // src port upper bound
#define DSTPORTLW 1     // dst port lower bound
#define DSTPORTHW 1024  // dst port upper bound
...

You see, the AGGREGATIONFLAG is 0x18 from the last chapter. Close the file now and change the upper range to 23, aka Telnet. So we throw all services into class 1, which should not appear in modern networks anymore. Then switch on the appropriate aggregation mode, recompile the whole core + loaded plugins and rerun t2.

t2conf tranalyzer2 -D SRCPORTHW=23 -D DSTPORTHW=23 -D AGGREGATIONFLAG=0x06 && t2build -R and for yikes check it: t2conf tranalyzer2 -G SRCPORTHW=23 -G DSTPORTHW=23 -G AGGREGATIONFLAG

SRCPORTHW = 23
DSTPORTHW = 23
AGGREGATIONFLAG = 0x06

Now invoke t2 on the pcap

t2 -r ~/data/annoloc2.pcap -w ~/results/

<pre><samp>
...
--------------------------------------------------------------------------------
<span class="code-bold">basicStats:</span> Flow max(pktload): 1460 (1.46 K)
<span class="code-bold">basicStats:</span> Flow max(b/s), pkts: 4712000000 (4.71 Gb/s), 2
<span class="code-bold">basicStats:</span> Biggest L2 flow talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
<span class="code-bold">basicStats:</span> Biggest L2 flow talker: 00:d0:02:6d:78:00: 2622 (2.62 K) [0.00%] bytes
<span class="code-bold">basicStats:</span> Biggest L3 flow talker: 139.45.174.202 (US): 30343 (30.34 K) [2.49%] packets
<span class="code-bold">basicStats:</span> Biggest L3 flow talker: 139.45.174.202 (US): 44264308 (44.26 M) [69.07%] bytes
<span class="code-bold">connStat:</span> Number of unique source IPs: 3655 (3.65 K)
<span class="code-bold">connStat:</span> Number of unique destination IPs: 3200 (3.20 K)
<span class="code-bold">connStat:</span> Number of unique source/destination IPs connections: 4
<span class="code-bold">connStat:</span> Max unique number of source IP / destination port connections: 586
<span class="code-bold">connStat:</span> IP connF=connSipDprt/connSip: 0.160328
<span class="code-bold">connStat:</span> IP connG=connSipDprt/connSipDip: 146.500000
<span class="code-bold">connStat:</span> Source IP with max connections: 138.212.189.66 (JP): 369 connections
<span class="code-bold">connStat:</span> Destination IP with max connections: 138.212.184.235 (JP): 400 connections
<span class="code-bold">connStat:</span> Biggest L3 talker: 138.212.189.38 (JP): 33706 (33.71 K) [2.77%] packets
<span class="code-bold">connStat:</span> Biggest L3 talker: 138.212.189.38 (JP): 48279870 (48.28 M) [75.34%] bytes
--------------------------------------------------------------------------------
Headers count: min: 2, max: 5, avg: 3.01
Number of ARP packets: 247 [0.02%]
Number of GRE packets: 20 [0.00%]
Number of IGMP packets: 12 [0.00%]
Number of ICMP packets: 3059 (3.06 K) [0.25%]
Number of ICMPv6 packets: 11 [0.00%]
Number of TCP packets: 948743 (948.74 K) [77.83%]
Number of TCP bytes: 52643546 (52.64 M) [82.15%]
Number of UDP packets: 266900 (266.90 K) [21.89%]
Number of UDP bytes: 11234272 (11.23 M) [17.53%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 12971 (12.97 K)
Number of processed L2   flows: 99 [0.76%]
Number of processed IPv4 flows: 12810 (12.81 K) [98.76%]
Number of processed IPv6 flows: 62 [0.48%]
Number of processed A    flows: 7518 (7.52 K) [57.96%]
Number of processed B    flows: 5453 (5.45 K) [42.04%]
Number of request        flows: 7517 (7.52 K) [57.95%]
Number of reply          flows: 5454 (5.45 K) [42.05%]
Total   A/B    flow asymmetry: 0.16
Total req/rply flow asymmetry: 0.16
Number of processed A+B packets/A+B flows: 93.98
Number of processed A   packets/A   flows: 82.26
Number of processed   B packets/  B flows: 110.14
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B   packets/s: 48859.83 (48.86 K)
Number of processed A     packets/s: 24786.46 (24.79 K)
Number of processed   B   packets/s: 24073.37 (24.07 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
...

See! Less flows, because now we aggregate all non port 23 flows into one. Let’s find the first 10 biggest talkers using unencrypted legacy services, you do not want to use anymore in your corporate network:

tawk '$srcPort == 1 { print $srcIP, $pktsSnt }' ~/results/annoloc2_flows.txt | sort -nr -k2 | head -n 10

138.212.185.150	8696
138.212.190.31	5276
138.212.188.251	5107
138.212.185.102	4056
138.212.186.210	3259
138.212.190.31	3080
193.86.146.215	2885
138.212.188.139	2466
138.212.184.244	2273
138.212.187.170	2156

Oups, that is bad news.

Protocol flow aggregation

The following configs are equivalent

t2conf tranalyzer2 -D AGGREGATIONFLAG=L4PROT

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x01

Enable protocol flow aggregation, recompile and rerun t2 on the pcap.

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x01 && t2build -R

t2conf tranalyzer2 -G AGGREGATIONFLAG

AGGREGATIONFLAG = 0x01
t2 -r ~/data/annoloc2.pcap -w ~/results

...
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 1472 (1.47 K)
basicStats: Flow max(b/s), pkts: 4712000000 (4.71 Gb/s), 2
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 2622 (2.62 K) [0.00%] bytes
basicStats: Biggest L3 flow talker: 138.212.189.38 (JP): 23601 (23.60 K) [1.94%] packets
basicStats: Biggest L3 flow talker: 138.212.189.38 (JP): 33731054 (33.73 M) [52.64%] bytes
connStat: Number of unique source IPs: 3774 (3.77 K)
connStat: Number of unique destination IPs: 3090 (3.09 K)
connStat: Number of unique source/destination IPs connections: 182
connStat: Max unique number of source IP / destination port connections: 413
connStat: IP connF=connSipDprt/connSip: 0.109433
connStat: IP connG=connSipDprt/connSipDip: 2.269231
connStat: Source IP with max connections: 138.212.189.66 (JP): 368 connections
connStat: Destination IP with max connections: 138.212.184.235 (JP): 402 connections
connStat: Biggest L3 talker: 138.212.189.38 (JP): 33706 (33.71 K) [2.77%] packets
connStat: Biggest L3 talker: 138.212.189.38 (JP): 48279870 (48.28 M) [75.34%] bytes
--------------------------------------------------------------------------------
Headers count: min: 2, max: 5, avg: 3.01
Number of ARP packets: 247 [0.02%]
Number of GRE packets: 20 [0.00%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 17098 (17.10 K)
Number of processed L2   flows: 99 [0.58%]
Number of processed IPv4 flows: 16939 (16.94 K) [99.07%]
Number of processed IPv6 flows: 60 [0.35%]
Number of processed A    flows: 9718 (9.72 K) [56.84%]
Number of processed B    flows: 7380 (7.38 K) [43.16%]
Number of request        flows: 9675 (9.68 K) [56.59%]
Number of reply          flows: 7423 (7.42 K) [43.41%]
Total   A/B    flow asymmetry: 0.14
Total req/rply flow asymmetry: 0.13
Number of processed A+B packets/A+B flows: 71.30
Number of processed A   packets/A   flows: 58.06
Number of processed   B packets/  B flows: 88.72
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B   packets/s: 48859.83 (48.86 K)
Number of processed A     packets/s: 22615.09 (22.61 K)
Number of processed   B   packets/s: 26244.74 (26.24 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

Now the protocol field is forced to 0 in the hash, and all flows are aggregated independently of the l4proto field of the IP header.

tawk 't2sort(pktsSnt, 10)' ~/results/annoloc2_flows.txt | tcol

%dir  flowInd  flowStat            timeFirst             timeLast              duration      numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP            srcIPCC  srcIPOrg                            srcPort  dstIP            dstIPCC  dstIPOrg                            dstPort  l4Proto  pktsSnt  pktsRcvd  padBytesSnt  l7BytesSnt  l7BytesRcvd  minL7PktSz  maxL7PktSz  avgL7PktSz  stdL7PktSz  minIAT  maxIAT    avgIAT        stdIAT       pktps     bytps     pktAsm     bytAsm      connSip  connDip  connSipDip  connSipDprt  connF  connG      connNumPCnt  connNumBCnt
B     91       0x0400000200004001  1022171701.699480000  1022171726.636773000  24.937293000  1           3        eth:ipv4:tcp  00:00:21:d2:cc:72  00:d0:02:6d:78:00  0x0800           138.212.189.38   jp       "!ASAHI KASEI CORPORATION"          139      138.212.86.201   jp       "AKNWS-NET Asahi Kasei Networks C"  3429     0        23601    12342     0            33731054    42462        6           1460        1429.221    189.2884    0       0.253336  0.001056625   0.003716458  946.4139  1352635   0.313246   0.9974855   1        1        1           1            1      1          33706        48279870
A     91       0x0400000a00004000  1022171701.699996000  1022171726.637210000  24.937214000  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:21:d2:cc:72  0x0800           138.212.86.201   jp       "AKNWS-NET Asahi Kasei Networks C"  3429     138.212.189.38   jp       "!ASAHI KASEI CORPORATION"          139      0        12342    23601     68670        42462       33731054     0           63          3.440447    14.32136    0       0.36365   0.002020519   0.005329602  494.923   1702.756  -0.313246  -0.9974855  1        1        2           2            2      1          12342        42462
B     6227     0x0400000200004001  1022171714.045827000  1022171722.457644000  8.411817000   1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:10:5a:c5:96:1a  0x0800           139.45.174.202   us       "STRIPE-139-45-174-0-24"            56071    138.212.190.117  jp       "!ASAHI KASEI CORPORATION"          3837     0        10159    5692      6            14821880    0            0           1460        1458.99     32.96766    0       1.465593  0.0008280156  0.01485064   1207.706  1762031   0.2818119  1           1        2        3           1            1      0.3333333  30343        44264308
B     3583     0x0400000200004001  1022171705.686717000  1022171714.043794000  8.357077000   1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:10:5a:c5:96:1a  0x0800           139.45.174.202   us       "STRIPE-139-45-174-0-24"            56070    138.212.190.117  jp       "!ASAHI KASEI CORPORATION"          3820     0        10048    5709      6            14656900    0            0           1460        1458.688    37.40672    0       1.39519   0.0008317156  0.01430882   1202.334  1753831   0.2753697  1           1        2        5           1            1      0.2        30343        44264308
A     327      0x0400000200004000  1022171701.712093000  1022171726.638722000  24.926629000  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:50:bf:08:44:81  0x0800           19.59.134.250    us       "!Ford Motor Company"               65230    138.212.187.240  jp       "!ASAHI KASEI CORPORATION"          58290    0        9459     5223      0            13696632    0            1448        1448        1448        0           0       0.067445  0.00263523    0.006631293  379.4737  549477.9  0.2885166  1           1        1        2           2            2      1          9459         13696632
B     69       0x0400000200004001  1022171701.698940000  1022171726.629403000  24.930463000  1           3        eth:ipv4:tcp  00:50:fc:23:7a:c0  00:d0:02:6d:78:00  0x0800           138.212.187.219  jp       "!ASAHI KASEI CORPORATION"          139      138.212.36.145   jp       "AKNWS-NET Asahi Kasei Networks C"  2860     0        8978     4413      0            12814184    25156        68          1460        1427.287    188.1383    0       0.070768  0.002776838   0.005433218  360.1217  513997    0.3409006  0.9960814   1        2        1           3            3      3          0            4198402
B     77       0x0400000200004001  1022171701.699040000  1022171726.629407000  24.930367000  1           3        eth:ipv4:tcp  00:48:54:63:7b:6c  00:d0:02:6d:78:00  0x0800           138.212.190.224  jp       "!ASAHI KASEI CORPORATION"          139      138.212.36.145   jp       "AKNWS-NET Asahi Kasei Networks C"  2861     0        7319     3622      0            10446036    20736        39          1460        1427.249    188.6036    0       0.131045  0.003406252   0.006908146  293.5777  419008.5  0.3379033  0.9960377   1        1        1           1            1      1          0            4198402
B     31       0x0400000200004001  1022171701.715914000  1022171726.608383000  24.892469000  1           3        eth:ipv4:tcp  00:01:02:b4:36:56  00:d0:02:6d:78:00  0x0800           138.212.187.109  jp       "!ASAHI KASEI CORPORATION"          139      138.212.77.73    jp       "AKNWS-NET Asahi Kasei Networks C"  61340    0        7289     3811      114          10398387    10773        0           1460        1426.586    205.6423    0       0.204505  0.003415072   0.01081335   292.8195  417732.2  0.3133333  0.9979301   4        2        1           3            0.75   3          23691        33466670
A     4781     0x0400000200004000  1022171709.260746000  1022171716.327308000  7.066562000   1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:b4:a9:15:71  0x0800           133.26.75.121    jp       "MEIJI-NET Meiji University"        36237    138.212.185.150  jp       "!ASAHI KASEI CORPORATION"          20       0        6865     4465      24           10008944    0            0           1460        1457.967    53.03892    0       0.404642  0.00102936    0.009474586  971.4766  1416381   0.211827   1           1        3        6           6            6      1          13412        19470658
B     8561     0x0400000200004001  1022171722.458182000  1022171726.637621000  4.179439000   1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:10:5a:c5:96:1a  0x0800           139.45.174.202   us       "STRIPE-139-45-174-0-24"            56072    138.212.190.117  jp       "!ASAHI KASEI CORPORATION"          3854     0        6044     3358      0            8817920     0            0           1460        1458.954    33.76375    0       0.145115  0.0006915019  0.00266886   1446.127  2109833   0.2856839  1           1        1        1           1            1      1          30343        44264308

VLAN flow aggregation

If you acquire your traffic from an LNS or on a trunk port the VLANs must be integrated in the flow hash as different VLANs with the same five tuple should be separated. If your traffic is stripped of VLANs, then you can ignore the VLAN ID, as it will always be 0.

Notwithstanding, it might be interesting to aggregate all VLANs with the same five tuple in one flow. One interesting incident, where T2 came to the rescue was a case where egress/ingress traffic was separated in two different VLANs; an accidental misconfiguration.

So the VLAN mode produced perfect flows and when I switched VLAN aggregation off I had twice the amount of flows, and that made me go to the customer and ask a question like: WTF? And he answered: WTF! And then we found a lot of WTF stuff including illegal access…

First, reset the aggregation mode:

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x00 && t2build -R

t2conf tranalyzer2 -G AGGREGATIONFLAG

AGGREGATIONFLAG = 0x00

Then, feed the 802.1Q_tunneling.cap PCAP file to T2 in order to see how the normal flow output looks like:

t2 -r ~/data/802.1Q_tunneling.cap -w ~/results

================================================================================
Tranalyzer 0.9.4 (Anteater), Cobra. PID: 47672, Prio: 0, SID: 666
================================================================================
Date: 1751992414.000874954 sec (Tue 08 Jul 2025 18:33:34 CEST)
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.4
    02: basicStats, 0.9.4
    03: connStat, 0.9.4
    04: txtSink, 0.9.4
[INF] IPv4 Ver: 6, Rev: 02072025, Range Mode: 0, subnet ranges loaded: 7237971 (7.24 M)
[INF] IPv6 Ver: 6, Rev: 02072025, Range Mode: 0, subnet ranges loaded: 1419101 (1.42 M)
Processing file: /home/user/data/802.1Q_tunneling.cap
Link layer type: Ethernet [EN10MB/1]
Snapshot length: 65535 (65.53 K)
Dump start: 1277840495.135052000 sec (Tue 29 Jun 2010 19:41:35 GMT)
Dump stop : 1277840530.538713000 sec (Tue 29 Jun 2010 19:42:10 GMT)
Total dump duration: 35.403661000 sec
Finished processing. Elapsed time: 0.000796337 sec
Finished unloading flow memory. Time: 0.000930254 sec
Percentage completed: 100.00%
Number of processed packets: 26
Number of processed bytes: 4686 (4.69 K)
Number of raw bytes: 4686 (4.69 K)
Number of pcap bytes: 5126 (5.13 K)
Number of L2 packets: 6 [23.08%]
Number of IPv4 packets: 20 [76.92%]
Number of A packets: 16 [61.54%]
Number of B packets: 10 [38.46%]
Number of A bytes: 3466 (3.47 K) [73.97%]
Number of B bytes: 1220 (1.22 K) [26.03%]
<A packet load>: 216.62
<B packet load>: 122.00
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 353
basicStats: Flow max(b/s), pkts: 483870 (483.87 Kb/s), 5
basicStats: Biggest L2 flow talker: 00:13:c3:df:ae:18: 1 [3.85%] packets
basicStats: Biggest L2 flow talker: 00:0f:34:5f:16:8d: 353 [7.53%] bytes
basicStats: Biggest L3 flow talker: 10.118.10.1: 5 [19.23%] packets
basicStats: Biggest L3 flow talker: 10.118.10.1: 360 [7.68%] bytes
connStat: Number of unique source IPs: 2
connStat: Number of unique destination IPs: 2
connStat: Number of unique source/destination IPs connections: 2
connStat: Max unique number of source IP / destination port connections: 2
connStat: IP connF=connSipDprt/connSip: 1.000000
connStat: IP connG=connSipDprt/connSipDip: 1.000000
connStat: Source IP with max connections: 10.118.10.1: 1 connections
connStat: Destination IP with max connections: 10.118.10.2: 1 connections
connStat: Biggest L3 talker: 10.118.10.1: 5 [19.23%] packets
connStat: Biggest L3 talker: 10.118.10.1: 360 [7.68%] bytes
--------------------------------------------------------------------------------
Headers count: min: 3, max: 5, avg: 4.69
Max VLAN header count: 2
Number of LLC packets: 6 [23.08%]
Number of ICMP packets: 20 [76.92%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 10
Number of processed L2   flows: 6 [60.00%]
Number of processed IPv4 flows: 4 [40.00%]
Number of processed A    flows: 8 [80.00%]
Number of processed B    flows: 2 [20.00%]
Number of request        flows: 8 [80.00%]
Number of reply          flows: 2 [20.00%]
Total   A/B    flow asymmetry: 0.60
Total req/rply flow asymmetry: 0.60
Number of processed A+B packets/A+B flows: 2.60
Number of processed A   packets/A   flows: 2.00
Number of processed   B packets/  B flows: 5.00
Number of processed total packets/s: 0.73
Number of processed A+B   packets/s: 0.73
Number of processed A     packets/s: 0.45
Number of processed   B   packets/s: 0.28
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<Number of processed flows/s>: 0.28
<Bandwidth>: 551 b/s
<Raw bandwidth>: 1059 b/s (1.06 Kb/s)
Max number of flows in memory: 10 [0.00%]
Memory usage: 0.04 GB [0.06%]
Aggregated flowStat=0x0400000000004104
[INF] Layer 2 flows
[INF] IPv4 flows
[INF] VLAN encapsulation

Max VLAN header count: 2… Hmmm, that means, the packets are encapsulated maximal in two VLANs. Right, let’s look at the flows:

tcol ~/results/802.1Q_tunneling_flows.txt

%dir  flowInd  flowStat            timeFirst             timeLast              duration     numHdrDesc  numHdrs  hdrDesc                srcMac             dstMac             ethType  vlanID  srcIP        srcIPCC  srcIPOrg            srcPort  dstIP        dstIPCC  dstIPOrg            dstPort  l4Proto  pktsSnt  pktsRcvd  padBytesSnt  l7BytesSnt  l7BytesRcvd  minL7PktSz  maxL7PktSz  avgL7PktSz  stdL7PktSz  minIAT  maxIAT    avgIAT     stdIAT        pktps     bytps     pktAsm  bytAsm  connSip  connDip  connSipDip  connSipDprt  connF  connG  connNumPCnt  connNumBCnt
A     1        0x0400000000004100  1277840495.135052000  1277840495.141708000  0.006656000  1           5        eth:vlan{2}:ipv4:icmp  00:13:c3:df:ae:18  00:1b:d4:1b:a4:d8  0x0800   118;10  10.118.10.1  04       "!Private network"  0        10.118.10.2  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.00188   0.0013312  0.0009233078  751.2019  54086.54  0       0       1        1        2           2            2      1      5            360
B     1        0x0400000000004101  1277840495.135910000  1277840495.142543000  0.006633000  1           5        eth:vlan{2}:ipv4:icmp  00:1b:d4:1b:a4:d8  00:13:c3:df:ae:18  0x0800   118;10  10.118.10.2  04       "!Private network"  0        10.118.10.1  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001721  0.0013266  0.0008943002  753.8067  54274.09  0       0       1        1        1           1            1      1      5            360
A     2        0x0400000000004100  1277840503.708352000  1277840503.714432000  0.006080000  1           5        eth:vlan{2}:ipv4:icmp  00:19:aa:7d:e6:88  00:21:55:c8:f1:3c  0x0800   209;20  10.209.20.3  04       "!Private network"  0        10.209.20.4  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001733  0.001216   0.0008592694  822.3684  59210.52  0       0       1        1        2           2            2      1      5            360
B     2        0x0400000000004101  1277840503.709181000  1277840503.715133000  0.005952000  1           5        eth:vlan{2}:ipv4:icmp  00:21:55:c8:f1:3c  00:19:aa:7d:e6:88  0x0800   209;20  10.209.20.4  04       "!Private network"  0        10.209.20.3  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001666  0.0011904  0.0007853201  840.0538  60483.87  0       0       1        1        1           1            1      1      5            360
A     3        0x0000000000000104  1277840510.969363000  1277840510.969363000  0.000000000  1           4        eth:vlan:llc:cdp       00:13:c3:df:ae:18  01:00:0c:cd:cd:d0  0x2000   41078   -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            349         0            349         349         349         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     4        0x0000000000000104  1277840511.384783000  1277840511.384783000  0.000000000  1           4        eth:vlan:llc:cdp       00:19:aa:7d:e6:88  01:00:0c:cd:cd:d0  0x2000   41169   -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            347         0            347         347         347         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     5        0x0000000000000004  1277840525.369320000  1277840525.369320000  0.000000000  1           3        eth:llc:cdp            00:0f:34:5f:16:8d  01:00:0c:cc:cc:cc  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            353         0            353         353         353         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     6        0x0000000000000004  1277840525.404193000  1277840525.404193000  0.000000000  1           3        eth:llc:cdp            00:13:c4:12:0f:0d  01:00:0c:cc:cc:cc  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            353         0            353         353         353         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     7        0x0000000000000104  1277840528.106320000  1277840528.106320000  0.000000000  1           4        eth:vlan:llc:cdp       00:1b:d4:1b:a4:d8  01:00:0c:cd:cd:d0  0x2000   41078   -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            349         0            349         349         349         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     8        0x0000000000000104  1277840530.538713000  1277840530.538713000  0.000000000  1           4        eth:vlan:llc:cdp       00:21:55:c8:f1:3c  01:00:0c:cd:cd:d0  0x2000   41169   -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            347         0            347         347         347         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0

Now switch on the VLAN aggregation mode using either of the following configs, which are equivalent:

t2conf tranalyzer2 -D AGGREGATIONFLAG=VLANID

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x20

So make the flow aggregation independent of the VLAN ID.

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x20 && t2build -R

t2 -r ~/data/802.1Q_tunneling.cap -w ~/results/

...
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 353
basicStats: Flow max(b/s), pkts: 483870 (483.87 Kb/s), 5
basicStats: Biggest L2 flow talker: 00:13:c3:df:ae:18: 1 [3.85%] packets
basicStats: Biggest L2 flow talker: 00:0f:34:5f:16:8d: 353 [7.53%] bytes
basicStats: Biggest L3 flow talker: 10.118.10.1: 5 [19.23%] packets
basicStats: Biggest L3 flow talker: 10.118.10.1: 360 [7.68%] bytes
connStat: Number of unique source IPs: 2
connStat: Number of unique destination IPs: 2
connStat: Number of unique source/destination IPs connections: 2
connStat: Max unique number of source IP / destination port connections: 2
connStat: IP connF=connSipDprt/connSip: 1.000000
connStat: IP connG=connSipDprt/connSipDip: 1.000000
connStat: Source IP with max connections: 10.118.10.1: 1 connections
connStat: Destination IP with max connections: 10.118.10.2: 1 connections
connStat: Biggest L3 talker: 10.118.10.1: 5 [19.23%] packets
connStat: Biggest L3 talker: 10.118.10.1: 360 [7.68%] bytes
--------------------------------------------------------------------------------
Headers count: min: 3, max: 5, avg: 4.69
Max VLAN header count: 2
Number of LLC packets: 6 [23.08%]
Number of ICMP packets: 20 [76.92%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed      flows: 10
Number of processed L2   flows: 6 [60.00%]
Number of processed IPv4 flows: 4 [40.00%]
Number of processed A    flows: 8 [80.00%]
Number of processed B    flows: 2 [20.00%]
Number of request        flows: 8 [80.00%]
Number of reply          flows: 2 [20.00%]
Total   A/B    flow asymmetry: 0.60
Total req/rply flow asymmetry: 0.60
Number of processed A+B packets/A+B flows: 2.60
Number of processed A   packets/A   flows: 2.00
Number of processed   B packets/  B flows: 5.00
Number of processed total packets/s: 0.73
Number of processed A+B   packets/s: 0.73
Number of processed A     packets/s: 0.45
Number of processed   B   packets/s: 0.28
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

Damn! But why do we have the same amount of flows? Is T2 broken? Or did we take something not into account? Ahhh, the VLANs and IPs are always different. If you look now at the ethVlanID it is empty, aka not used in the hash. The Ethernet flows have different srcMac, so they stay separate as well. Oups, …

tcol ~/results/802.1Q_tunneling_flows.txt

%dir  flowInd  flowStat            timeFirst             timeLast              duration     numHdrDesc  numHdrs  hdrDesc                srcMac             dstMac             ethType  vlanID  srcIP        srcIPCC  srcIPOrg            srcPort  dstIP        dstIPCC  dstIPOrg            dstPort  l4Proto  pktsSnt  pktsRcvd  padBytesSnt  l7BytesSnt  l7BytesRcvd  minL7PktSz  maxL7PktSz  avgL7PktSz  stdL7PktSz  minIAT  maxIAT    avgIAT     stdIAT        pktps     bytps     pktAsm  bytAsm  connSip  connDip  connSipDip  connSipDprt  connF  connG  connNumPCnt  connNumBCnt
A     1        0x0400000000004100  1277840495.135052000  1277840495.141708000  0.006656000  1           5        eth:vlan{2}:ipv4:icmp  00:13:c3:df:ae:18  00:1b:d4:1b:a4:d8  0x0800           10.118.10.1  04       "!Private network"  0        10.118.10.2  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.00188   0.0013312  0.0009233078  751.2019  54086.54  0       0       1        1        2           2            2      1      5            360
B     1        0x0400000000004101  1277840495.135910000  1277840495.142543000  0.006633000  1           5        eth:vlan{2}:ipv4:icmp  00:1b:d4:1b:a4:d8  00:13:c3:df:ae:18  0x0800           10.118.10.2  04       "!Private network"  0        10.118.10.1  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001721  0.0013266  0.0008943002  753.8067  54274.09  0       0       1        1        1           1            1      1      5            360
A     2        0x0400000000004100  1277840503.708352000  1277840503.714432000  0.006080000  1           5        eth:vlan{2}:ipv4:icmp  00:19:aa:7d:e6:88  00:21:55:c8:f1:3c  0x0800           10.209.20.3  04       "!Private network"  0        10.209.20.4  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001733  0.001216   0.0008592694  822.3684  59210.52  0       0       1        1        2           2            2      1      5            360
B     2        0x0400000000004101  1277840503.709181000  1277840503.715133000  0.005952000  1           5        eth:vlan{2}:ipv4:icmp  00:21:55:c8:f1:3c  00:19:aa:7d:e6:88  0x0800           10.209.20.4  04       "!Private network"  0        10.209.20.3  04       "!Private network"  0        1        5        5         0            360         360          72          72          72          0           0       0.001666  0.0011904  0.0007853201  840.0538  60483.87  0       0       1        1        1           1            1      1      5            360
A     3        0x0000000000000104  1277840510.969363000  1277840510.969363000  0.000000000  1           4        eth:vlan:llc:cdp       00:13:c3:df:ae:18  01:00:0c:cd:cd:d0  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            349         0            349         349         349         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     4        0x0000000000000104  1277840511.384783000  1277840511.384783000  0.000000000  1           4        eth:vlan:llc:cdp       00:19:aa:7d:e6:88  01:00:0c:cd:cd:d0  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            347         0            347         347         347         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     5        0x0000000000000004  1277840525.369320000  1277840525.369320000  0.000000000  1           3        eth:llc:cdp            00:0f:34:5f:16:8d  01:00:0c:cc:cc:cc  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            353         0            353         353         353         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     6        0x0000000000000004  1277840525.404193000  1277840525.404193000  0.000000000  1           3        eth:llc:cdp            00:13:c4:12:0f:0d  01:00:0c:cc:cc:cc  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            353         0            353         353         353         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     7        0x0000000000000104  1277840528.106320000  1277840528.106320000  0.000000000  1           4        eth:vlan:llc:cdp       00:1b:d4:1b:a4:d8  01:00:0c:cd:cd:d0  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            349         0            349         349         349         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0
A     8        0x0000000000000104  1277840530.538713000  1277840530.538713000  0.000000000  1           4        eth:vlan:llc:cdp       00:21:55:c8:f1:3c  01:00:0c:cd:cd:d0  0x2000           -            -        "-"                 0        -            -        "-"                 0        0        1        0         0            347         0            347         347         347         0           0       0         0          0             0         0         1       1       0        0        0           0            0      0      0            0

Try to add the network aggregation mode for srcIP and dstIP, how many flows do you expect now?

Subnet aggregation

For people who are interested in the big picture, being interested in flows between organizations or even bigger between countries, the SUBNET aggregation mode comes in handy. To achieve a maximal compression L4PROT, DSTPORT, SRCPORT are ignored as well in the flow hash.

So set the AGGREGATIONFLAG as follows (both commands are equivalent);

t2conf tranalyzer2 -D AGGREGATIONFLAG="(SUBNET | L4PROT | DSTPORT | SRCPORT)"

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x87

As indicated below, rebuild T2 and all active plugins and run it on annoloc2.pcap as it contains more organizations.

t2conf tranalyzer2 -D AGGREGATIONFLAG=0x87 && t2build -R

t2conf tranalyzer2 -G AGGREGATIONFLAG

AGGREGATIONFLAG = 0x87
t2 -r ~/data/annoloc2.pcap -w ~/results/

...
--------------------------------------------------------------------------------
basicStats: Flow max(pktload): 1460 (1.46 K)
basicStats: Flow max(b/s), pkts: 490666656 (490.67 Mb/s), 4
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
basicStats: Biggest L2 flow talker: 00:d0:02:6d:78:00: 2622 (2.62 K) [0.00%] bytes
basicStats: Biggest L3 flow talker: N/A (US): 68181 (68.18 K) [5.59%] packets
basicStats: Biggest L3 flow talker: N/A (JP): 87800789 (87.80 M) [137.01%] bytes
connStat: Number of unique source Nets: 713
connStat: Number of unique destination Nets: 545
connStat: Number of unique source/destination Net connections: 6
connStat: Max unique number of source Net / destination port connections: 1071 (1.07 K)
connStat: Net connF=connSipDprt/connSip: 1.502104
connStat: Net connG=connSipDprt/connSipDip: 178.500000
connStat: Source Net with max connections: N/A (JP): 539 connections
connStat: Destination Net with max connections: N/A (JP): 747 connections
connStat: Biggest Net talker: N/A (JP): 686348 (686.35 K) [56.30%] packets
connStat: Biggest Net talker: N/A (JP): 631042439 (631.04 M) [984.73%] bytes
--------------------------------------------------------------------------------
Headers count: min: 2, max: 5, avg: 3.01
Number of ARP packets: 247 [0.02%]
Number of GRE packets: 20 [0.00%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

Note that the amount of flows is now drastically reduced, IPbecomes Net and basicStats and connStat report only the country and not the IP anymore. L2 output stays the same as it is not aggregated according to country or organization.

Select the last 10 flows from the results directory and look at the IP columns. As flows are now aggregated into country and organizations, all the IPs covered by theses flow parameters are listed under srcIP and dstIP. Same for any other flow parameter such as VLAN ID or Ethernet address.

tail -n 10 ~/results/annoloc2_flows.txt | tcol

A  294  0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00                                      00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  0x0800    70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;70.96.180.209               us  "ABUL-14-7385"                      0  138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.187.4    jp  "!ASAHI KASEI CORPORATION"          0  0  358    592    1875    33379     767756    0   1414  93.23743  319.8525  0  0.848498  0.06950515    0.1386104     14.38742  1341.446  -0.2463158    -0.9166707  1  4  2  2  2          1  819     33379
B  294  0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.189.66   jp  "!ASAHI KASEI CORPORATION"          0  70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;216.187.138.121             us  "ABUL-14-7385"                      0  0  592    358    174     767756    33379     0   1452  1296.885  372.7865  0  0.843078  0.0417867     0.1191967     23.93106  31035.84  0.2463158     0.9166707   4  1  1  1  0.25       1  686348  631042439
A  25   0x0400000200004000  1022171701.692722000  1022171726.639225000  24.946503000  2  3;3      eth:ipv4:udp;eth:ipv4:icmp                                        00:d0:02:6d:78:00                                      00:10:a7:04:8c:e7;00:10:a7:02:4d:33                    0x0800    201.71.5.135;201.71.5.18                                                       br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  138.212.188.0;138.212.185.188                                                  jp  "!ASAHI KASEI CORPORATION"          0  0  1948   1521   4       64242     95562     15  81    32.97844  5.986069  0  0.091936  0.01280621    0.01253799    78.0871   2575.191  0.1230902     -0.1959901  1  3  2  2  2          1  1948    64242
B  25   0x0400000200004001  1022171701.699995000  1022171726.638731000  24.938736000  1  3        eth:ipv4:udp                                                      00:10:a7:02:4d:33;00:10:a7:04:8c:e7                    00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.0                                                  jp  "!ASAHI KASEI CORPORATION"          0  201.71.5.18;201.71.5.135                                                       br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  0  1521   1948   1883    95562     64242     2   1002  62.8284   48.25857  0  0.077883  0.01639627    0.01061916    60.98946  3831.87   -0.1230902    0.1959901   3  1  1  1  0.3333333  1  686348  631042439
A  26   0x0400000200004000  1022171701.692728000  1022171726.639232000  24.946504000  1  3        eth:ipv4:tcp                                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    00:d0:02:6d:78:00                                      0x0800    138.212.187.109;138.212.185.150                                                jp  "!ASAHI KASEI CORPORATION"          1  133.26.84.187;133.26.75.121                                                    jp  "MEIJI-NET Meiji University"        0  0  21733  21530  52272   18978704  19467995  0   1460  873.2667  715.4553  0  0.192009  0.001147856   0.003704329   871.1842  760776.1  0.004692231   -0.0127265  2  1  2  4  2          2  686348  631042439
B  26   0x0400000200004001  1022171701.700968000  1022171726.639226000  24.938258000  1  3        eth:ipv4:tcp                                                      00:d0:02:6d:78:00                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    0x0800    133.26.84.187;133.26.75.121                                                    jp  "MEIJI-NET Meiji University"        0  138.212.187.109;138.212.185.150                                                jp  "!ASAHI KASEI CORPORATION"          1  0  21530  21733  49080   19467995  18978704  0   1460  904.2265  708.7208  0  0.198618  0.001158304   0.003678098   863.3322  780647.8  -0.004692231  0.0127265   1  2  1  3  3          3  0       4198402
A  1    0x0400000200004000  1022171701.691172000  1022171726.640398000  24.949226000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:e0:4c:48:3a:a4;00:80:48:b3:21:69;00:4f:4e:02:84:25  00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:80:48:cd:85:0e  0x0800    138.212.185.53;138.212.188.131;138.212.188.165;138.212.179.166;138.212.185.72  jp  "!ASAHI KASEI CORPORATION"          1  138.212.213.164;138.212.233.169;138.212.233.135;138.212.232.65;138.212.226.85  jp  "!ASAHI KASEI CORPORATION"          0  0  46728  28494  1544    63806491  439407    0   1460  1365.487  293.3832  0  0.039757  0.000533925   0.0009780569  1872.924  2557454   0.2424025     0.9863211   1  2  2  2  2          1  686348  631042439
B  1    0x0400000a00004001  1022171701.691707000  1022171726.640388000  24.948681000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:48:54:7a:23:90  00:00:e8:87:02:d1;00:20:18:8a:fd:9b;00:e0:4c:41:eb:1b  0x0800    138.212.226.85;138.212.234.148;138.212.226.8;138.212.226.46;138.212.226.157    jp  "!ASAHI KASEI CORPORATION"          0  138.212.185.72;138.212.190.77;138.212.184.32;138.212.189.38;138.212.187.49     jp  "!ASAHI KASEI CORPORATION"          1  0  28494  46728  124271  439407    63806491  0   720   15.42104  31.61275  0  0.039945  0.000875577   0.001045347   1142.104  17612.43  -0.2424025    -0.9863211  2  1  1  1  0.5        1  686348  631042439
A  5    0x0401080a00005000  1022171701.691178000  1022171726.640398000  24.949220000  4  3;3;3;4  eth:ipv4:udp;eth:ipv4:tcp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x08a8)  00:d0:02:6d:78:00                                      00:50:04:2b:e6:2f;00:40:f4:21:e6:56;00:c0:26:2d:aa:4a  0x0800    19.32.78.208;19.132.119.69;19.67.220.218;19.6.61.6;19.49.74.170                us  "!Ford Motor Company"               0  138.212.184.93;138.212.188.21;138.212.187.94;138.212.190.146;138.212.186.231   jp  "!ASAHI KASEI CORPORATION"          0  0  68181  59416  85831   16813186  43551981  0   1460  246.5964  514.0296  0  0.038853  0.0003659278  0.0005407686  2732.791  673896.2  0.06869284    -0.4429507  1  1  2  2  2          1  77892   16914253
B  5    0x0401080a00005001  1022171701.691710000  1022171726.640389000  24.948679000  4  3;3;3;4  eth:ipv4:tcp;eth:ipv4:udp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x0051)  00:20:af:d2:17:09;00:c0:26:55:f7:dc;00:50:fc:20:14:df  00:d0:02:6d:78:00                                      0x0800    138.212.191.88;138.212.190.148;138.212.184.98;138.212.190.146;138.212.191.117  jp  "!ASAHI KASEI CORPORATION"          0  19.172.33.164;19.6.48.20;19.52.50.196;19.6.61.6;19.67.158.252                  us  "!Ford Motor Company"               0  0  59416  68181  18275   43551981  16813186  0   1460  733.0009  662.2905  0  0.039878  0.0004198974  0.0006543636  2381.529  1745663   -0.06869284   0.4429507   1  1  1  1  1          1  0       4198402

The length of each header in the flow file is defined in basicFlow.h below

basicFlow

vi src/basicFlow.h

...
// Maximum number of values to store

#define BFO_MAX_HDRDESC 4 // Maximum number of headers descriptions to store
#define BFO_MAX_MAC     3 // Maximum different MAC addresses to output
#define BFO_MAX_IP      5 // Maximum different IP addresses to output
#define BFO_MAX_MPLS    3 // Maximum MPLS headers/tags to output
#define BFO_MAX_VLAN    3 // Maximum VLAN headers/numbers to output
...

Let’s set the maximal IPs to 10 and see what happens.

t2conf basicFlow -D BFO_MAX_IP=10 && t2build basicFlow

t2 -r ~/data/annoloc2.pcap -w ~/results/

Now the list of IPs is longer

tail -n 10 ~/results/annoloc2_flows.txt | tcol

A  294  0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00                                      00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  0x0800    70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;70.96.180.209               us  "ABUL-14-7385"                      0  138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.187.4    jp  "!ASAHI KASEI CORPORATION"          0  0  358    592    1875    33379     767756    0   1414  93.23743  319.8525  0  0.848498  0.06950515    0.1386104     14.38742  1341.446  -0.2463158    -0.9166707  1  4  2  2  2          1  819     33379
B  294  0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.189.66   jp  "!ASAHI KASEI CORPORATION"          0  70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;216.187.138.121             us  "ABUL-14-7385"                      0  0  592    358    174     767756    33379     0   1452  1296.885  372.7865  0  0.843078  0.0417867     0.1191967     23.93106  31035.84  0.2463158     0.9166707   4  1  1  1  0.25       1  686348  631042439
A  25   0x0400000200004000  1022171701.692722000  1022171726.639225000  24.946503000  2  3;3      eth:ipv4:udp;eth:ipv4:icmp                                        00:d0:02:6d:78:00                                      00:10:a7:04:8c:e7;00:10:a7:02:4d:33                    0x0800    201.71.5.135;201.71.5.18                                                       br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  138.212.188.0;138.212.185.188                                                  jp  "!ASAHI KASEI CORPORATION"          0  0  1948   1521   4       64242     95562     15  81    32.97844  5.986069  0  0.091936  0.01280621    0.01253799    78.0871   2575.191  0.1230902     -0.1959901  1  3  2  2  2          1  1948    64242
B  25   0x0400000200004001  1022171701.699995000  1022171726.638731000  24.938736000  1  3        eth:ipv4:udp                                                      00:10:a7:02:4d:33;00:10:a7:04:8c:e7                    00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.0                                                  jp  "!ASAHI KASEI CORPORATION"          0  201.71.5.18;201.71.5.135                                                       br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  0  1521   1948   1883    95562     64242     2   1002  62.8284   48.25857  0  0.077883  0.01639627    0.01061916    60.98946  3831.87   -0.1230902    0.1959901   3  1  1  1  0.3333333  1  686348  631042439
A  26   0x0400000200004000  1022171701.692728000  1022171726.639232000  24.946504000  1  3        eth:ipv4:tcp                                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    00:d0:02:6d:78:00                                      0x0800    138.212.187.109;138.212.185.150                                                jp  "!ASAHI KASEI CORPORATION"          1  133.26.84.187;133.26.75.121                                                    jp  "MEIJI-NET Meiji University"        0  0  21733  21530  52272   18978704  19467995  0   1460  873.2667  715.4553  0  0.192009  0.001147856   0.003704329   871.1842  760776.1  0.004692231   -0.0127265  2  1  2  4  2          2  686348  631042439
B  26   0x0400000200004001  1022171701.700968000  1022171726.639226000  24.938258000  1  3        eth:ipv4:tcp                                                      00:d0:02:6d:78:00                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    0x0800    133.26.84.187;133.26.75.121                                                    jp  "MEIJI-NET Meiji University"        0  138.212.187.109;138.212.185.150                                                jp  "!ASAHI KASEI CORPORATION"          1  0  21530  21733  49080   19467995  18978704  0   1460  904.2265  708.7208  0  0.198618  0.001158304   0.003678098   863.3322  780647.8  -0.004692231  0.0127265   1  2  1  3  3          3  0       4198402
A  1    0x0400000200004000  1022171701.691172000  1022171726.640398000  24.949226000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:e0:4c:48:3a:a4;00:80:48:b3:21:69;00:4f:4e:02:84:25  00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:80:48:cd:85:0e  0x0800    138.212.185.53;138.212.188.131;138.212.188.165;138.212.179.166;138.212.185.72  jp  "!ASAHI KASEI CORPORATION"          1  138.212.213.164;138.212.233.169;138.212.233.135;138.212.232.65;138.212.226.85  jp  "!ASAHI KASEI CORPORATION"          0  0  46728  28494  1544    63806491  439407    0   1460  1365.487  293.3832  0  0.039757  0.000533925   0.0009780569  1872.924  2557454   0.2424025     0.9863211   1  2  2  2  2          1  686348  631042439
B  1    0x0400000a00004001  1022171701.691707000  1022171726.640388000  24.948681000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:48:54:7a:23:90  00:00:e8:87:02:d1;00:20:18:8a:fd:9b;00:e0:4c:41:eb:1b  0x0800    138.212.226.85;138.212.234.148;138.212.226.8;138.212.226.46;138.212.226.157    jp  "!ASAHI KASEI CORPORATION"          0  138.212.185.72;138.212.190.77;138.212.184.32;138.212.189.38;138.212.187.49     jp  "!ASAHI KASEI CORPORATION"          1  0  28494  46728  124271  439407    63806491  0   720   15.42104  31.61275  0  0.039945  0.000875577   0.001045347   1142.104  17612.43  -0.2424025    -0.9863211  2  1  1  1  0.5        1  686348  631042439
A  5    0x0401080a00005000  1022171701.691178000  1022171726.640398000  24.949220000  4  3;3;3;4  eth:ipv4:udp;eth:ipv4:tcp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x08a8)  00:d0:02:6d:78:00                                      00:50:04:2b:e6:2f;00:40:f4:21:e6:56;00:c0:26:2d:aa:4a  0x0800    19.32.78.208;19.132.119.69;19.67.220.218;19.6.61.6;19.49.74.170                us  "!Ford Motor Company"               0  138.212.184.93;138.212.188.21;138.212.187.94;138.212.190.146;138.212.186.231   jp  "!ASAHI KASEI CORPORATION"          0  0  68181  59416  85831   16813186  43551981  0   1460  246.5964  514.0296  0  0.038853  0.0003659278  0.0005407686  2732.791  673896.2  0.06869284    -0.4429507  1  1  2  2  2          1  77892   16914253
B  5    0x0401080a00005001  1022171701.691710000  1022171726.640389000  24.948679000  4  3;3;3;4  eth:ipv4:tcp;eth:ipv4:udp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x0051)  00:20:af:d2:17:09;00:c0:26:55:f7:dc;00:50:fc:20:14:df  00:d0:02:6d:78:00                                      0x0800    138.212.191.88;138.212.190.148;138.212.184.98;138.212.190.146;138.212.191.117  jp  "!ASAHI KASEI CORPORATION"          0  19.172.33.164;19.6.48.20;19.52.50.196;19.6.61.6;19.67.158.252                  us  "!Ford Motor Company"               0  0  59416  68181  18275   43551981  16813186  0   1460  733.0009  662.2905  0  0.039878  0.0004198974  0.0006543636  2381.529  1745663   -0.06869284   0.4429507   1  1  1  1  1          1  0       4198402
[stefan@wurble results]$ tail -n 10 ~/results/annoloc2_flows.txt | tcol | xclip -i
[stefan@wurble results]$ xclip -o
A  294  0x0400000200004000  1022171701.755879000  1022171726.638725000  24.882846000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00                                      00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  0x0800    70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;70.96.180.209;70.99.40.231                                                                               us  "ABUL-14-7385"                      0  138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.187.4;138.212.186.208;138.212.189.232;138.212.191.219                                  jp  "!ASAHI KASEI CORPORATION"          0  0  358    592    1875    33379     767756    0   1414  93.23743  319.8525  0  0.848498  0.06950515    0.1386104     14.38742  1341.446  -0.2463158    -0.9166707  1  4  2  2  2          1  819     33379
B  294  0x0400000200004001  1022171701.902659000  1022171726.640383000  24.737724000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:10:a7:02:4d:33;00:10:5a:9a:7b:b2;00:00:1c:b6:16:3f  00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.76;138.212.185.186;138.212.186.19;138.212.189.66;138.212.187.4;138.212.186.208;138.212.189.232;138.212.191.219                  jp  "!ASAHI KASEI CORPORATION"          0  70.98.46.15;70.99.42.203;70.96.59.158;70.99.32.168;216.187.138.121;70.96.180.209;70.99.40.231                                                                us  "ABUL-14-7385"                      0  0  592    358    174     767756    33379     0   1452  1296.885  372.7865  0  0.843078  0.0417867     0.1191967     23.93106  31035.84  0.2463158     0.9166707   4  1  1  1  0.25       1  686348  631042439
A  25   0x0400000200004000  1022171701.692722000  1022171726.639225000  24.946503000  2  3;3      eth:ipv4:udp;eth:ipv4:icmp                                        00:d0:02:6d:78:00                                      00:10:a7:04:8c:e7;00:10:a7:02:4d:33                    0x0800    201.71.5.135;201.71.5.18                                                                                                                                    br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  138.212.188.0;138.212.185.188                                                                                                                                jp  "!ASAHI KASEI CORPORATION"          0  0  1948   1521   4       64242     95562     15  81    32.97844  5.986069  0  0.091936  0.01280621    0.01253799    78.0871   2575.191  0.1230902     -0.1959901  1  3  2  2  2          1  1948    64242
B  25   0x0400000200004001  1022171701.699995000  1022171726.638731000  24.938736000  1  3        eth:ipv4:udp                                                      00:10:a7:02:4d:33;00:10:a7:04:8c:e7                    00:d0:02:6d:78:00                                      0x0800    138.212.185.188;138.212.188.0                                                                                                                               jp  "!ASAHI KASEI CORPORATION"          0  201.71.5.18;201.71.5.135                                                                                                                                     br  "SIMONE BARBOSA DO NASCIMENTO SOU"  0  0  1521   1948   1883    95562     64242     2   1002  62.8284   48.25857  0  0.077883  0.01639627    0.01061916    60.98946  3831.87   -0.1230902    0.1959901   3  1  1  1  0.3333333  1  686348  631042439
A  26   0x0400000200004000  1022171701.692728000  1022171726.639232000  24.946504000  1  3        eth:ipv4:tcp                                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    00:d0:02:6d:78:00                                      0x0800    138.212.187.109;138.212.185.150                                                                                                                             jp  "!ASAHI KASEI CORPORATION"          1  133.26.84.187;133.26.75.121                                                                                                                                  jp  "MEIJI-NET Meiji University"        0  0  21733  21530  52272   18978704  19467995  0   1460  873.2667  715.4553  0  0.192009  0.001147856   0.003704329   871.1842  760776.1  0.004692231   -0.0127265  2  1  2  4  2          2  686348  631042439
B  26   0x0400000200004001  1022171701.700968000  1022171726.639226000  24.938258000  1  3        eth:ipv4:tcp                                                      00:d0:02:6d:78:00                                      00:01:02:b4:36:56;00:00:b4:a9:15:71                    0x0800    133.26.84.187;133.26.75.121                                                                                                                                 jp  "MEIJI-NET Meiji University"        0  138.212.187.109;138.212.185.150                                                                                                                              jp  "!ASAHI KASEI CORPORATION"          1  0  21530  21733  49080   19467995  18978704  0   1460  904.2265  708.7208  0  0.198618  0.001158304   0.003678098   863.3322  780647.8  -0.004692231  0.0127265   1  2  1  3  3          3  0       4198402
A  1    0x0400000200004000  1022171701.691172000  1022171726.640398000  24.949226000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:e0:4c:48:3a:a4;00:80:48:b3:21:69;00:4f:4e:02:84:25  00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:80:48:cd:85:0e  0x0800    138.212.185.53;138.212.188.131;138.212.188.165;138.212.179.166;138.212.185.72;138.212.190.77;138.212.191.1;138.212.191.159;138.212.191.99;138.212.189.38    jp  "!ASAHI KASEI CORPORATION"          1  138.212.213.164;138.212.233.169;138.212.233.135;138.212.232.65;138.212.226.85;138.212.234.148;138.212.237.98;138.212.232.156;138.212.235.135;138.212.226.46  jp  "!ASAHI KASEI CORPORATION"          0  0  46728  28494  1544    63806491  439407    0   1460  1365.487  293.3832  0  0.039757  0.000533925   0.0009780569  1872.924  2557454   0.2424025     0.9863211   1  2  2  2  2          1  686348  631042439
B  1    0x0400000a00004001  1022171701.691707000  1022171726.640388000  24.948681000  2  3;3      eth:ipv4:tcp;eth:ipv4:udp                                         00:d0:02:6d:78:00;00:40:33:28:8b:1b;00:48:54:7a:23:90  00:00:e8:87:02:d1;00:20:18:8a:fd:9b;00:e0:4c:41:eb:1b  0x0800    138.212.226.85;138.212.234.148;138.212.226.8;138.212.226.46;138.212.226.157;138.212.235.98;138.212.212.51;138.212.237.98;138.212.232.156;138.212.233.28     jp  "!ASAHI KASEI CORPORATION"          0  138.212.185.72;138.212.190.77;138.212.184.32;138.212.189.38;138.212.187.49;138.212.189.132;138.212.184.52;138.212.191.1;138.212.191.159;138.212.186.178      jp  "!ASAHI KASEI CORPORATION"          1  0  28494  46728  124271  439407    63806491  0   720   15.42104  31.61275  0  0.039945  0.000875577   0.001045347   1142.104  17612.43  -0.2424025    -0.9863211  2  1  1  1  0.5        1  686348  631042439
A  5    0x0401080a00005000  1022171701.691178000  1022171726.640398000  24.949220000  4  3;3;3;4  eth:ipv4:udp;eth:ipv4:tcp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x08a8)  00:d0:02:6d:78:00                                      00:50:04:2b:e6:2f;00:40:f4:21:e6:56;00:c0:26:2d:aa:4a  0x0800    19.32.78.208;19.132.119.69;19.67.220.218;19.6.61.6;19.49.74.170;19.103.202.194;19.112.182.3;19.82.177.168;19.40.244.207;19.67.33.197                        us  "!Ford Motor Company"               0  138.212.184.93;138.212.188.21;138.212.187.94;138.212.190.146;138.212.186.231;138.212.187.16;138.212.185.98;138.212.186.166;138.212.184.165;138.212.188.166   jp  "!ASAHI KASEI CORPORATION"          0  0  68181  59416  85831   16813186  43551981  0   1460  246.5964  514.0296  0  0.038853  0.0003659278  0.0005407686  2732.791  673896.2  0.06869284    -0.4429507  1  1  2  2  2          1  77892   16914253
B  5    0x0401080a00005001  1022171701.691710000  1022171726.640389000  24.948679000  4  3;3;3;4  eth:ipv4:tcp;eth:ipv4:udp;eth:ipv4:icmp;eth:ipv4:gre:UNK(0x0051)  00:20:af:d2:17:09;00:c0:26:55:f7:dc;00:50:fc:20:14:df  00:d0:02:6d:78:00                                      0x0800    138.212.191.88;138.212.190.148;138.212.184.98;138.212.190.146;138.212.191.117;138.212.186.208;138.212.188.21;138.212.184.165;138.212.190.67;138.212.185.98  jp  "!ASAHI KASEI CORPORATION"          0  19.172.33.164;19.6.48.20;19.52.50.196;19.6.61.6;19.67.158.252;19.146.93.46;19.32.3.2;19.112.107.128;19.206.104.104;19.67.132.229                             us  "!Ford Motor Company"               0  0  59416  68181  18275   43551981  16813186  0   1460  733.0009  662.2905  0  0.039878  0.0004198974  0.0006543636  2381.529  1745663   -0.06869284   0.4429507   1  1  1  1  1          1  0       4198402

Conclusion

The aggregation modes are really useful to acquire a decent overview of the data and enable the analyist to drill down to the very problem without going through unnecessary large flow files initially. But mind the diffent operations of the plugins. If the aggregation mode is activated content plugins are not following streams correctly, thus should not be loaded.

So, that is enough for today and don’t forget to reset the aggregation configuration again for the next tutorials.

t2conf --reset -a && t2build -R

Have fun!