Tutorial: Flexible Flow and Packet Length Statistics
Contents
Introduction
In traffic mining or interface operations packet length and packets/flow are parameters with a high information gain. Thus, T2 provides the user with a user controlled packet length statistics and certain packets/flow indicators.
Preparation
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins and compile the following plugins:
$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow tcpStates basicStats nFrstPkts pktSIATHisto descriptiveStats txtSink
...
BUILD SUCCESSFULIf you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:
$ mkdir ~/data ~/results
$The anonymized sample PCAP used in this tutorial can be downloaded here: annoloc2.pcap Please extract it under your data folder. Now you are all set for T2 IP label experiments.
Basic packet statistics and packet/flow measure
The end report contains information about packet, bytes and protocol statistics, including measures which gives a practitioner an indication about the stress on a flow engine like T2 when operated on an interface.
$ t2 -r ~/data/annoloc2.pcap -w ~/results
================================================================================
Tranalyzer 0.8.8 (Anteater), Tarantula. PID: 46224
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
01: basicFlow, 0.8.8
02: basicStats, 0.8.8
03: tcpStates, 0.8.8
04: nFrstPkts, 0.8.8
05: pktSIATHisto, 0.8.8
06: descriptiveStats, 0.8.8
07: txtSink, 0.8.8
[INF] IPv4 Ver: 5, Rev: 28062020, Range Mode: 0, subnet ranges loaded: 405633 (405.63 K)
[INF] IPv6 Ver: 5, Rev: 28062020, Range Mode: 0, subnet ranges loaded: 49666 (49.67 K)
Processing file: /home/user/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1022171701.691172 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
Dump stop : 1022171726.640398 sec (Thu 23 May 2002 16:35:26 GMT)
Total dump duration: 24.949226 sec
Finished processing. Elapsed time: 1.136210 sec
Finished unloading flow memory. Time: 1.763557 sec
Percentage completed: 100.00%
Number of processed packets: 1219015 (1.22 M)
Number of processed bytes: 64082726 (64.08 M)
Number of raw bytes: 844642686 (844.64 M)
Number of pcap bytes: 83586990 (83.59 M)
Number of IPv4 packets: 1218608 (1.22 M) [99.97%]
Number of IPv6 packets: 160 [0.01%]
Number of A packets: 564232 (564.23 K) [46.29%]
Number of B packets: 654783 (654.78 K) [53.71%]
Number of A bytes: 29448132 (29.45 M) [45.95%]
Number of B bytes: 34634594 (34.63 M) [54.05%]
Average A packet load: 52.19
Average B packet load: 52.89
--------------------------------------------------------------------------------
basicStats: Biggest L2 Talker: 00:d0:02:6d:78:00: 57 [0.00%] packets
basicStats: Biggest L2 Talker: 00:d0:02:6d:78:00: 3420 (3.42 K) [0.01%] bytes
basicStats: Biggest L3 Talker: 138.212.189.38 (JP): 23601 (23.60 K) [1.94%] packets
basicStats: Biggest L3 Talker: 138.212.189.38 (JP): 35005508 (35.01 M) [54.63%] bytes
tcpStates: Aggregated tcpStates anomalies: 0xdf
--------------------------------------------------------------------------------
Headers count: min: 2, max: 4, average: 3.01
Number of GRE packets: 247 [0.02%]
Number of IGMP packets: 12 [0.00%]
Number of ICMP packets: 3059 (3.06 K) [0.25%]
Number of ICMPv6 packets: 11 [0.00%]
Number of TCP packets: 948743 (948.74 K) [77.83%]
Number of TCP bytes: 52643546 (52.64 M) [82.15%]
Number of UDP packets: 266900 (266.90 K) [21.89%]
Number of UDP bytes: 11234272 (11.23 M) [17.53%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed flows: 17589 (17.59 K)
Number of processed A flows: 9980 (9.98 K) [56.74%]
Number of processed B flows: 7609 (7.61 K) [43.26%]
Number of request flows: 9933 (9.93 K) [56.47%]
Number of reply flows: 7656 (7.66 K) [43.53%]
Total A/B flow asymmetry: 0.13
Total req/rply flow asymmetry: 0.13
Number of processed packets/flows: 69.31
Number of processed A packets/flows: 56.54
Number of processed B packets/flows: 86.05
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B packets/s: 48859.83 (48.86 K)
Number of processed A packets/s: 22615.21 (22.61 K)
Number of processed B packets/s: 26244.62 (26.24 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 704.99
Average full raw bandwidth: 270835712 b/s (270.84 Mb/s)
Average snapped bandwidth : 20548206 b/s (20.55 Mb/s)
Average full bandwidth : 270268576 b/s (270.27 Mb/s)
Max number of flows in memory: 15206 (15.21 K) [5.80%]
Memory usage: 0.26 GB [0.52%]
Aggregate flow status: 0x0c0098fa0202d044
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 payload length > framing length
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
[INF] Ethernet flows
[INF] IPv4 flows
[INF] IPv6 flows
[INF] ARP
[INF] IPv4/6 fragmentation
[INF] IPv4/6 in IPv4/6
[INF] GRE encapsulation
[INF] SSDP/UPnP
$Prominent measures are the packets/flows mean and the bandwidth estimation useful to assess the limits of on an interface. As the bandwidth is very low, we have a bad example her. But if it were in the range of 10GBit then we should look at higher tatistical moment are computed by the script t2flowstat from the flow file:
For both flow directions, 100 samples, 1 packet/bin, just the summary
$ t2flowstat annoloc2_flows.txt -c numPktsSnt -s 1 -m 200 -n
----------------------------------
#Flows: 17589
Col Sum: 1219015
Max Col: 23601
Median: 1.889559
Mean: 69.305532
Std Dev: 55.179076
NP Skew: 1.221767
$For the A direction only
$ t2flowstat annoloc2_flows.txt -c numPktsSnt -s 1 -m 200 -n -d A
----------------------------------
#Flows: 9933
Col Sum: 564232
Max Col: 12342
Median: 1.942525
Mean: 56.803785
Std Dev: 40.559900
NP Skew: 1.352599
$For the B direction only
$ t2flowstat annoloc2_flows.txt -c numPktsSnt -s 1 -m 200 -n -d B
----------------------------------
#Flows: 7656
Col Sum: 654783
Max Col: 23601
Median: 1.800890
Mean: 85.525470
Std Dev: 38.597294
NP Skew: 2.169183
$The overall median is around 1.9 pkts/flow, hence 50% of the flows contain two packets or less. So the flow engine has to create a flow every second to third packet in average, which brings every tool down when run at 10GBit++. For more information about this topic, refer to the Performance on interface et al tutorial. The mean and NP skew indicated a highly asymmetric but rapid declining distribution with increasing packets/flow.
We might integrate this statistics into the end report in a future version so that the generation of a flow file is not necessary.
Controlling the packet length
All the statistics available from the loaded plugin in the flow file is computed in the core, using different header information, hence the snap length does not alter the statistics. The content plugins use the snapped length information, there the information can be truncated with lower snap length.
The layer to be considered can be configured by the PACKETLENGTH variable residing in the core definition file packetCapture.h as shown below:
$ tranalyzer2
$ cat packetCapture.h
...
/ Interpretation of packet length
// Following plugins are influenced by PACKETLENGTH:
// nFrstPkts, pktSIATHisto, basicStats, descriptiveStats
#define PACKETLENGTH 3 // 0: including L2, L3 and L4 header,
// 1: including L3 and L4 header,
// 2: including L4 header,
// 3: only higher layer payload (Layer 7)
// If PACKETLENGTH == 1:
#define FRGIPPKTLENVIEW 1 // 0: IP header stays with 2nd++ fragmented packets,
// 1: IP header stripped from 2nd++ fragmented packets
...The constant FRGIPPKTLENVIEW defines for fragmented packets whether the length of the IP header for all following packets should be added or not. In Traffic Mining you like only to look at the content length, so all fragments following the first packet should be stripped of the IP header. For a troubleshooter or admin different story.
Let’s look at the default output of all relevant plugins first:
$ tawk '{ print wildcard("^(num)|Iat") }' ~/results/faf-exercise_flows.txt | head -n 10 | tcol
numHdrDesc numHdrs numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd L2L3L4Pl_Iat Ps_Iat_Cnt_PsCnt_IatCnt dsMinIat dsMaxIat dsMeanIat dsLowQuartileIat dsMedianIat dsUppQuartileIat dsIqdIat dsModeIat dsRangeIat dsStdIat dsRobStdIat dsSkewIat dsExcIat
1 3 1 0 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 0 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 0 0 0_0.000000 0_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
$You see the layer 7 length, e.g. the nFrstPkts output mostly starts with 0, so no content.
As a network person we are interested to see the full length of the packets, so set PACKETLENGTH=0 so that the full packet length including layer 2 is added to the length statistics.
$ t2conf tranalyzer2 -D PACKETLENGTH=0
$ t2build -R
...
$ t2 -r ~/data/faf-exercise.pcap -w ~/results
...
$As the end report and packet/flow statistics does not change, therefore it is not printed. Now look into the flow file again.
$ tawk '{ print wildcard("^(num)|Iat") }' ~/results/annoloc2_flows.txt | head -n 10 | tcol
umHdrDesc numHdrs numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd L2L3L4Pl_Iat Ps_Iat_Cnt_PsCnt_IatCnt dsMinIat dsMaxIat dsMeanIat dsLowQuartileIat dsMedianIat dsUppQuartileIat dsIqdIat dsModeIat dsRangeIat dsStdIat dsRobStdIat dsSkewIat dsExcIat
1 3 1 0 54 0 54_0.000000 54_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 0 54 0 54_0.000000 54_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 62 54 62_0.000000 62_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 54 62 54_0.000000 54_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 62 54 62_0.000000 62_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 54 62 54_0.000000 54_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 62 54 62_0.000000 62_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 54 62 54_0.000000 54_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
1 3 1 1 62 54 62_0.000000 62_0_1_1_1 0 0 0 0 0 0 0 0 0 0 0 0 0
$You easily spot the change in the numPktsSnt, numPktsRcvd, L2L3L4Pl_Iat and Ps_Iat_Cnt_PsCnt_IatCnt columns, now the numbers increased. Try the 1 and 2 yourself.
Fragmentation option
If fragmentation is present the packet length can be seen as one big packet, so the 2nd fragmented packet does not include the IP header. Or you want to see the true packet statistics on the wire and all packets count the length of each subsequent header.
The end report states that fragmented packets are present. Let’s look at these with our last configuration PACKETLENGTH=0 and FRGIPPKTLENVIEW=1.
$ tawk 'bitsanyset($flowStat, 0x0000001000000000)' ~/results/annoloc2_flows.txt | head -n 10 | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType ethVlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStates nFpCnt L2L3L4Pl_Iat tCnt Ps_Iat_Cnt_PsCnt_IatCnt dsMinPl dsMaxPl dsMeanPl dsLowQuartilePl dsMedianPl dsUppQuartilePl dsIqdPl dsModePl dsRangePl dsStdPl dsRobStdPl dsSkewPl dsExcPl dsMinIat dsMaxIat dsMeanIat dsLowQuartileIat dsMedianIat dsUppQuartileIat dsIqdIat dsModeIat dsRangeIat dsStdIat dsRobStdIat dsSkewIat dsExcIat
A 1894 0x040008d200004000 1022171702.614414 1022171702.614414 0.000000 1 3 eth:ipv4:udp 00:80:48:b3:0e:ed 00:d0:02:6d:78:00 0x0800 138.212.188.118 jp "ASAHI KASEI CORPORATION" 0 201.9.46.255 br "Telemar Norte Leste SA" 0 17 1 0 66 0 66 66 66 0 0 0 0 0 0 0 1 1 0x00 1 66_0.000000 1 66_0_1_1_1 66 66 66 66 66 66 0 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A 6282 0x0400085200004000 1022171713.796490 1022171713.796491 0.000001 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:80:c8:8f:ca:5e 0x0800 16.46.171.138 us "HEWLETT PACKARD ENTERPRISE COM" 0 138.212.189.231 jp "ASAHI KASEI CORPORATION" 0 17 2 0 2405 0 911 1494 1202.5 206.1216 0 1e-06 5e-07 3.535534e-07 2000000 2.405e+09 1 1 0x00 2 1494_0.000000;911_0.000001 2 911_0_1_1_2;1494_0_1_1_2 911 1494 1202.5 911 1202.5 1494 583 911 583 291.5 291.5 0 -2 0 0 0 0 0 0 0 0 0 0 0 0 0
A 984 0x0400081200004000 1022171701.848919 1022171726.366145 24.517226 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:e0:29:04:0a:18 0x0800 201.232.53.168 co "EPM Telecomunicaciones SA ESP" 3289 138.212.189.228 jp "ASAHI KASEI CORPORATION" 1533 17 203 0 207946 0 72 1514 1024.365 667.7642 0 0.391237 0.1207745 0.1718241 8.279893 8481.629 1 1 0x00 20 1514_0.000000;1494_0.000409;72_0.000000;1514_0.374473;1494_0.000018;72_0.000018;1514_0.375298;1494_0.000013;72_0.000007;1514_0.360322;1494_0.000009;72_0.000001;1514_0.374244;1494_0.000360;72_0.000001;1514_0.388981;1494_0.000444;72_0.000000;1514_0.360684;1494_0.000416 11 72_0_68_68_134;1494_0_65_67_134;1494_1_2_67_2;1514_0_1_68_134;1514_12_1_68_1;1514_355_3_68_3;1514_360_19_68_19;1514_370_32_68_32;1514_375_6_68_6;1514_385_5_68_5;1514_390_1_68_1 72 1514 1024.365 72 1494 1514 1442 72 1442 675.9622 675.9622 -0.6988212 -1.510996 0.5 392.5 121.5693 0.5 0.5 362.5 362 0.5 392 174.1365 174.1365 0.7409255 -1.44783
A 98 0x0400081200004000 1022171701.699706 1022171726.576813 24.877107 1 3 eth:ipv4:udp 00:80:48:b3:0e:ed 00:d0:02:6d:78:00 0x0800 138.212.188.118 jp "ASAHI KASEI CORPORATION" 655 201.9.46.255 br "Telemar Norte Leste SA" 655 17 1014 508 793668 70104 66 1514 782.7101 720.2989 0 0.833577 0.02453363 0.04786205 40.76036 31903.55 0.3324573 0.8376794 0x00 20 1514_0.000000;66_0.000002;1514_0.001272;66_0.000002;1514_0.089488;66_0.000001;1514_0.001204;66_0.000002;1514_0.074224;66_0.000001;174_0.000003;1514_0.083978;66_0.000001;1514_0.000027;66_0.000001;1514_0.093176;66_0.000000;1514_0.001097;66_0.000001;1514_0.106532 64 66_0_501_501_601;174_0_12_12_601;1514_0_88_501_601;1514_1_120_501_120;1514_2_29_501_29;1514_3_5_501_5;1514_4_1_501_1;1514_54_1_501_1;1514_64_1_501_1;1514_65_1_501_1;1514_66_2_501_2;1514_67_2_501_2;1514_68_3_501_3;1514_69_1_501_1;1514_70_6_501_6;1514_71_2_501_2;1514_72_1_501_1;1514_73_3_501_3;1514_74_4_501_4;1514_75_1_501_1;1514_76_2_501_2;1514_77_3_501_3;1514_78_3_501_3;1514_79_3_501_3;1514_80_5_501_5;1514_81_1_501_1;1514_82_5_501_5;1514_83_6_501_6;1514_84_9_501_9;1514_85_2_501_2;1514_86_11_501_11;1514_87_8_501_8;1514_88_10_501_10;1514_89_8_501_8;1514_90_13_501_13;1514_91_12_501_12;1514_92_10_501_10;1514_93_15_501_15;1514_94_13_501_13;1514_95_9_501_9;1514_96_9_501_9;1514_97_11_501_11;1514_98_4_501_4;1514_99_6_501_6;1514_100_8_501_8;1514_101_8_501_8;1514_102_5_501_5;1514_103_2_501_2;1514_104_1_501_1;1514_105_3_501_3;1514_106_4_501_4;1514_107_1_501_1;1514_108_5_501_5;1514_109_2_501_2;1514_110_2_501_2;1514_112_1_501_1;1514_114_1_501_1;1514_115_2_501_2;1514_119_1_501_1;1514_148_1_501_1;1514_169_1_501_1;1514_170_1_501_1;1514_179_2_501_2;1514_830_1_501_1 66 1514 782.7101 66 174 1514 1448 66 1448 722.7795 722.7795 0.02293284 -1.998584 0.5 835 24.8075 0.5 0.5 67 66.5 0.5 834.5 47.92335 47.92335 5.5217 79.30751
A 1289 0x0400081200004000 1022171702.058266 1022171726.575284 24.517018 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:10:60:59:f1:4b 0x0800 16.103.245.128 us "HEWLETT PACKARD ENTERPRISE COM" 1120 138.212.191.249 jp "ASAHI KASEI CORPORATION" 1461 17 134 0 107664 0 72 1514 803.4627 707.2161 0 0.747904 0.1829628 0.1880735 5.465591 4391.399 1 1 0x00 20 1514_0.000000;72_0.000001;1514_0.373331;72_0.001235;1514_0.372348;72_0.000001;1514_0.379557;72_0.000001;1514_0.357597;72_0.000001;1514_0.374397;72_0.000001;1514_0.372138;72_0.000001;1514_0.373593;72_0.000001;1514_0.366913;72_0.000000;1514_0.371722;72_0.000001 12 72_0_65_66_68;72_1_1_66_1;1494_0_2_2_68;1514_0_1_66_68;1514_355_2_66_2;1514_360_8_66_8;1514_365_11_66_11;1514_370_30_66_30;1514_375_9_66_9;1514_380_3_66_3;1514_390_1_66_1;1514_740_1_66_1 72 1514 803.4627 72 1494 1514 1442 72 1442 720.6296 720.6296 -0.02982101 -1.999067 0.5 745 184.5865 0.5 1 372.5 372 0.5 744.5 191.7651 191.7651 0.1838971 -1.55974
A 1290 0x040008f200004000 1022171702.058274 1022171726.575521 24.517247 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:10:60:59:f1:4b 0x0800 16.103.245.128 us "HEWLETT PACKARD ENTERPRISE COM" 0 138.212.191.249 jp "ASAHI KASEI CORPORATION" 0 17 64 0 95616 0 1494 1494 1494 0 0 0.747897 0.383082 0.08560037 2.610407 3899.948 1 1 0x00 20 1494_0.000000;1494_0.374560;1494_0.372355;1494_0.379555;1494_0.358766;1494_0.373236;1494_0.372266;1494_0.373462;1494_0.367221;1494_0.371415;1494_0.375320;1494_0.373893;1494_0.371685;1494_0.365597;1494_0.372201;1494_0.375167;1494_0.372508;1494_0.363928;1494_0.377676;1494_0.370959 11 1494_0_1_64_1;1494_355_1_64_1;1494_360_7_64_7;1494_365_10_64_10;1494_370_28_64_28;1494_375_10_64_10;1494_380_3_64_3;1494_390_1_64_1;1494_720_1_64_1;1494_730_1_64_1;1494_740_1_64_1 1494 1494 1494 1494 1494 1494 0 1494 0 0 0 0 0 357.5 745 389.2064 367.5 372.5 377.5 10 372.5 387.5 78.18096 7.413 4.213324 15.88362
A 126 0x0400083200004000 1022171701.700965 1022171726.594434 24.893469 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:80:c8:8f:ca:5e 0x0800 16.46.171.138 us "HEWLETT PACKARD ENTERPRISE COM" 4623 138.212.189.231 jp "ASAHI KASEI CORPORATION" 1490 17 742 0 969486 0 899 1514 1306.585 278.4473 0 0.202366 0.03354916 0.04754491 29.80701 38945.39 1 1 0x00 20 1514_0.000000;1494_0.000008;911_0.000002;1514_0.098384;1494_0.000011;911_0.000000;1514_0.093618;1494_0.000012;911_0.000002;1514_0.105108;1494_0.000225;911_0.000005;1514_0.095565;1494_0.000204;911_0.000001;1514_0.106943;1494_0.000007;911_0.000001;1514_0.094354;1494_0.000009 28 899_0_1_1_486;906_0_1_1_486;907_0_1_1_486;911_0_243_244_486;911_1_1_244_8;1494_0_239_247_486;1494_1_7_247_8;1494_2_1_247_1;1514_0_1_248_486;1514_91_2_248_2;1514_92_9_248_9;1514_93_9_248_9;1514_94_20_248_20;1514_95_46_248_46;1514_96_40_248_40;1514_97_12_248_12;1514_98_8_248_8;1514_99_1_248_1;1514_101_1_248_1;1514_103_5_248_5;1514_104_13_248_13;1514_105_24_248_24;1514_106_34_248_34;1514_107_14_248_14;1514_108_6_248_6;1514_109_1_248_1;1514_191_1_248_1;1514_200_1_248_1 899 1514 1306.585 911 1494 1514 603 1514 615 279.6179 279.6179 -0.7065479 -1.496943 0.5 202.5 33.87922 0.5 0.5 95.5 95 0.5 202 47.57575 47.57575 0.7782133 -1.190346
A 406 0x0400081200004000 1022171701.717743 1022171726.607895 24.890152 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:e0:29:04:0b:81 0x0800 18.14.224.62 us "Massachusetts Institute of Tec" 4383 138.212.191.34 jp "ASAHI KASEI CORPORATION" 2428 17 136 0 157964 0 809 1514 1161.5 346.0216 0 0.665421 0.1830158 0.1876362 5.464008 6346.446 1 1 0x00 20 1514_0.000000;809_0.000000;1514_0.359622;809_0.000001;1514_0.379067;809_0.000002;1514_0.372816;809_0.000001;1514_0.373867;809_0.000001;1514_0.362254;809_0.000001;1514_0.529715;809_0.000001;1514_0.219500;809_0.000001;1514_0.665421;809_0.000001;1514_0.203431;809_0.000001 12 809_0_68_68_69;1514_0_1_68_69;1514_87_1_68_1;1514_200_1_68_1;1514_215_1_68_1;1514_355_1_68_1;1514_360_12_68_12;1514_365_2_68_2;1514_370_36_68_36;1514_375_10_68_10;1514_520_2_68_2;1514_660_1_68_1 809 1514 1161.5 809 1161.5 1514 705 809 705 352.5 352.5 0 -2 0.5 665 184.3259 0.5 44 372.5 372 0.5 664.5 191.182 191.182 0.1759952 -1.692286
$Now we like to see what happens if we switch off the IP header length suppression. Set FRGIPPKTLENVIEW=0, recompile and run t2 on the pcap.
$ t2conf tranalyzer2 -D FRGIPPKTLENVIEW=0
$ t2build -R
...
$ t2 -r ~/data/annoloc2.pcap -w ~/results
...
$$ tawk '{ if (bitsanyset($flowStat, 0x0000001000000000)) print }' ~/results/annoloc2_flows.txt | head -n 10 | tcol
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType ethVlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStates nFpCnt L2L3L4Pl_Iat tCnt Ps_Iat_Cnt_PsCnt_IatCnt dsMinPl dsMaxPl dsMeanPl dsLowQuartilePl dsMedianPl dsUppQuartilePl dsIqdPl dsModePl dsRangePl dsStdPl dsRobStdPl dsSkewPl dsExcPl dsMinIat dsMaxIat dsMeanIat dsLowQuartileIat dsMedianIat dsUppQuartileIat dsIqdIat dsModeIat dsRangeIat dsStdIat dsRobStdIat dsSkewIat dsExcIat
A 1894 0x040008d200004000 1022171702.614414 1022171702.614414 0.000000 1 3 eth:ipv4:udp 00:80:48:b3:0e:ed 00:d0:02:6d:78:00 0x0800 138.212.188.118 jp "ASAHI KASEI CORPORATION" 0 201.9.46.255 br "Telemar Norte Leste SA" 0 17 1 0 86 0 86 86 86 0 0 0 0 0 0 0 1 1 0x00 1 86_0.000000 1 86_0_1_1_1 86 86 86 86 86 86 0 86 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A 6282 0x0400085200004000 1022171713.796490 1022171713.796491 0.000001 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:80:c8:8f:ca:5e 0x0800 16.46.171.138 us "HEWLETT PACKARD ENTERPRISE COM" 0 138.212.189.231 jp "ASAHI KASEI CORPORATION" 0 17 2 0 2445 0 931 1514 1222.5 206.1216 0 1e-06 5e-07 3.535534e-07 2000000 2.445e+09 1 1 0x00 2 1514_0.000000;931_0.000001 2 931_0_1_1_2;1514_0_1_1_2 931 1514 1222.5 931 1222.5 1514 583 931 583 291.5 291.5 0 -2 0 0 0 0 0 0 0 0 0 0 0 0 0
A 984 0x0400081200004000 1022171701.848919 1022171726.366145 24.517226 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:e0:29:04:0a:18 0x0800 201.232.53.168 co "EPM Telecomunicaciones SA ESP" 3289 138.212.189.228 jp "ASAHI KASEI CORPORATION" 1533 17 203 0 210646 0 92 1514 1037.665 663.0211 0 0.391237 0.1207745 0.1718241 8.279893 8591.755 1 1 0x00 20 1514_0.000000;1514_0.000409;92_0.000000;1514_0.374473;1514_0.000018;92_0.000018;1514_0.375298;1514_0.000013;92_0.000007;1514_0.360322;1514_0.000009;92_0.000001;1514_0.374244;1514_0.000360;92_0.000001;1514_0.388981;1514_0.000444;92_0.000000;1514_0.360684;1514_0.000416 10 92_0_68_68_134;1514_0_66_135_134;1514_1_2_135_2;1514_12_1_135_1;1514_355_3_135_3;1514_360_19_135_19;1514_370_32_135_32;1514_375_6_135_6;1514_385_5_135_5;1514_390_1_135_1 92 1514 1037.665 92 1514 1514 1422 1514 1422 671.1582 671.1582 -0.6992838 -1.511002 0.5 392.5 121.5693 0.5 0.5 362.5 362 0.5 392 174.1365 174.1365 0.7409255 -1.44783
A 98 0x0400081200004000 1022171701.699706 1022171726.576813 24.877107 1 3 eth:ipv4:udp 00:80:48:b3:0e:ed 00:d0:02:6d:78:00 0x0800 138.212.188.118 jp "ASAHI KASEI CORPORATION" 655 201.9.46.255 br "Telemar Norte Leste SA" 655 17 1014 508 803688 70104 86 1514 792.5917 710.5359 0 0.833577 0.02453363 0.04786205 40.76036 32306.33 0.3324573 0.8395408 0x00 20 1514_0.000000;86_0.000002;1514_0.001272;86_0.000002;1514_0.089488;86_0.000001;1514_0.001204;86_0.000002;1514_0.074224;86_0.000001;174_0.000003;1514_0.083978;86_0.000001;1514_0.000027;86_0.000001;1514_0.093176;86_0.000000;1514_0.001097;86_0.000001;1514_0.106532 64 86_0_501_501_601;174_0_12_12_601;1514_0_88_501_601;1514_1_120_501_120;1514_2_29_501_29;1514_3_5_501_5;1514_4_1_501_1;1514_54_1_501_1;1514_64_1_501_1;1514_65_1_501_1;1514_66_2_501_2;1514_67_2_501_2;1514_68_3_501_3;1514_69_1_501_1;1514_70_6_501_6;1514_71_2_501_2;1514_72_1_501_1;1514_73_3_501_3;1514_74_4_501_4;1514_75_1_501_1;1514_76_2_501_2;1514_77_3_501_3;1514_78_3_501_3;1514_79_3_501_3;1514_80_5_501_5;1514_81_1_501_1;1514_82_5_501_5;1514_83_6_501_6;1514_84_9_501_9;1514_85_2_501_2;1514_86_11_501_11;1514_87_8_501_8;1514_88_10_501_10;1514_89_8_501_8;1514_90_13_501_13;1514_91_12_501_12;1514_92_10_501_10;1514_93_15_501_15;1514_94_13_501_13;1514_95_9_501_9;1514_96_9_501_9;1514_97_11_501_11;1514_98_4_501_4;1514_99_6_501_6;1514_100_8_501_8;1514_101_8_501_8;1514_102_5_501_5;1514_103_2_501_2;1514_104_1_501_1;1514_105_3_501_3;1514_106_4_501_4;1514_107_1_501_1;1514_108_5_501_5;1514_109_2_501_2;1514_110_2_501_2;1514_112_1_501_1;1514_114_1_501_1;1514_115_2_501_2;1514_119_1_501_1;1514_148_1_501_1;1514_169_1_501_1;1514_170_1_501_1;1514_179_2_501_2;1514_830_1_501_1 86 1514 792.5917 86 174 1514 1428 86 1428 712.9836 712.9836 0.02316274 -1.99884 0.5 835 24.8075 0.5 0.5 67 66.5 0.5 834.5 47.92335 47.92335 5.5217 79.30751
A 1289 0x0400081200004000 1022171702.058266 1022171726.575284 24.517018 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:10:60:59:f1:4b 0x0800 16.103.245.128 us "HEWLETT PACKARD ENTERPRISE COM" 1120 138.212.191.249 jp "ASAHI KASEI CORPORATION" 1461 17 134 0 109024 0 92 1514 813.6119 697.696 0 0.747904 0.1829628 0.1880735 5.465591 4446.871 1 1 0x00 20 1514_0.000000;92_0.000001;1514_0.373331;92_0.001235;1514_0.372348;92_0.000001;1514_0.379557;92_0.000001;1514_0.357597;92_0.000001;1514_0.374397;92_0.000001;1514_0.372138;92_0.000001;1514_0.373593;92_0.000001;1514_0.366913;92_0.000000;1514_0.371722;92_0.000001 11 92_0_65_66_68;92_1_1_66_1;1514_0_3_68_68;1514_355_2_68_2;1514_360_8_68_8;1514_365_11_68_11;1514_370_30_68_30;1514_375_9_68_9;1514_380_3_68_3;1514_390_1_68_1;1514_740_1_68_1 92 1514 813.6119 92 1514 1514 1422 1514 1422 710.9208 710.9208 -0.0298541 -1.999109 0.5 745 184.5865 0.5 1 372.5 372 0.5 744.5 191.7651 191.7651 0.1838971 -1.55974
A 1290 0x040008f200004000 1022171702.058274 1022171726.575521 24.517247 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:10:60:59:f1:4b 0x0800 16.103.245.128 us "HEWLETT PACKARD ENTERPRISE COM" 0 138.212.191.249 jp "ASAHI KASEI CORPORATION" 0 17 64 0 96896 0 1514 1514 1514 0 0 0.747897 0.383082 0.08560037 2.610407 3952.157 1 1 0x00 20 1514_0.000000;1514_0.374560;1514_0.372355;1514_0.379555;1514_0.358766;1514_0.373236;1514_0.372266;1514_0.373462;1514_0.367221;1514_0.371415;1514_0.375320;1514_0.373893;1514_0.371685;1514_0.365597;1514_0.372201;1514_0.375167;1514_0.372508;1514_0.363928;1514_0.377676;1514_0.370959 11 1514_0_1_64_1;1514_355_1_64_1;1514_360_7_64_7;1514_365_10_64_10;1514_370_28_64_28;1514_375_10_64_10;1514_380_3_64_3;1514_390_1_64_1;1514_720_1_64_1;1514_730_1_64_1;1514_740_1_64_1 1514 1514 1514 1514 1514 1514 0 1514 0 0 0 0 0 357.5 745 389.2064 367.5 372.5 377.5 10 372.5 387.5 78.18096 7.413 4.213324 15.88362
A 126 0x0400083200004000 1022171701.700965 1022171726.594434 24.893469 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:80:c8:8f:ca:5e 0x0800 16.46.171.138 us "HEWLETT PACKARD ENTERPRISE COM" 4623 138.212.189.231 jp "ASAHI KASEI CORPORATION" 1490 17 742 0 979366 0 919 1514 1319.9 273.6272 0 0.202366 0.03354916 0.04754491 29.80701 39342.29 1 1 0x00 20 1514_0.000000;1514_0.000008;931_0.000002;1514_0.098384;1514_0.000011;931_0.000000;1514_0.093618;1514_0.000012;931_0.000002;1514_0.105108;1514_0.000225;931_0.000005;1514_0.095565;1514_0.000204;931_0.000001;1514_0.106943;1514_0.000007;931_0.000001;1514_0.094354;1514_0.000009 27 919_0_1_1_486;926_0_1_1_486;927_0_1_1_486;931_0_243_244_486;931_1_1_244_8;1514_0_240_495_486;1514_1_7_495_8;1514_2_1_495_1;1514_91_2_495_2;1514_92_9_495_9;1514_93_9_495_9;1514_94_20_495_20;1514_95_46_495_46;1514_96_40_495_40;1514_97_12_495_12;1514_98_8_495_8;1514_99_1_495_1;1514_101_1_495_1;1514_103_5_495_5;1514_104_13_495_13;1514_105_24_495_24;1514_106_34_495_34;1514_107_14_495_14;1514_108_6_495_6;1514_109_1_495_1;1514_191_1_495_1;1514_200_1_495_1 919 1514 1319.9 931 1514 1514 583 1514 595 274.7766 274.7766 -0.7092626 -1.496931 0.5 202.5 33.87922 0.5 0.5 95.5 95 0.5 202 47.57575 47.57575 0.7782133 -1.190346
A 406 0x0400081200004000 1022171701.717743 1022171726.607895 24.890152 1 3 eth:ipv4:udp 00:d0:02:6d:78:00 00:e0:29:04:0b:81 0x0800 18.14.224.62 us "Massachusetts Institute of Tec" 4383 138.212.191.34 jp "ASAHI KASEI CORPORATION" 2428 17 136 0 159324 0 829 1514 1171.5 336.2054 0 0.665421 0.1830158 0.1876362 5.464008 6401.086 1 1 0x00 20 1514_0.000000;829_0.000000;1514_0.359622;829_0.000001;1514_0.379067;829_0.000002;1514_0.372816;829_0.000001;1514_0.373867;829_0.000001;1514_0.362254;829_0.000001;1514_0.529715;829_0.000001;1514_0.219500;829_0.000001;1514_0.665421;829_0.000001;1514_0.203431;829_0.000001 12 829_0_68_68_69;1514_0_1_68_69;1514_87_1_68_1;1514_200_1_68_1;1514_215_1_68_1;1514_355_1_68_1;1514_360_12_68_12;1514_365_2_68_2;1514_370_36_68_36;1514_375_10_68_10;1514_520_2_68_2;1514_660_1_68_1 829 1514 1171.5 829 1171.5 1514 685 829 685 342.5 342.5 0 -2 0.5 665 184.3259 0.5 44 372.5 372 0.5 664.5 191.182 191.182 0.1759952 -1.692286
$If you look now numPktsRcvd or numBytesSnt or the second entry at Ps_Iat_Cnt_PsCnt_IatCnt the numbers now increased by the header length(s).
Don’t forget to reset constants and recompile for the next tutorials:
$ t2conf tranalyzer2 -D PACKETLENGTH=3 FRGIPPKTLENVIEW=1
$ t2build -R
...
$Have fun!