Tutorial: IP/TCP Troubleshooting and hidden figures

IP/TCP header features

In this tutorial we will show you the hidden power of the Layer3/4 for troubleshooting, security and admin applications. All is integrated in one plugin: tcpFlags. Its name is a bit misleading, as it evolved during practical application from a simple tcp Flags decoder to a full blown troubleshooting plugin for L3/4. So that it now provides the following features:

  • OS and application fingerprinting
  • Host load estimation
  • Options
  • Sequence/Acknowledge Number Tricks
  • NAT flow bundeling: boot time estimation, host clk estimation (OS fingerprinting)
  • Trip and Round Trip Time (RTT), jitter estimation
  • L3/4 Checksum evaluation
  • Protocol anomalies
  • Fragmentation anomalies
  • Flow health: Window Size statistics
  • Scan detection support

If you read the The Basics tutorial you already had a glips of some basic features from tcpFlags. Here we will explain the application of such features. Note that you need a version of the tcpFlags plugin 0.8.5 or higher, so that the output in your cmd line matches to the website.

Preparation

Before we start we need to prepare T2. If you did not complete the tutorials before just follow the procedure described below.

First I recommend to set T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins and compile the standard plugins. Just as a precaution if you have some old plugins or files there. If you like to keep them, please copy them away.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow tcpFlags tcpStates txtSink
...
BUILD SUCCESSFUL

$

If you did not create a separate data and results directory yet, please do it now in another cmd window, that facilitates your workflow:

$ mkdir ~/data
$ mkdir ~/results
$ cd data

Download the sample pcap if did not do it already: annoloc2.pcap. Now you’re all set. Let’s start with tcpFlags in minimal mode.

tcpFlags all off

tcpFlags is easier to comprehend if we first switch it to a minimal configuration, aka the fastest mode. Open tcpFlags.h in the tcpFlags plugin folder and set the constants as shown below:

$ cd tcpFlags/src
$ vi tcpFlags.h
...
// local defines

// -s option
#define SPKTMD_SEQACKREL 0 // -s option SEQ/ACK Numbers 0: absolute, 1: relative
#define SPKTMD_SEQACKHEX 0 // -s option SEQ/ACK Numbers 0: uint32_t, 1: hex32

// user defined constants 
#define RTT_ESTIMATE     0 // 1: Round trip time estimation
#define IPCHECKSUM       0 // 1: Calculation of L3 (IP) header checksum,
                           // 2: L3 + L4 (TCP,UDP) checksum
#define WINDOWSIZE       0 // 1: Calculation of TCP window size parameters
#define WINMIN           1 // Minimal window size threshold defining a healthy communication, packets below are counted
#define SEQ_ACK_NUM      0 // 1: SEQ/ACK number feature analysis
#define FRAG_ANALYZE     0 // 1: Fragmentation analysis
#define NAT_BT_EST       0 // 1: NAT boot time estimation
#define SCAN_DETECTOR    0 // 1: Scan flow detector
...

The constants SPKTMD_SEQACKREL and SPKTMD_SEQACKHEX refer to the packet mode for absolute and relative seq/ack number output and decimal/hex representation respectively, see packetmode

Save the file, recompile tcpFlags and invoke T2 with the -s option and store it in your results folder.

$ t2build tcpFlags
...

$ t2 -r ~/data/annoloc2.pcap -w ~/results/ -s
================================================================================
Tranalyzer 0.8.5 (Anteater), Tarantula. PID: 26054
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.5
    02: tcpFlags, 0.8.5
    03: tcpStates, 0.8.5
    04: txtSink, 0.8.5
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312745 (312.75 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
Processing file: /home/wurst/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1022171701.691172 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
Dump stop : 1022171726.640398 sec (Thu 23 May 2002 16:35:26 GMT)
Total dump duration: 24.949226 sec
Finished processing. Elapsed time: 7.350005 sec
Finished unloading flow memory. Time: 7.550096 sec
Percentage completed: 100.00%
Number of processed packets: 1219015 (1.22 M)
Number of processed bytes: 64082726 (64.08 M)
Number of raw bytes: 844642686 (844.64 M)
Number of pcap bytes: 83586990 (83.59 M)
Number of IPv4 packets: 1218588 (1.22 M) [99.96%]
Number of IPv6 packets: 180 [0.01%]
Number of A packets: 564233 (564.23 K) [46.29%]
Number of B packets: 654782 (654.78 K) [53.71%]
Number of A bytes: 29448166 (29.45 M) [45.95%]
Number of B bytes: 34634560 (34.63 M) [54.05%]
Average A packet load: 52.19
Average B packet load: 52.89
--------------------------------------------------------------------------------
tcpFlags: Aggregated ipFlags: 0x3866
tcpFlags: Aggregated tcpAnomaly: 0x0347
tcpFlags: Number of TCP scans, succ scans, syn retries, seq retries: 0, 0, 114, 0
tcpStates: Aggregated anomaly flags: 0xdf
--------------------------------------------------------------------------------
Headers count: min: 2, max: 4, average: 3.01
Number of GRE packets: 20 [0.00%]
Number of IGMP packets: 12 [0.00%]
Number of ICMP packets: 3059 (3.06 K) [0.25%]
Number of ICMPv6 packets: 11 [0.00%]
Number of TCP packets: 948743 (948.74 K) [77.83%]
Number of TCP bytes: 52643546 (52.64 M) [82.15%]
Number of UDP packets: 266900 (266.90 K) [21.89%]
Number of UDP bytes: 11234272 (11.23 M) [17.53%]
Number of IPv4 fragmented packets: 2284 (2.28 K) [0.19%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 17603 (17.60 K)
Number of processed A flows: 9995 (9.99 K) [56.78%]
Number of processed B flows: 7608 (7.61 K) [43.22%]
Number of request     flows: 9948 (9.95 K) [56.51%]
Number of reply       flows: 7655 (7.66 K) [43.49%]
Total   A/B    flow asymmetry: 0.14
Total req/rply flow asymmetry: 0.13
Number of processed   packets/flows: 69.25
Number of processed A packets/flows: 56.45
Number of processed B packets/flows: 86.06
Number of processed total packets/s: 48859.83 (48.86 K)
Number of processed A+B packets/s: 48859.83 (48.86 K)
Number of processed A   packets/s: 22615.25 (22.61 K)
Number of processed   B packets/s: 26244.58 (26.24 K)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 705.55
Average full raw bandwidth: 270835712 b/s (270.84 Mb/s)
Average snapped bandwidth : 20548206 b/s (20.55 Mb/s)
Average full bandwidth : 270445248 b/s (270.45 Mb/s)
Max number of flows in memory: 15220 (15.22 K) [5.81%]
Memory usage: 0.08 GB [0.11%]
Aggregate flow status: 0x000018fa0202d044
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
[INF] IPv4
[INF] IPv6
[INF] IPv4/6 fragmentation
[INF] IPv4/6 in IPv4/6
[INF] GRE encapsulation
[INF] SSDP/UPnP flows
[INF] Ethernet flows
[INF] ARP flows
$

We got a lot of warnings about fragmentation and L4/7 clipping, which is understandable, because this flow is annonymized so all content is gone.

As you might not always in control of the pcap acquisition process, these warnings and the line at the beginning

[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500

gives you an indication about the snaplength being imposed. So without even looking at the flows or packets you can assess the quality of a large pcap.

Between the dashed lines tcpFlags reports a summary of the flow variables ipFlags, tcpAnomaly and syn retries. The latter denotes mostly benign syn connection retries if the destination host does not answer.

Looking at the ipFlags and tcpAnomaly values we see checksum errors and fragmentationi, mostly due to snapped and annonymized traffic.

Moreover Null scans, syn retransmissions and that the L4 option field is corrupt in some flows.

We could now select flows where the said bits are set, but let’s select the first 20 flows containing time stamp options, because we’ll need them later. Moreover we can compare the different columns of tcpFlags better.

If you look at the ipFlags column, you imediately see that the snapped L3 header bit correlates with the L4 checksum errors. It is also understandable that the flows where the L4 option is corrupt, you cannot trust all extracted tcp options. Like the ipOptions they are hex coded: 2Option

Nevertheless, some are extracted such as the MSS and Windows Scale value. Moreover important values from the L3 Header containing information about senders performance, routing, applications, etc are extracted. E.g. IPv4: if the IPID is incremented by 1 in an OS then $ipMindIPID, $ipMaxdIPID indicate the load of the sending host. The rest, see my boot camp I’m giving from time to time if I’m invited; hint hint.

$ tawk 'bitsanyset($tcpOptions, 0x00000100)' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipTOS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpStates
A     2984     0x0000100200004000  1022171704.302738  1022171704.494135  0.191397  1           3        eth:ipv4:tcp  00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800              138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x0100    0           14454       64        64        0         0x00   0x1842   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0101      10            30         0x00000102  0       1      0x52
B     2984     0x0000100200004001  1022171704.302742  1022171704.489628  0.186886  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800              138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x0100    0           0           63        63        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0103      10            30         0x00000102  0       1      0x02
A     3415     0x0000000200004000  1022171705.224036  1022171705.383786  0.159750  1           3        eth:ipv4:tcp  00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800              138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x0100    1           42596       64        64        0         0x00   0x1842   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0101      10            30         0x00000102  0       1      0x42
B     3415     0x0000100200004001  1022171705.224038  1022171705.374555  0.150517  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800              138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x0100    0           0           63        63        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0103      9             27         0x00000102  0       1      0x02
A     3477     0x0000000200004000  1022171705.346276  1022171705.504551  0.158275  1           3        eth:ipv4:tcp  00:01:02:b7:bb:d4  00:d0:02:6d:78:00  0x0800              138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x0100    1           1           64        64        0         0x08   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0xdb      0x0101      8             24         0x00000102  0       1      0x06
B     3477     0x0000000200004001  1022171705.367204  1022171705.559621  0.192417  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:01:02:b7:bb:d4  0x0800              201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x0100    1           3           120       120       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0xd6      0x0102      4             12         0x00000102  0       1      0x46
A     2811     0x0000000000004000  1022171706.205501  1022171706.999557  0.794056  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:50:fc:3b:62:78  0x0800              216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0100    1           12          51        242       1         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x95      0x0001      2             6          0x00000102  0       1      0x53
B     2811     0x0000000200004001  1022171703.964220  1022171706.365041  2.400821  1           3        eth:ipv4:tcp  00:50:fc:3b:62:78  00:d0:02:6d:78:00  0x0800              138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x0100    1           1           64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x90      0x0000      3             9          0x00000102  0       1      0x03
A     4982     0x0000000200004000  1022171709.624612  1022171709.687159  0.062547  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x0100    256         256         48        48        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0101      6             18         0x00000102  0       1      0x0a
B     4982     0x0000000200004001  1022171709.626249  1022171709.687168  0.060919  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x0100    1           24          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0103      3             9          0x00000102  0       1      0x4a
A     5035     0x0000000200004000  1022171709.784259  1022171709.836533  0.052274  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x0100    256         768         48        48        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0101      6             18         0x00000102  0       1      0x0a
B     5035     0x0000000200004001  1022171709.786501  1022171709.837980  0.051479  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x0100    1           17          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0103      3             9          0x00000102  0       1      0x4a
A     5036     0x0000000200004000  1022171709.786494  1022171709.836536  0.050042  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x0100    256         768         48        48        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0101      6             18         0x00000102  0       1      0x0a
B     5036     0x0000000200004001  1022171709.786504  1022171709.837981  0.051477  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x0100    1           16          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0103      3             9          0x00000102  0       1      0x4a
A     5083     0x0000000200004000  1022171709.912864  1022171709.969239  0.056375  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x0100    256         768         48        48        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5b      0x0101      6             18         0x00000102  0       1      0x0a
B     5083     0x0000000200004001  1022171709.912872  1022171709.969252  0.056380  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x0100    1           21          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x5f      0x0103      3             9          0x00000102  0       1      0x4a
A     24       0x0000000200004000  1022171701.692693  1022171710.923400  9.230707  1           3        eth:ipv4:tcp  00:50:04:e8:e3:41  00:d0:02:6d:78:00  0x0800              138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x0100    1           4           64        64        0         0x08   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x19      0x0001      895           2685       0x00000102  0       1      0x07
B     24       0x0000000200004001  1022171701.706576  1022171710.959603  9.253027  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:50:04:e8:e3:41  0x0800              36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x0100    1           6           117       117       0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x14      0x0100      378           1134       0x00000102  0       1      0x47
A     776      0x0000000200004000  1022171701.778309  1022171701.793349  0.015040  1           3        eth:ipv4:tcp  00:00:1c:b6:1a:e1  00:d0:02:6d:78:00  0x0800              138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x0100    1           1           64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  0x1b      0x0101      5             15         0x00000102  0       1      0x00
$

But there is something more suspicious, the tcp NULL flaged flows. If you extract the packets, then you see that is really weird traffic. Have a look at the first flowStat

$tawk -V flowStat=0x0000100200004000

The flowStat column with value 0x0000100200004000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    14 | 0x0000 0000 0000 4000 | IPv4
    33 | 0x0000 0002 0000 0000 | Acquired packet length < packet length in L3 header
    44 | 0x0000 1000 0000 0000 | Duplicate IP ID

$

As indicated by the endreport, the packets are clipped, and if you look at the header description, we got ipv4 in ipv4, or ipv6 packets, so the tcp header is clipped. Just take the $pktNo or extract the flow index with pcapd and look with wireshark, if you don’t believe the Anteater.

$ tawk 'bitsanyset($tcpAnomaly, 0x0040)' annoloc2_packets.txt | head -n 20 | tcol
%pktNo  flowInd  flowStat            time               pktIAT    flowDuration  numHdrs  hdrDesc            ethVlanID  srcMac             dstMac             ethType  srcIP                                    srcIPCC  srcIPWho             srcPort  dstIP                                    dstIPCC  dstIPWho             dstPort  l4Proto  ipTOS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq         ack         tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpOptLen  tcpOpts  l7Content
5132    852      0x0000000a00024000  1022171701.803251  0.000000  0.000000      4        eth:ipv4:ipv4:tcp             00:d0:02:6d:78:00  00:00:21:ef:64:92  0x0800   201.98.74.248                            mx       Uninet S.A. de C.V.  5642     19.54.248.131                            us       Ford Motor Company   997      6        0x00   59907  0         0x4000  57     0x375f       0x0000       0x0000       0x0000       0x1840   0                       0                     0                 4122508806  140052278   0x0101    0x00      0x0040      0       0                   
5501    887      0x0000000a00008000  1022171701.811168  0.000000  0.000000      3        eth:ipv6:tcp                  00:60:08:2c:ca:8e  00:40:05:56:05:f0  0x86dd   3ffe:7c9b:e2:4ca6:4c::b0                 --       --                   48458    3ffe:7c9b:f5:8b05::2f50                  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 1014442254  2518458740  0x0101    0x00      0x0040      0       0                   
9876    852      0x0000000a00024001  1022171701.903727  0.000000  0.000000      4        eth:ipv4:ipv4:tcp             00:00:21:ef:64:92  00:d0:02:6d:78:00  0x0800   19.54.248.131                            us       Ford Motor Company   997      201.98.74.248                            mx       Uninet S.A. de C.V.  5642     6        0x00   1306   0         0x4000  64     0x1596       0x0000       0x0000       0x0000       0x1840   0                       0                     0                 140052278   4122508883  0x0101    0x00      0x0040      0       0                   
24426   1514     0x0000000a00008000  1022171702.203819  0.000000  0.000000      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2584580845  2580084521  0x0101    0x00      0x0040      0       0                   
24437   1514     0x0000000a00008001  1022171702.204108  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2580084521  2584580875  0x0101    0x00      0x0040      0       0                   
25555   852      0x0000000a00024001  1022171702.226012  0.322285  0.322285      4        eth:ipv4:ipv4:tcp             00:00:21:ef:64:92  00:d0:02:6d:78:00  0x0800   19.54.248.131                            us       Ford Motor Company   997      201.98.74.248                            mx       Uninet S.A. de C.V.  5642     6        0x00   58190  56884     0x4000  64     0x36fc       0x0000       0x0000       0x0000       0x1840   0                       0                     0                 140052278   4122508883  0x0100    0x00      0x0040      0       0                   
31166   852      0x0000000a00024000  1022171702.336097  0.532846  0.532846      4        eth:ipv4:ipv4:tcp             00:d0:02:6d:78:00  00:00:21:ef:64:92  0x0800   201.98.74.248                            mx       Uninet S.A. de C.V.  5642     19.54.248.131                            us       Ford Motor Company   997      6        0x00   63400  3493      0x4000  57     0x2a07       0x0000       0x0000       0x0000       0x1840   0                       0                     0                 4122508883  140052379   0x0100    0x00      0x0040      0       0                   
31572   1514     0x0000000a00008001  1022171702.344368  0.140260  0.140260      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2580084521  2584580875  0x0100    0x00      0x0040      0       0                   
31598   1514     0x0000000a00008000  1022171702.344744  0.140925  0.140925      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2584580875  2580084583  0x0100    0x00      0x0040      0       0                   
50204   1514     0x0000000a00008000  1022171702.726427  0.381683  0.522608      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2584580875  2580084583  0x0100    0x00      0x0040      0       0                   
50216   1514     0x0000000a00008001  1022171702.726437  0.382069  0.522329      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2580084583  2584580923  0x0100    0x00      0x0040      0       0                   
51918   2014     0x0000000a00008000  1022171702.761268  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:08:2c:ca:8e  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:37ff:a296:305e  --       --                   43441    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 1243673968  1230389220  0x0101    0x00      0x0040      0       0                   
51924   2015     0x0000000a00008000  1022171702.761273  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:50:da:51:3a:cb  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     2001:70e8:d3ce:e200:de29:56ff:4bed:207d  --       --                   2019     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 812694489   794313900   0x0101    0x00      0x0040      0       0                   
51932   2017     0x0000000a00008000  1022171702.761279  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:a0:c9:91:ff:b8  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de8b:71ff:caa5:ff02  --       --                   32770    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2900618548  2966996295  0x0101    0x00      0x0040      0       0                   
51935   2018     0x0000000a00008000  1022171702.761282  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:d0:b7:e8:2d:3f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:dec7:1cff:dc30:cc6f  --       --                   42686    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 1889731448  1977700919  0x0101    0x00      0x0040      0       0                   
51937   2015     0x0000000a00008001  1022171702.762093  0.000000  0.000000      3        eth:ipv6:tcp                  00:50:da:51:3a:cb  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de29:56ff:4bed:207d  --       --                   2019     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 794313900   812694584   0x0101    0x00      0x0040      0       0                   
51939   2019     0x0000000a00008000  1022171702.762094  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:10:4b:c6:e4:be  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     2001:70e8:d3ce:e200:de56:abff:19a1:c470  --       --                   59975    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 2842260568  2812244838  0x0101    0x00      0x0040      0       0                   
51947   2020     0x0000000a00008000  1022171702.762100  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:01:02:af:4a:b4  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de45:6dff:c7ad:c251  --       --                   50942    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 1943392398  1869635866  0x0101    0x00      0x0040      0       0                   
51950   2021     0x0000000a00008000  1022171702.762102  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:08:51:96:fa  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:37ff:a2c1:225b  --       --                   1739     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x0000       0x1800   0                       0                     0                 777085060   780206791   0x0101    0x00      0x0040      0       0                   
$

tcpFlags all on

In the following all functions will be switched on in tcpFlags. So we only need to run T2 once and then select the appropriate colums in the flow and packet files corresponding to the topic at hand.

Open tcpFlags.h and set all user constants to the default state as indicated below:

$ tcpFlags
$ vi src/tcpFlags.h
...
// -s option
#define SPKTMD_SEQACKREL 0 // -s option SEQ/ACK Numbers 0: absolute, 1: relative
#define SPKTMD_SEQACKHEX 0 // -s option SEQ/ACK Numbers 0: uint32_t, 1: hex32

// user defined constants 
#define RTT_ESTIMATE     1 // 1: Round trip time estimation
#define IPCHECKSUM       2 // 1: Calculation of L3 (IP) header checksum,
                           // 2: L3 + L4 (TCP,UDP) checksum
#define WINDOWSIZE       1 // 1: Calculation of TCP window size parameters
#define WINMIN           1 // Minimal window size threshold defining a healthy communication, packets below are counted
#define SEQ_ACK_NUM      1 // 1: SEQ/ACK number feature analysis
#define FRAG_ANALYZE     1 // 1: Fragmentation analysis
#define NAT_BT_EST       1 // 1: NAT boot time estimation
#define SCAN_DETECTOR    1 // 1: Scan flow detector
...

Now compile tcpFlags and rerun T2 with the -s option to also produce a packet file.

$ t2build tcpFlags
...
$ t2 -r ~/data/annoloc2.pcap -w ~/results/ -s
...
---------------------------------------------------------------------------------------
tcpFlags: Aggregated ipFlags: 0x3966
tcpFlags: Aggregated tcpAnomaly: 0xff47
tcpFlags: Number of TCP scans, succ scans, syn retries, seq retries: 685, 2569 (2.57 K), 114, 933
tcpFlags: Number WinSz below 1: 2415 (2.42 K) [0.25%]
tcpStates: Aggregated anomaly flags: 0xdf
---------------------------------------------------------------------------------------
...
$

Several bits provide an indication about abnormalities in the IPv4/6 header, such as IPv4 options, fragmentation problems or attacks, L3/4 16Bit checksum problems. L4 protocols such as SCTP using Adler32 or even better CRC checks will be calculated in sctpDecode.

The next two bit fields provide aggregated information about TCP flags and anomaly flags

$ tawk -V ipFlags=0x3966

The ipFlags column with value 0x3966 is to be interpreted as follows:

   bit | ipFlags | Description
   =============================================================================
     1 | 0x0002  | IPv4 packets out of order
     2 | 0x0004  | IPv4 ID roll over
     5 | 0x0020  | More Fragment bit
     6 | 0x0040  | IPv4: Don't Fragment bit, IPv6: reserve bit
     8 | 0x0100  | Fragmentation position error
    11 | 0x0800  | L4 checksum error
    12 | 0x1000  | L3 header length snapped
    13 | 0x2000  | Packet interdistance = 0

$ 

The bits match their position in the IP header. It gives you an overview which flags occur. If ECE or CWR are set, we know that there are throughput problems in the traffic. The presents of the PSH flag denotes that an application does not want to wait for a buffer to fill, either a strong indication for Windows OS or multimedia traffic. RST denotes an unexpected break in communication, which can happen, mostly benign.

tawk -V tcpAnomaly=0xff47

The tcpAnomaly column with value 0xff47 is to be interpreted as follows:

   bit | tcpAnomaly | Description
   =============================================================================
     0 | 0x0001     | FIN-ACK flag
     1 | 0x0002     | SYN-ACK flag
     2 | 0x0004     | RST-ACK flag
     6 | 0x0040     | Null flag, potential NULL scan packet, or malicious channel
     8 | 0x0100     | L4 option field corrupt or not acquired
     9 | 0x0200     | SYN retransmission
    10 | 0x0400     | Sequence Number retry
    11 | 0x0800     | Sequence Number out of order
    12 | 0x1000     | Sequence mess in flow order due to pcap packet loss
    13 | 0x2000     | Sequence number jump forward
    14 | 0x4000     | ACK number out of order
    15 | 0x8000     | Duplicate ACK

$

If the timestamp options are selected again. Spot all the anomalies per flow but beware, the traffic is clipped.

$ tawk 'bitsanyset($tcpOptions, 0x00000100)' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipTOS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN    tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpTmS      tcpTmER     tcpEcI  tcpUtm           tcpBtm             tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStates
A     2984     0x0000100200004000  1022171704.302738  1022171704.494135  0.191397  1           3        eth:ipv4:tcp  00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800              138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0           14454       64        64        0         0x00   0x1842   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1805940851  7           662             3               6           519                    0               5840          2189.454     0            6432         3               1              2                  0.2142857     0x5f      0x8101      10            30         0x00000102  0       1      69386246    2835653573  0.01    693862.444491    1021477842.045133  0              2e-06             0.106798          0.01663386        0.02559675           4e-06         -1               0x52
B     2984     0x0000100200004001  1022171704.302742  1022171704.489628  0.186886  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800              138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x00d4    0           0           63        63        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1803485413  11          573             0               7           663                    2               5792          6278.336     5792         6432         0               1              1                  0             0x5b      0x8103      10            30         0x00000102  0       1      2835653584  69386246    0.01    28356535.206182  993815169.283446   4e-06          4e-06             0.006369          0.002007818       0.002346763          0.01864167    0.02570411       0x02
A     3415     0x0000000200004000  1022171705.224036  1022171705.383786  0.159750  1           3        eth:ipv4:tcp  00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800              138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    1           42596       64        64        0         0x00   0x1842   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1813158365  7           1304            3               6           518                    0               5840          4468.272     0            6432         1               1              2                  0.08333334    0x5f      0x8101      10            30         0x00000102  0       1      69386334    2775938367  0.01    693863.324491    1021477842.050055  0              0.005417          0.084087          0.02240858        0.02751403           2e-06         -1               0x42
B     3415     0x0000100200004001  1022171705.224038  1022171705.374555  0.150517  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800              138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x00d4    0           0           63        63        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1815487924  10          572             0               7           1305                   2               5792          7307.699     5792         8099         0               1              1                  0             0x5b      0x8103      9             27         0x00000102  0       1      2775938375  69386334    0.01    27759383.129529  994412322.245026   2e-06          1e-06             0.005877          0.0018579         0.002009845          0.02426648    0.02758734       0x02
A     3477     0x0000000200004000  1022171705.346276  1022171705.504551  0.158275  1           3        eth:ipv4:tcp  00:01:02:b7:bb:d4  00:d0:02:6d:78:00  0x0800              138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x0054    1           1           64        64        0         0x08   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1807265661  6           8087            2               0           0                      1               5840          5840         5840         5840         0               0              0                  0             0xdb      0x8101      8             24         0x00000102  0       1      1061542895  340264      0.01    10615428.712727  1011556276.791825  0              0.000604          0.009891          0.002937333       0.003204388          0.020928      -1               0x06
B     3477     0x0000000200004001  1022171705.367204  1022171705.559621  0.192417  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:01:02:b7:bb:d4  0x0800              201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x0054    1           3           120       120       0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2926843921  5           0               0               4           9123                   1               17680         12158.86     0            17680        2               0              0                  0.1666667     0xd6      0x0102      4             12         0x00000102  0       1      340264      1061542895  0.1     34026.400507     1022137679.155481  0.020928       0.020928          0.0663            0.0477415         0.01142701           0.05067883    0.0118678        0x46
A     2811     0x0000000000004000  1022171706.205501  1022171706.999557  0.794056  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:50:fc:3b:62:78  0x0800              216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0054    1           12          51        242       1         0x00   0x0840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3658275921  2           0               0               0           0                      1               31856         15609.44     0            31856        2               0              0                  0.5           0x95      0x0001      2             6          0x00000102  0       1      114631611   785166946   0.01    1146316.084378   1021025390.893442  0              0.612778          2.241281          1.029833          0.5246621            0             -1               0x53
B     2811     0x0000000200004001  1022171703.964220  1022171706.365041  2.400821  1           3        eth:ipv4:tcp  00:50:fc:3b:62:78  00:d0:02:6d:78:00  0x0800              138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x0054    1           1           64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1443222362  3           2896            0               0           0                      0               6432          6432         6432         6432         0               0              0                  0             0x90      0x8000      3             9          0x00000102  0       1      785167185   114631558   0.01    7851671.674501   1014320034.690540  0              0.146206          0.15954           0.1019153         0.05374213           0             -1               0x03
A     4982     0x0000000200004000  1022171709.624612  1022171709.687159  0.062547  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    256         256         48        48        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2793035386  2           607             4               2           191                    2               16384         17366.7      16384        17520        1               2              3                  0             0x5b      0x8101      6             18         0x00000102  0       1      416107719   122103994   0.01    4161077.096993   1018010632.590167  0              0.015591          0.017495          0.01418957        0.004487047          0.001637      -1               0x0a
B     4982     0x0000000200004001  1022171709.626249  1022171709.687168  0.060919  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x0054    1           24          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1428902644  3           190             1               2           608                    2               17376         12163.2      0            17376        1               0              0                  0.2           0x5f      0x0103      3             9          0x00000102  0       1      122103994   416107717   0.01    1221039.912708   1020950669.757342  0.001637       9e-06             0.009642          0.0024842         0.002829067          0.01667377    0.005304452      0x4a
A     5035     0x0000000200004000  1022171709.784259  1022171709.836533  0.052274  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    256         768         48        48        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3050959365  2           606             4               2           191                    2               16384         17366.7      16384        17520        1               2              3                  0             0x5b      0x8101      6             18         0x00000102  0       1      416106512   122104009   0.01    4161065.026993   1018010644.809541  0              0.015441          0.016874          0.01371386        0.004394183          0.002242      -1               0x0a
B     5035     0x0000000200004001  1022171709.786501  1022171709.837980  0.051479  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x0054    1           17          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  531896816   3           190             1               2           607                    2               17376         12163.2      0            17376        1               0              0                  0.2           0x5f      0x0103      3             9          0x00000102  0       1      122104009   416106510   0.01    1221040.062708   1020950669.758146  0.002242       0.000763          0.002242          0.0012948         0.0004338192         0.01500866    0.004415546      0x4a
A     5036     0x0000000200004000  1022171709.786494  1022171709.836536  0.050042  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    256         768         48        48        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3051015772  2           602             4               2           191                    2               16384         17366.7      16384        17520        1               2              3                  0             0x5b      0x8101      6             18         0x00000102  0       1      416106512   122104009   0.01    4161065.026993   1018010644.809544  0              0.01545           0.016871          0.01371414        0.004393462          1e-05         -1               0x0a
B     5036     0x0000000200004001  1022171709.786504  1022171709.837981  0.051477  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x0054    1           16          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1004739114  3           190             1               2           603                    2               17376         12163.2      0            17376        1               0              0                  0.2           0x5f      0x0103      3             9          0x00000102  0       1      122104009   416106510   0.01    1221040.062708   1020950669.758149  1e-05          1e-05             0.001445          0.0008488         0.0003890621         0.01456294    0.004410655      0x4a
A     5083     0x0000000200004000  1022171709.912864  1022171709.969239  0.056375  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:00:e8:8f:b7:93  0x0800              201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    256         768         48        48        0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1823582276  2           603             4               2           191                    2               16384         17366.7      16384        17520        1               2              3                  0             0x5b      0x8101      6             18         0x00000102  0       1      416112905   122104022   0.01    4161128.956992   1018010581.012248  0              0.01591           0.019702          0.01492329        0.004987202          8e-06         -1               0x0a
B     5083     0x0000000200004001  1022171709.912872  1022171709.969252  0.056380  1           3        eth:ipv4:tcp  00:00:e8:8f:b7:93  00:d0:02:6d:78:00  0x0800              138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x0054    1           21          64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3262093770  3           190             1               2           604                    2               17376         12163.2      0            17376        1               0              0                  0.2           0x5f      0x0103      3             9          0x00000102  0       1      122104022   416112904   0.01    1221040.192708   1020950669.756830  8e-06          8e-06             0.002355          0.0004854         0.0007110852         0.01540869    0.005037641      0x4a
A     24       0x0000000200004000  1022171701.692693  1022171710.923400  9.230707  1           3        eth:ipv4:tcp  00:50:04:e8:e3:41  00:d0:02:6d:78:00  0x0800              138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x0054    1           4           64        64        0         0x08   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2542607003  862         1546464         30              0           0                      0               5840          5840         5840         5840         0               0              0                  0             0x19      0xa801      895           2685       0x00000102  0       1      113405465   80039       0.01    1134054.624652   1021037656.298749  0              2e-06             0.350364          0.002736811       0.01985572           0             -1               0x07
B     24       0x0000000200004001  1022171701.706576  1022171710.959603  9.253027  1           3        eth:ipv4:tcp  00:d0:02:6d:78:00  00:50:04:e8:e3:41  0x0800              36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x0054    1           6           117       117       0         0x00   0x3840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1969468418  537         0               0               392         1272105                144             16072         12261        0            17520        59              58             114                0.001858736   0x14      0x8100      378           1134       0x00000102  0       1      80039       113405465   0.1     8003.900119      1022163707.059475  0              1e-06             0.24066           0.01920191        0.03091631           0             -1               0x47
A     776      0x0000000200004000  1022171701.778309  1022171701.793349  0.015040  1           3        eth:ipv4:tcp  00:00:1c:b6:1a:e1  00:d0:02:6d:78:00  0x0800              138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x00d4    1           1           64        64        0         0x00   0x1840   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1804752764  3           124             2               3           2229                   1               5840          9188.096     5840         11584        0               2              1                  0             0x1b      0x8101      5             15         0x00000102  0       1      195084195   174526427   0.01    1950841.906395   1020220859.886954  0              0.000366          0.001362          0.0006781667      0.0003840317         0.005777      -1               0x00
$

The tcpFlags column indicates that some packets have the CWR set. Below is the decoded aggregated tcpFlags parameter.

$ tawk -V tcpFlags

The tcpFlags column is to be interpreted as follows:

   bit | tcpFlags | Description
   =============================================================================
     0 | 0x01     | FIN: No more data, finish connection
     1 | 0x02     | SYN: Synchronize sequence numbers
     2 | 0x04     | RST: Reset connection
     3 | 0x08     | PSH: Push data
     4 | 0x10     | ACK: Acknowledgement field value valid
     5 | 0x20     | URG: Urgent pointer valid
     6 | 0x40     | ECE: ECN-Echo
     7 | 0x80     | CWR: Congestion Window Reduced flag is set

$

Let’s select all packets where no tcp flags are set, which is clearly not normal

$ tawk 'bitsanyset($tcpAnomaly, 0x0040)' annoloc2_packets.txt | head -n 20 | tcol
pktNo  flowInd  flowStat            time               pktIAT    flowDuration  numHdrs  hdrDesc            ethVlanID  srcMac             dstMac             ethType  srcIP                                    srcIPCC  srcIPWho             srcPort  dstIP                                    dstIPCC  dstIPWho             dstPort  l4Proto  ipTOS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq         ack         seqDiff  ackDiff  seqPktLen  ackPktLen  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpTmS  tcpTmER  tcpOptLen  tcpOpts  l7Content
5132    852      0x0000000a00024000  1022171701.803251  0.000000  0.000000      4        eth:ipv4:ipv4:tcp             00:d0:02:6d:78:00  00:00:21:ef:64:92  0x0800   201.98.74.248                            mx       Uninet S.A. de C.V.  5642     19.54.248.131                            us       Ford Motor Company   997      6        0x00   59907  0         0x4000  57     0x375f       0x375f       0x0000       0xdf77       0x1840   0                       0                     0                 4122508806  140052278   0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
5501    887      0x0000000a00008000  1022171701.811168  0.000000  0.000000      3        eth:ipv6:tcp                  00:60:08:2c:ca:8e  00:40:05:56:05:f0  0x86dd   3ffe:7c9b:e2:4ca6:4c::b0                 --       --                   48458    3ffe:7c9b:f5:8b05::2f50                  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x7ce3       0x1800   0                       0                     0                 1014442254  2518458740  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
9876    852      0x0000000a00024001  1022171701.903727  0.000000  0.000000      4        eth:ipv4:ipv4:tcp             00:00:21:ef:64:92  00:d0:02:6d:78:00  0x0800   19.54.248.131                            us       Ford Motor Company   997      201.98.74.248                            mx       Uninet S.A. de C.V.  5642     6        0x00   1306   0         0x4000  64     0x1596       0x1596       0x0000       0xdfc4       0x1840   0                       0                     0                 140052278   4122508883  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
24426   1514     0x0000000a00008000  1022171702.203819  0.000000  0.000000      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0xa9a9       0x1800   0                       0                     0                 2584580845  2580084521  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
24437   1514     0x0000000a00008001  1022171702.204108  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0xa9c7       0x1800   0                       0                     0                 2580084521  2584580875  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
25555   852      0x0000000a00024001  1022171702.226012  0.322285  0.322285      4        eth:ipv4:ipv4:tcp             00:00:21:ef:64:92  00:d0:02:6d:78:00  0x0800   19.54.248.131                            us       Ford Motor Company   997      201.98.74.248                            mx       Uninet S.A. de C.V.  5642     6        0x00   58190  56884     0x4000  64     0x36fc       0x36fc       0x0000       0xbf24       0x1840   0                       0                     0                 140052278   4122508883  0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
31166   852      0x0000000a00024000  1022171702.336097  0.532846  0.532846      4        eth:ipv4:ipv4:tcp             00:d0:02:6d:78:00  00:00:21:ef:64:92  0x0800   201.98.74.248                            mx       Uninet S.A. de C.V.  5642     19.54.248.131                            us       Ford Motor Company   997      6        0x00   63400  3493      0x4000  57     0x2a07       0x2a07       0x0000       0xbf3c       0x1840   0                       0                     0                 4122508883  140052379   0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
31572   1514     0x0000000a00008001  1022171702.344368  0.140260  0.140260      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x5351       0x1800   0                       0                     0                 2580084521  2584580875  0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
31598   1514     0x0000000a00008000  1022171702.344744  0.140925  0.140925      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x5371       0x1800   0                       0                     0                 2584580875  2580084583  0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
50204   1514     0x0000000a00008000  1022171702.726427  0.381683  0.522608      3        eth:ipv6:tcp                  00:60:97:b9:10:2f  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0xfd08       0x1800   0                       0                     0                 2584580875  2580084583  0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
50216   1514     0x0000000a00008001  1022171702.726437  0.382069  0.522329      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:97:b9:10:2f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:c1ff:aee0:32d   --       --                   1031     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0xfd18       0x1800   0                       0                     0                 2580084583  2584580923  0        0        0          0          0x0040    0x00      0x0040      0       0       0        0                   
51918   2014     0x0000000a00008000  1022171702.761268  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:08:2c:ca:8e  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:37ff:a296:305e  --       --                   43441    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x1282       0x1800   0                       0                     0                 1243673968  1230389220  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51924   2015     0x0000000a00008000  1022171702.761273  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:50:da:51:3a:cb  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     2001:70e8:d3ce:e200:de29:56ff:4bed:207d  --       --                   2019     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x59f3       0x1800   0                       0                     0                 812694489   794313900   0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51932   2017     0x0000000a00008000  1022171702.761279  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:a0:c9:91:ff:b8  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de8b:71ff:caa5:ff02  --       --                   32770    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0xe152       0x1800   0                       0                     0                 2900618548  2966996295  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51935   2018     0x0000000a00008000  1022171702.761282  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:d0:b7:e8:2d:3f  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:dec7:1cff:dc30:cc6f  --       --                   42686    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x571f       0x1800   0                       0                     0                 1889731448  1977700919  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51937   2015     0x0000000a00008001  1022171702.762093  0.000000  0.000000      3        eth:ipv6:tcp                  00:50:da:51:3a:cb  00:80:48:cd:88:83  0x86dd   2001:70e8:d3ce:e200:de29:56ff:4bed:207d  --       --                   2019     2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x5a52       0x1800   0                       0                     0                 794313900   812694584   0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51939   2019     0x0000000a00008000  1022171702.762094  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:10:4b:c6:e4:be  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6668     2001:70e8:d3ce:e200:de56:abff:19a1:c470  --       --                   59975    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x931e       0x1800   0                       0                     0                 2842260568  2812244838  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51947   2020     0x0000000a00008000  1022171702.762100  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:01:02:af:4a:b4  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de45:6dff:c7ad:c251  --       --                   50942    6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x2542       0x1800   0                       0                     0                 1943392398  1869635866  0        0        0          0          0x0041    0x00      0x0040      0       0       0        0                   
51950   2021     0x0000000a00008000  1022171702.762102  0.000000  0.000000      3        eth:ipv6:tcp                  00:80:48:cd:88:83  00:60:08:51:96:fa  0x86dd   2001:70e8:d3ce:e200:deaf:b9ff:1f0e:becf  --       --                   6667     2001:70e8:d3ce:e200:de10:37ff:a2c1:225b  --       --                   1739     6        0x00   0      0         0x0000  64     0x0000       0x0000       0x0000       0x205a       0x1800   0                       0                     0                 777085060   780206791   0        0        0          0          0x0041    0x00      0x0040      0       0       0        0  
$

WTF is that? Malicious scan? Or is it due to clipping?

RTT estimate

The Round Trip Time(RTT) estimate is a vital tool for troubleshooting. The mode is controlled by RTT_ESTIMATE in tcpFlags. The RTT features are estimated for all L4 protocols and gives additional information about tcp connection anomalies such as ack retries and syn connection timeout retries.

  • $tcpSSASAATrip denotes the RTT during the tcp connection phase, which has no influence of the src and dst host.
  • $tcpSSASAATrip denotes the Time from the measurement point to the dst host and back.
  • $tcpRTTAckTripMin, $tcpRTTAckTripMax, $tcpRTTAckTripAve denote the minimal, maximal and average Trip time
  • $tcpRTTSseqAA denotes the total RTT, which also includes delays at the hosts
  • $tcpRTTAckJitAve denotes the average jitter, useful for voice communication, see also the voipDetector plugin

RTT can also be estimated from the TCP timestamp option. Using the MSS, which is supplied in the tcp options part and the RTT a bandwidth / flow can be calculated.

$ tawk '{if (bitsanyset($tcpOptions, 0x00000100)) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $l4Proto, $tcpFStat, $ipFlags, $tcpFlags, $tcpAnomaly, $tcpSSASAATrip, $tcpRTTAckTripMin, $tcpRTTAckTripMax, $tcpRTTAckTripAve, $tcpRTTAckTripJitAve, $tcpRTTSseqAA, $tcpRTTAckJitAve}' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  tcpFStat  ipFlags  tcpFlags  tcpAnomaly  tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve
A     2984     0x0000100200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101      0              2e-06             0.106798          0.01663386        0.02559675           4e-06         -1
B     2984     0x0000100200004001  138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x00d4    0x3840   0x5b      0x8103      4e-06          4e-06             0.006369          0.002007818       0.002346763          0.01864167    0.02570411
A     3415     0x0000000200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101      0              0.005417          0.084087          0.02240858        0.02751403           2e-06         -1
B     3415     0x0000100200004001  138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x00d4    0x1840   0x5b      0x8103      2e-06          1e-06             0.005877          0.0018579         0.002009845          0.02426648    0.02758734
A     3477     0x0000000200004000  138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x0054    0x1840   0xdb      0x8101      0              0.000604          0.009891          0.002937333       0.003204388          0.020928      -1
B     3477     0x0000000200004001  201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x0054    0x1840   0xd6      0x0102      0.020928       0.020928          0.0663            0.0477415         0.01142701           0.05067883    0.0118678
A     2811     0x0000000000004000  216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0054    0x0840   0x95      0x0001      0              0.612778          2.241281          1.029833          0.5246621            0             -1
B     2811     0x0000000200004001  138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x0054    0x1840   0x90      0x8000      0              0.146206          0.15954           0.1019153         0.05374213           0             -1
A     4982     0x0000000200004000  201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101      0              0.015591          0.017495          0.01418957        0.004487047          0.001637      -1
B     4982     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x0054    0x1840   0x5f      0x0103      0.001637       9e-06             0.009642          0.0024842         0.002829067          0.01667377    0.005304452
A     5035     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101      0              0.015441          0.016874          0.01371386        0.004394183          0.002242      -1
B     5035     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x0054    0x1840   0x5f      0x0103      0.002242       0.000763          0.002242          0.0012948         0.0004338192         0.01500866    0.004415546
A     5036     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101      0              0.01545           0.016871          0.01371414        0.004393462          1e-05         -1
B     5036     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x0054    0x1840   0x5f      0x0103      1e-05          1e-05             0.001445          0.0008488         0.0003890621         0.01456294    0.004410655
A     5083     0x0000000200004000  201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101      0              0.01591           0.019702          0.01492329        0.004987202          8e-06         -1
B     5083     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x0054    0x1840   0x5f      0x0103      8e-06          8e-06             0.002355          0.0004854         0.0007110852         0.01540869    0.005037641
A     24       0x0000000200004000  138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x0054    0x1840   0x19      0xa801      0              2e-06             0.350364          0.002736811       0.01985572           0             -1
B     24       0x0000000200004001  36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x0054    0x3840   0x14      0x8100      0              1e-06             0.24066           0.01920191        0.03091631           0             -1
A     776      0x0000000200004000  138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x00d4    0x1840   0x1b      0x8101      0              0.000366          0.001362          0.0006781667      0.0003840317         0.005777      -1
$

L3/4 Checksums

Looking at Checksums reveals whether there are manipulations of the L4header or the content. Moreover you can determine whether a pcap is acquired on a computer itself or at a network intercept, such as a span port. Only works, if the checksum off load option is present and activated on the HW. Think how a checksum would look like if a pcap is acquired on a computer.

As the end report ipFlags: 0x3966 and tcpAnomaly: 0xff47 indicate, there are lots of broken packets, due to anonymization. If you look at the ipFlags and tcpAnomaly column you see the status of the each flow and you can select them using tawk.

In the flowStat column all flows have a L3 packet length field warning, resulting in wrong checksums. Why? Look at $flowStat.

$ tawk '{if (bitsanyset($tcpOptions, 0x00000100)) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $l4Proto, $tcpFStat, $ipFlags, $tcpFlags, $tcpAnomaly}' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  tcpFStat  ipFlags  tcpFlags  tcpAnomaly
A     2984     0x0000100200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101
B     2984     0x0000100200004001  138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x00d4    0x3840   0x5b      0x8103
A     3415     0x0000000200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101
B     3415     0x0000100200004001  138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x00d4    0x1840   0x5b      0x8103
A     3477     0x0000000200004000  138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x0054    0x1840   0xdb      0x8101
B     3477     0x0000000200004001  201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x0054    0x1840   0xd6      0x0102
A     2811     0x0000000000004000  216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0054    0x0840   0x95      0x0001
B     2811     0x0000000200004001  138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x0054    0x1840   0x90      0x8000
A     4982     0x0000000200004000  201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101
B     4982     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x0054    0x1840   0x5f      0x0103
A     5035     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101
B     5035     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x0054    0x1840   0x5f      0x0103
A     5036     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101
B     5036     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x0054    0x1840   0x5f      0x0103
A     5083     0x0000000200004000  201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101
B     5083     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x0054    0x1840   0x5f      0x0103
A     24       0x0000000200004000  138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x0054    0x1840   0x19      0xa801
B     24       0x0000000200004001  36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x0054    0x3840   0x14      0x8100
A     776      0x0000000200004000  138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x00d4    0x1840   0x1b      0x8101
$

Below the flowInd = 2984 is extracted to indicate that each packet has a wrong checksum, compare l4HdrChkSum and l4CalChkSum. The L3 Checksum add up.

$ tawk 'flow(2984)' annoloc2_packets.txt | tcol
%pktNo  flowInd  flowStat            time               pktIAT    flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP            srcIPCC  srcIPWho                     srcPort  dstIP            dstIPCC  dstIPWho                     dstPort  l4Proto  ipTOS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq         ack         seqDiff  ackDiff  seqPktLen  ackPktLen  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpTmS      tcpTmER     tcpOptLen  tcpOpts                                                                                              l7Content
127191  2984     0x0000000200004000  1022171704.302738  0.000000  0.000000      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51072  0         0x4000  64     0x8c5b       0x8c5b       0x4b80       0x18f1       0x1840   0                       0                     0                 1805940851  0           0        0        0          0          0x0041    0x02      0x0100      5840    0           0           20         0x02;0x04;0x05;0xb4;0x04;0x02;0x08;0x0a;0x04;0x22;0xbf;0xf3;0x00;0x00;0x00;0x00;0x00;0x00;0xc1;0x00  
127197  2984     0x0000000200004001  1022171704.302742  0.000000  0.000000      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54dc       0x54dc       0x9338       0x18f1       0x1840   0                       0                     0                 1803485413  1805940852  0        0        0          0          0x0061    0x52      0x0102      5792    0           0           20         0x02;0x04;0x05;0xb4;0x04;0x02;0x08;0x0a;0xa9;0x04;0xa3;0xbd;0x00;0x00;0x00;0x00;0x00;0x00;0xc1;0x00  
127728  2984     0x0000000200004000  1022171704.312314  0.009576  0.009576      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51073  1         0x4000  64     0x8c62       0x8c62       0xc1cc       0x86bd       0x0840   0                       0                     0                 1805940852  1803485414  0        0        0          0          0x0050    0x50      0x0000      5840    69386228    2835653565  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xf4;0xa9;0x04;0xa3;0xbd                                          
128012  2984     0x0000000200004001  1022171704.318407  0.015665  0.015665      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x5485       0x5485       0xce55       0x318b       0x1840   0                       0                     0                 1803485414  1805940852  0        0        0          0          0x0070    0x58      0x8000      5792    2835653567  69386228    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xbf;0x04;0x22;0xbf;0xf4                                          
128518  2984     0x0000000200004000  1022171704.328953  0.016639  0.026215      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51074  1         0x4000  64     0x8c61       0x8c61       0xc169       0x865a       0x0840   0                       0                     0                 1805940852  1803485509  0        95       0          95         0x0050    0x50      0x0000      5840    69386230    2835653567  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xf6;0xa9;0x04;0xa3;0xbf                                          
128538  2984     0x0000000200004000  1022171704.329610  0.000657  0.026872      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51075  1         0x4000  64     0x8c3d       0x8c3d       0x7922       0x9f30       0x1840   0                       0                     0                 1805940852  1803485509  0        0        0          95         0x0050    0x58      0x8000      5840    69386230    2835653567  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xf6;0xa9;0x04;0xa3;0xbf                                          
128546  2984     0x0000100200004001  1022171704.329616  0.011209  0.026874      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54e4       0x54e4       0xc175       0x8666       0x0840   0                       0                     0                 1803485509  1805940887  95       35       95         35         0x0050    0x50      0x0000      5792    2835653568  69386230    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc0;0x04;0x22;0xbf;0xf6                                          
128553  2984     0x0000100200004001  1022171704.329622  0.000006  0.026880      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x541e       0x541e       0xbeb1       0x9e99       0x1840   0                       0                     0                 1803485509  1805940887  0        0        95         35         0x0050    0x58      0x8000      5792    2835653568  69386230    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc0;0x04;0x22;0xbf;0xf6                                          
129128  2984     0x0000000200004000  1022171704.341855  0.012245  0.039117      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51076  1         0x4000  64     0x8c27       0x8c27       0x68c2       0xb7f1       0x1840   0                       0                     0                 1805940887  1803485707  35       198      35         293        0x00d0    0x58      0x0000      6432    69386231    2835653568  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xf7;0xa9;0x04;0xa3;0xc0                                          
129478  2984     0x0000100200004001  1022171704.348224  0.018602  0.045482      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54a8       0x54a8       0xed96       0xb756       0x1840   0                       0                     0                 1803485707  1805940943  198      56       293        91         0x0050    0x58      0x0000      5792    2835653570  69386231    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc2;0x04;0x22;0xbf;0xf7                                          
129917  2984     0x0000000200004000  1022171704.357746  0.015891  0.055008      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51077  1         0x4000  64     0x8c31       0x8c31       0xcb9f       0xd0bd       0x1840   0                       0                     0                 1805940943  1803485767  56       60       91         353        0x00d0    0x58      0x0000      6432    69386233    2835653570  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xf9;0xa9;0x04;0xa3;0xc2                                          
130029  2984     0x0000100200004001  1022171704.360412  0.012188  0.057670      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54a5       0x54a5       0x6468       0xd010       0x1840   0                       0                     0                 1803485767  1805940988  60       45       353        136        0x0050    0x58      0x0000      5792    2835653571  69386233    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc3;0x04;0x22;0xbf;0xf9                                          
130382  2984     0x0000000200004000  1022171704.367598  0.009852  0.064860      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51078  1         0x4000  64     0x8c57       0x8c57       0x17ae       0xe9b0       0x1840   0                       0                     0                 1805940988  1803485830  45       63       136        416        0x00d0    0x58      0x0000      6432    69386234    2835653571  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xfa;0xa9;0x04;0xa3;0xc3                                          
130441  2984     0x0000100200004001  1022171704.368228  0.007816  0.065486      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54b2       0x54b2       0x6b5f       0xe8d7       0x1840   0                       0                     0                 1803485830  1805940994  63       6        416        142        0x0050    0x58      0x0000      5792    2835653572  69386234    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc4;0x04;0x22;0xbf;0xfa                                          
130817  2984     0x0000000200004000  1022171704.376527  0.008929  0.073789      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51079  1         0x4000  64     0x8a59       0x8a59       0x16ed       0x00a7       0x1840   0                       0                     0                 1805940994  1803485880  6        50       142        466        0x00d0    0x58      0x0000      6432    69386235    2835653572  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xbf;0xfb;0xa9;0x04;0xa3;0xc4                                          
131162  2984     0x0000100200004001  1022171704.382810  0.014582  0.080068      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54ae       0x54ae       0x3627       0x019b       0x1840   0                       0                     0                 1803485880  1805941509  50       515      466        657        0x00d0    0x58      0x0000      6432    2835653573  69386235    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xc5;0x04;0x22;0xbf;0xfb                                          
133164  2984     0x0000000200004000  1022171704.426806  0.050279  0.124068      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51080  1         0x4000  64     0x8c5b       0x8c5b       0xbacf       0x7fc0       0x0840   0                       0                     0                 1805941509  1803485934  515      54       657        520        0x00d0    0x50      0x0000      6432    69386240    2835653573  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xc0;0x00;0xa9;0x04;0xa3;0xc5                                          
135844  2984     0x0000000200004000  1022171704.489608  0.062802  0.186870      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51081  1         0x4000  64     0x8c54       0x8c54       0x1308       0x98b3       0x1840   0                       0                     0                 1805941509  1803485934  0        0        657        520        0x00d0    0x58      0x8000      6432    69386246    2835653573  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xc0;0x06;0xa9;0x04;0xa3;0xc5                                          
135860  2984     0x0000100200004001  1022171704.489621  0.106811  0.186879      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54ae       0x54ae       0xbdbe       0x1a5e       0x1840   0                       0                     0                 1803485934  1805941515  54       6        520        663        0x00d0    0x58      0x0000      6432    2835653584  69386246    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xd0;0x04;0x22;0xc0;0x06                                          
135863  2984     0x0000000200004000  1022171704.489623  0.000015  0.186885      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   51082  1         0x4000  64     0x8c59       0x8c59       0xbac2       0x7fb3       0x0840   0                       0                     0                 1805941515  1803485934  6        0        663        520        0x00d0    0x51      0x0001      6432    69386246    2835653573  12         0x01;0x01;0x08;0x0a;0x04;0x22;0xc0;0x06;0xa9;0x04;0xa3;0xc5                                          
135869  2984     0x0000100200004001  1022171704.489628  0.000007  0.186886      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54e4       0x54e4       0xba81       0x7f72       0x0840   0                       0                     0                 1803485988  1805941515  54       0        574        663        0x00d0    0x51      0x0001      6432    2835653584  69386246    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xd0;0x04;0x22;0xc0;0x06                                          
135870  2984     0x0000100200004001  1022171704.489628  0.000000  0.186886      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:d0:b7:e8:9e:bb  0x0800   138.212.18.149   jp       Asahi Kasei Networks Corpor  25       138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    6        0x00   0      0         0x4000  63     0x54e4       0x54e4       0xba80       0x7f71       0x2840   0                       0                     0                 1803485989  1805941516  0        1        574        664        0x00d0    0x50      0x0000      6432    2835653584  69386246    12         0x01;0x01;0x08;0x0a;0xa9;0x04;0xa3;0xd0;0x04;0x22;0xc0;0x06                                          
136098  2984     0x0000000200004000  1022171704.494132  0.004509  0.191394      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   0      14454     0x4000  64     0x53f0       0x53f0       0x8c67       0x5158       0x0842   0                       0                     0                 1805941515  0           0        0        663        520        0x0050    0x44      0x0000      0       69386246    2835653573  0                                                                                                               
136100  2984     0x0000100200004000  1022171704.494134  0.000002  0.191396      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   0      0         0x4000  64     0x53f0       0x53f0       0x8c67       0x5158       0x0840   0                       0                     0                 1805941515  0           0        0        663        520        0x0050    0x44      0x0000      0       69386246    2835653573  0                                                                                                               
136101  2984     0x0000100200004000  1022171704.494135  0.000001  0.191397      3        eth:ipv4:tcp             00:d0:b7:e8:9e:bb  00:d0:02:6d:78:00  0x0800   138.212.190.162  jp       ASAHI KASEI CORPORATION      37023    138.212.18.149   jp       Asahi Kasei Networks Corpor  25       6        0x00   0      0         0x4000  64     0x53f0       0x53f0       0x8c66       0x5157       0x0840   0                       0                     0                 1805941516  0           0        0        663        520        0x0050    0x44      0x0000      0       69386246    2835653573  0 
$

Window size features

The tcp window size is part of tcp flow control receiving buffer and gives an indication about the digestive performance of the dstination host. The initial window size in combination with the TTL is still a reasonable feature to estimate the type of OS. If $tcpMinWinSz hits 0 the buffer of the receiving host is full, and the sender has to wait.

As in the end report the variable $tcpWinSzThRt counts the occurences where the window size drops below a configurable threshold WINMIN in tcpFlags.h; default 1. So it counts how often the receiver performs backpressure to the sender. The parameters $tcpWinSzDwnCnt, $tcpWinSzUpCnt, $tcpWinSzChgDirCnt are experimental, trying to aggregate the evolution of the Window size, which is governed by several algorithms. They denote the up or down count/packet of the window size and how often these counts reverse the direction. Giving an indication about the unsteadiness of the regulation for the specific flow.

$ tawk '{if (bitsanyset($tcpOptions, 0x00000100)) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $l4Proto, $tcpFStat, $ipFlags, $tcpFlags, $tcpAnomaly, $tcpInitWinSz, $tcpAveWinSz, $tcpMinWinSz, $tcpMaxWinSz, $tcpWinSzDwnCnt, $tcpWinSzUpCnt, $tcpWinSzChgDirCnt, $tcpWinSzThRt}' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  tcpFStat  ipFlags  tcpFlags  tcpAnomaly  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt
A     2984     0x0000100200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101      5840          2189.454     0            6432         3               1              2                  0.2142857
B     2984     0x0000100200004001  138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x00d4    0x3840   0x5b      0x8103      5792          6278.336     5792         6432         0               1              1                  0
A     3415     0x0000000200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x0054    0x1842   0x5f      0x8101      5840          4468.272     0            6432         1               1              2                  0.08333334
B     3415     0x0000100200004001  138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x00d4    0x1840   0x5b      0x8103      5792          7307.699     5792         8099         0               1              1                  0
A     3477     0x0000000200004000  138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x0054    0x1840   0xdb      0x8101      5840          5840         5840         5840         0               0              0                  0
B     3477     0x0000000200004001  201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x0054    0x1840   0xd6      0x0102      17680         12158.86     0            17680        2               0              0                  0.1666667
A     2811     0x0000000000004000  216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0054    0x0840   0x95      0x0001      31856         15609.44     0            31856        2               0              0                  0.5
B     2811     0x0000000200004001  138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x0054    0x1840   0x90      0x8000      6432          6432         6432         6432         0               0              0                  0
A     4982     0x0000000200004000  201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101      16384         17366.7      16384        17520        1               2              3                  0
B     4982     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x0054    0x1840   0x5f      0x0103      17376         12163.2      0            17376        1               0              0                  0.2
A     5035     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101      16384         17366.7      16384        17520        1               2              3                  0
B     5035     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x0054    0x1840   0x5f      0x0103      17376         12163.2      0            17376        1               0              0                  0.2
A     5036     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x1840   0x5b      0x8101      16384         17366.7      16384        17520        1               2              3                  0
B     5036     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x0054    0x1840   0x5f      0x0103      17376         12163.2      0            17376        1               0              0                  0.2
A     5083     0x0000000200004000  201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x00f4    0x3840   0x5b      0x8101      16384         17366.7      16384        17520        1               2              3                  0
B     5083     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x0054    0x1840   0x5f      0x0103      17376         12163.2      0            17376        1               0              0                  0.2
A     24       0x0000000200004000  138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x0054    0x1840   0x19      0xa801      5840          5840         5840         5840         0               0              0                  0
B     24       0x0000000200004001  36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x0054    0x3840   0x14      0x8100      16072         12261        0            17520        59              58             114                0.001858736
A     776      0x0000000200004000  138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x00d4    0x1840   0x1b      0x8101      5840          9188.096     5840         11584        0               2              1                  0
$

Sequence and Acknowledge numbers

TCP error control is achieved by Seq and Ack numbers which are a formidable tool for troubleshooting and contain information about network or host problems. The differences between these parameters denote the bytes being transfered between both peers. The packet ack count is only correct if each packet being sent is acknowledged, so Idle Repeat Request mode, which would acktualy defeat the purpose of efficient TCP communication. So just a lower limit.

The fault counts from both directions are an indicator for the health of the connection. If divided by the packet sent from basicStats you have a good performance measure.

The initial sequence number $tcpISeqN is used for covert channels and it can help to identify crafting tools if more than one flow from the same IP is available. Make sure that the flow is complete by checking the $tcpFlags for a SYN, otherwise $tcpISeqN denotes just the sequence number of the first packet being seen.

$ tawk '{if (bitsanyset($tcpOptions, 0x00000100)) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $l4Proto, $ipFlags, $tcpFStat, $tcpFlags, $tcpAnomaly, $tcpISeqN, $tcpSeqSntBytes, $tcpSeqFaultCnt, $tcpPAckCnt, $tcpFlwLssAckRcvdBytes, $tcpAckFaultCnt}' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                       dstPort  l4Proto  ipFlags  tcpFStat  tcpFlags  tcpAnomaly  tcpISeqN    tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt
A     2984     0x0000100200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       6        0x1842   0x0054    0x5f      0x8101      1805940851  662             3               6           519                    0
B     2984     0x0000100200004001  138.212.18.149   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37023    6        0x3840   0x00d4    0x5b      0x8103      1803485413  573             0               7           663                    2
A     3415     0x0000000200004000  138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       6        0x1842   0x0054    0x5f      0x8101      1813158365  1304            3               6           518                    0
B     3415     0x0000100200004001  138.212.18.145   jp       "Asahi Kasei Networks Corpor"  25       138.212.190.162  jp       "ASAHI KASEI CORPORATION"      37024    6        0x1840   0x00d4    0x5b      0x8103      1815487924  572             0               7           1305                   2
A     3477     0x0000000200004000  138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     201.98.187.64    mx       "Uninet S.A. de C.V."          2169     6        0x1840   0x0054    0xdb      0x8101      1807265661  8087            2               0           0                      1
B     3477     0x0000000200004001  201.98.187.64    mx       "Uninet S.A. de C.V."          2169     138.212.190.102  jp       "ASAHI KASEI CORPORATION"      4266     6        0x1840   0x0054    0xd6      0x0102      2926843921  0               0               4           9123                   1
A     2811     0x0000000000004000  216.249.75.129   us       "Smithville Digital"           18380    138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       6        0x0840   0x0054    0x95      0x0001      3658275921  0               0               0           0                      1
B     2811     0x0000000200004001  138.212.187.186  jp       "ASAHI KASEI CORPORATION"      80       216.249.75.129   us       "Smithville Digital"           18380    6        0x1840   0x0054    0x90      0x8000      1443222362  2896            0               0           0                      0
A     4982     0x0000000200004000  201.192.5.72     cr       "SAN JOSE"                     10533    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x1840   0x00f4    0x5b      0x8101      2793035386  607             4               2           191                    2
B     4982     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.72     cr       "SAN JOSE"                     10533    6        0x1840   0x0054    0x5f      0x0103      1428902644  190             1               2           608                    2
A     5035     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3198     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x3840   0x00f4    0x5b      0x8101      3050959365  606             4               2           191                    2
B     5035     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3198     6        0x1840   0x0054    0x5f      0x0103      531896816   190             1               2           607                    2
A     5036     0x0000000200004000  201.192.5.81     cr       "SAN JOSE"                     3199     138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x1840   0x00f4    0x5b      0x8101      3051015772  602             4               2           191                    2
B     5036     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.81     cr       "SAN JOSE"                     3199     6        0x1840   0x0054    0x5f      0x0103      1004739114  190             1               2           603                    2
A     5083     0x0000000200004000  201.192.5.64     cr       "SAN JOSE"                     51240    138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       6        0x3840   0x00f4    0x5b      0x8101      1823582276  603             4               2           191                    2
B     5083     0x0000000200004001  138.212.185.98   jp       "ASAHI KASEI CORPORATION"      80       201.192.5.64     cr       "SAN JOSE"                     51240    6        0x1840   0x0054    0x5f      0x0103      3262093770  190             1               2           604                    2
A     24       0x0000000200004000  138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     36.89.79.225     id       "Telekomunikasi Indonesia"     1370     6        0x1840   0x0054    0x19      0xa801      2542607003  1546464         30              0           0                      0
B     24       0x0000000200004001  36.89.79.225     id       "Telekomunikasi Indonesia"     1370     138.212.190.146  jp       "ASAHI KASEI CORPORATION"      6999     6        0x3840   0x0054    0x14      0x8100      1969468418  0               0               392         1272105                144
A     776      0x0000000200004000  138.212.185.212  jp       "ASAHI KASEI CORPORATION"      41468    201.201.212.213  --       "--"                           80       6        0x1840   0x00d4    0x1b      0x8101      1804752764  124             2               3           2229                   1
$

SPKTMD_SEQACKREL in tcpFlags controles the output of the seq/ack numbers in the packet mode. Switching to relative it facilitates the analysis of irregularities in throughput. Look into the packet mode tutorial to see an example of Absolute Relative Seq Ack Numbers.

TCP Options

TCP Options contain vital information about connection characteristics and even let us guess something about the type of applicaton being involved. Let us select MSS and WSC options to see whether the decoding works, so bit postion 2 and 3 in $tcpOptions.

$ tawk '{if (bitsanyset($tcpOptions, 0x0000000c) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $l4Proto, $tcpFStat, $tcpFlags, $ipFlags, $tcpAnomaly, $tcpOptCnt, $tcpOptions, $tcpMSS, $tcpWS}' annoloc2_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho                       srcPort  dstIP            dstIPCC  dstIPWho                   dstPort  l4Proto  tcpFStat  tcpFlags  ipFlags  tcpAnomaly  tcpOptCnt  tcpOptions  tcpMSS  tcpWS
A     392      0x0000000000004000  36.242.181.230   jp       "SoftBank Corp."               4685     138.212.188.67   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0xc2      0x0840   0x0000      4          0x00000016  1436    1
A     906      0x0000000000004000  161.135.53.11    us       "Federal Express Corp."        5001     138.212.191.94   jp       "ASAHI KASEI CORPORATION"  80       6        0x0046    0xc2      0x0840   0x0000      3          0x0000000e  1460    1
A     1027     0x0000000000004000  146.162.158.230  gb       "Norwich Union Insurance Lim"  2849     138.212.184.193  jp       "ASAHI KASEI CORPORATION"  6346     6        0x0046    0x42      0x0840   0x0000      4          0x00000016  1460    1
A     1154     0x0000000000004000  193.133.224.57   gb       "Verizon UK Limited"           3286     138.212.188.67   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x02      0x0840   0x0000      4          0x00000016  1460    1
A     1336     0x0000000000004000  216.21.10.20     us       "XNS Technology Group Inc."    1305     138.212.191.94   jp       "ASAHI KASEI CORPORATION"  80       6        0x0046    0x02      0x0840   0x0000      4          0x00000016  536     1
A     1534     0x0000000000004000  216.233.229.167  us       "MCI Communications Services"  3782     138.212.185.86   jp       "ASAHI KASEI CORPORATION"  1058     6        0x0046    0x42      0x0840   0x0000      4          0x00000016  1460    1
A     1586     0x0000000200004000  130.92.198.110   ch       "Universitaet Bern"            1249     138.212.191.248  jp       "ASAHI KASEI CORPORATION"  1214     6        0x00c4    0xdb      0x1840   0x8001      16         0x00000036  1460    1
B     1586     0x0000000200004001  138.212.191.248  jp       "ASAHI KASEI CORPORATION"      1214     130.92.198.110   ch       "Universitaet Bern"        1249     6        0x0044    0xdf      0x3840   0x8003      4          0x00000016  1460    1
A     1836     0x0000000000004000  209.114.247.93   us       "Ideal Technology Solutions "  1335     138.212.187.11   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x42      0x0840   0x0000      4          0x00000016  536     1
A     1909     0x0000000000004000  19.27.88.236     us       "Ford Motor Company"           4045     138.212.186.88   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x82      0x0840   0x0000      4          0x00000016  1452    1
A     1959     0x0000000000004000  216.21.10.20     us       "XNS Technology Group Inc."    1305     138.212.191.94   jp       "ASAHI KASEI CORPORATION"  80       6        0x0046    0x02      0x0840   0x0000      4          0x00000016  536     1
A     1904     0x0000000000004000  138.212.186.27   jp       "ASAHI KASEI CORPORATION"      1396     83.220.134.126   --       "--"                       4661     6        0x0046    0x02      0x0840   0x0000      4          0x00000016  1452    1
A     1969     0x0000000000004000  83.0.129.97      pl       "Orange Polska Spolka Akcyjn"  1395     138.212.187.11   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0xc2      0x0840   0x0000      4          0x00000016  1460    1
A     2060     0x0000000000004000  18.97.211.233    us       "Massachusetts Institute of "  3448     138.212.187.247  jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0xc2      0x0840   0x0000      4          0x00000016  1460    1
A     2113     0x0000000000004000  36.92.31.200     id       "Telekomunikasi Indonesia"     48337    138.212.185.86   jp       "ASAHI KASEI CORPORATION"  1052     6        0x0046    0x42      0x0800   0x0000      4          0x00000016  1460    1
A     2183     0x0000000000004000  201.133.193.218  mx       "Uninet S.A. de C.V."          3134     138.212.187.11   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x02      0x0840   0x0000      4          0x00000016  1440    1
A     2236     0x0000000000004000  83.45.182.68     es       "Telefonica de Espana SAU"     1322     138.212.187.10   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x82      0x0840   0x0000      4          0x00000016  1460    1
A     2274     0x0000000000004000  201.53.22.207    br       "CLARO S.A."                   4810     138.212.187.11   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x02      0x0840   0x0000      4          0x00000016  1460    1
A     2333     0x0000000000004000  193.99.26.18     de       "Verizon Deutschland GmbH"     1925     138.212.188.67   jp       "ASAHI KASEI CORPORATION"  1214     6        0x0046    0x82      0x0840   0x0000      6          0x0000001e  1452    1
$

Time stamp options are a formidable tool for RTT estimation and for revealing the boot time of the src host.

Boot time estimation

The tcp timestamp option originally being created for Round Trip Time (RTT) measurements can be abused for boot time estimation, because OS used the uptime as a derivative for the tcp time stamp.

As different machines boot at different times this measure separates these machines even behind a NAT where you normally see only one IP address. The only problem is that different OS have different incremental clocks, which can be calculated if several packets / flow are available, otherwise it comes down to OS guessing. The column $tcpEcI below denotes this increment.

Unfortunately, newer version of different OS use a random value per connection, aka flow. Hence, a comparision of different flows from the same machine will yield in different up or boot times, becoming useless. Nevertheless, the estimation of $tcpEcI is still useful for newer OS.

The annoloc2.pcap was acquired in 2002, so if you look at the boot time below you will see a correlation; it works.

$ tawk '{ print $srcIP, $tcpTmS, $tcpTmER, $tcpEcI, $tcpUtm, $tcpBtm}' ~/test_data/results/BW_2013/annoloc2_flows.txt | sort -V | uniq | tawk '{ if ($2) print }' | head -n 40 | tcol
18.2.89.211     7748617     849533919  0.01  77486.168268     1022094226.113857
18.2.89.211     7748924     849534263  0.01  77489.238268     1022094226.117551
18.2.89.211     7749342     849534680  0.01  77493.418268     1022094226.112002
18.2.89.211     7749343     849534252  0.01  77493.428268     1022094226.110962
18.2.89.211     7749424     204508834  0.01  77494.238268     1022094226.109821
18.2.89.211     7749524     849534270  0.01  77495.238268     1022094226.114326
18.2.89.211     7749624     849534698  0.01  77496.238268     1022094226.112043
18.2.89.211     7749726     849535064  0.01  77497.258268     1022094226.108770
18.2.89.211     7749831     849534861  0.01  77498.308268     1022094226.110499
18.2.89.211     7749831     849535169  0.01  77498.308268     1022094226.106990
18.2.89.211     7749838     849535176  0.01  77498.378268     1022094226.108500
18.2.89.211     7749938     849535064  0.01  77499.378268     1022094226.111827
18.2.89.211     7749967     849535305  0.01  77499.668268     1022094226.108143
18.85.17.135    22846249    826368     0.01  228462.484893    1021943255.914858
18.85.17.135    22846930    826851     0.01  228469.294893    1021943255.920749
18.107.26.21    103819943   373699105  0.01  1038199.406794   1021133503.026880
18.155.23.221   33847443    43376223   0.01  338474.422435    1021833231.196510
18.155.23.221   33848762    248428209  0.01  338487.612434    1021833231.192676
18.155.23.221   33848960    182165005  0.01  338489.592434    1021833231.189961
18.155.23.221   33849237    6322419    0.01  338492.362434    1021833231.189718
19.24.4.45      1693583440  72029656   0.01  16935834.021455  1005235877.235269
19.24.4.45      1693583872  793132504  0.01  16935838.341455  1005235877.273548
19.29.161.16    19986317    34285718   0.01  199863.165533    1021971858.270386
19.55.36.202    1098236     390486740  0.01  10982.359755     1022160744.225946
19.55.36.202    1098240     8297346    0.01  10982.399755     1022160744.232882
19.59.134.250   199361062   113909808  0.01  1993610.575439   1020178116.063283
19.67.192.174   35424473    785167233  0.01  354244.722082    1021817463.220258
19.67.210.218   4323074     689732     0.1   432307.406442    1021739419.180875
19.114.68.45    78574708    17008889   0.01  785747.062437    1021385960.390762
19.114.68.45    78574918    34284567   0.01  785749.162437    1021385960.391068
19.114.68.45    78575088    72029658   0.01  785750.862437    1021385960.391782
19.139.46.124   29251       785168750  0.1   2925.100044      1022168799.345162
19.169.122.89   1806691     853448585  0.01  18066.909596     1022153659.006621
19.173.18.204   17719695    785166685  0.1   1771969.526404   1020399734.567682
19.173.18.204   17719720    785168065  0.01  177197.196039    1021994520.359783
19.173.18.204   17719731    785168629  0.01  177197.306039    1021994526.080392
19.182.177.87   49516646    34285659   0.01  495166.448932    1021676554.013431
19.182.177.87   144345608   34285961   0.01  1443456.047736   1020728267.433657
19.182.178.138  23254415    34285407   0.01  232544.144802    1021939173.792873
19.182.178.197  139712      853446502  0.1   13971.200208     1022157733.965144
$

The plugin will evolve, as soon we find something else for this feature it will be implemented in tcpFlags. So if you have an idea, please contact are happy to cooperate with you doing research of such things.

Fragmentation

Fragmentation is a diverse subject. In IPv4 it should not occour anymore, because the MTU today is generally large enough throughout the whole networks. If you see it in your corporate network, it should be investigated. IPv6 different story, here fragmentation is an established tool.

The constant FRAG_ANALYZE in tcpFLags controls the fragmentation analysis. Moreover the constant FRAGMENTATION has to be enabled in tranalyzer.h under the tranalyzer2/src directory, actually being the default.

I prepared a pcap which illustrates a pitfall of flow based representation when fragmentation is present. So download frag.pcap and add basicStats so that we can look at the packet and payload statistics.

$ t2build basicStats
...
$

Then rerun t2 using the -s option, as we also want to look at the packets.

$ t2 -r ~/data/frag.pcap -w ~/results/ -s
================================================================================
Tranalyzer 0.8.5 (Anteater), Tarantula. PID: 25417
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.5
    02: basicStats, 0.8.5
    03: tcpFlags, 0.8.5
    04: tcpStates, 0.8.5
    05: txtSink, 0.8.5
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312762 (312.76 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
Processing file: /home/wurst/data/frag.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1294260264.274530 sec (Wed 05 Jan 2011 20:44:24 GMT)
Dump stop : 1294260291.961272 sec (Wed 05 Jan 2011 20:44:51 GMT)
Total dump duration: 27.686742 sec
Finished processing. Elapsed time: 0.000914 sec
Finished unloading flow memory. Time: 0.001272 sec
Percentage completed: 100.00%
Number of processed packets: 82
Number of processed bytes: 14857 (14.86 K)
Number of raw bytes: 14857 (14.86 K)
Number of pcap bytes: 16193 (16.19 K)
Number of IPv4 packets: 38 [46.34%]
Number of A packets: 80 [97.56%]
Number of B packets: 2 [2.44%]
Number of A bytes: 14737 (14.74 K) [99.19%]
Number of B bytes: 120 [0.81%]
Average A packet load: 184.21
Average B packet load: 60.00
--------------------------------------------------------------------------------
basicStats: Biggest Talker: 192.168.203.131: 26 [31.71%] packets
basicStats: Biggest Talker: 192.168.203.131: 10904 (10.90 K) [73.39%] bytes
tcpFlags: Aggregated ipFlags: 0x0060
tcpFlags: Aggregated tcpAnomaly: 0x0044
tcpFlags: Number of TCP scans, succ scans, syn retries, seq retries: 1, 2, 0, 0
tcpFlags: Number WinSz below 1: 1 [50.00%]
tcpStates: Aggregated anomaly flags: 0xc3
--------------------------------------------------------------------------------
Headers count: min: 2, max: 4, average: 2.67
Number of LLC packets: 16 [19.51%]
Number of ICMP packets: 3 [3.66%]
Number of TCP packets: 27 [32.93%]
Number of TCP bytes: 10964 (10.96 K) [73.80%]
Number of UDP packets: 5 [6.10%]
Number of UDP bytes: 763 [5.14%]
Number of IPv4 fragmented packets: 26 [68.42%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 20
Number of processed A flows: 18 [90.00%]
Number of processed B flows: 2 [10.00%]
Number of request     flows: 18 [90.00%]
Number of reply       flows: 2 [10.00%]
Total   A/B    flow asymmetry: 0.80
Total req/rply flow asymmetry: 0.80
Number of processed   packets/flows: 4.10
Number of processed A packets/flows: 4.44
Number of processed B packets/flows: 1.00
Number of processed total packets/s: 2.96
Number of processed A+B packets/s: 2.96
Number of processed A   packets/s: 2.89
Number of processed   B packets/s: 0.07
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.72
Average full raw bandwidth: 4293 b/s (4.29 Kb/s)
Average full bandwidth : 3515 b/s (3.52 Kb/s)
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.05 GB [0.08%]
Aggregate flow status: 0x0000081000004044
[INF] IPv4
[INF] IPv4/6 fragmentation
[INF] Ethernet flows
[INF] ARP flows

We see that 26 [68.42%] packets are fragmented. Two warnings about fragmentation in the end report indicate abnormalities of the IPv4 fragmented traffic. Below is fragmented traffic including abnormalities are selected from the flow file. We have a perfect fragmented packet, all packets sum up in the numBytesSnt, Rcvd column.

$ tawk '{if (bitsanyset($ipFlags, 0x03b8) || bitsanyset($tcpAnomaly, 0x00ff)) print $dir, $flowInd, $flowStat, $srcIP, $srcIPCC, $srcIPWho, $srcPort, $dstIP, $dstIPCC, $dstIPWho, $dstPort, $tcpFStat, $ipFlags, $tcpFlags, $tcpAnomaly, $numPktsSnt, $numPktsRcvd, $numBytesSnt, $numBytesRcvd}' frag_flows.txt | tcol
%dir  flowInd  flowStat            srcIP            srcIPCC  srcIPWho           srcPort  dstIP            dstIPCC  dstIPWho           dstPort  tcpFStat  ipFlags  tcpFlags  tcpAnomaly  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd
A     4        0x0000081000004000  192.168.203.131  09       "Private network"  1509     192.168.203.134  09       "Private network"  0        0x0046    0x0020   0x00      0x0040      26          1            10000        0
B     4        0x0000000000004001  192.168.203.134  09       "Private network"  0        192.168.203.131  09       "Private network"  1509     0x0044    0x0040   0x14      0x0004      1           26           0            10000
$

Looking at the packet file the first packet contains a Layer4 header with the checksum. At the last fragment T2 adds the IP pseudo header and calculates the final checksum which matches 0x7366 from the TCP header of the initial packet.

$ tawk 'bitsanyset($ipFlags, 0x0020) || bitsanyset($ipFrag, 0x1fff)' frag_packets.txt | tcol
%pktNo  flowInd  flowStat            time               pktIAT    flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP            srcIPCC  srcIPWho         srcPort  dstIP            dstIPCC  dstIPWho         dstPort  l4Proto  pktLen  l7Len  ipTOS  ipID  ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq        ack        seqDiff  ackDiff  seqPktLen  ackPktLen  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpTmS  tcpTmER  tcpOptLen  tcpOpts  l7Content
4       4        0x0000009000004000  1294260266.528280  0.000000  0.000000      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     380    0x00   249   0         0x2000  64     0x4000       0x4000       0x7366       0x0000       0x0020   0                       0                     0                 280548844  777151161  0        0        0          0          0x0141    0x00      0x0040      512     0       0        0                   XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
5       4        0x0000089000004000  1294260266.528318  0.000038  0.000038      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2032  64     0x3fce       0x3fce       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
6       4        0x0000089000004000  1294260266.528335  0.000017  0.000055      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2064  64     0x3f9c       0x3f9c       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
7       4        0x0000089000004000  1294260266.528348  0.000013  0.000068      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2096  64     0x3f6a       0x3f6a       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
8       4        0x0000089000004000  1294260266.528363  0.000015  0.000083      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x20c8  64     0x3f38       0x3f38       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
9       4        0x0000089000004000  1294260266.528383  0.000020  0.000103      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x20fa  64     0x3f06       0x3f06       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
10      4        0x0000089000004000  1294260266.528404  0.000021  0.000124      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x212c  64     0x3ed4       0x3ed4       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
11      4        0x0000089000004000  1294260266.528424  0.000020  0.000144      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x215e  64     0x3ea2       0x3ea2       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
12      4        0x0000089000004000  1294260266.528443  0.000019  0.000163      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2190  64     0x3e70       0x3e70       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
13      4        0x0000089000004000  1294260266.528462  0.000019  0.000182      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x21c2  64     0x3e3e       0x3e3e       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
14      4        0x0000089000004000  1294260266.528480  0.000018  0.000200      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x21f4  64     0x3e0c       0x3e0c       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
15      4        0x0000089000004000  1294260266.528497  0.000017  0.000217      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2226  64     0x3dda       0x3dda       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
16      4        0x0000089000004000  1294260266.528512  0.000015  0.000232      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2258  64     0x3da8       0x3da8       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
17      4        0x0000089000004000  1294260266.528526  0.000014  0.000246      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x228a  64     0x3d76       0x3d76       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
18      4        0x0000089000004000  1294260266.528544  0.000018  0.000264      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x22bc  64     0x3d44       0x3d44       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
19      4        0x0000089000004000  1294260266.528561  0.000017  0.000281      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x22ee  64     0x3d12       0x3d12       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
20      4        0x0000089000004000  1294260266.528575  0.000014  0.000295      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2320  64     0x3ce0       0x3ce0       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
21      4        0x0000089000004000  1294260266.528588  0.000013  0.000308      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2352  64     0x3cae       0x3cae       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
22      4        0x0000089000004000  1294260266.528601  0.000013  0.000321      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x2384  64     0x3c7c       0x3c7c       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
23      4        0x0000089000004000  1294260266.528613  0.000012  0.000333      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x23b6  64     0x3c4a       0x3c4a       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
24      4        0x0000089000004000  1294260266.528626  0.000013  0.000346      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x23e8  64     0x3c18       0x3c18       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
25      4        0x0000089000004000  1294260266.528776  0.000150  0.000496      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x241a  64     0x3be6       0x3be6       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
26      4        0x0000089000004000  1294260266.528818  0.000042  0.000538      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x244c  64     0x3bb4       0x3bb4       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
27      4        0x0000089000004000  1294260266.528854  0.000036  0.000574      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x247e  64     0x3b82       0x3b82       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
28      4        0x0000089000004000  1294260266.528889  0.000035  0.000609      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        434     400    0x00   249   0         0x24b0  64     0x3b50       0x3b50       0x0000       0x0000       0x0020   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
29      4        0x0000081000004000  1294260266.528923  0.000034  0.000643      3        eth:ipv4:tcp             00:24:e8:ed:3f:10  00:0f:ea:e8:f5:51  0x0800   192.168.203.131  09       Private network  1509     192.168.203.134  09       Private network  0        6        54      20     0x00   249   0         0x04e2  64     0x5c9a       0x5c9a       0x7366       0x7366       0x0000   0                       0                     0                                                                                                                                                            XXXXXXXXXXXXXXXXXXXX

Detecting Scans

Scans are normally an initiation of some serious attacks. Nevertheless, a lot of normal tcp traffic looks like scanning. Anybody who used SNORT, the defacto standard of an IDS, might have suffered from interpreting all the scan alarms. So it needs filtering.

I needed once an indication in the end report and the flow/packet file whether there is a tcp malicious scan around. It is not perfect, but often served its purpose.

To see its effect clearly please download nmap_v_sT.pcap, copy it under your data folder and rerun t2. The pcap is generated by the nmap scanning tool.

$ t2 -r ~/data/nmap_v_sT.pcap -w ~/results/
===============================================================================
Tranalyzer 0.8.5 (Anteater), Tarantula. PID: 30415
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.5
    02: basicStats, 0.8.5
    03: tcpFlags, 0.8.5
    04: tcpStates, 0.8.5
    05: txtSink, 0.8.5
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312762 (312.76 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
Processing file: /home/wurst/data/nmap_v_sT.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1416313200.358106 sec (Tue 18 Nov 2014 12:20:00 GMT)
Dump stop : 1416313214.101341 sec (Tue 18 Nov 2014 12:20:14 GMT)
Total dump duration: 13.743235 sec
Finished processing. Elapsed time: 0.003624 sec
Finished unloading flow memory. Time: 0.025292 sec
Percentage completed: 100.00%
Number of processed packets: 1081 (1.08 K)
Number of processed bytes: 83786 (83.79 K)
Number of raw bytes: 83786 (83.79 K)
Number of pcap bytes: 101106 (101.11 K)
Number of IPv4 packets: 1081 (1.08 K) [100.00%]
Number of A packets: 1081 (1.08 K) [100.00%]
Number of A bytes: 83786 (83.79 K) [100.00%]
Average A packet load: 77.51
Average B packet load: 0.00
--------------------------------------------------------------------------------
basicStats: Biggest Talker: 10.20.6.125: 16 [1.48%] packets
basicStats: Biggest Talker: 10.20.6.125: 2832 (2.83 K) [3.38%] bytes
tcpFlags: Aggregated ipFlags: 0x0046
tcpFlags: Aggregated tcpAnomaly: 0x8080
tcpFlags: Number of TCP scans, succ scans, syn retries, seq retries: 1033 (1.03 K), 0, 0, 0
tcpStates: Aggregated anomaly flags: 0x83
--------------------------------------------------------------------------------
Headers count: min: 3, max: 4, average: 3.00
Number of ICMP packets: 16 [1.48%]
Number of TCP packets: 1057 (1.06 K) [97.78%]
Number of TCP bytes: 78218 (78.22 K) [93.35%]
Number of UDP packets: 8 [0.74%]
Number of UDP bytes: 2736 (2.74 K) [3.27%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 1042 (1.04 K)
Number of processed A flows: 1042 (1.04 K) [100.00%]
Number of request     flows: 1042 (1.04 K) [100.00%]
Total   A/B    flow asymmetry: 1.00
Total req/rply flow asymmetry: 1.00
Number of processed   packets/flows: 1.04
Number of processed A packets/flows: 1.04
Number of processed total packets/s: 78.66
Number of processed A+B packets/s: 78.66
Number of processed A   packets/s: 78.66
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 75.82
Average full raw bandwidth: 48772 b/s (48.77 Kb/s)
Average full bandwidth : 48772 b/s (48.77 Kb/s)
Max number of flows in memory: 1042 (1.04 K) [0.40%]
Memory usage: 0.06 GB [0.09%]
Aggregate flow status: 0x0000100002004000
[WRN] Consecutive duplicate IP ID
[INF] IPv4
[INF] SSDP/UPnP flows

So you see 1081 packet with flow asymmetry of 1.0; 1033 are tagged as rogue TCP scans.

tawk -V tcpAnomaly=0x8080

The tcpAnomaly column with value 0x8080 is to be interpreted as follows:

   bit | tcpAnomaly | Description
   =============================================================================
     7 | 0x0080     | XMas flag, potential Xmas scan packet, or malicious channel
    15 | 0x8000     | Duplicate ACK

Below the first 20 rows of the flow file containing scans are printed.

$ tawk 'bitsanyset(tcpFStat, 0x0006)' nmap_v_sT_flows.txt | head -n 20 | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP        srcIPCC  srcIPWho           srcPort  dstIP        dstIPCC  dstIPWho           dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT  aveIAT  stdIAT  pktps  bytps  pktAsm  bytAsm  tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipTOS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN    tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpTmS   tcpTmER  tcpEcI  tcpUtm        tcpBtm             tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStates
A     1        0x0000000000004000  1416313200.358106  1416313200.358106  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  54118    10.20.0.125  01       "Private network"  587      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2868435842  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3992845  0        0.004   15971.380759  1416297228.977348  0              65535             0                 0                 0                    0             -1               0x03
A     2        0x0000000000004000  1416313200.457149  1416313200.457149  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  33056    10.20.0.125  01       "Private network"  1720     6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2506148143  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3992875  0        0.004   15971.500759  1416297228.956391  0              65535             0                 0                 0                    0             -1               0x03
A     3        0x0000000000004000  1416313201.458313  1416313201.458313  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  45750    10.20.0.125  01       "Private network"  1720     6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1770850395  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993175  0        0.004   15972.700759  1416297228.757555  0              65535             0                 0                 0                    0             -1               0x03
A     4        0x0000000000004000  1416313201.458361  1416313201.458361  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  38704    10.20.0.125  01       "Private network"  587      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3336324932  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993175  0        0.004   15972.700759  1416297228.757603  0              65535             0                 0                 0                    0             -1               0x03
A     5        0x0000000000004000  1416313201.557900  1416313201.557900  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  50322    10.20.0.125  01       "Private network"  995      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3712758514  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993205  0        0.004   15972.820759  1416297228.737142  0              65535             0                 0                 0                    0             -1               0x03
A     6        0x0000000000004000  1416313201.558981  1416313201.558981  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  45384    10.20.0.125  01       "Private network"  135      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2792320487  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993206  0        0.004   15972.824759  1416297228.734223  0              65535             0                 0                 0                    0             -1               0x03
A     7        0x0000000000004000  1416313201.559756  1416313201.559756  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  55726    10.20.0.125  01       "Private network"  443      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  742007355   0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993206  0        0.004   15972.824759  1416297228.734998  0              65535             0                 0                 0                    0             -1               0x03
A     8        0x0000000000004000  1416313201.759706  1416313201.759706  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  55879    10.20.0.125  01       "Private network"  443      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3280881276  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993266  0        0.004   15973.064759  1416297228.694948  0              65535             0                 0                 0                    0             -1               0x03
A     9        0x0000000000004000  1416313201.759861  1416313201.759861  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  45598    10.20.0.125  01       "Private network"  135      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3916203706  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993266  0        0.004   15973.064759  1416297228.695103  0              65535             0                 0                 0                    0             -1               0x03
A     10       0x0000000000004000  1416313201.759942  1416313201.759942  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  50612    10.20.0.125  01       "Private network"  995      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1615214400  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993266  0        0.004   15973.064759  1416297228.695184  0              65535             0                 0                 0                    0             -1               0x03
A     11       0x0000000000004000  1416313201.859335  1416313201.859335  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  48186    10.20.0.125  01       "Private network"  445      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  822741617   0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993296  0        0.004   15973.184759  1416297228.674577  0              65535             0                 0                 0                    0             -1               0x03
A     12       0x0000000000004000  1416313201.959507  1416313201.959507  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  48279    10.20.0.125  01       "Private network"  445      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1080378516  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993326  0        0.004   15973.304759  1416297228.654749  0              65535             0                 0                 0                    0             -1               0x03
A     13       0x0000000000004000  1416313201.960575  1416313201.960575  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  39825    10.20.0.125  01       "Private network"  1723     6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1607881361  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993326  0        0.004   15973.304759  1416297228.655817  0              65535             0                 0                 0                    0             -1               0x03
A     14       0x0000000000004000  1416313202.058441  1416313202.058441  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  33611    10.20.0.125  01       "Private network"  554      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2605352890  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993355  0        0.004   15973.420759  1416297228.637683  0              65535             0                 0                 0                    0             -1               0x03
A     15       0x0000000000004000  1416313202.059850  1416313202.059850  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  58233    10.20.0.125  01       "Private network"  53       6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1614553615  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993356  0        0.004   15973.424759  1416297228.635092  0              65535             0                 0                 0                    0             -1               0x03
A     16       0x0000000000004000  1416313202.060987  1416313202.060987  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  39924    10.20.0.125  01       "Private network"  1723     6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  889726309   0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993356  0        0.004   15973.424759  1416297228.636229  0              65535             0                 0                 0                    0             -1               0x03
A     17       0x0000000000004000  1416313202.158552  1416313202.158552  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  33706    10.20.0.125  01       "Private network"  554      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  3984096420  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993385  0        0.004   15973.540759  1416297228.617794  0              65535             0                 0                 0                    0             -1               0x03
A     18       0x0000000000004000  1416313202.159998  1416313202.159998  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  58329    10.20.0.125  01       "Private network"  53       6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2107676260  0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993386  0        0.004   15973.544759  1416297228.615240  0              65535             0                 0                 0                    0             -1               0x03
A     19       0x0000000000004000  1416313202.161062  1416313202.161062  0.000000  1           3        eth:ipv4:tcp  e0:3f:49:7e:59:79  00:00:5e:00:01:11  0x0800              10.20.6.125  01       "Private network"  41868    10.20.0.125  01       "Private network"  199      6        1           0            0            0             0         0         0           0           0       0       0       0       0      0      1       0       0x0052    65535       0           64        64        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  776114555   0           0               0               0           0                      0               3737600       3737600      3737600      3737600      0               0              0                  0             0x02      0x0000      1             5          0x0000011e  1460    128    3993386  0        0.004   15973.544759  1416297228.616304  0              65535             0                 0                 0                    0             -1               0x03
$

If you look at the status flags the bit 0x02 is a measure to select unsuccessful SYN scan flows. The aggregated TCP flags indicate that these flows are unanswered SYN scans.

Odd is the high window scale factor, the randomn initial Window size and the TTL indicate a linux platform. Is it malicious?

Clearly tcpFStat claims a scan.

$ tawk -V tcpFStat=0x0152

The tcpFStat column with value 0x0152 is to be interpreted as follows:

   bit | tcpFStat | Description
   =============================================================================
     1 | 0x0002   | Scan detected in flow
     4 | 0x0010   | TCP option init
     6 | 0x0040   | Window state machine initialized
     9 | 0x0100   | L4 checksum calculation if present

$ tawk -V tcpFlags=0x02

The tcpFlags column with value 0x02 is to be interpreted as follows:

   bit | tcpFlags | Description
   =============================================================================
     1 | 0x02     | SYN: Synchronize sequence numbers

$

Look at the other pcaps and checkout the scan alarms. Have fun!