Tutorial: File Transport Protocol (FTP)

data carving FTP layer 7

Introduction

This tutorial discusses the plugin ftpDecode. FTP is the oldest file transfer protocol and not encrypted. So we can scrutinize and extract all content. It is sometimes even used today, you won’t believe it.

Preparation

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:

t2build -e -y

Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Then compile the core (tranalyzer2) and the following plugins:

t2build tranalyzer2 basicFlow tcpStates ftpDecode txtSink

...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:

mkdir ~/data ~/results

The sample PCAP used in this tutorial can be downloaded here: ftp-dpic.pcap.

Please save it in your ~/data folder.

Now you are all set for analyzing FTP traffic!

ftpDecode

Let’s look at the plugin configuration first:

ftpDecode

vi src/ftpDecode.h

This plugin is data carving capable, so it extracts the content of the flows including user names and passwords. The latter are also printed in the flow file. If FTP_SAVE=1 all the content including commands, up and downloads are extracted under the path FTP_F_PATH. If no file name is found, e.g. if you only recorded the B flow, then the FTP_NONAME is used. Note that FTP_RMDIR=1 defines that the file directory is deleted for each new t2 execution. So if you want to keep earlier files, switch it to 0. Here we keep the default value.

The lengths of the extracted usernames and passwords can be configured as the form of the FTP command flow output. We leave it in human readable and aggregated and leave data carving off.

So execute t2 on the supplied pcap including packet mode.

t2 -r ~/test_data/data/ftp-dpic.pcap -w ~/results -s

================================================================================
Tranalyzer 0.8.14 (Anteater), Tarantula. PID: 6198
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.14
    02: tcpStates, 0.8.14
    03: ftpDecode, 0.8.14
    04: txtSink, 0.8.14
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K)
Processing file: /home/wurst/data/ftp-dpic.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1168195766.739929 sec (Sun 07 Jan 2007 18:49:26 GMT)
Dump stop : 1168195799.748737 sec (Sun 07 Jan 2007 18:49:59 GMT)
Total dump duration: 33.008808 sec
Finished processing. Elapsed time: 0.002874 sec
Finished unloading flow memory. Time: 0.002956 sec
Percentage completed: 100.00%
Number of processed packets: 92
Number of processed bytes: 69464 (69.46 K)
Number of raw bytes: 69464 (69.46 K)
Number of pad bytes: 26
Number of pcap bytes: 70960 (70.96 K)
Number of IPv4 packets: 92 [100.00%]
Number of A packets: 37 [40.22%]
Number of B packets: 55 [59.78%]
Number of A bytes: 2096 (2.10 K) [3.02%]
Number of B bytes: 67368 (67.37 K) [96.98%]
Average A packet load: 56.65
Average B packet load: 1224.87 (1.22 K)
--------------------------------------------------------------------------------
tcpStates: Aggregated tcpStatesAFlags=0xc3
ftpDecode: Aggregated ftpStat=0x0b
ftpDecode: Number of FTP control packets: 20 [21.74%]
ftpDecode: Number of FTP-DATA packets: 69 [75.00%]
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 92 [100.00%]
Number of TCP bytes: 69464 (69.46 K) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 8
Number of processed A flows: 4 [50.00%]
Number of processed B flows: 4 [50.00%]
Number of request     flows: 4 [50.00%]
Number of reply       flows: 4 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 11.50
Number of processed A packets/flows: 9.25
Number of processed B packets/flows: 13.75
Number of processed total packets/s: 2.79
Number of processed A+B   packets/s: 2.79
Number of processed A     packets/s: 1.12
Number of processed   B   packets/s: 1.67
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.24
Average full raw bandwidth: 16835 b/s (16.84 Kb/s)
Average full bandwidth : 16829 b/s (16.83 Kb/s)
Max number of flows in memory: 6 [0.00%]
Memory usage: 0.02 GB [0.02%]
Aggregated flowStat=0x0400000000004000
[INF] IPv4 flows

The end report states that we have 8 flows with 20 control and 69 data packets. Looking at the aggregated ftpStat reveals that we have a passive and active FTP flow.

tawk -V ftpStat=0x0b

The ftpStat column with value 0x0b is to be interpreted as follows:

   bit | ftpStat | Description
   =============================================================================
     0 | 0x01    | FTP control port found
     1 | 0x02    | FTP passive parent flow
     3 | 0x08    | FTP active parent flow

So a promising error free download, lets look at the flow file now. Unfortunately we have no user name and passwords in the pcap, so try it with your own, you will see them in ftpUser and ftpPass. Flow 1 being denoted by ftpStat=0x09{.awk} contains four commands: TYPE;PASV;SIZE;RETR, hence there is a download initiated. The B flow shows the aggregated return codes, 226 indicates a successful download. A list of codes can be found in ftpDecode.h. ftpCDFindex denotes the link between the command flow with its spawned data flow 2 for the user initiated A flow 1. B flow 1 did not spawn any data flow, so the A flow 1 is linked. The data flow 2 denotes flow 1 as its parent. The column ftpCP shows all command parameters of all ftp flows. In flow 2 we see the name of the downloaded file.

tcol ~/results/ftp-dpic_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration   numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPOrg                        srcPort  dstIP            dstIPCC  dstIPOrg                        dstPort  l4Proto  tcpStatesAFlags  ftpStat  ftpCDFindex  ftpCC                ftpRC                ftpNumUser  ftpUser  ftpNumPass  ftpPass  ftpNumCP  ftpCP
A     2        0x0400000000004000  1168195766.764244  1168195766.976730  0.212486   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4076     128.121.136.217  us       "NTT America"                   30012    6        0x42             0x02     1                                                      0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
B     2        0x0400000000004001  1168195766.782725  1168195766.965575  0.182850   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   30012    67.180.72.76     us       "Comcast Cable Communications"  4076     6        0x02             0x02     1                                                      0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
A     1        0x0400000000004000  1168195766.739929  1168195799.748737  33.008808  1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4075     128.121.136.217  us       "NTT America"                   21       6        0x43             0x09     2            TYPE;PASV;SIZE;RETR                       0                    0                    2         "I";"/funwithbill/Microsoft-1978.jpg"
B     1        0x0400000000004001  1168195766.751652  1168195767.017088  0.265436   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4075     6        0x03             0x09     2                                 200;227;213;150;226  0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
A     3        0x0400000000004000  1168195798.794148  1168195798.808733  0.014585   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4072     128.121.136.217  us       "NTT America"                   21       6        0x83             0x01                                                            0                    0                    0
B     3        0x0400000000004001  1168195798.808082  1168195798.808646  0.000564   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4072     6        0x03             0x01                                                            0                    0                    0
A     4        0x0400000000004000  1168195799.515059  1168195799.527966  0.012907   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4071     128.121.136.217  us       "NTT America"                   21       6        0x83             0x01                                                            0                    0                    0
B     4        0x0400000000004001  1168195799.527449  1168195799.527938  0.000489   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4071     6        0x03             0x01

The packet file shows the ftp status for each packet and the content, which matches the extracted command and parameter columns in the flow file. Here you can see also the text for the ftp return codes for the control flow. You can see the content being downloaded in flow 2.

tcol ~/results/ftp-dpic_packets.txt

%pktNo  flowInd  flowStat            time               pktIAT     pktTrip    flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP            srcIPCC  srcIPOrg                      srcPort  dstIP            dstIPCC  dstIPOrg                      dstPort  l4Proto  tcpStatesAFlags  ftpStat  l7Content
1       1        0x0400000000004000  1168195766.739929  0.000000   0.000000   0.000000      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4075     128.121.136.217  us       NTT America                   21       6        0x01             0x01     TYPE I\r\n
2       1        0x0400000000004001  1168195766.751652  0.000000   0.011723   0.000000      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4075     6        0x01             0x01     200 Type set to I\r\n
3       1        0x0400000000004000  1168195766.752010  0.012081   0.000358   0.012081      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4075     128.121.136.217  us       NTT America                   21       6        0x01             0x01     PASV\r\n
4       1        0x0400000000004001  1168195766.764051  0.012399   0.012041   0.012399      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4075     6        0x01             0x09     227 Entering Passive Mode (128,121,136,217,117,60).\r\n
5       2        0x0400000000004000  1168195766.764244  0.000000   0.000000   0.000000      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4076     128.121.136.217  us       NTT America                   30012    6        0x00             0x00
6       2        0x0400000000004001  1168195766.782725  0.000000   0.018481   0.000000      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   30012    67.180.72.76     us       Comcast Cable Communications  4076     6        0x00             0x00
7       2        0x0400000000004000  1168195766.782811  0.018567   0.000086   0.018567      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4076     128.121.136.217  us       NTT America                   30012    6        0x00             0x00
8       1        0x0400000000004000  1168195766.782932  0.030922   0.018881   0.043003      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4075     128.121.136.217  us       NTT America                   21       6        0x01             0x09     SIZE /funwithbill/Microsoft-1978.jpg\r\n
9       1        0x0400000000004001  1168195766.806945  0.042894   0.024013   0.055293      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4075     6        0x01             0x09     213 64170\r\n
10      1        0x0400000000004000  1168195766.807113  0.024181   0.000168   0.067184      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4075     128.121.136.217  us       NTT America                   21       6        0x01             0x09     RETR /funwithbill/Microsoft-1978.jpg\r\n
11      1        0x0400000000004001  1168195766.818968  0.012023   0.011855   0.067316      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4075     6        0x01             0x09     150 Opening BINARY mode data connection for /funwithbill/Microsoft-1978.jpg (64170 bytes)\r\n
12      2        0x0400000000004001  1168195766.827519  0.044794   0.044708   0.044794      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   30012    67.180.72.76     us       Comcast Cable Communications  4076     6        0x00             0x02     ......JFIF.....,.,.....C......................\n.....\n...\n..\r...\r............................C.......\t..\t.\r.\r.............................................................".....................................\t\n.....................}........!1A..Qa."q.2....#B...R..$3br.\t\n.....%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz.......................................................................................................\t\n.....................w.......!1..AQ.aq."2...B....\t#3R..br.\n.$4.%.....&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz....................................................................................?..E..+SP..(...(...(...(...(...(.....> .|'.-{Z.4...WN....KY<....|...x....4O..@......7>!.....R~..._.u...~...P..Q^.7.U...5x{....\rGP.R..M.!........Q'<..`...F._..?..\r....+...3u$V.Ms.2....(....2:Z+...*.kZ.b. ...Z..4V.i%.....H..fA..i\..C..}.....<Awm%.....,Q..+[.......|?/.$.`.......[.,.../..y.O3..4..W........eo....V..W..y,..,^_.....N....|G....K-7X.m.5[...O........2.......Rx.....gc....o._../CI..w?j./&H....f.
13      2        0x0400000000004001  1168195766.828740  0.001221   0.045929   0.046015      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   30012    67.180.72.76     us       Comcast Cable Communications  4076     6        0x00             0x02     2=..+.....O.x.].tM/W..W...-J.....Fc....5:Z+...w...S.,t}R.B.G.67............J.o..\n....!.....X..-.....H..y...q..E.w.VU..4.......V.Z4..j...O2)".|.........5..k..x\.}..T......(.....-j..Z...i.)w....^[\..H..H..V......:.Y...Y[Gwu..w:.v..G/....(..}......<...1.....~.I..e.......2.._..lyqbIq..^....:>.-...p.E..-k#"......./.x.-.\r^(..a.LV7q.k4...H..~.}+R.......C....G_.....O....E'..Z...V.......+..5c.[]x.G.9o...l$..U.~U...b...}.mq.G.ig.....s...&......+##...iZ...cm}es....^lWP..G,.Z....O.|..kK%.....\...Hm"...2..ryq..q..@.-..xc..j...<Quql-f.....O.I...+..)|...............m.5..Sk>.!.f..9%..?.k ......=....M......4.KT.....J.+[[....._._..O.W..?..........u...2....=".w?.z...h.s.~<.|.%.z..\r.....[v?....".~.O.4xc....%...5H.o4..ol..S[7....G..~............jz........./b.....4..._*3..5............u(n....)o.../6Y".?0..ZB:Z+..Y.OD....;]..R.+..o..P..)c.A.......4.../.h........Y....r\.e.I..|..i.......I...G?............hy...^3.y........?.|....>3........u..Ib.....Rc.....h?h{...K...S....*-9~...s../.....x............x.....d....t.V.............K.....&....z..QZ...Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@..Q@....J.G.w.../.|q..Q...~..z....M.9%.m....4I.....C....M.....O.;.>...u.j:.....'W..'..H...../...w..#..X9.W.|g.1.............\.\.B=3V.Hm/b....2...T.9....+##..u......$..[..I.....Dj.......nzy^a.!.^..<.3Z~;.o.Y~<x_......E...Z-...s%....K.. ._............\t.v.....?..a...>...:~.q,.C7....d.G.......o.w..F._.i....<.,..F..=k.E.....M$....Q.....?.<O....|P...N..............l....H"..X.U...%..
...

From the end report alone we can deduct that there is extractable content available. From the flow file we know its name and consistency status. Now we like to see it. Let’s data carve.

Data Carving with ftpDecode

In order to enable the Data Carving mode FTP_SAVE has to be switched on. Use t2conf and t2build to reconfigure and recompile ftpDecode and rerun t2 on the pcap.

t2conf ftpDecode -D FTP_SAVE=1 && t2build ftpDecode

t2 -r ~/test_data/data/ftp-dpic.pcap -w ~/results

...
--------------------------------------------------------------------------------
tcpStates: Aggregated tcpStatesAFlags=0xc3
ftpDecode: Aggregated ftpStat=0x0f
ftpDecode: Number of FTP control packets: 20 [21.74%]
ftpDecode: Number of FTP-DATA packets: 69 [75.00%]
ftpDecode: Number of files extracted: 1
--------------------------------------------------------------------------------
...

We see that on content file was extracted and the status confirms that the file was properly extracted.

tawk -V ftpStat=0x0f

The ftpStat column with value 0x0f is to be interpreted as follows:

   bit | ftpStat | Description
   =============================================================================
     0 | 0x01    | FTP control port found
     1 | 0x02    | FTP passive parent flow
     2 | 0x04    | FTP passive parent flow write finished
     3 | 0x08    | FTP active parent flow

The flow file looks the same, except that the B flow 2 has the bit 0x04 set, indicating successful extraction.

tcol ~/results/ftp-dpic_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration   numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPOrg                        srcPort  dstIP            dstIPCC  dstIPOrg                        dstPort  l4Proto  tcpStatesAFlags  ftpStat  ftpCDFindex  ftpCC                ftpRC                ftpNumUser  ftpUser  ftpNumPass  ftpPass  ftpNumCP  ftpCP
A     2        0x0400000000004000  1168195766.764244  1168195766.976730  0.212486   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4076     128.121.136.217  us       "NTT America"                   30012    6        0x42             0x02     1                                                      0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
B     2        0x0400000000004001  1168195766.782725  1168195766.965575  0.182850   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   30012    67.180.72.76     us       "Comcast Cable Communications"  4076     6        0x02             0x06     1                                                      0                    0                    1         "_funwithbill_Microsoft-1978.jpg"
A     1        0x0400000000004000  1168195766.739929  1168195799.748737  33.008808  1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4075     128.121.136.217  us       "NTT America"                   21       6        0x43             0x09     2            TYPE;PASV;SIZE;RETR                       0                    0                    2         "I";"/funwithbill/Microsoft-1978.jpg"
B     1        0x0400000000004001  1168195766.751652  1168195767.017088  0.265436   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4075     6        0x03             0x09     2                                 200;227;213;150;226  0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
A     3        0x0400000000004000  1168195798.794148  1168195798.808733  0.014585   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4072     128.121.136.217  us       "NTT America"                   21       6        0x83             0x01                                                            0                    0                    0
B     3        0x0400000000004001  1168195798.808082  1168195798.808646  0.000564   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4072     6        0x03             0x01                                                            0                    0                    0
A     4        0x0400000000004000  1168195799.515059  1168195799.527966  0.012907   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4071     128.121.136.217  us       "NTT America"                   21       6        0x83             0x01                                                            0                    0                    0
B     4        0x0400000000004001  1168195799.527449  1168195799.527938  0.000489   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4071     6        0x03             0x01                                                            0                    0                    0

By default the extracted files reside under the /tmp folder.

ls /tmp/FTPFILES/

ftp_flow_1_A.txt  ftp_flow_1_B.txt  _funwithbill_Microsoft-1978.jpg_2_B

The first two files are the command and response from flow 1, the third is our extracted file. T2 adds the file index and the flow direction to the file name. Let’s look into the files.

cat /tmp/FTPFILES/ftp_flow_1_A.txt

TYPE I
PASV
SIZE /funwithbill/Microsoft-1978.jpg
RETR /funwithbill/Microsoft-1978.jpg

cat /tmp/FTPFILES/ftp_flow_1_B.txt

200 Type set to I
227 Entering Passive Mode (128,121,136,217,117,60).
213 64170
150 Opening BINARY mode data connection for /funwithbill/Microsoft-1978.jpg (64170 bytes)
226 Transfer complete.

eog _funwithbill_Microsoft-1978.jpg_2_B

You know these murky guys, right?

We missed something from the tcpStates plugin in the end report:

tawk -V tcpStatesAFlags=0xc3

The tcpStatesAFlags column with value 0xc3 is to be interpreted as follows:

   bit | tcpStatesAFlags | Description
   =============================================================================
     0 | 0x01            | Malformed connection establishment
     1 | 0x02            | Malformed teardown
     6 | 0x40            | Reset from sender
     7 | 0x80            | Potential evil behavior (scan)

Bits 0, 1 and 6 can happen, sometimes computers are very impatient, especially when the OS of the guys above is involved. The evil behavior is an estimate, which is often a false positive, because of the tear down sequence, which might happen in scans.

In order to verify load tcpFlags, basicFlow and tp0f, recompile, rerun t2 and look into the flow file again.

You will see that Windows is indeed the originator of abnormal tear-downs in flow 1 and 2 and the evil bit in the Microsoft part of flow 3 and 4. That seems appropriate :-)

Additionally in tcpAnomaly flags scan behavior in flow 3 and 4.

t2build basicStats tcpFlags tp0f

t2 -r ~/test_data/data/ftp-dpic.pcap -w ~/results

tcol ~/results/ftp-dpic_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration   numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPOrg                        srcPort  dstIP            dstIPCC  dstIPOrg                        dstPort  l4Proto  tp0fStat  tp0fDis  tp0fClName  tp0fPrName  tp0fVerName  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT       stdIAT        pktps      bytps     pktAsm      bytAsm  tcpFStat  ipMindIPID  ipMaxdIPID  ipMinTTL  ipMaxTTL  ipTTLChg  ipToS  ipFlags  ipOptCnt  ipOptCpCl_Num    ip6OptCntHH_D  ip6OptHH_D             tcpISeqN    tcpPSeqCnt  tcpSeqSntBytes  tcpSeqFaultCnt  tcpPAckCnt  tcpFlwLssAckRcvdBytes  tcpAckFaultCnt  tcpBFlgtMx  tcpInitWinSz  tcpAveWinSz  tcpMinWinSz  tcpMaxWinSz  tcpWinSzDwnCnt  tcpWinSzUpCnt  tcpWinSzChgDirCnt  tcpWinSzThRt  tcpFlags  tcpAnomaly  tcpOptPktCnt  tcpOptCnt  tcpOptions  tcpMSS  tcpWS  tcpMPTBF  tcpMPF  tcpMPAID  tcpMPdssF  tcpTmS  tcpTmER  tcpEcI  tcpUtm    tcpBtm    tcpSSASAATrip  tcpRTTAckTripMin  tcpRTTAckTripMax  tcpRTTAckTripAve  tcpRTTAckTripJitAve  tcpRTTSseqAA  tcpRTTAckJitAve  tcpStatesAFlags  ftpStat  ftpCDFindex  ftpCC                ftpRC                ftpNumUser  ftpUser  ftpNumPass  ftpPass  ftpNumCP  ftpCP
A     2        0x0400000000004000  1168195766.764244  1168195766.976730  0.212486   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4076     128.121.136.217  us       "NTT America"                   30012    6        0x01      0        win         Windows     XP           26          46           0            64170         0         0         0           0           0       0.04601   0.008172538  0.008155256   122.361    0         -0.2777778  -1      0x0811    1           3           128       128       0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2515816506  25          0               0               25          64171                  0               0           16384         12186.99     0            17520        2               1              3                  0             0x0416    0x0000      1             4          0x00000016  1460    0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0.018481       0.001129          0.045929          0.007008825       0.008882974          0.018567      0                0x42             0x02     1                                                      0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
B     2        0x0400000000004001  1168195766.782725  1168195766.965575  0.182850   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   30012    67.180.72.76     us       "Comcast Cable Communications"  4076     6        0x02      12       unix        HP-UX       11.x         46          26           64170        0             0         1460      1395        240.5472    0       0.044794  0.003975001  0.006068233   251.5723   350943.4  0.2777778   1       0x0811    1           141         52        52        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  1720743169  46          63804           0               46          0                      0               2920        32768         33580        32768        33580        0               1              1                  0             0x031b    0x0000      1             1          0x00000004  1460    0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  8.6e-05        3.7e-05           0.011155          0.0004865769      0.002092311          0.007495402   0.009126062      0x02             0x06     1                                                      0                    0                    1         "_funwithbill_Microsoft-1978.jpg"
A     1        0x0400000000004000  1168195766.739929  1168195799.748737  33.008808  1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4075     128.121.136.217  us       "NTT America"                   21       6        0x00      0        win         Windows     unknown      7           5            90           198           0         38        12.85714    14.05432    0       32.58816  4.715544     10.53506      0.2120646  2.726545  0.1666667   -0.375  0x0011    1           21          128       128       0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2953041913  7           90              0               7           198                    0               38          17440         12126.16     0            17440        6               0              0                  0             0x041c    0x0000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              0.011723          0.057773          0.023481          0.01581577           0             0                0x43             0x09     2            TYPE;PASV;SIZE;RETR                       0                    0                    2         "I";"/funwithbill/Microsoft-1978.jpg"
B     1        0x0400000000004001  1168195766.751652  1168195767.017088  0.265436   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4075     6        0x00      12       unix        Linux       unknown      5           7            198          90            11        91        39.6        24.76444    0       0.19812   0.0530872    0.0658703     18.83693   745.9426  -0.1666667  0.375   0x0011    1           967         52        52        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2248198007  5           174             0               5           82                     0               91          33580         33580        33580        33580        0               0              0                  0             0x0018    0x0000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              0.000168          32.73165          4.719271          10.58782             4.742752      10.58783         0x03             0x09     2                                 200;227;213;150;226  0                    0                    1         "/funwithbill/Microsoft-1978.jpg"
A     3        0x0400000000004000  1168195798.794148  1168195798.808733  0.014585   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4072     128.121.136.217  us       "NTT America"                   21       6        0x00      0        win         Windows     unknown      2           2            0            0             0         0         0           0           0       0.014585  0.0072925    0.005156576   137.1272   0         0           0       0x0011    1           1           128       128       0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  214829931   2           0               0               2           0                      0               0           16860         16860        16860        16860        0               0              0                  0             0x0111    0x3000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              0.013934          0.014498          0.014216          0.000199404          0             0                0x83             0x01                                                            0                    0                    0
B     3        0x0400000000004001  1168195798.808082  1168195798.808646  0.000564   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4072     6        0x00      12       unix        Linux       unknown      2           2            0            0             0         0         0           0           0       0.000564  0.000282     0.0001994041  3546.099   0         0           0       0x0011    1           1           52        52        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  308530437   2           0               0               2           0                      0               0           33580         33580        33580        33580        0               0              0                  0             0x0111    0x2000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              8.7e-05           8.7e-05           4.35e-05          3.075914e-05         0.0142595     0.0002017624     0x03             0x01                                                            0                    0                    0
A     4        0x0400000000004000  1168195799.515059  1168195799.527966  0.012907   1           3        eth:ipv4:tcp  00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800              67.180.72.76     us       "Comcast Cable Communications"  4071     128.121.136.217  us       "NTT America"                   21       6        0x00      0        win         Windows     unknown      2           2            0            0             0         0         0           0           0       0.012907  0.0064535    0.004563314   154.9547   0         0           0       0x0011    1           1           128       128       0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  2886500455  2           0               0               2           0                      0               0           17218         17218        17218        17218        0               0              0                  0             0x0111    0x3000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              0.01239           0.012879          0.0126345         0.0001728877         0             0                0x83             0x01                                                            0                    0                    0
B     4        0x0400000000004001  1168195799.527449  1168195799.527938  0.000489   1           3        eth:ipv4:tcp  00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800              128.121.136.217  us       "NTT America"                   21       67.180.72.76     us       "Comcast Cable Communications"  4071     6        0x00      12       unix        Linux       unknown      2           2            0            0             0         0         0           0           0       0.000489  0.0002445    0.0001728876  4089.979   0         0           0       0x0011    1           1           52        52        0         0x00   0x0040   0         0x00_0x00000000  0_0            0x00000000_0x00000000  785817037   2           0               0               2           0                      0               0           33580         33580        33580        33580        0               0              0                  0             0x0111    0x2000      0             0          0x00000000  0       0      0x0000    0x00    0         0x00       0       0        0       0.000000  0.000000  0              2.8e-05           2.8e-05           1.4e-05           9.899495e-06         0.0126485     0.0001731709     0x03             0x01                                                            0                    0                    0

So does the Anteater smell evil OS or behavior? Let’s look at flow 4 in detail:

tawk 'hdr() || flow(4)' ~/results/ftp-dpic_packets.txt | tcol

%pktNo  flowInd  flowStat            time               pktIAT    pktTrip   flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP            srcIPCC  srcIPOrg                      srcPort  dstIP            dstIPCC  dstIPOrg                      dstPort  l4Proto  pktLen  l7Len  ipToS  ipID   ipIDDiff  ipFrag  ipTTL  ipHdrChkSum  ipCalChkSum  l4HdrChkSum  l4CalChkSum  ipFlags  ip6HHOptLen  ip6HHOpts  ip6DOptLen  ip6DOpts  ipOptLen  ipOpts  seq         ack         seqMax      seqDiff  ackDiff  seqLen  ackLen  seqFlowLen  ackFlowLen  tcpMLen  tcpBFlgt  tcpFStat  tcpFlags  tcpAnomaly  tcpWin  tcpWS  tcpMSS  tcpTmS  tcpTmER  tcpOptLen  tcpOpts  tcpStatesAFlags  ftpStat  l7Content
88      4        0x0400000000004000  1168195799.515059  0.000000  0.000000  0.000000      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4071     128.121.136.217  us       NTT America                   21       6        54      0      0x00   34267  0         0x4000  128    0xdfa1       0xdfa1       0xce2a       0xce2a       0x0040   0                       0                     0                 2886500455  785817037   2886500455  0        0        0       0       0           0           0        0          0x0010    0x0111    0x0000      17218   0      0       0       0        0                   0x01             0x01
89      4        0x0400000000004001  1168195799.527449  0.000000  0.012390  0.000000      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4071     6        60      0      0x00   20400  0         0x4000  52     0x61cd       0x61cd       0x8e40       0x8e40       0x0040   0                       0                     0                 785817037   2886500456  785817037   0        0        0       0       0           0           0        0          0x0010    0x0010    0x0000      33580   0      0       0       0        0                   0x01             0x01
90      4        0x0400000000004001  1168195799.527938  0.000489  0.012879  0.000489      3        eth:ipv4:tcp             00:01:5c:22:a5:82  00:16:36:a9:08:20  0x0800   128.121.136.217  us       NTT America                   21       67.180.72.76     us       Comcast Cable Communications  4071     6        60      0      0x00   20401  1         0x4000  52     0x61cc       0x61cc       0x8e3f       0x8e3f       0x0040   0                       0                     0                 785817037   2886500456  785817037   0        0        0       0       0           0           0        0          0x0011    0x0111    0x0000      33580   0      0       0       0        0                   0x01             0x01
91      4        0x0400000000004000  1168195799.527966  0.012907  0.000028  0.012907      3        eth:ipv4:tcp             00:16:36:a9:08:20  00:01:5c:22:a5:82  0x0800   67.180.72.76     us       Comcast Cable Communications  4071     128.121.136.217  us       NTT America                   21       6        54      0      0x00   34268  1         0x4000  128    0xdfa0       0xdfa0       0xce29       0xce29       0x0040   0                       0                     0                 2886500456  785817038   2886500456  1        1        0       0       0           0           0        0          0x0011    0x0010    0x0000      17218   0      0       0       0        0                   0x01

No indication in the packet file.

So why the flags in the flow file?

On flow terminate the tcpStates collects all information about the compliance of the RFC TCP state-machine. If it sees certain aberration, it compares it also with facts known from scans, and if all fits it flags it as EVIL. Now, that can go wrong, as you saw here, because sometimes programmers do not comply with standards. Even tcpFlags tripped.

So always check your 6 and look into the packet file if in doubt.

Conclusion

Try your own FTP traffic. Don’t forget to reset the plugin configuration for the next tutorial.

t2conf ftpDecode --reset && t2build ftpDecode

Have fun with FTP!