Tranalyzer - Plugin Packet Mode

Tutorial: Plugin Packet Mode

Adding the Packet Mode
See Also

The packet mode of T2 is enabled with the -s command line option. Each plugin can implement code activated by a sPktFile switch activated by -s. Then, its contribution is added in plugin order to the packet file.

Before we start doing this, clean out your plugin .so directory if this is the first tutorial you follow. Then all unnecessary plugins should be deleted from the plugin folder ~/.tranalyzer/plugins and compile basicFlow, basicStats and txtSink.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow txtSink
...
BUILD SUCCESSFUL
$

If you didn’t read the tutorials before, here is the basis plugin which we will extend: tcpWin

The anonymized sample pcap can be downloaded here: annoloc2.pcap. Please extract it under your data folder: ~/data/, if you haven’t already. Now you are all set for packet mode fun.

Adding the Packet Mode

In order to implement the packet mode, the sPktFile switch has to be added to the initialize() function of your plugin and the header of the packet file has to be defined: (see also the t2PSkel skeleton plugin)

So open tcpWin.c and add in the initialize() callback the line marked by <--, a simple fputs function into the packet file, denoting the header description in the packet file.

...
// * This function is called before processing any packet.
void initialize() {
    // allocate struct for all flows and initialise to 0
    T2_PLUGIN_STRUCT_NEW(tcpWinFlows);

    // Packet mode
    if (sPktFile) fputs("winSize\twinThPktCnt\t", sPktFile); // <-- Note the trailing tab (\t)
}
...

Now we need to output data for every packet. Add in claimLayer4Information(...) callback the line marked by <--, again a simple fprintf into the packet file. Note the trailing \t in the format, do NOT forget it!

...
void claimLayer4Information(packet_t *packet, unsigned long flowIndex) {
    flow_t * const flowP = &flows[flowIndex];
    if (flowP->layer4Protocol != L3_TCP) {             // <-- process only TCP
                if (sPktFile) fputs("\t\t", sPktFile); // <-- if not tcp we need to print empty columns
                return;                                // <-- go back to core
        }

    // only 1. frag packet will be processed
    if (!t2_is_first_fragment(packet)) return;

    tcpWinFlow_t * const tcpWinFlowP = &tcpWinFlows[flowIndex];
    const tcpHeader_t * const tcpHeader = (tcpHeader_t*)packet->layer4Header; // cast layer4Header to tcpHeader struct
    const uint32_t tcpWin = ntohs(tcpHeader->window);                         // convert window size to little endian

    if (tcpWin < TCPWIN_THRES) {          // is windowsize below threshold?
        tcpWinFlowP->winThCnt++;          // count the packet / flow
        tcpWinFlowP->stat |= TCPWIN_THU;  // set the status bit
    }

    // Packet mode
    if (sPktFile) fprintf(sPktFile, "%"PRIu32"\t%"PRIu32"\t", tcpWin, tcpWinFlowP->winThCnt); // <-- Note the trailing tab (\t)
}
...

Done? No, here comes the catch, as already explained before in buildyourownplugin, we need now to compensate the missing two L4 columns of our plugin in case of a pure L2 flow appears. It won’t in our pcap, but it may if you use your own traffic. In case of ‘-s’ option set you need to output two tabs, so that the columns match at the L7content column produced by the core. Place the following code before the claimLayer4Information function.

#if ETH_ACTIVATE > 0
/*
 * This function is called for every packet with a layer 2.
 * If flowIndex is HASHTABLE_ENTRY_NOT_FOUND, this means the packet also
 * has a layer 4 and thus a call to claimLayer4Information() will follow.
 */
void claimLayer2Information(packet_t *packet, unsigned long flowIndex) {

    if (flowIndex == HASHTABLE_ENTRY_NOT_FOUND) return;

    // This packet does not have a layer 4.
    // Print the appropriate amount of tabs to keep the packet file aligned
    if (sPktFile) fputs("\t\t", sPktFile);
}
#endif // ETH_ACTIVATE > 0

The code is added if ETH_ACTIVATE is activated in the core.

Now compile tcpWin and rerun T2 with the -s option. So T2 will produce a flow and packet file now.

$ t2build tcpWin
Plugin 'tcpWin'

ninja: Entering directory `build'
[1/2] Compiling C object libtcpWin.so.p/src_tcpWin.c.o
../src/tcpWin.c: In function ‘claimLayer2Information’:
../src/tcpWin.c:84:39: warning: unused parameter ‘packet’ [-Wunused-parameter]
   84 | void claimLayer2Information(packet_t *packet, unsigned long flowIndex) {
      |                             ~~~~~~~~~~^~~~~~
[2/2] Linking target libtcpWin.so

tcpWin successfully built


Plugin tcpWin copied into /home/wurst/.tranalyzer/plugins


BUILDING SUCCESSFUL

$ t2 -r ~/data/annoloc2.pcap -w ~/results -s
================================================================================
Tranalyzer 0.8.11 (Anteater), Tarantula. PID: 33774
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.11
    02: tcpWin, 0.8.11
    03: txtSink, 0.8.11
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406208 (406.21 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51196 (51.20 K)
Processing file: /home/wurst/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1022171701.691172 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
...
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 payload length > framing length
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
...
$

Ah ETH_ACTIVATE is off by default, therefore the warning. As we also loaded txtSink, which is not necessary, but you can look at it, that T2 does everything in parallel correctly. So you do not need a onFlowTerminated callback if you only want to produce a packet file.

Now change to your results window and look at the packet file. I extracted some interesting lines already for you.

$ tawk 'hdr() || $winThPktCnt > 0 && $winSize == 0' annoloc2_packets.txt | head | tcol
%pktNo  flowInd  flowStat            time               pktIAT    flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP            srcIPCC  srcIPOrg                        srcPort  dstIP            dstIPCC  dstIPOrg                        dstPort  l4Proto  winSize  winThPktCnt  l7Content
521     265      0x0400000000004000  1022171701.709116  0.000000  0.000000      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:50:fc:0e:21:56  0x0800   209.171.12.143   ca       TELUS Communications Inc        4987     138.212.185.230  jp       ASAHI KASEI CORPORATION         41250    6        0        1
1159    60       0x0400000000004001  1022171701.720657  0.000000  0.000000      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:50:fc:26:95:88  0x0800   193.87.5.62      sk       Zdruzenie pouzivatelov Slovens  62486    138.212.188.178  jp       ASAHI KASEI CORPORATION         2100     6        0        1
1167    447      0x0400000000004000  1022171701.721366  0.000000  0.000000      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:50:fc:3b:62:78  0x0800   217.41.129.13    gb       BT Infrastructure Layer         58872    138.212.187.186  jp       ASAHI KASEI CORPORATION         80       6        0        1
1497    523      0x0400000000004001  1022171701.729052  0.000000  0.000000      3        eth:ipv4:tcp             00:00:b4:a9:15:71  00:d0:02:6d:78:00  0x0800   138.212.185.150  jp       ASAHI KASEI CORPORATION         1207     212.223.121.197  de       Ratiokontakt GmbH               8000     6        0        1
1684    392      0x0400000000004001  1022171701.732313  0.000000  0.000000      3        eth:ipv4:tcp             00:50:bf:59:85:48  00:d0:02:6d:78:00  0x0800   138.212.188.67   jp       ASAHI KASEI CORPORATION         1214     36.242.181.230   jp       SoftBank Corp                   4685     6        0        1
2004    176      0x0400000000004000  1022171701.739385  0.035669  0.035669      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:80:48:b3:13:27  0x0800   216.237.125.166  us       Infortech Corporation           3507     138.212.184.193  jp       ASAHI KASEI CORPORATION         8080     6        0        1
2232    633      0x0400000200004001  1022171701.743464  0.000000  0.000000      3        eth:ipv4:tcp             00:60:08:78:1b:63  00:d0:02:6d:78:00  0x0800   138.212.187.203  jp       ASAHI KASEI CORPORATION         6699     19.123.222.7     us       Ford Motor Company              1430     6        0        1
2295    642      0x0400000000004000  1022171701.744654  0.000000  0.000000      3        eth:ipv4:tcp             00:80:48:cd:8c:82  00:d0:02:6d:78:00  0x0800   138.212.186.160  jp       ASAHI KASEI CORPORATION         1217     70.196.57.198    us       Cellco Partnership DBA Verizon  9000     6        0        1
3975    176      0x0400000000004000  1022171701.779710  0.039558  0.075994      3        eth:ipv4:tcp             00:d0:02:6d:78:00  00:80:48:b3:13:27  0x0800   216.237.125.166  us       Infortech Corporation           3507     138.212.184.193  jp       ASAHI KASEI CORPORATION         8080     6        0        2

That was not so hard, right? . For researchers: Note that you already have the packet inter-distances per flowIndex and flow direction available. Why is there no L7 content? Have a look what T2 warns you about… the snap length! Use another pcap: faf-exercise.pcap and you will see L7 content. I extracted the FTP command/control flow 35 below.

$ t2 -r ~/data/faf-exercise.pcap -w ~/results -s
...
$ tawk 'flow(35)' faf-exercise_packets.txt | tcol
%pktNo  flowInd  flowStat            time               pktIAT     flowDuration  numHdrs  hdrDesc       ethVlanID  srcMac             dstMac             ethType  srcIP          srcIPCC  srcIPOrg         srcPort  dstIP          dstIPCC  dstIPOrg         dstPort  l4Proto  winSize  winThPktCnt  l7Content
1266    35       0x0400000000004000  1258594162.928342  0.000000   0.000000      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        8192     0
1267    35       0x0400000000004001  1258594163.008594  0.000000   0.000000      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4140     0
1268    35       0x0400000000004000  1258594163.009292  0.080950   0.080950      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64860    0
1269    35       0x0400000000004001  1258594163.087792  0.079198   0.079198      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4140     0            220 Microsoft FTP Service\r\n
1270    35       0x0400000000004000  1258594163.088491  0.079199   0.160149      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64833    0            USER anonymous\r\n
1271    35       0x0400000000004001  1258594163.166256  0.078464   0.157662      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4156     0            331 Anonymous access allowed, send identity (e-mail name) as password.\r\n
1272    35       0x0400000000004000  1258594163.168693  0.080202   0.240351      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        64761    0            PASS IEUser@\r\n
1273    35       0x0400000000004001  1258594163.247178  0.080922   0.238584      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4170     0            230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.\r\n    For information about DELL, call +1 800 999 3355 All transfers are logged with\r\n    your host name and email address. If you don't like this policy please disconnect now.\r\n    Please be advised that use constitutes consent to monitoring (Elec Comm Priv Act,\r\n    18 USC 2701-2711). Please see the file readme.txt for disclaimers pertaining to this\r\n    service. If your FTP client crashes or hangs shortly after login, try using a dash\r\n    (-) as the first character of your password. This will turn off the informational\r\n    messages which may be confusing your ftp client.\r\n    ********IN CASE OF PROBLEMS*************************\r\n    ** File Content: send EMAIL to dellbbs@dell.com   **\r\n    ** FTP Server: send EMAIL to hostmaster@dell.com  **\r\n    ** WWW Server: send EMAIL to webmaster@dell.com   **\r\n    ****************************************************\r\n
1274    35       0x0400000000004001  1258594163.247187  0.000009   0.238593      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4170     0            230 User logged in.\r\n
1275    35       0x0400000000004000  1258594163.247637  0.078944   0.319295      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63790    0
1276    35       0x0400000000004000  1258594163.249385  0.001748   0.321043      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63790    0            TYPE I\r\n
1277    35       0x0400000000004001  1258594163.327121  0.079934   0.318527      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4178     0            200 Type set to I.\r\n
1278    35       0x0400000000004000  1258594163.327845  0.078460   0.399503      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63770    0            PASV\r\n
1279    35       0x0400000000004001  1258594163.407582  0.080461   0.398988      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4184     0            227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283    35       0x0400000000004000  1258594163.487490  0.159645   0.559148      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63720    0            SIZE /video/R79733.EXE\r\n
1284    35       0x0400000000004001  1258594163.565990  0.158408   0.557396      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4208     0            213 4255056\r\n
1285    35       0x0400000000004000  1258594163.566694  0.079204   0.638352      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63707    0            RETR /video/R79733.EXE\r\n
1286    35       0x0400000000004001  1258594163.644188  0.078198   0.635594      3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4232     0            125 Data connection already open; Transfer starting.\r\n
1303    35       0x0400000000004000  1258594163.838277  0.271583   0.909935      3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63653    0
5898    35       0x0400000000004001  1258594185.427515  21.783327  22.418921     3        eth:ipv4:tcp             00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800   143.166.11.10  us       Dell             21       192.168.1.105  07       Private network  49329    6        4232     0            226 Transfer complete.\r\n
5900    35       0x0400000000004000  1258594185.618346  21.780069  22.690004     3        eth:ipv4:tcp             00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800   192.168.1.105  07       Private network  49329    143.166.11.10  us       Dell             21       6        63629    0

Why are there no packets below the defined windows threshold? Can you find the flow which has some? I leave that to you. Or change the threshold in tcpWin.h.

Have fun with packet mode programming!

The next tutorial will teach you how to produce summary files

Tutorial: Plugin Packet Mode

Contents

Adding the Packet Mode

See Also