Tutorial: Plugin Packet Mode
Contents
The packet mode of T2 is enabled with the -s
command line option. Each plugin can implement code activated by a sPktFile switch activated by -s
. Then, its contribution is added in plugin order to the packet file.
Before we start doing this, clean out your plugin .so directory if this is the first tutorial you follow. Then all unnecessary plugins should be deleted from the plugin folder ~/.tranalyzer/plugins
and compile basicFlow, basicStats and txtSink.
$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow txtSink
...
BUILD SUCCESSFUL
$
If you didn’t read the tutorials before, here is the basis plugin which we will extend: tcpWin
The anonymized sample pcap can be downloaded here: annoloc2.pcap. Please extract it under your data folder: ~/data/, if you haven’t already. Now you are all set for packet mode fun.
Adding the Packet Mode
In order to implement the packet mode, the sPktFile switch has to be added to the initialize()
function of your plugin and the header of the packet file has to be defined: (see also the t2PSkel skeleton plugin)
So open tcpWin.c and add in the initialize()
callback the line marked by <--
, a simple fputs
function into the packet file, denoting the header description in the packet file.
...
// * This function is called before processing any packet.
void initialize() {
// allocate struct for all flows and initialise to 0
T2_PLUGIN_STRUCT_NEW(tcpWinFlows);
// Packet mode
if (sPktFile) fputs("winSize\twinThPktCnt\t", sPktFile); // <-- Note the trailing tab (\t)
}
...
Now we need to output data for every packet. Add in claimLayer4Information(...)
callback the line marked by <--
, again a simple fprintf
into the packet file. Note the trailing \t
in the format, do NOT forget it!
...
void claimLayer4Information(packet_t *packet, unsigned long flowIndex) {
flow_t * const flowP = &flows[flowIndex];
if (flowP->layer4Protocol != L3_TCP) { // <-- process only TCP
if (sPktFile) fputs("\t\t", sPktFile); // <-- if not tcp we need to print empty columns
return; // <-- go back to core
}
// only 1. frag packet will be processed
if (!t2_is_first_fragment(packet)) return;
tcpWinFlow_t * const tcpWinFlowP = &tcpWinFlows[flowIndex];
const tcpHeader_t * const tcpHeader = (tcpHeader_t*)packet->layer4Header; // cast layer4Header to tcpHeader struct
const uint32_t tcpWin = ntohs(tcpHeader->window); // convert window size to little endian
if (tcpWin < TCPWIN_THRES) { // is windowsize below threshold?
tcpWinFlowP->winThCnt++; // count the packet / flow
tcpWinFlowP->stat |= TCPWIN_THU; // set the status bit
}
// Packet mode
if (sPktFile) fprintf(sPktFile, "%"PRIu32"\t%"PRIu32"\t", tcpWin, tcpWinFlowP->winThCnt); // <-- Note the trailing tab (\t)
}
...
Done? No, here comes the catch, as already explained before in buildyourownplugin, we need now to compensate the missing two L4 columns of our plugin in case of a pure L2 flow appears. It won’t in our pcap, but it may if you use your own traffic. In case of ‘-s’ option set you need to output two tabs, so that the columns match at the L7content
column produced by the core. Place the following code before the claimLayer4Information
function.
#if ETH_ACTIVATE > 0
/*
* This function is called for every packet with a layer 2.
* If flowIndex is HASHTABLE_ENTRY_NOT_FOUND, this means the packet also
* has a layer 4 and thus a call to claimLayer4Information() will follow.
*/
void claimLayer2Information(packet_t *packet, unsigned long flowIndex) {
if (flowIndex == HASHTABLE_ENTRY_NOT_FOUND) return;
// This packet does not have a layer 4.
// Print the appropriate amount of tabs to keep the packet file aligned
if (sPktFile) fputs("\t\t", sPktFile);
}
#endif // ETH_ACTIVATE > 0
The code is added if ETH_ACTIVATE
is activated in the core.
Now compile tcpWin and rerun T2 with the -s
option. So T2 will produce a flow and packet file now.
$ t2build tcpWin
Plugin 'tcpWin'
ninja: Entering directory `build'
[1/2] Compiling C object libtcpWin.so.p/src_tcpWin.c.o
../src/tcpWin.c: In function ‘claimLayer2Information’:
../src/tcpWin.c:84:39: warning: unused parameter ‘packet’ [-Wunused-parameter]
84 | void claimLayer2Information(packet_t *packet, unsigned long flowIndex) {
| ~~~~~~~~~~^~~~~~
[2/2] Linking target libtcpWin.so
tcpWin successfully built
Plugin tcpWin copied into /home/wurst/.tranalyzer/plugins
BUILDING SUCCESSFUL
$ t2 -r ~/data/annoloc2.pcap -w ~/results -s
================================================================================
Tranalyzer 0.8.11 (Anteater), Tarantula. PID: 33774
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
01: basicFlow, 0.8.11
02: tcpWin, 0.8.11
03: txtSink, 0.8.11
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406208 (406.21 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51196 (51.20 K)
Processing file: /home/wurst/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1022171701.691172 sec (Thu 23 May 2002 16:35:01 GMT)
[WRN] snapL2Length: 54 - snapL3Length: 40 - IP length in header: 1500
...
[WRN] L3 SnapLength < Length in IP header
[WRN] L4 header snapped
[WRN] Consecutive duplicate IP ID
[WRN] IPv4/6 payload length > framing length
[WRN] IPv4/6 fragmentation header packet missing
[WRN] IPv4/6 packet fragmentation sequence not finished
...
$
Ah ETH_ACTIVATE
is off by default, therefore the warning. As we also loaded txtSink, which is not necessary, but you can look at it, that T2 does everything in parallel correctly. So you do not need a onFlowTerminated callback if you only want to produce a packet file.
Now change to your results window and look at the packet file. I extracted some interesting lines already for you.
$ tawk 'hdr() || $winThPktCnt > 0 && $winSize == 0' annoloc2_packets.txt | head | tcol
%pktNo flowInd flowStat time pktIAT flowDuration numHdrs hdrDesc ethVlanID srcMac dstMac ethType srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto winSize winThPktCnt l7Content
521 265 0x0400000000004000 1022171701.709116 0.000000 0.000000 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:fc:0e:21:56 0x0800 209.171.12.143 ca TELUS Communications Inc 4987 138.212.185.230 jp ASAHI KASEI CORPORATION 41250 6 0 1
1159 60 0x0400000000004001 1022171701.720657 0.000000 0.000000 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:fc:26:95:88 0x0800 193.87.5.62 sk Zdruzenie pouzivatelov Slovens 62486 138.212.188.178 jp ASAHI KASEI CORPORATION 2100 6 0 1
1167 447 0x0400000000004000 1022171701.721366 0.000000 0.000000 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:fc:3b:62:78 0x0800 217.41.129.13 gb BT Infrastructure Layer 58872 138.212.187.186 jp ASAHI KASEI CORPORATION 80 6 0 1
1497 523 0x0400000000004001 1022171701.729052 0.000000 0.000000 3 eth:ipv4:tcp 00:00:b4:a9:15:71 00:d0:02:6d:78:00 0x0800 138.212.185.150 jp ASAHI KASEI CORPORATION 1207 212.223.121.197 de Ratiokontakt GmbH 8000 6 0 1
1684 392 0x0400000000004001 1022171701.732313 0.000000 0.000000 3 eth:ipv4:tcp 00:50:bf:59:85:48 00:d0:02:6d:78:00 0x0800 138.212.188.67 jp ASAHI KASEI CORPORATION 1214 36.242.181.230 jp SoftBank Corp 4685 6 0 1
2004 176 0x0400000000004000 1022171701.739385 0.035669 0.035669 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:80:48:b3:13:27 0x0800 216.237.125.166 us Infortech Corporation 3507 138.212.184.193 jp ASAHI KASEI CORPORATION 8080 6 0 1
2232 633 0x0400000200004001 1022171701.743464 0.000000 0.000000 3 eth:ipv4:tcp 00:60:08:78:1b:63 00:d0:02:6d:78:00 0x0800 138.212.187.203 jp ASAHI KASEI CORPORATION 6699 19.123.222.7 us Ford Motor Company 1430 6 0 1
2295 642 0x0400000000004000 1022171701.744654 0.000000 0.000000 3 eth:ipv4:tcp 00:80:48:cd:8c:82 00:d0:02:6d:78:00 0x0800 138.212.186.160 jp ASAHI KASEI CORPORATION 1217 70.196.57.198 us Cellco Partnership DBA Verizon 9000 6 0 1
3975 176 0x0400000000004000 1022171701.779710 0.039558 0.075994 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:80:48:b3:13:27 0x0800 216.237.125.166 us Infortech Corporation 3507 138.212.184.193 jp ASAHI KASEI CORPORATION 8080 6 0 2
That was not so hard, right? . For researchers: Note that you already have the packet inter-distances per flowIndex and flow direction available. Why is there no L7 content? Have a look what T2 warns you about… the snap length! Use another pcap: faf-exercise.pcap and you will see L7 content. I extracted the FTP command/control flow 35 below.
$ t2 -r ~/data/faf-exercise.pcap -w ~/results -s
...
$ tawk 'flow(35)' faf-exercise_packets.txt | tcol
%pktNo flowInd flowStat time pktIAT flowDuration numHdrs hdrDesc ethVlanID srcMac dstMac ethType srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto winSize winThPktCnt l7Content
1266 35 0x0400000000004000 1258594162.928342 0.000000 0.000000 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 8192 0
1267 35 0x0400000000004001 1258594163.008594 0.000000 0.000000 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4140 0
1268 35 0x0400000000004000 1258594163.009292 0.080950 0.080950 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 64860 0
1269 35 0x0400000000004001 1258594163.087792 0.079198 0.079198 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4140 0 220 Microsoft FTP Service\r\n
1270 35 0x0400000000004000 1258594163.088491 0.079199 0.160149 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 64833 0 USER anonymous\r\n
1271 35 0x0400000000004001 1258594163.166256 0.078464 0.157662 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4156 0 331 Anonymous access allowed, send identity (e-mail name) as password.\r\n
1272 35 0x0400000000004000 1258594163.168693 0.080202 0.240351 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 64761 0 PASS IEUser@\r\n
1273 35 0x0400000000004001 1258594163.247178 0.080922 0.238584 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4170 0 230-Welcome to the Dell FTP site. A service of Dell Inc., Round Rock, Texas.\r\n For information about DELL, call +1 800 999 3355 All transfers are logged with\r\n your host name and email address. If you don't like this policy please disconnect now.\r\n Please be advised that use constitutes consent to monitoring (Elec Comm Priv Act,\r\n 18 USC 2701-2711). Please see the file readme.txt for disclaimers pertaining to this\r\n service. If your FTP client crashes or hangs shortly after login, try using a dash\r\n (-) as the first character of your password. This will turn off the informational\r\n messages which may be confusing your ftp client.\r\n ********IN CASE OF PROBLEMS*************************\r\n ** File Content: send EMAIL to dellbbs@dell.com **\r\n ** FTP Server: send EMAIL to hostmaster@dell.com **\r\n ** WWW Server: send EMAIL to webmaster@dell.com **\r\n ****************************************************\r\n
1274 35 0x0400000000004001 1258594163.247187 0.000009 0.238593 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4170 0 230 User logged in.\r\n
1275 35 0x0400000000004000 1258594163.247637 0.078944 0.319295 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63790 0
1276 35 0x0400000000004000 1258594163.249385 0.001748 0.321043 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63790 0 TYPE I\r\n
1277 35 0x0400000000004001 1258594163.327121 0.079934 0.318527 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4178 0 200 Type set to I.\r\n
1278 35 0x0400000000004000 1258594163.327845 0.078460 0.399503 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63770 0 PASV\r\n
1279 35 0x0400000000004001 1258594163.407582 0.080461 0.398988 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4184 0 227 Entering Passive Mode (143,166,11,10,251,78)\r\n
1283 35 0x0400000000004000 1258594163.487490 0.159645 0.559148 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63720 0 SIZE /video/R79733.EXE\r\n
1284 35 0x0400000000004001 1258594163.565990 0.158408 0.557396 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4208 0 213 4255056\r\n
1285 35 0x0400000000004000 1258594163.566694 0.079204 0.638352 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63707 0 RETR /video/R79733.EXE\r\n
1286 35 0x0400000000004001 1258594163.644188 0.078198 0.635594 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4232 0 125 Data connection already open; Transfer starting.\r\n
1303 35 0x0400000000004000 1258594163.838277 0.271583 0.909935 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63653 0
5898 35 0x0400000000004001 1258594185.427515 21.783327 22.418921 3 eth:ipv4:tcp 00:19:e3:e7:5d:23 00:08:74:38:01:b4 0x0800 143.166.11.10 us Dell 21 192.168.1.105 07 Private network 49329 6 4232 0 226 Transfer complete.\r\n
5900 35 0x0400000000004000 1258594185.618346 21.780069 22.690004 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 Private network 49329 143.166.11.10 us Dell 21 6 63629 0
Why are there no packets below the defined windows threshold? Can you find the flow which has some? I leave that to you. Or change the threshold in tcpWin.h.
Have fun with packet mode programming!
The next tutorial will teach you how to produce summary files
See Also
- Plugin Programming Cheatsheet
- The basics: your first flow plugin
- Adding plugin end report
- Adding plugin monitoring output
- Producing summary files
- geo-whois-labeling
- All about plugin dependencies
- Plugin sinks
- Manipulating flow timeouts
- Alarm mode
- Force mode
- Pcap extraction
- Developing Tranalyzer plugins in C++
- Developing Tranalyzer plugins in Rust