Tutorial: Plugin Pcap Extraction

As detailed in the pcap extraction tutorial, T2 is capable to extract flows into pcaps using the pcapd plugin. The regex_pcre is an example for this operation regex_pcre flow extraction. In order to implement this feature in your own plugin, specialized macros are supplied, which will be detailed in this tutorial. You do not need to enable the ALARM_MODE it also works in the normal flow mode.

Reset your plugin directory into a pristine state and compile the following basic plugins:

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow tcpStates txtSink pcapd
...
$

If you did not read the tutorials before, here is the basic plugin which we will extend: tcpWin Extract it and copy it under your plugins directory if you have not already. You may also use any of your versions developed during earlier tutorials.

The anonymized sample pcap can be downloaded here: faf-exercise.pcap. Please extract it under your data folder: ~/data, if you haven’t already. Now you are all set for the alarm mode.

Implementing the FL_ALARM capability

Open tcpWin.c in an editor, move to the claimLayer4Information() callback and add the T2_SET_STATUS() line marked by <--.

Now pcapd will extract all packets of a flow direction where the FL_ALARM is set.

If you want to have the global FL_ALARM status including flow and alarm count shown as a warning in the end report, then add the T2_REPORT_ALARMS() macro in the onFlowTerminate() callback as marked by <--.

Save the file, recompile and run t2 on the pcap.

$ t2build tcpWin
...
$ t2 -r ~/data/faf-exercise.pcap -w ~/results/
================================================================================
Tranalyzer 0.8.11 (Anteater), Tarantula. PID: 5402
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.11
    02: tcpStates, 0.8.11
    03: tcpWin, 0.8.11
    04: txtSink, 0.8.11
    05: pcapd, 0.8.11
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406208 (406.21 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51196 (51.20 K)
Processing file: /home/wurst/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.006089 sec
Finished unloading flow memory. Time: 0.006109 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 1986 (1.99 K) [33.65%]
Number of B packets: 3916 (3.92 K) [66.35%]
Number of A bytes: 209315 (209.31 K) [4.19%]
Number of B bytes: 4784099 (4.78 M) [95.81%]
Average A packet load: 105.40
Average B packet load: 1221.68 (1.22 K)
--------------------------------------------------------------------------------
tcpStates: Aggregated tcpStatesAFlags=0x4a
pcapd: number of packets extracted: 1467 (1.47 K) [24.86%]
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 72
Number of processed A flows: 36 [50.00%]
Number of processed B flows: 36 [50.00%]
Number of request     flows: 36 [50.00%]
Number of reply       flows: 36 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 81.97
Number of processed A packets/flows: 55.17
Number of processed B packets/flows: 108.78
Number of processed total packets/s: 0.12
Number of processed A+B   packets/s: 0.12
Number of processed A     packets/s: 0.04
Number of processed   B   packets/s: 0.08
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.00
Average full raw bandwidth: 795 b/s
Average full bandwidth : 792 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.01 GB [0.02%]
Aggregated flowStat=0x0402000000004000
[WRN] 4 alarms in 1 flows [1.39%]
[INF] IPv4 flows
[INF] IPAlarm
$

The plugin report states that 1467 packets were extracted of one flow. Note the [WRN] which is 0x0402000000004000 describes the global aggregate flow status, which T2_SET_STATUS also sets. Try a tawk -V to decode the flow status:

$ tawk -V flowStat=0x0402000000004000

The flowStat column with value 0x0402000000004000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    14 | 0x0000 0000 0000 4000 | IPv4 flow
    49 | 0x0002 0000 0000 0000 | Alarm mode & pcapd dumps packets from this flow to new pcap if not -e option
    58 | 0x0400 0000 0000 0000 | IPv4 packet

$

See?!

Now move to the results directory and look into it.

$ cd ~/results
$ ls
faf-exercise_flows.txt  faf-exercise_headers.txt  faf-exercise_pcapd.pcap
$

Note the file faf-exercise_pcapd.pcap, which represents the extracted packets belonging to the flow where the FL_ALARM bit is set. To verify, select these flows using tawk.

$ tawk 'bitsanyset($flowStat, 0x0002000000000000)' faf-exercise_flows.txt | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration   numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP          srcIPCC  srcIPOrg           srcPort  dstIP          dstIPCC  dstIPOrg  dstPort  l4Proto  tcpStates  tcpWinStat  tcpWinThCnt
A     36       0x0402000000004000  1258594163.408285  1258594191.015208  27.606923  1           3        eth:ipv4:tcp  00:08:74:38:01:b4  00:19:e3:e7:5d:23  0x0800              192.168.1.105  07       "Private network"  49330    143.166.11.10  us       "Dell"    64334    6        0x42       0x01        4
$

Only one flow? Why? Look at the config of pcapd:

$ t2conf -G PD_OPP pcapd
PD_OPP = 0
$

So set it to 1 and see whether you get the other flow, as homework. If you are uneasy working with pcapd refer to the tutorial pcapextraction

If you are just interested in pcap extraction, then you do not need to load txtSink. It is advisable to use the -w option, otherwise t2 stores the pcap in the data folder, and you start desperately looking for your extracted pcap. It happened to people ;-)

Have fun extracting pcaps!

The next tutorial will teach you how to develop a plugin in C++.

See Also