Tutorial: Force Mode

Description

This tutorial details the use of the Force mode of the Anteater, which enables the user to produce packet based plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow at any time. An easy way to simulate L2-7 content based flow release which immediately produces a flow when a certain packet of interest is detected. The following plugins implement the force mode:

  • basicStats (if 64 bit count registers overrun)
  • dnsDecode (if arrays which hold names overrun)
  • radiusDecode (when a access accept or reject is received)

Preparation

In order to do so, we need to prepare T2. If you did not complete the tutorials before, just follow the procedure described below.

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins and compile only basicFlow, basicStats, tcpStates and txtSink.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build basicFlow basicStats tcpStates txtSink
...
BUILD SUCCESSFUL

The -e option for t2build also leads to the subnet file for geolocation being regenerated when building basicFlow the next time, which can take a bit of time.

An alternative would be to just remove the .so files, and preserve the old subnet files:

$ rm ~/.tranalyzer/plugins/*.so
$ t2build basicFlow basicStats tcpStates regex_pcre txtSink

Then the compilation will be a bit faster, as the subnet file already exists. If you did not create a separate data and results directory yet, please do it now in another cmd window, that facilitates your workflow:

$ mkdir ~/data
$ mkdir ~/results
$ cd data

The anonymized sample PCAP being used here, can be downloaded here: faf-exercise.pcap. Please extract it under your data folder. Now you are all set for T2 force mode experiments.

Core Force Control

The Force mode controls of the Anteater are residing in the tranalyzer.h file:

To enable the force mode switch from FORCE_MODE 1 as shown above. Several plugins can trigger an alarm concurrently, as we do not know what kind of plugin armada you might develop in future.

Now the force mode is activated in the core and all plugins which implement it.

Plugin Force Register and Control

Plugins which implement FORCE mode are basicStats, dnsDecode and radiusDecode. Let’s look at basicStats.

$ cd basicStats/src
$ ls
basicStats.c  basicStats.h  Makefile.am
$

open basicStats.c in an editor, move to the bl_claimInfo function and search for the FORCE_MODE pragmas. If numTBytes or numTPkts are about to be overrun by the next packet the current flow is terminated and a new flow begins. The macro T2_RM_FLOW{flowP} does all that for you, so you could now add the force mode to other plugins to your liking in a heartbeat.

If you are interested to add the force mode to your own plugin please refer to the plugin force mode tutorial. Now let’s see how it works by changing the threshold of numTPkts down to 1023. Just copy the original line for easier change back.

compile and run t2:

$ t2build basicStats
...
$ t2 -r ~/data/faf-exercise.pcap -w ~/results
================================================================================
Tranalyzer 0.8.4 (Anteater), Tarantula. PID: 16448
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.8.4
    02: basicStats, 0.8.4
    03: tcpStates, 0.8.4
    04: txtSink, 0.8.4
[INF] basicFlow: IPv4 Ver: 3, Rev: 1062019, Range Mode: 0, subnet ranges loaded: 222252 (222.25 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 1062019, Range Mode: 0, subnet ranges loaded: 26541 (26.54 K)
Processing file: /home/stefan/tranalyzer-website/tranalyzer/download/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.004803 sec
Finished unloading flow memory. Time: 0.004829 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 3068 (3.07 K) [51.98%]
Number of B packets: 2834 (2.83 K) [48.02%]
Number of A bytes: 3076981 (3.08 M) [61.62%]
Number of B bytes: 1916433 (1.92 M) [38.38%]
Average A packet load: 1002.93
Average B packet load: 676.23
--------------------------------------------------------------------------------
basicStats: Biggest Talker: 143.166.11.10 (US): 1023 [17.33%] packets
basicStats: Biggest Talker: 143.166.11.10 (US): 1410797 (1.41 M) [28.25%] bytes
tcpStates: Aggregated anomaly flags: 0x4b
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 78
Number of processed A flows: 39 [50.00%]
Number of processed B flows: 39 [50.00%]
Number of request     flows: 39 [50.00%]
Number of reply       flows: 39 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 75.67
Number of processed A packets/flows: 78.67
Number of processed B packets/flows: 72.67
Number of processed total packets/s: 0.12
Number of processed A+B packets/s: 0.12
Number of processed A   packets/s: 0.06
Number of processed   B packets/s: 0.06
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.00
Average full raw bandwidth: 795 b/s
Average full bandwidth : 792 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.04 GB [0.06%]
Aggregate flow status: 0x0000000000004000
[WRN] Number of flows terminated by force mode: 3 [3.85%]
[INF] IPv4

So 3 flows are terminated early. The relased flows are flaged in the flowStat column by the RMFLOW bit, as define in global.h in the t2 core:

so lets select all early terminated flows

$ tawk '{ if (bitsanyset($flowStat,0x0000020000000000)) print }' faf-exercise_flows.txt
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP          srcIPCC  srcIPWho  srcPort  dstIP          dstIPCC  dstIPWho           dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT       stdIAT      pktps     bytps     pktAsm     bytAsm  tcpStates
B     36       0x0000020000004001  1258594163.487027  1258594172.495175  9.008148  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        523          1405365      0             0         1380      1373.768    84.85606    0       0.67109   0.008805609  0.03729355  113.5639  156010.4  0.3234153  1       0x02
A     37       0x0000020000004000  1258594172.495183  1258594178.674264  6.179081  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        478          1410797      0             696       1380      1379.078    22.22461    0       0.124927  0.006040155  0.02058982  165.5586  228318.2  0.3630913  1       0x03
A     38       0x0000020000004000  1258594178.674299  1258594184.879209  6.204910  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        486          1410771      0             529       1380      1379.053    25.77088    0       0.079432  0.006065406  0.02059324  164.8694  227363.7  0.3558648  1       0x03
$

So numPktsSnt does not exceed 1023 as requested. Please, don’t forget to switch back to the original condition for the next tutorial.

dnsDecode terminates flows when arrays which hold DNS names, so it acts on DNS_QRECMAX or DNS_ARECMAX:

// user config
#define DNS_MODE         4 // 0: Only aggregated header info,
                           // 1: +Req Content Info,
                           // 2: +Answer Records,
                           // 3: +AUX records,
                           // 4: +Add records
#define DNS_HEXON        0 // 0: Hex Output flags off, 1: Hex output flags on
#define DNS_REQA         0 // 1: Request record aggregation mode
#define DNS_ANSA         0 // 1: Answer record aggregation mode
#define DNS_QRECMAX     15 // Max # of query records / flow
#define DNS_ARECMAX     20 // Max # of answer records / flow

#define MAL_TEST         0 // activate Malware domain test
#define MAL_TYPE         0 // 1: Type string; 0: Code

...

As an exercise, release DNS flows if there is more than one reply record in the flow, DNS_ARECMAX 2. Unload basicStats to reduce complexity of the flow output and run T2 on this pcap 2015-05-08-traffic-analysis-exercise.pcap (Source: malware-traffic-analysis.net)

$ t2build dnsDecode
...
$ t2build -u basicStats
...
$ t2 -r ~/data/t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results
...
--------------------------------------------------------------------------------
tcpStates: Aggregated anomaly flags: 0x42
dnsDecode: Number of DNS packets: 16 [2.10%]
dnsDecode: Number of DNS Q packets: 8 [50.00%]
dnsDecode: Number of DNS R packets: 8 [50.00%]
dnsDecode: Aggregated status: 0x0401
--------------------------------------------------------------------------------
...
Max number of flows in memory: 40 [0.02%]
Memory usage: 0.04 GB [0.06%]
Aggregate flow status: 0x0000000000004000
[WRN] Number of flows terminated by force mode: 8 [11.76%]
[INF] IPv4

Select the flows in question under your results directory and try to comprehend the outcome. You may also change DNS_QREMAX, this might be useful if you are interested in only one query/answer per flow at once not at flow terminate. Have fun.