Tutorial: Force Mode

Description

This tutorial details the use of the Force mode of the Anteater, which enables the user to produce packet based plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow at any time. An easy way to simulate L2-7 content based flow release which immediately produces a flow when a certain packet of interest is detected. The following plugins implement the force mode:

  • basicStats (if 64 bit count registers overrun)
  • dnsDecode (if arrays which hold names overrun)
  • radiusDecode (when a access accept or reject is received)

Preparation

In order to do so, we need to prepare T2. If you did not complete the tutorials before, just follow the procedure described below.

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins and compile only basicFlow, basicStats, tcpStates and txtSink.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow basicStats tcpStates txtSink
...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another cmd window, that facilitates your workflow:

$ mkdir ~/data
$ mkdir ~/results
$ cd data

The anonymized sample PCAP being used here, can be downloaded here: faf-exercise.pcap. Please extract it under your data folder. Now you are all set for T2 force mode experiments.

Core Force Control

The Force mode controls of the Anteater are residing in the tranalyzer.h file:

To enable the force mode edit tranalyzer.h and set FORCE_MODE 1 as shown above or use t2conf as schown below, then rebuild all loaded plugins. Several plugins can trigger an alarm concurrently, as we do not know what kind of plugin armada you might develop in future.

$ t2conf tranalyzer2 -D FORCE_MODE=1
$ t2build -R

...
$

Now the force mode is activated in the core and all plugins which implement it.

Plugin Force Register and Control

Plugins which implement FORCE mode are basicStats, dnsDecode and radiusDecode. Let’s look at basicStats first.

$ basicStats
$ cd src
$ ls
basicStats.c  basicStats.h  Makefile.am
$

open basicStats.c in an editor, move to the bl_claimInfo function and search for the FORCE_MODE pragmas. If numTBytes or numTPkts are about to be overrun by the next packet the current flow is terminated and a new flow begins. The macro T2_RM_FLOW{flowP} does all that for you, so you could now add the force mode to other plugins to your liking in a heartbeat.

If you are interested to add the force mode to your own plugin please refer to the plugin force mode tutorial. Now let’s see how it works by changing the threshold of numTPkts down to 1023. Just copy the original line for easier change back.

compile and run t2:

$ t2build basicStats
...
$ t2 -r ~/data/faf-exercise.pcap -w ~/results
================================================================================
Tranalyzer 0.8.5 (Anteater), Tarantula. PID: 17644
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [FORCE]
Active plugins:
    01: basicFlow, 0.8.5
    02: basicStats, 0.8.5
    03: tcpStates, 0.8.5
    04: txtSink, 0.8.5
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312796 (312.80 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
Processing file: /home/wurst/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.004808 sec
Finished unloading flow memory. Time: 0.004835 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 3068 (3.07 K) [51.98%]
Number of B packets: 2834 (2.83 K) [48.02%]
Number of A bytes: 3076981 (3.08 M) [61.62%]
Number of B bytes: 1916433 (1.92 M) [38.38%]
Average A packet load: 1002.93
Average B packet load: 676.23
--------------------------------------------------------------------------------
basicStats: Biggest Talker: 143.166.11.10 (US): 1023 [17.33%] packets
basicStats: Biggest Talker: 143.166.11.10 (US): 1460615 (1.46 M) [29.25%] bytes
tcpStates: Aggregated anomaly flags: 0x4b
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 78
Number of processed A flows: 39 [50.00%]
Number of processed B flows: 39 [50.00%]
Number of request     flows: 39 [50.00%]
Number of reply       flows: 39 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 75.67
Number of processed A packets/flows: 78.67
Number of processed B packets/flows: 72.67
Number of processed total packets/s: 0.12
Number of processed A+B packets/s: 0.12
Number of processed A   packets/s: 0.06
Number of processed   B packets/s: 0.06
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.00
Average full raw bandwidth: 795 b/s
Average full bandwidth : 792 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.05 GB [0.08%]
Aggregate flow status: 0x0000020000004000
[WRN] Number of flows terminated by force mode: 3 [3.85%]
[INF] IPv4
$

So 3 flows are terminated early. The relased flows are flaged in the flowStat column by the RMFLOW bit, as define in global.h in the t2 core:

so let’s select all early terminated flows

$ tawk '{ if (bitsanyset($flowStat,0x0000020000000000)) print }' faf-exercise_flows.txt | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP          srcIPCC  srcIPWho  srcPort  dstIP          dstIPCC  dstIPWho           dstPort  l4Proto  numPktsSnt  numPktsRcvd  numBytesSnt  numBytesRcvd  minPktSz  maxPktSz  avePktSize  stdPktSize  minIAT  maxIAT    aveIAT       stdIAT      pktps     bytps     pktAsm     bytAsm  tcpStates
B     36       0x0000020000004001  1258594163.487027  1258594172.495175  9.008148  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        523          1405365      0             0         1380      1373.768    84.85606    0       0.67109   0.008805609  0.03729355  113.5639  156010.4  0.3234153  1       0x02
A     37       0x0000020000004000  1258594172.495183  1258594178.674264  6.179081  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        478          1410797      0             696       1380      1379.078    22.22461    0       0.124927  0.006040155  0.02058982  165.5586  228318.2  0.3630913  1       0x03
A     38       0x0000020000004000  1258594178.674299  1258594184.879209  6.204910  1           3        eth:ipv4:tcp  00:19:e3:e7:5d:23  00:08:74:38:01:b4  0x0800              143.166.11.10  us       "Dell"    64334    192.168.1.105  09       "Private network"  49330    6        1023        486          1410771      0             529       1380      1379.053    25.77088    0       0.079432  0.006065406  0.02059324  164.8694  227363.7  0.3558648  1       0x03
$

So numPktsSnt does not exceed 1023 as requested. Please, DO NOT forget to switch back to the original condition for the next tutorial, by removing our changes to basicStats.c and uncommenting the original line.

dnsDecode terminates flows when arrays which hold DNS names, so it acts on DNS_QRECMAX or DNS_ARECMAX:

// user config
#define DNS_MODE         4 // 0: Only aggregated header info,
                           // 1: +Req Content Info,
                           // 2: +Answer Records,
                           // 3: +AUX records,
                           // 4: +Add records
#define DNS_HEXON        0 // 0: Hex Output flags off, 1: Hex output flags on
#define DNS_REQA         0 // 1: Request record aggregation mode
#define DNS_ANSA         0 // 1: Answer record aggregation mode
#define DNS_QRECMAX     15 // Max # of query records / flow
#define DNS_ARECMAX     20 // Max # of answer records / flow

#define MAL_TEST         0 // activate Malware domain test
#define MAL_TYPE         1 // 1: Type string; 0: Code

...

As an exercise, release DNS flows if there is more than one reply record in the flow, DNS_ARECMAX 2. Unload basicStats to reduce complexity of the flow output and run T2 on this pcap 2015-05-08-traffic-analysis-exercise.pcap (Source: malware-traffic-analysis.net)

$ t2build -u basicStats
...
$ t2conf dnsDecode -D DNS_ARECMAX=2
$ t2build dnsDecode
...
$ t2 -r ~/data/t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results
================================================================================
Tranalyzer 0.8.5 (Anteater), Tarantula. PID: 19601
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [FORCE]
Active plugins:
    01: basicFlow, 0.8.5
    02: tcpStates, 0.8.5
    03: dnsDecode, 0.8.5
    04: txtSink, 0.8.5
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312796 (312.80 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
Processing file: /home/wurst/data/2015-05-08-traffic-analysis-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1431031896.723375 sec (Thu 07 May 2015 20:51:36 GMT)
Dump stop : 1431032021.842982 sec (Thu 07 May 2015 20:53:41 GMT)
Total dump duration: 125.119607 sec (2m 5s)
Finished processing. Elapsed time: 0.001292 sec
Finished unloading flow memory. Time: 0.001583 sec
Percentage completed: 100.00%
Number of processed packets: 761
Number of processed bytes: 495665 (495.67 K)
Number of raw bytes: 495665 (495.67 K)
Number of pcap bytes: 507865 (507.87 K)
Number of IPv4 packets: 761 [100.00%]
Number of A packets: 305 [40.08%]
Number of B packets: 456 [59.92%]
Number of A bytes: 34638 (34.64 K) [6.99%]
Number of B bytes: 461027 (461.03 K) [93.01%]
Average A packet load: 113.57
Average B packet load: 1011.02
--------------------------------------------------------------------------------
tcpStates: Aggregated anomaly flags: 0x42
dnsDecode: Number of DNS packets: 16 [2.10%]
dnsDecode: Number of DNS Q packets: 8 [50.00%]
dnsDecode: Number of DNS R packets: 8 [50.00%]
dnsDecode: Aggregated status: 0x0401
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 745 [97.90%]
Number of TCP bytes: 493885 (493.88 K) [99.64%]
Number of UDP packets: 16 [2.10%]
Number of UDP bytes: 1780 (1.78 K) [0.36%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 68
Number of processed A flows: 34 [50.00%]
Number of processed B flows: 34 [50.00%]
Number of request     flows: 34 [50.00%]
Number of reply       flows: 34 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 11.19
Number of processed A packets/flows: 8.97
Number of processed B packets/flows: 13.41
Number of processed total packets/s: 6.08
Number of processed A+B packets/s: 6.08
Number of processed A   packets/s: 2.44
Number of processed   B packets/s: 3.64
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.54
Average full raw bandwidth: 31692 b/s (31.69 Kb/s)
Average full bandwidth : 31574 b/s (31.57 Kb/s)
Max number of flows in memory: 40 [0.02%]
Memory usage: 0.05 GB [0.08%]
Aggregate flow status: 0x0000020000004000
[WRN] Number of flows terminated by force mode: 8 [11.76%]
[INF] IPv4
$

At the 4.th line of the report, T2 informs about the [FORCE] mode. At the end we learn that eight flows matched the force criterias and were release early. Select the as above these flows as above these flows

$ tawk '{ if (bitsanyset($flowStat,0x0000020000000000)) print }' 2015-05-08-traffic-analysis-exercise_flows.txt | tcol
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP          srcIPCC  srcIPWho           srcPort  dstIP            dstIPCC  dstIPWho           dstPort  l4Proto  tcpStates  dnsStat  dnsHdriOPField  dnsHStat_OpC_RetC   dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname                                                                         dnsAname                                                                            dnsAPname                                          dns4Aaddress             dns6Aaddress  dnsAType  dnsAClass  dnsATTL    dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat             dnsOptCodeOwn
B     1        0x0000020000004001  1431031896.874326  1431031896.874326  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  60078    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.168421  "va872g.g90e1h.b8.642b63u.j985a2.v33e.37.pa269cc.e8mfzdgrf7g0.groupprograms.in"  "va872g.g90e1h.b8.642b63u.j985a2.v33e.37.pa269cc.e8mfzdgrf7g0.groupprograms.in";""  "";""                                              62.75.195.236;0.0.0.0                  1;0       1;0        29;0       0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     3        0x0000020000004001  1431031897.655926  1431031897.655926  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  65315    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.170213  "ubb67.3c147o.u806a4.w07d919.o5f.f1.b80w.r0faf9.e8mfzdgrf7g0.groupprograms.in"   "ubb67.3c147o.u806a4.w07d919.o5f.f1.b80w.r0faf9.e8mfzdgrf7g0.groupprograms.in";""   "";""                                              62.75.195.236;0.0.0.0                  1;0       1;0        29;0       0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     4        0x0000020000004001  1431031897.669844  1431031897.669844  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  50683    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.172043  "r03afd2.c3008e.xc07r.b0f.a39.h7f0fa5eu.vb8fbl.e8mfzdgrf7g0.groupprograms.in"    "r03afd2.c3008e.xc07r.b0f.a39.h7f0fa5eu.vb8fbl.e8mfzdgrf7g0.groupprograms.in";""    "";""                                              62.75.195.236;0.0.0.0                  1;0       1;0        29;0       0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     13       0x0000020000004001  1431031902.778136  1431031902.778136  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  53571    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.571429  "ip-addr.es"                                                                     "ip-addr.es";""                                                                     "";""                                              188.165.164.184;0.0.0.0                1;0       1;0        21599;0    0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     17       0x0000020000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  61720    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"                                                                     "runlove.us";""                                                                     "";""                                              204.152.254.221;0.0.0.0                1;0       1;0        14069;0    0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     19       0x0000020000004001  1431031903.474197  1431031903.474197  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  50509    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_0_1_0               2.191489  "kritischerkonsum.uni-koeln.de"                                                  "uni-koeln.de";""                                                                   "noc2.rrz.uni-koeln.de";"hostmaster.uni-koeln.de"  0.0.0.0;0.0.0.0                        6;0       1;0        1799;3600  0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     20       0x0000020000004001  1431031903.507883  1431031903.507883  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  56753    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.444444  "comarksecurity.com"                                                             "comarksecurity.com";""                                                             "";""                                              72.34.49.86;0.0.0.0                    1;0       1;0        13888;0    0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
B     28       0x0000020000004001  1431031941.364104  1431031941.364104  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2  09       "Private network"  53       192.168.138.158  09       "Private network"  50329    17       0x00       0x0401   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.326531  "7oqnsnzwwnm6zb7y.gigapaysun.com"                                                "7oqnsnzwwnm6zb7y.gigapaysun.com";""                                                "";""                                              95.163.121.204;0.0.0.0                 1;0       1;0        14277;0    0;0        0;0         0;0        0;0        0x00000000;0x00000000  0;0
$

You may also change DNS_QREMAX, this might be useful if you are interested in only one query/answer per flow at once not at flow terminate.

Do not forget to reset the config to default for the other tutorials.

$ t2conf dnsDecode -D DNS_ARECMAX=20
$ t2conf tranalyzer2 -D FORCE_MODE=0
$ t2build -R
...
$ 

Have fun.