Tutorial: Alarm Mode

Description

This tutorial details the use of the Alarm mode of the Anteater, which enables the user to produce plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow, otherwise no flows are produced. An easy way to transform T2 into an elaborate NIDS. The following plugins implement currently the alarm mode:

dnsDecode if a DNS name is detected in a black list
regex_pcre if a regex triggers

Moreover, with the help of the pcapd plugin you can also extract all packets of a flow processed after the specific alarm.

Preparation

In order to do so, we need to prepare T2. If you did not complete the tutorials before, just follow the procedure described below.

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins and compile only basicFlow, basicStats, tcpStates, dnsDecode and txtSink.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow tcpStates dnsDecode txtSink
...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:

$ mkdir ~/data
$ mkdir ~/results
$ cd data

The anonymized sample PCAP being used can be downloaded here: 2015-05-08-traffic-analysis-exercise.pcap. (Source: malware-traffic-analysis.net). Please extract it under your data folder. Now you are all set for T2 alarm experiments.

Core alarm control

The alarm mode controls of the Anteater are residing in tranalyzer2/src/tranalyzer.h:

To enable the alarm mode switch ALARM_MODE to 1 as shown above. Several plugins can trigger an alarm concurrently, as we do not know what kind of plugin armada you might develop in the future. So the ALARM_AND switch denotes whether the flow release is triggered by a logical AND or OR. So either all alarm registered plugins must agree that this is worth a flow or either one can release a flow. The default value is OR, so any plugin can release a flow. Edit tranalyzer.h or use t2conf. Recompile t2 including all existing plugins:

Plugin alarm register and control

Let’s look at the dnsDecode plugin to illustrate the principle. It is capable to look for malware domains/IPs from a blacklist. This can be combined with the alarm mode, so that only the malware flows are released.

Move to the dnsDecode plugin and look into it:

$ dnsDecode
$ ls
AUTHORS  autogen.sh  ChangeLog  configure.ac  COPYING  doc  Makefile.am   maldomain.txt  maltype.txt  NEWS  README  src  t2plconf  tests  utils

The file maldomain.txt contains a list of malware domains, not the newest, so feel free to compile your own. There are lots of lists available online. autogen.sh or t2build will convert that list into a T2 formatted list maldm.txt and copy it under your ~/.tranalyzer/plugins directory. More about that later when we edit the maldomain.txt list.

Now move to the src directory

$ cd src
$ ls
dnsDecode.c  dnsDecode.h  dnsType.h  Makefile.am  malsite.c  malsite.h
$

malsite.c contains the search algorithm and malsite.h defines the type of search: domain or IP. Not important at the moment. Now open dsnDecode.h and enable the flow based malware test.

MAL_TEST set to 1 produces the malware test on flow terminated, so it is more performant, than option 2 which does it at every packet.

MAL_TYPE is set to a human readable string output. A code number, the 2nd entry in the generated list maldm.txt under ~/.tranalyzer/plugins shown below, might be better for AI training purposes.

...
ruivabretof.com 173     phishing
rujva.co.uk     173     phishing
rukn-aljamal.com        173     phishing
rumahhufazh.or.id       173     phishing
rumahmakannusantara.biz.id      173     phishing
runawaydragons.com      173     phishing
rundll.co.in    202     scam
runlove.us      164     nudel
runngineszservices.co.uk        173     phishing
runvtkk.tel.lv  173     phishing
runzemaoye.com  145     malware
rupanic.webescuela.cl   173     phishing
rupeewiz.com    173     phishing
rupertsigns.com 173     phishing
...

If you want to add some more domains or download a new file store it under the name maldomain.txt in the dnsDecode directory.

$ tail maldomain.txt
...
posb-dbs-server.com     phishing
smileswab.joomla.com    phishing
posb-dbs-bank.com       phishing
samwebstudio.co.in      phishing
zariya.hu       phishing
zonasegurasbeta.com     phishing
flashobmen.ru   phishing
owmobmen.ru     phishing
paxful.co.in    phishing
repairshoppr.com        phishing
runlove.us      wursteimer			<-- add this here at the end of the file

I added here the runlove.us entry with a categorization. I picked a ludicrous one. In any case, you can download your own domain list. Just rename it to maldomain.txt and recompile with t2build -f.

Alternatively, invoke the dmt script in the dnsDecode/utils/ directory and copy the resulting file to ~/.tranalyzer/plugins/maldm.txt. Note however that, if you invoke t2build -f, a new list will be downloaded and your changes will be overwritten.

$ dnsDecode
$ t2conf dnsDecode -D MAL_TEST=1
$ t2build dnsDecode
...
$ utils/dmt maldomain.txt
$ cp maldm.txt ~/.tranalyzer/plugins
$ t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/
================================================================================
Tranalyzer 0.8.6 (Anteater), Tarantula. PID: 3531
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [ALARM]
Active plugins:
    01: basicFlow, 0.8.6
    02: tcpStates, 0.8.6
    03: dnsDecode, 0.8.6
    04: txtSink, 0.8.6
[INF] basicFlow: IPv4 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 312796 (312.80 K)
[INF] basicFlow: IPv6 Ver: 3, Rev: 01072019, Range Mode: 0, subnet ranges loaded: 21494 (21.49 K)
[INF] dnsDecode: 26826 blacklisted domains
Processing file: /home/wurst/data/2015-05-08-traffic-analysis-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1431031896.723375 sec (Thu 07 May 2015 20:51:36 GMT)
Dump stop : 1431032021.842982 sec (Thu 07 May 2015 20:53:41 GMT)
Total dump duration: 125.119607 sec (2m 5s)
Finished processing. Elapsed time: 0.000813 sec
Finished unloading flow memory. Time: 0.000903 sec
Percentage completed: 100.00%
Number of processed packets: 761
Number of processed bytes: 495665 (495.67 K)
Number of raw bytes: 495665 (495.67 K)
Number of pcap bytes: 507865 (507.87 K)
Number of IPv4 packets: 761 [100.00%]
Number of A packets: 305 [40.08%]
Number of B packets: 456 [59.92%]
Number of A bytes: 34638 (34.64 K) [6.99%]
Number of B bytes: 461027 (461.03 K) [93.01%]
Average A packet load: 113.57
Average B packet load: 1011.02
--------------------------------------------------------------------------------
tcpStates: Aggregated anomaly flags: 0x42
dnsDecode: Number of DNS packets: 16 [2.10%]
dnsDecode: Number of DNS Q packets: 8 [50.00%]
dnsDecode: Number of DNS R packets: 8 [50.00%]
dnsDecode: Aggregated status: 0x0001
dnsDecode: Number of flows 2 with 2 alarms
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 745 [97.90%]
Number of TCP bytes: 493885 (493.88 K) [99.64%]
Number of UDP packets: 16 [2.10%]
Number of UDP bytes: 1780 (1.78 K) [0.36%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 68
Number of processed A flows: 34 [50.00%]
Number of processed B flows: 34 [50.00%]
Number of request     flows: 34 [50.00%]
Number of reply       flows: 34 [50.00%]
Total   A/B    flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed   packets/flows: 11.19
Number of processed A packets/flows: 8.97
Number of processed B packets/flows: 13.41
Number of processed total packets/s: 6.08
Number of processed A+B packets/s: 6.08
Number of processed A   packets/s: 2.44
Number of processed   B packets/s: 3.64
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.54
Average full raw bandwidth: 31692 b/s (31.69 Kb/s)
Average full bandwidth : 31574 b/s (31.57 Kb/s)
Max number of flows in memory: 56 [0.02%]
Memory usage: 0.06 GB [0.08%]
Aggregate flow status: 0x0002000000004000
[WRN] 2 alarms in 2 [2.94%] flows
[INF] IPv4
$

T2 indicates the alarm mode in the third line of the endreport:

[INF] Creating flows for L2, IPv4, IPv6 [ALARM]

A warning at the end reports the total number of alarms and alarm flows. So two alarms in two flows total and all are comming from dnsDecode, because it is the only plugin loaded. The corresponding flows can be inspected in the flow file via a simple tawk statement. You see the detected runlove.us request and our type description in the two flows being written to the flow file out of 68 flows total.

$ tawk '$dnsMalCnt' 2015-05-08-traffic-analysis-exercise_flows.txt
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPWho           srcPort  dstIP            dstIPCC  dstIPWho           dstPort  l4Proto  tcpStates  dnsStat  dnsHdriOPField  dnsHStat_OpC_RetC   dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType    dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat  dnsOptCodeOwn
A     17       0x0000000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.158  09       "Private network"  61720    192.168.138.2    09       "Private network"  53       17       0x00       0x0001   0x0100          0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "wursteimer"                                                          1         1          0        0          0           0          0          0x00000000  0
B     17       0x0000000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2    09       "Private network"  53       192.168.138.158  09       "Private network"  61720    17       0x00       0x0001   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "wursteimer"  "runlove.us"  ""         204.152.254.221                1         1          14069    0          0           0          0          0x00000000  0
$

Alarm controlled packet extraction

The pcapd plugin discussed in the pcap extraction tutorial is either controlled by t2 -e option, aka an external flow index file, or if omitted by an internal flag FL_ALARM in the flow and global status. If one or more plugins set that bit, pcapd will then extract all packets of that flow following the appearance of the FL_ALARM. Unfortunately, we cannot store all packets before that alarm, so they do not appear in the flow file. The packet mode has no alarm mode, so all packets will appear.

In order to inform pcapd to extract packets from flows, dnsDecode has to produce a signal at the packet level. So switch MAL_TEST to 2 in dnsDecode.h and add thepcapd plugin:

$ t2conf dnsDecode -D MAL_TEST=2
$ t2build dnsDecode pcapd
...
$ t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/
================================================================================
Tranalyzer 0.8.6 (Anteater), Tarantula. PID: 5568
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [ALARM]
Active plugins:
    01: basicFlow, 0.8.6
    02: tcpStates, 0.8.6
    03: dnsDecode, 0.8.6
    04: txtSink, 0.8.6
    05: pcapd, 0.8.6
...
--------------------------------------------------------------------------------
tcpStates: Aggregated anomaly flags: 0x42
dnsDecode: Number of DNS packets: 16 [2.10%]
dnsDecode: Number of DNS Q packets: 8 [50.00%]
dnsDecode: Number of DNS R packets: 8 [50.00%]
dnsDecode: Aggregated status: 0x0001
dnsDecode: Number of flows 2 with 2 alarms
pcapd: number of packets extracted: 2 [0.26%]
--------------------------------------------------------------------------------
...
$

So we see that pcapd extracted 2 packets and created a pcap file named 2015-05-08-traffic-analysis-exercise_pcapd.pcap.

$ ls ~/results
015-05-08-traffic-analysis-exercise_flows.txt  2015-05-08-traffic-analysis-exercise_headers.txt  2015-05-08-traffic-analysis-exercise_pcapd.pcap
$

Note that in the flowStat, you see now the notification from dnsDecode to pcapd. If you switched off ALARM_MODE, these flows could be isolated from the 68 flows by using this very bit:

tawk -V flowStat=0x0002000000000000

The flowStat column with value 0x0002000000004000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    49 | 0x0002 0000 0000 0000 | pcapd dumps packets from this flow to new pcap if not -e option
$ tcol 2015-05-08-traffic-analysis-exercise_flows.txt
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPWho           srcPort  dstIP            dstIPCC  dstIPWho           dstPort  l4Proto  tcpStates  dnsStat  dnsHdriOPField  dnsHStat_OpC_RetC   dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType    dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat  dnsOptCodeOwn
A     17       0x0002000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.158  09       "Private network"  61720    192.168.138.2    09       "Private network"  53       17       0x00       0x0001   0x0100          0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "wursteimer"                                                          1         1          0        0          0           0          0          0x00000000  0
B     17       0x0002000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2    09       "Private network"  53       192.168.138.158  09       "Private network"  61720    17       0x00       0x0001   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "wursteimer"  "runlove.us"  ""         204.152.254.221                1         1          14069    0          0           0          0          0x00000000  0
$

So unload pcapd and run t2 with the new pcap and store it in another file, otherwise the original flow file will be overwritten.

$ t2build -u pcapd
...
$ t2 -r ~/results/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/nudel
===============================================================================
Tranalyzer 0.8.6 (Anteater), Tarantula. PID: 5568
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [ALARM]
Active plugins:
    01: basicFlow, 0.8.6
    02: tcpStates, 0.8.6
    03: dnsDecode, 0.8.6
    04: txtSink, 0.8.6
...
$

If you compare the nudel file with the original flow file above, they are the same. Note, that is due to fact that in the first packet the alarm was produced.

$ tcol nudel_flows.txt
%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  ethVlanID  srcIP            srcIPCC  srcIPWho           srcPort  dstIP            dstIPCC  dstIPWho           dstPort  l4Proto  tcpStates  dnsStat  dnsHdriOPField  dnsHStat_OpC_RetC   dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType    dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat  dnsOptCodeOwn
A     17       0x0002000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.158  09       "Private network"  61720    192.168.138.2    09       "Private network"  53       17       0x00       0x0001   0x0100          0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "wursteimer"                                                          1         1          0        0          0           0          0          0x00000000  0
B     17       0x0002000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800              192.168.138.2    09       "Private network"  53       192.168.138.158  09       "Private network"  61720    17       0x00       0x0001   0x8180          0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "wursteimer"  "runlove.us"  ""         204.152.254.221                1         1          14069    0          0           0          0          0x00000000  0
$

Alarm your own plugin

If you intent to integrate the ALARM mode in your own plugin, refer to the Plugin Alarm Mode tutorial. And do not forget to reset t2 and dnsDecode for your next tutorial, so that your output always matches the one on the webpage:

$ t2conf tranalyzer2 -D ALARM_MODE=0
$ t2conf dnsDecode -D MAL_TEST=0
$ t2build -R
...
$

Have fun!