Alarm mode

alarm ALARM_MODE DNS pcre regex

Introduction

This tutorial details the use of the Alarm mode of the Anteater, which enables the user to produce plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow, otherwise no flows are produced. An easy way to transform T2 into an elaborate NIDS. The following plugins implement currently the alarm mode:

dnsDecode if a DNS name is detected in a black list
regex_pcre if a regex triggers

Moreover, with the help of the pcapd plugin you can also extract all packets of a flow processed after the specific alarm.

Preparation

First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:

t2build -e -y

Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied

Then compile the following plugins:

t2build tranalyzer2 basicFlow tcpStates dnsDecode txtSink

...
BUILD SUCCESSFUL

If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:

mkdir ~/data ~/results

The anonymized sample PCAP used in this tutorial can be downloaded here: 2015-05-08-traffic-analysis-exercise.pcap. (Source: malware-traffic-analysis.net). Please extract it under your data folder.

Now you are all set for T2 alarm experiments.

Core alarm control

The alarm mode controls of the Anteater are residing in tranalyzer2/src/tranalyzer.h:

tranalyzer2

vi src/tranalyzer.h

...
// Tranalyzer User Operational modes

// Operation modes, Plugins which use these modes have to be recompiled

#define ALARM_MODE 0 // only flow output if an Alarm based plugin fires
#define ALARM_AND  0 // if (ALARM_MODE == 1) then 1: AND, 0: OR

#define FORCE_MODE 0 // netflow mode: parameter induced flow termination, implemented by plugins
#define BLOCK_BUF  0 // block unnecessary buffer output when non tranalyzer format event-based plugins are active: e.g. Syslog, ArcSight crap

#define USE_T2BUS  0 // XXX experimental (not implemented yet)
...

To enable the alarm mode switch ALARM_MODE to 1 as shown above. Several plugins can trigger an alarm concurrently, as we do not know what kind of plugin armada you might develop in the future. So the ALARM_AND switch denotes whether the flow release is triggered by a logical AND or OR. So either all alarm registered plugins must agree that this is worth a flow or either one can release a flow. The default value is OR, so any plugin can release a flow. Edit tranalyzer.h or use t2conf. Recompile t2 including all existing plugins:

t2conf tranalyzer2 -D ALARM_MODE=1 && t2build -R

Plugin alarm register and control

Let’s look at the dnsDecode plugin to illustrate the principle. It is capable to look for malware domains/IPs from a blacklist. This can be combined with the alarm mode, so that only the malware flows are released.

Move to the dnsDecode plugin folder and look into it:

dnsDecode

ls

autogen.sh  CMakeLists.txt  configure.ac  COPYING  default.config  doc  Makefile.am  maldomain.txt  meson.build  src  t2plconf  tests  utils

The file maldomain.txt contains a list of malware domains, not the newest, so feel free to compile your own. There are lots of lists available online. autogen.sh or t2build will convert that list into a T2 formatted list maldm.txt and copy it under your ~/.tranalyzer/plugins directory. More about that later when we edit the maldomain.txt list.

Now move to the src directory

ls src

dnsDecode.c  dnsDecode.h  dnsType.h  Makefile.am  malsite.c  malsite.h

malsite.c contains the search algorithm and malsite.h defines the type of search: domain or IP. Not important at the moment. Now open dsnDecode.h and enable the flow based malware test.

vi src/dnsDecode.h

...
/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */

#define DNS_MODE         4 // 0: Only aggregated header info,
                           // 1: +Req Content Info,
                           // 2: +Answer Records,
                           // 3: +AUX records,
                           // 4: +Add records

#define DNS_HEXON        0 // 0: Hex output flags off, 1: Hex output flags on
#define DNS_HDRMD        0 // Header Flags_OpCode_RetCode: 0: Bitfield; 1: Integer; 2: String
#define DNS_TYPE         0 // Q/A Type: 0: Numeric; 1: String
#define DNS_AGGR         0 // 1: Record aggregation mode

#define DNS_QRECMAX     15 // Max # of query records / flow
#define DNS_ARECMAX     20 // Max # of answer records / flow

#define DNS_WHO          0 // 1: Country and organization of DNS reply address; 0: no

#define DNS_MAL_TEST     0 // 2: Mal test @ L4Callback, pcap ops; 1: Mal test @ flow terminated
#define DNS_MAL_TYPE     1 // 1: Type string; 0: Code

/* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */

/*        No env / runtime configuration flags available for dnsDecode        */

/* ========================================================================== */
/* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */
/* ========================================================================== */
...

MAL_TEST=1 produces the malware test on flow terminated, so it is more performant, than option 2 which does it at every packet.

MAL_TYPE is set to a human readable string output. A code number, the 2nd entry in the generated list maldm.txt under ~/.tranalyzer/plugins shown below, might be better for AI training purposes. So set MAL_TEST first.

t2conf dnsDecode -D MAL_TEST=1 && t2build dnsDecode

dnsDecode

lsx maldm.txt


...
uivabretof.com                          48                                      phishing
rujva.co.uk                             48                                      phishing
rukn-aljamal.com                        48                                      phishing
rumahhufazh.or.id                       48                                      phishing
rumahmakannusantara.biz.id              48                                      phishing
runawaydragons.com                      48                                      phishing
rundll.co.in                            57                                      scam
runngineszservices.co.uk                48                                      phishing
runvtkk.tel.lv                          48                                      phishing
runzemaoye.com                          42                                      malware
rupanic.webescuela.cl                   48                                      phishing
rupeewiz.com                            48                                      phishing
rupertsigns.com                         48                                      phishing
rushrepublic.co.uk                      48                                      phishing
rusiapromo2018.ml                       48                                      phishing
rusmondf.com                            48                                      phishing
russelakic.com                          60                                      suspicious
russianfossils.com                      48                                      phishing
...

If you want to add some more domains or download a new file store it under the name maldomain.txt in the dnsDecode directory. Because I do not have an appropriate pcap yet, add at the end of the maldomain.txt an entry indicated below:

vi maldomain.txt


...
bankhapoalim-online.com phishing
bestmLxer.in    phishing
rnyuthewallet.com       phishing
bankhapoailm.com        phishing
stevenmyersphotography.com      phishing
www.cambridge-solutions.online  phishing
bankhapoailm-login.com  phishing
btcbozdurma.ml  phishing
rnuelherwallet.com      phishing
techincpo.club  phishing
kbcbankieren.com        phishing
runlove.us      trojan      		<-- add this here at the end of the file, mind the tab between the parameters, but do not add this comment

Note columns are tab separated, so don’t just copy the last line without replacing all blanks. Invoke now the prepdl script in the dnsDecode/utils/ directory and copy the resulting file to ~/.tranalyzer/plugins/maldm.txt. If you invoke t2build -f, a new list will be downloaded and your changes will be overwritten.

Then invoke t2 on the pcap:

utils/prepdl -c

converting malware domains list 'maldomain.txt' to 'maldm.txt'

t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/

T2 indicates the alarm mode in the third line of the end report:


[INF] Creating flows for L2, IPv4, IPv6 [ALARM]

A warning at the end reports the total number of alarms and alarm flows. So two alarms in two flows total and all are coming from dnsDecode, because it is the only plugin loaded. The corresponding flows can be inspected in the flow file via a simple tawk statement. You see the detected runlove.us request and our type description in the two flows being written to the flow file out of 68 flows total.

tcol ~/results/2015-05-08-traffic-analysis-exercise_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP            srcIPCC  srcIPOrg           srcPort  dstIP            dstIPCC  dstIPOrg           dstPort  l4Proto  tcpStatesAFlags  dnsStat  dnsHdrOPField  dnsHFlg_OpC_RetC    dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType  dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsQType  dnsQClass  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat
A     17       0x0400000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.158  07       "Private network"  61720    192.168.138.2    07       "Private network"  53       17       0x00             0x0001   0x0100         0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "trojan"                                                            1         1
B     17       0x0400000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.2    07       "Private network"  53       192.168.138.158  07       "Private network"  61720    17       0x00             0x0001   0x8180         0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "trojan"    "runlove.us"  ""         204.152.254.221                1         1          1         1          14069    0          0           0          0          0x00000000

Alarm controlled packet extraction

The pcapd plugin discussed in the pcap extraction tutorial is either controlled by t2 -e option, aka an external flow index file, or if omitted by an internal flag FL_ALARM in the flow and global status. If one or more plugins set that bit, pcapd will then extract all packets of that flow following the appearance of the FL_ALARM. Unfortunately, we cannot store all packets before that alarm, so they do not appear in the flow file. The packet mode has no alarm mode, so all packets will appear.

In order to inform pcapd to extract packets from flows, dnsDecode has to produce a signal at the packet level. So switch MAL_TEST to 2 in dnsDecode.h and add the pcapd plugin:

t2conf dnsDecode -D MAL_TEST=2 && t2build dnsDecode pcapd

t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/


================================================================================
Tranalyzer 0.9.1 (Anteater), Tarantula. PID: 37008
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
    01: basicFlow, 0.9.1
    02: tcpStates, 0.9.1
    03: dnsDecode, 0.9.1
    04: txtSink, 0.9.1
    05: pcapd, 0.9.1
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K)
[INF] dnsDecode: 26771 blacklisted domains
...
--------------------------------------------------------------------------------
tcpStates: Aggregated tcpStatesAFlags=0x42
dnsDecode: Aggregated dnsStat=0x0001
dnsDecode: Aggregated dnsHFlg=0x98, dnsOpC=0x0001, dnsRetC=0x0001
dnsDecode: Number of DNS packets: 16 [2.10%]
dnsDecode: Number of DNS Q packets: 8 [50.00%]
dnsDecode: Number of DNS R packets: 8 [50.00%]
dnsDecode: 2 alarms in 2 flows [0.00%]
pcapd: number of packets extracted: 2 [0.26%]
--------------------------------------------------------------------------------
...
Aggregated flowStat=0x0402000000004000
[WRN] 2 alarms in 2 flows [2.94%]
[INF] IPv4 flows
[INF] IPAlarm

So we see that pcapd extracted 2 packets and created a pcap file named 2015-05-08-traffic-analysis-exercise_pcapd.pcap.

ls ~/results

015-05-08-traffic-analysis-exercise_flows.txt  2015-05-08-traffic-analysis-exercise_headers.txt  2015-05-08-traffic-analysis-exercise_pcapd.pcap

Note that in the flowStat, you see now the notification from dnsDecode to pcapd. If you switched off ALARM_MODE, these flows could be isolated from the 68 flows by using this very bit:

tawk -V flowStat=0x0002000000000000


The flowStat column with value 0x0002000000000000 is to be interpreted as follows:

   bit | flowStat              | Description
   =============================================================================
    49 | 0x0002 0000 0000 0000 | Alarm mode & pcapd dumps packets from this flow to new pcap if not -e option

tcol ~/results/2015-05-08-traffic-analysis-exercise_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP            srcIPCC  srcIPOrg           srcPort  dstIP            dstIPCC  dstIPOrg           dstPort  l4Proto  tcpStatesAFlags  dnsStat  dnsHdrOPField  dnsHFlg_OpC_RetC    dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType  dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsQType  dnsQClass  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat
A     17       0x0402000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.158  07       "Private network"  61720    192.168.138.2    07       "Private network"  53       17       0x00             0x0001   0x0100         0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "trojan"                                                            1         1
B     17       0x0402000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.2    07       "Private network"  53       192.168.138.158  07       "Private network"  61720    17       0x00             0x0001   0x8180         0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "trojan"    "runlove.us"  ""         204.152.254.221                1         1          1         1          14069    0          0           0          0          0x00000000

So unload pcapd and run t2 with the new pcap and store it in another file, otherwise the original flow file will be overwritten.

t2build -u pcapd

t2 -r ~/results/2015-05-08-traffic-analysis-exercise_pcapd.pcap -w ~/results/nudel


================================================================================
Tranalyzer 0.9.1 (Anteater), Tarantula. PID: 20263
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [ALARM]
Active plugins:
    01: basicFlow, 0.9.1
    02: tcpStates, 0.9.1
    03: dnsDecode, 0.9.1
    04: txtSink, 0.9.1
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K)
[INF] dnsDecode: 26771 blacklisted domains
...
--------------------------------------------------------------------------------
dnsDecode: Aggregated dnsStat=0x0001
dnsDecode: Aggregated dnsHFlg=0x98, dnsOpC=0x0001, dnsRetC=0x0001
dnsDecode: Number of DNS packets: 2 [100.00%]
dnsDecode: Number of DNS Q packets: 1 [50.00%]
dnsDecode: Number of DNS R packets: 1 [50.00%]
dnsDecode: 2 alarms in 2 flows [100.00%]
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of UDP packets: 2 [100.00%]
Number of UDP bytes: 156 [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed   flows: 2
Number of processed A flows: 1 [50.00%]
Number of processed B flows: 1 [50.00%]
Number of request     flows: 1 [50.00%]
Number of reply       flows: 1 [50.00%]
...
Aggregated flowStat=0x0402000000004000
[WRN] 2 alarms in 2 flows [100.00%]
[INF] IPv4 flows
[INF] IPAlarm

If you compare the nudel flow file with the original flow file above, they are the same. Note, that is due to fact that in the first packet the alarm was produced.

tcol ~/results/nudel_flows.txt

%dir  flowInd  flowStat            timeFirst          timeLast           duration  numHdrDesc  numHdrs  hdrDesc       srcMac             dstMac             ethType  vlanID  srcIP            srcIPCC  srcIPOrg           srcPort  dstIP            dstIPCC  dstIPOrg           dstPort  l4Proto  tcpStatesAFlags  dnsStat  dnsHdrOPField  dnsHFlg_OpC_RetC    dnsCntQu_Asw_Aux_Add  dnsAAAqF  dnsQname      dnsMalCnt  dnsMalType  dnsAname      dnsAPname  dns4Aaddress     dns6Aaddress  dnsQType  dnsQClass  dnsAType  dnsAClass  dnsATTL  dnsMXpref  dnsSRVprio  dnsSRVwgt  dnsSRVprt  dnsOptStat
A     1        0x0402000000004000  1431031903.052371  1431031903.052371  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.158  07       "Private network"  61720    192.168.138.2    07       "Private network"  53       17       0x00             0x0001   0x0100         0x10_0x0001_0x0001  1_0_0_0               0         "runlove.us"  1          "trojan"                                                            1         1
B     1        0x0402000000004001  1431031903.089942  1431031903.089942  0.000000  1           3        eth:ipv4:udp  00:00:00:00:00:00  00:00:00:00:00:00  0x0800           192.168.138.2    07       "Private network"  53       192.168.138.158  07       "Private network"  61720    17       0x00             0x0001   0x8180         0x98_0x0001_0x0001  1_1_0_0               1.571429  "runlove.us"  1          "trojan"    "runlove.us"  ""         204.152.254.221                1         1          1         1          14069    0          0           0          0          0x00000000

Alarm your own plugin

If you intent to integrate the ALARM mode in your own plugin, refer to the Plugin alarm mode tutorial.

Conclusion

Do not forget to reset tranalyzer2 and dnsDecode for your next tutorial, so that your output always matches the one on the webpage:

t2conf tranalyzer2 -D ALARM_MODE=0

t2conf dnsDecode -D MAL_TEST=0

t2build -R

or use the new command

t2conf --reset -a && t2build -R

Have fun!