Alarm mode
Contents
Introduction
This tutorial details the use of the Alarm mode of the Anteater, which enables the user to produce plugin controlled flow release. Hence, any plugin can invoke an internal signal to release a specific flow, otherwise no flows are produced. An easy way to transform T2 into an elaborate NIDS. The following plugins implement currently the alarm mode:
| dnsDecode | if a DNS name is detected in a black list |
| regex_pcre | if a regex triggers |
Moreover, with the help of the pcapd plugin you can also extract all packets of a flow processed after the specific alarm.
Preparation
First, restore T2 into a pristine state by removing all unnecessary or older plugins from the plugin folder ~/.tranalyzer/plugins:
t2build -e -y
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
Then compile the following plugins:
t2build tranalyzer2 basicFlow tcpStates dnsDecode txtSink
...
BUILD SUCCESSFUL
If you did not create a separate data and results directory yet, please do it now in another bash window, that facilitates your workflow:
mkdir ~/data ~/results
The anonymized sample PCAP used in this tutorial can be downloaded here: 2015-05-08-traffic-analysis-exercise.pcap. (Source: malware-traffic-analysis.net). Please extract it under your data folder.
Now you are all set for T2 alarm experiments.
Core alarm control
The alarm mode controls of the Anteater are residing in tranalyzer2/src/tranalyzer.h:
tranalyzer2
vi src/tranalyzer.h
...
// Tranalyzer User Operational modes
// Operation modes, Plugins which use these modes have to be recompiled
#define ALARM_MODE 0 // only flow output if an Alarm based plugin fires
#define ALARM_AND 0 // if (ALARM_MODE == 1) then 1: AND, 0: OR
#define FORCE_MODE 0 // netflow mode: parameter induced flow termination, implemented by plugins
#define BLOCK_BUF 0 // block unnecessary buffer output when non tranalyzer format event-based plugins are active: e.g. Syslog, ArcSight crap
#define USE_T2BUS 0 // XXX experimental (not implemented yet)
...To enable the alarm mode switch ALARM_MODE to 1 as shown above. Several plugins can trigger an alarm
concurrently, as we do not know what kind of plugin armada you might develop in the future. So the ALARM_AND switch
denotes whether the flow release is triggered by a logical AND or OR. So either all alarm registered plugins
must agree that this is worth a flow or either one can release a flow. The default value is OR, so any plugin
can release a flow. Edit tranalyzer.h or use t2conf. Recompile t2 including all existing plugins:
t2conf tranalyzer2 -D ALARM_MODE=1 && t2build -R
Plugin alarm register and control
Let’s look at the dnsDecode plugin to illustrate the principle. It is capable to look for malware domains/IPs from a blacklist. This can be combined with the alarm mode, so that only the malware flows are released.
Move to the dnsDecode plugin folder and look into it:
dnsDecode
lsautogen.sh CMakeLists.txt configure.ac COPYING default.config doc Makefile.am maldomain.txt meson.build src t2plconf tests utils
The file maldomain.txt contains a list of malware domains, not the newest, so feel free to compile your own. There are
lots of lists available online. autogen.sh or t2build will convert that list into a T2 formatted list maldm.txt and copy
it under your ~/.tranalyzer/plugins directory. More about that later when we edit the maldomain.txt list.
Now move to the src directory
ls src
dnsDecode.c dnsDecode.h dnsType.h Makefile.am malsite.c malsite.h
malsite.c contains the search algorithm and malsite.h defines the type of search: domain or IP. Not important at the moment. Now open dsnDecode.h and enable the flow based malware test.
vi src/dnsDecode.h
...
/* ========================================================================== */
/* ------------------------ USER CONFIGURATION FLAGS ------------------------ */
/* ========================================================================== */
#define DNS_MODE 4 // 0: Only aggregated header info,
// 1: +Req Content Info,
// 2: +Answer Records,
// 3: +AUX records,
// 4: +Add records
#define DNS_HEXON 0 // 0: Hex output flags off, 1: Hex output flags on
#define DNS_HDRMD 0 // Header Flags_OpCode_RetCode: 0: Bitfield; 1: Integer; 2: String
#define DNS_TYPE 0 // Q/A Type: 0: Numeric; 1: String
#define DNS_AGGR 0 // 1: Record aggregation mode
#define DNS_QRECMAX 15 // Max # of query records / flow
#define DNS_ARECMAX 20 // Max # of answer records / flow
#define DNS_WHO 0 // 1: Country and organization of DNS reply address; 0: no
#define DNS_MAL_TEST 0 // 2: Mal test @ L4Callback, pcap ops; 1: Mal test @ flow terminated
#define DNS_MAL_TYPE 1 // 1: Type string; 0: Code
/* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */
/* No env / runtime configuration flags available for dnsDecode */
/* ========================================================================== */
/* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */
/* ========================================================================== */
...MAL_TEST=1 produces the malware test on flow terminated, so it is more performant, than
option 2 which does it at every packet.
MAL_TYPE is set to a human readable string output. A code number, the 2nd entry in the generated list maldm.txt under ~/.tranalyzer/plugins
shown below, might be better for AI training purposes.
So set MAL_TEST first.
t2conf dnsDecode -D MAL_TEST=1 && t2build dnsDecode
dnsDecode
lsx maldm.txt
... uivabretof.com 48 phishing rujva.co.uk 48 phishing rukn-aljamal.com 48 phishing rumahhufazh.or.id 48 phishing rumahmakannusantara.biz.id 48 phishing runawaydragons.com 48 phishing rundll.co.in 57 scam runngineszservices.co.uk 48 phishing runvtkk.tel.lv 48 phishing runzemaoye.com 42 malware rupanic.webescuela.cl 48 phishing rupeewiz.com 48 phishing rupertsigns.com 48 phishing rushrepublic.co.uk 48 phishing rusiapromo2018.ml 48 phishing rusmondf.com 48 phishing russelakic.com 60 suspicious russianfossils.com 48 phishing ...
If you want to add some more domains or download a new file store it under the name maldomain.txt in the dnsDecode directory. Because I do not have an appropriate pcap yet, add at the end of the maldomain.txt an entry indicated below:
vi maldomain.txt
...
bankhapoalim-online.com phishing
bestmLxer.in phishing
rnyuthewallet.com phishing
bankhapoailm.com phishing
stevenmyersphotography.com phishing
www.cambridge-solutions.online phishing
bankhapoailm-login.com phishing
btcbozdurma.ml phishing
rnuelherwallet.com phishing
techincpo.club phishing
kbcbankieren.com phishing
runlove.us trojan <-- add this here at the end of the file, mind the tab between the parameters, but do not add this comment
Note columns are tab separated, so don’t just copy the last line without replacing all blanks.
Invoke now the prepdl script in the dnsDecode/utils/ directory and copy the resulting file to ~/.tranalyzer/plugins/maldm.txt.
If you invoke t2build -f, a new list will be downloaded and your changes will be overwritten.
Then invoke t2 on the pcap:
utils/prepdl -c
converting malware domains list 'maldomain.txt' to 'maldm.txt'
t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/
T2 indicates the alarm mode in the third line of the end report:
[INF] Creating flows for L2, IPv4, IPv6 [ALARM]
A warning at the end reports the total number of alarms and alarm flows.
So two alarms in two flows total and all are coming from dnsDecode, because it
is the only plugin loaded. The corresponding flows can be inspected in the flow file via a simple
tawk statement. You see the detected runlove.us request and our type description in
the two flows being written to the flow file out of 68 flows total.
tcol ~/results/2015-05-08-traffic-analysis-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto tcpStatesAFlags dnsStat dnsHdrOPField dnsHFlg_OpC_RetC dnsCntQu_Asw_Aux_Add dnsAAAqF dnsQname dnsMalCnt dnsMalType dnsAname dnsAPname dns4Aaddress dns6Aaddress dnsQType dnsQClass dnsAType dnsAClass dnsATTL dnsMXpref dnsSRVprio dnsSRVwgt dnsSRVprt dnsOptStat
A 17 0x0400000000004000 1431031903.052371 1431031903.052371 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.158 07 "Private network" 61720 192.168.138.2 07 "Private network" 53 17 0x00 0x0001 0x0100 0x10_0x0001_0x0001 1_0_0_0 0 "runlove.us" 1 "trojan" 1 1
B 17 0x0400000000004001 1431031903.089942 1431031903.089942 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 61720 17 0x00 0x0001 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.571429 "runlove.us" 1 "trojan" "runlove.us" "" 204.152.254.221 1 1 1 1 14069 0 0 0 0 0x00000000Alarm controlled packet extraction
The pcapd plugin discussed in the pcap extraction tutorial is either controlled by t2 -e option,
aka an external flow index file, or if omitted by an internal flag FL_ALARM in the flow and global status.
If one or more plugins set that bit, pcapd will then extract all packets of that flow following the appearance
of the FL_ALARM. Unfortunately, we cannot store all packets before that alarm, so they do not appear in the
flow file. The packet mode has no alarm mode, so all packets will appear.
In order to inform pcapd to extract packets from flows, dnsDecode has to produce a signal at the packet level.
So switch MAL_TEST to 2 in dnsDecode.h and add the pcapd plugin:
t2conf dnsDecode -D MAL_TEST=2 && t2build dnsDecode pcapd
t2 -r ~/data/2015-05-08-traffic-analysis-exercise.pcap -w ~/results/
================================================================================ Tranalyzer 0.9.1 (Anteater), Tarantula. PID: 37008 ================================================================================ [INF] Creating flows for L2, IPv4, IPv6 Active plugins: 01: basicFlow, 0.9.1 02: tcpStates, 0.9.1 03: dnsDecode, 0.9.1 04: txtSink, 0.9.1 05: pcapd, 0.9.1 [INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K) [INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K) [INF] dnsDecode: 26771 blacklisted domains ... -------------------------------------------------------------------------------- tcpStates: Aggregated tcpStatesAFlags=0x42 dnsDecode: Aggregated dnsStat=0x0001 dnsDecode: Aggregated dnsHFlg=0x98, dnsOpC=0x0001, dnsRetC=0x0001 dnsDecode: Number of DNS packets: 16 [2.10%] dnsDecode: Number of DNS Q packets: 8 [50.00%] dnsDecode: Number of DNS R packets: 8 [50.00%] dnsDecode: 2 alarms in 2 flows [0.00%] pcapd: number of packets extracted: 2 [0.26%] -------------------------------------------------------------------------------- ... Aggregated flowStat=0x0402000000004000 [WRN] 2 alarms in 2 flows [2.94%] [INF] IPv4 flows [INF] IPAlarm
So we see that pcapd extracted 2 packets and created a pcap file named 2015-05-08-traffic-analysis-exercise_pcapd.pcap.
ls ~/results
015-05-08-traffic-analysis-exercise_flows.txt 2015-05-08-traffic-analysis-exercise_headers.txt 2015-05-08-traffic-analysis-exercise_pcapd.pcap
Note that in the flowStat, you see now the notification from dnsDecode to pcapd.
If you switched off ALARM_MODE, these flows could be isolated from the 68 flows by using this very bit:
tawk -V flowStat=0x0002000000000000
The flowStat column with value 0x0002000000000000 is to be interpreted as follows: bit | flowStat | Description ============================================================================= 49 | 0x0002 0000 0000 0000 | Alarm mode & pcapd dumps packets from this flow to new pcap if not -e option
tcol ~/results/2015-05-08-traffic-analysis-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto tcpStatesAFlags dnsStat dnsHdrOPField dnsHFlg_OpC_RetC dnsCntQu_Asw_Aux_Add dnsAAAqF dnsQname dnsMalCnt dnsMalType dnsAname dnsAPname dns4Aaddress dns6Aaddress dnsQType dnsQClass dnsAType dnsAClass dnsATTL dnsMXpref dnsSRVprio dnsSRVwgt dnsSRVprt dnsOptStat
A 17 0x0402000000004000 1431031903.052371 1431031903.052371 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.158 07 "Private network" 61720 192.168.138.2 07 "Private network" 53 17 0x00 0x0001 0x0100 0x10_0x0001_0x0001 1_0_0_0 0 "runlove.us" 1 "trojan" 1 1
B 17 0x0402000000004001 1431031903.089942 1431031903.089942 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 61720 17 0x00 0x0001 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.571429 "runlove.us" 1 "trojan" "runlove.us" "" 204.152.254.221 1 1 1 1 14069 0 0 0 0 0x00000000So unload pcapd and run t2 with the new pcap and store it in another file, otherwise the original flow file will be overwritten.
t2build -u pcapd
t2 -r ~/results/2015-05-08-traffic-analysis-exercise_pcapd.pcap -w ~/results/nudel
================================================================================ Tranalyzer 0.9.1 (Anteater), Tarantula. PID: 20263 ================================================================================ [INF] Creating flows for L2, IPv4, IPv6 [ALARM] Active plugins: 01: basicFlow, 0.9.1 02: tcpStates, 0.9.1 03: dnsDecode, 0.9.1 04: txtSink, 0.9.1 [INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406105 (406.11 K) [INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51345 (51.34 K) [INF] dnsDecode: 26771 blacklisted domains ... -------------------------------------------------------------------------------- dnsDecode: Aggregated dnsStat=0x0001 dnsDecode: Aggregated dnsHFlg=0x98, dnsOpC=0x0001, dnsRetC=0x0001 dnsDecode: Number of DNS packets: 2 [100.00%] dnsDecode: Number of DNS Q packets: 1 [50.00%] dnsDecode: Number of DNS R packets: 1 [50.00%] dnsDecode: 2 alarms in 2 flows [100.00%] -------------------------------------------------------------------------------- Headers count: min: 3, max: 3, average: 3.00 Number of UDP packets: 2 [100.00%] Number of UDP bytes: 156 [100.00%] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of processed flows: 2 Number of processed A flows: 1 [50.00%] Number of processed B flows: 1 [50.00%] Number of request flows: 1 [50.00%] Number of reply flows: 1 [50.00%] ... Aggregated flowStat=0x0402000000004000 [WRN] 2 alarms in 2 flows [100.00%] [INF] IPv4 flows [INF] IPAlarm
If you compare the nudel flow file with the original flow file above, they are the same. Note, that is due to fact that in the first packet the alarm was produced.
tcol ~/results/nudel_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto tcpStatesAFlags dnsStat dnsHdrOPField dnsHFlg_OpC_RetC dnsCntQu_Asw_Aux_Add dnsAAAqF dnsQname dnsMalCnt dnsMalType dnsAname dnsAPname dns4Aaddress dns6Aaddress dnsQType dnsQClass dnsAType dnsAClass dnsATTL dnsMXpref dnsSRVprio dnsSRVwgt dnsSRVprt dnsOptStat
A 1 0x0402000000004000 1431031903.052371 1431031903.052371 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.158 07 "Private network" 61720 192.168.138.2 07 "Private network" 53 17 0x00 0x0001 0x0100 0x10_0x0001_0x0001 1_0_0_0 0 "runlove.us" 1 "trojan" 1 1
B 1 0x0402000000004001 1431031903.089942 1431031903.089942 0.000000 1 3 eth:ipv4:udp 00:00:00:00:00:00 00:00:00:00:00:00 0x0800 192.168.138.2 07 "Private network" 53 192.168.138.158 07 "Private network" 61720 17 0x00 0x0001 0x8180 0x98_0x0001_0x0001 1_1_0_0 1.571429 "runlove.us" 1 "trojan" "runlove.us" "" 204.152.254.221 1 1 1 1 14069 0 0 0 0 0x00000000Alarm your own plugin
If you intent to integrate the ALARM mode in your own plugin, refer to the Plugin alarm mode tutorial.
Conclusion
Do not forget to reset tranalyzer2 and dnsDecode for your next tutorial, so that your output always matches the one on the webpage:
t2conf tranalyzer2 -D ALARM_MODE=0
t2conf dnsDecode -D MAL_TEST=0
t2build -R
or use the new command
t2conf --reset -a && t2build -R
Have fun!