Plugin alarm mode
Introduction
As detailed in the alarm mode tutorial, T2 is capable to operate in a plugin controlled flow release. The regex_pcre and dnsDecode plugins are examples. In order to implement this feature in your own plugin, specialized macros are supplied, which will be detailed in this tutorial.
Getting started
Create folders for your data and results
If you have not created a separate data and results directory yet, please do it now. This will greatly facilitate your workflow:
mkdir ~/data ~/results
Reset tranalyzer2 and the plugins configuration
If you have followed the other tutorials, you may have modified some of the core and plugins configuration. To ensure your results match those in this tutorial, make sure to reset everything:
t2conf -a --reset
You can also clean all build files:
t2build -a -c
Empty the plugin folder
To ensure we are not left with some unneeded plugins or plugins which were built using different core configuration, it is safer to empty the plugins folder:
t2build -e -y
Are you sure you want to empty the plugin folder '/home/user/.tranalyzer/plugins' (y/N)? yes
Plugin folder emptied
Download the PCAP file
The PCAP file used in this tutorial can be downloaded here:
Please save it in your ~/data folder:
wget --no-check-certificate -P ~/data https://tranalyzer.com/download/data/faf-exercise.pcap
Configure Tranalyzer2 core
For this tutorial, we need to activate the alarm mode. The required flag reside in tranalyzer.h:
tranalyzer2/src/tranalyzer.h
ALARM_MODE=1(alarm mode)
Change the configuration of Tranalyzer2 using the following t2conf command:
t2conf tranalyzer2 -D ALARM_MODE=1
Build tranalyzer2 and the required plugins
For this tutorial, we will need to build the core (tranalyzer2) and the following plugins:
As you may have modified some of the automatically generated files, it is safer to use the -r and -f options.
...
BUILDING SUCCESSFUL
Source code
In this tutorial, we will extend tcpWin09.tar.gz, the final version of the previous tutorial (Plugin Dependencies).
If you are impatient, you can download the intermediate and final versions of the tcpWin plugin we will develop in this tutorial.
To use one of those plugins, just unpack it in the plugins folder of your T2 installation.
tranpl
tar -xf ~/Downloads/tcpWin11.tar.gz
And let t2_aliases know about it:
source "$T2HOME/scripts/t2_aliases"
Implementing the alarm mode capability
If your plugin wants to contribute information to the alarm mode,
then global variables and a T2_REPORT_ALARMS(numAlarms) macro
have to be added.
So open tcpWin.c in an editor and add two global variables after the tcpWinFlows definition. Look for the // <-- markers below.
Add at the beginning in the global plugin block add a static definition of an alarm count variable for the end report:
tcpWin
vi src/tcpWin.c
...
/*
* Static variables are only visible in this file
*/
// window size counts
static gwz_t gwz; // global window size structure
static uint32_t winAlarms; // <-- For end report: Number of alarms
static uint32_t winThFlows; // <-- For end report: Number of flows which created an alarm
static uint32_t winThCntG, winThCntG0; // Aggregated win threshold count and variable for the last threshold count
static uint8_t tcpWinStat; // Aggregated status
...Then add in the t2OnFlowTerminate(...) callback the lines marked by // <-- which
triggers an alarm if the threshold count is > 0.
You can replace the variable in T2_REPORT_ALARMS(...) by any other logical statement or variable.
vi src/tcpWin.c
...
void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf) {
...
flow_t * const flowP = &flows[flowIndex]; // <-- Remove the const keyword
if (!FLOW_IS_IPV4(flowP) || !PROTO_IS_TCP(flowP)) return; // IPv4 and TCP only
if (tcpWinFlowP->winThCnt == 0 || bSFlowP->numTPkts < TCPWIN_MINPKTS) return;
winAlarms += tcpWinFlowP->winThCnt; // <-- global alarm count
winThFlows++; // <-- count number of alarms flows
T2_REPORT_ALARMS(tcpWinFlowP->winThCnt); // <-- report alarm to the core
T2_SET_STATUS(flowP, FL_ALARM); // <-- Set the alarm bit in flow and global
const int wzi = gwz.wzi; // store element count in const local variable, makes the compiler happy
...
}
...In order to make the alarm mode work, you only need the macro T2_REPORT_ALARMS(...).
It implements the following state-machine:
utils
vi t2Plugin.h
...
#if ALARM_MODE == 1
extern unsigned char supOut; // suppress output
#define T2_REPORT_ALARMS(num) { \
numAlarmFlows++; \
numAlarms += (num); \
if (!ALARM_AND) { \
if (num) supOut = 0; \
} else { \
if (!(num)) { \
supOut = 1; \
return; \
} \
} \
}
#else // ALARM_MODE == 0
#define T2_REPORT_ALARMS(num) { \
numAlarmFlows++; \
numAlarms += (num); \
}
#endif // ALARM_MODE == 0
...That complicated code is kept from you now, just use the macro and don’t worry.
If you are indeed interested in its functionality, here is the explanation:
The variable supOut is a global switch suppressing the output.
numAlarms denotes the aggregate of all flow alarms.
numAlarmFlows denotes the number of flows which bear an alarm.
If ALARM_MODE is not active the latter global counts will still be reported.
As we count the number of alarm flows in the macro, you may at the present version only use it ONCE
in t2OnFlowTerminate(...) XOR onFlowGenerate(), but NOT during packet callbacks.
This will be fixed in a later 0.9.X version.
So your plugin can reimplement any state-machine you want and control the output suppression to your liking.
The easiest would be to set suppress output supOut=1, then no flow would ever appear in the flow file, rather useless, right.
If you set ALARM_AND=1 in tranalyzer.h then any plugin can reset a flow terminate,
so all plugins implementing ALARM_MODE must agree.
For reporting edit the t2PluginReport(...) callback to report the number of alarms and alarm flows.
tcpWin
vi src/tcpWin.c
...
void t2PluginReport(FILE *stream) {
if (winThCntG) {
T2_FPLOG_AGGR_HEX(stream, plugin_name, tcpWinStat);
if (winThFlows) {
T2_FPWRN_NP(stream, plugin_name,
"%" PRIu32" alarms in %" PRIu32 " flows below win threshold " STR(TCPWIN_THRES),
winAlarms, winThFlows);
}
T2_FPLOG_NUMP(stream, plugin_name,
"Number of TCP winsize packets below threshold " STR(TCPWIN_THRES),
winThCntG, numPacketsL3[L3_TCP]);
...
}
}
...Now save your new tcpWin.c.
After you edited the skeleton code you should compare your implementation with tcpWin10.tar.gz.
Compile tcpWin and execute T2:
t2build tcpWin
t2 -r ~/data/faf-exercise.pcap -w ~/results/================================================================================ Tranalyzer 0.9.0 (Anteater), Cobra. PID: 44541, SID: 666 ================================================================================ [INF] Creating flows for L2, IPv4, IPv6 [ALARM] Active plugins: 01: basicFlow, 0.9.0 02: basicStats, 0.9.0 03: tcpStates, 0.9.0 04: tcpWin, 0.9.0 05: txtSink, 0.9.0 [INF] IPv4 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 481503 (481.50 K) [INF] IPv6 Ver: 5, Rev: 09082023, Range Mode: 0, subnet ranges loaded: 41497 (41.50 K) Processing file: /home/wurst/data/faf-exercise.pcap Link layer type: Ethernet [EN10MB/1] Snapshot length: 65535 Dump start: 1258544215.37210000 sec (Wed 18 Nov 2009 11:36:55 GMT) ... -------------------------------------------------------------------------------- basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 3101 (3.10 K) [52.54%] packets basicStats: Biggest L3 flow talker: 143.166.11.10 (US): 4268858 (4.27 M) [85.49%] bytes tcpStates: Aggregated tcpStatesAFlags=0x4a tcpWin: Aggregated tcpWinStat=0x01 tcpWin: 4 alarms in 1 flows below win threshold 1 tcpWin: Number of TCP winsize packets below threshold 1: 4 [0.07%] tcpWin: IP: 192.168.1.105, country: 07, org: Private network -------------------------------------------------------------------------------- ... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Number of average processed flows/s: 0.00 Average full raw bandwidth: 795 b/s Average full bandwidth : 792 b/s Max number of flows in memory: 18 [0.01%] Memory usage: 0.01 GB [0.02%] Aggregated flowStat=0x0402000000004000 [WRN] 4 alarms in 1 flows [1.39%] [INF] IPv4 flows [INF] IPAlarm
Note the info in the plugin block and the warning at the end of the T2 report.
Decoding the aggregated flow status, we see that you indeed set the FL_ALARM
bit in the global status.
The flowStat column with value 0x0402000000004000 is to be interpreted as follows: bit | flowStat | Description ============================================================================= 14 | 0x0000 0000 0000 4000 | IPv4 flow 49 | 0x0002 0000 0000 0000 | Alarm mode & pcapd dumps packets from this flow to new pcap if not -e option 58 | 0x0400 0000 0000 0000 | IPv4 packet
~/results/faf-exercise_flows.txt shows that out of 72 flows only one was released producing four alarms. Don’t worry about pcapd info under description, the alarm bit is used also for pcap extraction, we come to that in a later tutorial.
As you set the FL_ALARM in t2OnFlowTerminate(...) and the flowStat is printed by
basicFlow
which is executed before tcpWin, we do not see the bit in the flow.
tcol ~/results/faf-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStatesAFlags tcpWinStat tcpWinIpTTL tcpInitWinSz tcpWinThCnt tcpWinSzThRt
A 36 0x0400000000004000 1258594163.408285000 1258594191.015208000 27.606923000 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell Technologies" 64334 6 1514 3101 0 4268858 0 0 0 0 0 5.58724 0.01823444 0.1478493 54.84132 0 -0.3438787 -1 0x42 0x01 128 8192 4 0.002642008If you want to see the alarm bit set in the flowStat, you have to set it in the packet callback.
Just move the T2_SET_STATUS(...) statement in t2OnFlowTerminate(...) and add it in t2OnLayer4(...)
below where the // <-- is present.
vi src/tcpWin.c
...
void t2OnLayer4(packet_t *packet, unsigned long flowIndex) {
flow_t * const flowP = &flows[flowIndex]; // <-- Remove the const keyword
if (flowP->l4Proto != L3_TCP) { // process only TCP
TCPWIN_SPKTMD_PRI_NONE(); // Packet mode
return; // go back to core
}
// only 1. frag packet will be processed
if (!t2_is_first_fragment(packet)) {
TCPWIN_SPKTMD_PRI_NONE(); // Packet mode
return; // go back to core
}
tcpWinFlow_t * const tcpWinFlowP = &tcpWinFlows[flowIndex];
const tcpHeader_t * const tcpHeader = TCP_HEADER(packet);
const uint32_t tcpWin = ntohs(tcpHeader->window);
if (tcpWin < TCPWIN_THRES) { // is the window size below the threshold?
tcpWinFlowP->winThCnt++; // count the packet / flow
winThCntG++; // increment global packet counter
tcpWinFlowP->stat |= TCPWIN_STAT_THU; // set the status bit
tcpWinStat |= tcpWinFlowP->stat; // Aggregate all packet flags
T2_SET_STATUS(flowP, FL_ALARM); // <-- Set the alarm bit in flow and global
}
// Packet mode
if (sPktFile) {
fprintf(sPktFile,
"%" PRIu32 /* tcpWinSize */ SEP_CHR
"%" PRIu32 /* tcpWinThPktCnt */ SEP_CHR
, tcpWin, tcpWinFlowP->winThCnt);
}
}
...Save the file.
Recompile, rerun t2 and reopen ~/results/faf-exercise_flows.txt:
t2build tcpWin
t2 -r ~/data/faf-exercise.pcap -w ~/results/
tcol ~/results/faf-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType vlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStatesAFlags tcpWinStat tcpWinIpTTL tcpInitWinSz tcpWinThCnt tcpWinSzThRt
A 36 0x0402000000004000 1258594163.408285000 1258594191.015208000 27.606923000 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell Technologies" 64334 6 1514 3101 0 4268858 0 0 0 0 0 5.58724 0.01823444 0.1478493 54.84132 0 -0.3438787 -1 0x42 0x01 128 8192 4 0.002642008See now it is set.
If you want to separate the functionality of your alarm mode from other modes such as
force mode, then you have to surround your code with ALARM_MODE pragmas as shown below:
vi src/tcpWin.c
...
#if ALARM_MODE == 1 // <--
static uint32_t winAlarms; // Number of alarms
static uint32_t winThFlows; // Number of flows which created an alarm
#endif // ALARM_MODE == 1 // <--
static uint32_t winThCntG, winThCntG0; // Aggregated win threshold count and variable for the last threshold count
static uint8_t tcpWinStat; // Aggregated status
...
void t2OnLayer4(packet_t *packet, unsigned long flowIndex) {
...
#if ALARM_MODE == 1 // <--
T2_SET_STATUS(flowP, FL_ALARM); // Set the alarm bit in flow and global
#endif // ALARM_MODE == 1 // <--
...
}
...
void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf) {
...
#if ALARM_MODE == 1 // <--
winAlarms += tcpWinFlowP->winThCnt; // global alarm count
winThFlows++; // count number of alarm flows
T2_REPORT_ALARMS(tcpWinFlowP->winThCnt); // report alarm to the core
#endif // ALARM_MODE == 1 // <--
...
}
...
void t2PluginReport(FILE *stream) {
if (winThCnt) {
T2_FPLOG_AGGR_HEX(stream, plugin_name, tcpWinStat);
#if ALARM_MODE == 1 // <--
if (winThFlows) {
T2_FPWRN_NP(stream, plugin_name,
"%" PRIu32" alarms in %" PRIu32 " flows below win threshold " STR(TCPWIN_THRES),
winAlarms, winThFlows);
}
#endif // ALARM_MODE == 1 // <--
...
}
}
...Compile your plugin again and see whether something changed. What will happen if you switch off the ALARM bit in the core,
with and without the ALARM_MODE pragma? Try it out. But don’t forget to compile all plugins with t2build -R as the
ALARM mode is also implemented in your and other plugins, if loaded.
Conclusion
Play a bit around and improve the program, e.g. modify the alarm condition if more than one win threshold occurrences happen.
And do not forget to reset the alarm mode for the next tutorial
t2conf tranalyzer2 -D ALARM_MODE=0 && t2build -R -r -f
Have fun writing alarm mode plugins!
You can download the final version of the tcpWin plugin.
The next tutorial will teach you how to implement the force mode.
See also
- Plugin programming cheatsheet
- The basics: your first flow plugin
- Plugin end report
- Plugin monitoring
- Plugin packet mode
- Plugin summary files
- Plugin geo labeling
- Plugin dependencies
- Plugin force mode
- Plugin pcap extraction
- Plugin flow timeout
- Plugin sink
- Developing Tranalyzer plugins in C++
- Developing Tranalyzer plugins in Rust