Tutorial: Plugin Force Mode
As detailed in force mode tutorial T2 is capable to operate in a plugin controlled flow release. For example, the basicStats plugin can force the termination of a flow. In order to implement this feature in your own plugin, specialized macros are supplied, which will be detailed in this tutorial. First, switch on the FORCE_MODE
in tranalyzer.h.
$ tranalyzer2
$ vi tranalyzer.h
...
// Tranalyzer User Operational modes
// Operation modes, Plugins which use these modes have to be recompiled
...
#define FORCE_MODE 1 // netflow mode: parameter induced flow termination, implemented by plugins
...
Or use the following t2conf
command:
$ t2conf tranalyzer2 -D FORCE_MODE=1
$
Then reset your plugin directory into a pristine state and compile the following basic plugins:
$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow tcpStates txtSink
...
$
If you did not read the tutorials before, here is the basis plugin which we will extend: tcpWin Also here I want you to work a bit and write some code yourself, or at least copy it from the tutorial below.
The anonymized sample pcap can be downloaded here: faf-exercise.pcap. Please extract it under your data folder: ~/data
, if you haven’t already. Now you are all set for the force mode implementation.
Implementing the force mode capability
If you want to implement the force mode, then the specific code related to the force mode, but be surrounded by #if FORCE_MODE == 1
and #endif // FORCE_MODE == 1
instructions and the T2_RM_FLOW(flowP)
macro must be used.
So open tcpWin.c in an editor and add two global variables after the tcpWinFlows definition. Look for the <--
markers below.
Then add in the claimLayer4Information(...)
callback the code surrounded by the FORCE_MODE
pragmas.
void claimLayer4Information(packet_t *packet, unsigned long flowIndex) {
flow_t * const flowP = &flows[flowIndex];
if (flowP->layer4Protocol != L3_TCP) return; // process only TCP
// only 1. frag packet will be processed
if (!t2_is_first_fragment(packet)) return;
tcpWinFlow_t * const tcpWinFlowP = &tcpWinFlows[flowIndex];
const tcpHeader_t * const tcpHeader = (tcpHeader_t*)packet->layer4Header;
const uint32_t tcpWin = ntohs(tcpHeader->window);
if (tcpWin < TCPWIN_THRES) { // is windowsize below threshold?
tcpWinFlowP->winThCnt++; // count the packet
tcpWinFlowP->stat |= TCPWIN_THU; // set the status bit
}
// force mode implementation starts here
#if FORCE_MODE == 1 // <--
if (tcpWinFlowP->winThCnt > 1) T2_RM_FLOW(flowP); // <--
#endif // FORCE_MODE == 1 // <--
}
Just use the T2_RM_FLOW(...)
macro and your flow will be terminated if your condition applies and all core and flow status bits will be set automatically. So save the file, compile tcpWin and execute T2.
$ t2build tcpWin
...
$ t2 -r ~/data/faf-exercise.pcap -w ~/results/
================================================================================
Tranalyzer 0.8.11 (Anteater), Tarantula. PID: 17890
================================================================================
[INF] Creating flows for L2, IPv4, IPv6 [FORCE]
Active plugins:
01: basicFlow, 0.8.11
02: tcpStates, 0.8.11
03: tcpWin, 0.8.11
04: txtSink, 0.8.11
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406208 (406.21 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51196 (51.20 K)
Processing file: /home/wurst/data/faf-exercise.pcap
Link layer type: Ethernet [EN10MB/1]
Dump start: 1258544215.037210 sec (Wed 18 Nov 2009 11:36:55 GMT)
Dump stop : 1258594491.683288 sec (Thu 19 Nov 2009 01:34:51 GMT)
Total dump duration: 50276.646078 sec (13h 57m 56s)
Finished processing. Elapsed time: 0.006016 sec
Finished unloading flow memory. Time: 0.006036 sec
Percentage completed: 100.00%
Number of processed packets: 5902 (5.90 K)
Number of processed bytes: 4993414 (4.99 M)
Number of raw bytes: 4993414 (4.99 M)
Number of pcap bytes: 5087870 (5.09 M)
Number of IPv4 packets: 5902 (5.90 K) [100.00%]
Number of A packets: 3533 (3.53 K) [59.86%]
Number of B packets: 2369 (2.37 K) [40.14%]
Number of A bytes: 4423178 (4.42 M) [88.58%]
Number of B bytes: 570236 (570.24 K) [11.42%]
Average A packet load: 1251.96 (1.25 K)
Average B packet load: 240.71
--------------------------------------------------------------------------------
tcpStates: Aggregated tcpStatesAFlags=0x4b
--------------------------------------------------------------------------------
Headers count: min: 3, max: 3, average: 3.00
Number of TCP packets: 5902 (5.90 K) [100.00%]
Number of TCP bytes: 4993414 (4.99 M) [100.00%]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of processed flows: 74
Number of processed A flows: 37 [50.00%]
Number of processed B flows: 37 [50.00%]
Number of request flows: 37 [50.00%]
Number of reply flows: 37 [50.00%]
Total A/B flow asymmetry: 0.00
Total req/rply flow asymmetry: 0.00
Number of processed packets/flows: 79.76
Number of processed A packets/flows: 95.49
Number of processed B packets/flows: 64.03
Number of processed total packets/s: 0.12
Number of processed A+B packets/s: 0.12
Number of processed A packets/s: 0.07
Number of processed B packets/s: 0.05
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of average processed flows/s: 0.00
Average full raw bandwidth: 795 b/s
Average full bandwidth : 792 b/s
Max number of flows in memory: 18 [0.01%]
Memory usage: 0.01 GB [0.02%]
Aggregated flowStat=0x0400020000004000
[WRN] Number of flows terminated by force mode: 2 [2.70%]
[INF] IPv4 flows
Note the warning at the end of the T2 report. So we have total now 74 flows (originally 72), and two were released early. The following packets are deposited into a new flow having the same 6-tuple. Let’s select the early terminated flows by a logical AND using the RMFLOW
bit in flowStat as defined in flow.h in T2 core:
$ tawk -V flowStat=0x0400020000004000
The flowStat column with value 0x0400020000004000 is to be interpreted as follows:
bit | flowStat | Description
=============================================================================
14 | 0x0000 0000 0000 4000 | IPv4 flow
41 | 0x0000 0200 0000 0000 | Force mode: remove this flow instantly
58 | 0x0400 0000 0000 0000 | IPv4 packet
The tawk command to select all forced flow is then:
$ tawk 'bitsanyset($flowStat, 0x0000020000000000)' faf-exercise_flows.txt
%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType ethVlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto tcpStates tcpWinStat tcpWinThCnt
A 36 0x0400020000004000 1258594163.408285 1258594164.647755 1.239470 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 0x02 0x01 2
B 37 0x0400020000004001 1258594165.319087 1258594191.015208 25.696121 1 3 eth:ipv4:tcp 00:08:74:38:01:b4 00:19:e3:e7:5d:23 0x0800 192.168.1.105 07 "Private network" 49330 143.166.11.10 us "Dell" 64334 6 0x43 0x01 2
See? Play a bit around!
If you are interested in details, the macro T2_RM_FLOW(...)
implements the following code in utils/t2Plugin.h:
#if FORCE_MODE == 1
#define T2_RM_FLOW(flowP) { \
(flowP)->status |= RMFLOW; \
globalWarn |= (RMFLOW); \
rm_flows[num_rm_flows++] = flowP; \
numForced++; \
}
#else // FORCE_MODE == 0
#define T2_RM_FLOW(flowP)
#endif // FORCE_MODE
And defines including the flow removal code are located in main.c:
#if (FORCE_MODE == 1 || FDURLIMIT > 0)
unsigned long num_rm_flows;
flow_t *rm_flows[10];
#endif // (FORCE_MODE == 1 || FDURLIMIT > 0)
...
The T2_RM_FLOW(...)
macro sets the RMFLOW
bit in flow status. The core will then act on that signal and releases all flows in the rm_flows[num_rm_flows]
stack holding all flow pointers to be terminated next time the core is active. Moreover, it increments the numForced
counter for the end report.
The dimension of rm_flows
is defined also in main.c. Currently there are only 10 elements defined, so you can also terminate dependent flows, such as in ICMP. You normally require maximum two elements, but when we go parallel then we will need more.
A very powerful concept, but always be sure of what you are doing. If you have problems don’t hesitate to contact the Anteater, he’ll help
Note, as for the ALARM mode
, if you change the FORCE_MODE
in the core then you have to recompile all loaded plugins with t2build -R
, otherwise you have inconsistent settings in core and plugins.
$ t2conf tranalyzer2 -D FORCE_MODE=0 && t2build -R
...
$
Or use the new command: t2conf --reset tranalyzer2 && t2build -R
Have fun writing plugins!
The next tutorial will teach you how to add support for pcap extraction.
See Also
- Plugin Programming Cheatsheet
- The basics: your first flow plugin
- Adding plugin end report
- Adding plugin monitoring output
- Adding plugin packet output
- Producing summary files
- geo-whois-labeling
- All about plugin dependencies
- Plugin sinks
- Manipulating flow timeouts
- Alarm mode
- Pcap extraction
- Developing Tranalyzer plugins in C++
- Developing Tranalyzer plugins in Rust