Tutorial: Plugin dependencies
Contents
In order to avoid reinventing the wheel, T2 provides the means to access other plugins results. Before we start doing this, clean out your plugin .so directory if this is the first tutorial you follow. Then all unnecessary plugins should be deleted from the plugin folder ~/.tranalyzer/plugins and compile basicFlow, basicStats and txtSink.
$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build tranalyzer2 basicFlow basicStats tcpStates txtSink
...
$If you didn’t read the tutorials before, here is the basis plugin which we will extend: tcpWin If you just try the end result: tcpWinD, but then you learn nothing. Implementing code yourself is the best method of learning.
The anonymized sample pcap can be downloaded here: annoloc2.pcap. Please extract it under your data folder: ~/data, if you haven’t already. Now you are all set for dependency stuff.
Accessing core internal information
If you need the number of TCP packets or any other global counts defined in t2stats.h, which is included in t2Plugins.h every plugin includes. So no dependency is needed.
$ tranalyzer2
$ vi src/t2stats.h...
// counter and monitoring diff mode vars
extern struct timeval startTStamp, startTStamp0;
extern uint32_t numAlarms, numAlarms0;
extern uint16_t numHdrDesc;
extern uint64_t numOfPackets, numOfPackets0;
extern uint64_t numOfV4Packets, numOfV4Packets0;
extern uint64_t numOfV6Packets, numOfV6Packets0;
extern uint64_t numOfVxPackets, numOfVxPackets0;
extern uint64_t totalFlows, totalFlows0;
extern uint64_t numOfAPackets, numOfAPackets0;
extern uint64_t numOfBPackets, numOfBPackets0;
extern uint64_t numOfABytes, numOfABytes0;
extern uint64_t numOfBBytes, numOfBBytes0;
...
// global L3 protocols
extern uint64_t numPacketsL3[256], numPackets0L3[256];
extern uint64_t numPacketsL2[65536], numPackets0L2[65536];
...So if you need any of these variables, which denote the aggregated counts over all packets up to the time of request, just use them. BUT NEVER EVER WRITE TO THEM !!! We give you as developer the freedom to access core variables and function as we do not try to treat programmers as airbus treats pilots. So we give you access, please treat the anteater respectfully.
Accessing other plugins information
If you need the number of TCP packets or any other global counts defined in t2stat.h, no dependency is needed.
For plugins, a bit differently. Hence, the following rules apply:
- the plugin supplying data must have a number lower than yours
- the header file (pluginName.h) of the plugin must be included at the beginning of your plugin
- Replace
T2_PLUGIN_INIT(...)withT2_PLUGIN_INIT_WITH_DEPS(...)(and add the dependencies) - the path of the plugin must be added in your Makefile.am
- variables you are interested in must be declared as
extern ... __attribute__((weak));
It may sound complicated, but it actually is not. Just stick to the plugin coding rules and you’ll be fine.
In this example, we will add a dependency to the basicStats plugin, to access the number of transmitted packets numTPkts. Open your tcpWin.c and replace T2_PLUGIN_INIT(...) with T2_PLUGIN_INIT_WITH_DEPS(...). Just add the lines marked by <--
...
#include "tcpWin.h"
#include "basicStats.h" // <--
tcpWinFlow_t *tcpWinFlows;
extern bSFlow_t *bSFlow __attribute__((weak)); // <--
T2_PLUGIN_INIT_WITH_DEPS("tcpWin", "0.8.11", 0, 8, "basicStats"); // <--
...So we can access the basicStats flow struct now, as shown below:
typedef struct {
uint64_t numTPkts; // Number of packets transmitted.
uint64_t numTBytes; // Number of bytes transmitted (depends on PACKETLENGTH)
uint64_t totTBytes; // Number of bytes transmitted (total rawLength)
#if BS_STATS == 1
#if BS_XCLD > 0
uint64_t numTPkts0; // Number of packets transmitted pktlen > 0
#endif // BS_XCLD > 0
struct timeval lst;
float avePktSzf;
#if BS_VARSTD > 0
float varPktSz;
float varIATSz;
#endif // BS_VARSTD > 0
float aveIATSzf;
float minIAT;
float maxIAT;
uint16_t minL3PktSz; // Smallest L3 packet size detected
uint16_t maxL3PktSz; // Largest L3 packet size detected
#endif // BS_STATS == 1
} bSFlow_t;Open tcpWin.c and add the lines marked by <-- in the onFlowTerminate(...) callback function. There we define a pointer to the struct of that very flowIndex. Then we replace tcpWinFlowP->pktTcpCnt with bSFlowP->numTPkts.
void onFlowTerminate(unsigned long flowIndex) {
tcpWinFlow_t * const tcpWinFlowP = &tcpWinFlows[flowIndex];
bSFlow_t * const bSFlowP = &bSFlow[flowIndex]; // <--
OUTBUF_APPEND_U8(main_output_buffer, tcpWinFlowP->stat);
OUTBUF_APPEND_U32(main_output_buffer, tcpWinFlowP->winThCnt);
float f = 0.0; // <--
if (bSFlowP->numTPkts) f = (float)tcpWinFlowP->winThCnt/(float)bSFlowP->numTPkts; // <--
OUTBUF_APPEND_FLT(main_output_buffer, f); // <--
}And add an output column for the relative window size threshold measure in the printHeader().
binary_value_t* printHeader() {
binary_value_t *bv = NULL;
BV_APPEND_H8(bv, "tcpWinStat", "tcp Windowsize threshold status"); // 8 bit hex variable
BV_APPEND_U32(bv, "tcpWinThCnt", "tcp Windowsize theshold count"); // 32 unsigned int variable
BV_APPEND_FLT(bv, "tcpRelWinThCnt", "tcp Relative Windowsize threshold count"); // float variable
return bv;
}That’s it for the code part. Now we need to tell the compiler what we want.
For meson add the depending plugin in the inc statement in meson.build
$ tcpWin
$ vi meson.build
...
inc = include_directories(
join_paths('..', '..', 'utils'),
join_paths('..', '..', 'tranalyzer2', 'src'),
join_paths('..', 'basicStats', 'src'), # <--
)
...For cmake in the target_include_directories of CMakeLists.txt
$ vi CMakeLists.txt...
target_include_directories(${_plugin_name}
PRIVATE
../../utils
../../tranalyzer2/src
../../basicStats/src
)
...And for autotools, open the Makefile.am and add the path to the source of the basicStats plugin in libtcpWin_la_CFLAGS as outlined below
$ vi src/Makefile.amlib_LTLIBRARIES = libtcpWin.la
libtcpWin_la_SOURCES = tcpWin.c
libtcpWin_la_CFLAGS = \
-I$(top_srcdir)/../../utils \
-I$(top_srcdir)/../../tranalyzer2/src \
-I$(top_srcdir)/../../basicStats/src
if APPLE
libtcpWin_la_CFLAGS += -D_DARWIN_C_SOURCE # OSX specific flags
else
libtcpWin_la_CFLAGS += -D_GNU_SOURCE
endif
libtcpWin_la_LDFLAGS = -shrext .so # extension for shared libraryThen your plugin does its job for all build methods. Fine, now try to build your changes and see whether it complains. If all is good run t2 on the pcap, otherwise, debug. If you have difficulties, please don’t hesitate to send the Anteater an mail.
$ t2build tcpWin
...
$ t2 -r ~/data/annoloc2.pcap -w ~/results
================================================================================
Tranalyzer 0.8.11 (Anteater), Tarantula. PID: 6508
================================================================================
[INF] Creating flows for L2, IPv4, IPv6
Active plugins:
01: basicFlow, 0.8.11
02: basicStats, 0.8.11
03: tcpStates, 0.8.11
04: tcpWin, 0.8.11
05: txtSink, 0.8.11
[INF] IPv4 Ver: 5, Rev: 16122020, Range Mode: 0, subnet ranges loaded: 406208 (406.21 K)
[INF] IPv6 Ver: 5, Rev: 17122020, Range Mode: 0, subnet ranges loaded: 51196 (51.20 K)
Processing file: /home/wurst/data/annoloc2.pcap
Link layer type: Ethernet [EN10MB/1]
...
$Change to your results window and look at the flow file. You will notice that the relative measure matches the ones from previous tutorials. At least record displayed might convince you that plugin dependency works.
$ tcol annoloc2_flows.txt%dir flowInd flowStat timeFirst timeLast duration numHdrDesc numHdrs hdrDesc srcMac dstMac ethType ethVlanID srcIP srcIPCC srcIPOrg srcPort dstIP dstIPCC dstIPOrg dstPort l4Proto numPktsSnt numPktsRcvd numBytesSnt numBytesRcvd minPktSz maxPktSz avePktSize stdPktSize minIAT maxIAT aveIAT stdIAT pktps bytps pktAsm bytAsm tcpStates tcpWinStat tcpWinThCnt tcpRelWinThCnt
A 265 0x0400000000004000 1022171701.709116 1022171701.709116 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:fc:0e:21:56 0x0800 209.171.12.143 ca "TELUS Communications Inc" 4987 138.212.185.230 jp "ASAHI KASEI CORPORATION" 41250 6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0xc3 0x01 1 1
A 447 0x0400000000004000 1022171701.721366 1022171701.721366 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:fc:3b:62:78 0x0800 217.41.129.13 gb "BT Infrastructure Layer" 58872 138.212.187.186 jp "ASAHI KASEI CORPORATION" 80 6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0xc3 0x01 1 1
A 392 0x0400000000004000 1022171701.716998 1022171701.716998 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:bf:59:85:48 0x0800 36.242.181.230 jp "SoftBank Corp" 4685 138.212.188.67 jp "ASAHI KASEI CORPORATION" 1214 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x03 0x00 0 0
B 392 0x0400000000004001 1022171701.732313 1022171701.732313 0.000000 1 3 eth:ipv4:tcp 00:50:bf:59:85:48 00:d0:02:6d:78:00 0x0800 138.212.188.67 jp "ASAHI KASEI CORPORATION" 1214 36.242.181.230 jp "SoftBank Corp" 4685 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x43 0x01 1 1
A 905 0x0400000000004000 1022171701.816638 1022171701.816638 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:60:08:69:80:dd 0x0800 161.135.53.11 us "Federal Express Corp" 5001 138.212.191.94 jp "ASAHI KASEI CORPORATION" 80 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x03 0x00 0 0
B 905 0x0400000000004001 1022171701.817195 1022171701.817195 0.000000 1 3 eth:ipv4:tcp 00:60:08:69:80:dd 00:d0:02:6d:78:00 0x0800 138.212.191.94 jp "ASAHI KASEI CORPORATION" 80 161.135.53.11 us "Federal Express Corp" 5001 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x43 0x01 1 1
A 1026 0x0400000000004000 1022171701.872817 1022171701.872817 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:80:48:b3:13:27 0x0800 146.162.158.230 gb "Norwich Union Insurance Limite" 2849 138.212.184.193 jp "ASAHI KASEI CORPORATION" 6346 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x03 0x00 0 0
B 1026 0x0400000000004001 1022171701.873426 1022171701.873426 0.000000 1 3 eth:ipv4:tcp 00:80:48:b3:13:27 00:d0:02:6d:78:00 0x0800 138.212.184.193 jp "ASAHI KASEI CORPORATION" 6346 146.162.158.230 gb "Norwich Union Insurance Limite" 2849 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x43 0x01 1 1
A 1153 0x0400000000004000 1022171701.939627 1022171701.939627 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:50:bf:59:85:48 0x0800 193.133.224.57 gb "Verizon UK Limited" 3286 138.212.188.67 jp "ASAHI KASEI CORPORATION" 1214 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x03 0x00 0 0
B 1153 0x0400000000004001 1022171701.947575 1022171701.947575 0.000000 1 3 eth:ipv4:tcp 00:50:bf:59:85:48 00:d0:02:6d:78:00 0x0800 138.212.188.67 jp "ASAHI KASEI CORPORATION" 1214 193.133.224.57 gb "Verizon UK Limited" 3286 6 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0x43 0x01 1 1
A 867 0x0400000200004000 1022171701.805350 1022171701.805350 0.000000 1 3 eth:ipv4:tcp 00:60:b0:b5:da:10 00:d0:02:6d:78:00 0x0800 138.212.184.48 jp "ASAHI KASEI CORPORATION" 6666 36.74.248.27 id "Telekomunikasi Indonesia" 1108 6 1 1 137 0 137 137 137 0 0 0 0 0 0 0 0 1 0x03 0x00 0 0
B 867 0x0400000000004001 1022171702.012658 1022171702.012658 0.000000 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:60:b0:b5:da:10 0x0800 36.74.248.27 id "Telekomunikasi Indonesia" 1108 138.212.184.48 jp "ASAHI KASEI CORPORATION" 6666 6 1 1 0 137 0 0 0 0 0 0 0 0 0 0 0 -1 0x43 0x01 1 1
A 864 0x0400000200004000 1022171701.805329 1022171702.066438 0.261109 1 3 eth:ipv4:tcp 00:d0:02:6d:78:00 00:60:b0:ec:34:27 0x0800 19.54.241.65 us "Ford Motor Company" 6667 138.212.191.209 jp "ASAHI KASEI CORPORATION" 45891 6 3 3 324 0 0 185 108 59.8867 0 0.214542 0.08703633 0.06619374 11.48946 1240.861 0 1 0x03 0x00 0 0
B 864 0x0400000000004001 1022171701.806695 1022171702.066682 0.259987 1 3 eth:ipv4:tcp 00:60:b0:ec:34:27 00:d0:02:6d:78:00 0x0800 138.212.191.209 jp "ASAHI KASEI CORPORATION" 45891 19.54.241.65 us "Ford Motor Company" 6667 6 3 3 0 324 0 0 0 0 0 0.206711 0.08666233 0.06270833 11.53904 0 0 -1 0x43 0x01 1 0.3333333
...As an exercise, try to use more information from the basicStats plugin structure. If you use variables which can be influenced by a reconfiguration of basicStats you need to apply the same compiler pragmas as in bSFlow_t.
Make your Plugin a dependency
This is very easy, open tcpWin.h and look at the end of the file:
...
// plugin structures
typedef struct { // always large variables first to limit memory fragmentation
uint32_t winThCnt;
uint8_t stat;
} tcpWinFlow_t;
// plugin struct pointer for potential dependencies
extern tcpWinFlow_t *tcpWinFlows;
#endif // __TCP_WIN_H__The extern tcpWinFlow_t *tcpWinFlows; statement assures that whoever includes your tcpWin.h file will have access to the tcpWinFlow_t flow memory. Any plugin global variable or structure you define here as extern is in principle available for other plugins to access.
The real pointers to the whole struct flow memory have to be defined in your tcpWin.c file as global variables.
...
/*
* Plugin variables that may be used by other plugins (MUST be declared in
* the header file e.g. 'extern tcpWinFlow_t *tcpWinFlows;'
*/
tcpWinFlow_t *tcpWinFlows;
...So the tcpWinFlows structure of tcpWin was already eligible as a dependency.
Have fun!
The next tutorial will teach you how to write a sink plugin
See Also
- Plugin Programming Cheatsheet
- The basics: your first flow plugin
- Adding plugin end report
- Adding plugin monitoring output
- Adding plugin packet output
- Producing summary files
- geo-whois-labeling
- Plugin sinks
- Manipulating flow timeouts
- Alarm mode
- Force mode
- Pcap extraction
- Developing Tranalyzer plugins in C++
- Developing Tranalyzer plugins in Rust