Plugin programming cheatsheet
Contents
- Tutorials
- Creating a new plugin
- Plugin initialization
- Callbacks
- void t2Init()
- binary_value_t* t2PrintHeader()
- void t2OnNewFlow(packet_t *packet, unsigned long flowIndex)
- void t2OnLayer2(packet_t *packet, unsigned long flowIndex)
- void t2OnLayer4(packet_t *packet, unsigned long flowIndex)
- void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf)
- void t2Finalize()
- void t2PluginReport(FILE *stream)
- void t2Monitoring(FILE *stream, uint8_t state)
- void t2SaveState(FILE *stream)
- void t2RestoreState(const char *str)
- void t2BufferToSink(outputBuffer_t buf, binary_value_t bv)
- Accessing the plugin name
- Accessing the various layers
- Selecting flows by port/protocol
- Identifying flows with flow index and direction, e.g., for filenames
- Describing flow file output in t2PrintHeader()
- Bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)
- Signed integers (8, 16, 32 and 64 bits)
- Unsigned integers (8, 16, 32 and 64 bits)
- Floating point (float, double)
- MAC addresses
- IPv4, IPv6 and IPvX (IP version and IP) addresses
- Timestamp, duration
- Characters, strings and string classes (strings without quotes)
- If the type of the column varies
- Compounds, e.g., field1_field2
- Existing types
- Generating flow file output in t2OnFlowTerminate()
- Unsigned integers, bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)
- Signed integers (8, 16, 32 and 64 bits)
- Floating point (float, double)
- MAC
- IPv4, IPv6 and IPvX (IP version and IP) addresses
- Timestamp, duration
- Characters, strings and string classes (strings without quotes)
- If the type of the column varies
- Repetitive values
- Adding support for environment/runtime configuration flags
- Using T2 file manager
- Parsing data with t2buf
- Structures
- Reporting errors, warnings and other information
- Reporting aggregated bitfields
- Reporting hexadecimal numbers
- Reporting numbers
- Reporting MAC addresses
- Reporting IP addresses
- Reporting subnet information
- Working with strings
- Miscellaneous macros
- Handy C snippets
- Debugging
- Toggle test
- Scripts
- Documentation
This cheatsheet summarizes the most important callbacks, functions, macros and structures required to write your own plugins. For more details about the different aspects of writing your own plugins, refer to the t2PSkel plugin or to one of the tutorials.
Tutorials
- The basics: your first flow plugin
- Plugin end report
- Plugin monitoring
- Plugin packet mode
- Plugin summary files
- Plugin geo labeling
- Plugin dependencies
- Plugin alarm mode
- Plugin force mode
- Plugin pcap extraction
- Plugin flow timeout
- Plugin sink
- Developing Tranalyzer plugins in C++
- Developing Tranalyzer plugins in Rust
Creating a new plugin
Creating a new C plugin
t2plugin -c pluginName
Creating a new minimal C plugin
t2plugin -c pluginName -m
Creating a new minimal sink plugin
t2plugin -c pluginName -s
Creating a new minimal t2buf plugin
t2plugin -c pluginName -t
Creating a new C plugin with a specific plugin number
t2plugin -c pluginName -n 123
Creating a new C++ plugin
[Tutorial] t2plugin -c pluginName --cpp
Creating a new Rust plugin
[Tutorial] t2plugin -c pluginName --rust
Plugin initialization
Without any dependencies to other plugins
("yourPluginName", "0.9.1", 0, 9); T2_PLUGIN_INIT
With dependency to one plugin
("pluginName", "0.9.1", 0, 9, "tcpFlags"); T2_PLUGIN_INIT_WITH_DEPS
With dependency to several plugins
("pluginName", "0.9.1", 0, 9, "tcpFlags,tcpStates"); T2_PLUGIN_INIT_WITH_DEPS
Callbacks
void t2Init()
Initialization: allocate memory here [Tutorial]
void t2Init() {
// allocate struct for all flows, initialize to 0 and check for errors
(pluginNameFlows);
T2_PLUGIN_STRUCT_NEW}
binary_value_t* t2PrintHeader()
Describe your flow output here [Tutorial]
* t2PrintHeader() {
binary_value_t*bv = NULL;
binary_value_t // Describe your output fields with the BV_APPEND macros detailed in the next section, e.g.:
(bv, "mypluginStat" , "description...");
BV_APPEND_H8(bv, "mypluginCnt" , "description...");
BV_APPEND_U64return bv;
}
void t2OnNewFlow(packet_t *packet, unsigned long flowIndex)
A new flow has been generated: reset its memory and try to determine if it is relevant for the plugin (protocol, ports, …) [Tutorial]
void t2OnNewFlow(packet_t *packet, unsigned long flowIndex) {
// Reset the structure for this flow
(pluginNameFlows, flowIndex);
T2_PLUGIN_STRUCT_RESET_ITEM//pluginNameFlow_t * const pnFlowP = &pluginNameFlows[flowIndex];
//memset(pnFlowP, '\0', sizeof(*pnFlowP));
}
void t2OnLayer2(packet_t *packet, unsigned long flowIndex)
A new packet with a layer 2 (if it also has a layer 4, then flowIndex
is equal to HASHTABLE_ENTRY_NOT_FOUND
)
[Tutorial]
void t2OnLayer2(packet_t *packet, unsigned long flowIndex) {
if (flowIndex == HASHTABLE_ENTRY_NOT_FOUND) return;
// This packet does not have a layer 4.
}
void t2OnLayer4(packet_t *packet, unsigned long flowIndex)
A new packet with a layer 4 [Tutorial]
void t2OnLayer4(packet_t *packet, unsigned long flowIndex)
void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf)
A flow has been terminated: finish computations for this flow and push your data to the output buffer [Tutorial]
void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf) {
* const pnFlowP = &pluginNameFlows[flowIndex];
pluginNameFlow_t (buf, pnFlowP->mypluginStat);
OUTBUF_APPEND_U8(buf, pnFlowP->mypluginCnt);
OUTBUF_APPEND_U64}
void t2Finalize()
Free the memory here [Tutorial]
void t2Finalize() {
(pluginNameFlows);
free}
void t2PluginReport(FILE *stream)
Contribute plugin specific information to the end report [Tutorial]
static uint8_t mypluginStat;
static uint64_t mypluginCnt;
void t2PluginReport(FILE *stream) {
if (!mypluginCnt) return;
(stream, plugin_name, "Number of myplugin packets", mypluginCnt);
T2_FPLOG_NUMP0(stream, plugin_name, mypluginStat);
T2_FPLOG_AGGR_HEX// => T2_FPLOG(stream, plugin_name, "Aggregated mypluginStat=0x%02" B2T_PRIX8, mypluginStat);
}
void t2Monitoring(FILE *stream, uint8_t state)
Monitoring mode [Tutorial]
static uint8_t mypluginStat;
static uint64_t mypluginCnt, mypluginCnt0;
void t2Monitoring(FILE *stream, uint8_t state) {
switch (state) {
case T2_MON_PRI_HDR: // Print the name of the variables that will be output
("mypluginStat\tmypluginCnt\t", stream); // Note the trailing tab (\t)
fputsreturn;
case T2_MON_PRI_VAL: // Print the variables to monitor
(stream, "%" PRIu64 "\t0x%02" B2T_PRIX8 "\t", // Note the trailing tab (\t)
fprintf- mypluginCnt0, mypluginStat);
mypluginCnt break;
case T2_MON_PRI_REPORT: // print a report similar to t2PluginReport()
(stream, plugin_name, "Number of myplugin packets", mypluginCnt);
T2_FPLOG_DIFFNUMP(stream, plugin_name, mypluginStat);
T2_FPLOG_AGGR_HEX// => T2_FPLOG(stream, plugin_name, "Aggregated mypluginStat=0x%02" B2T_PRIX8, mypluginStat);
break;
default: // Invalid state, do nothing
return;
}
#if DIFF_REPORT == 1
= mypluginCnt;
mypluginCnt0 #endif
}
void t2SaveState(FILE *stream)
Save the state of your plugin
static uint8_t mypluginStat;
static uint64_t mypluginCnt;
void t2SaveState(FILE *stream)
(stream, "0x%02" PRIx8 "\t%" PRIu64, mypluginStat, mypluginCnt);
fprintf}
void t2RestoreState(const char *str)
Restore the state of your plugin
static uint8_t mypluginStat;
static uint64_t mypluginCnt;
void t2RestoreState(const char *str)
(str, "0x%02" SCNx8 "\t%" SCNu64, &mypluginStat, &mypluginCnt);
sscanf}
void t2BufferToSink(outputBuffer_t buf, binary_value_t bv)
Dump the content of buf
to a file, stream, …
Use bv
to decode it. (Sink plugins only)
[Tutorial]
void t2BufferToSink(outputBuffer_t *buf, binary_value_t *bv)
Accessing the plugin name
The T2_PLUGIN_INIT()
and T2_PLUGIN_INIT_WITH_DEPS()
macros both define the following static variable:
static const char * const plugin_name = "pluginName";
This variable being static, it can only be accessed in the pluginName.c file.
However, nothing prevents you from redefining it in a different file!
As an example, the plugin_name
variable can be used with the macros defined here, e.g., T2_PLOG(plugin_name, "Andy was here!");
Accessing the various layers
Layer 2 header
The layer 2 header (Ethernet) can be accessed as follows:
const uint16_t ethType = L2_PROTO(packet);
//const uint16_t ethType = packet->ethType;
const ethernetHeader_t * const ethHdrP = ETH_HEADER(packet);
//const ethernetHeader_t * const ethHdrP = (ethernetHeader_t*) packet->l2HdrP;
const lapdHdr_t * const lapdHdrP = LAPD_HEADER(packet);
//const lapdHdr_t * const lapdHdrP = (ethernetHeader_t*) packet->l2HdrP;
const uint8_t * const l2HdrP = L2_HEADER(packet);
//const uint8_t * const l2HdrP = packet->l2HdrP;
Layer 3 header
The layer 3 header (IPv4, IPv6) can be accessed as follows:
if (PACKET_IS_IPV6(packet)) {
const ip6Header_t * const ip6HdrP = IPV6_HEADER(packet); // IPv6
//const ip6Header_t * const ip6HdrP = (ip6Header_t*) packet->l3HdrP; // IPv6
} else if (PACKET_IS_IPV4(packet)) {
const ipHeader_t * const ipHdrP = IPV4_HEADER(packet); // IPv4
//const ipHeader_t * const ipHdrP = (ipHeader_t*) packet->l3HdrP; // IPv4
} else {
const uint8_t * const l3HdrP = L3_HEADER(packet);
//const uint8_t * const l3HdrP = packet->l3HdrP;
}
Layer 4 header
The layer 4 header (TCP, UDP, ICMP, SCTP, …) can be accessed as follows:
switch (L4_PROTO(packet)) {
//switch (packet->l4Proto) {
case L3_TCP: {
const tcpHeader_t * const tcpHdrP = TCP_HEADER(packet);
//const tcpHeader_t * const tcpHdrP = (tcpHeader_t*) packet->l4HdrP;
break;
}
case L3_UDP: {
const udpHeader_t * const udpHdrP = UDP_HEADER(packet);
//const udpHeader_t * const udpHdrP = (udpHeader_t*) packet->l4HdrP;
break;
}
case L3_ICMP: {
const icmpHeader_t * const icmpHdrP = ICMP_HEADER(packet);
//const icmpHeader_t * const icmpHdrP = (icmpHeader_t*) packet->l4HdrP;
break;
}
case L3_SCTP: {
const sctpHeader_t * const sctpHdrP = SCTP_HEADER(packet);
//const sctpHeader_t * const sctpHdrP = (sctpHeader_t*) packet->l4HdrP;
break;
}
case L3_PIM: {
const pimHeader_t * const pimHdrP = PIM_HEADER(packet);
//const pimHeader_t * const pimHdrP = (pimHeader_t*) packet->l4HdrP;
break;
}
default: {
const uint8_t * const l4HdrP = L4_HEADER(packet);
//const uint8_t * const l4HdrP = packet->l4HdrP;
break;
}
}
Layer 7 header
The layer 7 header can be accessed as follows:
const myProto_t * const myProtoHdrP = (myProto_t*) L7_HEADER(packet);
//const myProto_t * const myProtoHdrP = (myProto_t*) packet->l7HdrP;
Selecting flows by port/protocol
Selecting flows by port
Accessing the source/destination port:
const uint16_t srcPort = flowP->srcPort;
const uint16_t dstPort = flowP->dstPort;
Selecting flows by protocol
Accessing the layer 2 protocol:
const uint16_t ethType = L2_PROTO(packet); //const uint16_t ethType = packet->ethType; const uint16_t ethType = L2_PROTO(flowP); //const uint16_t ethType = flowP->ethType;
Accessing the layer 3 protocol:
const uint8_t proto = L3_PROTO(packet); //const uint8_t proto = packet->l3Proto; if (proto == ETHERTYPE_IP) { // ... } else if (proto == ETHERTYPE_IPV6) { // ... }
Accessing the layer 4 protocol (TCP, UDP, ICMP, …):
const uint8_t proto = L4_PROTO(packet); //const uint8_t proto = packet->l4Proto; const uint8_t proto = L4_PROTO(flowP); //const uint8_t proto = flowP->l4Proto;
Identifying flows with flow index and direction, e.g., for filenames
// Flow index
const uint64_t flowInd = flowP->findex;
// Flow direction
const char * const dir_s = FLOW_DIR_S(flowP); // "A" or "B"
const char dir_c = FLOW_DIR_C(flowP); // 'A' or 'B'
// Flow direction (alternative)
char *dir_s;
char dir_c;
if (FLOW_IS_B(flowP)) {
= FLOW_DIR_S_B;
dir_s = FLOW_DIR_C_B;
dir_c } else { // FLOW_IS_A(flowP))
= FLOW_DIR_S_A;
dir_s = FLOW_DIR_C_A;
dir_c }
// Filename example
char *filename = t2_strdup_printf("/tmp/prefix_%" PRIu64 "_%c.bin", flowInd, dir_c); // direction as char
//char *filename = t2_strdup_printf("/tmp/prefix_%" PRIu64 "_%s.bin", flowInd, dir_s); // direction as string
(filename); free
Describing flow file output in t2PrintHeader()
Bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)
// Single values, e.g., 0x01
( bv, "myHex8Field" , "description...");
BV_APPEND_H8(bv, "myHex16Field", "description...");
BV_APPEND_H16(bv, "myHex32Field", "description...");
BV_APPEND_H32(bv, "myHex64Field", "description...");
BV_APPEND_H64
// Repetitive values, e.g., 0x01;0x02
( bv, "myHex8Field" , "description...");
BV_APPEND_H8_R(bv, "myHex16Field", "description...");
BV_APPEND_H16_R(bv, "myHex32Field", "description...");
BV_APPEND_H32_R(bv, "myHex64Field", "description..."); BV_APPEND_H64_R
Signed integers (8, 16, 32 and 64 bits)
// Single values, e.g., -1
( bv, "myInt8Field" , "description...");
BV_APPEND_I8(bv, "myInt16Field", "description...");
BV_APPEND_I16(bv, "myInt32Field", "description...");
BV_APPEND_I32(bv, "myInt64Field", "description...");
BV_APPEND_I64
// Repetitive values, e.g., -1;-2
( bv, "myInt8Field" , "description...");
BV_APPEND_I8_R(bv, "myInt16Field", "description...");
BV_APPEND_I16_R(bv, "myInt32Field", "description...");
BV_APPEND_I32_R(bv, "myInt64Field", "description..."); BV_APPEND_I64_R
Unsigned integers (8, 16, 32 and 64 bits)
// Single values, e.g., 1
( bv, "myInt8Field" , "description...");
BV_APPEND_U8(bv, "myInt16Field", "description...");
BV_APPEND_U16(bv, "myInt32Field", "description...");
BV_APPEND_U32(bv, "myInt64Field", "description...");
BV_APPEND_U64
// Repetitive values, e.g., 1;2
( bv, "myInt8Field" , "description...");
BV_APPEND_U8_R(bv, "myInt16Field", "description...");
BV_APPEND_U16_R(bv, "myInt32Field", "description...");
BV_APPEND_U32_R(bv, "myInt64Field", "description..."); BV_APPEND_U64_R
Floating point (float, double)
// Single values, e.g., 1.1
(bv, "myFloatField" , "description...");
BV_APPEND_FLT(bv, "myDoubleField", "description...");
BV_APPEND_DBL
// Repetitive values, e.g., 1.1;2.2
(bv, "myFloatField" , "description...");
BV_APPEND_FLT_R(bv, "myDoubleField", "description..."); BV_APPEND_DBL_R
MAC addresses
// Single values, e.g., 00:11:22:33:44:55
(bv, "myMACField" , "description...");
BV_APPEND_MAC
// Repetitive values, e.g., 00:11:22:33:44:55;01:23:45:67:89:01
(bv, "myMACField" , "description..."); BV_APPEND_MAC_R
IPv4, IPv6 and IPvX (IP version and IP) addresses
// Single values, e.g., 1.2.3.4
(bv, "myIPv4Field" , "description...");
BV_APPEND_IP4(bv, "myIPv6Field" , "description...");
BV_APPEND_IP6(bv, "myIPvXField" , "description...");
BV_APPEND_IPX
// Repetitive values, e.g., 1.2.3.4;2.3.4.5
(bv, "myIPv4Field" , "description...");
BV_APPEND_IP4_R(bv, "myIPv6Field" , "description...");
BV_APPEND_IP6_R(bv, "myIPvXField" , "description..."); BV_APPEND_IPX_R
Timestamp, duration
// Single values, e.g., 0.1
(bv, "myTimestampField", "description...");
BV_APPEND_TIMESTAMP( bv, "myDurationField" , "description...");
BV_APPEND_DURATION
// Repetitive values, e.g., 0.1;0.2
(bv, "myTimestampField", "description...");
BV_APPEND_TIMESTAMP_R( bv, "myDurationField" , "description..."); BV_APPEND_DURATION_R
Characters, strings and string classes (strings without quotes)
// Single values, e.g., "string1"
(bv, "myCharField" , "description...");
BV_APPEND_CHAR( bv, "myStringField" , "description...");
BV_APPEND_STR(bv, "myStringClassField", "description...");
BV_APPEND_STRC
// Repetitive values, e.g., "string1";"string2"
(bv, "myCharField" , "description...");
BV_APPEND_CHAR_R( bv, "myStringField" , "description...");
BV_APPEND_STR_R(bv, "myStringClassField", "description..."); BV_APPEND_STRC_R
If the type of the column varies
#if VAL_UINT8 == 1
= bt_hex_8;
binary_type_t myType #else // VAL_UINT8 == 0
= bt_hex_16;
binary_type_t myType #endif // VAL_UINT8 == 0
// Single values, e.g., val1
(bv, "myField", "description...", myType);
BV_APPEND_TYPE
// Repetitive values, e.g., val1;val2
(bv, "myField", "description...", myType); BV_APPEND_TYPE_R
Compounds, e.g., field1_field2
// Single values, e.g., 1_2
(bv, "myCompoundPart1_Part2", "description...", number_of_fields, type1, type2, ...);
BV_APPEND(bv, "myCompoundPart1_Part2", "description...", number_of_fields, type1, type2, ...);
BV_APPEND
// Repetitive values, e.g., 1_2;3_4
(bv, "myCompoundPart1_Part2", "description...", number_of_fields, type1, type2, ...);
BV_APPEND_R(bv, "myCompoundPart1_Part2", "description...", number_of_fields, type1, type2, ...); BV_APPEND_R
Existing types
type | description |
---|---|
bt_int_8 |
8-bit signed integer |
bt_int_16 |
16-bit signed integer |
bt_int_32 |
32-bit signed integer |
bt_int_64 |
64-bit signed integer |
bt_uint_8 |
8-bit unsigned integer |
bt_uint_16 |
16-bit unsigned integer |
bt_uint_32 |
32-bit unsigned integer |
bt_uint_64 |
64-bit unsigned integer |
bt_hex_8 |
8-bit hexadecimal number (unsigned integer) |
bt_hex_16 |
16-bit hexadecimal number (unsigned integer) |
bt_hex_32 |
32-bit hexadecimal number (unsigned integer) |
bt_hex_64 |
64-bit hexadecimal number (unsigned integer) |
bt_float |
Floating point number |
bt_double |
Double precision floating point number |
bt_char |
Single character |
bt_string |
String |
bt_string_class |
Unquoted string |
bt_flow_direction |
A or B (see FLOW_DIR_[CS]_[AB] ) |
bt_timestamp |
Date whose representation depends on B2T_TIMESTR in utils/bin2txt.h |
bt_duration |
One uint64_t value for the seconds and one uint32_t value for the micro-/nano-seconds |
bt_mac_addr |
MAC address |
bt_ip4_addr |
IPv4 address |
bt_ip6_addr |
IPv6 address |
bt_ipx_addr |
IP address: version (8 bits), address (0, 4 or 16 bytes) |
Generating flow file output in t2OnFlowTerminate()
Unsigned integers, bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)
uint8_t val8 = 0;
(buf, val8);
OUTBUF_APPEND_U8
uint16_t val16 = 0;
(buf, val16);
OUTBUF_APPEND_U16
uint32_t val32 = 0;
(buf, val32);
OUTBUF_APPEND_U32
uint64_t val64 = 0;
(buf, val64);
OUTBUF_APPEND_U64
// Automatically converted from network to host order
(buf, val16);
OUTBUF_APPEND_U16_NTOH(buf, val32);
OUTBUF_APPEND_U32_NTOH(buf, val64);
OUTBUF_APPEND_U64_NTOH
// Special cases: append unsigned value 0
(buf);
OUTBUF_APPEND_U8_ZERO(buf);
OUTBUF_APPEND_U16_ZERO(buf);
OUTBUF_APPEND_U32_ZERO(buf); OUTBUF_APPEND_U64_ZERO
Signed integers (8, 16, 32 and 64 bits)
int8_t val8 = 0;
(buf, val8);
OUTBUF_APPEND_I8
int16_t val16 = 0;
(buf, val16);
OUTBUF_APPEND_I16
int32_t val32 = 0;
(buf, val32);
OUTBUF_APPEND_I32
int64_t val64 = 0;
(buf, val64);
OUTBUF_APPEND_I64
// Special cases: append signed value 0
(buf);
OUTBUF_APPEND_I8_ZERO(buf);
OUTBUF_APPEND_I16_ZERO(buf);
OUTBUF_APPEND_I32_ZERO(buf); OUTBUF_APPEND_I64_ZERO
Floating point (float, double)
float valf = 0.0f;
(buf, valf);
OUTBUF_APPEND_FLT
double vald = 0.0;
(buf, vald);
OUTBUF_APPEND_DBL
// Special cases: append floating point value 0
(buf);
OUTBUF_APPEND_FLT_ZERO(buf); OUTBUF_APPEND_DBL_ZERO
MAC
uint8_t mac[ETH_ALEN] = { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55 };
(buf, mac);
OUTBUF_APPEND_MAC
// Special case: append MAC address 00:00:00:00:00:00
(buf); OUTBUF_APPEND_MAC_ZERO
IPv4, IPv6 and IPvX (IP version and IP) addresses
(buf, ip); // ip MUST be of type ipAddr_t or ip4Addr_t
OUTBUF_APPEND_IP4(buf, ip); // ip MUST be of type ipAddr_t
OUTBUF_APPEND_IP6(buf, ip); // ip MUST be of type ipVAddr_t
OUTBUF_APPEND_IPV(buf, version, ip); // ip MUST be of type ipAddr_t or ip4Addr_t
OUTBUF_APPEND_IPVX
// Alternatively:
uint32_t val32 = 0x04030201;
(buf, val32); // val MUST be an IPv4 address (uint32_t) in network order
OUTBUF_APPEND_U32
uint8_t ip6[16] = {};
(buf, ip6, sizeof(ip6)); // ip6 MUST be an IPv6 address (uint8_t array of size 16) in network order
OUTBUF_APPEND
// Special cases: append IPv4 address 0.0.0.0 or IPv6 address ::
(buf);
OUTBUF_APPEND_IP4_ZERO(buf); OUTBUF_APPEND_IP6_ZERO
Timestamp, duration
uint64_t sec = 0;
uint32_t usec = 0;
(buf, sec, usec);
OUTBUF_APPEND_TIME
// Seconds only
(buf, sec);
OUTBUF_APPEND_TIME_SEC
// Special case: append timestamp 0
(buf); OUTBUF_APPEND_TIME_ZERO
Characters, strings and string classes (strings without quotes)
char *str = "string";
(buf, str);
OUTBUF_APPEND_STR
// If there is no string to append:
(buf);
OUTBUF_APPEND_STR_EMPTY//OUTBUF_APPEND_STR(buf, ""); // Alternative
// If the string needs to be free'd:
char *str = strdup("string");
(buf, str); // Also works if str is NULL OUTBUF_APPEND_STR_AND_FREE
If the type of the column varies
#if VAL_UINT8 == 1
uint8_t val = 0;
#else // VAL_UINT8 == 0
uint16_t val = 0;
#endif // VAL_UINT8 == 0
(buf, val, sizeof(val)); OUTBUF_APPEND
Repetitive values
For repetitive values, use one of the OUTBUF_APPEND_ARRAY_*()
macros
uint32_t size = 3; // size MUST be a uint32_t variable
uint8_t array[size] = {};
(buf , array, size);
OUTBUF_APPEND_ARRAY_U8
uint16_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_U16
uint32_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_U32
uint64_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_U64
int8_t array[size] = {};
(buf , array, size);
OUTBUF_APPEND_ARRAY_I8
int16_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_I16
int32_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_I32
int64_t array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_I64
float array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_FLT
double array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_DBL
uint8_t array[size][ETH_ALEN] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_MAC
[size] = {}; // or ipAddr_t array[size];
ip4Addr_t array(buf, array, size);
OUTBUF_APPEND_ARRAY_IP4
[size] = {};
ipAddr_t array(buf, array, size);
OUTBUF_APPEND_ARRAY_IP6
[size] = {};
ipVAddr_t array(buf, array, size);
OUTBUF_APPEND_ARRAY_IPV
[size] = {};
ipAddr_t arrayconst uint_fast8_t version = FLOW_IPVER(flowP);
(buf, version, array, size);
OUTBUF_APPEND_ARRAY_IPVX
char *array[size] = {};
(buf, array, size);
OUTBUF_APPEND_ARRAY_STR
char *array[size] = {};
[0] = strdup("string1");
array[1] = strdup("string2");
array(buf, array, size); OUTBUF_APPEND_ARRAY_STR_AND_FREE
Optional strings (repetitive with 0 or 1 repetition)
char *str = "andy";
(buf, str);
OUTBUF_APPEND_OPT_STR//=> OUTBUF_APPEND_NUMREP(buf, ONE);
//=> OUTBUF_APPEND_STR(buf, str);
char *str = "";
(buf, str);
OUTBUF_APPEND_OPT_STR//=> OUTBUF_APPEND_NUMREP(buf, ZERO);
More complex cases of repetitive values
For more complex cases of repetitive values, start by adding the number of repetitions, then add the values in a for loop
uint32_t size = 5;
(buf, size);
OUTBUF_APPEND_NUMREP//OUTBUF_APPEND_U32(buf, size); // Alternative
for (uint_fast32_t i = 0; i < size; i++) {
// Append repetition 'i'
//OUTBUF_APPEND...(buf, array[i]);
}
Special cases: 0 or 1 repetition
(buf);
OUTBUF_APPEND_NUMREP_ZERO
(buf);
OUTBUF_APPEND_NUMREP_ONE...(buf, ...) // append the actual value OUTBUF_APPEND
Adding support for environment/runtime configuration flags
In your header file, e.g.,
myPlugin.h
:/* ========================================================================== */ /* ------------------------ USER CONFIGURATION FLAGS ------------------------ */ /* ========================================================================== */ ... /* +++++++++++++++++++++ ENV / RUNTIME - conf Variables +++++++++++++++++++++ */ #define MYPLUGIN_FLAG_UINT 1 // Description #define MYPLUGIN_FLAG_INT -1 // Description #define MYPLUGIN_FLAG_STR "text" // Description /* ========================================================================== */ /* ------------------------- DO NOT EDIT BELOW HERE ------------------------- */ /* ========================================================================== */ // The elements in the enum below MUST be in the same order as above enum { , // Name of the configurable flag, preceded with ENV_ ENV_MYPLUGIN_FLAG_UINT, ENV_MYPLUGIN_FLAG_INT, ENV_MYPLUGIN_FLAG_STR// Sentinel to be used as number of environment/runtime config flags ENV_MYPLUGIN_N };
In your source file, e.g.,
myPlugin.c
:Case 1: declare global/local variables:
#if ENVCNTRL > 0 [ENV_MYPLUGIN_N] = {}; // => !! This array MUST be called env for the macros to work t2_env_t env(PLUGIN_SRCH, ENV_MYPLUGIN_N, env); t2_get_envconst uint32_t flag_uint = T2_ENV_VAL_UINT(MYPLUGIN_FLAG_UINT): const int32_t flag_int = T2_ENV_VAL_INT(MYPLUGIN_FLAG_UINT): const char * const flag_str = T2_ENV_VAL(MYPLUGIN_FLAG_STR); // => !! flag_str will be free'd when t2_free_env() is called // To preserve the value of flag_str after the call to t2_free_env() use one of the next two instructions: //char * const flag_str = T2_STEAL_ENV_VAL(MYPLUGIN_FLAG_STR); // => !! flag_str MUST be free'd with free() //char * const flag_str = T2_DUP_ENV_VAL(MYPLUGIN_FLAG_STR); // => !! flag_str MUST be free'd with free() #else // ENVCNTRL == 0 const uint32_t flag_uint = MYPLUGIN_FLAG_UINT: const int32_t flag_int = MYPLUGIN_FLAG_UINT: const char * const flag_str = MYPLUGIN_FLAG_STR; #endif // ENVCNTRL ... // Use flag_uint, flag_int and flag_str as normal variables ... #if ENVCNTRL > 0 // This releases the memory used by env variables // => flag_str will NOT point to a valid address after this call!! (ENV_MYPLUGIN_N, env); t2_free_env#endif
Case 2: use the
env
array to access variables:[ENV_MYPLUGIN_N] = {}; // => !! This array MUST be called env for the macros to work t2_env_t env #if ENVCNTRL > 0 (PLUGIN_SRCH, ENV_MYPLUGIN_N, env); t2_get_env#else // ENVCNTRL == 0 (MYPLUGIN_FLAG_UINT); T2_SET_ENV_NUM(MYPLUGIN_FLAG_INT); T2_SET_ENV_NUM(MYPLUGIN_FLAG_STR); T2_SET_ENV_STR#endif // ENVCNTRL ... // Access the variables with the same macros as before: // - T2_ENV_VAL_UINT(MYPLUGIN_FLAG_UINT) // - T2_ENV_VAL_INT(MYPLUGIN_FLAG_INT) // - T2_ENV_VAL(MYPLUGIN_FLAG_STR) ... #if ENVCNTRL > 0 // This releases the memory used by env variables (ENV_MYPLUGIN_N, env); t2_free_env#endif // ENVCNTRL > 0
Using T2 file manager
Opening a file:
const char * const filename = "/path/to/file.txt"; *file_object = file_manager_open(t2_file_manager, filename, "w"); file_object_t
Writing a single character:
(t2_file_manager, file_object, '\n'); file_manager_fputc // Alternative: FILE * const file = file_manager_fp(t2_file_manager, file_object); ('\n', file); fputc
Writing text:
(t2_file_manager, file_object, "Hello Andy!\n"); file_manager_fputs // Alternative: FILE * const file = file_manager_fp(t2_file_manager, file_object); ("Hello Andy!\n", file); fputs
Writing formatted text:
(t2_file_manager, file_object, "Hello %s!\n", "Andy"); file_manager_fprintf // Alternative: FILE * const file = file_manager_fp(t2_file_manager, file_object); (file, "Hello %s!\n", "Andy"); fprintf
Writing binary:
const uint8_t * const ptr = packet->l7HdrP; const uint16_t snaplen = packet->snapL7Len; (t2_file_manager, file_object, ptr, snaplen, 1); file_manager_fwrite // Alternative: FILE * const file = file_manager_fp(t2_file_manager, file_object); (ptr, snaplen, 1, file); fwrite
Getting the filename:
const char * const path = file_object_get_path(file_object);
Closing the file:
(t2_file_manager, file_object); file_manager_close
Parsing data with t2buf
Include the t2buf header
#include "t2buf.h"
Let your build system know about
t2buf.c
:meson.build
:...= [ src 'src', plugin_name + '.c'), join_paths('..', '..', 'utils', 't2buf.c'), join_paths( ] ...
CMakeLists.txt
:... add_library(${_plugin_name} MODULE ${_plugin_name}.c src/ ../../utils/t2buf.c )...
src/Makefile.am
:...libyourPluginName_la_SOURCES = \ yourPluginName.c \ ../../../utils/t2buf.c ...
Initialization in t2OnLayer2() or t2OnLayer4()
const uint16_t snaplen = packet->snapL7Len;
const uint8_t * const l7Hdr = packet->l7HdrP;
= t2buf_create(l7Hdr, snaplen); t2buf_t t2buf
Navigating the buffer
while (t2buf_left(&t2buf) > 0) { /* Read some data */ }
// Get the current position in the buffer
const long pos = t2buf_tell(&t2buf);
// Get a pointer to the current position in the buffer
uint8_t *ptr = t2buf_ptr(&t2buf);
const long offset = 5;
// Seek from the start of the buffer
if (!t2buf_seek(&t2buf, offset, SEEK_SET)) return; // => new position = 5
// Seek from the current position in the buffer
if (!t2buf_seek(&t2buf, offset, SEEK_CUR)) return; // => new position = pos + 5
// Seek from the end of the buffer
if (!t2buf_seek(&t2buf, -offset, SEEK_END)) return; // => new position = snaplen - 5
// Seek to the start of the buffer
if (!t2buf_rewind(&t2buf)) return; // => new position = 0
// Seek to the start of the buffer (alternative)
if (!t2buf_seek(&t2buf, 0, SEEK_SET)) return; // => new position = 0
Reading data
uint8_t u8;
uint16_t u16;
uint32_t u32;
uint64_t u64;
// Big endian (network order)
if (!t2buf_read_u8( &t2buf, &u8)) return;
if (!t2buf_read_u16(&t2buf, &u16)) return;
if (!t2buf_read_u24(&t2buf, &u32)) return;
if (!t2buf_read_u32(&t2buf, &u32)) return;
if (!t2buf_read_u48(&t2buf, &u64)) return;
if (!t2buf_read_u64(&t2buf, &u64)) return;
uint8_t ip[16];
if (!t2buf_read_n(&t2buf, ip, sizeof(ip))) return;
// Little endian
if (!t2buf_read_le_u8( &t2buf, &u8)) return;
if (!t2buf_read_le_u16(&t2buf, &u16)) return;
if (!t2buf_read_le_u24(&t2buf, &u32)) return;
if (!t2buf_read_le_u32(&t2buf, &u32)) return;
if (!t2buf_read_le_u48(&t2buf, &u64)) return;
if (!t2buf_read_le_u64(&t2buf, &u64)) return;
Peeking into data (reading without consuming any bytes)
uint8_t u8;
uint16_t u16;
uint32_t u32;
uint64_t u64;
// Big endian (network order)
if (!t2buf_peek_u8( &t2buf, &u8)) return;
if (!t2buf_peek_u16(&t2buf, &u16)) return;
if (!t2buf_peek_u24(&t2buf, &u32)) return;
if (!t2buf_peek_u32(&t2buf, &u32)) return;
if (!t2buf_peek_u48(&t2buf, &u64)) return;
if (!t2buf_peek_u64(&t2buf, &u64)) return;
// Little endian
if (!t2buf_peek_le_u16(&t2buf, &u16)) return;
if (!t2buf_peek_le_u24(&t2buf, &u32)) return;
if (!t2buf_peek_le_u32(&t2buf, &u32)) return;
if (!t2buf_peek_le_u48(&t2buf, &u64)) return;
if (!t2buf_peek_le_u64(&t2buf, &u64)) return;
String encodings
const t2buf_encoding encoding = T2BUF_ASCII;
const t2buf_encoding encoding = T2BUF_ANSI;
const t2buf_encoding encoding = T2BUF_UTF8;
const t2buf_encoding encoding = T2BUF_UTF16;
const t2buf_encoding encoding = T2BUF_UTF16_LE;
Read strings
#define MYDST_SIZE 5 // includes NULL terminator
uint8_t dst[MYDST_SIZE] = {};
const bool ignore_errors = false; // continue reading on encoding errors
// (invalid characters are replaced with '.')
const long read = t2buf_readstr(&t2buf, dst, MYDST_SIZE, encoding, ignore_errors);
//const long read = t2buf_readnstr(&t2buf, dst, MYDST_SIZE, (size_t)((MYDST_SIZE-1) / 2), T2BUF_UTF16, true);
if (read == T2BUF_EMPTY) return; // Nothing was read
else if (read == T2BUF_DST_FULL) return; // Destination buffer was not big enough
else if (read == T2BUF_ENC_ERROR) exit(1); // Provided encoding parameter not supported or unknown
Hexdecode bytes
const size_t src_bytes = 4; // Number of source bytes to read
//char hex[2 * src_bytes + 1];
//char sep = 0; // No separator
char hex[3 * src_bytes];
char sep = ':';
const size_t read = t2buf_hexdecode(&t2buf, src_bytes, hex, sep); // 01:02:03:04
if (read != src_bytes) return; // Could not read enough bytes => snapped
Read entire lines
#define MYLINELEN 64 // includes NULL terminator
uint8_t line[MYLINELEN];
//bool trim = false; // Store the line return in line
bool trim = true; // Do not store the line return in line
const long read = t2buf_readline(&t2buf, line, MYLINELEN, trim);
if (read == T2BUF_EMPTY) return; // Nothing was read
else if (read == T2BUF_DST_FULL) return; // Destination buffer was not big enough
else if (read == T2BUF_NULL) return; // NULL character was encountered before end of line
Skipping data
if (!t2buf_skip_u8( &t2buf)) return;
if (!t2buf_skip_u16(&t2buf)) return;
if (!t2buf_skip_u24(&t2buf)) return;
if (!t2buf_skip_u32(&t2buf)) return;
if (!t2buf_skip_u48(&t2buf)) return;
if (!t2buf_skip_u64(&t2buf)) return;
const size_t n = 32;
if (!t2buf_skip_n(&t2buf, n)) return;
// Skip strings
const long skipped = t2buf_skipstr(&t2buf, encoding); // 'skipped' bytes were skipped
if (skipped == T2BUF_EMPTY) return; // Nothing was read
else if (skipped == T2BUF_ENC_ERROR) exit(1); // Provided encoding parameter not supported or unknown
const long skipped = t2buf_skipnstr(&t2buf, skipped_max, encoding); // 'skipped' bytes were skipped (<= skipped_max)
if (skipped == T2BUF_EMPTY) return; // Nothing was read
else if (skipped == T2BUF_ENC_ERROR) exit(1); // Provided encoding parameter not supported or unknown
// Skip entire lines
const long skipped = t2buf_skipline(&t2buf); // 'skipped' bytes were skipped
if (skipped == T2BUF_EMPTY) return; // Nothing was skipped
const size_t skipped_max = 5;
const long skipped = t2buf_skipnline(&t2buf, skipped_max); // 'skipped' bytes were skipped (<= skipped_max)
if (skipped == T2BUF_EMPTY) return; // Nothing was skipped
Computing lengths
const long len = t2buf_strlen(&t2buf, T2BUF_UTF8); // NULL character is included in 'len'
const long len = t2buf_linelen(&t2buf); // 'len' includes terminating character(s)
if (len == T2BUF_EMPTY) return; // Nothing was read
Searching for patterns
const void *needle = "ANTEATER";
const size_t needle_len = strlen(needle);
const bool found = t2buf_memmem(&t2buf, needle, needle_len); // search for 'needle' of 'needle_len' in the buffer
if (found) {
// t2buf now points to the start of needle
}
Structures
flow_t
The flow_t
structure is defined in tranalyzer2/src/flow.h as follows:
typedef struct flow_s {
// Pointers to the next and previous flow in LRU list
struct flow_s * lruNextFlow;
struct flow_s * lruPrevFlow;
uint64_t findex; // flow index
uint64_t status; // status of flow, e.g., fragmentation processing
unsigned long flowIndex;
unsigned long oppositeFlowIndex;
#if (SCTP_ACTIVATE > 0 && SCTP_STATFINDEX == 1)
unsigned long sctpFindex;
#endif // (SCTP_ACTIVATE > 0 && SCTP_STATFINDEX == 1)
struct timeval lastSeen; // last time we've seen this flow
struct timeval firstSeen; // first time we've seen this flow
struct timeval duration; // lastSeen - firstSeen
// (NOT available before flow completely terminated)
/* --------------------------------------------------------------------- */
/* Begin flow identification */
/* --------------------------------------------------------------------- */
#if IPV6_ACTIVATE > 0
;
ipAddr_t srcIP;
ipAddr_t dstIP#else // IPV6_ACTIVATE == 0
;
ip4Addr_t srcIP;
ip4Addr_t dstIP#endif // IPV6_ACTIVATE == 0
#if ETH_ACTIVATE > 0
;
ethDS_t ethDS#endif
#if (IPV6_ACTIVATE == 2 || ETH_ACTIVATE > 0 || LAPD_ACTIVATE == 1)
uint16_t ethType;
#endif
uint16_t vlanId;
union {
struct {
uint16_t srcPort;
uint16_t dstPort;
};
uint32_t fragID;
};
#if SCTP_ACTIVATE & 2
uint32_t sctpVtag;
#endif // SCTP_ACTIVATE & 2
#if SCTP_ACTIVATE & 1
uint16_t sctpStrm;
#endif // SCTP_ACTIVATE & 1
uint8_t l4Proto;
/* --------------------------------------------------------------------- */
/* End flow identification */
/* --------------------------------------------------------------------- */
float timeout; // timeout of this flow in seconds
uint32_t lastIPID; // for duplicate IP ID detection
#if SUBNET_INIT != 0
uint32_t subnetNrSrc;
uint32_t subnetNrDst;
#endif // SUBNET_INIT != 0
// for fragPend hash cleanup
#if IPV6_ACTIVATE > 0
uint32_t lastFragIPID;
#else // IPV6_ACTIVATE == 0
uint16_t lastFragIPID;
#endif // IPV6_ACTIVATE == 0
} __attribute__((packed)) flow_t;
packet_t
The packet_t
structure is defined in tranalyzer2/src/packet.h as follows:
typedef struct {
#if (FDURLIMIT > 0 && FDLSFINDEX == 1)
uint64_t findex;
#endif //(FDURLIMIT > 0 && FDLSFINDEX == 1)
uint64_t status;
/* --------------------------------------------------------------------- */
/* Pointers (network order) */
/* --------------------------------------------------------------------- */
const struct pcap_pkthdr * const pcapHdrP;
const uint8_t * const raw_packet; // Beginning of the packet
const uint8_t * const end_packet; // Beyond the end of the packet
// (raw_packet + snapLen);
const uint8_t * l2HdrP; // Layer 2 (Ethernet, LLC, LAPD, ...)
const uint8_t * l3HdrP; // Layer 3 (IPv4, IPv6, ...)
const uint8_t * l4HdrP; // Layer 4 (TCP, UDP, ICMP, IGMP, SCTP, ...)
const uint8_t * l7HdrP; // Layer 7 (payload)
const etherLLCHeader_t* etherLLC; // Ethernet header LLC part if present (set but not used)
const pppHu_t * pppHdrP; // PPP header
const pppoEH_t * pppoeHdrP; // PPPoE header
const uint32_t * mplsHdrP; // MPLS pointer
const uint32_t * vlanHdrP; // VLAN pointer
// GRE headers
const greHeader_t * greHdrP; // GRE v1,2 header
const uint8_t * greL3HdrP; // L3 header before GRE header
// GTP headers
const uint8_t * gtpHdrP; // GTP v0,1,2 header (set but no used)
// L2TP headers
const uint16_t * l2tpHdrP; // L2TPv2 header
const uint8_t * l2tpL3HdrP; // L3 header before L2TP header
// IPv6 headers
const ip6OptHdr_t * ip6HHOptHdrP; // IPv6 Hop-by-Hop Option header
const ip6OptHdr_t * ip6DOptHdrP; // IPv6 Destination Option header
const ip6FragHdr_t * ip6FragHdrP; // IPv6 Fragment header
const ip6RouteHdr_t * ip6RouteHdrP; // IPv6 Routing header (set but not used)
// Teredo headers
const uint8_t * trdoOIHdrP; // Teredo Origin Indication header
const uint8_t * trdoAHdrP; // Teredo Authentication header
#if SCTP_ACTIVATE > 0
const uint8_t * l7SctpHdrP; // First SCTP payload
#endif // SCTP_ACTIVATE > 0
/* --------------------------------------------------------------------- */
/* Lengths (host order) */
/* - Snap lengths can be truncated due to limited snaplength */
/* (derived by header dissection) */
/* --------------------------------------------------------------------- */
const uint32_t rawLen; // extracted from pcapHdrP
uint32_t l2Len; // derived from IP header length field + length of L2 header (set but not used)
uint32_t len; // derived from IP header length field, defined by PACKETLENGTH in packetCapture.h:
// 0: Including L2-4 header,
// 1: including L3-4 header,
// 2: Including L4 header,
// 3: Only payload L7
// Snap lengths (can be truncated due to limited snaplength, derived by header dissection)
const uint32_t snapLen; // extracted from pcapHdrP
uint32_t snapL2Len; // includes L2 header
uint32_t snapL3Len; // includes L3 header
uint16_t snapL4Len; // includes L4 header
uint16_t snapL7Len; // only higher packet payload (L7)
uint16_t l7Len; // L7 length
#if SCTP_ACTIVATE > 0
uint16_t snapSctpL7Len; // only higher packet payload (L7)
#endif // SCTP_ACTIVATE > 0
uint16_t l2HdrLen; // set but not used
uint16_t l3HdrLen;
uint16_t l4HdrLen;
/* --------------------------------------------------------------------- */
/* Flow identification (IP addresses, ports and protocols (host order), */
/* headers count and description, ... */
/* --------------------------------------------------------------------- */
#if IPV6_ACTIVATE > 0
;
ipAddr_t srcIP;
ipAddr_t dstIP#else // IPV6_ACTIVATE == 0
;
ip4Addr_t srcIP;
ip4Addr_t dstIP#endif // IPV6_ACTIVATE == 0
#if ((SUBNET_INIT != 0) || (AGGREGATIONFLAG & (SUBNET | SRCIP | DSTIP)))
#if IPV6_ACTIVATE > 0
;
ipAddr_t srcIPC;
ipAddr_t dstIPC#else // IPV6_ACTIVATE == 0
;
ip4Addr_t srcIPC;
ip4Addr_t dstIPC#endif // IPV6_ACTIVATE == 0
uint32_t subnetNrSrc;
uint32_t subnetNrDst;
uint16_t srcPortC;
uint16_t dstPortC;
uint8_t l4ProtoC;
#endif // ((SUBNET_INIT != 0) || (AGGREGATIONFLAG & (SUBNET | SRCIP | DSTIP)))
uint16_t vlanId;
uint16_t srcPort;
uint16_t dstPort;
uint16_t ethType;
uint16_t outerEthType;
uint16_t l3Proto; // IPv4, IPv6, L2TPv2/3, ...
// Headers description, e.g., eth:ipv4:tcp
#if T2_PRI_HDRDESC == 1
uint16_t numHdrDesc; // Number of headers description
uint16_t hdrDescPos; // Headers description position
char hdrDesc[T2_HDRDESC_LEN]; // Headers description
#endif // T2_PRI_HDRDESC == 1
uint8_t l4Proto; // TCP, UDP, ICMP, IGMP, ...
uint8_t mplsHdrCnt;
uint8_t vlanHdrCnt;
#if SCTP_ACTIVATE > 0
uint8_t sctpPad; // SCTP padding of content
#endif // SCTP_ACTIVATE > 0
} packet_t;
MAC addresses
The ethDS_t
structure is defined in tranalyzer2/src/networkHeaders.h as follows:
typedef struct {
uint8_t ether_dhost[ETH_ALEN]; // destination eth addr
uint8_t ether_shost[ETH_ALEN]; // source ether addr
} __attribute__((packed)) ethDS_t;
IP addresses
The ipAddr_t
, ipVAddr_t
and ip4Addr_t
, structures are defined in tranalyzer2/src/ipaddr.h as follows:
typedef union {
uint32_t IPv4x[4];
struct in_addr IPv4; // IPv4 address
struct in6_addr IPv6; // IPv6 address
uint64_t IPv6L[2]; // IPv6 address 2*64 bit max chunk for masking ops
} __attribute__((packed)) ipAddr_t;
typedef struct {
uint8_t ver; // version
;
ipAddr_t addr} ipVAddr_t;
typedef union {
uint32_t IPv4x[1];
struct in_addr IPv4;
} __attribute__((packed)) ip4Addr_t;
ethernetHeader_t
The ethernetHeader_t
structure is defined in tranalyzer2/src/networkHeaders.h as follows:
typedef struct {
;
ethDS_t ethDSuint16_t ether_type; // packet type ID field or length
uint16_t data;
} __attribute__((packed)) ethernetHeader_t;
ipHeader_t
The ipHeader_t
structure is defined in tranalyzer2/src/proto/ipv4.h as follows:
#define IP_V(ip) (((ip)->ip_vhl & 0xf0) >> 4) // Version
#define IP_HL(ip) ((ip)->ip_vhl & 0x0f) // Header length
...
typedef struct {
uint8_t ip_vhl; // version, header length
uint8_t ip_tos; // type of service
uint16_t ip_len; // total length
uint16_t ip_id; // identification
uint16_t ip_off; // fragment offset field
uint8_t ip_ttl; // time to live
uint8_t ip_p; // protocol
uint16_t ip_sum; // checksum
struct in_addr ip_src; // source address
struct in_addr ip_dst; // destination address
} __attribute__((packed)) ipHeader_t;
ip6Header_t
The ip6Header_t
structure is defined in tranalyzer2/src/proto/ipv6.h as follows:
typedef struct {
uint32_t vtc_flw_lbl; // first word: ver, tcl, flow
uint16_t payload_len; // payload length
uint8_t next_header; // next protocol
union {
uint8_t hop_limit; // hop limit
uint8_t ip_ttl; // TTL
};
; // source address
ipAddr_t ip_src; // destination address
ipAddr_t ip_dst} __attribute__((packed)) ip6Header_t;
tcpHeader_t
The tcpHeader_t
structure is defined in tranalyzer2/src/proto/tcp.h as follows:
typedef struct {
uint16_t source;
uint16_t dest;
uint32_t seq;
uint32_t ack_seq;
uint8_t res1:4;
uint8_t doff:4;
uint8_t flags;
uint16_t window;
uint16_t check;
uint16_t urg_ptr;
} __attribute__((packed)) tcpHeader_t;
udpHeader_t
The udpHeader_t
structure is defined in tranalyzer2/src/proto/udp.h as follows:
typedef struct {
uint16_t source;
uint16_t dest;
uint16_t len;
uint16_t check;
} __attribute__((packed)) udpHeader_t;
icmpHeader_t
The icmpHeader_t
structure is defined in tranalyzer2/src/proto/icmp.h as follows:
typedef struct {
uint8_t type; // message type
uint8_t code; // type sub-code
uint16_t checksum;
union {
// echo datagram
struct {
uint16_t id;
uint16_t sequence;
} echo;
// gateway address
uint32_t gateway;
// path mtu discovery
struct {
uint16_t unused;
uint16_t mtu;
} frag;
};
} __attribute__((packed)) icmpHeader_t;
sctpHeader_t
The sctpHeader_t
structure is defined in tranalyzer2/src/sctp.h as follows:
typedef struct {
uint16_t source; // Source port
uint16_t dest; // Destination port
uint32_t verTag; // Verification tag
uint32_t chkSum; // Checksum
uint32_t data;
} __attribute__((packed)) sctpHeader_t;
Reporting errors, warnings and other information
char *msg = "text";
The following macros output to stdout or to the
_log.txt
file ift2 -l
option is used:("pluginName", "Your message is %s", msg); // Message (black) : pluginName: Your message is text T2_PLOG("pluginName", "Your message is %s", msg); // Info (blue) : [INF] pluginName: Your message is text T2_PINF( "pluginName", "Your message is %s", msg); // OK (green) : [OK] pluginName: Your message is text T2_POK("pluginName", "Your message is %s", msg); // Warning (orange): [WRN] pluginName: Your message is text T2_PWRN // The following macro always output to stderr ("pluginName", "Your message is %s", msg); // Error (red) : [ERR] pluginName: Your message is text T2_PERR
If you do not want to output the [INF], [OK], [WRN] or [ERR] prefix:
("pluginName", "Your message is %s", msg); // Info (blue) : pluginName: Your message is text T2_PINF_NP( "pluginName", "Your message is %s", msg); // OK (green) : pluginName: Your message is text T2_POK_NP("pluginName", "Your message is %s", msg); // Warning (orange): pluginName: Your message is text T2_PWRN_NP // The following macro always output to stderr ("pluginName", "Your message is %s", msg); // Error (red) : pluginName: Your message is text T2_PERR_NP
Alternatively, specify the output stream:
FILE *stream = stdout; (stream, "pluginName", "Your message is %s", msg); // Message (black) : pluginName: Your message is text T2_FPLOG(stream, "pluginName", "Your message is %s", msg); // Info (blue) : [INF] pluginName: Your message is text T2_FPINF( stream, "pluginName", "Your message is %s", msg); // OK (green) : [OK] pluginName: Your message is text T2_FPOK(stream, "pluginName", "Your message is %s", msg); // Warning (orange): [WRN] pluginName: Your message is text T2_FPWRN(stream, "pluginName", "Your message is %s", msg); // Error (red) : [ERR] pluginName: Your message is text T2_FPERR
If you do not want to output the [INF], [OK], [WRN] or [ERR] prefix:
(stream, "pluginName", "Your message is %s", msg); // Info (blue) : pluginName: Your message is text T2_FPINF_NP( stream, "pluginName", "Your message is %s", msg); // OK (green) : pluginName: Your message is text T2_FPOK_NP(stream, "pluginName", "Your message is %s", msg); // Warning (orange): pluginName: Your message is text T2_FPWRN_NP(stream, "pluginName", "Your message is %s", msg); // Error (red) : pluginName: Your message is text T2_FPERR_NP
Reporting aggregated bitfields
pluginName.h:
... typedef struct { uint8_t hex8; uint16_t hex16; uint32_t hex32; uint64_t hex64; } pluginNameFlow_t *pnFlows; ...
pluginName.c:
... // Global variables for aggregated bitfields uint8_t myProtHex8; uint16_t myProtHex16; uint32_t myProtHex32; uint64_t myProtHex64; ... void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf) { *pnFlowP = &pnFlows[flowIndex]; pluginNameFlow_t ... // Aggregate flow bitfields into global bitfields |= pnFlowP->hex8; myProtHex8 |= pnFlowP->hex16; myProtHex16 |= pnFlowP->hex32; myProtHex32 |= pnFlowP->hex64; myProtHex64 } void t2PluginReport(FILE *stream) { // The T2_FPLOG_AGGR_HEX() macro use sizeof() to determine the size of the variable to print (stream, plugin_name, myProtHex8); // pluginName: Aggregated myProtHex8=0xff T2_FPLOG_AGGR_HEX(stream, plugin_name, myProtHex16); // pluginName: Aggregated myProtHex16=0xffff T2_FPLOG_AGGR_HEX(stream, plugin_name, myProtHex32); // pluginName: Aggregated myProtHex32=0xffffffff T2_FPLOG_AGGR_HEX(stream, plugin_name, myProtHex64); // pluginName: Aggregated myProtHex64=0xffffffffffffffff T2_FPLOG_AGGR_HEX // If the value is known to be different from 0, use T2_FPLOG_AGGR_HEX0() to omit the test for val > 0 if (myProtHex8) T2_FPLOG_AGGR_HEX0(stream, plugin_name, myProtHex8); // pluginName: Aggregated myProtHex8=0xff if (myProtHex16) T2_FPLOG_AGGR_HEX0(stream, plugin_name, myProtHex16); // pluginName: Aggregated myProtHex16=0xffff if (myProtHex32) T2_FPLOG_AGGR_HEX0(stream, plugin_name, myProtHex32); // pluginName: Aggregated myProtHex32=0xffffffff if (myProtHex64) T2_FPLOG_AGGR_HEX0(stream, plugin_name, myProtHex64); // pluginName: Aggregated myProtHex64=0xffffffffffffffff // Alternatively, if you want to enforce a specific size ( stream, plugin_name, myProtHex8); // pluginName: Aggregated myProtHex8=0xff T2_FPLOG_AGGR_H8(stream, plugin_name, myProtHex16); // pluginName: Aggregated myProtHex16=0xffff T2_FPLOG_AGGR_H16(stream, plugin_name, myProtHex32); // pluginName: Aggregated myProtHex32=0xffffffff T2_FPLOG_AGGR_H32(stream, plugin_name, myProtHex64); // pluginName: Aggregated myProtHex64=0xffffffffffffffff T2_FPLOG_AGGR_H64}
Reporting hexadecimal numbers
T2 has a
HEX_CAPITAL
flag to output hex numbers in upper or lower case (default).In order to respect this flag, use the following
B2T_PRIX{8,16,32,64}
macros:uint8_t hex8 = 0x12; ("Aggregated hex8=0x%02" B2T_PRIX8, hex8); T2_LOG uint16_t hex16 = 0x1234; ("Aggregated hex16=0x%04" B2T_PRIX16, hex16); T2_LOG uint32_t hex32 = 0x12345678; ("Aggregated hex32=0x%08" B2T_PRIX32, hex32); T2_LOG uint64_t hex64 = 0x0123456789abcdef; ("Aggregated hex64=0x%016" B2T_PRIX64, hex64); T2_LOG
Reporting numbers
Simplest way to report a number (converted to human readable if necessary):
uint64_t num = 12345678; ("pluginName", "Your number is", num); // pluginName: Your number is 12345678 (12.35M) T2_PLOG_NUM // Alternatively, specify the output stream: FILE *stream = stdout; (stream, "pluginName", "Your number is", num); // pluginName: Your number is 12345678 (12.35M) T2_FPLOG_NUM
Report the percentage as well:
uint64_t num = 12345678; uint64_t tot = 987654321; ("pluginName", "Your number is", num, tot); // pluginName: Your number is 12345678 (12.35M) [1.25%] T2_PLOG_NUMP // Alternatively, specify the output stream: FILE *stream = stdout; (stream, "pluginName", "Your number is", num, tot); // pluginName: Your number is 12345678 (12.35M) [1.25%] T2_FPLOG_NUMP
For more flexibility, convert numbers to human readable manually:
uint64_t num = 12345678; char str[64]; (num, str); T2_CONV_NUM("%" PRIu64 "%s\n", num, str); // 12345678 (12.35 M) printf (num, str, "b/s"); T2_CONV_NUM_SFX("%" PRIu64 "%s\n", num, str); // 12345678 (12.35 Mb/s) printf (num, str, sizeof(str), "b/s"); t2_conv_readable_num("%" PRIu64 "%s\n", num, str); // 12345678 (12.35 Mb/s) printf
Reporting MAC addresses
Convert a MAC address to string. Note that the actual representation depends on the value of
MAC_FORMAT
andMAC_SEP
in utils/bin2txt.h (see table below).uint8_t mac[ETH_ALEN] = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06 }; char dest[T2_MAC_STRLEN+1]; (mac, dest, sizeof(dest)); t2_mac_to_str
To force a specific representation use one of the functions below:
Function MAC_FORMAT
Output t2_mac_to_mac(mac, dest, sizeof(dest))
0 01:02:03:04:05:06 t2_mac_to_hex(mac, dest, sizeof(dest))
1 0x0000010203040506 t2_mac_to_uint(mac, dest, sizeof(dest))
2 1108152157446
Reporting IP addresses
Convert an IPv4 address to string. Note that the actual representation depends on the value of
IP4_FORMAT
in utils/bin2txt.h (see table below).struct in_addr ipv4 = {}; char dest[INET_ADDRSTRLEN]; (ipv4, dest, sizeof(dest)); t2_ipv4_to_str
To force a specific representation use one of the functions below:
Function IP4_FORMAT
Output t2_ipv4_to_compressed(ipv4, dest, sizeof(dest))
0 1.2.3.4 t2_ipv4_to_uncompressed(ipv4, dest, sizeof(dest))
1 001.002.003.004 t2_ipv4_to_hex(ipv4, dest, sizeof(dest))
2 0x01020304 t2_ipv4_to_uint(ipv4, dest, sizeof(dest))
3 16909060 Convert an IPv6 address to string. Note that the actual representation depends on the value of
IP6_FORMAT
in utils/bin2txt.h (see table below).struct in6_addr ipv6 = {}; char dest[INET6_ADDRSTRLEN]; (ipv6, dest, sizeof(dest)); t2_ipv6_to_str
To force a specific representation use one of the functions below:
Function IP6_FORMAT
Output t2_ipv6_to_compressed(ipv6, dest, sizeof(dest))
0 2001:db8::1 t2_ipv6_to_uncompressed(ipv6, dest, sizeof(dest))
1 2001:0db8:0000:0000:0000:0000:0000:0001 t2_ipv6_to_hex128(ipv6, dest, sizeof(dest))
2 0x20010db8000000000000000000000001 t2_ipv6_to_hex64_hex64(ipv6, dest, sizeof(dest))
3 0x20010db800000000_0x0000000000000001 IPv4 or IPv6:
char dest[INET6_ADDRSTRLEN]; const uint_fast8_t version = PACKET_IPVER(packet); (packet->srcIP, version, dest, sizeof(dest)); T2_IP_TO_STR if (PACKET_IS_IPV6(packet)) { (packet->srcIP, dest, sizeof(dest)); // for IPv4 T2_IPV4_TO_STR} else if (PACKET_IS_IPV6(packet)) { (packet->srcIP, dest, sizeof(dest)); // for IPv6 T2_IPV6_TO_STR}
Reporting subnet information
// Tranalyzer2 already looked up source and destination address for each packet
const uint32_t num = packet->subnetNrSrc;
//const uint32_t num = packet->subnetNrDst;
const uint_fast8_t ipver = PACKET_IPVER(packet);
// To lookup other addresses:
uint32_t netNum;
= packet->srcIP;
ipAddr_t ip (netNum, ip, ipver);
SUBNET_TEST_IP//SUBNET_TEST_IP4(netNum, ip); // for IPv4
//SUBNET_TEST_IP6(netNum, ip); // for IPv6
// Autonomous System Number
uint32_t asn;
(asn, ipver, num);
SUBNET_ASN//SUBNET4_ASN(asn, num); // for IPv4
//SUBNET6_ASN(asn, num); // for IPv6
// Country
char *country;
(country, ipver, num);
SUBNET_LOC//SUBNET4_LOC(country, num); // for IPv4
//SUBNET6_LOC(country, num); // for IPv6
// County
char *county;
(county, ipver, num);
SUBNET_CNTY//SUBNET_CNTY(county, num); // for IPv4
//SUBNET_CNTY(county, num); // for IPv6
// City
char *city;
(city, ipver, num);
SUBNET_CTY//SUBNET4_CTY(city, num); // for IPv4
//SUBNET6_CTY(city, num); // for IPv6
// Organization
char *org;
(org, ipver, num);
SUBNET_ORG//SUBNET4_ORG(org, num); // for IPv4
//SUBNET6_ORG(org, num); // for IPv6
// Hexadecimal code
uint32_t netID;
(netID, ipver, num);
SUBNET_NETID//SUBNET4_NETID(netID, num); // for IPv4
//SUBNET6_NETID(netID, num); // for IPv6
// Latitude, longitude and precision
float lat, lng, prec;
(dest, ipver, num); // Latitude
SUBNET_LAT(dest, ipver, num); // Longitude
SUBNET_LNG(dest, ipver, num); // Precision
SUBNET_PREC//SUBNET4_LAT(dest, num); // Latitude for IPv4
//SUBNET4_LNG(dest, num); // Longitude for IPv4
//SUBNET4_PREC(dest, num) // Precision for IPv4
//SUBNET6_LAT(dest, num); // Latitude for IPv6
//SUBNET6_LNG(dest, num); // Longitude for IPv6
//SUBNET6_PREC(dest, num) // Precision for IPv6
Working with strings
const char * const str = "Andy was here";
if (t2_str_has_prefix(str, "Andy")) {
// 'str' starts with 'Andy'
}
if (t2_str_has_suffix(str, "here")) {
// 'str' ends with 'here'
}
char dest[9];
(dest, str, sizeof(dest), T2_STRCPY_EXIT); // abort if dest too small
t2_strcpy(dest, str, sizeof(dest), T2_STRCPY_EMPTY); // dest will be empty if dest too small: dest[0] = '\0'
t2_strcpy(dest, str, sizeof(dest), T2_STRCPY_TRUNC); // truncate string if dest too small: dest = "Andy was"
t2_strcpy(dest, str, sizeof(dest), T2_STRCPY_ELLIPSIS); // truncate and ellipsize string if dest too small: dest = "Andy ..."
t2_strcpy
const char * const andy = "Andy";
const char * const was_here = "was here";
char *andy_was_here = t2_strdup_printf("%s %s", andy, was_here);
("%s\n", andy_was_here); // -> Andy was here
printf(andy_was_here); free
Miscellaneous macros
Reporting alarms (alarm mode)
The following snippet of code can be implemented in any callback
receiving the flowIndex
, i.e., t2OnNewFlow()
, t2OnLayer2()
, t2OnLayer4()
and/or t2OnFlowTerminate()
.
void t2OnFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf) {
* const flowP = &flows[flowIndex];
flow_t *mypluginFlowP = &mypluginFlows[flowIndex];
mypluginFlow_t
#if ALARM_MODE == 1
(flowP, FL_ALARM); // Set the alarm bit in flowStat
T2_SET_STATUS(mypluginFlowP->alarms); // Report alarm(s) to the core
T2_REPORT_ALARMS#endif // ALARM_MODE == 1
...
}
For more details, refer to the Implementing the alarm mode capability tutorial.
Forcing termination of a flow (force mode)
*flowP = &flows[flowIndex];
flow_t (flowP); T2_RM_FLOW
For more details, refer to the Implementing the force mode capability tutorial.
Handy C snippets
Creating (and deleting) folder for saving data
- src/myPlugin.h
#define MYPLUGIN_SAVE 0 // Save data
#define MYPLUGIN_RMDIR 1 // Empty MYPLUGIN_PATH before starting
#define MYPLUGIN_PATH "/tmp/myPluginData/" // Path for extracted files
- src/myPlugin.c
#if MYPLUGIN_SAVE == 1
(MYPLUGIN_PATH, MYPLUGIN_RMDIR);
T2_MKPATH#endif // MYPLUGIN_SAVE == 1
Loading a file from the plugin folder
FILE *file = t2_fopen_in_dir(pluginFolder, "myFile.txt", "r");
if (UNLIKELY(!file)) exit(EXIT_FAILURE);
size_t len;
ssize_t read;
char *line = NULL;
while ((read = getline(&line, &len, file)) != -1) {
if (line[0] == '#') continue; // skip comments
if (line[read-1] == '\n') line[--read] = '\0';
// ...
}
(line);
free(file); fclose
Saving data in a new file
FILE *file = t2_fopen_with_suffix(baseFileName, "_mySuffix.txt", "w");
if (UNLIKELY(!file)) exit(EXIT_FAILURE);
("Write something in the file...\n", file);
fputs(file, "Number of %s packets: %" PRIu64 " packets\n", plugin_name, numPackets);
fprintf(file); fclose
Searching for a needle in a haystack
const uint8_t * const payload = packet->l7HdrP;
const uint16_t snaplen = packet->snapL7Len;
const char *needle = "Andy";
char *andy = memmem(payload, snaplen, needle, strlen(needle));
if (andy) {
// Andy was here
} else {
// where is Andy? Not in the payload...
}
Working with Perl Compatible Regular Expressions (PCRE)
meson.build:
...= dependency('libpcre') pcre_dep = [ deps pcre_dep, ] ... shared_module(plugin_name, sources: src, dependencies: deps, include_directories: inc,'so', name_suffix: )
myplugin.c:
#include <pcre.h> const uint8_t * const payload = packet->l7HdrP; const uint16_t snaplen = packet->snapL7Len; const char *error; int erroffset; int ovecsize = 30; int ovector[ovecsize]; // The regex to search for const char *code = "^[A-Z][a-z]+[0-9]+$"; // Initialization *regex = pcre_compile(code, 0, &error, &erroffset, NULL); pcre // Search const int num_matches = pcre_exec(regex, NULL, payload, snaplen, 0, 0, ovector, ovecsize); if (num_matches == PCRE_ERROR_NOMATCH) { // No match found } else if (num_matches < 0) { // An error occurred... ("%s\n", error); printf} else { // Results for (int i = 0; i < num_matches; i++) { const char *substring; (payload, ovector, num_matches, i, &substring); pcre_get_substring("Regex was matched: '%s'\n", substring); printf(substring); pcre_free_substring} } // Release memory (regex); pcre_free
Debugging
Compile in debug mode:
t2build -d myPlugin
Run
t2
in thegdb
(Linux) orlldb
(macOS) debugger:gt2 -r file.pcap
Set a breakpoint:
- On a specific line: b myFile.c:1234
- On a specific function: b myFunc
Set a conditional breakpoint:
- On a specific line: b myFile.c:1234 if numPackets == 3
- On a specific function: b myFunc if numPackets == 3
Add a condition to an existing breakpoint:
b myFile.c:1234
Breakpoint 1 at 0x5674
cond 1 numPackets == 3
b myFunc
Breakpoint 2 at 0x8535
cond 2 numPackets == 3
Start debugging:
r
Print the program trace (backtrace):
bt
Inspect the content of a variable:
- Default format: p myVar
- Hexadecimal: p /x myHexVar
Jump to the next program line:
n
Step into a function:
s
Continue running the program:
c
Quit
gdb
:q
Toggle test
List the flags to toggle in tests/myPlugin.flags:
cat $T2PLHOME/myPlugin/tests/myPlugin.flags
... ETH_ACTIVATE ../../tranalyzer2/src/networkHeaders.h ... MYBOOLEAN_FLAG src/myPlugin.h
Run
t2test
:t2test myPlugin
Or if you have a lot of flags, start with:
t2test myPlugin -J
which is not exhaustive, but faster.
Scripts
If you need to write supporting scripts, the t2utils.sh helper file provides easy access to Tranalyzer2 scripts and programs and help ensures portability for different OS.
Use it as follows (assuming your script resides in ${T2PLHOME}/myPlugin/scripts/
, otherwise you will need to adapt the path in the third line (source ...
)):
#!/usr/bin/env bash
source "$(dirname "${0}")/../../../scripts/t2utils.sh"
check_dependency gnuplot
usage() {
echo "Usage:"
echo " ${SNAME} [OPTION...] <FILE>"
echo
echo "Optional arguments:"
echo " -v, --verbose Enable verbose output"
echo
echo "Help and documentation arguments:"
echo " -h, --help Show this help, then exit"
}
while [[ "${#}" -gt 0 ]]; do
case "${1}" in
-v|--verbose)
VERBOSE=1
;;
-h|-\?|--help)
usage
exit 0
;;
*)
if [[ ! -f "${1}" ]]; then
abort_option_unknown "${1}"
fi
FILE="${1}"
;;
esac
shift
done
if [[ ! -f "${FILE}" ]]; then
abort_required_file
elif [[ ! -s "${FILE}" ]]; then
printerr "File size MUST be greater than zero"
abort_with_help
fi
Access Tranalyzer scripts and utilities
T2 alias/script/utility | T2 variable/function | Description | Example |
---|---|---|---|
t2 |
T2 |
tranalyzer executable | T2 -r file.pcap |
t2whois |
T2WHOIS |
query the anteaters subnet file | T2WHOIS 1.2.3.4 |
t2b2t |
T2B2T |
convert T2 binary files | T2B2T -l f_flows.bin |
tran |
"${T2HOME}" |
location of tranalyzer2 source files, e.g., /home/user/tranalyzer2-0.9.3/ |
cd "${T2PLHOME}" |
tranpl |
"${T2PLHOME}" |
location of the plugins source files, e.g., ${T2HOME}/plugins/ |
cd "${T2PLHOME}/basicFlow" |
"${SHOME}" |
location of the script, e.g., ${T2PLHOME}/myPlugin/scripts/ |
cd "${SHOME}" |
|
"${SNAME}" |
name of the script | echo "${SNAME}" |
|
t2build |
"${T2BUILD}" |
build plugins | "${T2BUILD}" basicFlow |
t2conf |
"${T2CONF}" |
configure plugins | "${T2CONF}" basicFlow -D BFO_VLAN=1 |
t2plot |
"${T2PLOT} |
2D/3D plot | "${T2PLOT}" file.txt |
tawk |
"${TAWK}" |
post-processing with Tawk | "${TAWK}" 'aggr($l4Proto)' f_flows.txt |
Function/variable wrappers for common utitilies
For maximum portability, make sure to replace your calls to awk
, sed
, … as follows:
Program | T2 variable/function | Example |
---|---|---|
awk |
AWK |
AWK '{ print $1 }' file.txt |
gawk |
AWK |
AWK '{ print $1 }' file.txt |
awk -v OFS='\t' -F '\t' |
AWKF |
AWKF '{ print $2 }' file.txt |
gawk -v OFS='\t' -F '\t' |
AWKF |
AWKF '{ print $2 }' file.txt |
sed |
"${SED}" |
"${SED}" 's/$/ Andy/' <<< "Hello" |
gsed |
"${SED}" |
"${SED}" 's/$/ Andy/' <<< "Hello" |
open (macOS) |
"${OPEN}" |
"${OPEN}" file.txt |
xdg-open |
"${OPEN}" |
"${OPEN}" file.txt |
readlink |
"${READLINK}" |
"${READLINK}" -f /tmp/../tmp/file.txt |
greadlink |
"${READLINK}" |
"${READLINK}" -f /tmp/../tmp/file.txt |
t2utils.sh: list of available functions
For usage details, refer to the well documented source code of t2utils.sh
in ${T2HOME}/scripts/t2utils.sh
:
Functions
printbold
printerr
printinf
printok
printwrn
printfbold
printferr
printfinf
printfok
printfwrn
fatal
check_dependency
check_dependency_linux
check_dependency_macos
test_min_version
has_define
get_define
set_define
ask_default_no
ask_default_yes
find_most_recent_dir
find_most_recent_file
t2_build_exec
get_t2_exec
get_t2b2t_exec
get_t2whois_exec
abort_if_t2_exec_not_found
abort_if_t2b2t_exec_not_found
abort_if_t2whois_exec_not_found
t2_wget
t2_wget_n
replace_suffix
get_nproc
arg_is_option
validate_next_arg
validate_next_arg_exists
validate_next_dir
validate_next_file
validate_next_file_or_dir
validate_next_pcap
validate_next_num
validate_next_int
validate_next_float
validate_float
validate_int
validate_num
validate_ip
validate_pcap
abort_missing_arg
abort_option_unknown
abort_required_file
abort_required_dir
abort_required_file_or_dir
abort_with_help
Scripts and programs
Note that all the variables are defined as readonly
.
Functions
AWK
AWKF
T2
T2B2T
T2WHOIS
Variables
${AWK_EXEC}
${OPEN}
${READLINK}
${SED}
${T2BUILD}
${T2CONF}
${T2PLOT}
${TAWK}
Folders
${SHOME}
${T2HOME}
${T2PLHOME}
Colors
${BLUE}
${GREEN}
${ORANGE}
${RED}
${BLUE_BOLD}
${GREEN_BOLD}
${ORANGE_BOLD}
${RED_BOLD}
${BLUE_ITALIC}
${GREEN_ITALIC}
${ORANGE_ITALIC}
${RED_ITALIC}
${BLUE_UNDERLINE}
${GREEN_UNDERLINE}
${ORANGE_UNDERLINE}
${RED_UNDERLINE}
${BOLD}
${ITALIC}
${STRIKETHROUGH}
${UNDERLINE}
${NOCOLOR}
OS detection
${IS_LINUX}
${IS_MACOS}
Variables
${SNAME}
Adding support for Bash/ZSH completion
Completions files for T2 scripts and utilities reside in ${T2HOME}/scripts/completions/{bash,zsh}/
and are named according to the name of the script/program prepended by an underscore, e.g., for myScript
: ${T2HOME}/scripts/completions/bash/_myScript
and ${T2HOME}/scripts/completions/zsh/_myScript
. Make sure to replace all occurences of myScript
with the name of your script.
Bash
Bash completions reside in ${T2HOME}/scripts/completions/bash/
.
# Bash completion for myScript
_myScript() {
local cur prev words cword
if type -t _init_completion &> /dev/null; then
_init_completion -n = || return
else
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
prev="${COMP_WORDS[COMP_CWORD-1]}"
fi
case "${prev}" in
# -f file
-f)
if type -t _filedir &> /dev/null; then
_filedir
else
compopt -o filenames 2> /dev/null
COMPREPLY=($(compgen -f -- "${cur}"))
fi
return
;;
# -d directory
-d)
if type -t _filedir &> /dev/null; then
_filedir -d
else
compopt -o dirnames 2> /dev/null
COMPREPLY=($(compgen -d -- "${cur}"))
fi
return
;;
esac
if [[ "${cur}" == -* ]]; then
COMPREPLY=($(compgen -W "$(_parse_help "${T2PLHOME}/myPlugin/scripts/myScript" -h)" -- "${cur}"))
# Or if you want more control:
#local opts=(
# -f --file
# -d --dir
# -v --verbose
# -\? -h --help
#)
#COMPREPLY=($(compgen -W "${opts[*]}" -- "${cur}"))
fi
}
complete -F _myScript myScript
ZSH
ZSH completions reside in ${T2HOME}/scripts/completions/zsh/
.
# ZSH completion for myScript
_myScript() {
#compdef myScript
local cur="${words[CURRENT]}"
[[ "${CURRENT}" -gt 2 ]] && local prev="${words[CURRENT-1]}"
case "${prev}" in
# -f file
-f)
_files
return
;;
# -d directory
-d)
_files -/
return
;;
esac
if [[ "${cur}" == -* ]]; then
_arguments --
# Or if you want more control:
#local opts=(
# -f --file
# -d --dir
# -v --verbose
# -\? -h --help
#)
#compadd "${opts[@]}"
fi
}
compdef _myScript myScript
Documentation
tawk -V
In order for tawk to know about your variables,
you need to create a file with the name of your variable in $T2HOME/scripts/tawk/vars/
.
The easiest is to copy one of the existing files, e.g.:
Bitfields:
bt_hex_8
$T2HOME/scripts/tawk/vars/arpStat
bt_hex_16
$T2HOME/scripts/tawk/vars/dhcpStat
bt_hex_32
$T2HOME/scripts/tawk/vars/tcpOptions
bt_hex_64
$T2HOME/scripts/tawk/vars/flowStat
Fixed values:
Hexadecimals $T2HOME/scripts/tawk/vars/vtpCode
Integers $T2HOME/scripts/tawk/vars/tftpErrC
Strings $T2HOME/scripts/tawk/vars/tftpOpcode
Workflow
tran
cd scripts/tawk/vars
cp arpStat myprotoStat
vi myprotoStat
#H bit | myprotoStat | Description
#H =============================================================================
0 | 0x01 | Myproto detected
#I 1 | 0x02 | This is an information (blue)
#W 2 | 0x04 | This is a warning (orange)
#E 3 | 0x08 | This is an error (red)
4 | 0x10 | Use a dash ("-") if a bit is not used
5 | 0x20 | -
6 | 0x40 | -
7 | 0x80 | -
tawk -V myprotoStat
The myprotoStat column is to be interpreted as follows: bit | myprotoStat | Description ============================================================================= 0 | 0x01 | Myproto detected 1 | 0x02 | This is an information (blue) 2 | 0x04 | This is a warning (orange) 3 | 0x08 | This is an error (red) 4 | 0x10 | Use a dash ("-") if a bit is not used 5 | 0x20 | - 6 | 0x40 | - 7 | 0x80 | -tawk -V myprotoStat=0x13
The myprotoStat column with value 0x13 is to be interpreted as follows: bit | myprotoStat | Description ============================================================================= 0 | 0x01 | Myproto detected 1 | 0x02 | This is an information (blue) 4 | 0x10 | Use a dash ("-") if a bit is not used