Tutorial: Plugin Programming Cheatsheet

Contents

This cheatsheet summarises the most important callbacks, functions, macros and structures required to write your own plugins. For more details about the different aspects of writing your own plugins, refer to the t2PSkel plugin or to one of the tutorials.

Tutorials

Creating a New Plugin

Creating a New C Plugin

$ t2plugin -c pluginName

Creating a New C Plugin with a Specific Plugin Number

$ t2plugin -c pluginName -n 123

Creating a New C++ Plugin

$ t2plugin -c pluginName --cpp

Creating a New Rust Plugin

$ t2plugin -c pluginName --rust

Plugin Initialization

Without Any Dependencies to Other Plugins

With Dependency to One Plugin

With Dependency to Several Plugins

Callbacks

void initialize()

Initialization: allocate memory here [Tutorial]

binary_value_t* printHeader()

Describe your flow output here [Tutorial]

void onFlowGenerated(packet_t *packet, unsigned long flowIndex)

A new flow has been generated: reset its memory and try to determine if it is relevant for the plugin (protocol, ports, …) [Tutorial]

void claimLayer2Information(packet_t *packet, unsigned long flowIndex)

A new packet with a layer 2 (if it also has a layer 4, then flowIndex is equal to HASHTABLE_ENTRY_NOT_FOUND) [Tutorial]

void claimLayer4Information(packet_t *packet, unsigned long flowIndex)

A new packet with a layer 4 [Tutorial]

void onFlowTerminate(unsigned long flowIndex)

A flow has been terminated: finish computations for this flow and push your data to the output buffer [Tutorial]

void onApplicationTerminate()

Free the memory here [Tutorial]

void pluginReport(FILE *stream)

Contribute plugin specific information to the end report [Tutorial]

void monitoring(FILE *stream, uint8_t state)

Monitoring mode [Tutorial]

void saveState(FILE *stream)

Save the state of your plugin

void restoreState(const char *str)

Restore the state of your plugin

void bufferToSink(outputBuffer_t *buffer)

Dump the content of buffer to a file, stream, … (Sink plugins only) [Tutorial]

Accessing the Plugin Name

The T2_PLUGIN_INIT() and T2_PLUGIN_INIT_WITH_DEPS() macros both define the following static variable:

This variable being static, it can only be accessed in the pluginName.c file. However, nothing prevents you from redefining it in a different file! As an example, the plugin_name variable can be used with the macros defined here, e.g., T2_PLOG(plugin_name, "Andy was here!");

Accessing the Various Layers

Layer 2 Header

The layer 2 header (Ethernet) can be accessed as follows:

Layer 3 Header

The layer 3 header (IPv4, IPv6) can be accessed as follows:

Layer 4 Header

The layer 4 header (TCP, UDP, ICMP, SCTP, …) can be accessed as follows:

Layer 7 Header

The layer 7 header can be accessed as follows:

Selecting Flows by Port/Protocol

Selecting Flows by Port

Accessing the source/destination port:

Selecting Flows by Protocol

Describing Flow File Output in printHeader()

Bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)

Signed integers (8, 16, 32 and 64 bits)

Unsigned integers (8, 16, 32 and 64 bits)

Floating point (float, double)

MAC addresses

IPv4, IPv6 and IPvX (IP version and IP) addresses

Timestamp, duration

Characters, strings and string classes (strings without quotes)

If the type of the column varies

Compounds, e.g., field1_field2

Existing types

type description
bt_int_8 8-bit signed integer
bt_int_16 16-bit signed integer
bt_int_32 32-bit signed integer
bt_int_64 64-bit signed integer
bt_uint_8 8-bit unsigned integer
bt_uint_16 16-bit unsigned integer
bt_uint_32 32-bit unsigned integer
bt_uint_64 64-bit unsigned integer
bt_hex_8 8-bit hexadecimal number (unsigned integer)
bt_hex_16 16-bit hexadecimal number (unsigned integer)
bt_hex_32 32-bit hexadecimal number (unsigned integer)
bt_hex_64 64-bit hexadecimal number (unsigned integer)
bt_float Floating point number
bt_double Double precision floating point number
bt_char Single character
bt_string String
bt_string_class Unquoted string
bt_flow_direction A or B
bt_timestamp Date whose representation depends on B2T_TIMESTR in utils/bin2txt.h
bt_duration One uint64_t value for the seconds and one uint32_t value for the micro-/nano-seconds
bt_mac_addr MAC address
bt_ip4_addr IPv4 address
bt_ip6_addr IPv6 address
bt_ipx_addr IP address: version (8 bits), address (0, 4 or 16 bytes)

Generating Flow File Output in onFlowTerminate()

Unsigned integers, bitfields and hexadecimal numbers (8, 16, 32 and 64 bits)

Signed integers (8, 16, 32 and 64 bits)

Floating point (float, double)

MAC

MAC addresses structures.

IPv4, IPv6 and IPvX (IP version and IP) addresses

IP Addresses structures

Timestamp, duration

Characters, strings and string classes (strings without quotes)

If the type of the column varies

Repetitive values

For repetitive values, use one of the OUTBUF_APPEND_ARRAY_*() macros

Optional strings (repetitive with 0 or 1 repetition)

More complex cases of repetitive values

For more complex cases of repetitive values, start by adding the number of repetitions, then add the values in a for loop

Parsing Data with t2buf

Structures

flow_t

The flow_t structure is defined in tranalyzer2/src/flow.h as follows:

typedef struct flow_s {
    struct flow_s *lruNextFlow, *lruPrevFlow;   // pointers to the next and previous flow in LRU list

    struct timeval lastSeen;                    // last time we've seen this flow
    struct timeval firstSeen;                   // first time we've seen this flow
    struct timeval duration;                    // lastSeen - firstSeen
                                                // (NOT available before flow completely terminated)

    // -------------------------------------------------------------------------
    // Begin flow identification
    // -------------------------------------------------------------------------

#if IPV6_ACTIVATE > 0
    ipAddr_t srcIP, dstIP;
#else // IPV6_ACTIVATE == 0
    ip4Addr_t srcIP, dstIP;
#endif // IPV6_ACTIVATE == 0

#if ETH_ACTIVATE > 0
    ethDS_t ethDS;
#endif

#if (IPV6_ACTIVATE == 2 || ETH_ACTIVATE > 0)
    uint16_t ethType;
#endif

    uint16_t vlanID;

    union {
        struct {
            uint16_t srcPort;
            uint16_t dstPort;
        };
        uint32_t fragID;
    };

#if SCTP_ACTIVATE == 1
    uint16_t sctpStrm;
#endif

    uint8_t layer4Protocol;

    // -------------------------------------------------------------------------
    // End flow identification
    // -------------------------------------------------------------------------

    uint64_t findex; // flow index

#if (SCTP_ACTIVATE == 1 && SCTP_STATFINDEX == 1)
    unsigned long sctpFindex;
#endif

    // for fragPend hash cleanup
#if IPV6_ACTIVATE > 0
    uint32_t lastFragIPID;
#else // IPV6_ACTIVATE == 0
    uint16_t lastFragIPID;
#endif // IPV6_ACTIVATE == 0

    uint32_t lastIPID;                  // for duplicate IP ID detection

#if SUBNET_INIT != 0
    uint32_t subnetNrSrc, subnetNrDst;
#endif // SUBNET_INIT != 0

    uint64_t status;                    // status of flow, e.g., fragmentation processing

    unsigned long flowIndex;
    unsigned long oppositeFlowIndex;

    float timeout;                      // the timeout of this flow in seconds
} __attribute__((packed)) flow_t;

packet_t

The packet_t structure is defined in tranalyzer2/src/packet.h as follows:

// a common packet pointer, holds pointers to each layer's header

typedef struct {
#if T2_PRI_HDRDESC == 1
    char hdrDesc[T2_HDRDESC_LEN];                // header description, e.g., eth:ipv4:tcp
    uint16_t hdrDescPos;                         // header description position
    uint16_t numHdrDesc;                         // number of headers description
#endif // T2_PRI_HDRDESC == 1

    const u_char * const raw_packet;             // Pointer to the beginning of the packet
    const u_char * const end_packet;             // Pointer to the end of the packet

    const struct pcap_pkthdr * const pcapHeader; // Network order

    const l2Header_t *layer2Header;              // Network order
    const uint32_t *vlans;                       // Network order, ptr to vlans
    const etherLLCHeader_t *etherLLC;            // Ethernet header LLC part if present
    const uint32_t *mpls;                        // Network order, MPLS pointer

    const l3Header_t *layer3Header;
    const l4Header_t *layer4Header;

    const greHeader_t *greHdr;                   // Network order pointer to GRE v1,2 header
    const uint16_t *l2TPHdr;                     // Network order uint16 pointer to L2TPv2 header
    const l3Header_t *greLayer3Hdr;              // Network order
    const l3Header_t *l2tpLayer3Hdr;             // Network order

    const pppHu_t  *pppHdr;                      // Network order, pointer to PPP header
    const pppoEH_t *pppoEHdr;                    // Network order, pointer to PPPoE header

    const ip6OptHdr_t   *ip6HHOptHdr;
    const ip6OptHdr_t   *ip6DOptHdr;
    const ip6FragHdr_t  *ip6FragHdr;
    const ip6RouteHdr_t *ip6RouteHdr;

    const uint8_t *trdoOIHdr;
    const uint8_t *trdoAHdr;

    const uint8_t *layer7Header;     // pointer to payload

#if SCTP_ACTIVATE == 1
    const uint8_t *layer7SCTPHeader; // pointer to 1. SCTP payload
    uint16_t snapSCTPL7Length;       // Host order, only higher packet payload (layer 7),
                                     // can be truncated due to limited snaplength
#endif // SCTP_ACTIVATE == 1

    uint16_t l2HdrLen;               // Host order
    uint16_t l3HdrLen;               // Host order
    uint16_t l4HdrLen;               // Host order

    const uint32_t rawLength;        // Host order, extracted from pcapHeader
    const uint32_t snapLength;       // Host order, extracted from pcapHeader

    uint32_t snapL2Length;           // Host order, includes layer2 header, can be truncated due to limited snaplength, will be processed by header dissection
    uint32_t snapL3Length;           // Host order, includes layer3 header, can be truncated due to limited snaplength, derived by header dissection

    uint32_t packetL2Length;         // Host order, derived from IP header length field + length of L2 header
    uint32_t packetLength;           // Host order, derived from IP header length field, defined by PACKETLENGTH in packetCapture.h
                                     // (0: including L2-4 header, including L3-4 header, 2: including L4 header, 3: Only payload L7)
    uint16_t snapL4Length;           // Host order, includes layer4 header, can be truncated due to limited snaplength, derived by header dissection
    uint16_t snapL7Length;           // Host order, only higher packet payload (layer 7), can be truncated due to limited snaplength, derived by header dissection
    uint16_t packetL7Length;         // Layer7 Length

    uint16_t srcPort, dstPort;       // Host order

    uint16_t innerVLANID;

    uint16_t outerL2Type;            // Ethernet, ...
    uint16_t layer2Type;             // Ethernet, ...
    uint16_t layer3Type;             // IPv4, IPv6, ...

    uint64_t status;
#if (FDURLIMIT > 0 && FDLSFINDEX == 1)
    uint64_t findex;
#endif //(FDURLIMIT > 0 && FDLSFINDEX == 1)
#if IPV6_ACTIVATE > 0
    ipAddr_t srcIP;
    ipAddr_t dstIP;
#else // IPV6_ACTIVATE == 0
    ip4Addr_t srcIP;
    ip4Addr_t dstIP;
#endif // IPV6_ACTIVATE == 0

#if ((SUBNET_INIT != 0) || (AGGREGATIONFLAG & (SUBNET | SRCIP | DSTIP)))
#if IPV6_ACTIVATE > 0
    ipAddr_t srcIPC;
    ipAddr_t dstIPC;
#else // IPV6_ACTIVATE == 0
    ip4Addr_t srcIPC;
    ip4Addr_t dstIPC;
#endif // IPV6_ACTIVATE == 0
    uint32_t subnetNrSrc, subnetNrDst;
    uint16_t srcPortC, dstPortC;
    uint8_t layer4TypeC;
#endif // SUBNET_INIT != 0

    uint8_t layer4Type;              // TCP, UDP, ICMP, IGMP, ...
    uint8_t vlanHdrCnt;
    uint8_t mplsHdrCnt;
} packet_t;

MAC Addresses

The ethDS_t structure is defined in tranalyzer2/src/networkHeaders.h as follows:

IP Addresses

The ipAddr_t, ipVAddr_t and ip4Addr_t, structures are defined in tranalyzer2/src/networkHeaders.h as follows:

ethernetHeader_t

The ethernetHeader_t structure is defined in tranalyzer2/src/networkHeaders.h as follows:

ipHeader_t

The ipHeader_t structure is defined in tranalyzer2/src/networkHeaders.h as follows:

ip6Header_t

The ip6Header_t structure is defined in tranalyzer2/src/networkHeaders.h as follows:

tcpHeader_t

The tcpHeader_t structure is defined in tranalyzer2/src/proto/tcp.h as follows:

udpHeader_t

The udpHeader_t structure is defined in tranalyzer2/src/proto/udp.h as follows:

icmpHeader_t

The icmpHeader_t structure is defined in tranalyzer2/src/icmp.h as follows:

sctpHeader_t

The sctpHeader_t structure is defined in tranalyzer2/src/sctp.h as follows:

Reporting Errors, Warnings and Other Information

Reporting Aggregated Bitfields

Reporting Hexadecimal Numbers

Reporting Numbers

Reporting MAC Addresses

Reporting IP Addresses

Reporting Subnet Information

Working with Strings

Miscellaneous Macros

Reporting the Number of Alarms

[Tutorial]

Forcing Termination of a Flow

[Tutorial]

Handy C Snippets

Loading a File From the Plugin Folder

Saving Data in a New File

Searching for a Needle in a Haystack

Working with Perl Compatible Regular Expressions (PCRE)

Debugging

  1. Compile in debug mode:

    $ t2build -d myPlugin
  2. Run t2 in the gdb (Linux) or lldb (macOS) debugger:

    $ gt2 -r file.pcap
  3. Set a breakpoint:

    (gdb) b myFile.c:1234
    (gdb) b myFunc
  4. Set a conditional breakpoint:

    (gdb) b myFile.c:1234 if numPackets == 3
    (gdb) b myFunc if numPackets == 3

    Or if the breakpoint already exists:

    (gdb) b myFile.c:1234
    Breakpoint 1 at 0x5674
    (gdb) cond 1 numPackets == 3
    (gdb) b myFunc
    Breakpoint 2 at 0x8535
    (gdb) cond 2 numPackets == 3
  5. Start debugging:

    (gdb) r
  6. Print the program trace (backtrace):

    (gdb) bt
  7. Inspect the content of a variable:

    (gdb) p myVar
    (gdb) p /x myHexVar
  8. Jump to the next program line:

    (gdb) n
  9. Step into a function:

    (gdb) s
  10. Continue running the program:

    (gdb) c
  11. Quit gdb:

    (gdb) q

Toggle Test

  1. List the flags to toggle in tests/myPlugin.flags:

  2. Run t2test:

    $ t2test myPlugin

    Or if you have a lot of flags, start with:

    $ t2test myPlugin -J

    which is not exhaustive, but faster.

Documentation

tawk -V

In order for tawk to know about your variables, you need to create a file with the name of your variable in $T2HOME/scripts/tawk/vars/. The easiest is to copy one of the existing files, e.g.:

  • Bitfields:

    bt_hex_8 vrrpStat
    bt_hex_16 dhcpStat
    bt_hex_32 tcpOptions
    bt_hex_64 flowStat
    $ tran
    $ cd scripts/tawk/vars
    $ cp vrrpStat myprotoStat
    $ vi myprotoStat
    #H bit | myprotoStat | Description
    #H =============================================================================
         0 | 0x01        | Myproto detected
    #I   1 | 0x02        | This is an information (blue)
    #W   2 | 0x04        | This is a warning (orange)
    #E   3 | 0x08        | This is an error (red)
         4 | 0x10        | Use a dash ("-") if a bit is not used
         5 | 0x20        | -
         6 | 0x40        | -
         7 | 0x80        | -