Tutorial: Plugin Sinks

If any of our sink plugins do not match your needs, you can either tweak them or develop your own. In this tutorial you will learn about the t2 functions supporting you in that task.

Preparation

You must have completed the plugin summary file and plugin dependencies tutorials because these prepares you for the comming material. If you are just interested in the sink part here is the necessary tcpWinSD.tar.gz.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$

If you use the tcpWin developed in the last two tutorials you need the basicFlow dependency.

$ t2build tranalyzer2 basicFlow tcpStates tcpGeoWin
...
$

The anonymized sample pcap can be downloaded here: annoloc2.pcap. Please extract it under your data folder: ~/data, if you haven’t already. Now you are all set for sink stuff.

Sink your own

If you like to send something via a socket just use the socketSink plugin, do not reinvent the wheel, or let yourself be inspired by it. Here we’d like to address a real case which is not covered by the flow buffer or our sink plugins.

Let’s build a customized summary sink which pushes info via a socket, like netflowSink, only simpler. So a customer tells you to produce from the summary file a periodic update, and send it to 127.0.0.1 on port 5555 via UDP.

I already prepared a tcpWinSink for you ready to be downloaded. So you can have it open in another bash window and follow the explanations below.

As sink plugins start with plugin numbers 9xx, I picked 950. We will also need a dependency: tcpGeoWin, which we already prepared in the previous tutorial plugingeolabel.

For meson you need to add the tcpGeoWIn in the include_directories function

$ tran
$ source scripts/t2_aliases
$ tcpWinSink
$ vi meson.build
...
inc = include_directories(
    join_paths('..', '..', 'utils'),
    join_paths('..', '..', 'tranalyzer2', 'src'),
    join_paths('..', 'tcpGeoWin', 'src'),
)
...

If you only want meson, you are good, but if you want fallback e.g. cmake, add the same to CMakeLists.txt

$ vi CMakeLists.txt

and for the generality of all maybe older systems autotools as an ultimate fallback, add it to the CFLAGS constant:

$ cd src
$ vi Makefile.am

Now the compiler is happy for all cases, it will always find your dependend plugin whatever version of OS.

Now open tcpWinSink.c and look for bufferToSink()

$ vi tcpWinSink.c

First we need to define the pointer to the window size counts tcpGeoWin produces (refer to the Plugin dependencies tutorial if you forgot). You have to add the tcpWin.h dependency and the socket also has to be defined.

As in the Plugin dependencies tutorial, we need a T2_PLUGIN_INIT_WITH_DEPS(...) macro, this time with tcpWin as a dependency.

The initialize() function mostly contains standard UDP/TCP socket code, and at the end, the initialization of the time marker. actTime denotes the actual pcap time of the core defined in global.h. So if you sniff from an interface, it will be the local time. The code is copied from the socketSink and adapted to our problem at hand.

Now we come to the important part of the job. As we are not using the flow buffer of T2, the bufferToSink(...) callback with the __attribute__((unused)) tag has to be appended to your tcpWinSink plugin. If you are using the tcpGeoWin I provided it does not contain any output buffer on flow terminated, as we are not using any other sink anyway. Now the compiler has no reason to complain about the fact that you do not use the buffer.

Implementing the bufferToSink(outputBuffer_t *buffer) callback

We like to print a winsize list reporting reporting every n seconds.

In bufferToSink(...) your code defines what the sink is doing. It consist normally of four main parts.

  1. A timer check when to send data. If not present, every flow terminating triggers your bufferToSink(...), that is inefficient. But you may try it without a timer.
  2. Data preparation and recoding section
  3. Buffer population as intended, so that the socket just needs to push it out.
  4. The end buffer part. You may reuse it in all your future applications, because it is back-pressure safe.
void bufferToSink(outputBuffer_t *buffer __attribute__((unused))) {
    // check core timer
    if (actTime.tv_sec - tmrk.tv_sec < TMINTVL || gwzP->wzi == 0) return; // timer expired and is something to print?
    tmrk = actTime; // reset timer

    // Data preparation part

    int i, sBufLen = 0;
    char srcIP[INET6_ADDRSTRLEN];

    char time[MAX_TM_BUF];
    strftime(time, MAX_TM_BUF, "%a %d %b %Y %X", localtime(&tmrk.tv_sec)); // make a nice time string

    // shovel output of tcpWin into sBuf

    sBufLen = snprintf(sBuf, MAXSBUF-sBufLen, "# %s\n# IP\tpktTcpCnt\twinRelThCnt\n", time); // send header
    for (i = 0; i < gwzP->wzi; i++) {
        T2_IP_TO_STR(gwzP->wzip[i].addr, gwzP->wzip[i].ver, srcIP, INET6_ADDRSTRLEN);                // transfer IP to string
        sBufLen += snprintf(&sBuf[sBufLen], MAXSBUF-sBufLen, "%s\t%"PRIu32"\t%f\n", srcIP, gwzP->tcpCnt[i], gwzP->wzCnt[i]); // push into sBuf
    }

    sBuf[sBufLen++] = '\n'; // final separation
    sBuf[sBufLen++] = '\0'; // just make sure that string is always properly terminated

    // Send sBuf via socket

    int32_t written = 0, act_written; // buffer positional indexes
    while (written < sBufLen) { // make sure that the socket sends the whole buffer regardless of flow control socket backpressure
#if SOCKTYPE == 1
        act_written = write(sfd, sBuf + written, sBufLen - written);
#else // SOCKTYPE == 0
        act_written = sendto(sfd, sBuf + written, sBufLen - written, 0, (struct sockaddr*)&server, sizeof(server));
#endif // SOCKTYPE
        if (UNLIKELY(act_written <= 0)) {
            T2_PERR("tcpWinSink", "Could not send message to socket: %s", strerror(errno));
            if (sfd) close(sfd);
            exit(-1);
        }
        written += act_written;
    }
}

Note that bufferToSink is called every time a flow timeouts. Therefore we either need to reduce our flow timeout, or add tcpStates, as we did above, assuring that TCP flows timeout immediately after a RST or FIN.

onApplicationTerminate(...) is easy, just close the socket handle. The if is good defensive programming, although we always exit in case of an error.

Now open tcpWinSink.h, there you already see the constants we need, the server address, port, the socket type, we choose a TCP socket for the start. The time interval is set to 5 seconds and the maximal buffer length is chosen large enough for simplicity, otherwise we need to malloc and stuff, which makes the tutorial unnecessary complicated.

So you can rewrite the buffer which is properly allocated according to the data to be sent as home work. Hint: It depends on the maximal number of entries in the gwz structure and the output format. You will only need a buffer pointer and malloc. And don’t forget to free the buffer at the end.

tcpWinSink.h holds now all necessary constants. I copied them from the socketSink, and adapted them to our task. The MAXBUF constant should suffice for our application. Nevertheless, if you intent to add more info into sBuf check whether you need to increase its size.

Now open a netcat TCP socket in another bash window: nc -l 127.0.0.1 -p 5555 Go back to your original window and invoke T2.

$ t2 -r ~/data/annoloc2.pcap -w ~/results
...

And you will see in the netcat window every five seconds a report

$ nc -l 127.0.0.1 -p 5555
# Thu 23 May 2002 18:35:05
# IP	pktTcpCnt	winRelThCnt
193.104.31.16	133	0.007519

# Thu 23 May 2002 18:35:10
# IP	pktTcpCnt	winRelThCnt
193.104.31.16	133	0.007519
19.112.1.129	94	0.063830

# Thu 23 May 2002 18:35:15
# IP	pktTcpCnt	winRelThCnt
193.104.31.16	133	0.007519
19.112.1.129	94	0.063830
138.212.191.84	92	0.010870
36.89.79.225	538	0.001859

# Thu 23 May 2002 18:35:20
# IP	pktTcpCnt	winRelThCnt
193.104.31.16	133	0.007519
19.112.1.129	94	0.063830
138.212.191.84	92	0.010870
36.89.79.225	538	0.001859
201.9.148.42	1332	0.000751
200.32.26.254	142	0.014085
83.42.68.176	62	0.016129

# Thu 23 May 2002 18:35:25
# IP	pktTcpCnt	winRelThCnt
193.104.31.16	133	0.007519
19.112.1.129	94	0.063830
138.212.191.84	92	0.010870
36.89.79.225	538	0.001859
201.9.148.42	1332	0.000751
200.32.26.254	250	0.016000
83.42.68.176	62	0.016129
19.123.222.7	72	0.013889
138.212.187.203	72	1.000000
193.87.239.57	365	0.016438
70.5.118.83	415	0.002410

$

Try t2 on your interface, but restart the netcat first.

$ st2 -i INTERFACE
[sudo] password for wurst:
...

Make sure you produce a lot of TCP traffic so that one connection produces back pressure situations or you have to lower TCPWIN_MINPKTS or increase TCPWIN_THRES in tcpGeoWin.h listed below. Use t2conf for that and don’t forget to t2build:

If you used the tcpGeoWin I provided, that is about it. Besides sending something over the socket you should see the summary report in the results directory from tcpGeoWin.

How about building a round robin list, where old IP’s are thrown out and new come in, so the amount of entries would be constant after some time and tcpWinSD would not just stop working when the maximum entries are reached.

Have fun writing plugins!

The next tutorial will teach you how to implement timeout handlers

See Also