Tutorial: Plugin Sinks

Contents

If any of our Sink plugins do not match your needs, you either tweak them or develop your own. In this tutorial you will learn about the t2 functions supporting you in that task.

Before we start doing this, clean out your plugin .so directory if this is the first tutorial you follow. Then all unnecessary plugins should be deleted from the plugin folder ./tranalyzer/plugins and compile basicFlow, basicStats and txtSink.

$ t2build -e
Are you sure you want to empty the plugin folder '/home/wurst/.tranalyzer/plugins' (y/N)? y
Plugin folder emptied
$ t2build basicFlow txtSink
...

Compiling the basicFlow took now a bit longer, because t2 had to rebuild the subnetfiles for geolocation. ‘t2build -e’ also removes the subnetfile. You can also use a rm, which does not remove the old subnetfile:

$ rm ~/.tranalyzer/plugins/*.so
$ t2build basicFlow txtSink

Then the compilation will be considerable faster, as the subnetfile already exists.

If you didn’t read the tutorials before, here is the basis plugin which we will extend: tcpWinSink

The annonymized sample pcap being used in the basic tutorial can be downloaded here: annoloc2.pcap. Please extract it under your data folder, if you not already have. Now you are all set for sink stuff.

Sink your own

If you like to send send something via a socket just use socketSink, do not reinvent the wheel, or let yourselve inspire by it. Here we like to address a real custom case which is not covered by the flow buffer or our sink plugins.

Let’s build a customized summary sink which pushes info via a socket, like netflowSink, only simpler. So a customer tells you to produce from the summary file a periodic update, being sent to 127.0.0.1 on port 5555 via udp.

I already prepared a tcpWinSinkTemplate for you ready to be downloaded. You only need to add the code, which we will do below.

As sinkPlugins start with plugin numbers 9xx, I picked 950. We will also need a dependency: tcpWin, which we already prepared in the previous tutorials.

$ tran
$ cd tcpWinSink
$ cat Makefile.am
lib_LTLIBRARIES = libtcpWinSink.la
  
libtcpWin_la_SOURCES = tcpWinSink.c tcpWinSink.h

libtcpWin_la_CFLAGS = -I../../tcpWin/src

Now open tcpWinSink.c and look for bufferToSink

$ cd src
$ vi tcpWinSink.c

First we need to define the pointer to the window size counts tcpWin produces. Then the socket has to be defined

As in the plugindependency tutorial we need a T2_PLUGIN_INIT_WITH_DEPS macro, this time with tcpWin as dependency.

The initialize mostly contains standard UDP/TCP socket code, and at the end the intialization of the time marker. actTime denotes the actual pcap time of the core defined in global.h. So if you sniff from an interface, it will be the local time.

Now we come to the important part of the job. As we are not using the flowbuffer of T2, the bufferToSink callback with the attribute((unused) has to be appended to your tcpWinSink plugin. Now the compiler has nothing to complain. Now the compiler has no reason to complain about the fact that you do not use the buffer.

In bufferToSink your code defines what the sink is doing. It consist normally of four main parts.

  1. A timer check when to send data. If not present, every flow terminating triggers your bufferToSink.
  2. Data preparation and recoding section
  3. Buffer population as intended, so that the socket just needs to push it out.
  4. The end buffer part. You may reuse it in all your future applications, because it is backpressure safe.
void bufferToSink(outputBuffer_t *buffer __attribute__((unused))) {
//	check core timer
        if (actTime.tv_sec - tmrk.tv_sec < TMINTVL || gwzP->wzi == 0) return; // timer expired and is something to print?
        tmrk = actTime; // reset timer

//	Data preparation part

        int i, sBufLen = 0;
        char srcIP[INET6_ADDRSTRLEN];

        char time[MAX_TM_BUF];
        strftime(time, MAX_TM_BUF, "%a %d %b %Y %X", localtime(&tmrk.tv_sec)); // make a nice time string

//      shovel output of tcpWin into sBuf

        sBufLen = snprintf(sBuf, MAXBHBUF-sBufLen, "# %s\n# IP\tpktTcpCnt\twinRelThCnt\n", time); // send header
        for (i = 0; i < gwzP->wzi; i++) {
                T2_IP_TO_STR(gwzP->wzip[i], 4, srcIP, INET6_ADDRSTRLEN);                // transfer IP to string
                sBufLen += snprintf(&sBuf[sBufLen], MAXBHBUF-sBufLen, "%s\t%"PRIu32"\t%f\n", srcIP, gwzP->tcpCnt[i], gwzP->wzCnt[i]); // push into sBuf 
        }
        sBuf[sBufLen++] = '\n'; // final separation
        sBuf[sBufLen++] = '\0'; // just make sure that string is always properly terminate


//      Send sBuf via socket

	int32_t written = 0, act_written; // buffer positional indexes
        while (written < sBufLen) { // make sure that the socket sends the whole buffer regardless of flow control socket backpressure
#if SOCKTYPE == 1
                act_written = write(sfd, sBuf + written, sBufLen - written);
#else // SOCKTYPE == 0
                act_written = sendto(sfd, sBuf + written, sBufLen - written, 0, (struct sockaddr*)&server, sizeof(server));
#endif // SOCKTYPE
                if (UNLIKELY(act_written <= 0)) {
                        T2_PERR("tcpWinSink", "Could not send message to socket: %s", strerror(errno));
                        if (sfd) close(sfd);
                        exit(-1);
                }
                written += act_written;
        }

}

onApplication is easy, just close the socket handle. The if is good defensive programming, although we always exit in case of an error.

Now open tcpWinSink.h, there you already see the constants we need, the server address, port, the socket type, we choose a tcp socket for the start. The time interval is set to 5 seconds. and the maximal buffer length large enough for simplicity. Actually the buffer should be allocated according to the data to be sent. I leave that to you as programming practise. Hint: It depends on the maximal number of in gwz structure and the output format. And don’t forget to free the buffer.

now open a netcat tcp socket in another bash window: nc -l 127.0.0.1 -p 5555 Go back to your original window and invoke T2

$ t2 -r ~/data/annoloc2.pcap -w ~/results/
...

And you will see in the netcat window 4 reports

$ nc -l 127.0.0.1 -p 5555 
# Thu 23 May 2002 18:35:11
# IP	pktTcpCnt	winRelThCnt
36.89.79.225	538	0.001859

# Thu 23 May 2002 18:35:16
# IP	pktTcpCnt	winRelThCnt
36.89.79.225	538	0.001859
201.9.148.42	1332	0.000751

# Thu 23 May 2002 18:35:21
# IP	pktTcpCnt	winRelThCnt
36.89.79.225	538	0.001859
201.9.148.42	1332	0.000751

# Thu 23 May 2002 18:35:26
# IP	pktTcpCnt	winRelThCnt
36.89.79.225	538	0.001859
201.9.148.42	1332	0.000751
193.87.239.57	365	0.016438
200.32.26.254	250	0.016000
70.5.118.83	415	0.002410

If you look into the annoloc2_tcpWin.txt file, you see more lines. Why?