PDF report generation from PCAP files or NetFlow records using t2fm

t2fm tawk clickhouse database data carving mongodb netflow nfdump pcap pdf postgresql

Introduction

This tutorial presents t2fm, a script which generates a PDF report out of a PCAP file or NetFlow records. Information provided in the report includes top source and destination addresses and ports, protocols and applications, DNS and HTTP activity and potential warnings, such as executable downloads or SSH connections.

Prerequisites

For this tutorial, it is assumed the user has a basic knowledge of Tranalyzer and that the file t2_aliases has been sourced in ~/.bashrc or ~/.bash_aliases as follows (Refer to the file README.md or to the documentation for more details):

# $HOME/.bashrc

if [ -f "$T2HOME/scripts/t2_aliases" ]; then
    . "$T2HOME/scripts/t2_aliases"             # Note the leading `.'
fi

Make sure to replace $T2HOME with the actual path, e.g., $HOME/tranalyzer2-0.9.3):

Required plugins

The following plugins must be loaded for t2fm to produce a useful report:

Optional plugins

The following plugins are optional:

If one of those plugin is not loaded, messages like N/A: dnsDecode plugin required will be displayed in the PDF where the information could not be accessed.

Packages

The following packages are required to build the PDF:

  • texlive-latex-extra
  • texlive-fonts-recommended

PCAP to PDF in one command

For simplicity, this tutorial assumes the user wants a complete report, i.e., requires all of the optional plugins.

The -b option builds and configures Tranalyzer and the plugins, the -A option opens the generated report.

t2fm -b -A -r file.pcap

Step-by-step instructions (PCAP to PDF)

Alternatively if you prefer to configure Tranalyzer, build the plugins and open the generated report yourself:

  1. Make sure all the plugins are configured as described in Section Prerequisites.

  2. Build Tranalyzer and the plugins (Hint: use the tab completion to avoid typing the full name of all the plugins: t2build tr<tab> ... ht<tab> ...):

    t2build tranalyzer2 basicFlow basicStats txtSink arpDecode dnsDecode httpSniffer nDPI portClassifier pwX sshDecode sslDecode

(Note that those first two steps can be omitted if t2fm -b option is used)

  1. Run t2fm directly on the PCAP file (the report will be named file.pdf):

    t2fm -r file.pcap

    (the output folder/prefix can be specified with the -w option: t2fm -r file.pcap -w ~/results/my_prefix)

  2. Open the generated PDF report file.pdf (Note that this step can be omitted if t2fm -A option is used:

    evince file.pdf

Step-by-step instructions (flow file to PDF)

Alternatively, if you prefer to run Tranalyzer yourself or already have access to a flow file, replace step 3 of the PCAP to PDF section with the following steps:

  • Run Tranalyzer on a pcap file as follows:

    t2 -r file.pcap -w out

  • The previous command should have created the following files:

    ls 

    
out_headers.txt     out_flows.txt

  • Run the t2fm script on the flow file previously generated previously:

    t2fm -F out_flows.txt

Step-by-step instructions (ClickHouse / MongoDB / PostgreSQL to PDF)

If one of the clickhouseSink, mongoSink or psqlSink plugins were loaded, t2fm can use the created databases to generate the report (faster).

  • Follow point 1 and 2 from the PCAP to PDF section

    (Note that HTTP_SAVE_* do not need to be set as EXE downloads detection is currently not implemented in the DB backends).

  • Build one the clickhouseSink, mongoSink or psqlSink plugin:

  • Run Tranalyzer on a pcap file as follows:

    t2 -r file.pcap -w out

  • Run the t2fm script on the database generated previously:

When generating a report from a database a time range to query can be specified with the -T option. The complete format is as follows: YYYY-MM-DD HH:MM:SS.USEC([+-]OFFSET|Z), e.g., 2018-10-01 12:34:56.912345+0100. Note that only the required fields must be specified, e.g., 2018-09-01 is equivalent to 2018-09-01 00:00:00.000000. For example, to generate a report from the 1st of September to the 11. of October 2018 at 14:59 from a PostgreSQL database, run the following command: t2fm -p tranalyzer -T "2018-09-01" "2018-10-11 14:59"

NetFlow records to PDF in one command

For simplicity, this tutorial assumes the user wants a complete report.

The -A option opens the generated report.

t2fm -A -N netflow/2024

Sample report

A sample report (IPs and passwords anonymized for privacy reasons) can be downloaded here.

Conclusion

This tutorial has presented how t2fm can be used to create a PDF report summarizing the traffic contained in a PCAP file or NetFlow records. Although not discussed in this tutorial, it is also possible to use t2fm on a live interface (-i option) or on a list of PCAP files (-R option). For more details, refer to t2fm man page or use t2fm --help.