Tawk Frequently Asked Questions (FAQ)

Can I use tawk with non Tranalyzer files?

Yes, refer to Using tawk With Non-Tranalyzer Files.

Can I use tawk functions with non Tranalyzer column names?

Yes, edit the my_vars file and load it using -i "$T2HOME/scripts/tawk/my_vars" option. Refer to Mapping External Column Names to Tranalyzer Column Names for more details.

Can I use tawk with files without column names?

Yes, but you won’t be able to use the functions which require a specific column, e.g., host().

The row listing the column names start with a ‘#’ instead of a ‘%’. . . Can I still use tawk?

Yes, use the -s option to specify the first character, e.g., tawk -s '#' 'program'

Can I process Bro/Zeek log files with tawk?

Yes, use the --zeek option.

Can I process a CSV (Comma Separated Value) file with tawk?

The simplest way to process CSV files is to use the --csv option. This sets the input and output separators to a comma and considers the first row to be the column names: tawk --csv 'program' file.csv.

Alternatively, the input field separator can be changed with the -F option and the output separator with -O ',' or -v OFS=','. Note that tawk expects the column names to be the first last row starting with a %. This can be changed with the -s option. To process a CSV file, run tawk as follows: tawk -F ',' -O ',' -s '' -N 1 'program' file.csv or tawk -F ',' -v OFS=',' -s '' -N 1 'program' file.csv

Can I produce a CSV (Comma Separated Value) file from tawk?

The output field separator (OFS) can be changed with the -O 'fs' or -v OFS='fs' option. To produce a CSV file, run tawk as follows: tawk -O ',' 'program' file.txt or tawk -v OFS=',' 'program' file.txt`

Can I write my tawk programs in a file instead of the command line?

Yes, copy the program (without the single quotes) in a file, e.g., prog.txt and run it as follows: tawk -f prog.txt file.txt

Can I still use column names if I pipe data into tawk?

Yes, you can specify a file containing the column names with the -I option as follows: cat file.txt | tawk -I colnames.txt 'program'

Can I use tawk if the row with the column names does not start with a special character?

Yes, you can specify the empty character with -s "". Refer to -s Option for more details.

I get a list of syntax errors from gawk… what is the problem?

The name of the columns is used to create variable names. If it contains forbidden characters, then an error similar to the following is reported:

gawk: /tmp/fileBndhdf:3: col-name = 3
gawk: /tmp/fileBndhdf:3:
^ syntax error

Although tawk will try to replace forbidden characters with underscore, the best practice is to use only alphanumeric characters (A-Z, a-z, 0-9) and underscore as column names. Note that a column name MUST NOT start with a number.

I get a function name previously defined error from gawk… What is the problem?

The name of the columns is used to create variable names. If a column is named after a tawk function or a builtin, then an error similar to the following is reported:

gawk: In file included from ah:21,
gawk:                  from /home/user/tranalyzer2/scripts/tawk/funcs/funcs.load:8,
gawk: proto:36: error: function name `proto' previously defined

In this case, you have two options. Either rename the column(s) in your file, e.g., proto -> l4Proto or use tawk -t option. With the -t option, Tawk tries to validate the column names by ensuring that no column names is equal to a function name and that all column names used in the program exist. Note that this verification process can be slow.

Tawk cannot find the column names… what is the problem?

First, make sure the comment char (-s option) is correctly set for your file (the default is "%"). Second, make sure the column names do not contain forbidden characters, i.e., use only alphanumeric and underscore and do not start with a number. If the row with column names is not the last one to start with the separator character, then specify the line number as follows: -N 2 or -s '#' -N 3. Refer to -s Option for more details.

Tawk reports errors similar to free(): double free detected in tcache 2

tawk uses gawk -M option to handle IPv6 addresses. For some reasons, this option is regularly affected by bugs… If you do not need IPv6 support, you can simply comment out line 653 in tawk:

OPTS=(
    #-M -v PREC=256         # <-- Add the leading sharp ('#') here
    -v __PRIHDR__=$PRIHDR
    -v __UNAME__="$(uname)"
)