ChangeLog for 0.9.2lmw2

• dnsDecode:
  • Extended decoding of NBNS names (discard padding, add suffix, …)
• mongoSink:
  • Fixed double escaping of quotes and double quotes
• voipDetector:
  • Renamed voipconv to t2voipconv
  • t2voipconv: added support for AMR, AMR-NB, AMR-WB, G.723.1, G.726, G.726le and GSM formats
  • Various fixes and improvements
• t2utils.[ch]:
  • New functions:
    • t2_strncpy_escape(), t2_strcpy_escape()
• tawk:
  • Fixed header printing when accumulating something other than flows
  • Added support for Termshark (-k option)
  • shark:
    • Added support for SDP
    • Extended support for RTP and SIP
  • New functions:
    • ientropy: compute the information entropy of each column, filter out columns with low entropy
    • isset: return true if a value is set, i.e., not empty
    • log2: compute the binary logarithm (log base 2) of a number
    • quote: add leading and trailing quotes to a string, escape quotes within string
    • printinf, printok, printwrn: print text in blue, green or orange
  • Improved documentation
  • Various fixes and improvements
• t2fm:
  • Added bottom N statistics
  • Added --top/--bottom options to only compute top or bottom stats
  • Do not count query names from responses when reporting top DNS queries
  • Various fixes and improvements
• t2utils.sh:
  • Added validate_next_file_or_dir function
  • Added {BLUE_,GREEN_,ORANGE_,RED_,}{ITALIC,UNDERLINE} and STRIKETHROUGH variables
  • Various fixes and improvements
• t2_aliases:
  • New t2voipconv alias
• New script:
  • t2voipconv: convert and manipulate raw VoIP files extracted from voipDetector

• tranalyzer2:
  • Added support for DPDK
  • Added support for DTLS dissection
• clickhouseSink
  • Improved documentation
• findexer
  • Added support for pcap with nanosecond precision
  • Various fixes and improvements
• mongoSink
  • Improved handling of timestamps with nanosecond precision
  • Improved documentation
• ospfDecode
  • Added missing column names in output files
  • Make sure each row in each file always have the same number of columns
  • Renamed column seq# to SeqNum
  • Bugfixes and improvements
• pcapd
  • Bugfixes and improvements
• psqlSink:
  • Improved handling of timestamps with nanosecond precision
  • Improved documentation
• sslDecode
  • Updated SSL blacklist
• tcpFlags
  • Improved performance
• voipDetector
  • Bugfixes and improvements
• t2fm
  • Added information about top known and unknown JA3 and JA3S fingerprints
  • Added information about top known and unknown JA4 and JA4S fingerprints
  • Added information about top blacklisted certificates
  • Replaced -C/--color option with --{chart,table-{odd,even}}-color
  • Replaced -c/--clickhouse option with -C/--clickhouse
  • Various fixes and improvements
• t2py
  • Added support for t2 -m option:
    • T2Utils.run_tranalyzer(…, monitoring_file=True)
    • t2 = T2(…, monitoring_file=True)
    • t2.monitoring_file()
    • t2.monitoring()
    • t2.print_monitoring()
    • t2.run(…, monitoring_file=True)
• t2utils.sh
  • Added ${IS_LINUX} and ${IS_MACOS} variables
  • Simplified find_most_recent_file function
  • Renamed check_dependency_osx to check_dependency_macos
  • Various fixes and improvements
• tawk
  • Added -b/--both-directions option to extract A and B flows (-x/-k options)
  • Added support for more custom defined columns (srcMac, ethType, …)
  • Various fixes and improvements
• scripts:
  • t2timeline: added -d option
• New script:
  • t2dpdk: run N instances of Tranalyzer in DPDK multi-process mode

• tranalyzer2:
  • Added LIVEBUFSIZE define to set libpcap internal buffer size on live captures
  • Added T2_USEC_PREC and T2_PRI_USEC macros
  • Added sensor ID to monitoring machine report
  • Added support for DTLS 1.2
  • Added -S/--snaplen and -B/--rx-bufsize command line options
  • Added -P/--priority option to set process priority (renice)
  • Added -M/--mon-interval option to set monitoring interval
  • Added -m/--monfile option to redirect monitoring output to _monitoring.txt
  • Added FLOW_IS_A() and FLOW_IS_B() macros
  • Extended support for Q-in-Q VLAN (ethertypes 0x9100 and 0x9200)
  • Reduced memory footprint of flow_t structure if FRAGMENTATION=0
  • Reduced list of L2/3 protocols to monitor (can be easily extended with MONPROTL[23])
  • Removed B2T_NANOSECS macro, used TSTAMP_PREC instead
  • Renamed ENABLE_IO_BUFFERING macro to IO_BUFFERING
• basicFlow:
  • Added MPLS information to packet mode
  • Added option to output MPLS labels as hexadecimal
  • Added BFO_VLAN=3 option to output decoded VLAN headers
  • Fixed nanoseconds representation in packet mode
• nDPI:
  • Updated nDPI library to version 4.8
• nFrstPkts:
  • Fixed nanoseconds representation for inter-arrival times
• pcapd:
  • Added PD_CHKSUM option to correct IPv4 checksum
• sslDecode:
  • Renamed SSL_PROTO_LIST to SSL_ALPN_LIST
  • Renamed sslProtoList and sslNumProto to sslALPNList and sslNumALPNList
  • Extract list of signature hash algorithms
  • Extract list of ALPN, NPN and ALPS
  • Extract list of record, handshake and supported versions
  • Extended sslProto to flag GREASE values and more
  • Added support for TLS 1.3 draft versions
  • Added support for missing TLS 1.3 ciphers
  • Added support for missing TLS 1.3 alerts
  • Added number of TLS 1.3 draft versions flows to plugin report
  • Added number of DTLS 1.3 flows to plugin report
  • Added support for JA4/JA4S fingerprints
  • Fixed handling of GREASE values in JA3 fingerprints
  • Updated list of insecure, weak, secure and recommended ciphers
  • Updated JA3 fingerprints
  • Updated SSL blacklist
• tcpFlags:
  • Added support for JA4T fingerprints
• tp0f:
  • Added packet mode
• txtSink:
  • Report process priority in headers file
• voipDetector:
  • Added VOIP_SIP, VOIP_RTP, VOIP_RTCP to control protocol dissection
  • Added VOIP_BUFMODE, RTPBUFSIZE, RTPSUBDIRS, VOIP_PERM macros
  • Decode RTCP by default
  • Output SIP contacts and Call-IDs
  • Output SDP session ID
  • Fixed description of RTP payload type 125
  • Code hardening
• fsutils.[ch]:
  • New helper macro:
    • T2_MKPATH_WITH_FLAGS()
• t2buf.[ch]:
  • New function:
    • t2buf_ptr()
• t2log.h:
  • New macros:
    • T2_FPLOG_DIFFNUM, T2_FPLOG_DIFFNUM0
• t2utils.[ch]:
  • New helper macros:
    • DTLS12_HEADER()
    • t2_calloc(), t2_malloc()
  • New functions:
    • t2_strncpy()
    • t2_tcp_socket_connect(), t2_tcp_socket_connect_to_server(), t2_udp_socket_init()
    • t2_calloc_fatal(), t2_malloc_fatal()
  • Fixed nanoseconds representation in t2_log_date() and t2_log_time()
• API break:
  • Renamed t2_calloc/t2_malloc to t2_[cm]alloc_fatal()
• tawk:
  • tawk is now faster
  • Inverted -t option behavior: use it to validate column names (slow)
• scripts:
  • t2build:
    • Added --lto option to enable link time optimization (meson only)
  • t2caplist:
    • Added -x option to filter by extension (faster, but less precise)
    • Added -t option to sort list by first packet time
  • t2conf:
    • Fixed t2conf tranalyzer2 --gui
    • Several other fixes and improvements
  • t2fm:
    • Added information about ASNs
    • Added -d/--data-carving option to report EXE downloads
  • t2fuzz:
    • Added -S/-P/-a options to start netcat (nc) before running t2

• tranalyzer2:
  • Switched to nanoseconds precision:
    • New default values: TSTAMP_PREC=1, B2T_NANOSECS=1
  • New subnet files
  • Added ENVCNTRL flag to control plugin configuration via environment variables
  • Added support for XXH3 (64-bits and 128-bits) hash functions
  • Monitoring mode output field separator can now be changed with SEP_CHR
  • Packet mode output field separator can now be changed with SEP_CHR
  • Report sensor ID and bound CPU number (if -c option was used) in final report
  • Report link layer type in monitoring (status) report
  • Report snapshot length in final and monitoring (status) report
  • Fallback to user’s default plugin folder when running T2 with sudo
  • Fixed dissection of IPv6/AH/IPv6
  • Updated t1ha to version v2.1.4
  • Updated wyhash to version wyhash_final4
  • Updated xxhash to version v0.8.1
  • New macros:
    • OUTBUF_APPEND_*_ZERO()
    • OUTBUF_APPEND_STR_AND_FREE()
    • OUTBUF_APPEND_ARRAY_STR_AND_FREE()
    • OUTBUF_APPEND_U{16,32,64}_NTOH()
  • Minor fixes and improvements
• plugins/*:
  • Added support for ENVCNTRL
  • Renamed configuration flags
  • Minor fixes and improvements
• basicFlow:
  • Renamed ethVlanID and ethVlanHdr to vlanID and vlanHdr
• basicStats:
  • Added udpLen, snapL[47]Len to packet mode
• connStat:
  • Added new connG feature
• dnsDecode:
  • Added geolocation information to packet mode
• geoip:
  • Also output geoStat column when GEOIP_LIB=0
• httpSniffer:
  • Added packet mode
• mndpDecode:
  • Fixed autotools backend
• nDPI:
  • Added packet mode
  • Updated nDPI library to version 4.6
• netflowSink:
  • Plugin now also working when BLOCK_BUF=1
• pcapd:
  • Added possibility to modify packets before saving them:
    • PD_TSHFT (time)
    • PD_MACSHFT (MAC addresses)
    • PD_VLNSHFT (VLAN ID)
    • PD_IPSHFT (IPv4/6 addresses)
    • More flags to control the modification process
  • Fixed PD_MODE_OUT=1 when -e option was used
• portClassifier:
  • Fixed packet mode for L2 flows
• sctpDecode:
  • Improved packet mode
• sslDecode:
  • Updated SSL blacklist
• tcpFlags:
  • Added flag for invalid length in UDP/UDP-Lite header
  • Output UDP length and snapped layer 4/7 length in packet mode
• txtSink:
  • Report libpcap version in _headers.txt file
• voipDetector:
  • Added VOIP_SIP to (de)activate SIP dissection
  • Added support for MPEG-2 transport stream (MP2T, RTP type 33)
  • Extract X-Real-IP from SIP header
  • Report number of SIP, SDP, RTP and RTCP packets
  • Fixed extraction of SIP User-Agent
• New plugins:
  • bayesClassifier: classification using Naive Bayes
  • kafkaSink: output into an Apache Kafka event streaming platform
• fsutils.[ch]:
  • New helper functions:
    • file_manager_{fprintf,fputc,fputs}()
    • file_manager_fwrite()
    • file_manager_fflush()
• iputils.[ch]:
  • New helper functions:
    • ipv4_to_mask(), ipv6_to_mask()
    • mask_to_ipv4(), mask_to_ipv6()
• subnetHL.h:
  • New helper macros:
    • SUBNET_POS_UNKNOWN and SUBNET_POS_IS_UNKNOWN()
    • SUBNET[46]_{ASN,CNTY,CTY,NETID,LAT,LNG,PREC,ORG}
• t2base64.[ch]:
  • New helper functions to base64 encode data
• t2buf.h:
  • New helper macro: t2buf_rewind()
• t2crypto.[ch]:
  • New helper functions to compute message digests (md5, sha1, …)
• t2Plugin.h:
  • New helper macro: T2_PLUGIN_STRUCT_RESET_ITEM()
• t2utils.[ch]:
  • New functions and macros for ENVCNTRL:
    • t2_get_env(), t2_free_env()
    • T2_ENV_KEY(), T2_ENV_VAL(),
    • T2_SET_ENV_NUM(), T2_SET_ENV_STR()
  • New helper macros:
    • FLOW_IS_IP(), PACKET_IS_IP()
    • L2_HEADER(), L3_HEADER(), L4_HEADER(), L7_HEADER(),
    • ETH_HEADER(), LAPD_HEADER()
    • IPV4_HEADER(), IPV6_HEADER()
    • ICMP_HEADER(), IGMP_HEADER(), PIM_HEADER(), SCTP_HEADER(), TCP_HEADER(), [UDP_HEADER()](/tutorial/plugindevcheatsheet#layer-4-header)
    • L2_PROTO(), L3_PROTO(), L4_PROTO(),
    • PROTO_IS_IPV4(), PROTO_IS_IPV6()
    • PROTO_IS_ICMP4(), PROTO_IS_ICMP6(), PROTO_IS_IGMP(), PROTO_IS_SCTP(), PROTO_IS_TCP(), PROTO_IS_UDP()
    • T2_MKPATH()
    • T2_MAC_STRLEN
    • T2_FREE_CONST()
    • T2_IPV4_TO_STR(), T2_IPV6_TO_STR()
  • New helper functions:
    • t2_alloc_strcat()
    • t2_swap_mac()
    • t2_fopen_in_dir(), t2_fopen_with_suffix()
    • t2_discard_trailing_chars()
• fpsGplt:
  • Added support for t2plot options
• statGplt:
  • Added -f/--flow option to plot specific flow only
  • Added -d/--dir option to plot specific direction only
  • Automatically derive output filename
  • Added support for t2plot options
  • Minor fixes and improvements
• tawk:
  • Improved texscape() function
  • Minor fixes and improvements
• t2build:
  • Fix for old meson versions (< 0.36.0)
• t2conf:
  • Added -e option to list plugins currently set environment variables
  • Added -E option to list plugins available environment variables
  • Fixed -D option for empty strings
  • Fixed handling of -m, --dual, --ip4 and --ip6 options
  • Minor fixes and improvements
• t2docker:
  • Improved error reporting
  • Minor fixes and improvements
• t2fm:
  • Report number of flows in Summary section
  • Report snapshot length in Summary section
  • Report unique VLAN tags in Summary section
  • Improved error reporting
  • Minor fixes and improvements
• t2netID:
  • Improved output readability
• t2plot:
  • Improved support for mouse interaction
• t2plugin:
  • Added -m/--minimal, -t/--t2buf and -s/--sink options
  • Added -y/--yes option
  • Improved plugin number testing/generation for sink plugins
  • Minor fixes and improvements
• t2py:
  • Improved readability of T2.status() and T2Plugin.status()
• t2test:
  • If *.flags is empty, run t2build instead of aborting
• t2utils.sh: * New helper functions: abort_required_dir, abort_required_file_or_dir
• New script:
  • t2fuzz: corrupt PCAP files and run T2 against them
• API break:

  • flow.h:

    • Some fields in flow_t have been renamed

  • packet.h:

    • Most fields in packet_t have been renamed
    • The type of some fields in packet_t has changed

  • outputBuffer.h:

    • Renamed OUTBUF_APPEND_OPTSTR() to OUTBUF_APPEND_OPT_STR()

  • t2utils.h:

    • Changed signatures of t2_alloc_filename() and t2_build_filename()
    • t2_open_filename() was renamed to t2_fopen()

  • binaryValue.h:

    • bv_new_bv(): swapped parameters name and desc to match order of BV_APPEND_*() macros

  • Renamed plugins callbacks:

    New callback function	Old callback function	Handled by
    t2PluginName	get_plugin_name	T2_PLUGIN_INIT*()
    t2PluginVersion	get_plugin_version	T2_PLUGIN_INIT*()
    t2SupportedT2Major	get_supported_tranalyzer_version_major	T2_PLUGIN_INIT*()
    t2SupportedT2Minor	get_supported_tranalyzer_version_minor	T2_PLUGIN_INIT*()
    t2Dependencies	get_dependencies	T2_PLUGIN_INIT_WITH_DEPS()
    t2Init	initialize
    t2PrintHeader	printHeader
    t2OnNewFlow	onFlowGenerated
    t2OnLayer2	claimLayer2Information
    t2OnLayer4	claimLayer4Information
    t2OnFlowTerminate	onFlowTerminate
    t2PluginReport	pluginReport
    t2Monitoring	monitoring
    t2Finalize	onApplicationTerminate
    t2BufferToSink	bufferToSink
    t2SaveState	saveState
    t2RestoreState	restoreState

  • New signatures for callbacks:

    • t2BufferToSink receives the binary_value_t to decode the buffer: void t2BufferToSink(outputBuffer_t *buf, binary_value_t *bv)
    • t2OnFlowTerminate receives the buffer to fill: void onFlowTerminate(unsigned long flowIndex, outputBuffer_t *buf)

• tranalyzer2:
  • Added support for configuring aggregation mode with t2conf --gui
  • Added support for bit operations in packet mode (SPKTMD_BOPS)
  • Added number of L2/IPv4/IPv6 flows to end report
  • Fixed reporting of ARP/RARP packets in final report
• descriptiveStats:
  • Added DS_QUARTILES flag to control quartiles calculation
  • Renamed ENABLE_{IAT,PS}_CALC to DS_{IAT,PS}_CALC
• nDPI:
  • Updated nDPI library to version 4.4
• portClassifier:
  • Added packet mode
• psqlSink:
  • Improved documentation
• regex_pcre:
  • Added packet mode
• sctpDecode:
  • Improved packet mode
  • Various fixes and improvements
• sslDecode:
  • Updated SSL blacklist
• tcpFlags:
  • Added MPTCP variables to packet mode
  • Various fixes and improvements
• New plugins:
  • clickhouseSink: output into a ClickHouse database
  • mndpDecode: MikroTik Neighbor Discovery Protocol
• tawk:
  • New functions: bitshift, isfloat, isint, isuint, nibble_swap
  • Added variables descriptions (-V option) for MPTCP
  • Various fixes and improvements
• t2fm:
  • Added -c option to generate a PDF report from a ClickHouse database
  • Various fixed and improvements
• scripts:
  • t2doc: added -n option to not open the generated PDF
  • t2timeline: bugfixes and improvements
  • t2conf: added support for bitfields in GUI mode

• tranalyzer2:
  • Added FLOW_IS_LAPD() macro
  • Fixed SCTP stream aggregation
  • New SCTP aggregation mode: Association
• findexer:
  • Added -P option to extract specific packets instead of whole flows
• nDPI:
  • Updated nDPI library to version 4.2
• ntlmsspDecode:
  • Produce separate files for NetNTLMv1 and NetNTLMv2 hashes
• sctpDecode:
  • Fixed SCTP stream aggregation in packet and flow mode
  • Chunk parameter extraction
• sslDecode:
  • Updated SSL blacklist
• t2conf:
  • Adapted script for C++ plugins and .hpp files
• tawk:
  • Added -P option to extract specific packets instead of whole flows
  • flow(), packet(): added support for filtering multiple ranges
  • proto(): added support for filtering multiple ranges
  • [sp]?port(): added support for filtering multiple ranges
  • t2sort(), t2rsort(): added support for sorting by multiple columns
  • tobits(): added option to force interpretation as hex
• setup.sh:
  • Fixed t2update check for new version

• tranalyzer2:
  • Added SPKTMD_PCNTL flag to control start of payload in packet mode
  • Added SPKTMD_PCNTH_PREF and SPKTMD_PCNTH_SEP to control byte prefix and separator in packet mode as hex
  • Removed bug in alarm mode when subnet is switched off
  • Improved error reporting
• arpDecode:
  • Improved detection of ARP spoofing
• basicFlow:
  • Fixed packet mode
• httpSniffer:
  • Extended monitoring report
• modbus:
  • Added monitoring report
• ntlmsspDecode:
  • Only print decoding warnings/errors if DEBUG > 0
• payloadDumper:
  • Fixed payload extraction based on port numbers
  • Added options to dump payload of layer 2 flows (PLDUMP_L2 and PLDUMP_ETHERTYPES)
  • Added option to start dumping L2 and UDP payload from a specific offset (PLDUMP_START_OFF)
• sshDecode:
  • Added monitoring report
  • Code hardening
• sslDecode:
  • Updated SSL blacklist
• tcpFlags:
  • Improved troubleshooting and anomaly info: tcpAnomaly, tcpFStat, tcpFlags, window, seq/ack number features, fault counts, etc
  • Extended packet mode including pktTrip: packet round-trip time and flags
• vrrpDecode:
  • Added monitoring report
  • Code hardening
• vtpDecode:
  • Fixed autotools backend
• t2test:
  • Renamed -r/--resume option to -b/--resume
  • Added -r/--configure option (t2build -r)
• tawk:
  • Added tobits() function
• t2_aliases:
  • New sortup alias (same as sortu, i.e., sort | uniq -C | sort -rn), but report the relative percentage instead of the absolute count
• setup.sh:
  • Force re-generation of build files

• bgpDecode:
  • Improved scripts and plugin report
• dhcpDecode:
  • Fixed reporting of DHCP messages with types > 8
• ftpDecode, ircDecode, payloadDumper:
  • Fixed handling of retransmissions and out-of-order packets
• payloadDumper:
  • Added capability to dump SCTP payload
  • Improved handling of TCP Keep-Alive packets
  • Added packet mode
• sshDecode:
  • Added packet mode
• sslDecode:
  • Updated SSL blacklist
• tcpFlags:
  • Flag TCP Keep-Alive packets in tcpFStat
  • Improved detection of TCP sequence number anomalies
• torDetector:
  • Improved detection for Tor version <= 0.4.5.10
• t2docker:
  • Minor improvements
• t2fm:
  • Added -y/--yes option
• t2locate:
  • Refactoring for compatibility with macOS and older systems
• t2py:
  • Added support for tawk: T2Utils.TAWK and T2Utils.tawk()
  • Added support for following streams: T2.follow_stream() and T2Utils.follow_stream()
  • Added support for streaming flows: T2.streaming and T2.stream()
  • Improved error reporting for functions using the subprocess module
  • Improved API documentation
  • Minor improvements
• tawk:
  • New base64() function
  • New follow_stream() function
  • New t2rsort() function to sort in reverse order
  • Added variables descriptions (-V option) for BGP and SSL/TLS
• t2_aliases:
  • Added alias for t2locate
  • Improved Bash and ZSH completions

• tranalyzer2/basicFlow:
  • Fixed detection of Tor addresses
  • Moved detection of Tor addresses from basicFlow to the core
• torDetector:
  • Added detection heuristics based on packet size
  • Bug fixes and minor improvements
• t2py:
  • Bug fixes and minor improvements
• See the news for Tranalyzer2 Tarantula version 0.8.11lmw1 and 0.8.11lmw2 below for more details

• torDetector:
  • Added packet mode
• t2py:
  • Improved alias and library documentation
• New plugin:
  • payloadDumper (similar functionality to tcpflow)
• See the news for Tranalyzer2 Tarantula version 0.8.11lmw1 below for more details

• tranalyzer2:
  • New T2_FPLOG_AGGR_{HEX,HEX0,H8,H16,H32,H64} macros
  • OUTBUF_APPEND_NUMREP() can now be used with non-uint32_t variables
  • Fixed flow direction correction for port 8080, 8081, 8088 and 8089
  • Updated uthash to version 2.3.0
• geoip:
  • Added GEOIP_ASN and GEOIP_CONNT to output source and destination AS number and connection type (only available in GeoLite2 Enterprise DB)
  • Added name of database (GEOIP_DB_FILE{,4,6}) to configuration flags
• macRecorder:
  • Faster conversion of EtherType and MAC addresses database
• mqttDecode:
  • Added packet mode
• nDPI:
  • Updated nDPI library to version 4.0
• radiusDecode:
  • New configuration flags
  • Added packet mode
  • Improved flow output
• sshDecode:
  • Updated HASSH fingerprints
• sslDecode:
  • Added SSL_DETECT_TOR configuration flag
  • Updated SSL blacklist
• New plugins:
  • bgpDecode
  • torDetector
  • vtpDecode
• t2plugin:
  • Added -N option to list plugin names only
  • Added -H option to remove section headers
• fpsGplt/statGplt/t2plot/t2timeline/t2viz:
  • Added --gif/--jpeg options
• New t2py library to control and operate T2 with Python

• tranalyzer2:
  • Added support for IEEE 802.3br mPackets encapsulation (DLT_ETHERNET_MPACKET)
  • flowInd is now printed by the core
  • New OUTBUF_APPEND_ARRAY macros
• New plugin:
  • ntlmsspDecode
• plugins/*:
  • Improved plugin report and packet mode
  • Improved default.config, t2plconf and documentation
• tcpFlags:
  • Fixed reported number of attempted/successful scans
• fpsGplt/statGplt/t2plot/t2timeline/t2viz:
  • Added --png/--svg options
• tawk:
  • Added -L option to decode all variables from Tranalyzer log file
• t2build/autogen.sh:
  • Added support for building/cleaning t2b2t, t2whois and fextractor
  • Use t2build -i tranalyzer2 to install tranalyzer in the plugin folder
• t2conf:
  • Added support for querying the default value of a config flag: t2conf pluginName -G flagName -g default
  • Added support for resetting a flag to its default value: t2conf pluginName -D flagName=default
  • Adapted -D/-G options to set/extract values from configuration files t2conf pluginName -g [file.config|default] -G name
  • Added -S option to list active plugins in a loading list
  • Use --gui with -g to graphically edit configuration files instead of headers: t2conf pluginName -g --gui
• t2docker:
  • Massively reduced image size
  • Added -m/--multi-stage option
  • Added support for t2whois and t2b2t
• t2plot:
  • Added --no-title option
• t2rrd:
  • New script combining the old rrdmonitor (t2rrd -m) and rrdplot
• t2test:
  • Added -W option to ignore warnings caused by #warning macro

• tranalyzer2:
  • Added support for long options
  • Added support for t1ha hash functions (meson build backend only)
  • PLLIST (plugin loading list) can now be specified as absolute path (previously only possible via tranalyzer -b option)
  • Removed global.h:
    • C plugins should include "t2Plugin.h" instead
    • C++ plugins should include "t2Plugin.hpp" instead
  • Updated MUM-hash to version 3
  • Updated uthash to version 2.1.0
  • Updated wyhash to final (?) version (Aug. 2020)
  • Updated xxhash to version 0.8.0
  • Improved computation of padding bytes for IPv4/6 and LLC
  • Bugfix in IPv6 fragmentation handling
• bin2txt.[ch]:
  • New B2T_NANOSECS flag replaces old and buggy B2T_TIME_IN_MICRO_SECS
  • Bugfix in human readable time string (B2T_TIMESTR)
• t2Plugin.h:
  • Added T2_PLUGIN_STRUCT_NEW() macro
• arpDecode:
  • Flag ARP Probes and Announcements
• ftpDecode:
  • Improved data carving capabilities
  • Improved plugin report
  • Fixed name of carved data
• ircDecode:
  • Extensive refactoring
  • Extended flow output
  • Improved data carving and decoding capabilities
• macRecorder:
  • Extended MR_MACLBL to output MAC labels as int, hex or string
  • Added src/dstMacLbl to packet mode
  • Fixed output of manufacturers in packet mode
• mongoSink, mysqlSink:
  • Store MAC and IPv4/6 addresses as requested in bin2txt.h (MAC_FORMAT, MAC_SEP, IP4_FORMAT and IP6_FORMAT)
• nDPI:
  • Updated nDPI library to version 3.4
• ospfDecode:
  • Added support for OSPFv3
  • Improved rospf script to map the network with graphviz
• telnetDecode:
  • Improved data carving and decoding capabilities
• tftpDecode:
  • Improved plugin report
  • Fixed typos in column names
  • Extended output of flow and packet mode
• voipDetector:
  • Improved plugin report
• New plugin:
  • mqttDecode
• t2b2t:
  • Added -l option to list the column names from a binary file
• t2conf:
  • -L option (edit plugin loading list) does not require --gui option anymore
• t2whois:
  • Added T2WHOIS_RANDOM flag in t2whois.h to (de)activate testing of random IPs (and drop the dependency to libbsd)
• t2build/autogen.sh:
  • Changed default build backend to meson (with a fallback to autotools-out-of-tree)
  • Deprecated autotools build backend
• tawk:
  • Improved shark() function (query T2 with wireshark/tshark syntax)
  • Added more variables descriptions (-V option): ethType, l4Proto, …
• New t2docker script:
  • create and manage Tranalyzer Docker containers
  • run T2 commands inside Docker containers
• fpsGplt:
  • Added -P/--plot option to directly plot the packet signal
• statGplt:
  • Added -P/--plot option to directly plot the signals
  • Added --iat/--ps/--ps-iat options to generate specific distributions
• t2plugin:
  • Renamed from new_plugin
  • Create new C, C++ or Rust plugins
  • List existing plugins

• tranalyzer2:
  • Improved error reporting
• macRecorder:
  • Updated manuf.txt
• sslDecode:
  • Updated sslblacklist.[ct]sv
• t2flowstat:
  • Improved and extended replacement of flowstat
• t2whois:
  • Fixed -k option to generate KML files
• setup.sh:
  • Added missing libbsd-devel and readline-devel dependencies for CentOS/Fedora/Red Hat

• tranalyzer2:
  • Updated subnet files
• dnsDecode:
  • New DNS_WHO configuration flag to add geo info to DNS A and AAA records
  • Added type and class of query
• macRecorder:
  • Updated manuf.txt
• nDPI:
  • Replaced buggy kerberos.c with latest development version from ntop/nDPI
• nFrstPkts:
  • Bugfix in absolute time computation (NFRST_IAT=2)
• sslDecode:
  • Updated sslblacklist.[ct]sv
• t2conf:
  • Added --gui option
• tawk:
  • Added t2whois() function
  • Added passivedns() function (loaded with tawk -e)

• setup.sh: added -C option to check for new releases
• tranalyzer2: (thx to Diaf Alaeddine to test this feature)
  • Corrected FDURLIMIT mode for unusual bursty traffic
  • Added FDLSFINDEX: sub flows can have now the same findex
• dnsDecode: updated maldomain.txt
• icmpDecode:
  • Improved packet mode
  • Report aggregated icmpStat in final and monitoring report
  • Detect covert channels such as Loki or OpenSSH in ICMP
• macRecorder: updated manuf.txt
• sslDecode: updated sslblacklist.[ct]sv
• autogen.sh/t2build:
  • Added -B option to change build backend:
    • autotools
    • autotools-out-of-tree
    • cmake
    • meson
  • Added -G option to select CMake generator
• t2fm: added --reset option
• New tutorial: flow mode

• tranalyzer2, basicFlow, utils:
  • Subnet control moved from basicFlow to tranalyzer2
  • Subnet routines moved from basicFlow to utils/subnet
  • New subnet aggregation mode
  • IPv4/6 Tor address labeling
  • Updated subnet files
  • Fixed subnet, Tor generation
• basicFlow, basicStats, connStat:
  • Added support for subnet aggregation mode
• tranalyzer2:
  • Fixed bug in SCTP engine
• dnsDecode, sslDecode, httpSniffer, tcpStates:
  • Used field name in Aggregated ... report (easier to grep and decode)
• jsonSink, mongoSink, mysqlSink, psqlSink, sqliteSink:
  • Added {JSON,MONGO,MYSQL,PSQL,T2_SQLITE}_SELECT options to only output/insert specific fields into the DB
• sqliteSink:
  • Automatically grow query buffer as required
  • Replaced SQLITE_QRY_LEN with SQLITE_QRY_MAXLEN to control maximum size of query buffer
  • Discard flows which could not de be deserialized instead of exiting
  • Use Tranalyzer -w option as database name
• dnsDecode:
  • Report percentage of flows with alarms
  • Updated domains blacklist
• entropy: added end report
• fnameLabel: added configuration flags: FNL_LBL, FNL_HASH, FNL_FLNM and FNL_FREL
• geoip:
  • Replaced GEOIP_LEGACY configuration flag with GEOIP_LIB=[0,1,2]
  • Faster direct MaxMindDB access
  • t2mmdb: fast direct request to MaxMindDB
  • t2mmdba: convert MaxMindDB to T2 subnet format
• macRecorder:
  • Improved MAC labeling
  • Updated manufacturers list
  • Reduced memory usage
• nDPI: updated nDPI library to version 3.2
• regex_pcre: report percentage of flows with alarms
• sshDecode:
  • Added SSH_ALGO to display chosen algorithms
  • Added SSH_LISTS to display lists of supported algorithms
  • Added SSH_FINGERPRINT to output fingerprints as MD5 or SHA256
  • Improved detection of Elliptic Curve Diffie-Hellman Key Exchange
• sslDecode: updated blacklist
• bin2txt: added B2T_NON_IP_STR macro to configure representation of non-IPv4/6 addresses in IP columns
• t2whois: added -D option to run as a server
• t2netID: Decode T2 hexadecimal country organization codes
• scripts:
  • t2_aliases: new t2mmdb and t2netID aliases
  • t2build/autogen.sh: new -U option to update databases, blacklists, …
  • t2fm:
    • Added top organizations section
    • Added SSH section with top connections and known HASSH signatures
    • Added --hide-{user,pass,user-pass} options to obfuscate usernames/passwords
    • Added --no-* options to discard specific sections of the report
    • New -NUM (-0, -1, …) option to control the number of queries to run in parallel
  • t2plot: allow for * in -s[xyz] options, e.g., -sx '0:*'
  • t2utils.sh: new helper functions: find_most_recent_{dir,file}, t2_wget[_n], t2_build_exec, ask_default_{no,yes}

• scripts:
  • t2conf, t2test, t2fm, tawk: bugfixes and improvements
  • Simplified configuration with t2conf:
    • Reset: t2conf pluginName --reset
    • Generate: t2conf pluginName -g myPluginName.conf
    • Apply: t2conf pluginName -C myPluginName.conf
• tranalyzer2:
  • Flag and handle IP packets with payload length > framing length
  • Flag IPv4 packets with header length < 20 bytes
  • Fixed column names for REPORT_HIST=1
  • Fixed l7Len for OSPFv2
  • Added support for Ethernet over MPLS
• arpDecode: fixed detection of gratuitous ARP
• basicFlow: new subnet files, hex coding for country now 9 bits
• ospfDecode: bugfixes, code hardening
• tcpFlags:
  • MPTCP new features, thx to Theresa TU Berlin
  • Flag and handle corrupt IPv4 options with length = 0
• regex_pcre: New engine and regfile format

• basicFlow:
  • t2whois: new program to query Tranalyzer databases
  • New subnet files, county city configurable
• basicStats: report L2 and L3 biggest talkers
• geoip: updated GeoLite2 database
• macRecorder:
  • Report min, max and average MAC pairs per flow
  • Updated manuf database
• nDPI: updated nDPI library to version 3.0
• radiusDecode, smtpDecode: bugfixes
• sctpDecode: merged SCTP_CHNKVAL and SCTP_CHNKSTR
• sshDecode: compute and lookup HASSH fingerprints
• sslDecode: updated blacklist
• t2caplist: added -z and -R options, various fixes
• t2conf: added bash/zsh completion for -D and -G options
• t2plot:
  • Added support for drawing histograms (-H and -D options)
  • Added -c option to customise chart color
• Tester.py:
  • Make sure to restore default configuration when toggle test failed
  • New options -S1, -S2 and -J (bit shift and Johnson counter)
  • New option -e to ignore compilation errors caused by #error macro
  • New t2test alias to run the tester from anywhere
• fpsGplt:
  • fpsEst was merged into fpsGplt as -j option
  • Improved -d option: -d 0|1 is now -d A|B
• protStat:
  • Added -C option to not output percentages
  • Added -r option to sort in reverse order
  • Added -H, -HR and -HH options to control the formatting of numbers
  • Added --color[=WHEN] option (default: no color if output redirected)
• t2b2t:
  • Utility to convert T2 binary files (renamed from tranalyzer-b2t)
  • Automatically compiled when building binSink or socketSink (CONTENT_TYPE=0)
• t2_aliases: new t2b2t and t2whois alias
• setup.sh: added -u/-U option to (not) update the databases

• basicFlow: bugfix teredo subnet labeling

• Windows 10 version
• tranalyzer2: bugfix in packetCapture: fragment hash lookup missing l4proto
• tcpFlags: Bugfixes
  • ipFlags: Fragmentation and OSPF checksum calculation
  • ipFlags: Min frag flag not at last packet
  • Limit pseudo header calculation, OSPF has not pseudo header
  • Packet Mode: relative seq/ack number calculation
  • TCP time option: fix of uptime clock estimation
  • Window scale value
  • Scan detector
• httpSniffer: robust against corrupted chunked pages

• basicFlow: improved subnet files
• dnsDecode: updated blacklists
• geoip: updated GeoLite2 database
• macRecorder: updated manuf database
• sslDecode: updated certificate blacklist

• Reorganization of source code
  • Plugins moved into plugins/ subfolder
• tranalyzer2:
  • Improved ALARM and FORCE mode
  • Added new hash functions
• basicFlow:
  • Simplified configuration for EtherType, MAC, VLAN and MPLS
  • Added src/dst-Mac to flow output (BFO_MAC=1)
  • New improved subnet files and Tor labeling
• dhcpDecode: added DHCP_FLAG_MAC flag
  • Added dhcpSrcMac and dhcpDstMac columns
• geoip: favour GeoLite2 over Legacy databases
• protoStats/nDPI: added number of bytes for each protocol
• scripts:
  • Improved protStats, t2stat, setup.sh
• New tutorials:
  • ALARM mode
  • FORCE mode
  • And many more!

• Landattack L2 removed
• scripts:
  • Improved protStat, t2plot and tawk
  • Fix of setup.sh

• The Anteater is currently in Finland giving a workshop at the BoostAcademy in Turku and eating some bugs, courtesy of ENTIS.
• Find out more about the workshop here!

• Fix for macOS

• New plugin: findexer
• basicFlow:
  • Updated IPv4/6 databases
  • Flag Tor addresses
• dnsDecode: blacklisted domain names detection
• ftpDecode: bug fixes
• geoip: updated databases
• nDPI: updated nDPI library to 2.6.0
• pwX: improved detection of HTTP based credentials
• sslDecode: updated JA3/JA3S database and SSL blacklist
• tranalyzer2:
  • Improved final and monitoring reports
  • Improved network aggregation mode IPv4/6
• autogen.sh/t2build:
  • Faster parallel compilation
  • New -P/--profile option
• Simpler control of MAC addresses representation (utils/bin2txt.h):
  • MAC_FORMAT: 0: string, 1: hex
  • MAC_SEP: separator for MAC addresses as string (default: ":")
• Avoid unnecessary dependency to zlib (*Sink)
• tawk: removed deprecated function bitisset
  • Use bitsanyset and bitsallset instead
• Bugfixes and code hardening

• basicFlow: bugfixes in teredo
• scripts:
  • Facilitated configuration of .h files via t2conf
  • Improved fpsStat mining script
• Output function refactoring
• Doc fixed
• Tutorials corrections

• More TM features in nFrstPkts scripts, tutorial improvements
• tcpFlags: minwinsz detection, doc
• telnetDecode: bug fixes
• Minor code refactoring

• Fix for older distributions where zlib version < 1.2.9 (big thanks to Ali Safari Khatouni from Dalhousie University for reporting the issue!)

• New plugins:
  • sslDecode: SSL/TLS, including JA3 hash,
  • p0f: OS fingerprinting based on SSL/TLS
• scripts:
  • Improved t2fm: create PDF report from MongoDB or PostgreSQL database
  • New t2plot and traffic mining scripts
• nFrstPkt: new signal preprocessing features
• Improved dnsDecode and arpDecode
• txtSink: added option to compress (gzip) the output
• Geo-labeling information for packet mode (-s option)
• Check out our tutorials

• Concurrent L2, IPv4/6 triple mode.
• Linux & MAC tested.
• It is a different and more powerful beast, so check it out.

This is the last Boeing version before the IPv4/6 dual mode Tarantula version!

• Linux & MAC tested.
• Improved end and t2fm report.
• Several bug fixes.
• Some protocol plugins added.
• Improved IPv4/6 geolabeling in basicFlow, now also non-CIDR ranges are possible, if enabled: SUBRNG=1.
• Improved packet/flow statistics for traffic mining.

• Linux & MAC tested.
• More support for L2 encapsulations
• Improved packet mode
• Core code refactored
• Fast and more precise IPv4/6 geolabeling in basicFlow (special thx to Lars from UniBW), so slow geoip might be obsolete some day.
• New plugin:
  • telnetDecode: because somebody insisted, here it is. Have fun!

• HashAutopilot: Protection against flow hash overflow, T2 finishes its job without complaining
• Added support for GENEVE, VXLAN-GPE and NSH
• Added support for WCCP, JUNIPER_PPPOE and JUMBO_LLC
• Added support for DLT_PPP_SERIAL
• New plugins:
  • cdpDecode
  • lldpDecode
  • radiusDecode
• Better fragmentation hashing

• Several encapsulations added, such as:
  • Ethernet over IP (EtherIP)
  • Control and Provisioning of Wireless Access Points (CAPWAP)
  • Anything in Anything (AYIYA)
  • … and more!
• Improved packet mode, now each plugin can contribute
• Improved protocol plugins including content downloads
• Improved SCTP support
• Better human readability of end report,
• Improved t2fm PDF summary report scripts
• New powerful tawk post processing scripts

We are also continuously fuzzing and testing Tranalyzer to keep it resilient against all kinds of attacks.

PDF Report Generation from PCAP using t2fm

Sample report (IPs and passwords anonymized for privacy reasons): (PDF)