The ITHITOOLS program include a set of tools designed to compute some of "Identifier Technology Health Indicators" (ITHI) metrics defined by ICANN. The identifiers include Names, Numbers, and Protocol Parameters. A general presentation of the ITHI work is available here: https://www.icann.org/ithi. The capture part of the tools are focused on a subset of the ITHI problem, specifically the computation of the IHTI metrics M3, M4 and M6:
- overhead in root traffic (M3.1, and M3.2 for duplicate requests contained in single PCAP file)
- leakage of RFC6761 names and other undelegated names (M4)
- usage of DNS protocol parameters defined in IANA registries (M6) plus list of unregistered parameters and frequencies.
The metric part of the tools reads data from the ITHI input folders, and produces the ITHI metrics.
The project builds a single executable, "ithitools.exe", which can be used in two modes:
-
analysis of a single PCAP file to produce a summary file (CSV format) containing the counts of interesting parameter found in the file.
-
aggregation of several summary files and extraction of supported metrics.
Calling ithitools with the option "-h" will produce a standard looking "usage" page. Further documentation is available in DnsProtocolParametersAnalysis.pdf.
On Linux systems, the project also builds a shared library, "ithicap" (libithicap.so). This library is meant to be used as an extension to "dnscap". A typical usage would be:
dnscap <dnscap-parameters> -P libithicap.so -o <ithi-capture-file.csv>
The "ithicap" capture options can be displayed with the option -h, as in:
dnscap <dnscap-parameters> -P libithicap.so -h
More information on configuring dnscap and the ithicap plugin can be found on the DBS capture settings wiki page.
ITHITOOLS was developed in C++, and can be built under Windows or Linux.
To build ITHITOOLS on Windows, you need to:
-
Have a version of Visual Studio 2017 installed. The freely available "Community" version will work.
-
Clone and compile ITHITOOLS, using the Visual Studio 2017 solution ithitoolsvs.sln included in the sources. Typical command will be:
git clone --recursive https://github.com/private-octopus/ithitools/
cd ithitools
.\ithitools.sln
- If you have already clones the project, typical update commands will be:
git submodule update
git pull --all
- You can use the unit tests included in the Visual Studio solution to verify the port.
To build ITHITOOLS on Linux, you need to:
- Clone and compile ITHITOOLS:
git clone --recursive https://github.com/private-octopus/ithitools/
cd ithitools
cmake .
make
- Run the test program "ithitest" to verify the port.
The --recursive
option will load the submodule cdns
from the github project
cdnsrdr, which implements a parser
for CBOR encoding of DNS captures, according to RFC8618.
Of course, if you want to just update to the latest release, you don't need to install again. You will do something like:
cd ithitools
git submodule init
git submodule update
git pull --all
cmake .
make
Building ITHITOOLS on Linux requires installation of CMAKE and of a C++ compiler. The installation tools depend on the Linux version. We have tested the installation on Ubuntu (16 and 17) and on Centos (6.9 and 7).
On Ubuntu, the recommended way to install the GCC/C++ compiler is by installing the "build essentials":
sudo apt-get install build-essential
GIT and CMAKE can of course be installed as:
sudo apt-get install cmake
sudo apt-get install git
On Centos, you will need the GCC/C++ compiler, CMAKE and git:
sudo yum install gcc
sudo yum install gcc-c++
sudo yum install cmake
sudo yum install git
By default, CMAKE will create a make file that reference the default C/C++ compilers for your system, typically gcc/g++. If you want to use a different compiler, for example CLANG, you can either change your system's defaults, or set explicit arguments to CMAKE, such as:
cmake -D CMAKE_C_COMPILER="/usr/bin/clang" -D CMAKE_CXX_COMPILER="/usr/bin/clang++" .
The exact value of the arguments depend of course of where the compilers are installed.
Binary downloads are supported on selected distributions of Linux.
Ithitools can be obtained through the COPR service on CentOS versions 6 and 7, and on Fedora versions 26, 27, 28 and Rawhide. The process has three steps:
- Subscribe to the ithitools project:
dnf copr enable chuitema/ithitools
- Install ithitools:
dnf install ithitools
- Once the package is installed, Linux will occasionally prompt for updates. This should be automatic, but just in case the command will be:
dnf update ithitools
On older systems, the "dnf" command is not supported -- use "yum" instead. These are system commands, so you may need to use "sudo" as appropriate.
Ithitools can be obtained through the Launchpad service for recent Ubuntu builds. The process has two steps:
- Subscribe to the ithitools project and get it:
sudo add-apt-repository ppa:chuitema/ppa
sudo apt-get update
- Install ithitools:
sudo apt-get install ithitools
- Updates should arrive as part of the regular updates for Ubuntu packages.
Sorry, but for the other distributions you will have to clone the sources from Github and compile.
- Clang static code analysis reports:
- issues found on 2018-12-13 01:57.40
- issues found on 2018-12-13 03:48.15
- issues found on 2018-12-13 16:52.58
- issues found on 2018-12-13 23:09.12
- issues found on 2018-12-15 21:57.21
- issues found on 2018-12-15 22:15.19
- issues found on 2019-02-28 03:12.45
- issues found on 2019-06-09 23:03.18
- issues found on 2019-06-09 23:08.04
- issues found on 2019-06-09 23:11.38
- issues found on 2019-06-09 23:32.46
- issues found on 2019-06-09 23:44.53
- issues found on 2019-06-15 02:19.25
- issues found on 2019-06-15 02:45.38
- issues found on 2019-06-26 22:56.49
- issues found on 2019-06-26 23:13.16
- issues found on 2019-08-07 21:52.10
- issues found on 2019-08-07 21:58.01
- issues found on 2019-08-07 23:03.48
- issues found on 2019-08-28 02:52.55
- issues found on 2019-08-28 03:15.24
- issues found on 2019-08-28 05:50.28
- issues found on 2019-08-28 06:28.31
- issues found on 2019-08-28 17:12.57
- issues found on 2019-08-28 18:16.33
- issues found on 2019-08-29 16:10.36
- issues found on 2019-08-29 16:29.02
- issues found on 2019-08-29 18:19.25
- issues found on 2019-08-29 22:06.00
- issues found on 2019-08-29 22:21.15
- issues found on 2019-09-07 06:19.07
- issues found on 2019-09-08 22:09.25
- issues found on 2019-11-28 01:27.01
- issues found on 2019-11-28 01:44.09
- issues found on 2020-05-02 00:07.45
- Code coverage from
ithitest
program:- from branch more-dnsstats-tables
- from branch tld-2-level
- from branch tally-only-if-not-dnscap
- from branch date-in-metric-line
- from branch ingest-m5-6
- from branch partner-check
- from branch kafka-prototype
- from branch tld-update-2020-05-01
- from branch explore-dga
- from branch export-partner
- from branch reduce-merge-memory
- from branch fix-enum-bug
- from branch explain-links
- from branch cbor-parse
- from branch cbor-debug-question
- from branch detect-anomaly
- from branch master
- from branch first-page-v2
- from branch stats-per-tld
- from branch crash-on-empty
- from branch DirectCapture
- from branch cdns-rfc
- from branch pygraphs
- from branch jumbo-column
- from branch fix-m56-definitions
- from branch Revisit-m46
- from branch cdns-module
- from branch l-root-to-imrs
- from branch addr-names-compress
- from branch tld-2ld-stats
- from branch parallel-count
- from branch centos-v-1-03
- from branch Fix-M1-bug
- from branch compact-first
- from branch fuzz-test-dev
- from branch odp-cron
- from branch isolate-privacy
- from branch tune-kafka
- from branch separate-a-e-files
- from branch cbor-bugs
- from branch web-site-bugs-2018-09
- from branch design-fixes
- from branch load-m3-bare
- from branch tld-leak-analyse
- from branch fix-tld-report
- from branch display-cur-ave-min-max
- from branch m2-www-bug
- from branch nb-gtld-in-m2
- Doxygen documentation (for master branch)