307 lines
14 KiB
Markdown
307 lines
14 KiB
Markdown
# Transdep
|
|
|
|
Transdep is a utility to discover single points of failure (SPOF) in DNS dependency graphs leading to unability to
|
|
resolve domain names.
|
|
|
|
DNS dependency graph is a notion that was introduced by Venugopalan Ramasubramanian and Emin Gün Sirer in
|
|
[Perils of Transitive Trust in the Domain Name System][1].
|
|
|
|
Current types of single points of failure that are detected are :
|
|
- domain names (which can be availability SPOF if DNSSEC is incorrectly configured);
|
|
- IP addresses of name servers;
|
|
- Longest network prefixes that may generally be announced over the Internet (/24 over IPv4 and /48 over IPv6);
|
|
- ASN of the AS announcing the IP addresses of name servers.
|
|
|
|
The ``transdep`` utility is the CLI version of the tool. The ``webserver`` utility spawns a REST/JSON webservice.
|
|
Endpoints are described below.
|
|
|
|
[1]: https://www.cs.cornell.edu/people/egs/papers/dnssurvey.pdf
|
|
|
|
## Licence
|
|
|
|
Transdep is licenced under the 2-clause BSD licence.
|
|
|
|
## Installation
|
|
|
|
Transdep uses the following external libraries:
|
|
- https://github.com/miekg/dns
|
|
- https://github.com/awalterschulze/gographviz
|
|
- https://github.com/hashicorp/golang-lru
|
|
- https://github.com/deckarep/golang-set
|
|
- https://github.com/hashicorp/go-immutable-radix
|
|
|
|
You may install them using the go get command or whichever other method you prefer:
|
|
|
|
```bash
|
|
$ go get github.com/miekg/dns
|
|
$ go get github.com/awalterschulze/gographviz
|
|
$ go get github.com/hashicorp/golang-lru
|
|
$ go get github.com/deckarep/golang-set
|
|
$ go get github.com/hashicorp/go-immutable-radix
|
|
```
|
|
|
|
You may then use the Makefile to compile the Transdep tools:
|
|
|
|
```bash
|
|
$ make all
|
|
```
|
|
|
|
## Usage
|
|
|
|
### CLI
|
|
|
|
The ``transdep`` utility can be used to analyze the dependencies of a single domain name, of multiple names, or a saved
|
|
dependency graph.
|
|
|
|
#### Analysis Target Types
|
|
To analyze a single name, the ``-domain`` option is to be used:
|
|
|
|
```bash
|
|
./transdep -domain www.example.net
|
|
```
|
|
|
|
To analyze multiple domain names, you must provide a list stored in a file with one domain name per line, with the
|
|
option ``-file``:
|
|
|
|
```bash
|
|
./transdep -file <(echo -ne "example.com\nexample.net")
|
|
|
|
./transdep -file /tmp/list_of_domain_names
|
|
```
|
|
|
|
If you saved a dependency graph into a file (that was generated using the ``-graph`` option ), you may analyze it by
|
|
loading the graph with the ``-load`` option:
|
|
|
|
```bash
|
|
./transdep -domain example.net -graph > /tmp/example_net_graph.json
|
|
|
|
./transdep -load /tmp/example_net_graph.json
|
|
```
|
|
|
|
#### Analysis Nature
|
|
|
|
Transdep can analyze a domain based on multiple criteria.
|
|
|
|
All analysis types consider that IP addresses and announcing network prefixes may be SPOF.
|
|
|
|
By default, SPOF discovery is conducted while considering that all names may break, including non-DNSSEC protected
|
|
domain names. This is used to analyze SPOF in the event of misconfigurations, zone truncation and all other types of
|
|
zone corruptions that may render a zone impossible to resolve.
|
|
|
|
If the analysis must be constrained to only consider that DNSSEC protected names may break, the ``-dnssec`` option must
|
|
be added to the command line:
|
|
|
|
```bash
|
|
./transdep -domain www.example.com -dnssec
|
|
```
|
|
|
|
By default, the SPOF discovery considers that resolvers are connected to both IPv4 and IPv6 networks. This means that if
|
|
an IPv4 address is unavailable, this unavailibility may be covered for by a server available over IPv6.
|
|
|
|
In some scenarii, this is unacceptable, because the IPv4 resolvers and the IPv6 resolvers are separate servers. Also,
|
|
one of these two networks might be unavailable (temporarily or permanently). To represent these situations, the
|
|
``-break4`` (resp. ``-break6``) options simulates that all IPv4 (resp. IPv6) addresses are always considered unavailable
|
|
when analyzing the SPOF potential of an IP address in the other network type:
|
|
|
|
```bash
|
|
./transdep -domain www.x-cli.eu -break4
|
|
www.x-cli.eu:this domain name requires some IPv4 addresses to be resolved properly
|
|
|
|
|
|
./transdep -domain www.example.com -break4
|
|
www.example.com.:Name:example.com.
|
|
www.example.com.:IP:2001:500:8d::53
|
|
www.example.com.:Name:iana-servers.net.
|
|
www.example.com.:Name:.
|
|
www.example.com.:Name:net.
|
|
www.example.com.:Name:com.
|
|
```
|
|
|
|
In the previous example, `www.x-cli.eu.` cannot be resolved by IPv6-only resolvers (because some names or delegations do
|
|
not have IPv6 addresses).
|
|
For `www.example.com`, the result shows that during that run, `Transdep` detected that when using a resolver that
|
|
has only access to the IPv6 network at the time of resolution, the name `www.example.com` might not be possible to
|
|
resolve if the IP address ``2001:500:8d::53`` is unavailable.
|
|
|
|
The ``-all`` option indicates to `Transdep` to analyze the requested domain name(s), using all possible compositions of
|
|
the previous options: with and without ``-dnssec``, and with and without ``-break4`` or with and without ``-break6``.
|
|
|
|
```bash
|
|
./transdep -domain www.x-cli.eu -all
|
|
AllNames:www.x-cli.eu.:Name:x-cli.eu.
|
|
AllNames:www.x-cli.eu.:Name:.
|
|
AllNames:www.x-cli.eu.:Name:eu.
|
|
DNSSEC:www.x-cli.eu.:Name:.
|
|
DNSSEC:www.x-cli.eu.:Name:eu.
|
|
AllNamesNo4:www.x-cli.eu.:this domain name requires some IPv4 addresses to be resolved properly
|
|
DNSSECNo4:www.x-cli.eu.:this domain name requires some IPv4 addresses to be resolved properly
|
|
AllNamesNo6:www.x-cli.eu.:Name:eu.
|
|
AllNamesNo6:www.x-cli.eu.:Name:x-cli.eu.
|
|
AllNamesNo6:www.x-cli.eu.:Name:.
|
|
DNSSECNo6:www.x-cli.eu.:Name:.
|
|
DNSSECNo6:www.x-cli.eu.:Name:eu.
|
|
```
|
|
|
|
`Transdep` may also consider an analysis criterion based on the ASN of the AS announcing the network prefixes covering
|
|
the IP addresses of the name servers. The association between the ASN and the IP address is done by using a file whose
|
|
format is as follows:
|
|
- one association per line;
|
|
- each line contains an ASN and an announced network prefix.
|
|
|
|
Here is an example of such a file:
|
|
```
|
|
64501 192.0.2.0/24
|
|
64501 198.51.100.0/24
|
|
64502 203.0.113.0/24
|
|
64502 2001:db8::/32
|
|
```
|
|
|
|
Such a file can be generated from a MRT dump file (bviews) such as the ones made available by the [RIS project][2],
|
|
using ANSSI's [`mabo`][3] tool with the sub-command ``prefixes``.
|
|
|
|
The ASN-prefix file is provided to `Transdep` using the ``-mabo`` option:
|
|
|
|
```bash
|
|
./mabo prefixes bview.20171013.0800.gz > prefixes-20171013.txt
|
|
./transdep -domain www.example.com -mabo prefixes-20171013.txt
|
|
```
|
|
|
|
[2]: https://www.ripe.net/analyse/internet-measurements/routing-information-service-ris
|
|
[3]: https://github.com/ANSSI-FR/mabo
|
|
|
|
#### Output Types
|
|
|
|
`Transdep` can generate several types of documents. By default, it generates a CSV containing the discovered SPOF for
|
|
the requested analysis.
|
|
|
|
If the ``-all`` option is provided, the format is ``AnalysisType:DomainName:TypeOfSPOF:SPOFReference``, where
|
|
``AnalysisType`` indicates one of the following combinations:
|
|
* ``AllNames``: default options (no ``-dnssec``, no ``-break4``, no ``-break6``);
|
|
* ``AllNamesNo4``: default options except that ``-break4`` is specified;
|
|
* ``AllNamesNo6``: default options except that ``-break6`` is specified;
|
|
* ``DNSSEC``: default options except that ``-dnssec`` is specified;
|
|
* ``DNSSECNo4``: ``-dnssec`` and ``-break4`` options are specified;
|
|
* ``DNSSECNo6``: ``-dnssec`` and ``-break6`` options are specified.
|
|
|
|
If the ``-all`` option is not specified, the format is ``DomainName:TypeOfSPOF:SPOFRerefence``.
|
|
|
|
In both formats, ``DomainName`` indicates the domain name that is analyzed.
|
|
|
|
``TypeOfSPOF`` can value:
|
|
* ``Name``: the next field specifies a domain name that must be resolvable for ``DomainName`` to be resolvable.
|
|
* ``IP``: the next field specifies an IP address that must be available and not hijacked for ``DomainName`` to be
|
|
resolvable.
|
|
* ``Prefix:``: the next field specifies a network prefix that must be available and not hijacked for ``DomainName`` to
|
|
be resolvable.
|
|
* ``ASN:``: the next field specifies an AS number whose whole network must not be totally broken for ``DomainName`` to
|
|
be resolvable.
|
|
|
|
TypeOfSPOF may also value a special value: ``Cycle``. ``Cycle`` indicates that there is a circular dependency in the
|
|
graph somewhere, or an overly long CNAME chain (for some definition of "overly long").
|
|
Having ``Cycle`` as dependency means that the name cannot be resolved at all using a RFC-compliant resolver at the time
|
|
of resolution.
|
|
|
|
The ``-graph`` output option generates an output that can be later loaded for analysis using the
|
|
``-load`` option, described above.
|
|
|
|
The ``-dot`` output option generates a DOT file output. This output may be passed to any Graphviz interpret for graph
|
|
drawing. The generated DOT file will highlight domain name and IP addresses that are SPOF by coloring the nodes in red.
|
|
|
|
```bash
|
|
./transdep -domain www.x-cli.eu | dot -T pdf -o /tmp/graph_x-cli.eu.pdf
|
|
```
|
|
|
|
#### Caches
|
|
|
|
`Transdep` maintains several caches in order to limit the number of requests to name servers during the discovery of the
|
|
dependency graph. There are in-memory caches, using LRU lists and go routines, and on-disk caches for long term cache
|
|
and to store value overflowing from the in-memory LRU lists.
|
|
|
|
In-memory cache sizes are controlled with the ``-nrlrusize``, ``-zcflrusize`` and ``-dflrusize`` options. The first two
|
|
options are associated with lists that contain data that is cached on disk when the LRU lists are overflowing.
|
|
The on-disk cache is leverage whenever possible and the entry is reinstated in the LRU list upon usage. Thus, an entry
|
|
is either in-memory or on-disk and is never lost unless the cache directoy is flushed manually. The third option is
|
|
associated with a LRU list whose entries may be very large. These entries are synthetized from the entries of the other
|
|
caches, and thus are not stored on disk when the list is overflowing.
|
|
|
|
If your computer swaps or consumes too much memory while running `Transdep`, you should try to lower these values,
|
|
trying to lower ``-dflrusize`` value first. If your computer spends too much time in "disk I/O wait" and you have
|
|
some RAM capacity available, you may try to increase the two first options.
|
|
|
|
On-disk caches consist of a lot very small JSON files. Please monitor the number of remaining inodes and adapt your
|
|
inode table accordingly.
|
|
|
|
On-disk caches are stored in the directory designated by the `TMPDIR` environment variable, the `-cachedir` command line
|
|
option. The default value is ``/tmp``.
|
|
|
|
`Transdep` caches never expire, with the current implementation. If you need to flush the cache, you may change the
|
|
cache directory to keep the previous one and yet start fresh. You may also delete the `nameresolver` and `zonecut`
|
|
directories that are present in the designated cache directory.
|
|
|
|
#### Root Zone Hint File
|
|
|
|
You may specify a root zone hint file with the `-hints` option. If left unspecified, a hard-coded list of root-servers
|
|
will be used by `Transdep`, when querying the root zone for delegations.
|
|
|
|
#### DNS Violations
|
|
|
|
Using a RFC-compliant implementation prevents you from resolving many domain names. Thus, some degree of DNS violation
|
|
tolerance was implemented in `Transdep`, with much grumble.
|
|
By default, `Transdep` will consider `rcode 3` status on non-terminal nodes equivalent to `rcode 0` answers with
|
|
`ancount=0`. You may reinstate RFC8020 compliance with the `-rfc8020` option.
|
|
|
|
Some devices are also unable to answer to non-A/AAAA queries and always return `rcode 2` answers for any other qtype,
|
|
including NS or DS. By default, `Transdep` considers this servers as broken, but you may use the `-servfail` option to
|
|
indicate `Transdep` to treat these answers as `rcode 0` answers with `ancount=0`. This may lead `Transdep` to return
|
|
incorrect results in some instances.
|
|
|
|
#### Script Friendliness
|
|
|
|
If you don't care about the nature of errors that may arise during the analysis of a domain name or if you want to have
|
|
a output that is easily parsable, you may use the `-script` option to return errors as the constant ``-ERROR-``.
|
|
|
|
`Transdep` will return an error if any name that is part of the dependency graph cannot be resolved at the time of
|
|
dependency graph discovery. Doing otherwise might have led to incorrect results from partial dependency graph discovery.
|
|
|
|
#### Concurrency
|
|
|
|
You may adapt the number of domain names whose dependency graphs are discovered simultaneously with the `-jobs` option.
|
|
The higher this option value, the more you will harass the name servers. You will want to keep this value relatively low,
|
|
to prevent blacklisting of your IP and false measurements.
|
|
|
|
### Web Service
|
|
|
|
The webservice uses the ``webserver`` binary.
|
|
|
|
The ``-bind`` and ``-port`` can be used to specify, respectively, on which address and port the web server should listen
|
|
on. By default, the service is available on `http://127.0.0.1:5000`.
|
|
|
|
The ``-nrlrusize``, ``-zcflrusize``, ``-dflrusize``, ``-jobs``, ``-hints`` and ``-cachedir`` options have the same usage
|
|
as for the `Transdep` CLI utility.
|
|
|
|
The web server exposes several endpoints:
|
|
* ``/allnames`` is the endpoint corresponding to the default behaviour of the `transdep` CLI utility.
|
|
* ``/dnssec`` is the endpoint corresponding to the ``-dnssec`` option of the `transdep` CLI utility.
|
|
* ``/break4`` is the endpoint corresponding to the ``-break4`` option of the `transdep` CLI utility.
|
|
* ``/break6`` is the endpoint corresponding to the ``-break6`` option of the `transdep` CLI utility.
|
|
|
|
Combination of ``-dnssec`` and ``-break4`` or ``-break6`` is not possible with the web server.
|
|
|
|
Each endpoint takes a ``domain`` parameter as part of the query string, to specify which domain name is to be analyzed.
|
|
|
|
Endpoints may also receive ``rfc8020`` and ``servfail`` query string parameters to indicate which DNS violations are
|
|
tolerated for this analysis. If these options are not specified, `rcode 3` answers on non-terminal nodes are treated as
|
|
`rcode 0` answers with `ancount=0` and `rcode 2` answers are considered as broken name servers.
|
|
|
|
When launched from a console, the `webserver` utility outputs a URL to query to gracefully stop the service. Gracefully
|
|
shutting down the service is strongly advised to prevent on-disk cache corruption or incompleteness.
|
|
|
|
```bash
|
|
$ ./webserver &
|
|
[1] 7416
|
|
To stop the server, send a query to http://127.0.0.1:5000/stop?secret=5942985ebdc9102663130752c1d21f23
|
|
$ curl http://127.0.0.1:5000/stop?secret=5942985ebdc9102663130752c1d21f23
|
|
Stopping.
|
|
Stopping the finder: OK
|
|
$
|
|
```
|