Linux DNS 查询剖析（第二部分）
DNS
![](/data/attachment/album/201808/24/232313rb2ozk8zbbb29bb6.jpeg)


在 [Linux DNS 查询剖析 - 第一部分](/article-9943-1.html) 中，我介绍了：


* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `ping` 与 `host` 查询方式对比


并且发现大多数程序选择要查询的 DNS 服务器时会参考 `/etc/resolv.conf` 配置文件。


这种方式在 Linux 上比较普遍<sup id="fnref1"> <a href="#fn1" rel="footnote">  1 </a></sup>。虽然我使用了特定的发行版 Ubuntu，但背后的原理与 Debian 甚至是那些基于 CentOS 的发行版有相通的地方；当然，与更低或更高的 Ubuntu 版本相比，差异还是存在的。


也就是说，接下来，你主机上的行为很可能与我描述的不一致。


在第二部分中，我将介绍 `resolv.conf` 的更新机制、`systemctl restart networking` 命令的运行机制 ，以及 `dhclient` 是如何参与其中。


### 1) 手动更新 /etc/resolv.conf


我们知道 `/etc/resolv.conf` （有极大的可能性）被用到，故你自然可以通过该文件增加一个 `nameserver`，那么主机也将会（与已有的 `nameserver` 一起）使用新加入的 `nameserver` 吧？


你可以尝试如下：



```
$ echo nameserver 10.10.10.10 >> /etc/resolv.conf
```

看上去新的 `nameserver` 已经加入：



```
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.3
search home
nameserver 10.10.10.10
```

但主机网络服务重启后问题出现了：



```
$ systemctl restart networking
$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.3
search home
```

我们的 `10.10.10.10` 的 `nameserver` 不见了！


在上一篇文章中我们忽略了这一点，本文进行补充说明。


### 2) resolvconf


你在 `/etc/resolv.conf` 文件中看到 `generated by resolvconf` 词组了吧？这就是我们的线索。


如果深入研究 `systemctl restart networking` 命令，你会发现它做了很多事情，结束时调用了 `/etc/network/if-up.d/000resolvconf` 脚本。在该脚本中，可以发现一次对 `resolvconf` 命令的调用：



```
/sbin/resolvconf -a "${IFACE}.${ADDRFAM}"
```

稍微研究一下 man 手册，发现`-a` 参数允许我们：



```
Add or overwrite the record IFACE.PROG then run the update scripts
if updating is enabled.
```

（增加或覆盖 IFACE.PROG 记录，如果开启更新选项，则运行更新脚本）


故而也许我们可以直接调用该命令增加 `namserver`：



```
echo 'nameserver 10.10.10.10' | /sbin/resolvconf -a enp0s8.inet
```

测试表明确实可以！



```
$ cat /etc/resolv.conf&nbsp; | grep nameserver
nameserver 10.0.2.3
nameserver 10.10.10.10
```

是否已经找到答案，这就是 `/etc/resolv.conf` 更新的逻辑？调用 `resolvconf` 将 `nameserver` 添加到某个地方的数据库，然后（“如果配置了更新”，先不管具体什么含义）更新 `resolv.conf` 文件。


并非如此。



```
$ systemctl restart networking
root@linuxdns1:/etc# cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.3
search home
```

呃！（网络服务重启后）新增的 `nameserver` 再次消失了。


可见，`systemctl restart networking` 不仅仅运行了 `resolvconf`，还在其它地方获取 `nameserver` 信息。具体是哪里呢？


### 3) ifup/ifdown


继续深入研究 `systemctl restart networking`，发现它完成了一系列工作：



```
cat /lib/systemd/system/networking.service
[...]
[Service]
Type=oneshot
EnvironmentFile=-/etc/default/networking
ExecStartPre=-/bin/sh -c '[ "$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment --list --exclude=lo)" ] && udevadm settle'
ExecStart=/sbin/ifup -a --read-environment
ExecStop=/sbin/ifdown -a --read-environment --exclude=lo
[...]
```

首先，网络服务的重启实质是运行一个<ruby> 单触发 <rt>  oneshot </rt></ruby>的脚本，脚本包含如下命令：



```
/sbin/ifdown -a --read-environment --exclude=lo
/bin/sh -c '[ "$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment --list --exclude=lo)" ] && udevadm settle'
/sbin/ifup -a --read-environment
```

第一行使用 `ifdown` 关闭全部的网络接口，但<ruby> 本地回环 <rt>  local, lo </rt></ruby>接口除外。<sup id="fnref2"> <a href="#fn2" rel="footnote">  2 </a></sup>


（LCTT 译注：其实这是因为很快就又启动了接口，间隔的时间没有超过 TCP 连接的超时时间，有人在评论中也做了类似回复）


第二行用于确认系统已经完成关闭网络接口相关的全部工作，以便下一步使用 `ifup` 启动接口。这也让我们了解到，网络服务实质运行的就是 `ifdown` 和 `ifup`。


文档中没有找到 `--read-environment` 参数的说明，该参数为 `systemctl` 正常工作所需。很多人以文档不完善为由不喜欢 `systemctl`。


很好。那么 `ifup` （和其成对出现的 `ifdown`） 到底做了哪些工作呢？长话短说，它运行了 `/etc/network/if-pre-up.d/` 和 `/etc/network/if-up.d/` 目录下的全部脚本；期间，这些脚本也可能会调用另外的脚本，依此类推。


其中一件工作就是运行了 `dhclient`，但我还不完全确定具体的机理，也许 `udev` 参与其中。


### 4) dhclient


`dhclient` 是一个程序，用于与 DHCP 服务器协商对应网络接口应该使用的 IP 地址的详细信息。同时，它也可以获取可用的 DNS 服务器并将其替换到 `/etc/resolv.conf` 中。


让我们开始跟踪并模拟它的行为，但仅在我实验虚拟机的 `enp0s3` 接口上。事先已经删除 `/etc/resolv.conf` 文件中的 nameserver 配置：



```
$ sed -i '/nameserver.*/d' /run/resolvconf/resolv.conf
$ cat /etc/resolv.conf | grep nameserver
$ dhclient -r enp0s3 && dhclient -v enp0s3
Killed old client process
Internet Systems Consortium DHCP Client 4.3.3
Copyright 2004-2015 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/enp0s8/08:00:27:1c:85:19
Sending on   LPF/enp0s8/08:00:27:1c:85:19
Sending on   Socket/fallback
DHCPDISCOVER on enp0s8 to 255.255.255.255 port 67 interval 3 (xid=0xf2f2513e)
DHCPREQUEST of 172.28.128.3 on enp0s8 to 255.255.255.255 port 67 (xid=0x3e51f2f2)
DHCPOFFER of 172.28.128.3 from 172.28.128.2
DHCPACK of 172.28.128.3 from 172.28.128.2
bound to 172.28.128.3 -- renewal in 519 seconds.

$ cat /etc/resolv.conf | grep nameserver
nameserver 10.0.2.3
```

可见这就是 `nameserver` 的来源。


但稍等一下，命令中的 `/run/resolvconf/resolv.conf` 是哪个文件，不应该是 `/etc/resolv.conf` 吗？


事实上，`/etc/resolv.conf` 并不一定只是一个普通文本文件。


在我的虚拟机上，它是一个软链接，指向位于 `/run/resolvconf` 目录下的“真实文件”。这也暗示了我们，该文件是在系统启动时生成的；同时，这也是该文件注释告诉我们不要直接修改该文件的原因。


（LCTT 译注：在 CentOS 7 中，没有 `resolvconf` 命令，`/etc/resolv.conf` 也不是软链接）


假如上面命令中 `sed` 命令直接处理 `/etc/resolv.conf` 文件，效果是不同的，会有警告消息告知待操作的文件不能是软链接（`sed -i` 无法很好的处理软链接，它只会创建一个新文件）。


（LCTT 译注：CentOS 7 测试时，`sed -i` 命令操作软链接并没有警告，但确实创建了新文件取代软链接）


如果你继续深入查看配置文件 `/etc/dhcp/dhclient.conf` 的 `supersede` 部分，你会发现 `dhclient` 可以覆盖 DHCP 提供的 DNS 服务器。


![linux-dns-2 (2)](/data/attachment/album/201808/24/232326asi1sbyjsylti4s7.png)


*（大致）准确的关系图*




---


### 第二部分的结束语


第二部分到此结束。信不信由你，这是一个某种程度上简化的流程版本，但我尽量保留重要和值得了解的部分，让你不会感到无趣。大部分内容都是围绕实际脚本的运行展开的。


但我们的工作还没有结束，在第三部分，我们会介绍这些之上的更多层次。


让我们简要列出我们已经介绍过的内容：


* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `/run/resolvconf/resolv.conf`
* `systemd` 和网络服务
* `ifup` 和 `ifdown`
* `dhclient`
* `resolvconf`




---


1. 事实上，这是相对于 POSIX 标准的，故不限于 Linux （我从上一篇文章的一条极好的[回复](https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/#comment-2312)中了解到这一点） [↩](#fnref1)
2. 我不明白为何这没有导致我例子中的 vagrant 会话中断 （有谁明白吗？）。 [↩](#fnref2)




---


via: <https://zwischenzugs.com/2018/06/18/anatomy-of-a-linux-dns-lookup-part-ii/>


作者：[ZWISCHENZUGS](https://zwischenzugs.com/)  
 译者：[pinewall](https://github.com/pinewall)  
 校对：[wxy](https://github.com/wxy)


本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出


OK
In [Anatomy of a Linux DNS Lookup – Part I](https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/) I covered:

`nsswitch`

`/etc/hosts`

`/etc/resolv.conf`

`ping`

vs`host`

style lookups

and determined that most programs reference `/etc/resolv.conf`

along the way to figuring out which DNS server to look up.

That stuff was more general linux behaviour **(*)** but here we move firmly into distribution-specific territory. I use ubuntu, but a lot of this will overlap with Debian and even CentOS-based distributions, and also differ from earlier or later Ubuntu versions.

###### (*) in fact, it’s subject to a POSIX standard, so

is not limited to Linux (I learned this from

a fantastic [comment](https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/#comment-2312) on the previous post)

In other words: your host is more likely to differ in its behaviour in specifics from here.

In Part II I’ll cover how `resolv.conf`

can get updated, what happens when `systemctl restart networking`

is run, and how `dhclient`

gets involved.

**Other posts in the series:**

[Anatomy of a Linux DNS Lookup – Part I](https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/)

[Anatomy of a Linux DNS Lookup – Part III](https://zwischenzugs.com/2018/07/06/anatomy-of-a-linux-dns-lookup-part-iii/)

[Anatomy of a Linux DNS Lookup – Part IV](https://zwischenzugs.com/2018/08/06/anatomy-of-a-linux-dns-lookup-part-iv/)

[Anatomy of a Linux DNS Lookup – Part V – Two Debug Nightmares](https://zwischenzugs.com/2018/09/13/anatomy-of-a-linux-dns-lookup-part-v-two-debug-nightmares/)

# 1) Updating /etc/resolv.conf by hand

We know that `/etc/resolv.conf`

is (highly likely to be) referenced, so surely you can just add a nameserver to that file, and then your host will use that nameserver in addition to the others, right?

If you try that:

`$ echo nameserver 10.10.10.10 >> /etc/resolv.conf`

it all looks good:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.0.2.3 search home nameserver 10.10.10.10

until the network is restarted:

$ systemctl restart networking $ cat /etc/resolv.conf # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.0.2.3 search home

our `10.10.10.10`

nameserver has gone!

This is where those comments we ignored in Part I come in…

# 2) resolvconf

You see the phrase `generated by resolvconf`

in the `/etc/resolv.conf`

file above? This is our clue.

If you dig into what `systemctl restart networking`

does, among many other things, it ends up calling a script: `/etc/network/if-up.d/000resolvconf`

. Within this script is a call to `resolvconf`

:

`/sbin/resolvconf -a "${IFACE}.${ADDRFAM}"`

A little digging through the man pages reveals that the `-a`

flag allows us to:

Add or overwrite the record IFACE.PROG then run the update scripts if updating is enabled.

So maybe we can call this directly to add a nameserver:

echo 'nameserver 10.10.10.10' | /sbin/resolvconf -a enp0s8.inet

Turns out we can!

$ cat /etc/resolv.conf | grep nameserver nameserver 10.0.2.3 nameserver 10.10.10.10

So we’re done now, right? This is how `/etc/resolv.conf`

gets updated? Calling `resolvconf`

adds it to a database somewhere, and then updates (if configured, whatever that means) the `resolv.conf`

file

No.

$ systemctl restart networking root@linuxdns1:/etc# cat /etc/resolv.conf # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.0.2.3 search home

Argh! It’s gone again.

So `systemctl restart networking`

does more than just run `resolvconf`

. It must be getting the nameserver information from somewhere else. Where?

# 3) ifup/ifdown

Digging further into what `systemctl restart networking`

does tells us a couple of things:

cat /lib/systemd/system/networking.service [...] [Service] Type=oneshot EnvironmentFile=-/etc/default/networking ExecStartPre=-/bin/sh -c '[ "$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment --list --exclude=lo)" ] && udevadm settle' ExecStart=/sbin/ifup -a --read-environment ExecStop=/sbin/ifdown -a --read-environment --exclude=lo [...]

First, the networking ‘service’ restart is actually a ‘oneshot’ script that runs these commands:

```
/sbin/ifdown -a --read-environment --exclude=lo
/bin/sh -c '[ "$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment --list --exclude=lo)" ] && udevadm settle'
/sbin/ifup -a --read-environment
```

The first line with `ifdown`

brings down all the network interfaces (but excludes the local interface). **(*)**

###### (*) I’m unclear why this doesn’t boot me out of my

vagrant session in my example code (anyone know?).

The second line makes sure the system has done all it needs to do regarding the bringing of network interfaces down before going ahead and bringing them all back up with `ifup`

in the third line. So the second thing we learn is that `ifup`

and `ifdown`

are what the networking service ‘actually’ runs.

The `--read-environment`

flag is undocumented, and is there so that `systemctl`

can play nice with it. A lot of people hate `systemctl`

for this kind of thing.

Great. So what does `ifup`

(and its twin, `ifdown`

) do? To cut another long story short, it runs all the scripts in `etc/network/if-pre-up.d/`

and `/etc/network/if-up.d/`

. These in turn might run other scripts, and so on.

One of the things it does (and I’m still not quite sure how – maybe `udev`

is involved?) `dhclient`

gets run.

# 4) dhclient

`dhclient`

is a program that interacts with DHCP servers to negotiate the details of what IP address the network interface specified should use. It also can receive a DNS nameserver to use, which then gets placed in the `/etc/resolv.conf`

.

Let’s cut to the chase and simulate what it does, but just on the `enp0s3`

interface on my example VM, having first removed the nameserver from the `/etc/resolv.conf`

file:

$ sed -i '/nameserver.*/d' /run/resolvconf/resolv.conf $ cat /etc/resolv.conf | grep nameserver $ dhclient -r enp0s3 && dhclient -v enp0s3Killed old client process Internet Systems Consortium DHCP Client 4.3.3 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/enp0s8/08:00:27:1c:85:19 Sending on LPF/enp0s8/08:00:27:1c:85:19 Sending on Socket/fallback DHCPDISCOVER on enp0s8 to 255.255.255.255 port 67 interval 3 (xid=0xf2f2513e) DHCPREQUEST of 172.28.128.3 on enp0s8 to 255.255.255.255 port 67 (xid=0x3e51f2f2) DHCPOFFER of 172.28.128.3 from 172.28.128.2 DHCPACK of 172.28.128.3 from 172.28.128.2 bound to 172.28.128.3 -- renewal in 519 seconds.$ cat /etc/resolv.conf | grep nameservernameserver 10.0.2.3

So that’s where the nameserver comes from…

But hang on a sec – what’s that `/run/resolvconf/resolv.conf`

doing there, when it should be `/etc/resolv.conf`

?

Well, it turns out that `/etc/resolv.conf`

isn’t always ‘just’ a file.

On my VM, it’s a symlink to the ‘real’ file stored in `/run/resolvconf`

. This is a clue that the file is constructed at run time, and one of the reasons we’re told not to edit the file directly.

If the `sed`

command above were to be run on the `/etc/resolv.conf`

file directly then the behaviour above would be different and a warning thrown about `/etc/resolv.conf`

not being a symlink (`sed -i`

doesn’t handle symlinks cleverly – it just creates a fresh file).

`dhclient`

offers the capability to override the DNS server given to you by DHCP if you dig a bit deeper into the `supersede`

setting in `/etc/dhcp/dhclient.conf`

…

**A (roughly) accurate map of what’s going on**

## End of Part II

That’s the end of Part II. Believe it or not that was a somewhat simplified version of what goes on, but I tried to keep it to the important and ‘useful to know’ stuff so you wouldn’t fall asleep. Most of that detail is around the twists and turns of the scripts that actually get run.

And we’re still not done yet. Part III will look at even more layers on top of these.

Let’s briefly list some of the things we’ve come across so far:

`nsswitch`

`/etc/hosts`

`/etc/resolv.conf`

`/run/resolvconf/resolv.conf`

`systemd`

and its`networking`

service`ifup`

and`ifdown`

`dhclient`

`resolvconf`


**If you like this, you might like one of my books:
**

Is there something like this article for how dns Is traversed in Kubernetes? KubeDNS vs CoreDNS etc?

There’s another article I wrote a while ago on DNS in OpenShift that might be relevant while you wait…

cat | grep? you can just grep, man! Nice write up!

You’re not “booted” because the commands happen in rapid succession — fast enough that your TCP connection remained intact (aside from one retransmitted packet).

If you were to put a ‘sleep 20’ in there, you’d likely get booted.

“The first line with ifup brings down all the network interfaces” shouldn’t this be ifdown?

You should write about local dns caching in part 3. It can be rather weird with timeouts, multiple retires, dual requests due to ipv4+6 etc.

Thanks – do you have useful resources on that?

Have you checked Ubuntu 18.04, for instance?

Now enters systemd-resolved to add even more to this complexity…

It is really a hell to get it to work fine with VPN, and to debug…

Dealing with an 18.04 bug that spiked cpu because systemd-resolved was fighting dnsmasq was the trigger for me to actually unpick this mess!

I also had a hard time trying to debug an issue with systemd-resolved… complexity and subtle bugs, a real pain for something as critical as name resolution.

And, great article by the way. Thanks !

once upon a time, dnsmasq plays a role in generating resolv.conf.

Hi Ian,

Thanks for your articles.

How did you find that “systemctl restart networking does, among many other things, it ends up calling a script:

/etc/network/if-up.d/000resolvconf” ?

Tried strace but only had either WSL (no no systemd) or Linux 18.04 which seems a little bit different.

Many thanks

strace won’t help you – I think I did a system-wide check of which files were opened while the command was running using sysdig I think

Thanks Ian, I will have a check.

Unrelated, but I see it all too often: These two are equivalent:

cat /etc/resolv.conf | grep nameserver

grep nameserver /etc/resolv.conf #no need to cut and pipe

Cheers

^^ “So systemctl restart networking does more than just run resolvconf. It must be getting the nameserver information from somewhere else. Where?” ^^

On Debian Systems take a look at /etc/network/interfaces for local network configuration.

Either a “nameserver” statement or “dhpc” advise if-up which nameserver to put in resolv.conf.