I moved my domain’s nameservers to Cloudflare. The apex, www and four subdomains kept working. Sixteen other subdomains, all apps on the same VPS, started returning NXDOMAIN.
Nothing was wrong with the server. Every one of those apps still answered when I sent the request straight to the VPS address. The new zone only had six records, and my old zone had no wildcard record, so every missing subdomain simply did not exist in DNS any more.
The fix is to add one A record per subdomain. I wrote them as a single zone file for Cloudflare’s import. This post covers how I told a DNS problem from an app problem, the import file, and the audit I should have run before switching nameservers.
The symptom
All the subdomains point at one VPS. Each has its own nginx site and its own certbot certificate. After the nameserver change:
- 6 hostnames resolved: the apex,
wwwand four subdomains - 16 hostnames returned
NXDOMAIN
NXDOMAIN is a useful error. It does not mean the server is down or the certificate is bad. It means the DNS zone has no record with that name. The request never reaches your server.
Proving the apps were fine
When a batch of sites goes dark at once, the first guess is that the server or nginx is broken. Check that before touching DNS, and check it without using DNS.
curl --resolve connects to an address you choose and still sends the right hostname for TLS and for nginx:
curl -sS -I --resolve app.example.com:443:203.0.113.10 https://app.example.comI ran this for the missing hostnames. Every app answered on port 443 of the VPS. So nginx, the certificates and the containers were all fine, and only the DNS records were gone.
To see what DNS itself says, ask a public resolver for the record:
Resolve-DnsName app.example.com -Type A -Server 1.1.1.1For the six working names this returns an address. For the other sixteen it fails with a name-does-not-exist error.
The two checks together separate the cases:
| Direct request to the server | DNS lookup | Problem is in |
|---|---|---|
| Works | Fails | DNS zone |
| Fails | Works | Server or app |
Why the records were missing
Two facts combined.
The new zone only had some of the records. When I looked at the Cloudflare zone, it held records for the apex, www and four subdomains. I do not know exactly why the rest were not carried over. The zone was moved around the same time I moved the apex off another host, and only some records got recreated. I did not compare the old and new zones before switching nameservers, so I did not notice.
My zone never had a wildcard record. I had learned this earlier the hard way. When I added a new subdomain a week before, it did not resolve until I created an explicit A record for it, and certbot could not issue its certificate before that. There is no *.example.com record that catches everything. Each subdomain has its own A record pointing at the VPS.
With a wildcard, a missing record falls through to the wildcard and the site keeps working. Without one, a missing record is NXDOMAIN. I prefer explicit records, but it means a zone move has to carry every single one.
The fix: one import file
Adding sixteen records by hand in a dashboard is slow and easy to get wrong. Cloudflare accepts a BIND zone file under DNS > Records > Import and Export. See Import and export records in the Cloudflare docs.
This is the file I wrote, trimmed and with the names changed. The real one has sixteen record lines.
;; BIND zone import for example.com (Cloudflare: DNS > Records > Import and Export)
;; Adds only the missing VPS subdomains. Existing records are left out to avoid duplicates.
;; Proxied (orange cloud) to hide the origin IP; files-console stays DNS-only (100MB upload cap through the proxy). Set SSL/TLS mode to Full.
$ORIGIN example.com.
$TTL 300
app.example.com. 300 IN A 203.0.113.10 ; cf_tags=cf-proxied:true
docs.example.com. 300 IN A 203.0.113.10 ; cf_tags=cf-proxied:true
api.example.com. 300 IN A 203.0.113.10 ; cf_tags=cf-proxied:true
files-console.example.com. 300 IN A 203.0.113.10 ; cf_tags=cf-proxied:false
staging.example.com. 300 IN A 203.0.113.10 ; cf_tags=cf-proxied:trueA few things about it.
Leave out records that already exist. The file contains only the missing names. The six records already in the zone are not repeated, so the import cannot create duplicates.
The proxy setting is in a comment. cf_tags=cf-proxied:true after the semicolon marks the record as proxied. cf-proxied:false marks it as DNS-only. This is the same format Cloudflare uses in its own zone exports.
Not everything should be proxied. I proxy most records so the public DNS answer is a Cloudflare address and not my server’s. The exception is anything that takes large uploads. Cloudflare’s proxy caps uploads at 100 MB on my plan, and one of my subdomains is the web console for self-hosted object storage. That record is DNS-only. The storage API hostname needs the same treatment. Its record was one of the six that already existed, and it was proxied, so it has to be switched in the dashboard.
Set SSL/TLS mode to Full. My server already has a certbot certificate for each subdomain. In Full mode Cloudflare connects to the origin over HTTPS, which matches that setup.
A side effect: SSH by hostname
With proxied records, a hostname resolves to Cloudflare, not to the VPS. Cloudflare’s proxy handles web traffic. It does not pass SSH through.
So SSH has to go straight to the server’s IP address, and my SSH config uses the IP. If you have a deploy pipeline that connects over SSH, check that its host setting is the IP address or a DNS-only name, not a proxied one.
The audit I should have run first
The outage was avoidable. Before changing nameservers, the new zone should have been checked against what the server actually serves.
1. Get the list of hostnames from the server, not from memory. On my VPS every subdomain has a site file in /etc/nginx/sites-enabled. That directory is the real inventory. With the apex and www I had 22 hostnames, and I would not have listed them all from memory.
2. Export the old zone. Most DNS providers can export a zone file. Keep it. It is also your rollback reference.
3. Compare both lists with the new zone. Every hostname that nginx serves needs a record in the new zone. Anything in the old zone that is not in the new one needs a decision: recreate it or drop it on purpose.
4. Query the new nameservers directly before switching. The new provider’s nameservers will answer for your zone before the registrar points at them. Ask them for each hostname:
Resolve-DnsName app.example.com -Type A -Server <new-nameserver>Any name that fails here will fail for everyone after the switch.
5. After the switch, test through real DNS. A --resolve test proves the server works. It cannot find a missing record, because it never asks DNS.
What I do now
When one of my subdomains stops loading, I check the DNS zone before I look at the app. It takes one lookup, and in this case it was the whole problem for sixteen sites at once.