Maint. Log Archive
A log of changes to the burble.dn42 network.
As you may have noticed, the maintenance log is no longer being kept up to date !
A new dedicated server us-nyc1 has joined burble.dn42, hosted in Virmach’s Buffalo, NY datacentre.
The server is not open for peering, however it does host a shell server that you are free to use. See the Shell Accounts service page for more info.
The shell servers now have apache2 installed to provide home directory
public_html access.
See the Shell Accounts service page
for more info.
Added shell.ca-bhs2.burble.dn42 as a new shell server.
Added new Shell Accounts service
dn42regsrv now supports publishing ROA in OpenBGPD format.
The ROA are now published at the following links:
- https://dn42.burble.com/roa/dn42_roa_obgpd_46.conf
- https://dn42.burble.com/roa/dn42_roa_obgpd_4.conf
- https://dn42.burble.com/roa/dn42_roa_obgpd_6.conf
us-nyc1 and us-chi1 have been removed from service and decommissioned.
If you were peered on these nodes, please contact me if you wish to re-peer on a different node.
The b.recursive-servers.dn42 DNS resovler is running an experimental build
of pdns-recursor to test a fix of this issue.
Please let me know if you spot any strange problems.
burble.dn42 websites are now using a TLS certificate issued by the DN42 ACME service.
A number of significant changes have been implemented for the global route collector
- Downstream peerings have been stopped, in favour of parsing the MRT dumps
- The collector has moved from de-fra1 to fr-rbx1, where bandwidth is no issue
- A special routing policy has been implemented for the collector to encourage traffic to go directly to fr-rbx1 and not transit through burble.dn42 nodes. See also the Routing Policy page.
- Internal rate limits on BGP sessions have been relaxed
The collector is now using a TLS certificate issued by the DN42 ACME service. The collector is behind an anycasted reverse proxy, so a normal ACME challenge will not work. Instead, the certificate is managed using dnscontrol to respond to an ACME DNS challenge.
DNSSEC has been enabled on all edge nodes.
There was a major DNS outage today as a minor change took out the entire service.
What should have been a trivial config change actually upgraded the container from Alpine 3.11 to Alpine 3.13 and caused a number of the DNS applications to stop working due to incompatibilities.
The lack of working DNS meant it was more complicated to bootstrap the service back again, leading to a long delay in restoring service.
Fixed a bug in bird that was preventing MRT dumps from the collector working. Hopefully the dumps can now be successfully parsed: https://mrt.collector.dn42
Bird 2.0.8 has been deployed across the network. Please let me know if you see problems.
burble.dn42 uses a custom bird build that includes additional debugging. The source code for the build is available on git.burble.dn42.
Advanced Notice
- us-nyc1 will be decommissioned before 15/04/21
- us-chi1 will be decommissioned before 14/05/21
Updated IPv6 address for hk-hkg1
Upgraded the looking glass to use bird-lg-go.
The main benefit of the go version is that it executes queries in parallel, greatly improving response times with a large number of nodes.
hk-hkg1 is now open for IPv4 peering; see the node information for details.
IPv6 connectivity is expected ~February.
Happy New Year DN42.
The new year brings a new website for burble.dn42 built using Hugo and statically delivered from each core node for speed. As always, the source for the website is available in the gitea repo.
The MTU for anycast services has been reduced to 1280 after a problem was seen with IPv6 path MTU discovery.
The problem was due to an asymmetric path, where a request to the wiki went to one node but the return path was via a different node. The other node also hosted a wiki instance, which meant that pmtud ICMP messages on the return path were being picked up by the wrong node. To fix this, the MTU has been clamped to the minimum allowable size of 1280.
Interestingly, Cloudflare also recognised the same type of issue and wrote up what they did in their blog.
The following services were impacted by the changes.
- DNS Services
- NGINX Reverse Proxy (and therefore also all websites, including the Wiki mirrors)
- WHOIS Service
es-mad1 in Madrid, Spain has already been delivered and is now open for peerings.
The new node in Hong Kong, hk-hkg1 has also been delivered and I’m now just waiting for IPv6 to be available before it too will also be ready for peering.