IPv6 only networks not working -- looks like a DNS issue (alpha 3)

[sgryphon]

Board

m5stack-core2

Device Description

M5Stack Core2 ESP32 IoT Development Kit for AWS IoT Kit, https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit-for-aws-iot-edukit?variant=37687799251116

Hardware Configuration

Just the M5Stack Core2

Version

latest master (checkout manually)

IDE Name

PlatformIO

Operating System

Ubuntu 22.04

Flash frequency

240Mhz

PSRAM enabled

yes

Upload speed

115200

Description

I have been testing out the alpha 3 master with the initial IPv6 code now merged in.

I have got it working on my M5Stack Core2 device for dual-stack, but on my IPv6-only network it can't make HTTP connections and is reporting a DNS issue. It does not list my IPv6 DNS server.

With IPv6 there are two available DNS mechanisms, RDNSS where DNS servers are part of Router Advertisement (RA) options, and DHCPv6 stateless, where RA is used for address prefixes and DHCPv6 is use for DNS only.

The relevant options in LWIP look to be CONFIG_LWIP_IPV6_RDNSS_MAX_DNS_SERVERS and CONFIG_LWIP_IPV6_DHCP6

Running on my IPv6 only network, it connects to the network and gets IPv6 addresses (I have multiple prefixes in RA, so it gets one general global and one ULA, as well as the link-local), but DNS is empty so HTTP isn't working:

[ 1682][D][WiFiNetworkManager.cpp:34] wifiOnEvent(): [Network] WiFi station start
[ 1700][D][WiFiNetworkManager.cpp:94] loop(): [Network] WiFi begin status 6
[ 1821][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA connected
[ 3583][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 fe80:0000:0000:0000:0a3a:f2ff:fe65:db28
[ 3597][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 2
[ 4585][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 2407:8800:bc61:1330:0a3a:f2ff:fe65:db28
[ 4599][I][Core2Logger.cpp:176] success(): [Core2Logger] Success
[ 4605][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 1
[ 4620][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 fd7c:e25e:67e8:0030:0a3a:f2ff:fe65:db28
[ 4635][I][Core2Logger.cpp:176] success(): [Core2Logger] Success
[ 4641][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 4

[ 18509][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Button 1, scenario 0, v0.1.0-150-g27c8f84-dev
[ 18523][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Global IPv6 2407:8800:bc61:1330:a3a:f2ff:fe65:db28
[ 18537][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: IPv4 0.0.0.0
[ 18545][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Link-Local IPv6 fe80::a3a:f2ff:fe65:db28%st1
[ 18558][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS0 0.0.0.0
[ 18566][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS1 0.0.0.0
[ 18574][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: URL: http://v4v6.ipv6-test.com/api/myip.php
[ 18605][D][WiFiGeneric.cpp:1649] hostByName(): Clearing DNS cache
[ 18611][E][WiFiGeneric.cpp:1678] hostByName(): DNS Failed for 'v4v6.ipv6-test.com' with error '-6' and result '0'
[ 18622][E][WiFiClient.cpp:258] connect(): connect on fd 48, errno: 118, "Host is unreachable"
[ 18631][D][HTTPClient.cpp:1163] connect(): failed connect to v4v6.ipv6-test.com:80
[ 18638][W][HTTPClient.cpp:1486] returnError(): error(-1): connection refused
[ 18645][E][Core2Logger.cpp:192] log(): [Core2Logger] CORE2: HTTP GET error -1: connection refused

Sketch

TBA

Debug Message

[ 18509][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Button 1, scenario 0, v0.1.0-150-g27c8f84-dev
[ 18523][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Global IPv6 2407:8800:bc61:1330:a3a:f2ff:fe65:db28
[ 18537][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: IPv4 0.0.0.0
[ 18545][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Link-Local IPv6 fe80::a3a:f2ff:fe65:db28%st1
[ 18558][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS0 0.0.0.0
[ 18566][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS1 0.0.0.0
[ 18574][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: URL: http://v4v6.ipv6-test.com/api/myip.php
[ 18605][D][WiFiGeneric.cpp:1649] hostByName(): Clearing DNS cache
[ 18611][E][WiFiGeneric.cpp:1678] hostByName(): DNS Failed for 'v4v6.ipv6-test.com' with error '-6' and result '0'
[ 18622][E][WiFiClient.cpp:258] connect(): connect on fd 48, errno: 118, "Host is unreachable"
[ 18631][D][HTTPClient.cpp:1163] connect(): failed connect to v4v6.ipv6-test.com:80
[ 18638][W][HTTPClient.cpp:1486] returnError(): error(-1): connection refused
[ 18645][E][Core2Logger.cpp:192] log(): [Core2Logger] CORE2: HTTP GET error -1: connection refused

Other Steps to Reproduce

I am not sure if it is a LWIP configuration issue or if processing of DNS server addresses does not handle IPv6.

IPv6 is working in dual stack, although DNS only shows the IPv4 DNS server.

[ 1679][D][WiFiNetworkManager.cpp:34] wifiOnEvent(): [Network] WiFi station start
[ 1715][D][WiFiNetworkManager.cpp:94] loop(): [Network] WiFi begin status 6
[ 2012][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA connected
[ 2356][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv4 192.168.1.147
[ 2366][I][Core2Logger.cpp:176] success(): [Core2Logger] Success
[ 3583][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 fe80:0000:0000:0000:0a3a:f2ff:fe65:db28
[ 3597][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 2
[ 4585][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 2407:8800:bc61:1340:0a3a:f2ff:fe65:db28
[ 4599][I][Core2Logger.cpp:176] success(): [Core2Logger] Success
[ 4605][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 1
[ 4620][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: WF STA IPv6 fd7c:e25e:67e8:0040:0a3a:f2ff:fe65:db28
[ 4634][I][Core2Logger.cpp:176] success(): [Core2Logger] Success
[ 4640][D][WiFiNetworkManager.cpp:47] wifiOnEvent(): [Network] IPv6 address type 4

Requests to an IPv6 test dual stack endpoint do resolve, via the IPv4 DNS, to their IPv6 address with the endpoint reporting the remote client address (i.e. my address) as IPv6:

[ 36859][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Button 1, scenario 0, v0.1.0-150-g27c8f84-dev
[ 36872][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Global IPv6 2407:8800:bc61:1340:a3a:f2ff:fe65:db28
[ 36886][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: IPv4 192.168.1.147
[ 36895][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: Link-Local IPv6 fe80::a3a:f2ff:fe65:db28%st1
[ 36908][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS0 192.168.1.1
[ 36917][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: DNS1 0.0.0.0
[ 36924][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: URL: http://v4v6.ipv6-test.com/api/myip.php
[ 39165][I][Core2Logger.cpp:198] log(): [Core2Logger] CORE2: response=<2407:8800:bc61:1340:a3a:f2ff:fe65:db28>
[ 39180][I][Core2Logger.cpp:176] success(): [Core2Logger] Success

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[TD-er]

Yep that's to be expected as there isn't any support yet for DNS servers via an IPv6 address.

Not sure how many people will run an IPv6-only network right now, but it is for sure something that needs to be fixed at some point in time.

[sgryphon]

It doesn't look too hard. There is a bit of parsing code in dnsIP() that only has IPv4 (affects display; shouldn't affect function), but the main issue looks to be that both IPv6 DNS mechanisms (stateless DHCPv6 and RDNSS) are configured as off. Most of the function, if you configure them on, seems to be automatic (DHCPv6 also needs to be enabled in code).

First I was trying to add IPv6 DNS via DHCPv6 to arduino-esp32 and I am getting a linker (ld?) error "undefined reference to `dhcp6_enable_stateless'".

I presume this means although the header is there, because the options at precompile time was missing CONFIG_LWIP_IPV6_DHCP6, then the functions are just not included in the compiled code (they are guarded).

How do I get a custom version of esp32-arduino-libs? The repository at espressif/esp32-arduino-libs says it has pre-compiled versions, and has headers but no source.

Do I need to download https://github.com/espressif/esp-idf and then build that myself (with the needed config changes)?

I will also have a look at adding via RDNSS.

[sgryphon]

Not sure how many people will run an IPv6-only network right now

Telcos are starting to switch to IPv6 only for their consumer mobile networks, e.g. Telstra here in Australia (who I work for): https://www.sidn.nl/en/news-and-blogs/australias-telstra-switches-mobile-users-to-ipv6-only

Some ISPs have started to provide IPv6 only connections, e.g. T-Mobile in USA and several ISPs in India. While global IPv6 traffic is about 45% it varies by country, with India being ~70%.

Corporations might deploy new networks as IPv6 only, or configure guest networks for reduced configuration and increase security (less vectors for attack with single stack). Both AWS and Azure will now be charging for (public) IPv4 address, while IPv6 will remain free, which will be another motivation.

Technologies such as Thread are now available, based on 6LowPAN, which, by definition, is IPv6 only. There is also some versions of ESP32 chips, e.g. the ESP32H2, that support Thread, which would (by definition) be an IPv6-only network.

[TD-er]

I've been running dual-stack IPv6 at home since 2010, as I'm also convinced for the need to have IPv6.
Here in the Netherlands some providers now have CG-NAT for IPv4 and native IPv6. Some also call it DS-lite (dual-stack lite)

I doubt providers would dare to completely drop IPv4 support right now as there are just too many sites out there without IPv6 addresses.
So I guess they would all switch to some form of CG-NAT so they don't need to use expensive public IPv4 addresses, but still allow users to use IPv4 to access those sites not accessible via IPv6.
This then also allows for DNS servers accessible via IPv4.

Therefore I don't expect IPv6-only to become really common in the next few years.
Still it should be fixed in the Arduino code to be able to use IPv6-addresses for DNS servers.