[ksk-rollover] IoT devices and KSK rollover

Michael Casadevall michael at casadevall.pro
Mon Jul 1 21:02:29 UTC 2019

Replies inline.

On 7/1/19 1:47 PM, Paul Wouters wrote:
> On Mon, 1 Jul 2019, Michael Casadevall wrote:
>> As for DoT/DoH, none of the above changes as there are no wire protocol
>> changes to DNS in either of these use cases, and the cost of
>> implementation is stupidly high. First off, it's impossible to deploy
>> DoT/DoH in an internal network that uses RFC1918 address space (aka
>>,, as CA/B forum CAs can't issue
>> certificates for this address space;
> what prevents you from requesting a certificate on an internet connect
> IP using ACME, and then moving or also using it on the internal IP
> pointed to by internal only split DNS ?

Most home networks (and a depressing number of corporate networks) on
the offhand chance they do dynamic DNS use .local, .lan, or .home.
Active Directory is a big offender here since Windows SBS was hardcoded
to only allow .local domains for years, and Microsoft's official advice
was to use .local unless you had a domain name for your practice up
until ~2008. CA/B only allows issuance of certificates that have a
publicly resolvable domain name that you can verify ownership of (or
ownership of a specific public IP address that you have allocated to you).

Furthermore, using a domain name to point at your DNS system has a
bootstrapping problem (which can be resolved via traditional DNS but is
very much non-ideal).

>> Device manufactors also need to update their devices to update
>> their root store from time to time.
> I guess it really depends on your definition of IoT.

For "higher end" devices running Linux, it is extremely common for the
entire OS to be stored as a read-only squashfs image with a few
directories in persistant flash marked read/write. I have yet to see any
device that has a root certificate store put it anywhere read/writable
(and the config paths that read these images are often hard coded).

For lower end devices, the situation is far worse. One specific project
I worked on as a contractor was built around a Cortex-M3 running in
16-bit Thumb mode; it had 128 kilobytes of working memory, 256 kb of
executable/addressable flash (I think), and 8 MiB of "storage" flash
(that could be read or written to, but not executed) that had to be
manually paged into working memory (aka, it wasn't in the memory map).

This was built around the WiCED RTOS and TCP/IP functionality was
provided by an add-on card that had wifi and used AT commands to make
TCP/IP and UDP connections (similar to

The device by default used Amazon's IoT service to push and receive
configuration settings, and could also connect to a customers on-premise
location. These operations were done with TLS and HTTPS (with plans to
also support MQTT at the time I left).

Due to the way the onboard TLS stack, the root certificate storage had
to be either in RAM or the executable/addressable flash memory. WiCED
used ~20-30 KiB of onboard memory, and the TLS stack used up to 90 KiB
(aka, almost everything the device had).

Ultimately, I was forced to hardcode in the roots Amazon was using, and
there was a large question mark with this device on how to handle
customer based installs short of telling them to build certs from a
specific CA, and I couldn't convince management we needed a more
powerful board. I choose not to renew my contract, but I suspect their
solution was to just disable all TLS validation.

>> It should also be noted that the current Mozilla NSS root store is
>> approximately 250 KiB in size uncompressed.
> Why would an IoT device need the whole firefox or other vendor root CA
> store? Again, it really depends on what you call an IoT device.

Because if you're supporting more than a single HTTPS site or resource
that requires TLS, need to access third party websites, or access
customer hosted equipment (for example, to download configuration
information, submit results, etc), you can't know in advance what CA has
signed your customer stuff, and it's the height of user unfriendly where
you need to download your webservers root certificate and upload it to
your device.

Real world example: I'm an Internet enabled scanner. I want to have an
IoT device that sends email to a given host. Due to privacy concerns,
the information I send should not pass through the vendors servers. The
SMTP relay my IT admin setup requires use of a username and password +
STARTTLS. I have no idea what root certificate(s) their servers chain up to.

I've encountered these devices at more than one customer's site, and can
easily be called IoT devices. There are plenty of other cases I can come
up with where you need an entire root store beyond DoH/DoT.

>> There are major issues with DoH/DoT in general, but a lot of devices in
>> general are going to simply incapable of supporting it because the cost
>> of full TLS support + full set of CA root certificates is simply too
>> high in terms of flash storage.
> Maybe IoT devices shouldn't need to talk to the internet at large? And
> their trust model can be minimzed to the little things it needs?

IoT devices should be well designed, secured and be regularly updated
too. Sadly, we don't live in that world. We live in a world where
vendors leave root enabled telnet on out of the box. This should be a
rather enlightening read:

The state of IoT is just sad and depressing.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pEpkey.asc
Type: application/pgp-keys
Size: 2468 bytes
Desc: not available
URL: <http://mm.icann.org/pipermail/ksk-rollover/attachments/20190701/ac9a2aaa/pEpkey.asc>

More information about the ksk-rollover mailing list