tldr; IoT startups need to think about firmware
At Ringtail, my focus is software and systems for the Internet of Things. In many ways, IoT is really just today’s name for embedded networked systems. This quote from @embeddedsec summarises it perfectly:
IoT is marketing speak for connected embedded devices. That is, inexpensive, low power, resource constrained computers that talk to each other, possibly on the capital-I Internet, to exchange data and command and control information. These devices are often installed in hard to reach places and can be expected to operate for years.
IoT is not a new discipline. These kind of systems have been around for a long time. The explosion we’re seeing today is due to falling hardware costs, a renaissance in manufacturing and widely available software toolkits.
Some of the work I do is with early stage companies to take a concept from drawing board to prototype and onto manufacture. However, many of my clients are established companies who need help maintaining, extending and updating their existing IoT devices.
Unfortunately, many companies who have invested in development of an IoT hardware solution find themselves without the documentation, know-how or tools to build, configure, deploy or properly use their devices. I help these companies get value out of their investment.
There’s a lot of advice already available to IoT entrepreneurs and fledgling startups on finance, design, manufacturing and marketing. Sadly, embedded software is too often ignored. For a hardware or device startup, software is key intellectual property and is an investment which must be looked after.
If you have bought in an IoT solution, or are developing your own, here are some of the first questions you should be asking. When I start working with a new client or handover a completed project, I make sure to cover all of these topics.
To minimise size and maximise lifetime, IoT designers are often forced to use unusual battery types. These can range from watch and coin cells, to lithium or even lead-acid battery chemistries. If your device uses batteries, make sure you know the voltage and size required. Be careful of non-standard cells such as 3.6v lithium batteries in a standard AA shape as these are particularly easy to mix up.
If your device is mains powered, do you have the correct adapter? During development, engineers will often power a board from a variable bench power supply allowing them to dial in the exact voltage and current requirements needed. If you do not have a bench power supply, take care to choose the right AC-DC adapter and double check the polarity of the output.
Most devices have some kind of debug connection used to load new software and monitor behaviour. For networked devices with no screen, a mechanism for debug and diagnostics is critical.
Devices deployed in industrial or harsh environments will sometimes provide a serial RS232 connection with a standard connector. However, it’s not uncommon to find TTL level debug available on the PCB. Whichever your device uses, make sure you know the baud rate settings.
Some devices eschew RS232/UART and provide serial emulation over USB. This fixes the problem of needing a special connector, but it is important to check if any special drivers or other software is needed to communicate with the device.
If specialised debug connectors or dongles are required for JTAG or SWD, it’s important to make sure all of the right connectors and software tools are available.
Programming new firmware into devices can be complex. Typically, each semiconductor vendor will provide specialised software and hardware tools to communicate with their chips.
Unfortunately, a common pattern I see is for companies to have “the special laptop” or “programming station”. This is a PC which has been setup with the software tools to flash firmware. While this can work OK, 9 times out of 10 the laptop isn’t backed up and there are sketchy instructions for how to recreate it. Remember - your backup strategy is only as good as your last restore.
For most customers, I recommend using a virtual machine image which is properly backed up. There should also be a document (or even better, script) to recreate this VM image with all of the required tools archived alongside.
Remember, if you cannot program your devices then you cannot fix bugs and you cannot manufacture more of them.
If you have paid for development of an IoT solution, you should already have the source code. Without source code, you are reliant on your single supplier to make changes and fix bugs on your behalf.
If you are at the planning or procurement stage of an IoT project, ensure that you will be receiving full source code - this is important IPR for your company.
Using the source code you have, are you able to build this into a firmware image for your device?
The first step is to compile the source. Typically, embedded firmware is developed using a cross-compiler (running on a PC). As with programming devices, compiling often requires vendor supplied tools and may even need site licenses or copy-prevention dongles. Again, a VM or dedicated PC is often the best solution here.
Once the code is compiled correctly, it must be linked to any required libraries. If these are not bundled with the source code then it’s important to know exactly what versions are required and where they can be fetched from. I have seen many examples of source code deliverables with implicit dependencies on special versions of external software - this can cause bugs and build problems.
Now that the compiler and linker have produced a binary, a special file format must often be prepared for the flash loader or bootloader. With some tools, this is all handled automatically, but it can be complex when working with open source toolchains and proprietary bootloaders.
Lubarsky’s Law of Cybernetic Entomology states “There is always one more bug”.
The unfortunate reality is that no matter how much you test, there will be issues with your software. Sometimes these are outright bugs, other times feedback from customers will change the requirements.
Issues need to be recorded, so that they can be prioritised, tracked and resolved. Sometimes, this activity needs to involve the end user, other times it’s a purely internal engineering activity. Whatever your requirements, bug tracking is a must. Over the years, I’ve seen companies track issues on whiteboards, in Excel spreadsheets and Word documents. Any kind of bug tracking is better than nothing - but the right way is to use a dedicated issue tracker which is linked to your source control system.
Over time, alterations will need to be made to your source code and firmware releases will produced. These source code changes must be tracked and versioned so that they can be reviewed, archived and tested. Knowing which changes to the code are meant to fix which issues speeds up testing and gives confidence about software releases.
Some form of revision control is essential for good software development. Today, most of the tools for this are free and sites like github and bitbucket make teamwork easy.
Once a device has been powered up and programmed with fresh firmware, do you have a way to prove it is working correctly?
For a networked device, testing may require configuring the device with a unique address or providing security credentials for a cloud service. If the device contains sensors or actuators, there should be a process to exercise each of these and ensure that the latest firmware release is working as expected.
If you are working with a third-party to build devices, they will need a clear process for testing manufactured devices. Often the best way to encapsulate that knowledge is to build an automated test jig which can be used to quickly check if a device is working.
Without automation, the burden of testing will grow until it slows the software release process. Too many times I have seen companies drowning under the weight of their manual test process forced to release untested software to meet customer deadlines.
One of the trickiest problems for IoT can be ensuring that batteries last long periods without putting each firmware release into test for a year at a time.
Managing a power budget will usually require a device to be asleep for much of its life, waking only to perform its function. The most straightforward way to estimate battery life is to look at the possible states the device can be in (e.g. asleep, transmitting, processing…) and take power measurements for each. A simple mathematical model can then be built to predict the power usage in normal operation. While this isn’t sufficient for every system, having a baseline to compare each firmware release against will often provide early warning of problems.
Toby Jaffey, Principal Consultant
+44 (0)7588 606410
Ringtail Software Ltd. Registered in England and Wales, Company Number 10085086. VAT Registration Number 242677787