NodeMCU is an interactive firmware, which allows running Lua interpreter on the ESP8266 microcontroller (ESP32 support is in development). Alongside with all the regular hardware interfaces, it has WiFi module and SPIFFS file system.

This article describes the new module for the NodeMCU — sdm. SDM stands for simple driver model and it provides device-driver model abstraction for the system. In the first part of this article we will discuss the model itself and in the second part will be a showcase of dynamically created web user interface using sdm with some commentaries.

Driver model basics

Two major components of the model are devices and drivers. Device is an abstract representation of some hardware or virtual device. It makes sense to place devices into tree hierarchy, with the microcontroller on top, buses in the middle and sensors as leaves.

             DEVICES                  +   DRIVERS
                                      |
             +-----+                  |   +-----+
             |1WIRE<----------------------+1WIRE|
             ++-+-++                  |   +-----+
              | | |                   |
    +---------+ | +--------+          |   +------+
    |           |          |       +------+DS1820|
+---v----+  +---v----+ +---v----+  |  |   +------+
|DS1820|0|  |DS1820|1| |DS1822|0|  |  |
+---^----+  +---^----+ +---^----+  |  |   +------+
    |           |          +--------------+DS1822|
    |           |                  |  |   +------+
    +-----------+------------------+  +

Device driver is a piece of logic associated with given device. Functions provided by driver are called methods, data containers associated with driver are called attributes. Both methods and attributes live inside driver.

Attributes have two functions associated with them: getter and setter hooks. So attributes superset method functionality, but they also take up more memory (microcontroller memory is scarce, remember?).

sdm.attr_add(drv, -- device handle
             "ref", -- attribute name
             "Reference voltage", -- attribute description
             5,
             function(dev) -- this is a getter function
                return sdm.attr_data(sdm.attr_handle(dev, "ref"))
             end,
             function(dev, value) -- this is a setter function
                sdm.attr_set(sdm.attr_handle(dev, "ref"), value)
             end
)

Device binding

Tricky part of the driver model is device-driver binding. The process itself is quite simple: we match device with each available driver until it fits. Only two parts are missing — matching logic and some data to match to.

In sdm matching logic lives in drivers under the name _poll(). It is a regular method that is called with device handle as parameter and returns true or false if device could or could not be attached to the driver respectively.

sdm.method_add(drv, "_poll", nil,
               function(dev, drv, par)
                  local attr = sdm.attr_data(sdm.local_attr_handle(dev, "id")) -- get device attribute "id"
                  if attr == nil then return false end -- if it does not have one, driver does not match
                  -- parent name must be "ESP8266_1W" and first byte of "id" must be "0x28"
                  return (sdm.device_name(par) == "ESP8266_1W") and (attr:byte(1) == 0x28)
               end
)

As seen in the example above, driver matches device using attribute. But as noted above, attributes associate only with driver. Generally it is true, but there are some attributes that cannot be retrieved via software. These are chip IDs, used pins etc. For those a special type of attribute was added to the sdm — local attribute. This attribute is associated with one instance of the device and usually immutable.

The only one thing left to say about driver binding. Usually devices require some kind of initialization on startup and cleanup after use. For this purpose sdm uses _init() and _free() methods.
If driver has _init() method then it will be called automatically after device binding. Same with _free().

sdm.method_add(drv, "_init", nil,
               function(dev, drv, par)
                  sdm.device_rename(dev, sdm.request_name("DS18B20")) -- rename device
                  sdm.attr_copy(dev, "temp") -- copy attribute
                  sdm.attr_copy(dev, "precision") -- copy attribute
                  local met = sdm.method_dev_handle(par, "setup") -- get 1Wire bus pin init function ..
                  local func = sdm.method_func(met) -- .. and ..
                  func(par, dev) -- .. call it
               end
)

sdm.method_add(drv, "_free", nil,
               function(dev, drv, par)
                  local met = sdm.method_dev_handle(par, "free") -- get 1Wire bus pin free function ..
                  local func = sdm.method_func(met) -- .. and ..
                  func(par, dev)  -- .. call it
               end
)

Attentive reader would probably ask: what does "copy attribute" in the example above mean? And he would be right, because this has to do with the third kind of attribute we have not discussed yet — private attribute. It does not make much sense to have all attribute data shared between all device instances. For this purpose sdm provides mechanism of copying attribute from driver and associate it with device. This makes driver attribute a prototype or template.

A quick summary:

local attributes are used for data which cannot be retrieved by software. Like device IDs, connected pins etc.
driver attributes are used for data shared between all instances of devices attached to this driver.
private attributes are copied from driver attributes and hold data associated with only one device instance. This type is the most common.

Property	Local attribute	Private attribute	Driver (public) attribute
Stored in	device	device	driver
Accessible using driver handle	-	-	+
Accessible using device handle	+	+	+
Shared between devices	-	-	+
Persist upon driver detach	+	-	+

Web user interface implementation

Server code

There's a lovely nodemcu-httpserver project that implements server code for NudeMCU. Sadly it seems to be dead. It was used as a basis for the server. Firstly, server functions were moved to LFS and then slightly modified to serve one static page for every call. Vue.js is a perfect choice for template based web pages. So it was used for frontend. It worth noting that NodeMCU may not be connected to the Internet. Because of this, vue.js library needs to be present locally and served by NodeMCU server.

Since all devices are organized in tree structure, they are accessed just like a directory: /ESP8266/ESP8266_1W/DS18S20-0. Here /ESP8266 is a NodeMCU page, /ESP8266/ESP8266_1W is a 1Wire bus page and finally /ESP8266/ESP8266_1W/DS18S20-0 is a temperature sensor.

As mentioned previously, all device pages are build from one template page which is served on every call. JS code inside this page then makes request to the same URL, prepended with /api. For the example above call URL would be /api/ESP8266/ESP8266_1W/DS18S20-0. On such requests the server responds with JSON-encoded device-specific data, which populates the page. Of course, the HTML page request may be skipped if only raw data is needed.

Device tree

Initial device configuration is done using simple device tree structure. It is like device tree, but simpler. It describes configuration of the hardware including device local attributes.

local root={
   -- local_attributes={},
   children={
      {
         name="ESP8266_1W",
         -- local_attributes={},
         children = {
            {
               name="DS18S20-0", -- static declaration alternative to 1Wire poll method
               local_attributes={
                  {
                     name="id",
                     desc=nil, -- empty description to save space
                     data=string.char(16) ..
                        string.char(221) ..
                        string.char(109) ..
                        string.char(104) ..
                        string.char(3) ..
                        string.char(8) ..
                        string.char(0) ..
                        string.char(150) -- ugly way to create byte array
                  },
                  {
                     datapin=2
                  }
               }
            },
         }
      },
      {
         name="ESP8266_SPI",
         -- local_attributes={},
         children = {
            {
               name="MCP3208-0"
            },
         }
      },
   }
}

Hardware setup

Here begins the showcase. For this purpose a bunch of sensors were connected to the NodeMCU:

1Wire sensors are connected to the same pin.

Web pages and drivers

root device

The main purpose of the root device (aka ESP8266) is to provide place for its children to connect to. However it's not restricted to have methods or attributes associated with it.

This code snippet is from here:

sdm.method_add(drv, "_init", nil,
               function(dev, drv, par)
                  local attr = sdm.attr_handle(dev, "id") -- get device "id" attribute
                  sdm.attr_set(attr, node.chipid()) -- set "id" value
                  attr = sdm.attr_handle(dev, "float") -- get device "float" attribute
                  sdm.attr_set(attr, 3 / 2 ~= 1) -- set to true if firmware supports floating point instructions
               end
)

sdm.attr_add(drv, "float", "Floating point build", false,
             function(drv) -- attribute value is set inside "_init" function
                local attr = sdm.attr_drv_handle(drv, "float")
                return sdm.attr_data(attr) -- just return stored value
             end,
             nil
)

This code adds attribute float which is used to hold firmware build type. Its value is initialized in the _init() hook which is a special function, that runs once when driver attaches to the device.

This is the generated page for the root device.

Here we can see that the root device has one method heap, two driver attributes float and id. Finally, it has two devices connected to it — SPI and 1Wire buses.

SPI

SPI driver is not very interesting. It just maps NodeMCU SPI functions.

MCP3208

MCP3208 is an ADC chip. It measures voltages from zero to ref and returns 12 bit code. What's interesting about this driver implementation is that attribute ref would be present build only if firmware supports floating point arithmetic. If it is not supported then instead of absolute voltage, voltage code is returned by both single and differential methods.

sdm.method_add(drv, "single", "Single ended measure 0|1|2|3|4|5|6|7",
               function(dev, channel)
                  -- ...
                  if ref ~= nil then
                     -- this part is executed only if floating point arithmetic is enabled
                     rv = ref * rv / 4096
                  end
                  return rv
               end
)

if 3/2~=1 then -- other alternative is to access ESP8266 "float" method
   sdm.attr_add(drv, "ref", "Reference voltage", 5,
                function(dev)
                   return sdm.attr_data(sdm.attr_handle(dev, "ref"))
                end,
                function(dev, value)
                   sdm.attr_set(sdm.attr_handle(dev, "ref"), value)
                end
   )
end

Also note that this device has attribute ref marked as private. It is set on per-device basis.

1Wire

1Wire driver implements poll method — dynamic search for devices.

Right after device discovery its type is not known. So its 1Wire unique address is used as a new device name (bytes represented as numbers separated by _ character).

sdm.method_add(drv, "poll", "Poll for devices",
               function(bus, pin)
                  local children = sdm.device_children(bus) or {} -- already attached
                  local ids = {}
                  -- get IDs of attached devices
                  for name, handle in pairs(children) do
                     local dpin = sdm.attr_data(sdm.local_attr_handle(handle, "pin"))
                     if dpin == pin then
                        ids[sdm.attr_data(sdm.local_attr_handle(handle, "id"))] = true
                     end
                  end
                  ow.reset_search(pin) -- reset previous search
                  while true do
                     -- for all found devices
                     local id = ow.search(pin)
                     if id == nil then break end
                     if ids[id] == nil then -- if not already present
                        local name = ""
                        for i=1,#id do name = name .. tostring(id:byte(i)) .. "_" end
                        name = name:sub(1,-2)
                        -- add to system with their ID used as name
                        local device = sdm.device_add(name, bus)
                        -- add "pin" attribute
                        local rv = sdm.local_attr_add(device, "datapin", nil, pin, nil, nil)
                        -- add "id" attribute
                        local rv = sdm.local_attr_add(device, "id", nil, id, nil, nil)
                        -- poll for driver
                        local rv = sdm.device_poll(device)
                     end
                  end
               end
)

This is the initial page for 1Wire driver.

After issuing poll call with argument 2 and refreshing page, children section appears. Note that children names are human readable. This is because device_rename() function was called during their _init.

DS18S20

Upon initialization, DS18S20 driver checks that device ID begins with 0x10, which is a device family code. When device is attached to driver, it is renamed to the DS18S20-X, where DS18S20 is a basename and X is an instance number.

sdm.method_add(drv, "_poll", nil,
               function(dev, drv, par)
                  local attr = sdm.attr_data(sdm.local_attr_handle(dev, "id"))
                  if attr == nil then return false end
                  return (sdm.device_name(par) == "ESP8266_1W") and (attr:byte(1) == 0x10) -- check family ID
               end
)

sdm.method_add(drv, "_init", nil,
               function(dev, drv, par)
                  sdm.device_rename(dev, sdm.request_name("DS18S20")) -- rename device
                  sdm.attr_copy(dev, "temp") -- copy attribute to device
                  local met = sdm.method_dev_handle(par, "setup")
                  local func = sdm.method_func(met)
                  -- use parent "setup" method on the device
                  func(par, dev)
               end
)

Local attributes id and datapin do not have getter and setter hooks, so only their names are visible.

DS18B20

DS18B20 driver is almost the same as DS18S20 driver. The only difference is the precision method. Both DS18?20 drivers assume integer build and do not use floating point division.

sdm.attr_add(drv, "precision", "Precision (9|10|11|12)", 12,
             function(dev, precision)
                local attr = sdm.attr_dev_handle(dev, "precision")
                return sdm.attr_data(attr)
             end,
             function(dev, precision)
                local par = sdm.device_parent(dev)
                local attr = sdm.attr_dev_handle(dev, "precision")
                local ex = sdm.method_func(sdm.method_dev_handle(par, "exchange"))
                local modes = {[9]=0x1f, [10]=0x3f, [11]=0x5f, [12]=0x7f}
                if modes[precision] ~= nil then
                   ex(par, dev, {0x4e, 0, 0, modes[precision]})
                   sdm.attr_set(attr, precision)
                end
             end
)

Memory usage

ESP8266 free memory is about 40k. Server code is moved to LFS, so it does not take any RAM space at initialization time (original code took about 10k).

SDM takes up about 10k for 5 device drivers and 5 devices. Slightly lesser for non-floating firmware build. So it's preferable to select in driver manifest only drivers needed for the task at hand. The most memory consuming task is to serve vue.js library.

In case of requesting raw JSON-encoded data (using curl) peak memory consumption may be significantly reduced.

Instead of an Epilogue

One of the first methods I implemented with sdm was the binding for
node.restart().
Trying it out using web user interface produced a curious result. Right after the web browser issued the request, chip restarted as expected. But because NodeMCU did not properly respond to the HTTP request, web browser tried the same request again. When NodeMCU server restarted and was up again, browser connected to it, reset internal try again counter and called the node.restart() method, thus beginning an infinite loop of NodeMCU restarting.

NodeMCU simple driver model (SDM) showcase: dynamic user interface