Skip to content
See packages

Engineering

FPGA peripheral to Linux driver: bringing up the stack layer by layer

A custom controller block on programmable logic crosses five layers before software can use it. Bring them up one at a time and every failure has one suspect.

Computing America Hardware. Custom hardware for physical automation and monitoring.

Send the board
4 min readComputing America
An FPGA development board on an anti-static mat with a programming cable and logic analyzer probes.

In short

  • Check the logic in simulation before any hardware is involved, and make the first hardware check a bare-metal register read and write, before an operating system boots.
  • Use the vendor's bus template instead of writing the bus interface by hand: AXI leaves the order of write address and write data open, and a hand-written slave has to accept either.
  • A finished driver makes the hardware look like a character device, so the application needs no knowledge of the hardware beneath it.
  • Support both polling and interrupts in the driver, and default to the one that suits the operation's duration.

The five layers are the logic, the bus interface, the build and device tree, the kernel driver and user space. We built one end to end, from the register-transfer-level description to a user-space program that doubles a number, a function small enough to leave every layer visible. Each layer gets one pass-or-fail check before the next is added.

Why does a robotics buyer care? Fixed-latency work such as encoder counting, motor timing and sensor sampling often lives in programmable logic beside the processor. A bring-up plan that skips a layer can't say where a fault is.

Five layers, five checks, one order

Fig. 1
  1. 5 User space

    Within: Checked on the board, one at a time.
  2. 4 Kernel driver

    Within: Checked on the board, one at a time.
  3. 3 Build and device tree

    Within: Checked on the board, one at a time.
  4. 2 Bus interface

    Within: Checked on the board, one at a time.
  5. 1 Logic

    Within: Checked in simulation, no hardware.

Build bottom-up: the logic is always checked without hardware, and each layer above it is checked on the board before the next is added.

SourceComputing America's FPGA peripheral build, as shown on the portfolio page.

The logic is a small state machine with two data registers (one software writes, one it reads back) and a control register carrying a start bit software sets and a done bit hardware sets. Synthesis turns it into a bitstream that configures the fabric at boot, and the tooling emits the register map and device description the kernel needs.

  • Logic: a simulation testbench drives the register writes and reads a bus master would issue and checks the done bit and the doubled value, with no hardware involved.
  • Bus interface: a bare-metal program on the processor reads and writes the registers by address, with no operating system. If the interface is hand-written or you changed the vendor's template, also run it in simulation under AMD's AXI Protocol Checker or Arm's AXI protocol assertions.
  • Build and device tree: boot the image and look for the peripheral's node under /proc/device-tree, with the address and compatible string the register map promises.
  • Kernel driver: load the module, confirm with dmesg that the probe succeeded, then write 42 to the device and read it back with the two commands below.
  • User space: run the six-line program and compare the number that comes back with twice the number written.
driver checkshell
printf '\052\000\000\000' | dd of=/dev/axi_acc0 bs=4 count=1dd if=/dev/axi_acc0 bs=4 count=1 | xxd
The write is 42 as a 4-byte little-endian value. A working driver reads back 5400 0000, which is 84, twice 42.

The temptation is to work several layers at once, since each has idle time while another builds. A bug that could be in the logic, the bus interface, the device description or the driver is a bug you can't localize.

Use the vendor's bus template

The peripheral talks to the processor over AXI4, Arm's on-chip bus. It has five channels, each with its own valid and ready handshake: write address, write data, write response, read address and read data. Our build used AXI4-Lite, the subset for control registers, which keeps all five.

The specification requires that a write response follow the last write transfer and that read data follow its address. In its words, "The protocol does not define any other relationship between the channels." (opens in a new tab) So the write data can arrive before its write address, or in the same cycle. A slave that assumes the address comes first is wrong the first time an interconnect delivers them the other way round.

The vendor toolchain ships a template for this interface. Use it unchanged. If you modify it, run it under the protocol checker like a hand-written interface.

The driver turns the hardware into a file

With the driver loaded, the user-space program is six lines: open, write one value, read back the answer, close. It's the same shape as talking to any other character device, and the application author doesn't need to know the accelerator exists.

userspace/main.cC
/* Five layers sit under this file, each brought up and verified before the   next: Verilog RTL, AXI4-Lite bus attachment, device tree, kernel character   driver, and then this. */int fd = open("/dev/axi_acc0", O_RDWR);write(fd, &(uint32_t){ 42 }, sizeof(uint32_t)); uint32_t out;read(fd, &out, sizeof(uint32_t));printf("%u\n", out);            /* 84 */close(fd);
The peripheral doubles a number, and the triviality is the point: a function this small leaves no layer anywhere to hide, so a failure is unambiguously attributable to the one you just brought up. Bringup order is the transferable skill; the accelerator is not.

The driver itself is short. Two decisions matter:

  • Polling or interrupts. An operation that completes in tens of nanoseconds wants polling, because interrupt overhead dominates. A longer one flips it.
  • Matching by compatibility string. If the string in the device tree and the one in the driver differ, the kernel does not bind the driver to the peripheral.

What to ask of a bring-up plan

  • Which check passes before each layer is added, and who signs it off?
  • Are registers exercised from a bare-metal program before an operating system boots?
  • How is enumeration confirmed before the driver loads?
  • Is the bus interface the vendor's template? If it's custom or a modified template, what test shows it handles every channel ordering?
  • Does the driver support polling and interrupts, and which one is the default for this operation's duration?

Next step

Send us the board.

A schematic, a datasheet, or the vendor's evaluation notes for the part you are stuck on. We will tell you which layer we would bring up first and what we would expect to be wrong when you do.

Reply
A person replies, not a sequence: within one business day, from someone who would be on the engagement.

Packages

Start with a package

Four packages, each with its price on the page. Pick one and buy it online.

Billing