While writing some software defined radio applications, I have found myself wanting to run them headless on some raspberry pi or similar, but then occasionally connect to it and inspect it live.

I want to stream sample data, and tags data, and drop data on overflow. The main use case is for a UI, so it should not block the rest of the processing if the network or UI is slow, or disconnected.

I also want the protocol to support lossless sources, for example a stream coming from a SigMF file.

I’ve been dragging my heels on this, because surely this protocol already exists. And if I make a new one then nobody else is compatible with it.

Requirements

  • Support drop-on-overflow, and lossless.
  • Support regular TCP, and connections from JS/WASM websockets.
  • Work with GNU Radio, and with RustRadio.
  • Because websocket messages are delivered via callback, and thus the TCP flow control does not propagate back to the sending application, the protocol needs its own flow control.
  • One port per application, but a client can request uploading or downloading different streams, by name.

Existing protocols

GNU Radio ZeroMQ

ZeroMQ example in GNU Radio

GNU Radio has ZeroMQ sink and source blocks, but a UI running in a browser would not be able to directly connect to it, since only websockets can be used from JS or WASM. One can set up a websocket to TCP gateway, but it would be nicer to not need to.

There would also be some complexity about how to handle overflows, and requiring the user to choose between the three ZeroMQ socket patterns.

The serialization is also GNU Radio specific, and doesn’t surface the flow control and loss policy I’d want.

And it’s one TCP port per stream, which is not ideal.

VITA49

VITA49 seems to be the most standard protocol for sending RF data. But it has very limited support for application metadata (in our case, tags). And no flow control.

It also doesn’t work directly on websockets, but has the same need for a websocket-to-TCP layer. Hell, it’s not even meant for TCP, so it’d be a bit nonstandard to even use it as an in-order stream.

It does have a message type for extensions, so in theory tags can be serialized in a custom format, put in VITA49, then wrapped in another custom protocol. Ok, but what was the point of using a “standard”, then? Just for the IQ samples? Might as well just put serialized floats on a TCP, in that case.

The vita49 rust crate also points out that one can’t “just” use vita49 and think it’s all one big happy standard.

V49.2 provides millions of options for data types and packet structures that
are likely impossible to implement as one application program interface (API)
without culling down to required attributes.

DIFI

DIFI builds on VITA49, and adds flow control. But still no arbitrary tag support, or anything that maps to it.

So I guess I’m making yet another protocol

Sigh. Well, at least this unblocks what I want to do. If I find a better protocol in the future, I can just swap out the implementation.

If this had been 5 years ago I would have worried about throwaway work, if a better protocol arrives. But because of AI, I’m OK with throwaway work and more complex proof of concepts.

I refer to this as “IQ streaming”, but it supports non-complex data too.

To at least avoid having yet another parser, and only work with messages, I decided to base the protocol on gRPC. There’s a protocol description and a .proto file, for other implementations.

Implementation

Implemented for RustRadio (behind the unstable feature, in case I want to change the API), and for GNU Radio.

I added a few IQ streaming taps to my electricity monitor “sparslog” written with RustRadio, and made a couple of blocks to receive the data in GNU Radio.

I also added a RustRadio WASM UI for sparslog.

Results

GNU Radio graph

GNU Radio graph running

RustRadio browser (WASM) based UI

Future work

I also want to add other controls, such as ability to change frequency, or change filter parameters. With a gRPC endpoint, this should be easy.