Sending Raw Data

There are two variants of raw data. There is raw data for larger transfers such as e.g. zip files and lightweight raw data for sending small messages with custom data.

Raw Data

Sending raw data can be used to send any arbitrary data, e.g. zip files. The data to be sent is split into chunks that are then sent through the mesh. The data to be sent is called the "payload".

General Overview

raw data flow

Every raw data transaction (except raw_data_light, which will be explained later) starts with a raw_data_start message. This message includes the amount of chunks in the transaction and the protocol of the payload. Once the receiver receives this message, it answers with a raw_data_start_received message, which indicates to the sender that the receiver is ready to receive the chunks.

When the sender receives the raw_data_start_received message, it starts sending all raw_data_chunk objects. In addition of payload data, every raw_data_chunk includes a chunk ID, which is a unique (regarding the current transaction) ascending number, starting at 1 for the first raw_data_chunk object. chunk ID 0 is reserved for the raw_data_start. Using this chunk ID, and the information of the amount of chunks form the raw_data_start message, the receiver is able to determine the last chunk in the transaction. Once it receives the last chunk, it reports back to the sender using a single raw_data_report. This message includes the chunk IDs of up to three missing chunks. If all chunks were received, the list of missing chunks is empty, which tells the sender that the transaction was successful. If however, the list of missing chunks is not empty, the sender must send the chunks with the corresponding chunk IDs again. The last chunk IDs of the previous raw_data_report message acts as a last chunk, regarding the sending of additional raw_data_report. This sending of raw_data_chunk and raw_data_report is repeated until raw_data_report has an empty list of missing chunk IDs.

All devices involved in the communication (sender, receiver, and the mesh) are able to cancel the communication by sending a raw_data_error message to the sender as well as the receiver. If the sender or the receiver is the device that hung up the transmission, it is allowed to not send the error to itself. The raw_data_error message includes an error code, indicating the reason for the cancellation. It is possible to receive a raw_data_error message without an open transmission. This can happen if the sender cancels the transmission using a raw_data_error, but this error is dropped during the transmission. The receiver then might send another raw_data_error indicating a timeout while the sender already canceled the communication. Such messages without an open transmission may be discarded.

Implementation

As any other message in the mesh, every message in the raw send protocol could be dropped. This section describes how an implementation must behave in such scenarios. It also describes the obligations of the sender and the receiver.

Dropped message Reaction

raw_data_start
raw_data_start_received

The sender must send the raw_data_start again after a timeout of 10 seconds or stop the transmission. The receiver has to be able to handle several successive raw_data_start messages with the same content (in case the raw_data_start_received message is dropped) and close a dropped connection after a timeout of 15 seconds (in case the sender does not send another raw_data_start).

raw_data_chunk

Missing chunks are reported in raw_data_report once the last chunk is received. These missing chunks must be resent.

Last raw_data_chunk or raw_data_report

Using the ChunkID, both sender and receiver are able to identify the last data chunk. If this message or the raw_data_report are dropped, the sender must send the last chunk again. This means that the receiver is only allowed to save the last chunk ID once the first chunk after a raw_data_report is received, not immediately after the raw_data_report is sent.

raw_data_error

If a raw_data_error message is dropped, the sender or receiver has already canceled the transmission, leading to the sending of another raw_data_error upon receiving an invalid out-of-transmission message or a raw_data_error indicating a timeout. In the rare cases where the origin of the raw_data_error is the mesh itself, it could happen that both raw_data_error messages are dropped. In such cases the connection is still up, but will probably create another raw_data_error once the ill-formed chunk is sent again.

Transmission Start

raw_data_start [receiverId] [destinationModuleId] [numChunks] [protocolId] {requestHandle = 0} {metadataHex}

This command starts a raw data transmission. The payload shall be sent using raw_data_chunk messages.

Command Parameter Type Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

u8

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

numChunks

u24

Number of Chunks for the total message. Must not be 0.

protocolId

u8

One of the protocol IDs mentioned in the table below

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

metadataHex

HexString or Base64

Optional binary metadata to send, e.g. 11:22:33:44 or ESIzRA==. The maximum length is 40 bytes of binary data (hex/base64 encoded string will be longer). It can be used to store request specific data like a UUID of a transmission.

Raw Data Protocol ID Name Description

0

Invalid

Invalid protocol ID

1

HTTP

A raw HTTP request or response

2

GZIPPED_JSON

A JSON that was gzipped

3

KBM

Mway proprietary building automation protocol

4 - 199

Reserved

Not yet used

200 - 255

User defined

May be different in each implementation

If received by a device with a terminal, raw_data_start is printed out like this:

// without metadata
{
	"nodeId":5,
	"type":"raw_data_start",
	"module":4,
	"numChunks":3,
	"protocol":1,
	"fmKeyId":2,
	"requestHandle":0
}

// with metadata
{
	"nodeId":5,
	"type":"raw_data_start",
	"module":4,
	"numChunks":3,
	"protocol":1,
	"fmKeyId":2,
	"requestHandle":0,
	"metadata":"ESIzRA=="
}

Accept Transmission

raw_data_start_received [receiverId] [destinationModuleId] {requestHandle = 0} {metadataHex = ""}

Once a raw_data_start is received, the receiver shall send the sender a raw_data_start_received message.

Command Parameter Type Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

u8

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

metadataHex

HexString or Base64

Optional binary metadata to send, e.g. 11:22:33:44 or ESIzRA==. The maximum length is 40 bytes of binary data (hex/base64 encoded string will be longer). It can be used to store request specific data like a UUID of a transmission.

If received by a JSON-capable device, the raw_data_start_received is printed out like this:

// without metadata
{
	"nodeId":5,
	"type":"raw_data_start_received",
	"module":4,
	"requestHandle":0
}

// with metadata
{
	"nodeId":5,
	"type":"raw_data_start_received",
	"module":4,
	"requestHandle":0,
	"metadata":"ESIzRA=="
}

Subsequent Chunk Messages

raw_data_chunk [receiverId] [destinationModuleId] [chunkId] [payloadHex] {requestHandle = 0}

Once a raw transmission has been started, the appropriate number of chunks should follow in the correct order. Once the last chunk is received by the receiver, it is possible to reassemble and parse the whole message. The moduleId is present in all chunks so that they can be assigned to the correct stream and to avoid clashes between different modules. A module can send intermittent data streams if is uses different request handles.

Command Parameter Type Description

receiverId

u16

The NodeID that this message should be sent to

destinationModuleId

u8

The ModuleId is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

chunkId

u24

ID of this data chunk starting from 1.

payloadHex

HexString or Base64String

The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

If received by a JSON-capable device, the raw_data_chunk is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_chunk",
	"module":4,
	"chunkId":1,
	"payload":"abcdeQ==",
	"requestHandle":0
}

Send Report

raw_data_report [receiverId] [destinationModuleId] [MissingChunkIds] {requestHandle = 0}

Once the last chunk is received, the receiver sends this message to the sender, indicating either a successful transmission (empty missing chunk IDs) or informs the sender about missing chunk IDs.

Command Parameter Type Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

u8

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

MissingChunkIds

Comma-separated integers or the literal string "-" (without "")

Up to three chunk IDs of missing chunks. Must not contain spaces! E.g. 2,17,312

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

If received by a JSON-capable device, the raw_data_report is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_report",
	"module":4,
	"missing":[2,17,312],
	"requestHandle":0
}

Or in cases where the transmission was successful:

{
	"nodeId":5,
	"type":"raw_data_report",
	"module":4,
	"missing":[],
	"requestHandle":0
}

Send Report Desired

raw_data_report_desired [receiverId] [destinationModuleId] {requestHandle = 0}

The "Report Desired" message is an optional message that can speed up transmissions in cases where the current last chunk was dropped. It is send from the sender to the receiver, indicating that the sender wishes to get a raw_data_report from the receiver. The receiver however may completely ignore this message, completely relying on the timeouts. The sender may also never send it, also completely relying on the timeouts.

Command Parameter Type Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

u8

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

If received by a JSON-capable device, the raw_data_report_desired is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_report_desired",
	"module":4,
	"requestHandle":0
}

Send Error

raw_data_error [receiverId] [destinationModuleId] [errorCode] [destination] {requestHandle = 0}

This command indicates that some error occurred and the transmission must be closed. The packet will be sent to the receiver as well as the sender.

Error Code Name Meaning

0

Reserved

1

Unexpected end of transmission

Three timeouts happened without receiving a message from the transmission partner.

2

Not in a transmission

A raw_data_chunk or raw_data_report was received without an open transmission.

3

Malformed Message

A message was received which was malformed and did not fit any other error code.

4

Unsupported Protocol

The receiver is unable to interpret the given protocol.

5

Malformed GZIP

The receiver got all chunks but could could not unpack the GZIP.

6

Malformed Type

Thrown in case of a GZIP communication. The unpacking worked, but the provided type inside the GZIP JSIN was unknown.

7

Invalid Chunk ID

The given chunk ID was out of range.

8 - 199

Reserved

Not yet used

200 - 255

User defined

May be different in each implementation.

Destination Code Name Meaning

1

Sender

The error is sent to the sender.

2

Receiver

The error is sent to the receiver.

3

Both

The error is sent both to the sender and the receiver.

If received by a JSON-capable device, the raw_data_error is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_error",
	"module":4,
	"error":1,
	"destination":1
	"requestHandle":0
}

Lightweight Raw data

Sending messages that fit into a single mesh packet can be done by using raw_data_light with minimal overhead and almost no implementation effort.

Terminal Command

raw_data_light [receiverId] [destinationModuleId] [protocolId] [payload] {requestHandle = 0}

Sends a single chunk of arbitrary data to the receiver. There is no guarantee that the message is transmitted if the mesh is in the process of restructuring or busy. The sender should provide some resending logic if desired.

Command Parameter Type Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

u8

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

protocolId

u8

One of the protocol IDs mentioned in the table of raw_data_start

payloadHex

HexString or Base64String

The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.

requestHandle

u8

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

Terminal Response

If received by a JSON-capable device, then raw_data_light is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_light",
	"module":4,
	"protocol":2,
	"payload":"abcdeQ==",
	"requestHandle":0
}

The mesh message is documented here.

Messages

Raw Data Start Message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 0.

3

u24

Number of Chunks

The total amount of raw_data_chunk messages for this transmission.

1

u8

Protocol ID

See above for a list of valid protocol IDs.

4

u32

fmKeyId

See EncryptionKeys Heading.

0-40

u8

metadataHex

Optional metadata

Raw Data Start Received Message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 1.

0-40

u8

metadataHex

Same metadata as included in the raw data start message

Raw Data Error Message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 4.

1

u8

Error Code

See above for a list of possible error codes.

1

u8

Destination Code

See above for a list of possible Destination Codes.

Raw Data Chunk message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 2.

3

u24

Chunk ID

The ID of this chunk. The first chunk has ID 1.

1

u8

reserved

A reserved value that must be set to 0.

1-60

u8[1-60]

payload

The payload.

Raw Data Report Message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 3.

12

u32[3]

missings

The IDs of the missing chunks.

Raw Data Report Desired Message

Bytes Type Name Description

8

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 5.

Raw Data Light Message

Bytes Type Name Description

5

connPacketHeader

header

Message Type = 55

1

u8

moduleId

The id of the module that should process the message. 0 to only print it at the gateway.

1

u8

requestHandle

Used to identify the message

1

RawDataProtocolId

protocolId

Raw Data Protocol Id as given above

1-60

u8[1-60]

payload

The custom payload to send