IMD SolutionComputing LibraryShop main store

Troubleshooting

SCADA MQTT points blank with nested JSON payloads

Edge computing and IIoT Published 8 min read

Short answer

Blank points on a live MQTT subscription mean the payload does not match what the configured driver parses, not a timestamp fault. Capture one raw message, validate its JSON, compare it with the driver's documented message formats, then either publish that format or flatten the payload through an edge intermediary. Keep timestamp handling separate until the values map.

On this page
  1. Why are my MQTT points blank when the broker connection is fine?
  2. Does the Geo SCADA MQTT driver support nested JSON?
  3. How do I capture the exact message the publisher sends?
  4. Is the JSON itself valid before I blame the driver?
  5. Which fix fits: change the publisher, flatten at the edge, or write a driver?
  6. How do I flatten the payload to the documented format?
  7. Is the timestamp really why the points stay blank?
  8. When should I move to Sparkplug B instead of custom JSON?
  9. How do I verify the fix point by point?
  10. Questions asked

Blank points on a live MQTT subscription are a payload problem, not a connection problem. MQTT transports bytes and defines nothing about the JSON inside them; the SCADA driver maps only the message formats its documentation defines. A payload that splits values under payload_fields and the timestamp under metadata sits outside the Geo SCADA JSON driver's documented formats, which is why payload_fields.sensor1 maps nothing. The fix path is fixed: capture one raw message, validate the JSON, compare it with the driver's documented formats, then publish a supported format, flatten the payload at an edge intermediary, or use a driver built for that format. The timestamp's position is not the cause.

Why are my MQTT points blank when the broker connection is fine? #

Because the payload does not match what the configured driver parses: MQTT only transports the bytes, and a working subscription proves delivery, not that the driver can address your nested fields.

Transport and payload are separate layers, and a green connection proves the first one only. The payload is an application-level choice: JSON, CSV, raw text, even base64-encoded content ride the same subscription, and field names, nesting, timestamp placement and value semantics all belong to the publisher. Two devices can both publish JSON and still be mutually unintelligible to a given driver.

What you observe Where the fault sits First move
No message reaches the subscriber at all Transport: broker address, credentials, topic filter Fix delivery before touching the driver
Messages arrive, no point ever updates Payload format versus the configured driver Capture one raw message and compare formats
Flat fields map, nested fields stay blank Field addressing outside the documented formats Stop respelling paths; flatten or change format
Values appear, point time is wrong or missing Timestamp extraction Check the documented timestamp format separately

When nothing arrives at all, that is transport: work through a PLC MQTT client that never reaches your broker before asking any payload question.

Does the Geo SCADA MQTT driver support nested JSON? #

Not in the standard JSON driver: its guide documents two message formats, and values under payload_fields with time under metadata sit outside both, so references like payload_fields.sensor1 have nothing to bind to.

The JSON Schema A driver's help documents two supported message formats, and a format defines which fields exist and how a point references them. A field the format does not describe cannot be exposed by spelling its JSON path differently, so the attempted dotted references mapping nothing is the expected result, not a syntax puzzle waiting for the right separator.

For payloads outside those formats, the vendor's exchange hosts a Basic MQTT sample driver whose stated purpose is telemetry in any payload format; you adapt and build it as a custom driver, which is a development task rather than a tag-configuration task. One reported route cloned and built Driver-BasicMQTTClient against The Things Stack using the topic descriptor my-app-id/devices/my-dev-id/up/<field>, and it worked once the region had uplink fields enabled. Treat that as one publisher's result with that build, not a capability of the packaged driver. The same driver family documents JSON, Text and Sparkplug message handling, so a payload change can be a driver change.

How do I capture the exact message the publisher sends? #

Subscribe with the same broker, credentials and topic filter the driver uses, and record one unmodified message — topic string and full payload, punctuation and quotation marks included — before changing any point configuration.

Subscribe beside the SCADA client, not through it: same broker address, same credentials, same topic filter string, character for character. MQTT topics are case-sensitive, and one wildcard level of difference hides messages silently. Any MQTT client that prints topic and payload together does the job; a Node-RED debug node wired to the same subscription shows the raw payload without reformatting it. Save one complete message, including whitespace and the exact timestamp text. That captured message is the test vector for every step that follows — a hand-typed copy of what the publisher should be sending is not.

Is the JSON itself valid before I blame the driver? #

Validate it first, because strict JSON rejects the sample as received: one comma is missing between the two sensor members and one trailing comma follows the time member — two faults, and a parser stops at the first.

As received (illustrative capture from a commissioning case):

{
  "payload_fields": {
    "sensor1": 67593
    "sensor2": 83463
  },
  "metadata": {
    "time": "2021-02-10T12:28:16.002839378Z"
  },
}

Strict JSON requires a comma between the sensor1 member and the sensor2 member, and forbids the comma after the time member's closing brace: 1 missing plus 1 trailing equals 2 faults, and a strict parser rejects the message before any field mapping is attempted. The corrected message parses:

{
  "payload_fields": { "sensor1": 67593, "sensor2": 83463 },
  "metadata": { "time": "2021-02-10T12:28:16.002839378Z" }
}

Valid JSON is necessary, not sufficient: it makes the message parseable, it does not make the driver map it. Run the capture through any strict JSON parser before comparing formats, and hold the publisher to the same standard — the publish-side version of this is fixing extra characters in a PLC MQTT JSON payload.

Which fix fits: change the publisher, flatten at the edge, or write a driver? #

Match the fix to who controls the payload: if the publisher can change, emit a documented format; if it cannot, flatten at an edge intermediary; commission a custom driver only when several publishers share one unsupported format.

  1. Publisher output is changeable: configure it to emit one of the driver's documented formats, or Sparkplug B, then capture the raw output again before touching the SCADA database.
  2. Publisher output is fixed: republish flat through an intermediary — a Node-RED flow or small script on an edge box, one flow per publisher.
  3. Several publishers share one unsupported format: a custom driver on the vendor's driver development kit, run as a software project with version control and tests, not a configuration tweak.
  4. Neither side can change and no driver fits: raise a driver-enhancement request through the vendor's official support channel; that is the documented route for capability asks.

How do I flatten the payload to the documented format? #

Lift each leaf to the top level in one intermediary — a Node-RED flow or small script that republishes sensor1, sensor2 and time as flat fields on the topic the driver subscribes to.

Worked example. Input is the corrected nested capture; output is the flat shape the point configuration expects. The field names come from the driver's documented format page; the shape below is illustrative:

{
  "sensor1": 67593,
  "sensor2": 83463,
  "time": "2021-02-10T12:28:16.002839378Z"
}

Count the work: the nested message holds 2 containers (payload_fields, metadata) and 3 leaves; the flat message holds 0 containers and the same 3 leaves. You configure 3 point-to-field mappings either way — flattening rewrites 3 paths and removes 2 levels of nesting, and adds no data.

A Node-RED function node doing the lift (illustrative; msg is Node-RED's message object):

// Flatten one publisher's nested uplink to the driver's flat format
const fields = msg.payload.payload_fields;   // object: nested values
const stamp = msg.payload.metadata.time;     // string: UTC timestamp text
msg.payload = {
  sensor1: fields.sensor1,
  sensor2: fields.sensor2,
  time: stamp
};
return msg;

Test it: feed the corrected capture through the flow and the debug output is the flat object above; change sensor1 to 67600 in the input and the output shows 67600 — proof the mapping follows the leaf, not the container. A community node builds the same key-value shape straight from the topic path, turning topic home/sensors/0/temp into payload {"temp": 23.1}, when the publisher can be steered toward flat output.

Host the flow on a fanless mini PC in the control cabinet — the same host pattern as bridging OPC UA PLC data to MQTT with an edge PC. Other platforms meet the same wall with the same answer: extracting embedded values from payloads in Ignition follows the identical capture-and-parse discipline.

Is the timestamp really why the points stay blank? #

No: a value maps or fails on its own path, and moving metadata.time into the data object cannot make payload_fields.sensor1 addressable — it only changes the layout and creates a second mismatch.

Handle precision as a separate, later question. The sample stamp carries nine fractional digits: 0.002839378 s is 2839378 ns, or 2.839378 ms past 12:28:16. If the driver's documented timestamp format accepts milliseconds only, a strict parser rejects the nine-digit form; one that truncates keeps .002 and drops 0.839378 ms, one that rounds keeps .003 and gains 0.160622 ms. That loss matters for sequence-of-events records and not for most process points — and none of it can blank a value point. Test value mapping first, timestamp handling second, and never move the timestamp to chase a mapping problem.

When should I move to Sparkplug B instead of custom JSON? #

When you control both ends: Sparkplug B fixes the payload structure by standard — metric names, birth and death certificates included — so a compliant driver maps metrics without any per-publisher parsing.

Sparkplug B exists because payload structure was otherwise everyone's private choice. A Sparkplug B publisher sends a defined metric structure with birth and death certificates, and Geo SCADA's MQTT driver set includes Sparkplug handling, so the mapping problem disappears on both ends of a greenfield link. It cannot rescue you from a fixed upstream JSON, such as a LoRaWAN network server's uplinks; that stays a flatten-at-the-edge case. For the wider protocol trade-offs, read OPC UA vs MQTT Sparkplug from a controller to SCADA.

How do I verify the fix point by point? #

Replay one known message with unambiguous test values — sensor1 set to 11111 and sensor2 to 22222 — confirm each point takes its own value, then replay a second message so the mapping is not a one-off.

  1. Confirm the configured subscriber received the exact test message on the expected topic.
  2. Confirm each point independently: each reads its own test value, not its neighbour's, and not a stale one.
  3. Confirm the point time equals the payload time, to the precision the driver documents.
  4. Replay a second message with different values to separate a working mapping from a one-off.
  5. If the message arrives and points stay unchanged, return to payload compatibility and read the driver's diagnostics for parse errors; if no message arrives, you are back to transport.

One caution before replaying: a replay on the live topic writes test values into the historian. Use a test topic for the trials, or annotate the entries so production reports can exclude them.

Questions asked

Why did payload_fields.sensor1 map nothing as a point reference?

Because a dotted JSON path is not one of that driver's documented addressing rules. The JSON driver exposes only the fields its two documented message formats define, so no spelling variant of the path can reach a field outside them. The working options are a supported format, an intermediary that flattens the payload, or a driver built for that message shape.

How much timestamp precision does the sample carry, and does it matter?

The stamp 2021-02-10T12:28:16.002839378Z has nine fractional digits: 2839378 ns, or 2.839378 ms past the second. A millisecond-only parser truncates to .002 and loses 0.839378 ms, or rounds to .003 and gains 0.160622 ms. It matters for sequence-of-events records, not for blank points: values map independently of where the time field sits.

Is a custom MQTT driver worth building for one publisher?

Usually no. One publisher justifies a republishing flow that flattens the payload: an afternoon of Node-RED on an edge box, transparent to audit. A custom driver on the vendor's development kit earns its cost when several publishers share one unsupported format and you need native points, alarms and history without an intermediary. File capability requests through official support.

Does Sparkplug B remove nested-payload problems for good?

Where you control both ends, yes: Sparkplug B defines the metric structure, names and birth and death certificates by standard, so compliant drivers map metrics without per-publisher parsing. It cannot change a fixed upstream payload such as a LoRaWAN network server's JSON uplinks, and that case still needs flattening at the edge before the SCADA driver sees it.

Keep reading

  • Comparison

    OPC UA vs MQTT Sparkplug from a controller to SCADA

    Pick OPC UA when the controller and the SCADA host share one plant LAN you control: the host subscribes inbound to the controller's browsable server. Pick MQTT Sparkplug B when the controller sits behind NAT, on cellular or at a remote site, or when several consumers need the same tags: outbound-only connections plus birth and death certificates give automatic tag discovery and stale detection.

    Edge computing and IIoT

  • How-to

    Push ClearSCADA values to MQTT through an edge gateway

    ClearSCADA and Geo SCADA do not publish straight to an MQTT broker: the value travels from the SCADA server through an edge gateway to the broker as two independent client connections. Prove the SCADA read on one known point and the MQTT publish on a fixed test payload separately, join them only after both pass, and chase a login failure in the server's own web logs, never in the broker settings.

    Edge computing and IIoT

  • Troubleshooting

    Fixing extra characters in a PLC MQTT JSON payload

    The extra characters come from publishing a UTF-16 WSTRING to a subscriber that reads UTF-8, and from sending SIZEOF bytes of the allocated buffer instead of the text. Convert the finished JSON with JSON.Stu.ConvertUTF16toUTF8 into a STRING(2000), then publish TO_UDINT(JSON.Stu.StrLenA(ADR(utf8JsonData))) bytes. Keep the MQTT 5 properties; they declare UTF-8 but convert nothing.

    Edge computing and IIoT