Explainer
Exporting Sparkplug B out of Geo SCADA to a historian
Short answer
No modified DDK driver will push Sparkplug B out of Geo SCADA, because the driver layer is pull-only in the outbound direction. Build the publisher on the Client API Sample DataFeeder pattern, which watches about 100,000 points per instance; for millions of points, use HSI or a database-level connector.
On this page
- Can a Sparkplug DDK driver be made event-driven to push data out?
- Which export paths out of Geo SCADA exist, and what limits each?
- How many points can a Client API export watch before it stops scaling?
- Why does the DataFeeder pattern fall apart at a million points?
- Does switching the payload from JSON to Sparkplug B change scalability?
- What must the Sparkplug B publisher itself handle?
- How do you size a Client API Sparkplug B export that stays inside the ceiling?
- Which export path should you choose for your point count?
- Questions asked
Geo SCADA has no event-driven push driver, so no modified Sparkplug DDK driver will export on change: the driver layer is pull-only in the outbound direction. The working paths are the Client API export, the Sample DataFeeder pattern, which reaches about 100,000 watched points per instance, and beyond that the database-level feeds, HSI or the vendor's own historian connectors. Below is what each path carries, where each one stops, and how to choose.
Can a Sparkplug DDK driver be made event-driven to push data out? #
No. A Geo SCADA driver is a pull mechanism: the server asks the driver for data, the DDK exposes no event hook that fires when a value lands, so even a Sparkplug B driver would remain a poller seen from the server.
Inbound MQTT is easy because a driver sits on the field side: it subscribes or polls the PLCs and writes values into the Geo SCADA database. The outbound direction has no mirror of that. A DDK driver formats and serves data when the server asks for it; nothing in the interface tells it a point has just changed. A custom Sparkplug driver would therefore still be a poller from the server's viewpoint, and polling is exactly what a millions-of-points requirement cannot afford. That closes the DDK route for this job, however the driver is written.
Which export paths out of Geo SCADA exist, and what limits each? #
Five outbound mechanisms exist — the Client API export, DNP3 Slave RTU, OPC DA/HDA, the vendor's own historian connectors, and the HSI historian-to-historian link — and point count plus destination decide between them.
| Path | Direction | Practical ceiling | What decides it |
|---|---|---|---|
| DDK driver output, including a custom Sparkplug driver | Pull from the server's viewpoint | Not a push path | No event trigger; the server still asks the driver for data |
| DNP3 Slave RTU | External master pulls | A few thousand points | Pull by a master; no Sparkplug metric shape or birth semantics |
| OPC DA/HDA server | Client pulls | Not established | A reported attempt could not extract device-timestamped time-series data |
| OPC UA server | Under consideration | Not shipping | Do not design around an output that has not shipped |
| Client API export, Sample DataFeeder pattern with on-change | Application-side publish | About 100,000 watched points | Per-point registration and whole-set reads load the server |
| eDNA and WWH/Insights connectors | Database-level feed | High | Feed only their own historian systems, not a custom historian |
| HSI historian-to-historian link | Database-level feed | Millions of points | Same internal mechanism as the eDNA and WWH feeds; a projects-team offering, not a download |
The OPC DA/HDA row records a reported result, not a verified limit: one engineer could not extract device-timestamped time-series data through it, and no working counterexample has surfaced. Treat it as unproven rather than impossible.
HSI is a historian-to-historian link from the vendor's UK projects team, using the same internal mechanism as the eDNA and WWH/Insights feeds. It is not published as a download; you reach it through your Schneider Electric sales representative, who routes the request to that team. The eDNA and WWH/Insights connectors also work at database level, but they feed only their own database systems.
How many points can a Client API export watch before it stops scaling? #
About 100,000 watched points per client instance, with deliberate architecture and update times; the figure is a practical deployment ceiling, not a published specification, and your change rate moves it in both directions.
The Sample DataFeeder shows the mechanism: it uses the Advanced section of the client to register each watched point for on-change reporting, so a point that has not moved costs nothing at export time. The load concentrates in two places — the periodic read that checks the whole watched set for changes, and the one-call-per-point registration at startup. Both execute against the server database, which is why the ceiling is a server-load ceiling rather than a protocol ceiling.
A supported product derived from the DataFeeder code has been announced, with Sparkplug B listed as a planned output. Planned is not shipping: confirm availability with the vendor before locking a design on it.
Why does the DataFeeder pattern fall apart at a million points? #
Two documented mechanisms: the change-check interval stretches to 15 to 30 or more minutes, and startup registration — one call per point — runs about an hour at a million points, loading the server hard enough to need a dedicated Standby-Only machine.
Interval first. Reading changed data across a million points loads the server so heavily that the check has to stretch to 15 or 30 minutes or more. That interval is latency added to every exported value, which defeats a near-real-time historian feed before a single message is published.
Registration second. On startup the feeder sets on-change in one call per point. At a million points that startup runs about an hour, and it repeats after every restart and every redundancy changeover — a failover that costs an hour of export coverage.
Both loads land on the server your operators are watching. The documented mitigation at that scale is a dedicated Standby-Only server carrying the export, so the control-room workload stays protected. None of this depends on the payload: the mechanism is per-point calls and database reads, so JSON, Sparkplug B or anything else hits the same wall.
Does switching the payload from JSON to Sparkplug B change scalability? #
No. Scalability is set by the source server and the destination's ingest rate, not the serialization; Sparkplug B changes bytes on the wire and topic structure, not the number of registration calls or database reads per change.
The source sets how fast data can leave; the destination sets how fast it can arrive — a cloud event hub or time-series service ingests only so much per second, and sizing it is part of the design. Between them, the serialization is a rounding error.
What Sparkplug B does change is the work your publisher has to do, which is the next question.
What must the Sparkplug B publisher itself handle? #
Births, timestamps and grouping: publish a birth message describing every metric before any data message, re-emit births after a broker reconnect or on a rebirth request, and carry value, quality and the point's source timestamp in each metric.
Map Geo SCADA's value, quality and source timestamp onto the metric's fields, and stamp the metric with the source timestamp — never the export time, or the historian's record of when things happened is destroyed.
Group the watched points into a bounded number of edge nodes or devices, each holding a few thousand metrics, so a rebirth payload stays small and a reconnect does not fire one enormous burst. Follow the sample feeder's conservative bias: bring points up gradually, partition by partition, rather than all at once.
Handle broker disconnects explicitly: re-publish births on reconnect, and buffer or re-read the changes since the last successful publish so the historian shows no gap. Throttle the publish rate to what the destination accepts rather than what the source produces.
Structurally this is the same job as pushing values to MQTT through an edge gateway, with a Sparkplug payload and stricter topic discipline in place of plain JSON. This article covers the data direction only; commands flowing back are a separate design.
How do you size a Client API Sparkplug B export that stays inside the ceiling? #
Keep each exporter instance well under roughly 100,000 watched points, set the check interval from measured read time rather than a target latency, and stagger the per-point registration at startup so the calls never arrive as one storm.
Take 60,000 points as an illustrative input — inside the ceiling with margin. Three numbers size the system.
Startup: registration is one call per point. The documented million-point case takes about an hour, which is 1,000,000 calls ÷ 3,600 s ≈ 280 calls per second. At the same rate, 60,000 points need 60,000 ÷ 280 ≈ 214 s, about three and a half minutes from start to steady state — and it repeats on every restart, so stagger the partitions.
Steady state: assume 5% of points change each minute, an illustrative rate for a slow process. 60,000 × 0.05 = 3,000 changes per minute, which is 3,000 ÷ 60 = 50 metrics per second on average. Peaks run several times the average, so size the publisher and the broker for the peak, not the mean.
Grouping: at 5,000 metrics per edge node, 60,000 points give 60,000 ÷ 5,000 = 12 nodes. A reconnect then rebirths at most 5,000 metric definitions per node instead of one 60,000-metric burst.
Set the check interval from measured read time on your own server — start long and shorten while watching server load — and run the exporter on a machine of its own, a fanless mini PC beside the SCADA server, so the export never competes with an operator workstation.
Which export path should you choose for your point count? #
Up to a few thousand points, DNP3 Slave RTU works but the Client API gives Sparkplug B directly; from a few thousand to about 100,000, Client API export with on-change; into the hundreds of thousands and millions, HSI or a database-level connector.
| Watched points | Path | Why |
|---|---|---|
| Up to a few thousand | DNP3 Slave RTU is workable, but Client API export gives Sparkplug B directly | DNP3 is pull and carries no metric shape of value, quality and source timestamp together |
| A few thousand to about 100,000 | Client API export, on-change, with a Sparkplug B publisher | The only application-side push path; needs partitioning, staggered registration, measured intervals |
| Hundreds of thousands to millions | HSI, or a database-level connector | Per-point registration and read load do not scale; HSI links historian to historian directly |
For the requirement as stated — millions of incoming points exported to a custom historian over Sparkplug B — the plan has two parts. Use HSI where the target historian can be reached through it, and ask your sales representative early, because it is a projects-team offering rather than a catalog item. Where HSI cannot reach the target, partition the export so no single Geo SCADA client watches more than the roughly 100,000-point range, and size every check interval from measurement. On the receiving side, apply the same discipline as any plant historian selection by scale and support needs decision, because a feed that outruns its destination is a feed with gaps.
Questions asked
Where do I find the Client API documentation and the Sample DataFeeder code?
The Client API is documented in the Client API Guide, a help file installed with Geo SCADA; find it on the Start menu in the EcoStruxure Geo SCADA group. The Sample DataFeeder is published in the vendor's public example-code repository and compiles into a working starting point for an MQTT publisher.
Does running the exporter on its own PC remove the load from the SCADA server?
No. The client host only runs your code; the on-change registrations and the change-check reads execute against the Geo SCADA server database wherever the client sits. A separate host protects your workstation, not the server, which is why the large-scale answer is a dedicated Standby-Only server rather than just another PC.
Is the OPC UA server a way out of this instead?
Not yet. It has been listed as under consideration for years without shipping, so it cannot anchor a design. When it does ship it will be a path a consumer attaches to, not an application-side publisher, and the OPC DA/HDA attempt that could not extract device-timestamped data suggests the pull routes deserve skepticism for time-series export.
How large should each Sparkplug edge node be in the export?
Small enough that one rebirth burst stays manageable. A node holding 5,000 metrics re-sends 5,000 metric definitions on reconnect, so 60,000 points across 12 nodes never fire more than 5,000 at once, and staggering the nodes spreads even that. One node holding everything turns every reconnect into a full re-birth of the whole plant.
Can DNP3 Slave RTU carry the export at small scale?
Yes, up to a few thousand points, if you accept a pull: an external master polls the RTU on its own schedule. What you lose is Sparkplug B itself, meaning the metric carrying value, quality and source timestamp together plus birth and rebirth semantics, which is why the Client API path is preferred even below the DNP3 point count.