How-to
MQTT store and forward backfill after a connection outage
Short answer
Yes. Tag data collected on the edge during the outage fills the server gap after reconnection, but only when Store and Forward is enabled in MQTT Transmission on the edge and history is enabled on the receiving MQTT Engine tags. Matching historian database names on both systems does nothing.
On this page
- Will the 10 to 11 AM data fill the gap when the link returns?
- Where do you enable Store and Forward on the edge?
- Where do you enable history on the server's MQTT Engine tags?
- Why doesn't the same database name on both historians backfill anything?
- How much data must Store and Forward hold during an outage?
- How do you prove the backfill before you rely on it?
- What do you check when the gap still doesn't fill?
- Questions asked
Yes. Tag data collected on the edge while the 4G link was down reaches the server and closes the 10 to 11 AM gap, but only when two settings are active: Store and Forward enabled in the MQTT Transmission configuration on the Ignition Edge machine, and history enabled on the matching MQTT Engine tags on the server. Enabling the historian on both systems under the same database name contributes nothing to backfill. The two settings, where each one lives, a sizing example for the outage window, and the test that proves the gap closes are below.
Will the 10 to 11 AM data fill the gap when the link returns? #
Yes, when MQTT Transmission on the edge has Store and Forward enabled and the receiving MQTT Engine tags have history enabled, the stored 10 to 11 AM values are forwarded after reconnection and written into server history at their original timestamps.
The transmitting side holds tag data on the edge while the connection is down and forwards it on reconnect; the receiving side writes what arrives into the history provider. Both are required, because either one missing breaks the chain: nothing stored means nothing to forward, and no history means nothing written.
| Side | Setting | What it does |
|---|---|---|
| Edge, MQTT Transmission | Enable Store and Forward for this Group | Holds tag data on the edge through the outage and forwards it on reconnect |
| Server, MQTT Engine tags | History enabled | Writes the forwarded values, at their collection timestamps, into the server history provider |
Where do you enable Store and Forward on the edge? #
The checkbox lives inside the MQTT Transmission configuration, and you set it on the group that carries the tags that dropped out.
- Open the MQTT Transmission configuration and locate the group that carries the tags you saw drop out.
- In the Store and Forward section of the MQTT Group Parameters configuration view, select the checkbox Enable Store and Forward for this Group.
- Save, then confirm the same checkbox on each group involved; the setting is per group, so a second unchecked group still leaves its own gap.
Where an MQTT client driver publishes instead of the Transmission module, the equivalents are the Enable Store and Forward property and the Store and Forward path that defines where the queue lives on disk. The same feature exists on other edge platforms; one documented walkthrough is configuring groov EPIC MQTT store-and-forward for data backfill.
Where do you enable history on the server's MQTT Engine tags? #
In the Configure tab, scroll down to the MQTT Engine section, click Settings, and enable history on the tags; history keys on the tag path, so a tag renamed between outage and flush starts a new history series.
- In the Configure tab, scroll down to the MQTT Engine section and click Settings.
- Enable history on the tags that receive the edge data.
- Leave the tag paths alone from here on: history uses the tag path as its identifier, so renaming a tag after the outage forks its history instead of extending it.
One history setting interacts directly with backfill: minimum time between samples. Set it at or below the edge's sample interval, 5 seconds in the example below, or the provider can treat backfilled points as too close together and thin them.
Why doesn't the same database name on both historians backfill anything? #
Because backfill travels as MQTT messages from the edge's store, not through the database; the database name only decides where each system writes what it receives, and no server-side setting can recreate values that were never sent.
The broker does not fill the role either: it holds queued messages, not an hour of tag history, which is the limit behind whether an MQTT broker stores historic data you can query. If Store and Forward was off during the outage, the MQTT path cannot recover the interval, and the remaining option is a manual export from the edge's own database into the server, the same job as backfilling trend history from a plant historian done once by hand instead of automatically.
How much data must Store and Forward hold during an outage? #
At a 5-second sample interval, a 60-minute outage leaves 720 samples per tag, and 40 tagged points total 28,800 samples, or about 8.6 MB at 300 bytes per stored message.
This is an illustrative worked example, not a documented limit; replace the inputs with your own scan rate, tag count and payload size.
- Samples per tag: 60 min × 60 s/min ÷ 5 s/sample = 720 samples per tag.
- Across 40 tags: 720 × 40 = 28,800 samples.
- Disk: 28,800 × 300 B = 8,640,000 B ≈ 8.64 MB.
Storage scales linearly with outage length, tag count and sample rate, so size the store's disk for the longest outage you accept rather than the typical one. The 300-byte figure is the assumption to verify: measure one of your own stored messages and redo the last line.
How do you prove the backfill before you rely on it? #
Force a controlled outage, watch the server trend develop the gap while the edge keeps collecting, then reconnect and confirm the interval fills with timestamped data; the whole test fits in one shift.
- Confirm the Store and Forward checkbox is set for the transmitting group and history is enabled on the receiving tags; write down the tag paths.
- Break the 4G link for a fixed test window, 15 minutes is enough to see the behavior.
- Watch the server trend: the gap grows while the edge keeps logging locally.
- Restore the link. The flush runs in chronological order, so the gap closes from its oldest end; wait for it to finish rather than judging the first minute.
- Open the trend across the test window and confirm the timestamps inside the gap are the original collection times, not the reconnect time.
Run the test in a window where nobody steers the plant off that trend: operators seeing a live gap, or its sudden fill, can act on it.
What do you check when the gap still doesn't fill? #
Check the two settings first, then the follow-ons that break an otherwise correct setup: a renamed tag path, a history provider thinning samples, and a store that ran out of disk during the outage.
- The checkbox is per group: verify Enable Store and Forward for this Group is set on the group that actually carries the missing tags, not just the first one you opened.
- The receiving tags have history on and their paths are unchanged, because a rename forks the history series and the old interval stays blank under the new name.
- Minimum time between samples is at or below the edge sample interval, or backfilled points arriving closer together get thinned.
- The store's disk held through the outage; a full Store and Forward path stops new storage even though the edge kept scanning.
- The flush was given time to finish: everything flushes in chronological order, and a deep queue takes longer than a shallow one.
The decision falls out of the two settings. With both confirmed and the five checks clean, the gap is recoverable data in transit and you can expect it to close. With Store and Forward off during the outage, the interval is gone from the MQTT path for good, and the manual backfill described above is the only recovery.
Questions asked
Does the edge's own historian already protect the outage data?
It protects a local copy, not the transfer. The edge's tag logging keeps collecting into its own database while the link is down, but nothing republishes that copy over MQTT unless Store and Forward is enabled. The two mechanisms answer different failures: the local historian covers loss on the edge, store and forward covers the path to the server.
How long does the flush take once the link returns?
Everything flushes in chronological order, oldest first, so the gap closes from its oldest end and a deep queue takes longer than a shallow one. The wall-clock duration depends on queue depth and link throughput; the worked example's 60-minute outage at a 5-second interval is 28,800 stored messages across 40 tags, which is the size of the job.
Does broker persistence on a forwarding gateway do the same job?
No. Broker persistence retains queued messages across a broker restart; it never captures tag values that were not published because the uplink was down. A gateway broker between the edge and the server cannot store what the edge never sent, which is why the store belongs on the transmitting side, in MQTT Transmission or the publishing driver.
Should history sit on the MQTT Engine tags or on tags that reference them?
The documented backfill path is history on the MQTT Engine tags themselves, and that is where this procedure enables it. Some deployments keep the permanent history on downstream tags that reference the MQTT tags and reserve direct history on the MQTT tags for diagnostics; if you use that pattern, run the outage test to confirm the flush drives the downstream tags.
How much disk does a multi-day outage need?
Storage scales linearly with outage time, tag count and sample rate. From the worked example, about 8.64 MB covers one hour for 40 tags at a 5-second interval and 300 bytes per message, so a full day at the same settings is roughly 24 times that, about 207 MB. Replace the 300-byte assumption with your measured payload size before trusting the total.