Selection guide
Choosing an edge data path for periodic records
Short answer
For independent periodic samples, Tag Historian with Edge Sync Services is the reliable path and buffers 35 days locally. For one atomic row per collection cycle, use MQTT Transmission records landing in MQTT Recorder. For true transaction groups, license a standard gateway for that node.
On this page
- Match the data contract before the platform
- What Edge Sync Services will and will not carry
- Records with MQTT Transmission and MQTT Recorder
- Buffer arithmetic for the outage you must survive
- Where the 00:00 timestamps come from
- Import rules that survive a reconnection
- When the answer is a standard license
- Questions asked
The choice is decided by what one row in your central table means, not by which module looks closest to store and forward. If the table accepts independent samples — one value per tag per timestamp — run Tag Historian in periodic mode and let Edge Sync Services carry it: that transfer is reliable, and the edge gateway holds 35 days of tag history locally. If the table needs one atomic record per collection cycle, build it with MQTT Transmission records landing in MQTT Recorder, because Sync Services moves samples, never records. And if you genuinely need transaction groups on that node, license a standard gateway for it. Do not bridge the gap with a bespoke in-memory queue.
Match the data contract before the platform #
Write down what one row means before touching a single tag. Two contracts cover every periodic table:
- Independent samples: each tag contributes its own value with its own timestamp; a row holds one tag's sample, or several tags pivoted by a matching rule you own.
- Atomic record: all tags of one collection cycle commit together or not at all; a partial cycle must never reach the table.
Record semantics are not exotic at the source. Reading ION7400 data recorder historical records via Modbus is a typical device that emits fixed records rather than loose samples, and your table inherits whichever contract the source and the transport preserve.
| The table needs | Build the path with | Edge-side persistence | Verify before cutover |
|---|---|---|---|
| Independent samples, one value per tag per timestamp | Tag Historian in periodic mode, Gateway Network, Edge Sync Services | 35 days of tag history, 7 days of alarm journal and audit trail | Outage window inside the buffer; watermark and idempotent import |
| One atomic record per collection cycle | MQTT Transmission records, MQTT Recorder at the central gateway, timer script into the table | Transmission's on-disk store-and-forward queue | Record completeness after replay, duplicate handling, queue disk growth |
| Transaction-group behavior on that node | A standard gateway license | Managed store-and-forward to your database | License includes the modules you verified |
One option is rejected outright: gateway messaging plus a bespoke in-memory queue. The queue dies with the process, and you inherit retry, ordering and duplicate handling with no persistent watermark. That is not store and forward; it is data loss waiting for a restart.
What Edge Sync Services will and will not carry #
On Ignition Edge the function is named Edge Sync Services, and the name is the honest one: it synchronizes tag history, alarm journal and audit trail to a full gateway over the Gateway Network. For that job it is dependable. If synchronization runs slowly, measure the transfer rate and the link before switching architectures — the samples still arrive.
The 8.1 manual documents the local hold at 35 days of tag history and 7 days of alarm journal and audit trail, and notes that earlier releases carried a single week, so confirm the buffer on the release you run.
What it will not carry is a record. Each tag's samples travel as independent historian entries with their own timestamps and availability, so a twenty-tag cycle arrives as twenty sample streams, not one row. Pivot them afterwards and skew between tags, a tag that skipped a cycle, and late-arriving backfill all become your code's problem. That burden is exactly what records exist to remove.
Records with MQTT Transmission and MQTT Recorder #
Define one record in MQTT Transmission containing every tag in the collection cycle. Each cycle publishes one record, and MQTT Recorder at the central gateway subscribes and writes its record tables, which your timer script copies into the custom table. The record is the unit end to end: a cycle is complete or it is absent, with no pivot logic at the center.
Persistence sits on both sides. Transmission queues unsent publishes on disk and replays them when the broker returns, and the broker itself keeps no history — whether an MQTT broker stores historic data you can query has a short answer: no, by design, so the recorder store is what your script drains. The recovery behavior is the pattern covered in MQTT store and forward backfill after a connection outage, and the same arrangement exists on other edge platforms; groov EPIC MQTT store-and-forward for data backfill documents the equivalent setup. Unlike the historian buffer, no fixed day window is published for these queues, so measure queue disk growth during your outage test instead of assuming one.
Verify three things on this path before cutover: record completeness after a replay, duplicate suppression when a record is delivered twice, and the values of the first records after reconnection — a historian storing zero after an MQTT edge node reconnects is the failure mode your test must rule out.
Buffer arithmetic for the outage you must survive #
Example inputs, chosen for the arithmetic and not from any manual: 24 tags, history sample mode periodic, 5-minute period.
- Samples per tag per hour: 60 min ÷ 5 min = 12
- Samples per tag per day: 12 × 24 h = 288
- Samples per day, all tags: 288 × 24 tags = 6,912
- Capacity of the 35-day window: 6,912 per day × 35 days = 241,920 samples
- A 4-hour outage: 12 per hour × 4 h = 48 per tag, then 48 × 24 tags = 1,152 samples
- A 14-day outage: 6,912 per day × 14 days = 96,768 samples, still inside the window
Every outage up to 35 days recovers in full at this rate, so commit the recovery SLA at 30 days and keep five days of margin. Loss begins on day 36 of disconnection, oldest samples first, at 6,912 samples per day for this fleet. Shorten the period to 1 second and the same tags produce 86,400 × 24 = 2,073,600 samples per day; the time window does not change, but the disk behind it now needs the same sizing discipline as choosing edge storage for high-rate vibration data. If this table is a quality or compliance record, run the full interruption test before cutover: an undetected gap there is a records failure, not a software inconvenience.
Where the 00:00 timestamps come from #
system.historian.queryRawPoints returns full date-and-time values, so a timestamp that lands at 00:00 had its clock stripped somewhere after the historian. Find where, in this order:
| Observation | Likely boundary | Check |
|---|---|---|
| Raw dataset shows the correct time, the table shows 00:00 | SQL binding or destination column | Store date and time in the column; bind the timestamp value, not a formatted string |
| Script log is correct, the report shows 00:00 | Display or report query | Remove date-only casts; read the stored value back directly |
| Raw dataset already shows 00:00 | Query bounds or the source sample | Compare the same point in local edge history and synchronized central history |
| Times differ by a fixed offset | Time-zone interpretation | Compare the edge gateway, central gateway, database session and display time zones |
Log one diagnostic row — tag path, the timestamp value and its type, the process value, the quality — and follow that row through the prepared statement into a read-back. Do not convert a timestamp to a string anywhere between the query and the insert.
Import rules that survive a reconnection #
A timer that only queries values newer than the last inserted timestamp skips samples that arrive late after an outage. Make the import idempotent instead:
- Keep the import watermark in the destination database, never in script memory.
- Query an overlapping window starting before the watermark, through the last completed collection cycle.
- Give every row a stable identity: for independent samples, tag identity plus source timestamp. Insert or ignore on that key.
- Commit the new rows and the updated watermark in one transaction.
- Rerun the same window twice as the acceptance test: the second run must add no rows and alter no stored timestamps.
- If you pivot tags into one row, match on the documented collection key with a stated skew allowance and hold incomplete sets until a timeout. If the table cannot tolerate an incomplete set, the contract is wrong — move to records.
One identity caveat decides the architecture: if the process can legitimately produce two values for the same tag and timestamp, tag-plus-timestamp is not unique, and the records path is the correct one.
When the answer is a standard license #
Ignition Edge does not include transaction groups — no data- or time-triggered groups, no OPC writes from a group, no managed store-and-forward into your database. When any of those is a hard requirement on that node, license a standard gateway for it and keep the edge node for collection; trim the license to the modules you have verified, and confirm module inclusion before purchase rather than after. If the only hard requirement is the atomic row and the site can carry MQTT, records remain the lighter answer and the edge license stays where it is.
Questions asked
Is Edge Sync Services reliable for moving tag history to a central gateway?
Yes, for the job it is built for: synchronizing historian samples to a full Ignition gateway over the Gateway Network. The 8.1 manual documents a 35-day tag history buffer plus 7 days of alarm journal and audit trail on the edge side. Loss starts only when a disconnection outlives the buffer, oldest samples first, so keep your recovery window inside those 35 days.
Why do queryRawPoints timestamps show 00:00 in my custom table?
The function returns full date-and-time values, so 00:00 means the clock was stripped downstream: a date-only column type, a formatted-string binding, or a date-only cast in a report query. Inspect the raw dataset first, bind the timestamp value itself rather than a string, and check for a fixed offset caused by mismatched gateway, database and display time zones.
Can gateway messaging with an in-memory queue replace store and forward?
Not durably. The queue lives in process memory, so a restart or power loss drops everything not yet written, and you inherit retry, ordering and duplicate handling with no persistent watermark. Use the persistence the platform already provides: the edge historian buffer or the on-disk MQTT store-and-forward queues.
How do I size the buffer for my own sample rate?
Samples per day per tag equal 1440 divided by the period in minutes. At a 5-minute period that is 288 samples per tag per day, so 24 tags produce 6,912 samples per day and 241,920 across the 35-day window. Halve the period and both numbers double; lengthen the tolerated outage and the required window grows day for day.