Troubleshooting
Why Ignition MQTT shows tagid None and what to key on
Short answer
tagid is None because the MQTT payload never carries it: tag IDs are indexes inside the Ignition tag history provider, assigned and retired as tag configuration changes. Key the lake on a site-scoped tag path with the site identity in the MQTT namespace, and trace duplicate sqlth_data_ table sets through their driver IDs in sqlth_drv before stopping the extra writer.
On this page
- Is tagid supposed to be in the MQTT message?
- Why does one tag path have two different tag IDs?
- What does the number in the sqlth_data_ table name mean?
- How do I find which gateway is writing the duplicate historian table?
- What should the data lake key on instead of tagid?
- Should I build a tagid-to-tag-path mapping table in the lake?
- Is it safe to delete the extra sqlth_data_ tables?
- Questions asked
No — tagid is not missing from the MQTT feed by accident, and no mapping table will make it reliable. Tag IDs are indexes inside the Ignition tag history provider: assigned when a tag begins logging, retired when its configuration changes, and meaningless outside the historian's own tables. The message shows tagid: None because the payload never carried that field. Key the lake on a site-scoped tag path you own, and treat the extra sqlth_data_ table sets as a second historian driver to trace through sqlth_drv — not as a tag identity problem.
Is tagid supposed to be in the MQTT message? #
No. The tagid is an index inside the Ignition tag history provider's database tables, assigned and retired by the historian itself, and the MQTT payload from Transmission carries tag paths and the metadata you configured — never a historian row index.
A displayed tagid: None is your consumer answering a question the message never heard. Three conditions produce it, and each has a different fix: the payload has no such field, the field is present but explicitly null, or the handler reads the wrong nesting level. Sort out which one at the boundary:
- Capture one raw message where MQTT meets your cloud, before any rule, transform, or lake serialization touches it.
- Record the topic and the complete payload shape, metadata included; redact process values if you must, but keep the structure.
- Compare the raw message with what the rule forwards and what the handler receives.
- Inspect the final lake record for fields that are absent, null, renamed, or reshaped at each hop.
Decision: stop expecting tagid from Transmission. If you want a stable identifier inside the payload, define one as tag metadata you maintain, so it travels with the message and never depends on the historian.
Why does one tag path have two different tag IDs? #
Because the historian treats the ID as a time-scoped association, not a permanent identity: sqlth_te records which tagid a tag path maps to and when, a configuration change retires the old ID and issues a new one, and a query spanning both must gather every relevant ID.
Two IDs on one path is normal historian behavior over the life of a tag, not a referential-integrity defect. Retired IDs still answer queries over the old rows, and more than one history provider can store the same tag path. Ignition's history query mechanisms perform that resolution; a direct SQL statement against one raw sqlth_data_ table does not, because the raw layout carries no knowledge of which IDs were ever valid for the path.
Decision: never collapse two tag IDs into one "real tag" downstream, and never read a new ID as a new physical tag. The tag path is the identity; the ID is bookkeeping.
What does the number in the sqlth_data_ table name mean? #
It is the historian driver ID — the drvid recorded in sqlth_drv — which stands for a source gateway name and origin tag provider, so two numbered table sets mean two drivers have provisioned tables in that database, not that one tag split in two.
The naming pattern is sqlth_data_#_yyyy_mm: the number is the drvid and the date is the monthly partition of raw rows. Driver identity follows the source gateway name and tag provider, so renaming a gateway can register a new driver even though the tags never changed. Two nearby faults are not this problem: a history provider whose sqlth_drv table never gets created has failed provisioning outright, and tag historian sqlth_drv table not created fix covers it. A gateway log repeating a sqlth_drv already-exists warning every 60 seconds is a schema-creation complaint rather than duplicate writes; sqlth_drv already exists warning fix addresses that one.
Decision: read the number as "which driver wrote this table," never as "which tag." When two driver numbers are live, the next job is naming both drivers.
How do I find which gateway is writing the duplicate historian table? #
Trace both driver IDs to their sqlth_drv records before touching any data: identify the gateway and provider behind each, check where the database connections originate, and the parallel writer is almost always a development or test gateway loaded from a production backup — often running under the same gateway name.
Work the trace in this order:
- Map each table-set number to its record in
sqlth_drv, and write down the source gateway and tag provider each record represents. - Mark which driver is supposed to be active in production, so the other one has a name and an owner.
- Check where connections to the historian database originate, in database-side connection records and in each gateway's connection settings, looking for a development or test instance still pointed at production.
- Confirm which tags have history enabled on each source; two providers can log the same logical tags even when one was meant to be disabled.
- For one tag path present in both table sets, compare timestamps over the same month: matching timestamp sets mean the same stream written twice, not two different measurements.
Then size what the duplicate costs. Illustrative example, with your tag count and logging rate setting the real numbers: 500 tags at one row each minute — the 60-second cadence this feed targets — gives 1 row/min × 1,440 min/day = 1,440 rows each tag each day, 1,440 × 30 = 43,200 rows each tag over a 30-day window, and 43,200 × 500 = 21,600,000 rows from that driver across the window. A second driver writing the same tags adds another 21,600,000 rows, all of it redundant, and doubles the insert volume your polling and retention jobs run against. If insert volume is the cost that hurts, the backend is the lever — evaluating an Ignition historian with TimescaleDB for your plant covers that comparison.
Decision: the writer to stop is the parallel gateway, and you stop it at the gateway — disable its history provider or repoint its connection — never by deleting tables. If a driver is supposedly disabled and rows still land, keep the sqlth_drv mapping and the connection evidence intact and take both to the integrator or the vendor's support.
What should the data lake key on instead of tagid? #
Key the lake on a canonical, site-scoped tag path — or a separate stable tag key — that your integration owns, with the site identity carried in the MQTT namespace itself, in its group, edge-node, or device fields, because a path parser cannot recover a site the publisher never stated.
The second site is what makes this urgent: if its paths arrive shaped like paths inside the first site, a consumer files both streams under one logical asset, and no tag ID fixes that — it is a routing defect. Build the contract before site two onboards:
- Put a unique site identity in a deliberate part of the published namespace, or in explicit message metadata, and keep it consistent from Transmission through Engine, rules, and lake records.
- Define what each level of the tag hierarchy means — site, area, line, asset — and keep the values stable, so the hierarchy supports rollups instead of hiding ambiguity.
- Keep ownership and access groupings out of the physical site identity: rollups by owner group and operator access across groups are queries over the site key, not extra namespace levels.
- Validate with two sites that have similar tag names: they must produce distinct canonical records before their production feeds turn on.
- Test the outage path at the same time — force a disconnect and check what the historian records when an MQTT edge node reconnects, because that gap behavior is what the lake inherits.
Decision: fix the namespace before cutting partitions or lake tables for site two. Retrofitting site identity by parsing ambiguous paths afterward costs more than the namespace ever will.
Should I build a tagid-to-tag-path mapping table in the lake? #
No. A mapping table adds a read on the hot ingestion path to resolve a value whose lifecycle belongs to a different subsystem, and it still keys your records on an index the historian can retire at any configuration change.
The added latency is real but secondary; the structural problem is ownership. The historian assigns, retires, and reissues IDs for its own query machinery, and nothing downstream maintains the mapping for you — whether an MQTT broker keeps historic data you can query is no, so there is no side-channel to reconcile against either. If a specific reconciliation genuinely needs the IDs, store them as lineage metadata on records already keyed by tag path, refreshed whenever tag configuration changes.
Decision: no mapping table as a key. tagid becomes, at most, an annotation — never the join.
Is it safe to delete the extra sqlth_data_ tables? #
Not yet. Deleting is the last step, after the extra writer is identified and stopped, because dropped history rows cannot be reconstructed from live tags and a provider still writing into a dropped table leaves the schema inconsistent.
Work in this order: name both drivers through sqlth_drv, stop the wrong one at its gateway, then watch that table set go quiet for a full monthly partition before deciding what history to remove. Check before assuming duplication, too — if the second driver's timestamps do not match the first, that table set may hold the only copy of some spans. One safety note: history deletions are irreversible, and nothing in the live tags can rebuild the rows once they are gone.
Decision: trace, stop, verify silence, then decide with support what to keep. A supposedly disabled driver that still writes goes to the vendor with the driver mapping and connection evidence, not to a DROP statement.
Questions asked
Why do new sqlth_data_ tables keep appearing?
The date suffix is a monthly partition: every driver gets a fresh table each month, so the count of sqlth_data_ tables grows with time and with each additional driver. Steady monthly growth is normal; a second driver number appearing where only one existed before is the signal to trace the new drvid through sqlth_drv.
Will the edge gateway's store-and-forward lose data during an outage?
Store-and-forward buffers published messages while the connection is down and releases them on reconnect, so the MQTT stream itself can survive an interruption. What to verify is the receiving end: check what the historian and the lake record across the gap, including any stale or repeated values the reconnect delivers.
Can the gateway use SQLite as its historian to cut database cost?
No. SQLite is not a supported backend for an Ignition tag history provider. For short local retention use the gateway's bundled internal database, or an external database documented for history providers; when insert volume is the real cost driver, the backend choice is what to evaluate.
Does moving to Ignition 8.3 change the tagid behavior?
Ignition 8.3 reworked the tag history system and added a new built-in local historian, which changes your storage and local-retention options. It does not change the identity rule: historian tag IDs remain internal indexes tied to tag configuration, so a cross-system lake still keys on a site-scoped tag path.