IMD SolutionComputing LibraryShop main store

Troubleshooting

Fix duplicate driver IDs after a Docker historian upgrade

Edge computing and IIoT Published 8 min read

Short answer

Duplicate rows in the historian driver table after a container upgrade mean the gateway identity changed: the data volume or hostname was not carried forward, so the historian opened a new driver ID and logged under it. Pin the volume and hostname, restart the history provider to flush its cached tag IDs, and rebuild the schema only after that. ID jumps of about 1,000 are database sequence caching, not a fault.

On this page
  1. What the driver row represents
  2. Why an upgrade opens a second row
  3. Confirm which case you have
  4. Restart the provider before touching any table
  5. Why tag IDs jump by about 1,000
  6. Rebuild the schema only when identity is stable
  7. The decision for each symptom
  8. Questions asked

A historian that stops storing after a Docker image upgrade, showing two gateway rows in sqlth_drv and data-table tag IDs that sqlth_te has never seen, has not lost its tags. It has lost its identity. The historian keys every stored value to a driver row that represents the writing gateway, so an upgrade that changes the gateway name — or arrives without the gateway's data volume — opens a new driver ID and logs under it, while the old entries stay because the writer that would retire them no longer exists. The fix has an order: pin the data volume and hostname first, restart the history provider to flush its cached tag IDs, and rebuild the schema only once identity is stable. ID jumps of about 1,000 are database sequence caching and need no action at all.

What the driver row represents #

A driver row is the identity of the writer, not a device driver: sqlth_drv stores one row for each gateway-and-tag-provider pair that writes tag history. The provider column holds the tag provider name and the name column holds the gateway System Name, and the rest of the schema hangs off that row:

  • sqlth_scinfo — the history execution groups (scan classes), each keyed to a drvid.
  • sqlth_sce — the periods during which each group actually ran; a missing row here is what a trend draws as a gap.
  • sqlth_te — one entry per historized tag, carrying id, tagpath, scid and retired.
  • the monthly partitions, named for driver ID, year and month in the sqlt_data_1_2026_09 style, holding tagid, value, quality and timestamp.

When a tag starts logging, the historian looks for a non-retired sqlth_te row for that tag path and scan class. Finding none, it inserts one, takes the generated id, and caches the path-to-ID mapping in memory. Because scid resolves through sqlth_scinfo to a drvid, a single new driver row cascades: new scan class rows, then a new tag entry for every historized tag, then a new partition series.

Worked example: a gateway historizing 240 tags across 2 scan classes gains 1 driver row plus 2 scan class rows plus 240 tag entries — 1 + 2 + 240 = 243 schema rows created without storing one new value — and from that day a year of data spreads across 24 monthly tables instead of 12.

Why an upgrade opens a second row #

Two deployment habits produce a fresh driver row on each upgrade:

  • The data volume is not carried forward. The gateway keeps its configuration, including the System Name, in its data directory — the official Ignition image stores it at /usr/local/bin/ignition/data. Recreating the container from the new image without the same named volume or bind mount starts a blank gateway, and a blank gateway is a new writer.
  • The identity changes under a kept volume. A container's default hostname is its container ID, which changes at every recreation, and Docker fixes the hostname at creation time with no rename for an existing container. If the System Name was derived from the hostname at first start instead of set explicitly, a gateway backup restored from a differently named gateway lands under a new name as well.

The check for both: read the System Name and the tag provider name in the gateway's web interface after the swap and compare them against the values from before it.

Two containers pointed at the same database on purpose — a blue/green cutover, a test gateway on the production connection — is legitimate: each earns its own driver ID and its own partition series, and the streams do not mix. Whether the historian belongs in a container at all is its own decision, covered in whether plant historian software can run in Docker containers.

Confirm which case you have #

Run these read-only queries against the historian database. They are generic SQL written against the documented tables, not vendor tools.

SELECT id, name, provider FROM sqlth_drv ORDER BY id;

Two rows with the same provider value mean the gateway name changed — the volume or hostname case. Two rows with different provider values mean the tag provider was renamed or recreated. One row per real gateway/provider pair means identity is intact and the fault is elsewhere.

SELECT tagpath, COUNT(*) AS copies
FROM sqlth_te
WHERE retired IS NULL
GROUP BY tagpath
HAVING COUNT(*) > 1;

A healthy result is 0 rows. A path listed here resolves to two live tag entries, so a chart can pick the idle one and draw nothing. Close the check by comparing MAX(id) in sqlth_drv with the value recorded before the upgrade: it must not move.

Restart the provider before touching any table #

Partition rows carry tagid as a plain integer with no foreign key to sqlth_te, so nothing at the database level stops a write whose ID has no tag entry. The path-to-ID mapping lives in the history provider's memory, and the store-and-forward engine can already hold buffered rows whose IDs were resolved minutes earlier. Emptying the tables empties neither one: delete every row from sqlth_te and the next flush still writes the old numbers — exactly the reported pattern of a value landing in a data table with an ID the tag entry table has never seen.

SELECT d.tagid, COUNT(*) AS rows_written
FROM sqlt_data_1_2026_09 d
LEFT JOIN sqlth_te t ON t.id = d.tagid
WHERE t.id IS NULL
GROUP BY d.tagid;

That reads the September 2026 partition of driver ID 1; substitute your own. The restart, not the DELETE, is what fixes it: restarting the history provider, or the whole gateway, rebuilds the cache from the table contents. Expected result once the provider has restarted and history has run one scan period: 0 rows. Restart before any schema surgery, because afterwards you can no longer tell cache effects from rebuild effects.

Why tag IDs jump by about 1,000 #

sqlth_te takes its id from the database's identity or sequence counter; the historian never picks the number. The engine hands those values out of a cache — SQL Server's identity cache reserves about 1,000 values at a time — and discards the unused remainder at every service restart, which under Docker can mean every stack restart.

Worked example: the first three tags took ids 1, 2 and 3. The cache had reserved the block 1–1000; the database restarted and discarded the 997 unused values, 1000 − 3 = 997; the next tag took id 1000. Each further restart walks the counter up a block, which is why later tags can surface just past the next thousand, at 2003 or 3003.

Gaps are cosmetic; duplicate IDs would be a fault, and gaps never collide with anything already written. Renumbering is not an option — every partition row already stores its tagid. If reporting needs dense numbers, change the identity caching at the database level, never inside the historian tables.

Rebuild the schema only when identity is stable #

When the driver table keeps growing after the identity fix, or orphan IDs survive a full gateway restart, rebuild in this order:

  1. Stop the writes. Disable the tag history provider in the gateway configuration or stop the gateway. This is the step that protects the data: renaming or emptying these tables under a writing gateway orphans every row in the next flush.
  2. Drain store-and-forward. On the gateway status pages, read the pending and quarantined counts for the historian's store-and-forward engine and wait for both to reach 0. Quarantined rows carry tag IDs resolved under the old identity; discard them unless you intend to reconcile IDs by hand.
  3. Rename, do not drop, anything with history in it. Move sqlth_drv, sqlth_scinfo, sqlth_sce, sqlth_te, sqlth_partitions, sqlth_annotations and the sqlt_data_* partitions to a suffixed archive set so the old series stays on disk and readable.
  4. Restart the whole gateway, not just the provider, so the tag-ID cache and the forward buffers both start empty.
  5. Re-enable history on a handful of tags and verify: one row in sqlth_drv, one scan class row, one sqlth_te entry per tag, and a first partition row within one scan period.
  6. If orphan IDs still arrive, remove the tag history module, restart the gateway and reinstall it so the schema is created from scratch — the step reported to clear the fault when cache flushing alone was not enough.

Two schema-creation faults have their own fixes: a recreated schema in which the sqlth_drv table never appears is covered by the page on a tag historian sqlth_drv table that is not created, and a gateway that repeats the sqlth_drv table already exists warning every 60 seconds (seen on 8.1.19) is covered by the page on clearing that repeating sqlth_drv warning.

The decision for each symptom #

  • A new sqlth_drv row after each upgrade, same provider name. Identity is not surviving the swap: carry the data volume forward, pin the hostname in the compose file, set the System Name explicitly. Keep both partition series; dropping the old one is the only variant of this fix that destroys data.
  • Rows landing with tag IDs absent from sqlth_te. A stale provider cache: restart the history provider or the gateway, then re-run the orphan query and expect 0 rows. Rebuild only if a full gateway restart still leaves orphans.
  • Tag entries that never retire. They belong to a driver ID the running gateway does not own, so nothing will ever write their retired flag. Join sqlth_te to sqlth_scinfo on scid to see which drvid each entry belongs to, leave the rows in place, and let the rebuild move live logging onto the current driver.
  • IDs that jump by about 1,000. Database identity caching across a restart. No action, and never a renumber.

Questions asked

Can two gateway containers write to the same historian database at once?

Yes, by design. A staged cutover or a test gateway pointed at the production historian database receives its own driver row and its own partition series, so the two streams never mix. The one requirement is that each container keeps its own data directory: two containers sharing one data directory corrupt the internal configuration, which is a different and worse failure than a duplicate driver row.

Should I renumber the tag IDs to remove the gaps?

No. The id value in sqlth_te is written into every partition row as tagid, so renumbering would mean rewriting each monthly table and re-pointing every existing trend. If reporting needs dense numbers, generate them in the reporting layer from row order or timestamps and leave the historian tables exactly as the database created them.

Does this duplicate-driver problem happen without Docker?

Yes. Anything that changes the gateway name opens a new driver row: restoring a gateway backup taken under a different System Name, reinstalling on a machine with a new hostname, or recreating the tag provider under a new name. The table names, the checks and the fix order are identical on a Windows or Linux install with no Docker at all.

How do I read old trends after rebuilding the schema?

The renamed archive tables keep every column, so point reporting queries at the suffixed names and keep the same joins: tag entries to scan class rows on scid, then partitions on tagid matching the entry id. A chart that must span the rebuild unions the old and new partition series ordered by timestamp.

Keep reading

  • Explainer

    Can plant historian software run in Docker containers

    The processes will start, but historian services hold retained state and are not documented to communicate statelessly, so a started container proves nothing about collection, calculations, events, or queries surviving replacement. Keep the working deployment in production and evaluate containers only in an isolated test, with a volume mapped for every path that holds data or configuration, passing a counted query across a restart, a recreate, and a host move.

    Edge computing and IIoT

  • Troubleshooting

    Fix PI calculations on slash-containing DeltaV historian tags

    Quote the entire point name — 'FIC101/PID1/ PV.CV' — in single quotes and put the arithmetic outside it: 'FIC101/PID1/ PV.CV' / 2. The slashes inside the quotes belong to the stored name; the parser divides only at the slash after the closing quote. Renaming points is a later decision, not the fix.

    Edge computing and IIoT

  • Troubleshooting

    Historian reads slow in Excel after adding tags

    Run one timed test before touching anything: read one old tag and one newly added tag over the same range in the historian's built-in client, then in the Excel macro. Fast in the client but slow in Excel means the macro's request pattern is the fault; the new tag slow in both means you run the historian's validate action and then confirm the SQL maintenance plan actually targets the historian database.

    Edge computing and IIoT