IMD SolutionComputing LibraryShop main store

Selection guide

Sizing an edge PC for oversampled PLC data to InfluxDB

Edge computing and IIoT Published 6 min read

Short answer

Yes — the database side carries it. 50 samples every 2 ms is 25,000 values per second, reaching InfluxDB as 2.5 writes per second at the documented 10,000-line batch, about 0.38 MB each. What decides the project is the controller's 2 ms buffer handshake and line serialization — no vendor rates that path in bytes per second, so each stage is proven on the box with a synthetic signal before a sensor is wired.

On this page
  1. Rate the demand before anything else
  2. The write limits InfluxDB actually documents
  3. Where the real bottleneck sits
  4. Buffer ownership rules that keep 2 ms acquisition intact
  5. Timestamps: derive them, never stamp them at write time
  6. Worked example: sizing the write path for 50 samples every 2 ms
  7. Choosing where the writer runs
  8. The test that decides it
  9. Questions asked

Yes — the database side carries it. Fifty samples every 2 ms is 25,000 values per second, and at InfluxDB's documented optimal batch of 10,000 lines that arrives as 2.5 authenticated writes per second. The decision sits on the controller: a 50-sample array completes every 2 ms, so the box must sustain 500 buffer handshakes and 25,000 lines of serialization per second, and no vendor publishes a bytes-per-second rating for that path on any controller tier. Rate the stages separately, then prove them with a synthetic signal before wiring a sensor.

Rate the demand before anything else #

Convert the requirement into the numbers each stage must carry, then check the stages one at a time. For 50 samples per 2 ms:

  • Sample rate: 50 ÷ 0.002 s = 25,000 values per second.
  • Sample spacing: 0.002 s ÷ 50 = 40 µs, or 40,000 ns — the timestamp resolution the stored records must preserve.
  • Array completions: 25,000 ÷ 50 = 500 completed arrays per second, one every 2 ms.
  • Accumulation: 25,000 × 5 = 125,000 values per 5 s, held in 2,500 arrays.

The decision this forces: the write unit cannot be the array. One array is 50 lines; the correct write unit is the documented batch, and everything between array completion and batch flush must be queued. Running Python and Node-RED beside PLC code on an edge PC covers where that transfer code lives; this article sizes it.

The write limits InfluxDB actually documents #

InfluxData's write-optimization guidance puts the optimal batch at 10,000 lines of line protocol or 10 MB, whichever threshold is met first. A later optimization discussion for InfluxDB 2 reports 5,000 as the working figure; anywhere in that range leaves every decision below unchanged. Line protocol — measurement, field, value and timestamp per line, separated by newlines — is the payload format built for these writes, not a JSON wrapper around the array.

Documented limit Value Scope
Optimal batch size 10,000 lines or 10 MB, whichever first General write guidance
Write request cap 50 MB Cloud service
Uncompressed payload 250 MB; lines beyond it are not written Cloud, TSM engine

For a self-hosted instance, no per-request cap appears in these references — the batch guidance is what applies. The server's own hardware sizing is documented separately by the vendor, and planning the disk and memory behind the whole self-hosted stack is its own exercise, covered in sizing a self-hosted Node-RED InfluxDB Grafana historian.

Where the real bottleneck sits #

Staged in the order they fail:

The buffer handshake. 500 times per second, a completed array must be noticed, read consistently and acknowledged. The reader must never see a half-filled buffer, and acquisition must never wait on the network.

Serialization. 25,000 lines per second of string building is the CPU cost on the controller. Building each line by concatenation allocates memory per sample; build the payload into one preallocated buffer instead — the discipline behind zero-allocation serialization for embedded C and C++ targets, where per-sample allocation is equally unaffordable.

The HTTP write. At correct batching this is 2.5 requests per second on one kept-alive connection — negligible. This is also why the controller tiers cannot be ranked by a bytes-per-second figure for the database: the discriminating stage is serialization, not the write.

Buffer ownership rules that keep 2 ms acquisition intact #

  • Raise a batch-ready indication only after the last valid sample is stored; completeness is asserted, never assumed from timing.
  • Hold the completed buffer stable while it is read; repeated reads of one completed batch return identical data.
  • Acknowledge — release or reset — only after the write response indicates success.
  • Run acquisition and database communication in separate tasks, with the transfer in the lower-priority one. A slow or retrying write shows up as queue occupancy, never as a changed 40 µs spacing.
  • Give the queue a fixed depth and an explicit overflow behavior: pause acquisition, or drop with a counted loss indication — never a silent overwrite.

If the same runtime also executes machine control, commission this on a stopped machine or on the bench first: a backed-up queue must not be able to stall the control scan.

Timestamps: derive them, never stamp them at write time #

Each record's timestamp is the acquisition start of its array plus index × 40 µs, expressed in the precision the request declares; line protocol's native unit is nanoseconds. Verify a stored spacing of 40,000 ns between consecutive samples. Stamp at write time instead and every point in a flush carries the same instant; with one measurement and series they overwrite each other, and 10,000 samples collapse into one point per flush. Downsampling a dashboard is a separate, later decision — reducing what is stored changes the information condition monitoring can later recover.

Worked example: sizing the write path for 50 samples every 2 ms #

Inputs #

  • Acquisition requirement: 50 samples per 2 ms.
  • Optimal batch: 10,000 lines (documented write guidance; the 10 MB alternative threshold needs roughly 1 kB per line to bind first).
  • Example line: vib,ax=1 s=0.4823 1699999999999999999 plus a trailing newline — 38 bytes, an illustrative line, counted.
  • Cloud write request cap: 50 MB (documented).

Rule #

Sample rate divides samples per array by the array period. Flush interval divides the batch size by the sample rate. Bytes per flush multiplies batch size by line size; sustained rate and daily volume multiply that by seconds.

Arithmetic #

  • 50 ÷ 0.002 s = 25,000 values/s; spacing = 0.002 s ÷ 50 = 40 µs.
  • 10,000 lines ÷ 25,000 lines/s = 0.4 s per flush, so 1 ÷ 0.4 s = 2.5 writes/s. Cross-check: 0.4 s ÷ 0.002 s = 200 arrays, and 200 × 50 = 10,000 lines.
  • 10,000 × 38 B = 380,000 B ≈ 0.38 MB per request — 3.8% of the 10 MB batch threshold, under 1% of the 50 MB request cap.
  • 25,000 × 38 B/s = 950,000 B/s ≈ 0.95 MB/s sustained.
  • 0.95 MB/s × 86,400 s ≈ 82 GB per day of raw line protocol.

Decision #

Queue completed arrays and flush at 10,000 lines — every 0.4 s. The database side passes with two orders of magnitude of margin on request size, so per-array posting at 500 writes per second is rejected: it buys nothing the queue does not. Retention is the failing stage at continuous 25 kHz: above roughly 80 GB per day, store full-rate data in capture windows and a downsampled trend continuously.

Choosing where the writer runs #

Writer placement Select when Trade you accept
On the controller The sustained test holds queue depth at zero with control tasks loaded One box to maintain; serialization competes with control CPU
Separate edge box Serialization on the controller cannot hold the rate A second node, its power and network, and one more hop to secure

Keep acquisition on the controller either way; when the writer moves, it pulls completed buffers and queues them locally, so the 2 ms timing never crosses the network. When the controller must stay dedicated to acquisition and control, give the write role to a fanless mini PC on the same subnet. The disk behind either arrangement is its own constraint — choosing edge storage for high-rate vibration data covers that selection.

The test that decides it #

Run a synthetic generator — the array filled with a ramp plus noise — at the full 25,000 values per second for a sustained period, an hour at minimum, before any sensor is connected. Pass criteria, all of them required:

  • Queue depth returns to zero after every flush, with no unbounded retries.
  • Every write response indicates success.
  • Queried timestamps are spaced 40,000 ns.
  • First and last values of a sampled batch match the PLC buffer.

The tier or model that passes with queue depth pinned at zero is the one you deploy; the InfluxDB settings are proven by the same run.

Questions asked

Does InfluxDB publish a maximum writes-per-second limit?

No points-per-second ceiling appears in the write documentation used here. What is documented: an optimal batch of 10,000 lines or 10 MB, whichever comes first; a 50 MB write request cap on the cloud service; and, on the cloud TSM engine, a 250 MB uncompressed payload limit. Lines beyond 250 MB in a single request are simply not written, so never drain a large backlog as one request.

Why not write each 50-sample array as soon as it completes?

That is 500 authenticated requests per second holding 50 lines each — 200 times below the documented optimal batch size and 200 times the request rate the batch path needs. The database would likely absorb it; the controller spends its CPU on connection handling and per-request overhead instead of serialization. Queue the arrays and flush at the 10,000-line batch instead.

How much disk does continuous 25 kHz storage need?

At the 38-byte example line, 0.95 MB per second of line protocol is about 3.4 GB per hour and 82 GB per day before compression. Compression depends on the engine version and the data itself, so treat those as the raw budget. If the disk cannot hold the capture schedule, store full-rate data in windows and a downsampled trend continuously.

Do the faster controller tiers publish a bytes-per-second rating for this?

None was found in the documentation behind this article: no controller model or performance tier carries a published rate for the PLC-to-historian path. The tiers differ in the serialization stage, not in the 2.5 writes per second the database sees. Run the same synthetic test on each candidate and select the one that holds queue depth at zero.

What happens if the writer falls behind for a minute?

Sixty seconds at 25,000 lines per second is 1.5 million lines, about 57 MB at the example line size, and 150 flushes to drain. Give the queue a fixed depth and an explicit loss counter, and pause acquisition or drop oldest-first with a recorded count. Never let the PLC overwrite a buffer the writer still owns, and never send the whole backlog as a single request.

Keep reading

  • Comparison

    Choosing edge storage for high-rate vibration data

    Keep the CSV-in-ZIP vibration archive that already works at 10 kHz, and treat Parquet as a candidate format, not a capacity fix. Move only after the Parquet writer proves exact sample fidelity, sustained throughput at the real acquisition rate, and recoverable completed archives after power loss. A format change is an engineering change with its own acceptance test, and compression in either format never repairs precision lost upstream.

    Edge computing and IIoT

  • Explainer

    MQTT to Kafka edge data pipeline without losing messages

    Keep MQTT on the sensor side of the plant, put the Kafka-class durable log on the site or cloud side of the uplink, and give the boundary one disk-backed store-and-forward buffer sized to the longest outage you will ride through. At 500 messages per second and an 8-hour outage, that queue holds 14.4 million messages and about 9.6 GB — set the ceiling above it and alarm on overflow.

    Edge computing and IIoT

  • How-to

    Route MQTT sensor data into Grafana on an edge PC

    An MQTT broker forwards each message once and keeps no history, so Grafana cannot trend from it directly. Subscribe with Node-RED, write each payload into InfluxDB, set a 365-day retention policy, and point a Grafana data source at that database. For the example below — 720 signals sampling every 5 s — that is about 0.9 TB for a year; provision roughly 2 TB.

    Edge computing and IIoT