Troubleshooting
Troubleshooting high CPU usage in a Node-RED agent
Short answer
Above 200% CPU in a Node-RED agent usually means upload-time payload work: serialization, retry amplification or burst queueing, not the network. Read the percentage against the logical core count first, then baseline records generated against records delivered over one window, and fix the step the shortfall points to.
On this page
- Why does the CPU percentage go above 200%?
- How do I measure how many records are actually lost?
- What makes CPU spike exactly when uploads start?
- Which agent or flow is responsible?
- Why do failed sends raise CPU without delivering anything?
- What does the queue do when a burst outruns the upload?
- When is the edge PC itself too small?
- Questions asked
Read the number before you tune anything: a Linux process can honestly report more than 200% CPU because per-core accounting multiplies the ceiling by the logical processor count, so the first question is how many cores the box has. The second question is how many records were generated and how many arrived, counted over the same window. Upload-time spikes in a Node-RED agent come from serialization, retry amplification and burst queueing far more often than from the network, and the checks below separate them in that order.
Why does the CPU percentage go above 200%? #
Linux process monitors such as top or htop charge each logical processor separately, so one multithreaded process can report up to 100% per logical processor — 400% on a four-core host — and 200% means two cores' worth of work, not a broken reading.
Run the command that prints the logical processor count (nproc on most Linux distributions) and divide: 210% against a 400% ceiling is 210 ÷ 400 = 0.525, so about 52.5% of a four-core machine, which leaves acquisition and the operating system room to run. The same 210% on a two-core host is past the whole machine, and every other task starves. Cross-check the whole-machine summary line in the same tool; that line is normalized to a 0–100% scale no matter the core count, and some tools can also divide a per-process figure by the core count, so confirm which convention yours uses before you compare readings with anyone else's.
The Node-RED runtime is one process, so its figure covers every flow, every installed node and the runtime's own helper threads; a high number says the runtime is busy, not which flow is busy. Decision: if the whole machine stays under its core count during sends, CPU starvation is not what drops your records — the loss is inside the agent, and the queue and retry checks below find it.
How do I measure how many records are actually lost? #
Define loss at three checkpoints — records the acquisition flow generates, records the agent accepts for upload, and records visible at the destination — then count all three over the same 10-minute window; the checkpoint where the counts diverge is where the records drop.
A send attempt is not a delivery, so count at the destination itself rather than at the upload node's output: a counting handler on the acquisition message, the agent's own input counter, and the destination's record statistics.
Worked example with illustrative numbers: acquisition emits 2,000 records per minute, so a 10-minute window holds 2,000 × 10 = 20,000 generated records. The agent accepts 19,800 of them and the destination shows 16,000. The 20,000 − 19,800 = 200-record gap (200 ÷ 20,000 = 1%) sits between acquisition and the agent; the 19,800 − 16,000 = 3,800-record gap sits inside the upload path, and 3,800 ÷ 20,000 = 0.19 means 19% of everything generated never arrives. Those two numbers point at different faults: a 1% acquisition-side gap is a hand-off problem, while a 19% upload-side gap is serialization, retries or queue overflow.
If the counts only diverge while CPU is high, the spike and the loss share a cause. If records vanish while CPU sits idle, look for silent discards instead: an overwrite rule, a filter, or a queue emptied when the process restarts.
What makes CPU spike exactly when uploads start? #
Upload adds work acquisition never runs — building the payload, serializing it, encrypting the connection — so a spike that starts with the send is local preparation, and its size scales with records per request rather than with bandwidth.
Measure records per request, bytes per request and requests per interval across one normal upload window. Small, frequent sends pay connection and request-setup overhead on every record; large batches concentrate serialization into one long block that stalls the entire runtime — timers, dashboards and the second agent's callbacks all wait until it finishes. Verbose diagnostic logging left on after commissioning multiplies the cost of every message, and a debug output on every message does the same; raise logging only for a short capture and put it back.
Not every spike belongs to your flow. Reported cases describe a fresh container with no deployed flows spiking on a fixed 30-minute schedule; a spike that recurs on the clock regardless of traffic points to runtime or host housekeeping, so find what runs on that schedule before touching the flows.
Decision: a spike that starts with the send and scales with payload size is preparation work — reduce the batch size or pace the submissions. A spike on a fixed schedule with little traffic is housekeeping — hunt for it there.
Which agent or flow is responsible? #
Run one path at a time: let the time-series agent work alone through one full upload window, then the event agent alone, and compare each run's CPU trace and generated-versus-delivered counts — the path that degrades on its own owns the fault.
Fix the window length and the counters first so the solo runs and the combined run are comparable, then isolate:
- Run the time-series path alone with representative input through one full upload window, logging CPU, queue depth and generated-versus-delivered counts.
- Repeat with the event path alone, same window length, same counters.
- Enable both with their normal input patterns and compare the combined numbers against each solo run.
- Check whether both agents live in one Node-RED runtime: one runtime means one event loop, so a heavy block in one path delays the other's timers and callbacks, while separate runtimes isolate the loop but still share cores, memory, disk and network.
If these records feed an environmental, quality or safety obligation, do not stop acquisition to run the isolation tests; put a durable capture in front first and treat any resulting gap as a record-keeping event, not a tuning side effect.
Decision: spikes under the event agent alone point at burst size and submission pacing; a spike that begins before the send points at local preparation. Two clean solo runs with a failing combined run mean the two paths together exceed what the host shares out, and the busier one needs pacing.
Why do failed sends raise CPU without delivering anything? #
A retry re-runs payload building, serialization and encryption for work already done, so every failed batch roughly doubles that batch's processor cost while delivering nothing — and a permanent error retried on a short fixed interval amplifies load without end.
Capture the outcome of every failed attempt with a timestamp — response status and error class only, never credentials or payload contents — and line it up beside the CPU trace. Four classes cover almost everything, and each names its own fix:
- Connection failure: the endpoint was unreachable; retry with backoff and let a durable queue hold the records meanwhile.
- Authentication failure: credentials or registration are wrong; stop retrying, because no attempt can succeed until the configuration is corrected.
- Payload rejected: format, size or a documented limit is exceeded; correct the request against the endpoint's documented limits.
- Rate limiting: the endpoint refuses the current pace; slow the submission rate instead of burning CPU on requests it will refuse.
Use retry logic that backs off, caps its attempts and keeps permanent errors visible instead of burying them in repeats. Where you would rather not build that yourself, the Node-RED flow library documents retry-and-buffer nodes such as @colinl/node-red-guaranteed-delivery, which buffers a failed send in a queue and retries it later, preserving order — that turns unbounded amplification into bounded, paced work.
Decision: classify before you tune. No batch-size change helps an authentication failure, and no bigger queue helps a rejected payload.
What does the queue do when a burst outruns the upload? #
A queue absorbs a burst only until its capacity or overwrite rule bites; in the worked example below, 10,000 records arriving inside 30 seconds against a drain of 2,000 records per minute and a 5,000-record memory cap loses 4,000 of them — a bigger queue postpones the boundary, throughput decides it.
Worked example with illustrative numbers: the upload path sustains 2,000 records per minute, and a burst pushes 10,000 event records into the queue within 30 seconds, an arrival rate of 20,000 records per minute. In that half minute the agent delivers 2,000 × 0.5 = 1,000 records, so the backlog reaches 10,000 − 1,000 = 9,000. With a 5,000-record in-memory cap, 9,000 − 5,000 = 4,000 records drop, and the CPU reads high the whole time because the runtime serializes the backlog flat out. The surviving 5,000 records then drain for 5,000 ÷ 2,000 = 2.5 minutes, or 150 seconds.
Healthy behavior is queue depth that returns toward its baseline after each burst; depth that ratchets upward from window to window means average arrivals exceed delivery capacity, and no queue size fixes that. Read the installed upload node's documented batching behavior and payload limits before changing request size, change one setting at a time, and keep event latency inside whatever the application requires. Check what a full queue does and whether the backlog survives a restart, because an in-memory queue loses everything in it when the process exits. For a structure that holds records until delivery is confirmed, the design in a message-loss-free MQTT to Kafka edge data pipeline moves that boundary from process memory to acknowledged storage.
Decision: size the queue to the worst burst minus what the drain delivers while it runs, give the overflow a durable home, or slow the burst at its source. A larger memory queue alone only postpones the same loss.
When is the edge PC itself too small? #
Add up every process percentage during a send: when the sum reaches the machine's ceiling of 100% per logical core — 400% on four cores — the host is saturated, acquisition starves, and a lean flow holding that level on current runtimes needs bigger hardware, not more tuning.
Before any comparison test, record your Node.js and Node-RED versions and bring the runtime current — the install steps in installing Node-RED natively on a Linux edge computer give the clean native path — then run the identical flows and input on a second PC. A spike that shrinks in proportion to the larger host was a sizing problem; a lean flow that still saturates a much larger host is a flow problem, and no hardware fixes it.
When it is sizing, work from the measured worst window — records per second, bytes per request, simultaneous requests — the way sizing a self-hosted Node-RED InfluxDB Grafana historian works from measured rates instead of guesses. When the measured load genuinely exceeds what the host can sustain and the flow is already lean, replace the unit with a fanless industrial mini PC specified for the worst recorded upload window rather than the average day.
Decision: run the two-host test first. If the same flows run comfortably elsewhere on current runtimes, fix the flow; if they saturate every host at the same load point, move the load to a box sized from the burst.
Questions asked
Will adding RAM stop the CPU spikes?
No. Memory changes how much backlog the queue can hold, not how much processor work each serialization and retry costs. Extra RAM postpones the queue-overflow boundary and can lengthen garbage-collection pauses, but a spike caused by payload preparation or retry amplification stays until those are fixed. Size memory for the worst burst; tune processor cost separately.
How much CPU headroom should an edge PC keep?
Plan for the worst recorded upload window to keep the sum of all process percentages at or below half the logical core count — 200% of 400% on a four-core host — so acquisition, operating-system tasks and logging keep the other half. A machine that runs near its core count on every send has no room left for a burst.
Why does one core sit at 100% while the others idle?
The Node-RED runtime executes flow logic on a single JavaScript event loop, so serialization-bound work pins one core while the rest wait. Percentages above 100% come from the runtime's helper threads and other processes. For this kind of bottleneck, per-core clock speed buys more delivery capacity than additional cores; more cores only help once several processes run flat out.
Can a spike this high damage the edge PC?
Processor spikes do not damage the hardware, but sustained full load raises the temperature inside a passively cooled box, and many industrial computers throttle clock speed when they overheat. Throttling slows delivery further and can look like a new problem. Give the heatsink fins clearance, check the enclosure temperature during a worst-case upload, and treat repeated throttling as a mounting or cooling defect.