Troubleshooting
Fixing an edge gateway MQTT TLS private key load failure
Short answer
The errors error:0906D06C PEM routines:PEM_read_bio:no start line and error:140B0009 SSL routines:SSL_CTX_use_PrivateKey_file mean the gateway cannot parse its own private-key file, so the failure is local and happens before the broker sees anything. On a Flexy 205 running firmware 15.0s1 the confirmed correction is the 15.0s2 firmware upgrade with the same certificate files, not new keys or AWS IoT policy edits.
On this page
Error:0906D06C (PEM routines:PEM_read_bio:no start line) followed by error:140B0009 (SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib) means the gateway could not read its own private-key file, so the MQTT connection died before any TLS handshake or broker authorization. On an Ewon Flexy 205 running firmware 15.0s1 this is the reported key-file handling fault in that firmware, and the confirmed correction is the 15.0s2 firmware upgrade with the script and certificate files left unchanged. Regenerating keys or editing AWS IoT policies misses, because this failure never reaches AWS.
What the three log lines say #
The log names the failing file first: Error: Unable to load client key file "/opt/ewon/usr/device-private.pem.key". The script asks for that file through the keyfile parameter, set just before the connect call:
MQTT "OPEN", device_id$, mqtt_broker$
MQTT "SETPARAM", "Port", "8883"
MQTT "SETPARAM", "cafile", "/usr/AWS_CA.crt"
MQTT "SETPARAM", "certfile", "/usr/device-certificate.pem.crt"
MQTT "SETPARAM", "keyfile", "/usr/device-private.pem.key"
MQTT "CONNECT"
ErrorReturned% = GETSYS PRG, "LSTERR"
device_id$ and mqtt_broker$ are the script's own string variables for the client ID and the broker endpoint. Decoded in order:
error:0906D06C:PEM routines:PEM_read_bio:no start line— the PEM parser scanned the key file and never found the boundary line that must open it. RFC 7468, which defines the text encoding for these files, requires a PEM object to start with a-----BEGIN-----line; that boundary is the "start line" the parser reports as missing.error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib— the TLS context then had no parsed key to install, so loading the private key into the session failed.BASIC [ERROR] Operation failed (28) 186 : MQTT "CONNECT"— the 28 is whatGETSYS PRG, "LSTERR"returns when theMQTT "CONNECT"operation fails, whatever stage broke.
The sequence is local first: parse the key, load it into the SSL context, run the handshake, then the broker authenticates the certificate and evaluates policy. A failure in the first step ends the connection with the broker never contacted.
Confirm it is the 15.0s1 firmware case #
Four checks settle it:
- The device is a Flexy 205 reporting firmware 15.0s1.
- The log carries the three lines above, with the key named under
/opt/ewon/usr/. - A key file copied from an identical unit that connects without error produces the same failure, so the key's contents are not the variable.
- The same CA file, certificate and key connect to the same AWS endpoint from a workstation, so the certificate set and the policy work outside the gateway.
Size check, worked from one reported case: the generated key file measured 1,706 bytes. Suppose the copy read back from the gateway measures 1,760 bytes — 1,760 − 1,706 = 54 bytes of growth — and its first line no longer reads -----BEGIN RSA PRIVATE KEY-----. A rewritten header reproduces error:0906D06C exactly, and on 15.0s1 the rewrite happens on the device, which is why no re-upload corrects it.
Apply the correction #
- Download firmware 15.0s2 for the Flexy 205 from the HMS Networks support site; firmware comes from the manufacturer or its distributors, nowhere else.
- Upgrade the gateway following the vendor's documented update procedure.
- Upload the three TLS files again — the CA to
/usr/AWS_CA.crt, the certificate to/usr/device-certificate.pem.crt, the key to/usr/device-private.pem.key— with binary-mode transfer. - Re-run the same MQTT sequence from the script above, unchanged.
- Read the log: the key-file line and both OpenSSL errors must be gone, then confirm a publish on a topic the existing policy allows.
Move the key to the gateway over a direct, restricted session rather than a shared staging folder. If a read-back size check still differs from the source after the upgrade, redo the transfer in binary mode before suspecting anything else.
Other causes of the same PEM error #
On gateways that are not on 15.0s1, the same error string has three usual origins:
- Key format: the file must open with
-----BEGIN RSA PRIVATE KEY-----(PKCS#1) or-----BEGIN PRIVATE KEY-----(PKCS#8). OpenSSL's key reader does not parse the OpenSSH format ssh-keygen writes by default. - Transfer mode: text-mode FTP rewrites line endings and can clip the header line the parser scans for. Use binary mode.
- Storage damage: re-download the key from the gateway and compare byte counts with the local copy. Matching sizes and a correct first line rule out the file and point back at firmware or format.
Why the AWS IoT policy is not involved #
The broker evaluates a policy only after a certificate has authenticated on a completed TLS handshake. This failure happens before the handshake starts, so no change to connect, publish or subscribe permissions can alter the outcome.
Rejection that appears only after the handshake — certificate trust, registration, thumbprints — is a different fault, and the steps in fixing X.509 certificate auth to Azure IoT Hub over MQTT cover that stage. A client that never reaches the broker at all, with timeouts and no OpenSSL lines, is a reachability problem; start with fixing a PLC MQTT client that never reaches your broker.
The decision for each case #
- Flexy 205 on firmware 15.0s1 with these three log lines: upgrade to 15.0s2 and keep the script and all three files unchanged.
- Key opens with the OpenSSH header: regenerate the pair as PKCS#1 or PKCS#8 PEM and issue the certificate for the new key.
- Read-back size differs from the source after a binary-mode upload on corrected firmware: redo the transfer; a second mismatch means the storage path is altering files.
- No OpenSSL lines and pure timeouts: reachability, not key loading.
And if this gateway was already scheduled for replacement rather than another firmware cycle, a fanless mini PC can take over the same AWS IoT publish job.
Questions asked
Does BASIC error 28 always mean the WAN interface is not ready?
No. The 28 comes from GETSYS PRG "LSTERR" after the MQTT CONNECT call and only says the operation failed. The sample script printed a WAN-not-ready message for that value, which pointed troubleshooting at the network. The MQTT log entries printed alongside it, with the two OpenSSL errors, are what identify the private-key load as the failing stage.
Why does the error name /opt/ewon/usr/device-private.pem.key when the script sets /usr/device-private.pem.key?
It is the same file. The SETPARAM keyfile entry uses the short /usr path, and the firmware reports the failed load under the device's full path /opt/ewon/usr/. The file name is identical in both messages, so no path correction is needed in the script.
My private key starts with BEGIN OPENSSH PRIVATE KEY. Does that cause this error?
It can. OpenSSL's private-key reader accepts PKCS 1 keys, which begin BEGIN RSA PRIVATE KEY, and PKCS 8 keys, which begin BEGIN PRIVATE KEY. It does not read the OpenSSH format that ssh-keygen writes by default. Regenerate the pair as PEM and issue the device certificate for the new key.
Should I widen the AWS IoT policy while troubleshooting this error?
No. The policy is evaluated only after a certificate authenticates on a completed TLS handshake, and this failure happens before the handshake starts. Fix the key loading first. Revisit permissions only if a later attempt connects and then has a publish or subscribe rejected.
How should the private key be handled on the gateway?
Treat it as a per-device secret. Move it to the gateway over a direct, restricted session instead of shared folders, keep FTP accounts limited to the people who need them, and delete workstation copies once the connection test passes. If a key may have been exposed, deactivate that certificate at the broker and issue a replacement pair.