CVE-2023-5678: OpenSSL DH Key Checks Can Trigger Denial of Service

Impact

CVE-2023-5678 affects applications that use OpenSSL to generate or validate X9.42 Diffie-Hellman keys and parameters from untrusted input. The result is denial of service. The process can stall. Worker threads can be consumed. Batch jobs can hang. API endpoints that validate submitted public keys can become unavailable.

The vulnerability is tracked by NVD with CVSS 3.1 score 5.3, Medium. Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L. OpenSSL rated the issue Low in its November 6, 2023 security advisory.

This is not a TLS handshake break. OpenSSL says its SSL/TLS implementation is not affected. That lowers exposure for standard HTTPS servers. It does not remove risk for applications that directly call OpenSSL key validation APIs, certificate tooling, security gateways, PKI automation, custom crypto services, or command-line workflows that process untrusted keys.

Affected Products

OpenSSL identified the following release lines as vulnerable:

• OpenSSL 3.1
• OpenSSL 3.0
• OpenSSL 1.1.1
• OpenSSL 1.0.2

The affected APIs include:

• DH_generate_key()
• DH_check_pub_key()
• DH_check_pub_key_ex()
• EVP_PKEY_public_check()
• EVP_PKEY_generate()

The affected command-line paths include:

• openssl pkey -pubcheck
• openssl genpkey

The OpenSSL 3.0 and 3.1 FIPS providers are not affected, according to OpenSSL. The core non-FIPS code paths still require review where applications accept attacker-controlled X9.42 DH material.

Technical Details

The bug is a resource-consumption flaw in Diffie-Hellman parameter handling. X9.42 DH parameters include values such as p, q, and g. These values define the mathematical group used for key agreement. They must be checked before use. Those checks can be expensive when inputs are very large.

The vulnerable path fails to reject oversized q values early enough in some operations. That matters because public-key validation and key generation can perform large-number arithmetic based on attacker-supplied values. If the value is excessive, the calculation can run for a long time before failing.

The failure mode is availability loss. No confidentiality impact is currently assigned. No integrity impact is currently assigned. The attacker goal is to make the target spend CPU time.

The pattern is familiar. Crypto libraries often handle complex formats that carry both data and algorithm parameters. If a parser accepts attacker-controlled parameters and validation has no strict upper bounds, the attacker can force the library into expensive work. The input does not need to be valid. It only needs to reach the expensive path.

CVE-2023-5678 also follows earlier OpenSSL DH hardening work. OpenSSL previously addressed related expensive DH checks in CVE-2023-3446 and CVE-2023-3817. Those fixes tightened handling around oversized p and q values in DH_check(). CVE-2023-5678 covers additional paths where DH_check_pub_key() and key generation behavior still left applications exposed.

Exploitation Conditions

Exposure depends on application behavior.

A server is more likely to be exposed if it accepts public keys, DH parameters, certificate-like objects, key exchange material, or cryptographic job requests from untrusted users, then passes that material to OpenSSL validation APIs.

A local or batch system is more likely to be exposed if it runs OpenSSL command-line validation against files supplied by users, customers, CI pipelines, ticket attachments, malware samples, or automated intake systems.

Standard TLS use is not the main concern. OpenSSL states that its SSL/TLS implementation is not affected by this issue. Do not treat that statement as a blanket exemption for products that bundle OpenSSL. Products often use libcrypto outside TLS.

Mitigation

Patch first.

Upgrade OpenSSL to a fixed vendor release. OpenSSL 3.2.0 includes the fix in its changelog. OpenSSL 3.0.13 includes the fix for the 3.0 branch in the OpenSSL 3.0 changelog. For older or vendor-maintained branches, apply the vendor backport that contains the relevant OpenSSL commits referenced in the advisory.

Take these actions now:

• Inventory systems and applications that bundle OpenSSL.
• Check both OS packages and statically linked application copies.
• Identify code that calls DH_generate_key(), DH_check_pub_key(), DH_check_pub_key_ex(), EVP_PKEY_public_check(), or EVP_PKEY_generate().
• Block untrusted oversized DH parameters before they reach OpenSSL.
• Add size limits to upload, parsing, and key-validation workflows.
• Avoid running openssl pkey -pubcheck or openssl genpkey on untrusted inputs without resource controls.
• Use CPU timeouts, job isolation, and worker limits for key-processing services.
• Monitor for repeated submissions of unusually large DH parameter files.

For Microsoft environments, check the Microsoft Security Update Guide for any product-specific servicing entry. Do not assume Windows Update covers third-party OpenSSL copies shipped inside applications, agents, appliances, containers, or developer tooling.

Timeline

• August 16, 2023: David Benjamin of Google reported the issue to OpenSSL.
• November 6, 2023: OpenSSL published its security advisory for CVE-2023-5678.
• November 23, 2023: OpenSSL 3.2.0 shipped with the fix.
• January 30, 2024: OpenSSL 3.0.13 shipped with the fix for the 3.0 branch.
• December 2, 2025: CISA-ADP added CVSS 3.1 scoring in NVD.
• May 12, 2026: NVD records show additional downstream vendor references added.

Required Response

Treat this as a denial-of-service exposure in cryptographic intake paths. Patch OpenSSL. Review direct libcrypto use. Limit untrusted key sizes. Put resource controls around validation jobs.

Do not stop at checking HTTPS. This flaw sits below the application layer, in key handling code that many products call outside normal TLS traffic.