Netty: Memory Leak in DNS Record Decoder via Malformed Domain Names

NVD Description

### Summary A memory leak can be caused in Netty's DNS codec by sending malicious DNS packets containing invalid domain names. Because the leak occurs incrementally per packet, sustained malicious requests will cause a gradual Denial of Service. ### Details Inside `io.netty.handler.codec.dns.AbstractDnsRecord`, the parsed domain name string is passed to `IDN.toASCII(name)`. If the domain name contains characters that violate IDNA rules, `IDN.toASCII` throws an `IllegalArgumentException`. Because this exception occurs inside the constructor before the `DnsRecord` instance can assign the buffer to its content field for later release, the ByteBuf whose reference count was incremented (or newly allocated) is never released, resulting in a direct memory leak. There are several places where variants of this leak happen: - `io.netty.handler.codec.dns.DefaultDnsRecordDecoder#decodeRecord(java.lang.String, io.netty.handler.codec.dns.DnsRecordType, int, long, io.netty.buffer.ByteBuf, int, int)` invokes `in.retainedDuplicate()` or creates a new buffer `out` when constructing `DefaultDnsRawRecord` - `io.netty.handler.codec.dns.DnsCodecUtil#decompressDomainName` allocates a new `ByteBuf` and passes it to `encodeDomainName()`. If the decompressed domain name contains a null byte (`\0`), `encodeDomainName()` throws an `IllegalArgumentException`, leaking the newly allocated buffer. ### Impact Resource Exhaustion. Any application utilizing Netty's DnsRecordDecoder (such as DnsNameResolver or custom DNS servers) is vulnerable.

Affected Packages (1)