CVE-2022-49292

In the Linux kernel, the following vulnerability has been resolved:

ALSA: oss: Fix PCM OSS buffer allocation overflow

We've got syzbot reports hitting INTMAX overflow at vmalloc() allocation that is called from sndpcmplugalloc(). Although we apply the restrictions to input parameters, it's based only on the hwparams of the underlying PCM device. Since the PCM OSS layer allocates a temporary buffer for the data conversion, the size may become unexpectedly large when more channels or higher rates is given; in the reported case, it went over INTMAX, hence it hits WARN_ON().

This patch is an attempt to avoid such an overflow and an allocation for too large buffers. First off, it adds the limit of 1MB as the upper bound for period bytes. This must be large enough for all use cases, and we really don't want to handle a larger temporary buffer than this size. The size check is performed at two places, where the original period bytes is calculated and where the plugin buffer size is calculated.

In addition, the driver uses arraysize() and array3size() for multiplications to catch overflows for the converted period size and buffer bytes.