Skip to content

Commit

Permalink
Fixed the issue that sw codec fails to validate 10-bit type buffer
Browse files Browse the repository at this point in the history
The function validateBufferSize fails when sw codec allocates buffer
with HAL_PIXEL_FORMAT_YCBCR_P010. The reason is that the pixel format
will be converted to drm format for allocating buffer, then
the format of the handle is assigned to HAL_PIXEL_ format type with
i915_private_invert_format, but the descriptor.format is still
the pixel format, they must be different when checked in
validateBufferSize. So we check the format type after uniformly
converting the pixel/HAL format to drm format.

Tracked-On: OAM-111243
Signed-off-by: zhangyichix <yichix.zhang@intel.com>
  • Loading branch information
zhangyichix committed Jul 19, 2023
1 parent d76a74d commit 3e4cb46
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion cros_gralloc/gralloc4/CrosGralloc4Mapper.cc
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,14 @@ Return<Error> CrosGralloc4Mapper::validateBufferSize(void* rawHandle,
}

PixelFormat crosHandleFormat = static_cast<PixelFormat>(crosHandle->droid_format);
if (descriptor.format != crosHandleFormat &&

uint32_t descriptor_drmFormat = 0, handle_drmFormat = 0;
handle_drmFormat = cros_gralloc_convert_format(static_cast<int32_t>(crosHandleFormat));
if (convertToDrmFormat(descriptor.format, &descriptor_drmFormat)) {
descriptor_drmFormat = cros_gralloc_convert_format(static_cast<int32_t>(descriptor.format));
}

if (descriptor_drmFormat != handle_drmFormat &&
!flex_format_match((uint32_t)descriptor.format, (uint32_t)crosHandleFormat, descriptor.usage)) {
drv_log("Failed to validateBufferSize. Format mismatch.\n");
return Error::BAD_BUFFER;
Expand Down

0 comments on commit 3e4cb46

Please sign in to comment.