Skip to content

Commit

Permalink
Fixed the issue that sw codec fails to validate 10-bit type buffer
Browse files Browse the repository at this point in the history
The function validateBufferSize fails when sw codec allocates buffer
with HAL_PIXEL_FORMAT_YCBCR_P010. The reason is that the pixel format
will be converted to drm format for allocating buffer, then
the format of the handle is assigned to HAL_PIXEL_ format type with
i915_private_invert_format, but the descriptor.format is still
the pixel format, they must be different when checked in
validateBufferSize. So we check the format type after uniformly
converting the pixel/HAL format to drm format.

Tracked-On: OAM-111243
Signed-off-by: zhangyichix <yichix.zhang@intel.com>
  • Loading branch information
zhangyichix committed Sep 25, 2023
1 parent d76a74d commit 7175d2e
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion cros_gralloc/gralloc4/CrosGralloc4Mapper.cc
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,20 @@ Return<Error> CrosGralloc4Mapper::validateBufferSize(void* rawHandle,
}

PixelFormat crosHandleFormat = static_cast<PixelFormat>(crosHandle->droid_format);
if (descriptor.format != crosHandleFormat &&

uint32_t descriptor_drmFormat = 0, handle_drmFormat = 0;

if (convertToDrmFormat(descriptor.format, &descriptor_drmFormat)) {
drv_log("Failed to convertToDrmFormat, descriptor.format = %d", (uint32_t)descriptor.format);
return Error::BAD_VALUE;
}

if (convertToDrmFormat(crosHandleFormat, &handle_drmFormat)) {
drv_log("Failed to convertToDrmFormat, crosHandleFormat = %d", (uint32_t)crosHandleFormat);
return Error::BAD_VALUE;
}

if (descriptor_drmFormat != handle_drmFormat &&
!flex_format_match((uint32_t)descriptor.format, (uint32_t)crosHandleFormat, descriptor.usage)) {
drv_log("Failed to validateBufferSize. Format mismatch.\n");
return Error::BAD_BUFFER;
Expand Down

0 comments on commit 7175d2e

Please sign in to comment.