Skip to content

Commit

Permalink
[abfs] Fix download by reading file correctly instead of extra 1 byte…
Browse files Browse the repository at this point in the history
… always with every chunk (#3472)

- When trying to read the file, we are giving wrong endByte value as per https://learn.microsoft.com/en-us/rest/api/storageservices/specifying-the-range-header-for-blob-service-operations#format-2-bytesstartbyte-endbyte.

- We were reading 1 byte extra for every 1MB chunk size value i.e instead of 0-1023 we were sending 0-1024. This was messing up with the downloaded file and lead to truncation of end bytes to fit the downloaded size. 

Header['range'] = headers['range'] = 'bytes=x-y' in this x is starting point and y is end point and both are included hence we removed the 1 byte from y to make sure we are not duplicating the bytes.

- We checked and this issue is only for ABFS, other FS in Hue should not show this behaviour.
  • Loading branch information
agl29 authored Sep 22, 2023
1 parent 86a7b42 commit b82b388
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion desktop/libs/azure/src/azure/abfs/abfs.py
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ def read(self, path, offset='0', length=0, *args, **kwargs):
path = Init_ABFS.strip_scheme(path)
headers = self._getheaders()
if length != 0 and length != '0':
headers['range'] = 'bytes=%s-%s' % (str(offset), str(int(offset) + int(length)))
headers['range'] = 'bytes=%s-%s' % (str(offset), str(int(offset) + int(length) - 1))

return self._root.get(path, headers=headers)

Expand Down

0 comments on commit b82b388

Please sign in to comment.