Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading a map. #5

Open
SaurabhGupta89 opened this issue Sep 13, 2017 · 0 comments
Open

Reading a map. #5

SaurabhGupta89 opened this issue Sep 13, 2017 · 0 comments

Comments

@SaurabhGupta89
Copy link

Hi,

I have exported data from DynamoDB which contains an attribute which is a map. Sample row (look at Bind attribute, which is a map):

{"CustomerReturns":{"n":"0"},"CustomerReturnsDamage":{"n":"0"},"ShipmentDamage":{"n":"0"},"WarehouseDamage":{"n":"0"},"CarrierDamage":{"n":"0"},"Bind":{"m":{"CUSTOMER-RETURN":{"n":"0"},"VENDOR-DAMAGE":{"n":"0"},"WAREHOUSE-DAMAGE":{"n":"0"},"DEFECTIVE":{"n":"0"},"CUSTOMER-RETURN-DAMAGE":{"n":"0"},"PRIME":{"n":"0"},"SHIPPER-DAMAGE":{"n":"0"},"DAMAGED_BY_VENDOR":{"n":"0"},"CARRIER-DAMAGE":{"n":"0"}}},"Defective":{"n":"0"},"ProductId":{"s":"EgkE1B77jF"},"Prime":{"n":"0"},"VendorDamage":{"n":"0"},"ClientReferenceId":{"s":"q4dBRrNJuj.5d14747c-56d5-4112-8c53-11fcb84b1ba4"},"WarehouseId":{"s":"TestWarehouse2"},"ContainerId":{"s":"BwxdjU7ziU"},"DamagedByVendor":{"n":"0"},"ContainerHandle":{"s":"TestWarehouse2#BwxdjU7ziU"}}

I am getting following error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException com.lyft.hive.serde.DynamoDbSerDe doesn't allow column [5] named bind with type map<string,bigint>)

Initially wanted to directly read DDB table in hive script but could not do it because DynamoDBStorageHandler does not support map data type. I turned to current SerDe just so that I can dump DDB data on S3 and read a map from the text.

Could you confirm if I am missing something here Or just that current Deserialize function can not parse a map?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant