We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have my Avro Schema as below:
schema = { "namespace": "example.avro", "type": "record", "name": "IoTData", "fields": [ {"name": "nodeId", "type": ["null", "string"], "default": None}, {"name": "displayName", "type": ["null", "string"], "default": None}, {"name": "dataType", "type": ["null", "string"], "default": None}, {"name": "statusCode", "type": ["null", "string"], "default": None}, {"name": "timestamp", "type": ["null", {"type": "string", "logicalType": "timestamp-micros"}], "default": None}, {"name": "sourceTimestamp", "type": ["null", {"type": "string", "logicalType": "timestamp-micros"}], "default": None}, {"name": "value", "type": ["null", "double"], "default": None} ] }
I had to put default values as None as sometimes these values maybe blank.
File content is as below:
[{'nodeId': 'ns=2;s=SCD30_CO2', 'displayName': 'SCD30_CO2', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 61.83}, {'nodeId': 'ns=2;s=SCD30_TEMPERATURE', 'displayName': 'SCD30_TEMPERATURE', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 27.35}, {'nodeId': 'ns=2;s=SCD30_HUMIDITY', 'displayName': 'SCD30_HUMIDITY', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 41.49}, {'nodeId': 'ns=2;s=SCD30_CO2', 'displayName': 'SCD30_CO2', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:22.704777', 'sourceTimestamp': '2024-02-26T03:56:22.250094', 'value': 63.27}]
When I convert this to Pandas data types are as below:
0 nodeId 60 non-null string 1 displayName 60 non-null string 2 dataType 60 non-null string 3 statusCode 60 non-null string 4 timestamp 60 non-null object 5 sourceTimestamp 60 non-null object 6 value 60 non-null float64
Its unable to infer timestamp type.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
I have my Avro Schema as below:
schema = { "namespace": "example.avro", "type": "record", "name": "IoTData", "fields": [ {"name": "nodeId", "type": ["null", "string"], "default": None}, {"name": "displayName", "type": ["null", "string"], "default": None}, {"name": "dataType", "type": ["null", "string"], "default": None}, {"name": "statusCode", "type": ["null", "string"], "default": None}, {"name": "timestamp", "type": ["null", {"type": "string", "logicalType": "timestamp-micros"}], "default": None}, {"name": "sourceTimestamp", "type": ["null", {"type": "string", "logicalType": "timestamp-micros"}], "default": None}, {"name": "value", "type": ["null", "double"], "default": None} ] }
I had to put default values as None as sometimes these values maybe blank.
File content is as below:
[{'nodeId': 'ns=2;s=SCD30_CO2', 'displayName': 'SCD30_CO2', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 61.83}, {'nodeId': 'ns=2;s=SCD30_TEMPERATURE', 'displayName': 'SCD30_TEMPERATURE', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 27.35}, {'nodeId': 'ns=2;s=SCD30_HUMIDITY', 'displayName': 'SCD30_HUMIDITY', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:21.622480', 'sourceTimestamp': '2024-02-26T03:56:20.224859', 'value': 41.49}, {'nodeId': 'ns=2;s=SCD30_CO2', 'displayName': 'SCD30_CO2', 'dataType': 'Double', 'statusCode': 'Good', 'timestamp': '2024-02-25T22:56:22.704777', 'sourceTimestamp': '2024-02-26T03:56:22.250094', 'value': 63.27}]
When I convert this to Pandas data types are as below:
0 nodeId 60 non-null string
1 displayName 60 non-null string
2 dataType 60 non-null string
3 statusCode 60 non-null string
4 timestamp 60 non-null object
5 sourceTimestamp 60 non-null object
6 value 60 non-null float64
Its unable to infer timestamp type.
The text was updated successfully, but these errors were encountered: