Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not allowing Infinite values for Double #1769

Open
FreCap opened this issue Oct 28, 2022 · 0 comments
Open

Not allowing Infinite values for Double #1769

FreCap opened this issue Oct 28, 2022 · 0 comments

Comments

@FreCap
Copy link

FreCap commented Oct 28, 2022

Problem you have encountered:

Error sending insert request, table: xxxx
com.google.cloud.bigquery.BigQueryException: java.lang.IllegalArgumentException
at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:100)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:979)
...
Caused by: java.lang.IllegalArgumentException
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
    at com.google.api.client.util.Preconditions.checkArgument(Preconditions.java:35)    
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:134)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:146)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:105)

It seems that the json serializer is very restrictive (not able to change/configure it) in handling Infinity values, which are actually supported by BigQuery and Double standard.

https://github.com/googleapis/google-http-java-client/blob/main/google-http-client/src/main/java/com/google/api/client/json/JsonGenerator.java#L134

What you expected to happen:

I would expect the JsonGenerator, to be able to pass Infinity (as a string?) and not hard fail.

e.g. draft

        double doubleValue = ((Number) value).doubleValue();
        if (Double.isInfinite(doubleValue)){ // or   if (Double.isInfinite(doubleValue && allowDoubleInfinityAsString))
          writeString("infinity");
        } else {
          Preconditions.checkArgument(!Double.isInfinite(doubleValue) && !Double.isNaN(doubleValue));
          writeNumber(doubleValue);
        }

The alternative of cleaning the data with Infinity values before inserting it would imply loss of information due to the transportation mechanism, which shouldn't be a limitation itself

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant