Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't query data from a CSV with empty fields #433

Open
Anphisa opened this issue Jan 22, 2023 · 10 comments
Open

Can't query data from a CSV with empty fields #433

Anphisa opened this issue Jan 22, 2023 · 10 comments
Assignees
@mehtasaurav
Labels
A-db Area: DB C-bug Category: This is a bug. E-medium

Comments

@Anphisa
Copy link

Anphisa commented Jan 22, 2023

Operating System (Polypheny-DB)

Windows

Polypheny-DB Version

0d7256f name:schema-integration

Affected Areas

DB

Browser

Chrome

Client Operating System

No response

Bug Description

When I load a CSV into Polypheny which has empty fields (not all values set), I can't query the respective columns.

After executing the steps from reproduction steps, I get a NullPointerException:

13:35:56.886 ERROR [qtp542743255-162]: Caught exception while executing a query from the console
 org.polypheny.db.webui.Crud$QueryExecutionException: java.lang.RuntimeException: java.lang.NullPointerException
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3667) ~[classes/:?]
	at org.polypheny.db.webui.Crud.anySqlQuery(Crud.java:810) ~[classes/:?]
	at org.polypheny.db.webui.crud.LanguageCrud.anyQuery(LanguageCrud.java:119) ~[classes/:?]
	at org.polypheny.db.webui.WebSocket.onMessage(WebSocket.java:119) ~[classes/:?]
	at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection.tryBeforeAndEndpointHandlers(WsConnection.kt:71) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection.onMessage(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:70) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.annotated.OptionalSessionCallableMethod.call(OptionalSessionCallableMethod.java:68) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextMessage(JettyAnnotatedEventDriver.java:301) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextFrame(JettyAnnotatedEventDriver.java:287) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:152) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:148) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:111) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:169) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:90) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:234) ~[classes/:?]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
	... 38 more
Caused by: java.lang.NullPointerException
	at org.polypheny.db.runtime.functions.Functions.cannotConvert(Functions.java:2322) ~[classes/:?]
	at org.polypheny.db.runtime.functions.Functions.toFloat(Functions.java:2530) ~[classes/:?]
	at Baz$1$1.current(Unknown Source) ~[?:?]
	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:683) ~[calcite-linq4j-1.18.0.jar:1.18.0]
	at org.polypheny.db.util.LimitIterator.next(LimitIterator.java:58) ~[classes/:?]
	at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:129) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
	at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:122) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:208) ~[classes/:?]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
	... 38 more

Reproduction Steps

Example: Use baeume as source csv.
baeume.csv

Then query on the console:
SELECT hoehe FROM public.baeume

Expected Behavior

Show the column that was queried

Relevant Log Output

No response
@Anphisa Anphisa added the C-bug Category: This is a bug. label Jan 22, 2023
@hennlo hennlo added A-db Area: DB E-medium labels Feb 17, 2023
@Raghavendrapara
Copy link

Let me check this out.

@Raghavendrapara
Copy link

The methods in the exception are the symptoms , underlying disease seems to be in the csv loader. Null handling cases might have been missed

@Raghavendrapara
Copy link

Polypheny-DB/plugins/csv-adapter/src/main/java/org/polypheny/db/adapter/csv/CsvEnumerator.java

Line 308 in 5e3257e

case FLOAT:

Looks like instead of universally setting all null we need to initialize something else or allow null handling

@Raghavendrapara
Copy link

From DBMS perspective null shall be OK, so ig the way injection of null is happening into the database is troubling

@Raghavendrapara
Copy link

Raghavendrapara commented Feb 24, 2023 •
edited
Loading

Filtering does not work on that too, we currently cannot sort based on such fields

@Raghavendrapara
Copy link

Interestingly this works
SELECT latitude,hoehe FROM public.baeume

@Deeppjp116
Copy link

The error message you provided is a stack trace that indicates an exception occurred in a software application. Here's a breakdown of the information provided:

  • The error occurred at 13:35:56.886.
  • The error is of type "ERROR" and is related to executing a query from the console.
  • The specific exception is a org.polypheny.db.webui.Crud$QueryExecutionException, which is caused by a java.lang.RuntimeException that, in turn, is caused by a java.lang.NullPointerException.
  • The error stack trace shows the sequence of method calls leading to the exception, starting from org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3667).

Based on this information, it seems that the error originates from a web-based user interface (UI) component called "Crud" in the Polypheny-DB system. Polypheny-DB is a database management system, and the error occurred while executing a SQL select query. The cause of the exception appears to be a NullPointerException that occurred within the Functions.toFloat method, which is part of the org.polypheny.db.runtime.functions package.

To further investigate and resolve the issue, it would be necessary to analyze the source code of the Polypheny-DB system, particularly the Crud class and the Functions class. Additionally, examining the specific query being executed and the data involved could provide more insights into the cause of the NullPointerException and potential fixes.

@Deeppjp116
Copy link

try:
# Execute the SQL query from the console
result = executeSqlSelect(query)
# Process the result or perform further actions
processResult(result)
except QueryExecutionException as e:
# Handle the query execution exception
handleQueryExecutionException(e)

def executeSqlSelect(query):
try:
# Perform necessary operations to execute the SQL select query
# ...
# Return the result
return queryResult
except RuntimeException as e:
# Log the error or perform error handling
logError(e)
# Raise the exception to the calling code
raise QueryExecutionException("Error executing SQL select query")

def processResult(result):
# Process the query result
# ...
pass

def handleQueryExecutionException(exception):
# Log the exception details or perform error handling
logError(exception)
# Handle the exception as required
# ...

def logError(error):
# Log the error details to a log file or console
# ...
pass

@YassinHemdan
Copy link

YassinHemdan commented Feb 24, 2024 •
edited
Loading

can you assign this issue for me @hennlo

@mehtasaurav
Copy link

Hi @hennlo ,
I think you all are facing this bug because in java empty String is by default null. And in your code you are finding the length of null which in case will give NullPointerException. I can help you fixing this.

 case FLOAT:
                    if ( string.length() == 0 ) {
                        return null;
                    }
                    return Float.parseFloat( string );

mehtasaurav pushed a commit to mehtasaurav/Polypheny-DB that referenced this issue Mar 11, 2024
Fixed Bug causing polypheny#433
f5487d6
@mehtasaurav mehtasaurav mentioned this issue Mar 11, 2024
Open
mehtasaurav pushed a commit to mehtasaurav/Polypheny-DB that referenced this issue Mar 12, 2024
move the check before the switch statement polypheny#433
359e1da
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mehtasaurav mehtasaurav

Labels
A-db Area: DB C-bug Category: This is a bug. E-medium
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@Anphisa @Raghavendrapara @hennlo @mehtasaurav @YassinHemdan @Deeppjp116