Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to parse Float #85

Open
GustavoAlves1993 opened this issue Apr 14, 2021 · 21 comments
Open

Failed to parse Float #85

GustavoAlves1993 opened this issue Apr 14, 2021 · 21 comments
Assignees

Comments

@GustavoAlves1993
Copy link

I'm trying to consume data from a table but I'm getting the following error:

com.linuxense.javadbf.DBFException: Failed to parse Float: null
	at com.linuxense.javadbf.DBFUtils.readNumericStoredAsText(DBFUtils.java:82)
	at com.linuxense.javadbf.DBFReader.getFieldValue(DBFReader.java:459)
	at com.linuxense.javadbf.DBFReader.nextRecord(DBFReader.java:343)
	at com.linuxense.javadbf.DBFReader.nextRow(DBFReader.java:395)

is giving error when trying to convert a double value, does anyone know how to proceed with this error?

@albfernandez albfernandez self-assigned this Apr 14, 2021
@GustavoAlves1993
Copy link
Author

does anyone have any idea how i can resolve it?

@albfernandez
Copy link
Owner

Hi, maybe more info would be useful to help you.
The version of javadbf you're using, the full stacktrace, and (if possible) the file causing the error.

@GustavoAlves1993
Copy link
Author

hi, first thanks for the feedback

i'm using version 1.11.1 of the library

the full stacktracer

com.linuxense.javadbf.DBFException: Failed to parse Float: null
	at com.linuxense.javadbf.DBFUtils.readNumericStoredAsText(DBFUtils.java:82)
	at com.linuxense.javadbf.DBFReader.getFieldValue(DBFReader.java:459)
	at com.linuxense.javadbf.DBFReader.nextRecord(DBFReader.java:343)
	at com.linuxense.javadbf.DBFReader.nextRow(DBFReader.java:395)
	at conversorconnexone.dao.DbfDAO.coletaEstoquePrincipal(DbfDAO.java:212)
	at conversorconnexone.bo.DdfBO.coletaEstoquePrincipal(DdfBO.java:35)
	at conversorconnexone.ui.Principal.coletarDBF(Principal.java:489)
	at conversorconnexone.ui.Principal.conectarDBF(Principal.java:440)
	at conversorconnexone.ui.Principal.jbConverterMouseClicked(Principal.java:1263)
	at conversorconnexone.ui.Principal.access$000(Principal.java:31)
	at conversorconnexone.ui.Principal$1.mouseClicked(Principal.java:1228)
	at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:270)
	at java.awt.Component.processMouseEvent(Component.java:6536)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6298)
	at java.awt.Container.processEvent(Container.java:2236)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2294)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4534)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
	at java.awt.Container.dispatchEventImpl(Container.java:2280)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.awt.EventQueue$4.run(EventQueue.java:729)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:109)
	at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:184)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.lang.NumberFormatException
	at java.math.BigDecimal.<init>(BigDecimal.java:494)
	at java.math.BigDecimal.<init>(BigDecimal.java:383)
	at java.math.BigDecimal.<init>(BigDecimal.java:806)
	at com.linuxense.javadbf.DBFUtils.readNumericStoredAsText(DBFUtils.java:77)
	... 54 more

my class:

public ArrayList coletaEstoquePrincipal(DBFReader connDBF) throws Exception {
        ArrayList estoquePrincipal = new ArrayList();
                
        DBFRow row;        
        
        while ((row = connDBF.nextRow()) != null) {            
                estoquePrincipal.add(new Object[]{
                    row.getString("SEQUENCIAL"), //0
                    row.getString("CODIGO"), //1
                    row.getString("GRUPO"), //2
                    row.getString("SUBGRUPO"), //3
                    row.getString("DESCRICAO"), //4
                    row.getString("UNIDADE"), //5
                    row.getString("COR"), //6
                    row.getString("CUSTO"), //7
                    row.getString("MARGEM"), //8
                    row.getString("VENDA"), //9
                    row.getString("MINIMO"), //10
                    row.getString("ATUAL"), //11
                    row.getString("ICMS"), //12
                    row.getString("BARRA"), //13
                    row.getString("FORNECEDOR"), //14
                    row.getString("MARCA"), //15
                    row.getString("ULTCOMPRA"), //16
                    row.getString("NCM"), //17
                    row.getString("CSOSN"), //18
                    row.getString("CODFAB"), //19
                    row.getString("GTIN"), //20
                    row.getString("INATIVO") //21
                });           
        }

        return estoquePrincipal;
    }

the error happens on the line while ((row = connDBF.nextRow()) != null) and drops the application

can I send the file in your email?

@albfernandez
Copy link
Owner

albfernandez commented Apr 15, 2021

you can post the file here if it doesn't contain sensible data or email me the file, as you like.

@GustavoAlves1993
Copy link
Author

I sent in your email

@albfernandez
Copy link
Owner

Hi
I've tested and it works for me.

I've updated some changes to be more verbose on exceptions, I've attached here
javadbf-1.12.0-SNAPSHOT.jar.zip

Can you try with this one to get a more detailed messages?

I've tried with https://github.com/albfernandez/javadbf/blob/main/src/test/java/com/linuxense/javadbf/testutils/DbfToTxtTest.java
It's basic the same you're using. I use this method to read a dbf and print the results.

public static void writeToConsole(File file) throws FileNotFoundException {
		DBFReader reader = null;
		try {

			
			// create a DBFReader object
			reader = new DBFReader(new FileInputStream(file));

			// get the field count if you want for some reasons like the following

			int numberOfFields = reader.getFieldCount();

			// use this count to fetch all field information
			// if required

			for (int i = 0; i < numberOfFields; i++) {

				DBFField field = reader.getField(i);

				// do something with it if you want
				// refer the JavaDoc API reference for more details
				//
				System.out.println(field.getType() + " (" + field.getLength() + ") "+ field.getName());
			}

			// Now, lets us start reading the rows

			Object[] rowObjects;

			System.out.println("-------------------");
			while ((rowObjects = reader.nextRecord()) != null) {
				for (int i = 0; i < rowObjects.length; i++) {
					System.out.println(rowObjects[i]);
				}
				System.out.println("-------------------");
			}

			// By now, we have iterated through all of the rows

		
		}
		finally {
			DBFUtils.close(reader);
		}
	}

@GustavoAlves1993
Copy link
Author

Alberto I replaced the library as you requested, and now the error presented is this:

com.linuxense.javadbf.DBFException: Error reading field SALDOFIN of record 2 Failed to parse Float value [0.00N] : null
	at com.linuxense.javadbf.DBFReader.nextRecord(DBFReader.java:383)
	at com.linuxense.javadbf.DBFReader.nextRow(DBFReader.java:400)
	at conversorconnexone.dao.DbfDAO.coletaEstoquePrincipal(DbfDAO.java:212)
	at conversorconnexone.bo.DdfBO.coletaEstoquePrincipal(DdfBO.java:35)
	at conversorconnexone.ui.Principal.coletarDBF(Principal.java:489)
	at conversorconnexone.ui.Principal.conectarDBF(Principal.java:440)
	at conversorconnexone.ui.Principal.jbConverterMouseClicked(Principal.java:1263)
	at conversorconnexone.ui.Principal.access$000(Principal.java:31)
	at conversorconnexone.ui.Principal$1.mouseClicked(Principal.java:1228)
	at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:270)
	at java.awt.Component.processMouseEvent(Component.java:6536)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6298)
	at java.awt.Container.processEvent(Container.java:2236)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2294)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4534)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
	at java.awt.Container.dispatchEventImpl(Container.java:2280)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.awt.EventQueue$4.run(EventQueue.java:729)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:109)
	at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:184)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: com.linuxense.javadbf.DBFException: Failed to parse Float value [0.00N] : null
	at com.linuxense.javadbf.DBFUtils.readNumericStoredAsText(DBFUtils.java:82)
	at com.linuxense.javadbf.DBFReader.getFieldValue(DBFReader.java:464)
	at com.linuxense.javadbf.DBFReader.nextRecord(DBFReader.java:342)
	... 52 more
Caused by: java.lang.NumberFormatException
	at java.math.BigDecimal.<init>(BigDecimal.java:494)
	at java.math.BigDecimal.<init>(BigDecimal.java:383)
	at java.math.BigDecimal.<init>(BigDecimal.java:806)
	at com.linuxense.javadbf.DBFUtils.readNumericStoredAsText(DBFUtils.java:80)
	... 54 more

when I connect to the DBF table, DBFReader has 106765 records, when I try to populate the arraylist it will be with only 1098 rows, in other words, the error must be giving in the 1099 line of DBFReader, it gives the error and stops populating the array

DBF records
img1

ArrayList size
img2

@albfernandez
Copy link
Owner

I need some time to investigate. I was wrong, it doesn't work for me, it stops at 1098 record, but without exception. there's some "buggy" in the file, as I can see, but others programs can open it, so I'll support it
Your problem seems related to #84, but a this moment I can't reproduce it

javadbf-1.12.0-SNAPSHOT.jar.zip

@GustavoAlves1993
Copy link
Author

I do not believe it is related to bug 84, because I have been using your library for a while and I have used it in tables with more records than this, for example one with more than 400 thousand records

I believe that it is incorrect filling in the double fields, or an excess of numbers that breaks the double, or alphanumeric characters

using while on mine it also doesn’t point out the error, I changed it and made a for loop and then it fires the exception, follow the example code

DBFRow row;
        
        for (int i=0; i<connDBF.getRecordCount(); i++) {
            row = connDBF.nextRow();
            System.out.println(i);
        }

@albfernandez
Copy link
Owner

I'm still not able to reproduce on my mahine, maybe some setup is diferent.
How do you create dbfReader? it's a normal FileInputStream?
What OS are you runing? JVM version ?

I'm testing in Linux with openjdk 8 and openjdk 11

@GustavoAlves1993
Copy link
Author

Hi, yes it's a normal FileInputStream

I'm using Windows 10 64 bits and JVM 1.8.0_281 and JDK 8 update 251 (64-bit)

@ApurbaPandey
Copy link

I am getting the same exception while reading the stream from S3 (S3ObjectInputStream) or HDFS (FSDataInputStream), but working fine with local files (FileInputStream).

@albfernandez
Copy link
Owner

have you tried to wrap your inputStream in a BufferedInputStream?

@GustavoAlves1993
Copy link
Author

GustavoAlves1993 commented May 12, 2021

yes, the problem occurs when I go through the DBFReader and consume the fields inside it, there is a field that is causing the library to drop and I still haven't found a solution for that

@thigg
Copy link

thigg commented May 21, 2021

Its funny, I have a similar error right now, but for me it's not [0.00N] but [0.00T].

Looking at the hexdump at that location, its actually in the file and not only T, but also F.
Funny is, that I wrote the rows with this library, 1.12.0. But this also happened with the old lib before the fork.

What I did was taking an already existing legacy file, opening it and writing 2 rows into it, which then have this error.
The fields are both Doubles (boxed - of course) with 0.0

I attached the hexdump with a few lines for context:

0002a030  20 20 20 20 20 32 30 31  35 30 34 32 36 30 39 3a  |     2015042609:|
0002a040  34 35 3a 34 33 20 20 31  20 20 20 20 30 2e 30 30  |45:43  1    0.00|
0002a050  20 20 20 31 36 39 37 30  30 30 30 30 32 35 20 20  |   16970000025  |
0002a060  20 20 33 37 2e 33 38 20  20 20 20 33 37 2e 33 38  |  37.38    37.38|
0002a070  20 20 20 20 20 30 2e 30  30 20 20 20 20 20 30 2e  |     0.00     0.|
0002a080  30 30 20 20 20 20 20 30  2e 30 30 20 20 20 20 20  |00     0.00     |
0002a090  30 2e 30 30 20 20 20 20  20 20 20 20 20 32 30 31  |0.00         201|
0002a0a0  35 30 34 32 36 31 33 3a  35 39 3a 32 39 20 20 34  |5042613:59:29  4|
0002a0b0  20 20 20 20 30 2e 30 30  20 31 36 39 38 20 30 30  |    0.00 1698 00|
0002a0c0  30 30 30 30 31 20 20 20  20 31 30 2e 30 30 20 20  |00001    10.00  |
0002a0d0  20 20 20 30 2e 30 30 20  20 20 20 31 30 2e 30 30  |   0.00    10.00|
0002a0e0  20 20 20 20 20 30 2e 30  30 20 20 20 20 20 30 2e  |     0.00     0.|
0002a0f0  30 30 20 20 20 20 20 30  2e 30 30 20 20 20 20 20  |00     0.00     |
0002a100  30 2e 30 31 32 30 32 31  30 35 32 31 31 35 3a 31  |0.012021052115:1|
0002a110  34 3a 30 31 20 20 31 20  20 20 20 30 2e 30 30 54  |4:01  1    0.00T|
0002a120  20 31 36 39 39 20 30 30  30 30 30 30 32 20 20 20  | 1699 0000002   |
0002a130  20 31 30 2e 30 30 20 20  20 20 31 30 2e 30 30 20  | 10.00    10.00 |
0002a140  20 20 20 20 30 2e 30 30  20 20 20 20 20 30 2e 30  |    0.00     0.0|
0002a150  30 20 20 20 20 20 30 2e  30 30 20 20 20 20 20 30  |0     0.00     0|
0002a160  2e 30 30 20 20 20 20 20  30 2e 30 31 32 30 32 31  |.00     0.012021|
0002a170  30 35 32 31 31 35 3a 31  34 3a 30 31 20 20 31 20  |052115:14:01  1 |
0002a180  20 20 20 30 2e 30 30 46  1a                       |   0.00F.|

@thigg
Copy link

thigg commented May 21, 2021

Noticed, that I see a T and a F here, and the next field is defined as B_CHECKED,L, apperantly these fields get mixed up somehow

@thigg
Copy link

thigg commented May 21, 2021

Looking closley at it, it looks like there is a space between the rows in the existing format, which is missing here

@albfernandez
Copy link
Owner

I've released 1.13 with the proposed changed, check if it works for you

@thigg
Copy link

thigg commented Jun 7, 2021

No, unfortunatley exatctly the same error

@pablosv94
Copy link

Hello,

I had the same error and to solve it what I did was comment the DBFException that is thrown in readNumericStoredAsText to return null.
First I tried to use the skip method but not wanting to see the deleted rows, it was re-entered by nextRecord and the error occurred again.

How do you see returning null instead of the exception? null is a value that is returned when something goes wrong reading fields and it shouldn't cause any problems.

Greetings and thanks for the effort!

@pablosv94
Copy link

The purpose of return null in readNumericStoreAsText when try to create new BigDecimal(aux) is for continue reading the file, if exceptions throws the execution is stopped.

@albfernandez albfernandez added this to the 2.0 milestone Jan 4, 2024
@albfernandez albfernandez removed this from the 2.0 milestone Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants