You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In testIoOctaveDouble.java I added the following test
@Test
public void testPrecision() {
final OctaveEngine octave = new OctaveEngineFactory().getScriptEngine();
octave.eval("format long");
octave.eval("a = (1/3) * 0.29");
OctaveDouble a = octave.get(OctaveDouble.class, "a");
//This assertion will fail.
//On evaluation, the console will report "a = 0.0966666666666667",
//But retrieving the value back from the wrapper reports the native-hex value cast to a double
assertEquals(0.096666666666667, a.get(1), 0.0);
}
Within the command window of octave, the value of the variable a will be "0.0966666666666667", but if the format is changed to 'native-hex' the value of a is 'be58f28b25bfb83f' which converts to "0.09666666666666665"
It seems that this is a bug in how the wrapper stores the variable value, though it could be that my expectations are incorrect.
I'm using Ubuntu 16.04
octaveInJava version 0.6.8
Octave version 4.3.0
The text was updated successfully, but these errors were encountered:
In testIoOctaveDouble.java I added the following test
Within the command window of octave, the value of the variable a will be "0.0966666666666667", but if the format is changed to 'native-hex' the value of a is 'be58f28b25bfb83f' which converts to "0.09666666666666665"
It seems that this is a bug in how the wrapper stores the variable value, though it could be that my expectations are incorrect.
I'm using Ubuntu 16.04
octaveInJava version 0.6.8
Octave version 4.3.0
The text was updated successfully, but these errors were encountered: