Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support IEEE 754 #31

Open
akotlar opened this issue Jun 23, 2016 · 3 comments
Open

Support IEEE 754 #31

akotlar opened this issue Jun 23, 2016 · 3 comments

Comments

@akotlar
Copy link

akotlar commented Jun 23, 2016

Single precision floats defined in msgpack spec, doesn't seem to be implemented here, except during decoding stage.

This would save a significant amount of data for our team ~40% of ~80G per database.

@syohex
Copy link
Collaborator

syohex commented Jun 24, 2016

Perl support only double type(NV) for floating point number. We can have float value as pointer type but it requires much bytes(float + pointer) than double.

@akotlar
Copy link
Author

akotlar commented Jun 24, 2016

Perl has the ability to pack / unpack 32 bit floats. We don't particularly care about whether perl will store a single precision float as double precision, what we care about is that msgpack doesn't use up 44% more space to store a single precision float. Also, message pack's goal is interoperability, else we could just pack / unpack in perl for every type and save the overhead of encoding the msgpack schema.

You support unpacking single precision floats in you PP package. I propose we add packing support for the same.

Even in the pure perl implementation it would be trivial to check whether a value fits in 32 bits, right (partial implementation):

if( unpack('f', pack('f', $num) ) == $num) { pack('f', $num) } else { pack('d', $num) }

I'd be happy to submit a pull request for the pure perl version.

If the runtime packing cost seems high, why don't we offer it as an option, much like utf8, prefer_integer, or your big int support.

@akotlar
Copy link
Author

akotlar commented Sep 26, 2017

Any chance you could take another look at this? Treating all floats as doubles means that our database is substantially larger than needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants