docker builder, wasm->libpglite(wasi/emsdk)->pglite all in one #45
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…7_5_WASM-pglite-builder
…7_5_WASM-pglite-builder
…7_5_WASM-pglite-builder
…7_5_WASM-pglite-builder
… queries (#47) * Improve MemoryContext management to prevent memory exhaustion between queries * Use MemoryContextReset; drop legacy MemoryContextResetAndDeleteChildren macro * Cleanup: remove extraneous comment; keep MemoryContextReset only
* first pass
* skip postgresql tests as usual
* preloaded files; pglite.data file autotools handling
* new pglite and install-pglite Makefile targets; workflow succesful;
* add pglite.{data,js,html} to gitignore
* remove the need for emsdk-shared script
* add password and PGPASSFILE static files
* use system zic from /usr/sbin/zic for timezone
* use static files from repo when preloading password and PGPASSFILE
* add empty file
* builder dockerfile; external deps source code
* use -sMAIN_MODULE=2 which reduces size code
* building with -sMAIN_MODULE=2 to reduce size
* use file containing needed EXPORTED_FUNCTIONS - tested on data generated from the *.o files of plpgsql + the already predefined ones;
-sDYLINK_DEBUG=2 flag is very useful in debugging emscripten's dynamic linking;
generating the EXPORTED_FUNCTIONS needs to be automated
* automate creation of imports needed by plpgsql;
automate creation of EXPORTED_FUNCTIONS emscripten parameter;
* infrastructure for contrib extensions
* more excluded imports
* link pglite.wasm against more libs so the exports are visible for emscripten
* more EXPORTED_FUNCTIONS excludes
* more EXPORTED_FUNCTIONS includes
* extended docker image to contain the exports of the prebuilt libraries
* change the exports format of shared libs
* extend backend makefile to better handle EXPORTED_FUNCTIONS depending on shared libs' exports
* git: ignore .map files
* added pg_ivm as submodule; added Makefile to build such submodules
* added pgvector as submodule and updated the Makefile to build it
* use env var to point to llvm-nm for exports and imports defs
* exclude libossp-uuid exports and dont link libossp-uuid to the exe
* git ignore excluded.imports file
* new simple bash script to build everything; allow debug builds
* cleanup
* added bashs script to clean everything
* unset DEBUG var if it is false - there is a bug somewhere and until we fix that, this works;
bug fix in passing PGLITE_CFLAGS
* further simplify build-with-docker2.sh script
* build pglite script remove SOURCE_MAP_PREFIXES
* comments
* rename script
* add 2 more exported functions; sort the always included imports
* separate dockerfiles for different archs - this is needed because emsdk does not publish different archs under the same tag; hopefully this will be fixed in the future
* contrib extensions packaging; integration into the build of the said extensions
* better error message when building/disting contribs
* change custom build root for pglite contrib
* add LDFLAGS_SL="-sSIDE_MODULE=1" when building contrib extensions
* renamed pgvector submodule to vector
* dist extra extensions insted of install (packaging pgvector and pg_ivm into tar.gz)
* use pushed docker image
* build only using native build platform, since wasm is the same on all platforms
* remove unused dockerfiles
* cleanup src/backend/Makefile
* remove wasm-builder folder
* added comment to other extensions Makefile
* comments
* remove binaries used for pglite-builder
* remove config.site since the param that it contains is already passed in the command line
* update clean-pglite.sh script
---------
Co-authored-by: tudor <tudor@swisstch.com>
* although <some> syscalls are supported by emscripten, this is not what we (always) want. in this particular case, initdb is querying the server configuration by executing postgres with various parameters. when running on node, the calls end up trying to execute on the OS, which leads to a series of errors and undesirable behavior. instead, these syscalls should remain inside the wasm sandbox somehow. until we decide on a proper way to do this, the initdb querying of postgres server configs will just fail (which is the case anyhow atm).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.