-
Notifications
You must be signed in to change notification settings - Fork 1.7k
minor: simplify docs build process & pin pip package versions #17816
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
mkdir temp | ||
cp -rf source/* temp/ | ||
# replace relative URLs with absolute URLs | ||
sed -i -e 's/\.\.\/\.\.\/\.\.\//https:\/\/github.com\/apache\/arrow-datafusion\/blob\/main\//g' temp/contributor-guide/index.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only ever did this to one file and it looks like the file doesn't have anymore relative URLs
"sphinx.ext.napoleon", | ||
"myst_parser", | ||
"sphinx_reredirects", | ||
"rustdoc_trim", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here we register rustdoc_trim
so it'll run during Sphinx build, meaning we don't need to manually fix it in source files that are in a temp dir, allowing us to use original build dir for simplicity.
Tested by checking the catalog page, can see it's properly trimmed:

Source code:
datafusion/docs/source/library-user-guide/catalogs.md
Lines 56 to 68 in a451920
`tables` is the key-value pair described above. The underlying state could also be another data structure or other storage mechanism such as a file or transactional database. | |
Then we implement the `SchemaProvider` trait for `MemorySchemaProvider`. | |
```rust | |
# use std::sync::Arc; | |
# use dashmap::DashMap; | |
# use datafusion::catalog::TableProvider; | |
# | |
# #[derive(Debug)] | |
# pub struct MemorySchemaProvider { | |
# tables: DashMap<String, Arc<dyn TableProvider>>, | |
# } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also double checked and it looks good to me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks beautiful to me -- thank you @Jefffrey
"sphinx.ext.napoleon", | ||
"myst_parser", | ||
"sphinx_reredirects", | ||
"rustdoc_trim", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also double checked and it looks good to me
needing to create a virtual environment: | ||
|
||
```sh | ||
uv run --with-requirements requirements.txt bash build.sh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice!
uv
instruction to make it easier to buildtemp
dir as we now run therustdocs_trim
as part of the Sphinx process as an extension itself, and also remove thesed
for relative links as there are no relative links anymore