Skip to content

Commit

Permalink
df.get, NL, streaming (#32)
Browse files Browse the repository at this point in the history
* adds processor for `df.get(xxx)` notation
* adds streaming data
* logit models are maskable (only a subset of rows are evaluated) 
* when converting dataframe to dataset, duplicate column names are generally not allowed, but if column data is identical then the duplicate column is dropped (this compatibility comes at a cost of some performance and should be avoided if possible)
* code now emits `CacheMissWarning` when compile is triggered, this is totally normal and expected on the first pass to compile things, but can give useful information when a function is unexpectedly recompiled (e.g. due to an accidental difference in input dtypes)
* intermediate functions are no longer cached, reducing the cache footprint on disk with no impact on performance
* ast processing now completes a first pass on each expression with each subspace before cascading down to fallback subspaces, this permits more complex expressions to complete
* implements nested logit, `mnl_draws` function name changed to `logit_draw`.
* no longer tested for compatibility with Python 3.7 (google colab updated to 3.8)
  • Loading branch information
jpn-- authored Jan 9, 2023
1 parent c15d9c0 commit 8587d40
Show file tree
Hide file tree
Showing 22 changed files with 7,820 additions and 384 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
strategy:
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.7", "3.9", "3.10"]
python-version: ["3.9", "3.10"]
defaults:
run:
shell: bash -l {0}
Expand Down
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
repos:

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0
rev: v4.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
exclude: .*\.ipynb
- id: trailing-whitespace

- repo: https://github.com/kynan/nbstripout
rev: 0.5.0
rev: 0.6.1
hooks:
- id: nbstripout

Expand All @@ -20,11 +20,11 @@ repos:
args: ["--profile", "black", "--filter-files"]

- repo: https://github.com/psf/black
rev: 21.12b0
rev: 22.10.0
hooks:
- id: black

- repo: https://github.com/PyCQA/flake8
rev: 4.0.1
rev: 5.0.4
hooks:
- id: flake8
2 changes: 1 addition & 1 deletion docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Dot

Logit
~~~~~~~~~
.. automethod:: sharrow.Flow.mnl_draws
.. automethod:: sharrow.Flow.logit_draws

Convenience
~~~~~~~~~~~
Expand Down
Loading

0 comments on commit 8587d40

Please sign in to comment.