Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for -a, --force-album argument #491

Merged
merged 4 commits into from
Jul 5, 2023

Conversation

KaibutsuX
Copy link
Contributor

Like --force, but can specify multiple specific album names or wildcard patterns to apply to entire album paths like:
-a Pics/*
-a Wedding
-a 'Landscape Photos'

@KaibutsuX
Copy link
Contributor Author

Test failures and permission denied when I run tox locally:

:~/prj/gh/sigal [ supportAlbumForce ] $ tox
py38-pillow80: skipped because could not find python interpreter with spec(s): py38
py38-pillow80: SKIP ⚠ in 0.01 seconds
py38-pillow90: skipped because could not find python interpreter with spec(s): py38
py38-pillow90: SKIP ⚠ in 0.01 seconds
py38-pillow-latest: skipped because could not find python interpreter with spec(s): py38
py38-pillow-latest: SKIP ⚠ in 0.01 seconds
py39-pillow80: skipped because could not find python interpreter with spec(s): py39
py39-pillow80: SKIP ⚠ in 0.01 seconds
py39-pillow90: skipped because could not find python interpreter with spec(s): py39
py39-pillow90: SKIP ⚠ in 0.01 seconds
py39-pillow-latest: skipped because could not find python interpreter with spec(s): py39
py39-pillow-latest: SKIP ⚠ in 0.01 seconds
py310-pillow80: skipped because could not find python interpreter with spec(s): py310
py310-pillow80: SKIP ⚠ in 0.01 seconds
py310-pillow90: skipped because could not find python interpreter with spec(s): py310
py310-pillow90: SKIP ⚠ in 0.01 seconds
py310-pillow-latest: skipped because could not find python interpreter with spec(s): py310
py310-pillow-latest: SKIP ⚠ in 0 seconds
.pkg: _optional_hooks> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_editable> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py311-pillow80: packaging backend failed (code=1), with PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 90, in run
    outcome = backend_proxy(parsed_message["cmd"], **parsed_message["kwargs"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 32, in __call__
    return getattr(on_object, name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 450, in get_requires_for_build_editable
    return self.get_requires_for_build_wheel(config_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
    self.run_setup()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 147, in setup
    _setup_distribution = dist = klass(attrs)
                                 ^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 496, in __init__
    _Distribution.__init__(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
    self.finalize_options()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 935, in finalize_options
    ep(self)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 127, in infer_version
    _assign_version(dist, config)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 60, in _assign_version
    maybe_version = _get_version(config)
                    ^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 162, in _get_version
    dump_version(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 77, in dump_version
    with open(target, "w") as fp:
         ^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Backend: run command get_requires_for_build_editable with args {'config_settings': None}
Backend: Wrote response {'code': 1, 'exc_type': 'PermissionError', 'exc_msg': "[Errno 13] Permission denied: 'src/sigal/version.py'"} to /tmp/pep517_get_requires_for_build_editable-thzh7dve.json
py311-pillow80: FAIL ✖ in 0.24 seconds
py311-pillow90: install_deps> python -I -m pip install Pillow==9.0.1
Collecting Pillow==9.0.1
  Using cached Pillow-9.0.1.tar.gz (49.5 MB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: Pillow
Failed to build Pillow
  WARNING: Building wheel for Pillow failed: [Errno 13] Permission denied: '/home/kaibutsux/.cache/pip/wheels/15'
ERROR: Could not build wheels for Pillow, which is required to install pyproject.toml-based projects

py311-pillow90: exit 1 (3.02 seconds) /home/kaibutsux/prj/gh/sigal> python -I -m pip install Pillow==9.0.1 pid=479535
py311-pillow90: FAIL ✖ in 3.03 seconds
.pkg: build_sdist> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py311-pillow-latest: packaging backend failed (code=1), with PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 90, in run
    outcome = backend_proxy(parsed_message["cmd"], **parsed_message["kwargs"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 32, in __call__
    return getattr(on_object, name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 420, in build_sdist
    return self._build_with_temp_dir(['sdist', '--formats', 'gztar'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
    self.run_setup()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 147, in setup
    _setup_distribution = dist = klass(attrs)
                                 ^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 496, in __init__
    _Distribution.__init__(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
    self.finalize_options()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 935, in finalize_options
    ep(self)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 127, in infer_version
    _assign_version(dist, config)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 60, in _assign_version
    maybe_version = _get_version(config)
                    ^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 162, in _get_version
    dump_version(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 77, in dump_version
    with open(target, "w") as fp:
         ^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Backend: run command build_sdist with args {'sdist_directory': '/home/kaibutsux/prj/gh/sigal/.tox/.pkg/dist', 'config_settings': None}
Backend: Wrote response {'code': 1, 'exc_type': 'PermissionError', 'exc_msg': "[Errno 13] Permission denied: 'src/sigal/version.py'"} to /tmp/pep517_build_sdist-yrio1qxo.json
py311-pillow-latest: FAIL ✖ in 0.04 seconds
pypy3: skipped because could not find python interpreter with spec(s): pypy3
pypy3: SKIP ⚠ in 0 seconds
.pkg: build_sdist> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
check: packaging backend failed (code=1), with PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 90, in run
    outcome = backend_proxy(parsed_message["cmd"], **parsed_message["kwargs"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pyproject_api/_backend.py", line 32, in __call__
    return getattr(on_object, name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 420, in build_sdist
    return self._build_with_temp_dir(['sdist', '--formats', 'gztar'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
    self.run_setup()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 147, in setup
    _setup_distribution = dist = klass(attrs)
                                 ^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 496, in __init__
    _Distribution.__init__(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
    self.finalize_options()
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools/dist.py", line 935, in finalize_options
    ep(self)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 127, in infer_version
    _assign_version(dist, config)
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/integration.py", line 60, in _assign_version
    maybe_version = _get_version(config)
                    ^^^^^^^^^^^^^^^^^^^^
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 162, in _get_version
    dump_version(
  File "/home/kaibutsux/prj/gh/sigal/.tox/.pkg/lib/python3.11/site-packages/setuptools_scm/__init__.py", line 77, in dump_version
    with open(target, "w") as fp:
         ^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'src/sigal/version.py'
Backend: run command build_sdist with args {'sdist_directory': '/home/kaibutsux/prj/gh/sigal/.tox/.pkg/dist', 'config_settings': None}
Backend: Wrote response {'code': 1, 'exc_type': 'PermissionError', 'exc_msg': "[Errno 13] Permission denied: 'src/sigal/version.py'"} to /tmp/pep517_build_sdist-8a4dwpmi.json
.pkg: _exit> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  py38-pillow80: SKIP (0.01 seconds)
  py38-pillow90: SKIP (0.00 seconds)
  py38-pillow-latest: SKIP (0.01 seconds)
  py39-pillow80: SKIP (0.01 seconds)
  py39-pillow90: SKIP (0.01 seconds)
  py39-pillow-latest: SKIP (0.00 seconds)
  py310-pillow80: SKIP (0.01 seconds)
  py310-pillow90: SKIP (0.01 seconds)
  py310-pillow-latest: SKIP (0.00 seconds)
  py311-pillow80: FAIL code 1 (0.24 seconds)
  py311-pillow90: FAIL code 1 (3.03 seconds)
  py311-pillow-latest: FAIL code 1 (0.04 seconds)
  pypy3: SKIP (0.00 seconds)
  check: FAIL code 1 (0.06 seconds)
  evaluation failed :( (3.64 seconds)

But when I checkout main the tests also fail albeit without the permission errors or verbosity.

Copy link
Owner

@saimn saimn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, seems a useful feature. Some comments below.

docs/getting_started.rst Outdated Show resolved Hide resolved
@@ -823,7 +824,7 @@ def get_albums(self, path):
for subname, album in self.get_albums(subdir):
yield subname, self.albums[subdir]

def build(self, force=False):
def build(self, force=None):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to change the default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I was trying to avoid the implying it could just be a bool

src/sigal/gallery.py Outdated Show resolved Hide resolved
src/sigal/gallery.py Outdated Show resolved Hide resolved
Comment on lines 949 to 950
if fnmatch(a.path, f):
return True
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if fnmatch(a.path, f):
return True
return fnmatch(a.path, f)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that works because a False would be an early return

Comment on lines 951 to 952
elif a.name == f:
return True
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elif a.name == f:
return True
else:
return a.name == f

src/sigal/gallery.py Outdated Show resolved Hide resolved
@saimn
Copy link
Owner

saimn commented Jun 3, 2023

Permission denied: this depends on your system, it seems the process cannot write a file (in /tmp ? ) while building the package.

@saimn
Copy link
Owner

saimn commented Jun 3, 2023

Also please rebase on main, I fixed an issue with tox for the CI jobs.

@KaibutsuX
Copy link
Contributor Author

Added tests and addressed comments.

After fixing all of the permissions, tox finally runs, but it it won't get past the 2nd dot in the cli_tests after running for an hour.

Pytest shows the same behavior and even on main without my changes, it looks like some tests fail, but without it actually completing, I can't tell what the errors are.

@KaibutsuX KaibutsuX requested a review from saimn June 5, 2023 14:45
@saimn
Copy link
Owner

saimn commented Jul 5, 2023

Sorry for the delay @KaibutsuX, I forgot about this...
There is an error in the code:

      def process_dir(self, album, force=False):
          """Process a list of images in a directory."""
          for f in album:
  >           if isfile(f.dst_path) and not should_reprocess_album(a.path, a.name, force):
  E           NameError: name 'a' is not defined

Like --force, but can specify multiple specific album names or wildcard
patterns to apply to entire album paths like:
-a Pics/*
-a *Wedding*
-a 'Landscape Photos'
@codecov
Copy link

codecov bot commented Jul 5, 2023

Codecov Report

Merging #491 (81c0d81) into main (b45d018) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #491      +/-   ##
==========================================
+ Coverage   87.53%   87.60%   +0.07%     
==========================================
  Files          24       24              
  Lines        2053     2065      +12     
==========================================
+ Hits         1797     1809      +12     
  Misses        256      256              
Impacted Files Coverage Δ
src/sigal/__init__.py 85.97% <100.00%> (+0.08%) ⬆️
src/sigal/gallery.py 89.75% <100.00%> (ø)
src/sigal/utils.py 97.59% <100.00%> (+0.36%) ⬆️

@KaibutsuX
Copy link
Contributor Author

FYI, I dont know what is going on with the local testing env. When I run pytest tests/ locally, mine hangs forever in test_serve because the invocation of serve actually starts to server and hangs forever.

@saimn
Copy link
Owner

saimn commented Jul 5, 2023

Maybe an issue with opening an HTTP port ? Does it work with sigal serve ?
Thanks for the quick update !

@saimn saimn merged commit 2dc1e55 into saimn:main Jul 5, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants