Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RESTful API endpoints for Groups customizations #570

Merged
merged 7 commits into from
Sep 28, 2022

Conversation

victorlin
Copy link
Member

@victorlin victorlin commented Jul 11, 2022

Description of proposed changes

Adds RESTful API endpoints for downloading/uploading/removing group overviews and logos.

Related issue(s)

Save for later

Testing

Automated:

Manual:

Local testing from a member of test-private/owners. Maybe automate?

Token setup

nextstrain login
id_token_line=$(grep id_token ~/.nextstrain/secrets)
token=${id_token_line#"id_token = "}

GET private overview works

curl http://localhost:5000/groups/test-private/settings/overview \
    --header "Authorization: Bearer $token" \
    --header 'Accept: text/markdown' \
    --compressed \
        | tee overview.md

DELETE private overview works

curl http://localhost:5000/groups/test-private/settings/overview \
    --header "Authorization: Bearer $token" \
    --request "DELETE"

PUT private overview works

curl http://localhost:5000/groups/test-private/settings/overview \
    --header "Authorization: Bearer $token" \
    --header 'Content-Type: text/markdown' \
    --upload-file overview.md

GET private logo works

curl http://localhost:5000/groups/test-private/settings/logo \
    --header "Authorization: Bearer $token" \
    --header 'Accept: image/png' \
    --compressed > logo.png

DELETE private logo works

curl http://localhost:5000/groups/test-private/settings/logo \
    --header "Authorization: Bearer $token" \
    --request "DELETE"

PUT private logo works

curl http://localhost:5000/groups/test-private/settings/logo \
    --header "Authorization: Bearer $token" \
    --header 'Content-Type: image/png' \
    --upload-file logo.png

Test these since the underlying code was refactored

GET private narrative still works

curl http://localhost:5000/groups/test-private/narratives/foo \
    --header "Authorization: Bearer $token" \
    --header 'Accept: text/markdown' \
    --compressed \
        > narrative.md

DELETE private narrative still works

curl http://localhost:5000/groups/test-private/narratives/victorlin-test \
    --header "Authorization: Bearer $token" \
    --request "DELETE"

PUT private narrative still works

curl http://localhost:5000/groups/test-private/narratives/victorlin-test \
    --header "Authorization: Bearer $token" \
    --header 'Content-Type: text/vnd.nextstrain.narrative+markdown' \
    --upload-file narrative.md

GET private dataset still works

curl http://localhost:5000/groups/test-private/victorlin-test \
    --header "Authorization: Bearer $token" \
    --header 'Accept: application/json' \
    --compressed \
        > dataset.json

DELETE private dataset still works

curl http://localhost:5000/groups/test-private/victorlin-test \
    --header "Authorization: Bearer $token" \
    --request "DELETE"

PUT private dataset still works

curl http://localhost:5000/groups/test-private/victorlin-test \
    --header "Authorization: Bearer $token" \
    --header 'Content-Type: application/vnd.nextstrain.dataset.main+json' \
    --upload-file dataset.json

@victorlin victorlin self-assigned this Jul 11, 2022
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 11, 2022 23:06 Inactive
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 12, 2022 22:17 Inactive
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from ab8390d to 4f5452c Compare July 14, 2022 19:36
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 14, 2022 19:36 Inactive
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 4f5452c to 46990fd Compare July 14, 2022 23:01
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 14, 2022 23:01 Inactive
Copy link
Member Author

@victorlin victorlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tsibley this is still in early stages, just focused on GET /groups/:groupName/overview. I have some https://github.com/nextstrain/nextstrain.org/labels/request%20for%20comments to make sure I'm doing this right!

src/sources/groups.js Outdated Show resolved Hide resolved
src/sources/models.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
@victorlin victorlin requested a review from tsibley July 14, 2022 23:17
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 46990fd to 4f5452c Compare July 18, 2022 18:27
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 18, 2022 18:28 Inactive
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 4f5452c to d817a3b Compare July 18, 2022 22:13
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 18, 2022 22:13 Inactive
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from d817a3b to 5dbab20 Compare July 18, 2022 22:37
@nextstrain-bot nextstrain-bot temporarily deployed to nextstrain-s-victorlin--ibcg12 July 18, 2022 22:37 Inactive
Copy link
Member Author

@victorlin victorlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tsibley progress here is just GET/PUT overview. Would you be able to review before I extend this approach to more endpoints?

src/endpoints/sources.js Outdated Show resolved Hide resolved
Copy link
Member

@tsibley tsibley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think this is the right direction!

Left lots of minor notes and comments for your discretion, plus a few actual requested changes.

src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
@tsibley
Copy link
Member

tsibley commented Jul 19, 2022

BTW, in your testing notes above, the PUT of the overview markdown uses Accept instead of Content-Type.

@victorlin
Copy link
Member Author

BTW, in your testing notes above, the PUT of the overview markdown uses Accept instead of Content-Type.

Good catch, but also this means it worked without the Content-Type header when it shouldn't. Will fix this.

@tsibley
Copy link
Member

tsibley commented Aug 4, 2022

Ah, good point. Looking closer, that's because the PUT endpoints are using contentTypesProvided instead of contentTypesConsumed.

src/endpoints/sources.js Outdated Show resolved Hide resolved
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 8aa733c to 25bccef Compare August 5, 2022 22:11
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
victorlin added a commit that referenced this pull request Aug 23, 2022
This is in line with the existing doc on groups access [[1]].
However, a better alternative would be to use a new authz tag for Group
as proposed in [[2]].

[1]: https://github.com/nextstrain/docs.nextstrain.org/blob/5b5f71fcfa65463c3920cfbdd340814d2be9e5d9/src/learn/groups/index.rst?plain=1#L122-L128
[2]: #570 (comment)
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 6f3867d to 2adabdb Compare August 23, 2022 20:27
src/endpoints/sources.js Outdated Show resolved Hide resolved
@victorlin
Copy link
Member Author

victorlin commented Aug 23, 2022

@tsibley I added DELETE which was simpler: d70cd69. Could you review before I use these approaches for /groups/:groupName/logo?

@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch from 1542f8f to d70cd69 Compare August 23, 2022 22:58
@victorlin victorlin requested a review from tsibley August 23, 2022 22:58
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch 2 times, most recently from e3736a2 to 0d8c4a6 Compare August 31, 2022 23:41
@victorlin victorlin marked this pull request as ready for review August 31, 2022 23:46
@victorlin victorlin requested a review from tsibley August 31, 2022 23:47
src/groups.js Outdated Show resolved Hide resolved
src/groups.js Outdated Show resolved Hide resolved
src/sources/groups.js Outdated Show resolved Hide resolved
src/sources/groups.js Outdated Show resolved Hide resolved
src/endpoints/groups.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Outdated Show resolved Hide resolved
src/endpoints/sources.js Show resolved Hide resolved
@tsibley
Copy link
Member

tsibley commented Sep 20, 2022

@victorlin Sorry this fell off my radar for a bit. I submitted some comments above (some of which were sitting pending for a week without me realizing they hadn't been submitted :-/). I want to get this across the line soon—it's been dragging out thru no fault of your own—so I'll prioritize more conversation around it. I also need to swap context back in for how it interacts with #581 and your comments there.

src/app.js Show resolved Hide resolved
…ception classes

Otherwise we have to do string comparison on the exception names instead
of cleaner instanceof checks.

I was confused for a bit because the documentation says you can import
these classes but that wasn't working in practice when I first tried it.
Turns out we require 3.52.0 and the classes weren't exported until
3.53.0, just one release later!
@victorlin
Copy link
Member Author

victorlin commented Sep 20, 2022

Force-pushing rebase onto latest changes from #583.

@tsibley see 9a1902b...ecca684 which is the new version of e98a3ea...39d4db2 (commits that were shared between this PR + #581). You may want to rebase your PR onto ecca684 and drop the old commits.

EDIT: see #570 (comment) for a better version of this.

src/groups.js Outdated Show resolved Hide resolved
victorlin and others added 3 commits September 21, 2022 15:55
Jest version 26 does not play well with ESM circular dependencies (to be
added in future commits) that work fine in production. Errors look like:

    RangeError: Maximum call stack size exceeded
    Exception in PromiseRejectCallback:
    internal/vm/module.js:321
            const module = await linker(identifier, this);

These errors go away with the next major version (27).
For actions on the group itself (e.g. membership management) rather than
the group's Source of datasets and narratives.
The Group management API (members, roles, overview, logo, etc) will live
under here.  Using a single namespace (…/settings/…) means we only add a
single new forbidden prefix for user-chosen dataset names in the group
(i.e. in addition to the existing …/narratives/… namespace).

Routes under this namespace are only visible to users with Read access
to the Group itself (not the GroupSource), i.e. access is limited to
group members even for public groups.
@victorlin
Copy link
Member Author

@tsibley see 9a1902b...833ea30 which is the new version of e98a3ea...39d4db2 (commits that were shared between this PR + #581). You may want to rebase your PR onto 833ea30 and drop the old commits.

Notably, 6ef1ec2 has been added so that tests will pass with the circular dependency used in bf7eecf. Context: #570 (comment)

Previously, only GroupSource.group was available. Having the inverse,
Group.source, allows for more direct access.

However, maintaining both of these properties in the object constructors
results in a bad cyclic dependency. Since GroupSource.group is already
set in its constructor, adding Group.source as a computed property.

Update existing code to use this new property as a shortcut to create
new GroupSource instance.
@victorlin victorlin force-pushed the victorlin/groups-api-endpoints branch 3 times, most recently from a18ffa3 to 8546855 Compare September 27, 2022 05:12
@tsibley
Copy link
Member

tsibley commented Sep 27, 2022

Does 7184cd7 still apply?

This allows for reuse later with other endpoints.

DELETE: Extract the logic of creating multiple DELETE requests in
deleteResource to its own function.

GET: Extract the call to proxyResponseBodyFromUpstream in
sendSubresource to its own function.

PUT: Extract the logic of creating a PUT request in receiveSubresource
to its own function. This requires a new type, upstreamUrlExtractor, to
be used since the URL for these requests are dependent on the method and
headers, which are defined in the new function.
Read-write access for group owners, read-only access for all other
members of a group, and no access for non-members.
Copy link
Member

@tsibley tsibley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ✨ Sorry for the several dead ends and fits and starts review that turned this into a bit of a slog. 🙏

@victorlin victorlin merged commit 73b0ca2 into master Sep 28, 2022
@victorlin victorlin deleted the victorlin/groups-api-endpoints branch September 28, 2022 17:05
@victorlin
Copy link
Member Author

@tsibley no worries! Thanks for your helpful comments here, learned a lot during the process!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Add RESTful API endpoints for Groups customizations
3 participants