-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Updating docs, fixing some places where configured ports weren't bein…
…g incldued properly (#189)
- Loading branch information
Showing
3 changed files
with
137 additions
and
50 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,31 +1,79 @@ | ||
| # API Gateway Integration | ||
|
|
||
| The Unified Ecommerce application expects to be run behind an API gateway of some sort to provide authentication support. The application supports two protocols for this: | ||
| The Unified Ecommerce application expects to be run behind the APISIX API gateway. APISIX's main job is to coordinate the integration between the application and SSO. | ||
|
|
||
| - Forward Authentication: the API gateway provides only username of the authenticated user to the application | ||
| - X-UserInfo: the API gateway provides a more complete user record to the application | ||
| ## Reasoning | ||
|
|
||
| Of these, the latter is the preferred method as it doesn't entail a further API call to another service to retrieve the user data. | ||
| The Unified Ecommerce application doesn't have (or need) its own login UI. It's intended to be used in conjunction with other systems, so it needs to share authentication and accounts with those systems. In addition, these accounts all need to be in sync with each other. | ||
|
|
||
| ## Forward Authentication | ||
| We've chosen Keycloak as the authentication system for Open Learning applications. It is the source of truth for user information and authentication, and other Open Learning applications are configured to use it for authentication. They redirect the user to Keycloak, and then Keycloak verifies the user and sends them back to the application (via OAuth2/OIDC). | ||
|
|
||
| Forward auth is provided by a number of API gateway (and gateway-like) services, including Traefik. To support this, the app extends the built-in Django `RemoteUserMiddleware` to use the `X-Forwarded-User` header. In addition to this, the app then attempts to load the user data from Keycloak to fill out the user account, as the only data that will be passed along will be the username. | ||
| For UE, APISIX handles this integration with Keycloak. For certain API endpoints, APISIX itself checks for a session and redirects the user through Keycloak. It then passes the user on to UE and attaches a payload of the user data in the headers. UE can then set up the Django session, create or update the local user account, and check permissions as it needs. | ||
|
|
||
| The app requires a service account in the relevant Keycloak realm so that it can pull user data. | ||
| UE doesn't have to coordinate with Keycloak or use OIDC at all in this scenario. APISIX controls that. Additionally, the APISIX configuration can be shared across services, so ideally everything routes through it, and users can seamlessly transition between individual applications after authenticating once. | ||
|
|
||
| ## X-UserInfo | ||
| ## Authentication Workflow | ||
|
|
||
| Unified Ecommerce API endpoints generally fall into one of three categories: | ||
|
|
||
| - Anonymous access: a number of APIs are accessible anonymously. (Product information falls into this category.) | ||
| - Authenticated access: other APIs require a session to be established within Unified Ecommerce. (Basket and order information APIs are in this category.) | ||
| - Transitional access: specific APIs that handle transition between anonymous and authenticated access. (Essentially, login.) | ||
|
|
||
| For anonymous access APIs, APISIX is configured to pass these along without change or processing. Any existing Django session will be used. | ||
|
|
||
| For authenticated access APIs, APISIX is configured in the same way, and passes these along as well. The user will receive an error if the Django session isn't established beforehand. | ||
|
|
||
| Transitional access APIs involve the APISIX OIDC integration. | ||
|
|
||
| ```{mermaid} | ||
| --- | ||
| title: Session Establishment | ||
| --- | ||
| flowchart LR | ||
| accessEndpoint["User hits the endpoint"] | ||
| hasApisixSession["User has an APISIX session"] | ||
| redirectSso["Redirected to Keycloak SSO"] | ||
| ssoAuth["Log in via SSO"] | ||
| ssoAuthOk["SSO Auth OK"] | ||
| ssoAuthBad["SSO Auth Fail"] | ||
| apisixAuth["Session setup in APISIX"] | ||
| intoDjango["Redirect into Django"] | ||
| fail["Auth failed"] | ||
| This method is useful when a more fully-fledged API gateway system is placed in front of the app. (APISIX is the canonical example and was the service that was used to write the initial integration, so this info is geared towards using APISIX.) | ||
| accessEndpoint --> hasApisixSession | ||
| hasApisixSession --> intoDjango | ||
| hasApisixSession --> redirectSso | ||
| redirectSso --> ssoAuth | ||
| ssoAuth --> ssoAuthBad | ||
| ssoAuth --> ssoAuthOk | ||
| ssoAuthOk --> apisixAuth | ||
| ssoAuthBad --> fail | ||
| apisixAuth --> intoDjango | ||
| ``` | ||
|
|
||
| Since APISIX sits before the Django app, it will first check to see if the user has a session established in APISIX. If it does, then the user is passed along to the Django app. If not, the user is redirected into Keycloak to log in. Assuming that succeeds, APISIX receives the user back, sets up its own session, and then sends the user to the Django app with the APISIX payload attached. (If the user can't get past Keycloak, the process stops.) | ||
|
|
||
| APISIX attaches user information in a special `X-UserInfo` header. A middleware within the Django app processes this header, either updates or creates a user account, and establishes a Django session for the account with the data contained within. | ||
|
|
||
| This workflow is used by the `/establish_session` endpoint. The frontend calls an endpoint to retrieve the current user data, and redirects the user to `/establish_session` if the user's not logged in. This endpoint then logs the user in with the processed APISIX data, starts a Django session, and sends the user back to the frontend. The user can then use the rest of the API as an authenticated user. | ||
|
|
||
| ## X-UserInfo | ||
|
|
||
| When configured to use authentication via OIDC Connect, APISIX returns the user data back to the application by injecting it into the HTTP headers sent to the app. A custom middleware in the application decodes this data, and takes action based on it. | ||
|
|
||
| For _local_ deployments, APISIX sends user data retrieved via OIDC in the `X-UserInfo` header. The data is sent as a base64-encoded JSON object, and its contents may vary but include: | ||
| APISIX sends user data retrieved via OIDC in the `X-UserInfo` header. The data is sent as a base64-encoded JSON object, and its contents may vary but include: | ||
|
|
||
| - The user's email address (`email`) | ||
| - The UUID associated with the user in the SSO system (`preferred_username`) | ||
| - The user's first and last name (`given_name`, `family_name`) | ||
|
|
||
| The middleware creates or updates the user account based on this data and sets the session user appropriately. Note that, unlike forward authentication, APISIX includes enough user data to construct and update the user record so the app does not need to make a separate call to Keycloak directly for this data. | ||
| The middleware creates or updates the user account based on this data and sets the session user appropriately. | ||
|
|
||
| ```{note} | ||
| Regular forward authentication doesn't include the user data. If we used that, the app would have to perform a round-trip to Keycloak to retrieve it. | ||
| ``` | ||
|
|
||
| For Heroku deployments, we do something different because we need to be able to trust the data in the `X-UserInfo` header. | ||
| ### Trust | ||
|
|
||
| > _TODO:_ Fill this out - we don't have the info here yet. The challenge here is preventing injection: the app won't be DMZed in production so we need to be able to verify the source of the data. | ||
| Having the app configured in this way means that it **must** sit behind APISIX. At time of writing, the APISIX middleware also blindly trusts the payload that APISIX sends along. So, the Django app must not be exposed directly to the Internet when it is deployed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,34 +1,75 @@ | ||
| # Events | ||
|
|
||
| Unified Ecommerce emits events when certain transaction states are hit. These are emitted to the relevant integrated system as a hit to a configured webhook. | ||
| Certain operations within Unified Ecommerce trigger events, and those events can send data to the relevant configured integrated systems. | ||
|
|
||
| ## Data Sent | ||
| The integrated system model has a field for a webhook URL. Data for all events are sent to this URL. The integrated system itself decides whether or not to take action on the data. | ||
|
|
||
| ## Events | ||
|
|
||
| The data that gets sent is: | ||
| These are the events that are triggered: | ||
|
|
||
| * `reference_number` - the reference number for the order | ||
| * `system_slug` - the system slug for the data being sent | ||
| * `user` - nested object containing user information | ||
| * `total_price_paid` - the total price paid for the order, inclusive of _all_ items on the order | ||
| * `state` - the order state | ||
| * `lines` - line items in the order | ||
| | Event (in UE) | Type | Description | | ||
| | ------------- | ------------ | ---------------------------------------------------------------- | | ||
| | `basket_add` | `presale` | Triggered when an item is added to the basket. | | ||
| | `post_sale` | `postsale` | Triggered when an order has been completed successfully. | | ||
| | `post_refund` | `postrefund` | Triggered when an item has been refunded from a completed order. | | ||
|
|
||
| Each system will only get the data that is relevant to itself, which will be indicated by the `system_slug` attribute. If the slug does not match what the system expects, the webhook target should return a 500 response so that Unified Ecommerce can log a Sentry error. | ||
| ```{note} | ||
| The Event tracks the plugin hook spec that is called to generate the event. | ||
| ``` | ||
|
|
||
| ## Data Sent | ||
|
|
||
| To that end, the `lines` attribute will only include the line items that are for the system that UE is talking to. Totalling the line cost will not necessarily match the `total_price_paid` value as the total may include line items not visible to the system. | ||
| The event data is wrapped in a standard container (implemented in `payments/serializers/v0` as the `WebhookBase` dataclass): | ||
|
|
||
| - `system_slug`: the system slug for the data being sent | ||
| - `system_key`: the shared key for the system | ||
| - `user`: nested object containing user information | ||
| - `type`: the event type (see table above) | ||
| - `data`: event-specific data | ||
|
|
||
| Each system will only get the data that is relevant to itself, which will be indicated by the `system_slug` attribute. The system should verify the slug and key sent are valid, and emit a 401 error if they aren't. | ||
|
|
||
| User data includes: | ||
|
|
||
| * `username` - the username of the purchaser | ||
| * `email` - the email address of the purchaser | ||
| * `first_name` - the purchaser's first name | ||
| * `last_name` - the purchaser's last name | ||
| - `id`: the ID of the purchaser (this is Unified Ecommerce's ID) | ||
| - `username`: the username of the purchaser (this will be a UUID corresponding to a Keycloak user) | ||
| - `email`: the email address of the purchaser | ||
| - `first_name`: the purchaser's first name | ||
| - `last_name`: the purchaser's last name | ||
|
|
||
| The `data` attribute differs depending on what event is being sent. | ||
|
|
||
| For `presale`: | ||
|
|
||
| - `action`: either "add" or "remove" | ||
| - `product`: the product added or removed to the basket | ||
|
|
||
| For `postsale`: | ||
|
|
||
| - `reference_number`: the reference number of the order. (Despite this saying "number" this is generally a string.) | ||
| - `total_price_paid`: the total amount paid for the order, inclusive of any discounts and taxes assessed. | ||
| - `state`: the state of the order. This should always be `fulfilled`. | ||
| - `lines`: array of line items for the order | ||
|
|
||
| `Line` data includes: | ||
|
|
||
| - `id`: an ID for the line item | ||
| - `quantity`: quantity on order | ||
| - `item_description`: description of the item | ||
| - `unit_price`: the unit price (before tax/discounts) of the item | ||
| - `total_price`: the amount charged for the item | ||
| - `product`: the product | ||
|
|
||
| `Product` data includes (just relevant fields): | ||
|
|
||
| Individual line items include: | ||
| - `id`: an ID for the product | ||
| - `sku`: the product's SKU. By convention, this should be the readable ID of the resource in the integrated system. | ||
| - `name`: the product's name | ||
| - `description`: the product's description | ||
| - `system_data`: JSON; system-specific data. This is defined by the integrated system. | ||
| - `price`: the base price of the product | ||
|
|
||
| * `quantity` - the number of items purchased for this line (this will generally be one) | ||
| * `discounted_price` - the discounted price of the line item | ||
| * `product_sku` - the line item's SKU | ||
| * `system_data` - the line item's system data | ||
| ## Architecture | ||
|
|
||
| Products configured in the Universal Ecommerce system can contain system-specific data in JSON format - that is what is returned in the `system_data` attribute. | ||
| The event system is built using Pluggy, REST framework serializers, and Celery tasks. The hookspecs listed in the table in Events have a hook implementation that queues a task to send the data to the target URL(s) without blocking the user. |