Add support for O365 client credentials grant flow (AccessAsApp) (#65)

Author: simonrob

README.md

@@ -6,7 +6,7 @@ Transparently add OAuth 2.0 support to IMAP/POP/SMTP client applications, script
 Email services that support IMAP, POP and/or SMTP access are increasingly requiring the use of OAuth 2.0 to authenticate connections, but not all clients support this method. This script creates a simple local proxy that intercepts the traditional IMAP/POP/SMTP authentication commands and transparently replaces them with the appropriate SASL (X)OAuth 2.0 commands and credentials. Your email client can continue to use the `login` or `auth`/`authenticate` options, with no need to make it aware of OAuth's existence. The proxy works in the background with a menu bar/taskbar helper or as a system service, and is compatible with macOS, Windows and Linux.
 
 ### Example use-cases
-- You need to use an Office 365 email account, but don't get on with Outlook. The email client you like doesn't support OAuth 2.0, which will be mandatory from [October 2021](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-september-2021-update/ba-p/2772210).
+- You need to use an Office 365 email account, but don't get on with Outlook. The email client you like doesn't support OAuth 2.0, which will be mandatory from [January 2023](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437).
 - You used to use Gmail via IMAP/POP/SMTP with your raw account credentials (i.e., your real password), but cannot do this now that Google has disabled this method, and don't want to use an [App Password](https://support.google.com/accounts/answer/185833) (or cannot enable this option).
 - You have an account already set up in an email client, and you need to switch it to OAuth 2.0 authentication. You can edit the server details, but the client forces you to delete and re-add the account to enable OAuth 2.0, and you don't want to do this.
 - You have made your own script or application that sends or receives email, but it doesn't support OAuth 2.0, and you don't want to have to modify it to implement this.
@@ -31,14 +31,16 @@ After your accounts are fully set-up and authorised, no further proxy interactio
 ### OAuth 2.0 client credentials
 As part of the proxy setup process you need to provide an OAuth 2.0 `client_id` and `client_secret` to allow it to authenticate with email servers on your behalf.
 
-If you have an existing client ID and secret for a desktop app, you can use these directly in the proxy. If this is not possible, you can also reuse the client ID and secret from any email client that supports IMAP/POP/SMTP OAuth 2.0 authentication with the email server you would like to connect to (such as the [various](https://github.com/mozilla/releases-comm-central/blob/master/mailnews/base/src/OAuth2Providers.jsm) [open](https://github.com/Foundry376/Mailspring/blob/master/app/internal_packages/onboarding/lib/onboarding-constants.ts) [source](https://gitlab.gnome.org/GNOME/evolution-data-server/-/blob/master/CMakeLists.txt) [clients](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/blob/master/meson_options.txt) with OAuth 2.0 support), but please do this with care and restraint as access through reused tokens will be associated with the token owner rather than your own client.
+If you have an existing client ID and secret for a desktop app, you can use these directly in the proxy. If this is not possible, you can also reuse the client ID and secret from any email client that supports IMAP/POP/SMTP OAuth 2.0 authentication with the email server you would like to connect to (such as [the](https://github.com/mozilla/releases-comm-central/blob/master/mailnews/base/src/OAuth2Providers.jsm) [various](https://github.com/Foundry376/Mailspring/blob/master/app/internal_packages/onboarding/lib/onboarding-constants.ts) [open](https://gitlab.gnome.org/GNOME/evolution-data-server/-/blob/master/CMakeLists.txt) [source](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/blob/master/meson_options.txt) [clients](https://github.com/M66B/FairEmail/blob/master/app/src/main/res/xml/providers.xml) with OAuth 2.0 support), but please do this with care and restraint as access through reused tokens will be associated with the token owner rather than your own client.
 
 If you do not have access to credentials for an existing client you will need to register your own. The process to do this is different for each provider, but the registration guides for several common ones are linked below. In all cases, when registering, make sure your client is set up to use an OAuth scope that will give it permission to access IMAP/POP/SMTP as desired. It is also highly recommended to use a scope that will grant "offline" access (i.e., a way to [refresh the OAuth 2.0 authentication token](https://oauth.net/2/refresh-tokens/) without user intervention). The [sample configuration file](emailproxy.config) provides example scope values for several common providers.
 
 - Office 365: register a new [Microsoft identity application](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app)
 - Gmail / Google Workspace: register a [Google API desktop app client](https://developers.google.com/identity/protocols/oauth2/native-app)
 - AOL and Yahoo Mail are not currently allowing new client registrations with the OAuth email scope – the only option here is to use the credentials from an existing client that does have this permission.
 
+The proxy also supports the [client credentials grant flow](https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#use-client-credentials-grant-flow-to-authenticate-imap-and-pop-connections) if needed, though please note that Office 365 only supports IMAP/POP in this mode, and _not_ SMTP. See the [sample configuration file](emailproxy.config) for further details.
+
 
 ## Optional arguments and configuration
 When starting the proxy there are several optional arguments that can be set to customise its behaviour.
@@ -51,7 +53,7 @@ When starting the proxy there are several optional arguments that can be set to
 
 `--config-file` allows you to specify the location of a [configuration file](emailproxy.config) that the proxy should load. If this argument is not provided, the proxy will look for `emailproxy.config` in the same directory as the script itself.
 
-`--log-file` allows you to specify the location of a file to send log output to. This option overrides the proxy's default behaviour, which varies by platform (see [`Log.initialise()`](https://github.com/simonrob/email-oauth2-proxy/search?q=staticmethod+initialise)).
+`--log-file` allows you to specify the location of a file to send log output to. This option overrides the proxy's default behaviour, which varies by platform (see [below](#troubleshooting) for details).
 
 `--debug` enables debug mode, printing more verbose output to the log as [discussed below](#troubleshooting). This argument is identical to enabling debug mode from the menu bar icon.
 
@@ -79,7 +81,7 @@ On macOS the setup and installation instructions above should automatically inst
 
 When first launching on Linux you may encounter errors similar to `Namespace […] not available`. This is caused by missing dependencies for [pystray](https://github.com/moses-palmer/pystray/) and [pywebview](https://github.com/r0x0r/pywebview/), which are used to display the menu bar icon and login windows. See the [pywebview dependencies](https://pywebview.flowrl.com/guide/installation.html#dependencies) page and [issue 1](https://github.com/simonrob/email-oauth2-proxy/issues/1#issuecomment-831746642) in this repository for a summary and suggestions about how to resolve this.
 
-A similar issue may occur on Windows with the [pythonnet](https://github.com/pythonnet/pythonnet) package, which is required by [pywebview](https://github.com/r0x0r/pywebview). If you are unable to resolve this by following the [pythonnet installation instructions](https://github.com/pythonnet/pythonnet/wiki/Installation), you may find that installing a [prebuilt wheel](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonnet) helps fix the issue. Note that the public releases of pythonnet can take some time to be compatible with the latest major python release, so it can be worth using a slightly older version of python, or a prerelease version of pythonnet. This should be handled by the proxy's `requirements.txt`, but see [issue 45](https://github.com/simonrob/email-oauth2-proxy/issues/45) for a workaround if needed.
+A similar issue may occur on Windows with the [pythonnet](https://github.com/pythonnet/pythonnet) package, which is required by [pywebview](https://github.com/r0x0r/pywebview). If you are unable to resolve this by following the [pythonnet installation instructions](https://github.com/pythonnet/pythonnet/wiki/Installation), you may find that installing a [prebuilt wheel](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonnet) helps fix the issue. Note that the public releases of pythonnet can take some time to be compatible with the latest major python release, so it can be worth using a slightly older version of python, or a pre-release version of pythonnet. This should be handled by the proxy's `requirements.txt`, but see [issue 45](https://github.com/simonrob/email-oauth2-proxy/issues/45) for a workaround if needed.
 
 If you are using the proxy in a non-GUI environment it is also possible to skip installation of dependencies that apply only to the interactive version. To do this, install the script's requirements via `python -m pip install -r requirements-no-gui.txt`, and pass the [`--no-gui`](#optional-arguments-and-configuration) argument when starting the proxy. Please note that the proxy was designed as a GUI-based tool from the outset due to the inherently interactive nature of OAuth 2.0 authorisation, and there are limitations to its ability to support fully no-GUI operation. See the [optional arguments and configuration](#optional-arguments-and-configuration) section of this file for further details.
 
@@ -110,7 +112,7 @@ The [plugins branch](https://github.com/simonrob/email-oauth2-proxy/tree/plugins
 ## Related projects and alternatives
 Michael Stepner has created a [Terraform configuration](https://github.com/michaelstepner/email-oauth2-proxy-aws) that helps run this proxy on a lightweight cloud server (AWS EC2). Philippe-Adrien Nousse has provided an [example Docker configuration](https://github.com/linka-cloud/email-oauth2-proxy/commit/67ca6b8fd0709d85480de2e3ea0af79439e6ba22) (though please note that the fork is otherwise outdated, and it is better to use this repository for the proxy script itself).
 
-If you already use postfix, the [sasl-xoauth2](https://github.com/tarickb/sasl-xoauth2) plugin is probably a better solution than running this proxy.
+If you already use postfix, the [sasl-xoauth2](https://github.com/tarickb/sasl-xoauth2) plugin is probably a better solution than running this proxy. Similarly, if you use an application that is able to handle OAuth 2.0 tokens but just cannot retrieve them itself, then [pizauth](https://github.com/ltratt/pizauth), [mailctl](https://github.com/pdobsan/mailctl) or [oauth-helper-office-365](https://github.com/ahrex/oauth-helper-office-365) may be more appropriate. There are also dedicated helpers available for specific applications (e.g., [mutt_oauth2](https://gitlab.com/muttmua/mutt/-/blob/master/contrib/mutt_oauth2.py)), and several open-source email clients that support OAuth 2.0 natively (e.g., [Thunderbird](https://www.thunderbird.net/), [Mailspring](https://getmailspring.com/), [FairEmail](https://email.faircode.eu/), [Evolution](https://wiki.gnome.org/Apps/Evolution), etc).
 
 [DavMail](http://davmail.sourceforge.net/) is an alternative that takes the same approach of providing a local IMAP/POP/SMTP server (and more) for Exchange/Office 365, though it does this by translating these protocols into Exchange API calls rather than proxying the connection. That approach is very useful in situations where server-side IMAP/POP/SMTP is not supported or enabled, or the full Exchange capabilities are needed, but it has limitations in terms of speed and the number of email messages that can be retrieved. This proxy was developed to work around these limitations for providers that do support IMAP/POP/SMTP natively.
 

emailproxy.config

@@ -2,7 +2,8 @@
 documentation = This is a sample Email OAuth 2.0 Proxy configuration file. Configure the proxy by adding items in the
 	[Server setup] and [Account setup] sections below. You may delete any servers or accounts that you do not intend to
 	use. Documentation is provided inline, with example setups for Gmail and Office 365 (though you will need to enter
-	your own desktop app API client credentials in the accounts section).
+	your own desktop app API client credentials in the accounts section). Use the `Reload configuration file` menu
+	option (or restart the proxy) to apply any changes.
 
 
 [Server setup]
@@ -96,6 +97,15 @@ documentation = Accounts are specified using your email address as the section h
 	- It is possible to create Office 365 clients that do not require a secret to be sent. If this is the case for your
 	setup, delete the `client_secret` line from your account's configuration entry (do not leave the default value).
 
+	- If your Office 365 configuration uses the client credentials grant flow rather than interactive authentication,
+	add your account entry as normal, but do not add a `permission_url` value (it does not apply in this mode, and its
+	absence signals to the proxy to use the appropriate token retrieval mechanism). Please note that with the client
+	credentials grant flow SMTP is not supported, and by default there is essentially no local access control when using
+	IMAP/POP. To alleviate this, you are encouraged to enable client secret encryption in the proxy - see the setting at
+	the end of this file. In addition, if you are using the proxy in an environment where there is any possibility of
+	malicious access attempts then pre-encrypting account entries is highly recommended - see the example setup script
+	at https://github.com/simonrob/email-oauth2-proxy/issues/61#issuecomment-1259110336.
+	
 	Advanced account configuration:
 	- For most configurations the default `redirect_uri` value of `http://localhost` is correct, unless you have
 	explicitly set your OAuth 2.0 client to use a different address here (which you will need to manually redirect to
@@ -157,6 +167,16 @@ documentation = The parameters below control advanced options for the proxy. In
 	if the proxy is used in a headless (often also public-facing) context, where authentication flows are more likely to
 	be laborious or need administrator intervention, this can potentially result in a denial-of-service issue, whether
 	malicious or not. Set to False and the proxy will instead return an error when an incorrect password is provided.
+	
+	- encrypt_client_secret_on_first_use (default = False): The proxy encrypts sensitive configuration values (e.g.,
+	cached access tokens) using the password that is given when accessing an account via IMAP/POP/SMTP. It does not do
+	this for values that are not sensitive. In the most common operation mode (i.e., interactively authorising account
+	access), the `client_secret` value falls into this category - it is not actually secret, and there is no real need
+	to prevent access to it. However, when using the Office 365 client credentials grant flow there is no user involved,
+	and possession of the secret grants full access to an account. If you use this method and it is possible that others
+	may gain access to the proxy's configuration file, set `encrypt_client_secret_on_first_use` and the proxy will
+	replace the `client_secret` value with a new property `client_secret_encrypted` at the next token refresh.
 
 [emailproxy]
 delete_account_token_on_password_error = True
+encrypt_client_secret_on_first_use = False