Stir/Shaken Refactor DRAFT

THIS IS A DRAFT PR/COMMIT and will not be merged as is. To make
reviewing easier, there's an earlier commit that removes the
existing completely.  Otherwise, the review diffs would be an
unfollowable mess of deletions and additions.  The final commit
will be a traditional single "update" commit without the removal.

There are also outstanding items that I'm waiting on resolution of.

* Do we need to support the compact form of the Identity PASSporT?
  Based on ATIS-10000074,  I don't _think_ we do but I'm awaiting
  clarification.  NOT AT THIS TIME.

* ATIS-10000074 also states that we must not follow redirections or
  attempt to retrieve certificates using URLs that have
  user:password components or path or query parameters.  I still need
  to implement that check.  DONE.

* RFC-8224 says we must include one or more "mky" Media Key entries
  in the PASSporT containing the fingerprints in the SDP if DTLS
  is in use.  Although this is now implemented, it defaults to
  "off" because I can't find any other implementation that respects
  them.  In fact OpenSIPS will fail to validate any Identity header
  that has them.
* Many enums and functions that are private to res_stir_shaken still
  have "ast_stir_shaken" prefixes on them.  They should be renamed
  just for clarity's sake.
* It was my intention to have the stir-shaken internals set error
  codes and text responses in the contexts to be passed back to the
  outside caller so the caller could decide whether to emit
  messages or not.  I never got to that and need to either
  implement that or remove the associated fields in the contexts
  and clean up the error messages.

Things that need to be done ouside this commit:
* Implement dialplan function that allows a dialplan author to
  reject a call with a specific SIP response code if they deem it
  necessary based on the stir-shaken results passed to them.
  DONE

* Add alembic scripts so the configuration can be database based.
* Decide if/how to implement RFC-9090 Certificate Delegation.
* Decide if/how to implement validation of the TNAuthList
  extension in certificates.  Right now we only validate that
  it's present, not its content.
* Decide if/how to implement RFC-8946 Diverted Calls.
* Decide if/how to handle multiple Identity headers.
* Decide if/how to handle passing received identity headers to an
  outgoing INVITE.

REAL COMMIT MESSAGE:

Why do we need a refactor?

The original stir/shaken implementation was started over 3 years ago
when little was understood about practical implementation.  The
result was an implementation that, until now, wouldn't interoperate
with any other stir-shaken implementations.

There were also a number of stir-shaken features and RFC
requirements that were never implemented such as TNAuthList
certificate validation, sending Reason headers in SIP responses
when verification failed but we wished to continue the call, and
the ability to send Media Key(mky) grants in the Identity header
when the call involved DTLS.

Finally, there were some performance concerns around outgoing
calls and selection of the correct certificate and private key.
The configuration was keyed by an arbitrary name which meant that
for every outgoing call, we had to scan the entire list of
configured TNs to find the correct cert to use.  With only a few
TNs configured, this wasn't an issue but if you have a thousand,
it could be.

What's changed?

* Configuration objects have been refactored to be clearer about
  their uses and to fix issues.
    * The "general" object was renamed to "verification" since it
      contains parameters specific to the incoming verification
      process.  It also never handled ca_path and crl_path
      correctly.
    * A new "attestation" object was added that controls the
      outgoing attestation process.  It sets default certificates,
      keys, etc.
    * The "certificate" object was renamed to "tn" and had it's key
      change to telephone number since outgoing call attestation
      needs to look up certificates by telephone number.
    * The "profile" object had more parameters added to it that can
      override default parameters specified in the "attestation"
      and "verification" objects.
    * The "store" object was removed altogther as it was never
      implemented.

* We now use libjwt to create outgoing Identity headers and to
  parse and validate signatures on incoming Identiy headers.  Our
  previous custom implementation was much of the source of the
  interoperability issues.

* General code cleanup and refactor.
    * Moved things to better places.
    * Separated some of the complex functions to smaller ones.
    * Using context objects rather than passing tons of parameters
      in function calls.
    * Removed some complexity and unneeded encapsuation from the
      config objects.

UserNote: Asterisk's stir-shaken feature has been refactored to
correct interoperability, RFC compliance, and performance issues.
See https://docs.asterisk.org/Deployment/STIR-SHAKEN for more
information.

UpgradeNote: The stir-shaken refactor is a breaking change but since
it's not working now we don't think it matters. The
stir_shaken.conf file has changed significantly which means that
existing ones WILL need to be changed.  The stir_shaken.conf.sample
file in configs/samples/ has quite a bit more information.  This is
also an ABI breaking change since some of the existing objects
needed to be changed or removed, and new ones added.

configs/samples/stir_shaken.conf.sample

@@ -0,0 +1,347 @@
+;--
+
+There are 4 object types used by the STIR/SHAKEN process...
+
+The "verification" object sets the parameters for verification
+of the Identity header and caller id on incoming INVITE requests.
+
+The "attestation" object sets the parameters for creating an Identity
+header which attests to the ownership of the caller id on outgoing
+INVITE requests.
+
+One or more "tn" objects that are used to create the outgoing Identity
+header.  Each object's "id" is a specific caller-id telephone number
+and the object contains the URL to the certificate that was used to
+attest to the ownership of the caller-id, the level (A,B,C) of the
+attestation you're making, and the private key the asterisk
+attestation service will use to sign the Identity header.  When
+an outgoing INVITE request is placed, the attestation service will
+look up the caller-id in the tn object list and if it's found, use
+the information in the object to create the Identity header.
+
+One or more "profile" objects that can be associated to channel
+driver endpoints (currently only chan_pjsip).  Profiles can set
+whether verification, attestation, both or neither should be
+performed on requests coming in to this endpoint or requests
+going out from this endpoint. They can also set acls on what URLs
+we should be allowed to retrieve certificates from on incoming
+requests.
+
+--;
+
+
+;--
+=======================================================================
+  Verification Object Description
+=======================================================================
+The "verification" object sets the parameters for verification
+of the Identity header on incoming INVITE requests.
+Only one "verification" object may exist.
+
+Parameters:
+
+-- global_disable -----------------------------------------------------
+If set, globally disables the verification service.
+Default: no
+
+-- load_system_certs---------------------------------------------------
+If set, loads the system Certificate Authority certificates
+(usually located in /etc/pki/CA) into the trust store used to
+validate the certificates in incoming requests.  This is not
+normally required as service providers will usually provide their
+CA certififcate to you separately.
+Default: no
+
+-- ca_file -----------------------------------------------------------
+Path to a single file containing a CA certificate or certificate chain
+to be used to validate the certificates in incoming requests.
+Default: none
+
+-- ca_path -----------------------------------------------------------
+Path to a directory containing one or more CA certificates to be used
+to validate the certificates in incoming requests.  The files in that
+directory must contain only one certificate each and the directory
+must be hashed using the OpenSSL 'c_rehash' utility.
+Default: none
+
+NOTE:  Both ca_file and ca_path can be specified but at least one
+MUST be.
+
+-- crl_file -----------------------------------------------------------
+Path to a single file containing a CA certificate revocation list
+to be used to validate the certificates in incoming requests.
+Default: none
+
+-- crl_path -----------------------------------------------------------
+Path to a directory containing one or more CA certificate revocation
+lists to be used to validate the certificates in incoming requests.
+The files in that directory must contain only one certificate each and
+the directory must be hashed using the OpenSSL 'c_rehash' utility.
+Default: none
+
+NOTE:  Neither crl_file nor crl_path are required.
+
+-- cert_cache_dir -----------------------------------------------------
+Incoming Identity headers will have a URL pointing to the certificate
+used to sign the header.  To prevent us from having to retrieve the
+certificate for every request, we maintain a cache of them at the
+'cert_cache_dir' specified.  The directory will be checked for
+existence and writability at startup.
+Default: <astvarlibdir>/keys/stir_shaken/cache
+
+-- max_cache_entry_age ------------------------------------------------
+Maximum number of seconds after retrieval a certificate in the cache
+can be used before re-retrieving it.
+Default: 3600 (1 hour)
+
+-- max_cache_size -----------------------------------------------------
+Maximum number of entries the cache can hold.
+Not presently implemented.
+
+-- curl_timeout -------------------------------------------------------
+The number of seconds we'll wait for a response when trying to retrieve
+the certificate specified in the incoming Identity header.
+Default: 2
+
+-- max_iat_age --------------------------------------------------------
+The "iat" parameter in the Identity header indicates the time the
+sender actually created their attestation. If that is older than the
+number of seconds set here, the request will be considered "failed".
+Default: 15
+
+-- max_date_header_age ------------------------------------------------
+The sender MUST also send a SIP Date header in their request.  If we
+receive one that is older than the number of seconds set here, the
+request will be considered "failed".
+Default: 15
+
+-- failure_action -----------------------------------------------------
+Indicates what will happen to requests that have failed verification.
+Must be one of:
+- continue -
+  Continue processing the request.  You can use the
+  STIR_SHAKEN dialplan function to determine whether
+  the request passed or failed verification and take
+  the action you deem appropriate.
+
+- reject_request -
+  Reject the request immediately using the SIP response codes
+  defined by RFC8224.
+
+- continue_return_reason -
+  Continue processing the request but send a SIP Reason header
+  back to the originator in the next provisional response indicating
+  the issue according to RFC8224.  You can use the STIR_SHAKEN
+  dialplan function to determine whether the request passed or
+  failed verification and take the action you deem appropriate.
+
+Default: continue
+NOTE: This parameter may be overridden in profile objects defined
+below.
+
+-- use_rfc9410_responses ----------------------------------------------
+If set, when sending Reason headers back to originators, the protocol
+header parameter will be set to "STIR" rather than "SIP".  This is a
+new protocol defined in RFC9410 and may not be supported by all
+participants.
+Default: no
+NOTE: This parameter may be overridden in profile objects defined
+below.
+
+Example:
+--;
+
+;[verification]
+;global_disable = yes
+;load_system_certs = no
+;ca_path = /var/lib/asterisk/keys/stir_shaken/verification_ca
+;cert_cache_dir = /var/lib/asterisk/keys/stir_shaken/verification_cache
+;failure_action = reject_request
+;curl_timeout=5
+;max_iat_age=60
+;max_date_header_age=60
+;max_cache_entry_age = 300
+
+;--
+=======================================================================
+  Attestation Object Description
+=======================================================================
+The "attestation" object sets the parameters for creating an Identity
+header which attests to the ownership of the caller id on outgoing
+INVITE requests.
+Only one "attestation" object may exist.
+
+Parameters:
+
+-- global_disable -----------------------------------------------------
+If set, globally disables the attestation service.  No Identity headers
+will be added to any outgoing INVITE requests.
+Default: no
+
+-- check_tn_cert_public_url -------------------------------------------
+Identity headers in outgoing requests must contain a URL that points
+to the certificate used to sign the header.  Setting this parameter
+tells Asterisk to actually try to retrieve the certificates defined
+in the "tn" objects defined below and fail loading that tn if the cert
+can't be retrieved or if its 'Not Valid Before" -> 'Not Valid After"
+date range doesn't include today.  This is a network intensive process
+so use with caution.
+Default: no
+
+-- default_public_cert_url --------------------------------------------
+The URL to the certificate you received from the issueing authority.
+They may give you a URL to use or you may have to host the certificate
+yourself and provide your own URL here.
+Default: none
+WARNING:  Make absolutely sure the file that's made public doesn't
+accidentally include the privite key as well as the certificate.
+If you set "check_tn_cert_public_url" in the "attestation" section
+above, the tn will not be loaded and a "DANGER" message will be output
+on the asterisk console if the file does contain a private key.
+NOTE: This parameter may be overridden in "tn" objects defined below.
+
+-- default_private_key_file -------------------------------------------
+The path to a file containing the private key you received from the
+issuing authority.  The file must NOT be group or world readable or
+writable so make sure the user the asterisk process is running as is
+the owner.
+Default: none
+NOTE: This parameter may be overridden in "tn" objects defined below.
+
+-- default_attest_level -----------------------------------------------
+The level of the attestation you're making.
+One of "A", "B", "C"
+Default: none
+NOTE: This parameter may be overridden in "tn" objects defined below.
+
+-- default_send_mky -----------------------------------------------------------
+If set and an outgoing call uses DTLS, an "mky" Media Key grant will
+be added to the Identity header.  Although RFC8224/8225 require this,
+not many implementations support it so a remote verification service
+may fail to verify the signature.
+Default: no
+
+Example:
+--;
+
+;[attestation]
+;global_disable = no
+;default_private_key_path = /var/lib/asterisk/keys/stir_shaken/tns/multi-tns-key.pem
+;default_public_cert_url = https://example.com/tncerts/multi-tns-cert.pem
+;default_attest_level = C
+
+;--
+=======================================================================
+  TN Object Description
+=======================================================================
+Each "tn" object contains the parameters needed to create the Identity
+header used to attest to the ownership of the caller-id on outgoing
+requests.  When an outgoing INVITE request is placed, the attestation
+service will look up the caller-id in this list and if it's found, use
+the information in the object to create the Identity header.
+The private key and certificate needed to sign the Identity header are
+usually provided to you by the telephone number issuing authority along
+with their certificate authority certificate.  You should give the CA
+certificate to any recipients who expect to receive calls from you
+although this has probably already been done by the issuing authority.
+
+The "id" of this object MUST be a canonicalized telephone nmumber which
+starts with a country code.  The only valid characters are the numbers
+0-9, '#' and '*'.
+
+Parameters:
+
+-- type (required) ----------------------------------------------------
+Must be set to "tn"
+Default: none
+
+-- public_cert_url (required) -----------------------------------------
+The URL to the certificate you received from the issueing authority.
+They may give you a URL to use or you may have to host the certificate
+yourself and provide your own URL here.
+Default: <default_public_cert_url from attestation>
+WARNING:  Make absolutely sure the file that's made public doesn't
+accidentally include the privite key as well as the certificate.
+If you set "check_tn_cert_public_url" in the "attestation" section
+above, the tn will not be loaded and a "DANGER" message will be output
+on the asterisk console if the file does contain a private key.
+
+-- private_key_file (required) ----------------------------------------
+The path to a file containing the private key you received from the
+issuing authority.  The file must NOT be group or world readable or
+writable so make sure the user the asterisk process is running as is
+the owner.
+Default: <default_private_key_file from attestation>
+
+-- attest_level (required) --------------------------------------------
+The level of the attestation you're making.
+One of "A", "B", "C"
+Default: <default_attest_level from attestation>
+
+Example:
+--;
+
+;[18005551515]
+;type = tn
+;private_key_path = /var/lib/asterisk/keys/stir_shaken/tns/18005551515-key.pem
+;public_cert_url = https://example.com/tncerts/18005551515-cert.pem
+;attest_level = C
+
+
+;--
+=======================================================================
+  Profile Object Description
+=======================================================================
+A "profile" object can be associated to channel driver endpoint
+(currently only chan_pjsip) and can set verification and attestation
+parameters specific to endpoints using this profile.  If you have
+multiple upstream providers, this is the place to set parameters
+specific to them.
+
+The "id" of this object is arbitrary and you'd specify it in the
+"stir_shaken_profile" parameter of the endpoint.
+
+Parameters:
+
+-- type (required) ----------------------------------------------------
+Must be set to "profile"
+Default: none
+
+-- permit/deny --------------------------------------------------------
+To help prevent MITM attacks, you can restrict from where you can
+retrieve certificates during the verification process.  This can prevent
+an attacker from sending you a request pretending to be a known
+originator with a mailcious certificate URL.  See acl.conf.sample to
+see examples of how to specify the permit/deny parameters.
+Default: none
+
+-- acllist ------------------------------------------------------------
+Rather than providing individual permit/deny parameters, you can set
+the acllist parameter to an acl list predefined in acl.conf.
+Default: none
+
+All of the "verification" parameters defined above can be set on a profile
+with the exception of 'global_disable' and 'load_system_certs'.
+
+All of the "attestation" parameters defined aboive can be set on a profile
+with the exception of 'global_disable'.
+
+Example:
+--;
+
+;[myprofile]
+;type = profile
+;behavior = verify
+;failure_action = continue_return_reason
+;acllist = myacllist
+
+;In pjsip.conf...
+;[myendpoint]
+;type = endpoint
+;stir_shaken_profile = myprofile
+
+;In acl.conf...
+;[myacllist]
+;permit=0.0.0.0/0.0.0.0
+;deny=10.24.20.171
+

include/asterisk/astdb.h

@@ -50,6 +50,16 @@ int ast_db_get(const char *family, const char *key, char *value, int valuelen);
  */
 int ast_db_get_allocated(const char *family, const char *key, char **out);
 
+/*!
+ * \brief Check if family/key exitsts
+ *
+ * \param family
+ * \param key
+ * \retval 1 if family/key exists
+ * \retval 0 if family/key does not exist or an error occurred
+ */
+int ast_db_exists(const char *family, const char *key);
+
 /*! \brief Store value addressed by family/key */
 int ast_db_put(const char *family, const char *key, const char *value);
 

include/asterisk/res_pjsip.h

@@ -975,6 +975,8 @@ struct ast_sip_endpoint {
 		AST_STRING_FIELD(accountcode);
 		/*! If set, we'll push incoming MWI NOTIFYs to stasis using this mailbox */
 		AST_STRING_FIELD(incoming_mwi_mailbox);
+		/*! STIR/SHAKEN profile to use */
+		AST_STRING_FIELD(stir_shaken_profile);
 	);
 	/*! Configuration for extensions */
 	struct ast_sip_endpoint_extensions extensions;
@@ -1044,6 +1046,8 @@ struct ast_sip_endpoint {
 	enum ast_sip_security_negotiation security_negotiation;
 	/*! Client security mechanisms (RFC 3329). */
 	struct ast_sip_security_mechanism_vector security_mechanisms;
+	/*! Set which STIR/SHAKEN behaviors we want on this endpoint */
+	unsigned int stir_shaken;
 	/*! Should we authenticate OPTIONS requests per RFC 3261? */
 	unsigned int allow_unauthenticated_options;
 	/*! The name of the geoloc profile to apply when Asterisk receives a call from this endpoint */