Refreshing auth tokens for SignalR

[analogrelay]

Below is one option we've considered, but I'm re-framing this issue to build some story for refreshing auth tokens.

To improve the ability to "refresh" expired tokens, we should consider caching the access token provided by the factory. Then, when an HTTP request gets a 401, we call the factory again before re-issuing the request. That way the user can configure a process to "refresh" the token without forcing the connection to be reestablished

• For the WebSockets transport, this has no effect. There is only ever a single request. This logic would not cover reconnecting in the event of something like An unhandled exception was thrown by the application. #1159 (where the WebSocket is terminated when the token expires)
• For the SSE transport, this only affects POST (send) requests. We would call the token factory again and re-issue the send. The unsent data would stay buffered in the pipe
• For Long Polling, this affects POST requests like SSE, and the GET (poll) requests. The client would assume that a 401 error indicates that the data is still in the pipe for them to read. The server would be expected to keep data in the pipe in the case of a 401

[analogrelay]

We need to make sure that if two requests are outstanding simultaneously, the access token factory is only called once. So we should use a shared component and lock properly

[JamesNK]

Also need to make sure any new new requests that need to get sent wait while the access token is refreshed.

The same behavior should happen while starting a connection. Requests that need to get sent with an access token should wait while a single call to the factory is made, and then get sent out once the token is ready.

I'm assuming the behavior between C# and TS should be the same.

[analogrelay]

@SteveSandersonMS @rynowak FYI, just something that popped up in our backlog grooming and @BrennanConroy mentioned this was something you cared about. So we put our special happy label on it.

[analogrelay]

Expanding this to also cover the possibility of in-band refresh of the token. We may want to build a way to refresh the user principal without terminating the connection.

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[BrennanConroy]

We should author a doc about handling auth expiration correctly. See #5283 (comment) for a code example.

@bradygaster asked me to add @IEvangelist

[gregoriusus]

Any news on this topic?

[Misiu]

So maybe .NET 10?

[davidfowl]

So maybe .NET 10?

At a minimum

[Skyppid]

Wow that's crazy... too bad there's not much good alternatives to SignalR as of now...

[reduckted]

At a minimum

The earliest we'll see it is in the next version? How about a maximum? 😆

[DeepWorksStudios]

At a minimum

The earliest we'll see it is in the next version? How about a maximum? 😆

It's sadly the fact that Microsoft does not invest enough in the development department in comparison to how many technologies they are invented and maintaining.

Best course is to build the solution yourself or even better try to avoid such incompleted frameworks.

[thirstyape]

6 years and 9 months and not even close to any real progress, just a bunch of classic MS bureaucracy.

Why is there not at least a manual way to refresh tokens or even just the stored value?

[Skyppid]

6 years and 9 months and not even close to any real progress, just a bunch of classic MS bureaucracy.

Why is there not at least a manual way to refresh tokens or even just the stored value?

From what I understand by using the accessTokenFactory you always fetch a fresh token when a request is being sended. To optimize it, you can cache it until expiry and request a new one only if expired.

    const hubConnectionBuilder = new HubConnectionBuilder().withUrl(this.SERVER_HUB_URL, {
      transport: HttpTransportType.None,
      logger: LogLevel.Debug,
      accessTokenFactory: () => this.identityService.getToken()
    });

[DeepWorksStudios]

6 years and 9 months and not even close to any real progress, just a bunch of classic MS bureaucracy.

Why is there not at least a manual way to refresh tokens or even just the stored value?

From what I understand by using the accessTokenFactory you always fetch a fresh token when a request is being sended. To optimize it, you can cache it until expiry and request a new one only if expired.

    const hubConnectionBuilder = new HubConnectionBuilder().withUrl(this.SERVER_HUB_URL, {
      transport: HttpTransportType.None,
      logger: LogLevel.Debug,
      accessTokenFactory: () => this.identityService.getToken()
    });

This does not solve the original issue. The issue is that a reconnect is required to revaildet the token if its expired and that requires a full reconnect and if a client is connected without any connection loss even the token is invalid he can still accces protected hub methods etc . And some other issues that'd come with the whole auth system of signalR just read the thread

[gregoriusus]

Interesting that this is not some major issue. No one is refreshing token until logout/login?

[Skyppid]

Is it? I remember in a previous company we used it and had the problem that after the expiry all requests failed. Back then we had to actually reconnect. But using the factory it requests a token on each request, so if the token is outdated it should still fail?

Or do you mean backend-side? Honestly, we haven't been using SignalR that much so far, as those features were just UX convenience and there were more pressing things to build. It works for us, but maybe I'm overlooking something here as I'm not that deep into it anymore.

Just wanted to hint that this helps at least with some common issues in low-frequency / non-critical scenarios.

[thirstyape]

Interesting that this is not some major issue. No one is refreshing token until logout/login?

By default this is pretty much the issue, the token request does not happen until an HTTP request goes out, which doesn't happen often as most communication is facilitated via WebSocket.

Maybe the thought process here is that since the connection remains open it doesn't require re-validation? Not sure I agree with this.

Just seems wrong when I'm adding methods on the hub for them to not check for a valid token on each request. And when an invalid token is found just abort the connection. My current workaround is two-fold: on the hub add a method that informs whether the token is valid; on the client check with the hub each request whether the token is valid and reconnect when it is not. Additionally, the hub will do a single retry if the request failed due to closure from the hub.

[gregoriusus]

Yes, we are solving now, on client side, to pool all request to signalr when we are refreshing token and then we disconnect and reconnect. But on high volume of calls to/from signalr this will pose a problem of messages being lost or invest on some kind of retry mechanism.

[thirstyape]

Yes, we are solving now, on client side, to pool all request to signalr when we are refreshing token and then we disconnect and reconnect. But on high volume of calls to/from signalr this will pose a problem of messages being lost or invest on some kind of retry mechanism.

Exactly!

Only dealing with maybe 100 low traffic connections at a time right now, but this is going to be a scaling nightmare.

[DeepWorksStudios]

Yes, we are solving now, on client side, to pool all request to signalr when we are refreshing token and then we disconnect and reconnect. But on high volume of calls to/from signalr this will pose a problem of messages being lost or invest on some kind of retry mechanism.

It's more about the actual backend issue that we can have the case an invalid token or a user that changed permissions on runtime can still access protected methods since the validation just happens on the initial handshake of the websocket connection afterwards if the connection persists than we got the issue with unauthorized users can still access protected resources. The only workaround is to attach to the signalR heartbeat and validate it yourself but on client side we got the reconnect issue it's a slight different issue but still all is kinda sharing the same issue that authentication is rather fragile in signalR

The workaround works but ain't optimal and causes scaling issues if the demand increases so yeah total conclusion is rather bad since it's a issue known for nearly 7 years

[DeepWorksStudios]

Yes, we are solving now, on client side, to pool all request to signalr when we are refreshing token and then we disconnect and reconnect. But on high volume of calls to/from signalr this will pose a problem of messages being lost or invest on some kind of retry mechanism.

Exactly!

Only dealing with maybe 100 low traffic connections at a time right now, but this is going to be a scaling nightmare.

Sadly we already experienced issues at our company in therms of scaling and there is no real solution sadly since it's a concept issue in signalR and can't be changed that easily that's why I wonder what did Microsoft all the 6 years. That would be sufficient time to fix such a major flaw

[thirstyape]

It's more about the actual backend issue that we can have the case an invalid token or a user that changed permissions on runtime can still access protected methods since the validation just happens on the initial handshake of the websocket connection afterwards if the connection persists than we got the issue with unauthorized users can still access protected resources. The only workaround is to attach to the signalR heartbeat and validate it yourself but on client side we got the reconnect issue it's a slight different issue but still all is kinda sharing the same issue that authentication is rather fragile in signalR

The workaround works but ain't optimal and causes scaling issues if the demand increases so yeah total conclusion is rather bad since it's a issue known for nearly 7 years

Wow! I hadn't even thought of token invalidation or permission changes, that's huge.

[gregoriusus]

Me niether regarding permission changes, this is additional issue :-(

At the time when they architect this, refresh token was not a big issue. We have one solution on old Signalr (NET framework) and we are using cookie based authentication. It works flawless. Anyone knows, if it is possible to mix and for signalr send cookie based authneticaton with some custom validation on server and for API calls token based?

[DeepWorksStudios]

It's more about the actual backend issue that we can have the case an invalid token or a user that changed permissions on runtime can still access protected methods since the validation just happens on the initial handshake of the websocket connection afterwards if the connection persists than we got the issue with unauthorized users can still access protected resources. The only workaround is to attach to the signalR heartbeat and validate it yourself but on client side we got the reconnect issue it's a slight different issue but still all is kinda sharing the same issue that authentication is rather fragile in signalR

The workaround works but ain't optimal and causes scaling issues if the demand increases so yeah total conclusion is rather bad since it's a issue known for nearly 7 years

Wow! I hadn't even thought of token invalidation or permission changes, that's huge.

It's a huge issue and no real solution wich would solve the root issue

I mean that's why nobody srs uses Microsoft tech because their them self don't use it in the extend their should to prove it's viable for production and sadly this case if you require high security that's a no no

It's a killer issue out of my perspective I wonder why nobody points that out as what it is

I mean I follow this issue since 4 years and still no progress at all only more people find out about the surface issue but only a small handful of people understands the overall impact on therms of auth security and I am no expert in that terms and still even I saw the issues that comes with this type of of implementation

[gregoriusus]

Do you think that token authentication for api calls and cookie based for signalr would help in this issue? Just a thought...

[thirstyape]

Do you think that token authentication for api calls and cookie based for signalr would help in this issue? Just a thought...

From what I can tell this would be subject to the same issue.

The documentation, and examples I googled, seem to suggest that the cookie would only be provided on the initial HTTP request, and subsequent requests over the WebSocket would be doing the same thing as token based requests.

[channeladam]

It has been a while since I have looked at the SignalR code in my project, but I think I mitigate the many SignalR issues/concerns with the following approach:

1. My SignalR access token is a very short-lived token just for the hubs - no other endpoints will accept them - they are different from my normal api endpoint access tokens.

2. I never send data to the hub via the SignalR connection! When calling the backend, I always use a normal api endpoint with the normal session security mechanism.

3. As you can guess, data is only sent from the backend to the frontend, for the specified user guid that was part of the original access token when initially connecting to SignalR. The backend can check the current role of the user (e.g. against the database) before sending the data if necessary or if concerned about a role change.
   I believe that makes it okay for the SignalR connection to remain open as long as the user is on that web page.
   If the user's web session times out, they are redirected to the login page, so their connection is closed from the frontend at that point.

4. Due to the javascript library having issues reconnecting server-sent events or long-polling (more specifically not bothering to properly inspect the response status code), I use a custom HttpClient that extends DefaultHttpClient and overrides send() in order to catch any error and and return a specific HttpResponse to address those issues to allow them to reconnect.

Having said all that, if I recall correctly (please be kind - I could be wrong!), the backend Hub Context keeps the initial connection request details - so the user's guid that was in the SignalR access token is always accessible via a claim via the Hub.Context.GetHttpContext(). I believe, if I wanted to, I could add other custom claims such as a session expiry timestamp if I wanted to check that later in a hub method, and then call Abort() if it had expired - or something similar if there was a role change and it had to be invalidated. Could be wrong!
Either way, given my approach, I don't need that.

Hope this helps someone.

[DeepWorksStudios]

It has been a while since I have looked at the SignalR code in my project, but I think I mitigate the many SignalR issues/concerns with the following approach:

1. My SignalR access token is a very short-lived token just for the hubs - no other endpoints will accept them - they are different from my normal api endpoint access tokens.

2. I never send data to the hub via the SignalR connection! When calling the backend, I always use a normal api endpoint with the normal session security mechanism.

3. As you can guess, data is only sent from the backend to the frontend, for the specified user guid that was part of the original access token when initially connecting to SignalR. The backend can check the current role of the user (e.g. against the database) before sending the data if necessary or if concerned about a role change.
   I believe that makes it okay for the SignalR connection to remain open as long as the user is on that web page.
   If the user's web session times out, they are redirected to the login page, so their connection is closed from the frontend at that point.

4. Due to the javascript library having issues reconnecting server-sent events or long-polling (more specifically not bothering to properly inspect the response status code), I use a custom HttpClient that extends DefaultHttpClient and overrides send() in order to catch any error and and return a specific HttpResponse to address those issues to allow them to reconnect.

Having said all that, if I recall correctly (please be kind - I could be wrong!), the backend Hub Context keeps the initial connection request details - so the user's guid that was in the SignalR access token is always accessible via a claim via the Hub.Context.GetHttpContext(). I believe, if I wanted to, I could add other custom claims such as a session expiry timestamp if I wanted to check that later in a hub method, and then call Abort() if it had expired - or something similar if there was a role change and it had to be invalidated. Could be wrong!
Either way, given my approach, I don't need that.

Hope this helps someone.

Sadly it won't work for my use case but I see how it can minimize the issues for many use cases but still they are just a workaround out of my perspective

[Gruski]

I need this a few years ago too but since there was nothing being done I ended up changing my auth from token based to where info was actually stored in token, to database based so now token never needs to be updated because I can just update the auth info of the static client token on the server side in DB user's record. That also helped with heterogeneous systems that could not share auth which I also needed and sliding expiration which I believe none of the other workarounds address. Then use filters in the hub to auth against DB on every SignalR request.

[DeepWorksStudios]

https://www.youtube.com/watch?v=kAGlATRcgY4
Security & Auth .NET 10 planning discussion

Thats currently live on going maybe it will tackel this issue in .net 10
At least i hope so

[DeepWorksStudios]

They mentioned that it will be worked on this issue in .Net 10

Gladly 🎉