Recommendations for apps with a large # of user-controlled roles

[steven-fox]

The following is fictional story outlining a real world problem we faced. Your mileage may vary.

Also, no one's perfect, and there's a chance we missed something vital that would completely change our situation. Please feel free to let me know if it's possible to have 20,000+ roles with this package and things still work smoothly.

Use Case | App Requirement

Hey dev team, it's your business owner here. So, I think it would be cool if we let each team in our system create their own set of user roles that have a custom set of permissions based on the needs of their team. Let me know when this is done! Would love to see it done yesterday. :D

To which the dev team responds, "Ok boss. You got it. There's this permission package we can use that permits the creation and assignment of roles on a per-team basis. Easy peasy."

Design Strategy

Seems easy enough. Install this Spatie package, turn on the teams feature, and every time a new team model is created, we also create a set of default roles (each having a default set of permissions from our pool of controllable permissions) for the team, allowing them to modify the permissions associated with each team-role if needed. So we have something like:

• Our set of ~50 core permissions that are integral to the app and are not user-controlled (alarm.create, alarm.delete, notification.create, notification.delete, billing.manage, adCampaign.create, adCampaign.delete, apiKey.create, apiKey.delete, apiKeys.rotate, ...).
• Our set of 8 default roles (admin, developer, marketer, ...) that each have a default set of permissions from our available pool above.
• Each time a new team is created, we create new versions of our 8 default roles, assign the team_id accordingly, and relate the new roles with their respective permissions for some sane defaults.
• Teams are then allowed to add new roles, edit existing ones, alter which permissions are associated with each role, assign roles to their team members, etc.

This gets implemented and we're off the races. No sweat! Even a junior developer could handle this one!

The Deployment

After some testing on our dev environment, we push to production, seed the new roles/permissions tables with existing data for each team in our system. It was soooo easy.

That is until:

APP ALARM: P95 response times exceeding 1000ms in the past 10 mins. Avg response time: 24617ms.
APP ALARM: Excessive memory usage during request: 742MB.
APP ALARM: P95 response times exceeding 1000ms in the past 10 mins. Avg response time: 25012ms.
APP ALARM: Excessive memory usage during request: 743MB.
APP ALARM: P95 response times exceeding 1000ms in the past 10 mins. Avg response time: 24935ms.
...

To which the junior dev responds with 😳🤷 and "I'm heading to lunch! Good luck!"

The Oversight

After desperately attempting to figure out what was going on with our system, we realized the following:

This package is set up to cache permissions.roles and roles - for every record in your permissions, roles, and role_has_permissions tables.

Ha! Oops.

• We had 2,412 teams, each with a minimum of 8 roles, and each of those relating to X permissions. Our role_has_permissions table had over 250,000 records.

When the PermissionRegistrar sets/reads the cache, it was loading over a quarter million model instances into and out of cache. Big oops indeed!

P.S. I believe the actual checking for permissions could be fast by just using the database layer with these sorts of numbers. I think it's just the cache and the HasPermissions::hasPermissionViaRole() method that makes the package slow and unusable for such a business requirement. Here's my thinking, subject to being completely wrong haha: When checking if a person ->hasPermissionTo(' ... '), all that needs to happen is 1) look up the permission record by identifier (probably < 1ms at the db because of the unique name index, and you could even continue to do this part via a more limited cache implementation), 2) see if the person has a relation to that permission record (which uses the model_has_permissions table and performs an index-based lookup in < 1ms), and if necessary, 3) check if the user has a role with that permission (which would still be ultra fast at the database layer because all of this can be done via index lookups). Sure, this might lead to 1 extra db query, but it should be lightening fast. Using the cache the way this package does is great for a small number of roles/permissions, but can quickly get out of hand.

If enough people face a similar situation as us (needing/wanting a lot of roles due to a large number of teams), then I may consider working on a PR that will update the PermissionRegistrar's handling of cache and the HasPermissions::hasPermissionViaRole() method. I would need to fully test the performance on a large mocked application though.

Our (Temporary?) Solution

Instead of creating a ton of duplicate, default, roles for each team in our system, we now maintain the default set of 8 roles with a team_id of null, allowing us to assign them to any individual user. Then, only if a team actually needs a custom role (either because of a name preference or unique set of permissions), we then create a custom role and update the rest of the tables accordingly. Thus, instead of thousands upon thousands of roles, we have < 100. This solution isn't perfect, however. Once you do have a team with a custom role (so, a role record where team_id is not null), you have to be mindful of that and change your role lookups accordingly. It's not as simple as Role::query()->where('team_id', $this->id)->orWhereNull('team_id'), because you can get back multiple roles with the same name, which will look confusing to a frontend user. You must find any duplicate name/guard combinations and favor the version that has a non-null team_id. Not hard, just a little extra work.

The Key Takeaway

Be cautious when using this package, as of version 5.10.1 if you need a lot of roles, which you might think of doing if you want each team in your system to have their own set of roles. Instead, try our strategy of maintaining a global set of roles and only creating new roles if absolutely necessary for a particular team, or use an alternate permission strategy.

[parallels999]

I may consider working on a PR that will update the PermissionRegistrar's handling of cache and the HasPermissions::hasPermissionViaRole() method. I would need to fully test the performance on a large mocked application though.

It could also be optional, through the configuration file to maintain the cache functionality for small apps, and be able to change it if necessary

[steven-fox]

Thanks for the recommendation. I had similar thoughts. A simple configuration value that alters which strategy is used. Only downside is that is also changes how the HasRoles and HasPermissions traits are written. I’ll have to think about a way to best handle that aspect.