RBAC denies upload unless chunk size is specified

[grzleadams]

Version
Deployed on K8s via Operator

{                                                                                                        
  "versions": [              
    {                          
      "component": "core",     
      "version": "3.49.1",   
      "package": "pulpcore",                     
      "module": "pulpcore.app",                                                                          
      "domain_compatible": true
    },                        
    {                                                                                                    
      "component": "ansible",                                                                            
      "version": "0.21.3",                                                                               
      "package": "pulp-ansible",                                                                         
      "module": "pulp_ansible.app",                                                                      
      "domain_compatible": false                                                                         
    },                      
    {                           
      "component": "container",
      "version": "2.19.2",
      "package": "pulp-container",
      "module": "pulp_container.app",
      "domain_compatible": false
    },
    {
      "component": "deb",
      "version": "3.2.0",
      "package": "pulp_deb",
      "module": "pulp_deb.app",
      "domain_compatible": false
    },
    {
      "component": "maven",
      "version": "0.8.0",
      "package": "pulp-maven",
      "module": "pulp_maven.app",
      "domain_compatible": false
    },
    {
      "component": "ostree",
      "version": "2.3.0",
      "package": "pulp-ostree",
      "module": "pulp_ostree.app",
      "domain_compatible": true
    },
    {
      "component": "python",
      "version": "3.11.0",
      "package": "pulp-python",
      "module": "pulp_python.app",
      "domain_compatible": false
    },
    {
      "component": "rpm",
      "version": "3.25.1",
      "package": "pulp-rpm",
      "module": "pulp_rpm.app",
      "domain_compatible": true
    },
    {
      "component": "certguard",
      "version": "3.49.1",
      "package": "pulpcore",
      "module": "pulp_certguard.app",
      "domain_compatible": true
    },
    {
      "component": "file",
      "version": "3.49.1",
      "package": "pulpcore",
      "module": "pulp_file.app",
      "domain_compatible": true
    }
  ],

Describe the bug
While uploading some files (I haven't been able to exactly pin down what they have in common yet) as a non-admin user, we get Error: {"detail":"You do not have permission to perform this action."} while doing a pulp file content upload despite permissions looking fine across the board. Specifying a sufficiently high chunk size is the only thing that seems to resolve it. For example:

~$ wget https://github.com/mstorsjo/llvm-mingw/releases/download/20231128/llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz
~$ pulp --config /tmp/config.toml file content upload --file llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz --relative-path llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz --repository file-local --chunk-size 10000000
Error: {"detail":"You do not have permission to perform this action."}
~$ stat --printf="%n,%s" llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz 
llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz,72800008
~$ pulp --config /tmp/config.toml file content upload --file llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz --relative-path llvm-mingw-20231128-ucrt-ubuntu-20.04-x86_64.tar.xz --repository file-local --chunk-size 100000000
Started background task /pulp/api/v3/tasks/018e860b-bde4-7a59-bacd-710eec7c94bd/
Done.
<snip>

Interestingly, I don't see this behavior when using the admin user, only a user that we created, which makes me think this is some permission I missed when creating the user, but I have no idea what it would be.

To Reproduce

1. Create a file repository and distribution (with autopublish).
2. Create a user with the following roles for the repository and distribution: file.filedistribution_creator, file.filedistribution_owner, file.filerepository_creator, file.filerepository_owner.
3. Download a large file of an affected type (whatever that is... .tar.xz seems to trigger it) and try to upload without --chunk-size set.
4. Upload should fail with permissions error.

Expected behavior
Upload should happen without an error.

Additional context
N/A

[mdellweg]

I suspect, the user is missing permissions to create upload. So can you verify that adding the upload_creator role would solve the problem?
In the long run, we should probably include the upload_create permission in all the content upload roles.

[grzleadams]

Adding core.upload_creator role to the user (even with --object "") without --chunk-size didn't resolve the issue. I was in a bit of a time crunch so I just gave file.filedistribution_owner, file.filerepository_owner, and core.upload_owner globally to the user, and it started working, so I'm not sure which of the three was the one that mattered. I assume it was the third one but I'll confirm tomorrow.

[grzleadams]

It's probably worth mentioning that the path in question wasn't in the top level of the repository but in a directory (i.e., coverage/myfile.txt) that already existed and contained files. I assumed the permissions would apply to all subpaths within the repository but is that not the case or something? I did see a similar issue here, which feels very similar, although it's obviously in pulp_container instead.

[mdellweg]

It's probably worth mentioning that the path in question wasn't in the top level of the repository but in a directory (i.e., coverage/myfile.txt) that already existed and contained files.

Permissions are scoped to the repository. So that should not matter.

Thanks for investigating. I'll have a look myself.

[mdellweg]

With the file.filerepository_creator and the core.upload_creator` roles, i get the following result:

pulp -vv -p user1 file content upload --file Makefile --relative-path Makefile --repository user1_test --chunk-size 10
uploads_create : post http://localhost:5001/pulp/api/v3/uploads/
  User-Agent: Pulp-CLI/0.25.0.dev
  Accept-Encoding: gzip, deflate
  Accept: application/json
  Connection: keep-alive
  Content-Length: 13
  Content-Type: application/json
  Authorization: Basic dXNlcjE6SXRvYUI4Y2g=
Response: 201
  Server: nginx/1.22.1
  Date: Thu, 11 Apr 2024 10:23:40 GMT
  Content-Type: application/json
  Content-Length: 180
  Connection: keep-alive
  Location: /pulp/api/v3/uploads/018eccaf-3c4f-7a10-bbdd-4fd1101427bf/
  Vary: Accept
  Allow: GET, POST, HEAD
  X-Frame-Options: DENY
  X-Content-Type-Options: nosniff
  Referrer-Policy: same-origin
  Cross-Origin-Opener-Policy: same-origin
  Correlation-ID: c4d2bced1fb24cceb2a790cc4ed78146
  Access-Control-Expose-Headers: Correlation-ID
uploads_read : get http://localhost:5001/pulp/api/v3/uploads/018eccaf-3c4f-7a10-bbdd-4fd1101427bf/
  User-Agent: Pulp-CLI/0.25.0.dev
  Accept-Encoding: gzip, deflate
  Accept: application/json
  Connection: keep-alive
  Correlation-ID: c4d2bced1fb24cceb2a790cc4ed78146
  Authorization: Basic dXNlcjE6SXRvYUI4Y2g=
Response: 403
  Server: nginx/1.22.1
  Date: Thu, 11 Apr 2024 10:23:40 GMT
  Content-Type: application/json
  Content-Length: 63
  Connection: keep-alive
  Vary: Accept
  Allow: GET, HEAD, PUT, DELETE
  X-Frame-Options: DENY
  X-Content-Type-Options: nosniff
  Referrer-Policy: same-origin
  Cross-Origin-Opener-Policy: same-origin
  Correlation-ID: c4d2bced1fb24cceb2a790cc4ed78146
  Access-Control-Expose-Headers: Correlation-ID
uploads_delete : delete http://localhost:5001/pulp/api/v3/uploads/018eccaf-3c4f-7a10-bbdd-4fd1101427bf/
  User-Agent: Pulp-CLI/0.25.0.dev
  Accept-Encoding: gzip, deflate
  Accept: application/json
  Connection: keep-alive
  Correlation-ID: c4d2bced1fb24cceb2a790cc4ed78146
  Content-Length: 0
  Authorization: Basic dXNlcjE6SXRvYUI4Y2g=
Response: 403
  Server: nginx/1.22.1
  Date: Thu, 11 Apr 2024 10:23:40 GMT
  Content-Type: application/json
  Content-Length: 63
  Connection: keep-alive
  Vary: Accept
  Allow: GET, HEAD, PUT, DELETE
  X-Frame-Options: DENY
  X-Content-Type-Options: nosniff
  Referrer-Policy: same-origin
  Cross-Origin-Opener-Policy: same-origin
  Correlation-ID: c4d2bced1fb24cceb2a790cc4ed78146
  Access-Control-Expose-Headers: Correlation-ID
Error: {"detail":"You do not have permission to perform this action."}

It looks like we are allowed to create the upload, but the user1 did not become its owner.

pulp user role-assignment list --username user1 also shows that the core.upload_owner role is missing for the user.