Restricted access

[vidstige]

To be able to use this sweet plugin for proprietary software, restricted access is needed. See this for example: https://debian-administration.org/article/513/Restrict_Access_To_Your_Private_Debian_Repository

Are the actual .deb files protected by normal TeamCity access? In that case I should just be able to use e.g. http://user:pass@teamcity.company.com/app/debrepo/

Please advice

[netwolfuk]

Currently, browsing the repo is not restricted in any way.

In TeamCity, when the webcontroller is registered at a certain URL, the authenticate flag is specified. Currently authentication is not required for that controller.

So in theory, I could sub-class the controller with an alternative URL and set the authenticate flag to true, then the controller would require authentication.

How this works in TeamCity is that accessing the URL would bounce you to the login page if there is no valid session. However, TeamCity also provides the /httpAuth/ prefix which does basic authentication instead of session authentication.

So:

1. I'd need a new controller registered at an alternative URL. Eg, /app/debrepo-restricted/*
2. that controller would also require authentication and would check that the user has read permission on the project that the repo is associated to.
3. Add a setting on a repo that indicates if authentication is required and then return 404 if accessed via the open URL.
4. Add UI to set this setting and update links to this repo to point to the relevant location.

Then the repo would be accessed at http://user:pass@teamcity.company.com/httpAuth/app/debrepo-restricted/MyRepoName/ or something like that.

We could come up with a better name than debrepo-restricted :-)

[netwolfuk]

The user:pass would be TeamCity username and password. The user would need read permission on the project that the repo is associated to (or a subproject).

[netwolfuk]

Should it return 404 or should it do a 301 redirect to the restricted URL?

[netwolfuk]

BTW. Thanks for the link. I was not aware a repository could be restricted.

[vidstige]

I guess 501 would be the least unexpected http code in that case. Your suggestion sounds awesome! I didn't quite get point 3 there, but this feature would be very helpful to us. Can we help out somehow? Testing? Coding? The name debrepo-restricted sounds very clear to me 👍

[netwolfuk]

I guess 501 would be the least unexpected http code in that case.

I don't agree with 501. The features are implemented, it's just that it is now in a different location. I think a redirect (301) and then 401 if auth required, 200 if OK or 404 if not found seems like the most logical to me. However, I will do some testing and see how APT handles it. Most of the time, APT is the client, so it should really be whatever is going to give the most useful response back to APT.

I didn't quite get point 3

The ability to toggle whether a repo is restricted. If it's restricted then the above logic will redirect the client to the relevant location.

Can we help out somehow? Testing? Coding?

I'll try to have a test build available in the next few days and you can let me know how it goes.

The name debrepo-restricted sounds very clear to me

OK, great.

[netwolfuk]

@vidstige I have a working build which implements all four of the items I listed above.

WRT the redirects, I have implemented the following for handling restricted repositories:

• Accessing the unrestricted URL /app/debrepo/Test01/ redirects (302) to /httpAuth/app/debrepo-restricted/Test01/
• Accessing /httpAuth/app/debrepo-restricted/Test01/ without credentials then returns a 401
• Accessing /httpAuth/app/debrepo-restricted/Test01/ with credentials returns 200.

The behaviour I have noticed from apt-get is that when it encounters the redirect (302) it does not resend the credentials after the redirect and therefore fails to install a package due to 401 Unauthorized.

So this redirecting is not much use for APT, but is still very useful when trying to obtain the correct URL by visiting in a browser. One can then simply copy the new URL into the sources.list file.

I will tidy up some code, and then push the branch to GitHub where it will build and I can make a test plugin ZIP file available for testing. It should be ready for you on Monday morning your time.

[vidstige]

This behavior makes lots of sense. That was very quick work. Thank you so much! 😊

[netwolfuk]

Note to @netwolfuk. Consider showing user:pass in the URL of the sources.list example.

[netwolfuk]

Test build is available here
Logon as guest to download the zip file.

[netwolfuk]

Hi @vidstige. Did you get a chance to test this?

[vidstige]

@netwolfuk sadly no. Our TeamCity expert @rasmusgo is skiing this week. But we're thrilled and will get to it asap. Thanks again for this feature, I'll let you know how it works out for us. If not, ping again! :)

[rasmusgo]

We have installed the plugin successfully and created a debian repository in the UI. However, sudo apt update complains about a missing Release file. We can login with basic http auth through the web browser and find a Packages file, but no Release file.

$ sudo apt update
Ign:1 https://build.volumental.com/httpAuth/app/debrepo-restricted/volumental xenial InRelease
Err:2 https://build.volumental.com/httpAuth/app/debrepo-restricted/volumental xenial Release
  404  Not Found
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]  
Hit:4 http://us.archive.ubuntu.com/ubuntu xenial InRelease                              
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Reading package lists... Done                                                                                                        
E: The repository 'https://build.volumental.com/httpAuth/app/debrepo-restricted/volumental xenial Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Do we need to enable insecure updates or is there a way for the plugin to generate this file?

[netwolfuk]

@rasmusgo Thanks for the feedback.

I am aware of this. I have tried a few things, but have not yet managed to get recent ubuntu versions to install from the plugin. See the original README from version 1.0.

I am making progress on fixing this. Please see issue #14 , which relies on #13 and #4.
Once those are solved, I may also need to do #3, unless there is a way to use a Release file, without having to have it signed. #3 will be interesting if I want try to implement a Java only solution. By that I mean, not going to the OS and executing gpg (which would require GPG on the server, which is also probably too hard if not on Linux).

[vidstige]

We missed that table with supported linux dists. Thanks for quick reply, we'll anticipate the 1.1 release then. Some quick googling reveals it seems tricky indeed with GPG signing. Some quick googling reveals it might be possible with some fiddling around using the bouncycastle library as described eg. here.

[netwolfuk]

Yes, I think I saw that post on SO the other night whilst searching. "Clear Signing" as per the Debian docs appears to be the keyword phrase to search for. Googling that reveals this example which might be promising (and is MIT licensed, yey).

It looks like I have some crypto to learn, after I get the Release files generated. If you're at all inclined, I'd happily accept a pull request or GIST of a code snippet that can clearsign a text string. The release file will be stored in the DB, so I need to take a chunk of text, clearsign it, and then persist it into the DB again.

My initial thoughts are that I need to generate the Package files and Release file after every build which produces a DEB file. I'm still working through that in my head.

Unless generating the Release file, and signing it is so lightweight I can do it on demand. As I say. Still mulling it over at the moment.