Remove references to postgresql-evr #3167

ianballou · 2024-07-25T20:11:56Z

With the merging of Katello/katello#11087 , the evr database type is now being added to postgres directly in Katello. This means users no longer need to install the RPM. So, for Katello 4.14, it will be safe to remove this reference.

Q: do we still need postgresql-contrib? I think we may need it for Pulpcore...

I am okay with my commits getting squashed when you merge this PR.
I am familiar with the contributing guidelines.

Please cherry-pick my commits into:

Foreman 3.11/Katello 4.13
Foreman 3.10/Katello 4.12
Foreman 3.9/Katello 4.11 (Satellite 6.15; orcharhino 6.8/6.9)
Foreman 3.8/Katello 4.10
Foreman 3.7/Katello 4.9 (Satellite 6.14)
Foreman 3.6/Katello 4.8
Foreman 3.5/Katello 4.7 (Satellite 6.13; orcharhino 6.6/6.7)
We do not accept PRs for Foreman older than 3.5.

github-actions · 2024-07-25T20:14:33Z

The PR preview for ef0b160 is available at theforeman-foreman-documentation-preview-pr-3167.surge.sh

The following output files are affected by this PR:

show diff

show diff as HTML

guides/common/modules/proc_installing-postgresql.adoc

ekohl

AFAIK this was the only reason why we installed the Satellite repositories and enabled the satellite module on the external DB. Please simply it so it only uses RHEL base OS & AppStream.

I'd also like to see it in the release notes as a headline feature, because it enables managed databases (like in cloud environments) to be used.

ianballou · 2024-07-26T17:05:30Z

AFAIK this was the only reason why we installed the Satellite repositories and enabled the satellite module on the external DB. Please simply it so it only uses RHEL base OS & AppStream.

Which part of the docs does this live in?

I'm not seeing any references to Satellite at least in https://docs.theforeman.org/nightly/Installing_Server/index-katello.html#using-external-databases_foreman, which seems to contain all of the external DB docs.

ekohl · 2024-07-30T20:43:56Z

Which part of the docs does this live in?

https://github.com/ekohl/foreman-documentation/blob/move-external-db-params/guides/common/modules/proc_preparing-a-host-for-external-databases.adoc which I think can pretty much go entirely.

I'm not seeing any references to Satellite at least in https://docs.theforeman.org/nightly/Installing_Server/index-katello.html#using-external-databases_foreman, which seems to contain all of the external DB docs.

Downstream: https://docs.theforeman.org/nightly/Installing_Server/index-satellite.html#preparing-a-host-for-external-databases_satellite

Upstream: https://docs.theforeman.org/nightly/Installing_Server/index-katello.html#preparing-a-host-for-external-databases_foreman

ianballou · 2024-07-30T21:58:03Z

Downstream: https://docs.theforeman.org/nightly/Installing_Server/index-satellite.html#preparing-a-host-for-external-databases_satellite

I see, I thought you meant we advised users to install Satellite repositories on normal Foreman smart proxies.

ianballou · 2024-07-31T15:47:12Z

@ekohl if I'm not wrong, I think that means we can also stop having users install the Katello repositories as well on external databases. I've removed references to installing our repos from both the Katello and Satellite docs.
Unless

guides/common/modules/proc_preparing-a-host-for-external-databases.adoc

guides/doc-Release_Notes/topics/katello.adoc

ianballou · 2024-08-05T19:05:28Z

This is what the entire section looks like now for upstream external DBs:

There is only a "prerequisites" section with no procedure. Looking for comments on how to deal with this following normal foreman-documentation conventions.

aneta-petrova · 2024-08-06T13:14:13Z

There is only a "prerequisites" section with no procedure. Looking for comments on how to deal with this following normal foreman-documentation conventions.

Conceptually speaking, it’s strange that a section named "Preparing" includes prerequisites. Because "Preparing" itself is a prerequisite by definition.

If you think the two bullets should indeed be considered prerequisites applicable to all the modules in 4.10. Using external databases with Foreman, you can split them off into a separate module and include it at the beginning of assembly_using-external-databases.adoc. See Prerequisites for bare-metal provisioning (source here) for an example of prereqs included at the assembly level.

If you feel the two bullets are actually procedure steps, you can merge them with .Procedure:

.Procedure
. Ensure the prepared host meets [...]
. Ensure the host has [...]
. Select the operating system [...]

Then, even if you hide the last step for upstream builds, the rest of the module will still make sense for these builds.

ekohl · 2024-08-06T15:40:37Z

guides/common/modules/proc_preparing-a-host-for-external-databases.adoc

I also think you can drop lines 6-9 because now a base RHEL subscription gives you access to the right repositories. Though with service level agreement I'm a bit uncertain.

Do we still need procedure? After removing the specific Satellite repositories, isn't it sufficient to say you need a RHEL 8 or 9 system with an attached subscription? Essentially, drop the whole chapter Preparing a host for external databases and make them prerequisites in the Installing PostgreSQL procedure

@asteflova @apinnick what do you think?

I've ripped out so much of this page that I think I could just include this info as prerequisites.

Sounds good to me from the docs perspective. Looking at the procedure steps, it looks like you could save a lot of space by shrinking it into a prerequisite bullet without losing much value. More importantly (to me), anything that's a prerequisite should be labeled as "prerequisite", not "preparing" or anything like that. We are supposed to apply the same wording and terminology consistently (this helps make the content predictable).

ekohl · 2024-08-06T15:46:23Z

guides/doc-Release_Notes/topics/katello.adoc

+=== postgresql-evr extension no longer required
+
+Installation of the Katello database on remote systems where root access is not available is now possible.
+Only a basic PostgreSQL installation is required, which should enable installs on systems like Amazon RDS or Azure Database for PostgreSQL.


This is out of scope, but we have a guide https://docs.theforeman.org/nightly/Deploying_Project_on_AWS/index-katello.html and using Amazon RDS would be a good thing to document, after it's been verified.

@maximiliankolb Just FYI, this might be useful to know with regards to your plans for the AWS guide.

ianballou · 2024-08-06T19:05:18Z

It's looking likely that I'll need to also add information for users upgrading their remote databases due to Katello/katello#11098.

Looks like https://docs.theforeman.org/nightly/Upgrading_Project/index-katello.html#Upgrading_the_External_Database_upgrading-connected is what will need updating.

ekohl · 2024-08-07T15:38:05Z

Q: do we still need postgresql-contrib? I think we may need it for Pulpcore...

We do. I forgot which extension they use, but it's shipped in postgresql-contrib.

guides/common/modules/proc_upgrading-the-external-database.adoc

ekohl · 2024-12-17T17:10:21Z

guides/common/modules/proc_installing-postgresql.adoc

+ifdef::orcharhino[]
+* The prepared host has the same content available as your {ProjectServer}.
+endif::[]


I think this is no longer true. Previously it was needed for postgresql-evr but now base OS repos should suffice.

Suggested change

ifdef::orcharhino[]

* The prepared host has the same content available as your {ProjectServer}.

endif::[]

The information is wrapped in an orcharhino check as it was on the old page -- @maximiliankolb does this need to stay for some orcharhino-specific reason?

Yes, Ewoud's suggestion is correct because external DBs no longer need packages from Foreman/Katello.

ekohl · 2024-12-17T17:14:42Z

guides/doc-Release_Notes/topics/katello.adoc

+
+Installation of the Katello database on remote systems where root access is not available is now possible.
+Only a basic PostgreSQL installation is required.
+With this feature, you can now install on systems like Amazon RDS or Azure Database for PostgreSQL.


Note to self: since Pulp 3.22 uses the HStore extension (which is why you need postgresql-contrib). It's on both https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html and https://learn.microsoft.com/en-us/azure/postgresql/extensions/concepts-extensions-versions#hstore so I'd expect it to indeed work.

maximiliankolb

Made some suggestions on your PR.

With the following commit, I got the build to work:

commit 63f7aaa499a8f65a7ce5e32592e3443b982e63a3 (HEAD -> pr_3167_ian_external_databases)
Author: Maximilian Kolb <kolb@atix.de>
Date:   Wed Dec 18 12:02:43 2024 +0100

    PATCH

diff --git a/guides/common/modules/proc_upgrading-the-external-database.adoc b/guides/common/modules/proc_upgrading-the-external-database.adoc
index 9f578a393e..d258060d23 100644
--- a/guides/common/modules/proc_upgrading-the-external-database.adoc
+++ b/guides/common/modules/proc_upgrading-the-external-database.adoc
@@ -15,8 +15,8 @@ If your {Project} deployment uses an external database, the database will be upg
 "psql -d foreman -c \"UPDATE pg_extension SET extowner = (SELECT oid FROM pg_authid WHERE rolname='foreman') WHERE extname='evr';\""
 ----
 ifdef::upgrading-connected[]
- . Follow xref:upgrading_a_connected_{project-context}_server_{context}[].
- endif::[]
- ifdef::upgrading-disconnected[]
- . Follow xref:upgrading_a_disconnected_{project-context}_server_{context}[].
- endif::[]
+. Follow xref:upgrading_a_connected_{project-context}_server_{context}[].
+endif::[]
+ifdef::upgrading-disconnected[]
+. Follow xref:upgrading_a_disconnected_{project-context}_server_{context}[].
+endif::[]

maximiliankolb · 2024-12-18T10:52:34Z

guides/common/modules/proc_upgrading-the-external-database.adoc

@@ -1,25 +1,22 @@
 [id="Upgrading_the_External_Database_{context}"]
 = Upgrading the external database

-You can upgrade an external database from {EL} 8 to {EL} 9 while upgrading {Project} from {ProjectVersionPrevious} to {ProjectVersion}.
+If your {Project} deployment uses an external database, the database will be upgraded when running {foreman-installer} on the {Project} server.


Suggested change

If your {Project} deployment uses an external database, the database will be upgraded when running {foreman-installer} on the {Project} server.

If your {ProjectServer} uses an external database, the database will be upgraded when running `{foreman-installer}` on your {ProjectServer}.

stupid question: can you also use an external DB for a Smart Proxy Server?

I don't think we support external DBs for smart proxies, but it is technically possible. I don't see docs in the proxy installation guide for external DBs, at least.

guides/common/modules/proc_upgrading-the-external-database-operating-system.adoc

maximiliankolb · 2024-12-18T10:55:32Z

guides/common/modules/proc_upgrading-the-external-database-operating-system.adoc

+[id="Upgrading_the_External_Database_Operating_System{context}"]
+= Upgrading the external database operating system
+
+If your {Project} uses an external database, you can upgrade the database from {EL} 8 to {EL} 9 while upgrading {Project} from {ProjectVersionPrevious} to {ProjectVersion}.


Is that still true for Foreman/Katello nightly? I am asking because it does not run on EL8 anymore. Latest release is 3.12.

The external DB has more flexible requirements than Foreman since it just needs to be a running postgres server, so I think it's still okay that users would have EL8 machines running their DBs for some time. @ekohl can probably keep me honest here though.

As @ianballou has indicated, it sort of has its own lifecycle. Technically I think we should loosen the requirement to any PostgreSQL 13+ compatible server instead of some (RH)EL server running PostgreSQL.

Within the platform team we've had some discussions on the level of support for the external DB server. Now that it's just a plain PostgreSQL server we can consider not having any instructions on setting up the PostgreSQL server. For example, we don't document how you do backups, partition, etc. Where do we draw the line?

I can see our external DB documentation being the recommendation that we can provide the best support for. Perhaps we should be strict with what the docs recommend but ensure users know that other solutions may be possible, just with limited support.

guides/common/modules/proc_upgrading-the-external-database-operating-system.adoc

ianballou · 2024-12-18T16:44:19Z

@maximiliankolb I believe I've addressed your comments. I addressed @ekohl 's earlier one too.

maximiliankolb

Two tiny suggestions, rest LGTM.

guides/common/modules/proc_upgrading-the-external-database.adoc

…command

ianballou · 2025-01-08T17:06:36Z

@maximiliankolb I've added your last suggestions in.

@ekohl you've got a change request on here.

maximiliankolb

Thanks Ian, LGTM!

@ianballou Do you want us to cherry-pick this to 3.13 and 3.12? Your PR states that it's part of Katello 4.14.

ianballou · 2025-01-13T19:20:38Z

Thanks all, no need to do any cherry-picks, that comment is old. Merging since I believe I've addressed the other requested changes.

aneta-petrova · 2025-01-14T07:19:41Z

Thanks @ianballou!

Otherwise DB restore using foreman-maintain will fail. Inspired by theforeman#3167

Otherwise DB restore using foreman-maintain will fail. Inspired by #3167 (cherry picked from commit 3ef26b5)

Otherwise DB restore using foreman-maintain will fail. Inspired by #3167

pr-processor bot added the Not yet reviewed label Jul 25, 2024

ianballou mentioned this pull request Jul 25, 2024

Fixes #37680 - Stop installing postgresql_evr extension theforeman/puppet-katello#502

Merged

ekohl reviewed Jul 25, 2024

View reviewed changes

guides/common/modules/proc_installing-postgresql.adoc Show resolved Hide resolved

ekohl requested changes Jul 25, 2024

View reviewed changes

pr-processor bot added Waiting on contributor Requires an action from the author and removed Not yet reviewed labels Jul 25, 2024

ianballou force-pushed the remove-postgresql-evr-refs branch from 9969c52 to 3a439ca Compare July 26, 2024 17:04

pr-processor bot added Needs re-review and removed Waiting on contributor Requires an action from the author labels Jul 26, 2024

ianballou requested a review from ekohl July 30, 2024 20:30

ekohl requested changes Jul 31, 2024

View reviewed changes

pr-processor bot added Waiting on contributor Requires an action from the author Needs re-review and removed Needs re-review Waiting on contributor Requires an action from the author labels Jul 31, 2024

ianballou force-pushed the remove-postgresql-evr-refs branch from 2d172de to 50b7976 Compare August 5, 2024 18:51

ianballou force-pushed the remove-postgresql-evr-refs branch from 50b7976 to fd58d72 Compare August 5, 2024 19:07

ekohl reviewed Aug 6, 2024

View reviewed changes

pr-processor bot removed the Needs re-review label Aug 6, 2024

ekohl reviewed Aug 6, 2024

View reviewed changes

ianballou mentioned this pull request Aug 7, 2024

Fixes #37717 - Update evr extension ownership to foreman theforeman/foreman-installer#955

Merged

ianballou force-pushed the remove-postgresql-evr-refs branch from 323cbdc to b6e08ea Compare December 13, 2024 21:40

ianballou added 4 commits December 13, 2024 16:41

Remove references to postgresql-evr

841ca5b

Remove references to Foreman repositories for external DBs

e8654e8

Add release note for new remote DB possibilities

a09f1f8

Remove excess parts of external DB host setup

4c7a0e5

ianballou force-pushed the remove-postgresql-evr-refs branch from b6e08ea to 478958a Compare December 13, 2024 21:48

ianballou commented Dec 13, 2024

View reviewed changes

guides/common/modules/proc_upgrading-the-external-database.adoc Show resolved Hide resolved

ekohl reviewed Dec 17, 2024

View reviewed changes

maximiliankolb requested changes Dec 18, 2024

View reviewed changes

pr-processor bot added the Waiting on contributor Requires an action from the author label Dec 18, 2024

ianballou force-pushed the remove-postgresql-evr-refs branch from 478958a to 8eca831 Compare December 18, 2024 16:43

pr-processor bot added Needs re-review and removed Waiting on contributor Requires an action from the author labels Dec 18, 2024

pr-processor bot removed the Needs re-review label Dec 18, 2024

maximiliankolb approved these changes Dec 20, 2024

View reviewed changes

guides/common/modules/proc_upgrading-the-external-database.adoc Outdated Show resolved Hide resolved

guides/common/modules/proc_upgrading-the-external-database.adoc Outdated Show resolved Hide resolved

Remove prepping a host for ext DBs guide + introduce evr permissions …

ef0b160

…command

ianballou force-pushed the remove-postgresql-evr-refs branch from 8eca831 to ef0b160 Compare January 8, 2025 17:03

maximiliankolb approved these changes Jan 9, 2025

View reviewed changes

aneta-petrova approved these changes Jan 10, 2025

View reviewed changes

ianballou merged commit cc8800a into theforeman:master Jan 13, 2025
9 checks passed

ianballou deleted the remove-postgresql-evr-refs branch January 13, 2025 19:20

evgeni mentioned this pull request Mar 7, 2025

Document to fix EVR permissions after PostgreSQL 13 upgrade #3716

Merged

10 tasks

evgeni added a commit to evgeni/foreman-documentation that referenced this pull request Mar 11, 2025

Document to fix EVR permissions after PostgreSQL 13 upgrade

d26a20a

Otherwise DB restore using foreman-maintain will fail. Inspired by theforeman#3167

aneta-petrova pushed a commit that referenced this pull request Mar 11, 2025

Document to fix EVR permissions after PostgreSQL 13 upgrade (#3716)

be42a08

Otherwise DB restore using foreman-maintain will fail. Inspired by #3167 (cherry picked from commit 3ef26b5)

aneta-petrova pushed a commit that referenced this pull request Mar 11, 2025

Document to fix EVR permissions after PostgreSQL 13 upgrade (#3716)

c30c269

Otherwise DB restore using foreman-maintain will fail. Inspired by #3167 (cherry picked from commit 3ef26b5)

aneta-petrova pushed a commit that referenced this pull request Mar 11, 2025

Document to fix EVR permissions after PostgreSQL 13 upgrade (#3716)

3ef26b5

Otherwise DB restore using foreman-maintain will fail. Inspired by #3167

	ifdef::orcharhino[]
	* The prepared host has the same content available as your {ProjectServer}.
	endif::[]

	If your {Project} deployment uses an external database, the database will be upgraded when running {foreman-installer} on the {Project} server.
	If your {ProjectServer} uses an external database, the database will be upgraded when running `{foreman-installer}` on your {ProjectServer}.

Remove references to postgresql-evr #3167

Remove references to postgresql-evr #3167

Uh oh!

Conversation

ianballou commented Jul 25, 2024

Uh oh!

github-actions bot commented Jul 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ekohl left a comment

Choose a reason for hiding this comment

Uh oh!

ianballou commented Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekohl commented Jul 30, 2024

Uh oh!

ianballou commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianballou commented Jul 31, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ianballou commented Aug 5, 2024

Uh oh!

aneta-petrova commented Aug 6, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ianballou commented Aug 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekohl commented Aug 7, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maximiliankolb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ianballou commented Dec 18, 2024

Uh oh!

maximiliankolb left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 25, 2024 •

edited

Loading

ianballou commented Jul 26, 2024 •

edited

Loading

ianballou commented Jul 30, 2024 •

edited

Loading

ianballou commented Aug 6, 2024 •

edited

Loading

maximiliankolb left a comment •

edited

Loading