-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Service Connector] az spring-cloud connection create postgres
: Add --system-identity
for springcloud-postgres connection
#22459
Conversation
az spring-cloud connection create postgres --system-identity
: Add new auth type of springcloud-postgres connectionaz spring-cloud connection create postgres --system-identity
: Add new auth type of springcloud-postgres connection
az spring-cloud connection create postgres --system-identity
: Add new auth type of springcloud-postgres connection--system-identity
: Add new auth type of springcloud-postgres connection
Service Connector |
--system-identity
: Add new auth type of springcloud-postgres connectionaz spring-cloud connection create
: Add --system-identity
for springcloud-postgres connection
az spring-cloud connection create
: Add --system-identity
for springcloud-postgres connectionaz spring-cloud connection create postgres
: Add --system-identity
for springcloud-postgres connection
@kairu-ms Could you help to check the build failure? What should I do to fix it? |
/azp run |
Azure Pipelines successfully started running 2 pipeline(s). |
@jiasli Hi Jiashuo, could you please help to check the "Build Rpm Package CentOS 7" pipeline error? What should I do to fix it? |
@@ -139,3 +139,4 @@ vsts==0.1.25 | |||
wcwidth==0.1.7 | |||
websocket-client==1.3.1 | |||
xmltodict==0.12.0 | |||
psycopg2-binary==2.9.3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@xfz11 do you add it in setup.py file of azure-cli
azure-cli/src/azure-cli/setup.py
Lines 53 to 156 in 58a8f65
DEPENDENCIES = [ | |
'antlr4-python3-runtime~=4.7.2', | |
'azure-appconfiguration~=1.1.1', | |
'azure-batch~=12.0.0', | |
'azure-cli-core=={}'.format(VERSION), | |
'azure-cosmos~=3.0,>=3.0.2', | |
'azure-data-tables==12.2.0', | |
'azure-datalake-store~=0.0.49', | |
'azure-graphrbac~=0.60.0', | |
'azure-keyvault-administration==4.0.0b3', | |
'azure-keyvault-keys==4.5.1', | |
'azure-keyvault~=1.1.0', | |
'azure-loganalytics~=0.1.0', | |
'azure-mgmt-advisor==9.0.0', | |
'azure-mgmt-apimanagement~=3.0.0', | |
'azure-mgmt-appconfiguration==2.1.0b2', | |
'azure-mgmt-applicationinsights~=1.0.0', | |
'azure-mgmt-authorization~=0.61.0', | |
'azure-mgmt-batchai==7.0.0b1', | |
'azure-mgmt-batch~=16.1.0', | |
'azure-mgmt-billing==6.0.0', | |
'azure-mgmt-botservice~=0.3.0', | |
'azure-mgmt-cdn==12.0.0', | |
'azure-mgmt-cognitiveservices~=13.1.0', | |
'azure-mgmt-compute~=26.1.0', | |
'azure-mgmt-consumption~=2.0', | |
'azure-mgmt-containerinstance~=9.1.0', | |
'azure-mgmt-containerregistry==8.2.0', | |
'azure-mgmt-containerservice~=19.1.0', | |
'azure-mgmt-cosmosdb==7.0.0b2', | |
'azure-mgmt-databoxedge~=1.0.0', | |
'azure-mgmt-datalake-analytics~=0.2.1', | |
'azure-mgmt-datalake-store~=0.5.0', | |
'azure-mgmt-datamigration~=10.0.0', | |
'azure-mgmt-deploymentmanager~=0.2.0', | |
'azure-mgmt-devtestlabs~=4.0', | |
'azure-mgmt-dns~=8.0.0', | |
'azure-mgmt-eventgrid==9.0.0', | |
'azure-mgmt-eventhub~=10.0.0', | |
'azure-mgmt-extendedlocation==1.0.0b2', | |
'azure-mgmt-hdinsight~=9.0.0', | |
'azure-mgmt-imagebuilder~=1.0.0', | |
'azure-mgmt-iotcentral~=9.0.0', | |
'azure-mgmt-iothub==2.2.0', | |
'azure-mgmt-iothubprovisioningservices==1.1.0', | |
'azure-mgmt-keyvault==9.3.0', | |
'azure-mgmt-kusto~=0.3.0', | |
'azure-mgmt-loganalytics==13.0.0b4', | |
'azure-mgmt-managedservices~=1.0', | |
'azure-mgmt-managementgroups~=1.0.0', | |
'azure-mgmt-maps~=2.0.0', | |
'azure-mgmt-marketplaceordering==1.1.0', | |
'azure-mgmt-media~=9.0', | |
'azure-mgmt-monitor~=3.0.0', | |
'azure-mgmt-msi~=6.0.0', | |
'azure-mgmt-netapp~=7.0.0', | |
'azure-mgmt-network~=19.3.0', | |
'azure-mgmt-policyinsights~=1.1.0b2', | |
'azure-mgmt-privatedns~=1.0.0', | |
'azure-mgmt-rdbms~=10.0.0', | |
'azure-mgmt-recoveryservicesbackup~=4.1.1', | |
'azure-mgmt-recoveryservices~=2.0.0', | |
'azure-mgmt-redhatopenshift==1.1.0', | |
'azure-mgmt-redis~=13.1.0', | |
'azure-mgmt-relay~=0.1.0', | |
'azure-mgmt-reservations==2.0.0', # TODO: Use requirements.txt instead of '==' #9781 | |
'azure-mgmt-resource==21.1.0b1', | |
'azure-mgmt-search~=8.0', | |
'azure-mgmt-security==2.0.0b1', | |
'azure-mgmt-servicebus~=7.1.0', | |
'azure-mgmt-servicefabricmanagedclusters~=1.0.0', | |
'azure-mgmt-servicelinker==1.0.0', | |
'azure-mgmt-servicefabric~=1.0.0', | |
'azure-mgmt-signalr==1.0.0b2', | |
'azure-mgmt-sqlvirtualmachine==1.0.0b2', | |
'azure-mgmt-sql==4.0.0b1', | |
'azure-mgmt-storage~=20.0.0', | |
'azure-mgmt-synapse==2.1.0b2', | |
'azure-mgmt-trafficmanager~=1.0.0', | |
'azure-mgmt-web~=6.1.0', | |
'azure-multiapi-storage~=0.8.0', | |
'azure-storage-common~=1.4', | |
'azure-synapse-accesscontrol~=0.5.0', | |
'azure-synapse-artifacts~=0.12.0', | |
'azure-synapse-managedprivateendpoints~=0.3.0', | |
'azure-synapse-spark~=0.2.0', | |
'chardet~=3.0.4', | |
'colorama~=0.4.4', | |
# On Linux, the distribution (Ubuntu, Debian, etc) and version are checked for `az feedback` | |
'distro; sys_platform == "linux"', | |
'fabric~=2.4', | |
'javaproperties~=0.5.1', | |
'jsondiff~=1.3.0', | |
'packaging>=20.9,<22.0', | |
'PyGithub~=1.38', | |
'PyNaCl~=1.5.0', | |
'scp~=0.13.2', | |
'semver==2.13.0', | |
'six>=1.10.0', # six is still used by countless extensions | |
'sshtunnel~=0.1.4', | |
'urllib3[secure]', | |
'websocket-client~=1.3.1', | |
'xmltodict~=0.12' | |
] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, I think it's the psycopg2-binary
causes the failing of Build Rpm Package CentOS7
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why you need the binary
library?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in setup.py but build still failed.
If we don't use the psycopg2-binary
, but install Psycopg
from sources, python3-dev
and libpq-dev
package are required to build it. Any guide to add those dependencies in CLI build environment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2022-05-18T07:10:56.2535244Z Provides: azure-cli = 2.36.0-1.el7 azure-cli(x86-64) = 2.36.0-1.el7 config(azure-cli) = 2.36.0-1.el7 libcom_err-2abe824b.so.2.1()(64bit) libcrypto-2f2fcd35.so.1.1()(64bit) libcrypto.so.1.1(OPENSSL_1_1_0)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0a)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0c)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0d)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0f)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0g)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0h)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0i)(64bit) libcrypto.so.1.1(OPENSSL_1_1_0j)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1b)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1c)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1d)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1e)(64bit) libcrypto.so.1.1(OPENSSL_1_1_1h)(64bit) libffi-c643fa1a.so.6.0.4()(64bit) libgssapi_krb5-497db0c6.so.2.2()(64bit) libgssapi_krb5.so.2(HIDDEN)(64bit) libgssapi_krb5.so.2(gssapi_krb5_2_MIT)(64bit) libk5crypto-b1f99d5c.so.3.1()(64bit) libk5crypto.so.3(HIDDEN)(64bit) libk5crypto.so.3(k5crypto_3_MIT)(64bit) libkeyutils-dfe70bd6.so.1.5()(64bit) libkeyutils.so.1(KEYUTILS_0.3)(64bit) libkeyutils.so.1(KEYUTILS_1.0)(64bit) libkeyutils.so.1(KEYUTILS_1.3)(64bit) libkeyutils.so.1(KEYUTILS_1.4)(64bit) libkeyutils.so.1(KEYUTILS_1.5)(64bit) libkrb5-fc820a1d.so.3.3()(64bit) libkrb5.so.3(HIDDEN)(64bit) libkrb5.so.3(krb5_3_MIT)(64bit) libkrb5support-a4e68a84.so.0.1()(64bit) libkrb5support.so.0(HIDDEN)(64bit) libkrb5support.so.0(krb5support_0_MIT)(64bit) liblber-2-f149c062.4.so.2.11.7()(64bit) libldap_r-2-aa060733.4.so.2.11.7()(64bit) libpcre-9513aab5.so.1.2.0()(64bit) libpq-30c1c89d.so.5.14()(64bit) libsasl2-8fd1a564.so.3.0.0()(64bit) libselinux-0922c95c.so.1()(64bit) libssl-6b7b94d5.so.1.1()(64bit) libssl.so.1.1(OPENSSL_1_1_0)(64bit) libssl.so.1.1(OPENSSL_1_1_0d)(64bit) libssl.so.1.1(OPENSSL_1_1_1)(64bit) libssl.so.1.1(OPENSSL_1_1_1a)(64bit)
2022-05-18T07:10:56.2544788Z Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
2022-05-18T07:10:56.2551092Z Requires: ld-linux-x86-64.so.2()(64bit) ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.12)(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.16)(64bit) libc.so.6(GLIBC_2.17)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.7)(64bit) libc.so.6(GLIBC_2.8)(64bit) libcom_err-2abe824b.so.2.1()(64bit) libcrypto-2f2fcd35.so.1.1()(64bit) libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0)(64bit) libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0d)(64bit) libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0f)(64bit) libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0i)(64bit) libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_1)(64bit) libdl.so.2()(64bit) libdl.so.2(GLIBC_2.2.5)(64bit) libffi-c643fa1a.so.6.0.4()(64bit) libgssapi_krb5-497db0c6.so.2.2()(64bit) libgssapi_krb5-497db0c6.so.2.2(gssapi_krb5_2_MIT)(64bit) libk5crypto-b1f99d5c.so.3.1()(64bit) libk5crypto-b1f99d5c.so.3.1(k5crypto_3_MIT)(64bit) libkeyutils-dfe70bd6.so.1.5()(64bit) libkeyutils-dfe70bd6.so.1.5(KEYUTILS_0.3)(64bit) libkeyutils-dfe70bd6.so.1.5(KEYUTILS_1.0)(64bit) libkeyutils-dfe70bd6.so.1.5(KEYUTILS_1.5)(64bit) libkrb5-fc820a1d.so.3.3()(64bit) libkrb5-fc820a1d.so.3.3(krb5_3_MIT)(64bit) libkrb5support-a4e68a84.so.0.1()(64bit) libkrb5support-a4e68a84.so.0.1(krb5support_0_MIT)(64bit) liblber-2-f149c062.4.so.2.11.7()(64bit) libldap_r-2-aa060733.4.so.2.11.7()(64bit) libpcre-9513aab5.so.1.2.0()(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpthread.so.0(GLIBC_2.3.2)(64bit) libresolv.so.2()(64bit) libresolv.so.2(GLIBC_2.2.5)(64bit) libsasl2-8fd1a564.so.3.0.0()(64bit) libselinux-0922c95c.so.1()(64bit) libssl-6b7b94d5.so.1.1()(64bit) libssl-6b7b94d5.so.1.1(OPENSSL_1_1_0)(64bit) libssl-6b7b94d5.so.1.1(OPENSSL_1_1_1)(64bit) libz.so.1()(64bit) rtld(GNU_HASH)
...
Step 13/13 : RUN rpm -i ./azure-cli-dev.rpm && az --version
---> Running in db374f835afc
error: Failed dependencies:
libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0d)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0f)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0i)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_1)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libgssapi_krb5-497db0c6.so.2.2(gssapi_krb5_2_MIT)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libk5crypto-b1f99d5c.so.3.1(k5crypto_3_MIT)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libkeyutils-dfe70bd6.so.1.5(KEYUTILS_0.3)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libkeyutils-dfe70bd6.so.1.5(KEYUTILS_1.0)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libkeyutils-dfe70bd6.so.1.5(KEYUTILS_1.5)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libkrb5-fc820a1d.so.3.3(krb5_3_MIT)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libkrb5support-a4e68a84.so.0.1(krb5support_0_MIT)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libssl-6b7b94d5.so.1.1(OPENSSL_1_1_0)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
libssl-6b7b94d5.so.1.1(OPENSSL_1_1_1)(64bit) is needed by azure-cli-2.36.0-1.el7.x86_64
The command '/bin/sh -c rpm -i ./azure-cli-dev.rpm && az --version' returned a non-zero code: 1
I think there is some problem with psycopg2-binary
that it requires libs like libcrypto-2f2fcd35.so.1.1(OPENSSL_1_1_0)(64bit)
, but doesn't provide it, meaning it is not compatible with rpmbuild
and rpm
. This could be similar to #20608.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should use psycopg2
as dependency when build package, which is suggested by psycopg owner. It has some pre-required package, to reduce the complexity, we may use psycopg2-binary
in develop environment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should use
psycopg2
as dependency when build package, which is suggested by psycopg owner. It has some pre-required package, to reduce the complexity, we may usepsycopg2-binary
in develop environment.
Thanks for your help! Since psycopg2
has other library dependencies on Linux and MacOS. I update the code to dynamically install the packages only when user runs the command the first time. For windows, it needs no other library, so I add the package in windows requirement files to improve user experience.
src/azure-cli/setup.py
Outdated
@@ -152,7 +152,8 @@ | |||
'sshtunnel~=0.1.4', | |||
'urllib3[secure]', | |||
'websocket-client~=1.3.1', | |||
'xmltodict~=0.12' | |||
'xmltodict~=0.12', | |||
'psycopg2-binary==2.9.3' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add dependencies alphabetically.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you leave more comments on the _creadential_free
implementation>?
src/azure-cli/azure/cli/command_modules/serviceconnector/_utils.py
Outdated
Show resolved
Hide resolved
src/azure-cli/azure/cli/command_modules/serviceconnector/_utils.py
Outdated
Show resolved
Hide resolved
src/azure-cli/azure/cli/command_modules/serviceconnector/_params.py
Outdated
Show resolved
Hide resolved
src/azure-cli/azure/cli/command_modules/serviceconnector/_credential_free.py
Outdated
Show resolved
Hide resolved
src/azure-cli/azure/cli/command_modules/serviceconnector/_credential_free.py
Outdated
Show resolved
Hide resolved
src/azure-cli/azure/cli/command_modules/serviceconnector/custom.py
Outdated
Show resolved
Hide resolved
oauthlib==3.0.1 | ||
packaging==21.3 | ||
paramiko==2.10.1 | ||
pbr==5.3.1 | ||
pkginfo==1.8.2 | ||
portalocker==2.3.2 | ||
psutil==5.9.0 | ||
psycopg2==2.9.3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is it required for windows only?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
psycopg2 has no pre-compiled binary version on Macos and some linux distribution. It needs to compile from source code and needs some extra library installed. In order not to impact too many CLI users, we will install packages dynamically when users run the specific command.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
psycopg2
currently cannot be installed on Python 3.11 (psycopg/psycopg2#1514), which blocks Azure CLI's Python 3.11 support (#24494).
@kairu-ms Could you please help review the PR? |
Related command
Description
Add
--system-identity
option for commands:az spring-cloud/spring/webapp/containerapp connection create postgres/postgres-flexible/mysql-flexible/sql
It will enable the system identity of spring cloud app, and create an aad user in postgresql. Then create the connection.
Testing Guide
History Notes
[Component Name 1] BREAKING CHANGE:
az command a
: Make some customer-facing breaking change[Component Name 2]
az command b
: Add some customer-facing featureThis checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.