Starting Docker container pulsar-manager-0.3.0 with internal PostgreSQL does not work #505
Comments
|
Hello @malaskowski It seems that there is an issue in startup script. Database can't be created because binary path is not in the environment variable PATH. Unfortunately, your container will fail each time during init so you will have to fix this manually until the patch is deployed Have a nice day |
|
@splio-rbechon , the fix was committed in August of 2022. Is there any chance of getting a new container image release? The last release is effectively broken. |
|
@tuteng , sorry to ping you directly, but it looks like you push the container images. Is there any chance of getting a new version of the pulsar-manager image with the fix for this pushed? |
|
@urfreespace Can you help take a look at it? thanks |
|
@urfreespace @splio-rbechon @nodece thank you! |
|
Just wanted to say I'm also hitting this, and waiting for a new image to be built with the fix. |
|
streamnative/pulsar-manager:v0.3.1-rc2 Since the official release may take some time, I tried to build an image manually, can you try it? @bpereto @Zand3r24 @doug-ba @malaskowski @splio-rbechon |
|
@tuteng Out of curiosity, are you working on v0.3.1 release based on current master branch? |
|
Hitting this as well, looks like a lot of this stuff is on hold or what? |
|
@tuteng But manager started correctly and works, thank you. |
tisonkun
changed the title
|
^
keeps crashing: |
|
@tisonkun I will test v0.4.0-candidate-2 if you will release a docker image. |
|
still have the back-end crashing in the 4.0.0 |
|
I builded the docker image with FROM openjdk:17-slim-bullseye and needed an extra java opts: JAVA_OPTS: "--add-opens java.base/java.lang=ALL-UNNAMED" with this setup it works fine. or use an external database ( ex. bitnami postgresql helm chart) for a production deployment, as I have. |
|
Hi, I am deploying Pulsar in Kubernetes (Azure) and am facing the same issue. I deploy using Helm. Is there a way to solve this for my deployment? If yes, how? Thanks in advance! |
|
@kocaki configData:
SPRING_CONFIGURATION_FILE: "/pulsar-manager/pulsar-manager/application.properties"
# Workaround for https://github.com/apache/pulsar-manager/issues/505
URL: ""This works with |
|
@malaskowski thanks for your answer! It looks like pulsar manager does more now because now I get " username or password is wrong". I try to login with pulsar and pulsar but it does not work. So there is still an issue somewhere. I also tried deploying 0.4.0. In the logs of that release I do see that the db is created but for some reason also that one does not work. Logs of 0.4.0: Do you have any suggestions? |
|
|
Hello @kocaki Do you have this in your helm template file that deployed pulsar manager ? On my side, this variable was the root cause of Supervisord issue ... I just removed it to fix my issue |
|
Hi @splio-rbechon, It is existing in the pulsar-manager-deployment.yaml but looks different: Tried a new deployment without it but it is still stuck on the same. Logs: So with the suggestion of @malaskowski the backend startup problem has been solved but now there is still a db issue. |
|
It finally started yesterday with pulsar 0.4.0 for me, can you try adding this conf : |
|
Hi @enzo-dechaene, Unfortunately also failing with this setup. Logs show me that the pular_manager_backend keeps restarting. Can you share your pulsar manager config so I can compare? |
|
@kocaki I followed the instructions for creating user via the backend API: MANAGER_POD=`kubectl get pods --namespace pulsar --selector component=pulsar-manager --no-headers -o custom-columns=":metadata.name"`
CSRF_TOKEN=$(kubectl exec --namespace pulsar -it $MANAGER_POD -- curl http://localhost:7750/pulsar-manager/csrf-token)
kubectl exec --namespace pulsar -it $MANAGER_POD -- curl \
-H 'X-XSRF-TOKEN: $CSRF_TOKEN' \
-H 'Cookie: XSRF-TOKEN=$CSRF_TOKEN;' \
-H "Content-Type: application/json" \
-X PUT http://localhost:7750/pulsar-manager/users/superuser \
-d '{"name": "admin", "password": "mypassword", "description": "console admin", "email": "admin@example.com"}'This is also fairly easy to automate. |
|
Hi @malaskowski, Thanks I will try it. With 0.4.0 I see that the db is being created in the pod but unfortunately the pulsar_manager_backend keeps restarting. Are you running the db embedded or using an external db and version 0.3.0 or 0.4.0? Because the embedded db will be destroyed once you redeploy the pod right or will it migrate the db to the new pod? |
|
Hello, @kocaki, I'm using db postgresql embedded with pulsar manager 0.4.0. For the deployment of Pulsar Manager, I only refer to the configmap configuration in the env parameter: The config map Notice that USERNAME and PASSWORD is not the credential for access the interface. It's the username of database. You can check all parameter on this link If you want to add user pulsar, you must connect to the pulsar manager pod and run this script |
|
Hi @enzo-dechaene, Thanks for the info and help. The problem is that the pulsar-manager-backend keeps restarting and we cannot find out why. The config looks as following: I attached the log of the pulsar manager where you can see that db is being created successfully with 0.4.0 but then it the backend keeps restarting so even if I should create a user I am unable to login. Our configuration is in Azure AKS. When I check the resources I see there is an internal and external ip assigned and necessary ports are set open automatically. Besides Pulsar Manager also the Proxy pod is restarting with the error it cannot start http server on port 80. No idea if it is related to each other. If I install version 0.1.0, I can login on the webinterface but receive a lot of errors (500 internal server error). And with 3.0 there is this known issue regarding the db which this topic is about. Tried also an external PostgreSql db but looks like it is not being used. DB stays empty and no errors in log. Update: I just tried the command to create a user with v0.3.0 (with fix suggested by @malaskowski) and despite the errors about the db in the log, I am now able to login in Pulsar Manager. I will test functionality and troubleshoot the proxy issue but that is finally some progress... thanks both! |
|
Just wanted to chip in with what I came out with after a bit of a rabbit chase and how I got things to not repeatledy restart. On version 0.4.0 (but some of the errrors are probably there in 0.3.0 as well) I found out that So both those ENV:s has to point to the same config file. Fixing that, the backend still restarts since the helm chart for the pulsar manager deployment creates an ENV named So that ENV also has to be fixed. My final version of the values.yaml file looks like this: With those three ENV:s configured I managed to get the backend into a running state but still had to manually create a user as described by @malaskowski here. |
|
Also stumbled upon the issue with the postgres not starting up in 0.3.0 as well as the restarting backend issue. Then it took a bit further reading here and after I had changed the Then I also discovered here that the pulsar:pulsar credentials is not for the frontend and that you have to generate that yourself using the API. Finally I can use pulsar-manager after two late nights :-) |
|
Hi! I reproduced the problem and @jesperbagge suggestion works also fine for me. I just submitted a PR to fix the boot issue. |
|
I think the chart deployment on the pulsar/pulsar repository needs to be updated as well ? |
|
Hi enzo-dechaene, |
|
Hi everyone! The PR is merged. Huge thanks for your investigations! |
|
new create Dockerfile file exegesis addgroup and adduser in the file of entrypoint.sh: ...
# addgroup pulsar
# adduser --disabled-password --ingroup -ms /bin/sh pulsar pulsar
...rebuild image and run it: $ docker build -t apachepulsar/pulsar-manager:20240416 .
$ docker run -dit \
-p 9527:9527 -p 7750:7750 \
-v $PWD/config/application.properties:/pulsar-manager/pulsar-manager/application.properties \
-v $PWD/secret/admin.key:/pulsar-manager/private.key \
-e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
-e URL='jdbc:postgresql://127.0.0.1:5432/pulsar_manager' \
-e USERNAME=pulsar \
-e PASSWORD=pulsar \
-e LOG_LEVEL=DEBUG \
-v $PWD/data:/data \
apachepulsar/pulsar-manager:20240416 /bin/sh
|
Running Pulsar Manager with the default Pulsar Helm Chart settings ends with not working manager instance.
Steps to reproduce
Run the container using the default:
Result
First, the user is asked for the information by the
addusercommand from the https://github.com/apache/pulsar-manager/blob/v0.3.0/docker/entrypoint.sh#L27 (looks like fix for this is pretty easy:adduser --disabled-password --gecos "" --ingroup pulsar pulsar). However, later running the/pulsar-manager/startup.shfails with:/pulsar-manager/startup.sh: 21: initdb: not found /pulsar-manager/startup.sh: 22: pg_ctl: not found createdb: error: could not connect to database template1: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? psql: error: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?Then in the
/pulsar-manager/pulsar-manager.logyou can see the connection error (as the DB didn't start):2023-01-13 10:49:51.463 ERROR 156 --- [pool-2-thread-1] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization. org.postgresql.util.PSQLException: Connection to 127.0.0.1:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.Finally, the backend never starts properly.
Expected result
The Pulsar Manager creates an internal DB instance and runs properly.
Additional details
I'm running the command above on M1 Apple Silicon processor and Docker Desktop for Mac
v4.15.0.The text was updated successfully, but these errors were encountered: