Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swam examples unable to start a notebook #363

Closed
zozo6015 opened this issue Mar 27, 2020 · 4 comments
Closed

swam examples unable to start a notebook #363

zozo6015 opened this issue Mar 27, 2020 · 4 comments

Comments

@zozo6015
Copy link

zozo6015 commented Mar 27, 2020

I am using the swarm examples from the repository when I try to start up a notebook I can see the following errors in the docker-compose console:

hub_1    | [D 2020-03-27 10:15:01.667 JupyterHub dockerspawner:811] Getting container 'jupyter-admin'
hub_1    | [D 2020-03-27 10:15:01.713 JupyterHub swarmspawner:133] Service lrmb1ld status: {'Message': 'preparing',
hub_1    |      'PortStatus': {},
hub_1    |      'State': 'preparing',
hub_1    |      'Timestamp': '2020-03-27T10:13:59.436022091Z'}
hub_1    | [I 2020-03-27 10:15:01.718 JupyterHub dockerspawner:1148] Stopping service jupyter-admin (id: lrmb1ld)
hub_1    | [I 2020-03-27 10:15:01.720 JupyterHub swarmspawner:232] Removing service lrmb1ld82y8sxrbe133vj0td4
hub_1    | [E 2020-03-27 10:15:01.836 JupyterHub gen:974] Exception in Future <Task finished coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn() done, defined at /opt/conda/lib/python3.6/site-packages/jupyterhub/handlers/base.py:619> exception=TimeoutError('Timeout',)> after timeout
hub_1    |     Traceback (most recent call last):
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 970, in error_callback
hub_1    |         future.result()
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 626, in finish_user_spawn
hub_1    |         await spawn_future
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/user.py", line 486, in spawn
hub_1    |         raise e
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/user.py", line 406, in spawn
hub_1    |         url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
hub_1    |     tornado.util.TimeoutError: Timeout
hub_1    |     
hub_1    | [I 2020-03-27 10:15:01.862 JupyterHub log:158] 200 GET /hub/api/users/admin/server/progress (admin@::ffff:62.231.93.77) 50076.50ms
hub_1    | [D 2020-03-27 10:15:06.962 JupyterHub swarmspawner:144] Getting task of service 'jupyter-admin'
hub_1    | [D 2020-03-27 10:15:06.963 JupyterHub dockerspawner:811] Getting container 'jupyter-admin'
hub_1    | [I 2020-03-27 10:15:06.983 JupyterHub dockerspawner:818] Service 'jupyter-admin' is gone
hub_1    | [I 2020-03-27 10:15:06.995 JupyterHub swarmspawner:166] Task for service 'jupyter-admin' is gone
hub_1    | [E 2020-03-27 10:15:06.999 JupyterHub gen:974] Exception in Future <Future finished exception=RuntimeError('Service jupyter-admin not found',)> after timeout
hub_1    |     Traceback (most recent call last):
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/tornado/gen.py", line 970, in error_callback
hub_1    |         future.result()
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/dockerspawner.py", line 1055, in start
hub_1    |         yield self.start_object()
hub_1    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/swarmspawner.py", line 252, in start_object
hub_1    |         raise RuntimeError("Service %s not found" % self.service_name)
hub_1    |     RuntimeError: Service jupyter-admin not found

I was trying to set this up as close as possible to the examples because my custom environment has the same problems and I wanted to see what is different

@songabc
Copy link

songabc commented May 27, 2020

seems like I have the same problem.
first I set up docker cluster
then I follow the swarm example instroduction, move work dir to sample/swarm/
and run docker-compose build && docker-compose up
when I login with username 'zl', page response with 'Spawn failed: Service jupyter-zl not found' log event.

I also found that on my node machine, singleuserjupyterhub container was try to run with every login request, but finally with exited status.

I think it maybe due to wrong network setting which casuse jupyterhub can't communicate with singleuser container
I am trying to figure it out

@z3ky
Copy link

z3ky commented Jun 20, 2020

I run into the same problem. Did you find a solution?

@zozo6015
Copy link
Author

Yes, I had to create the stack name swarm any other name did not worked. Once the stack name was set as swarm it started to work and it's still working as a charm.

@minrk
Copy link
Member

minrk commented Mar 12, 2021

Closing since OP reported a resolution. Other discussion of the issue may be related to #333 where a previous service instance is not fully deleted before creating a new one

@minrk minrk closed this as completed Mar 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants