Migrate start-notebook & start-singleuser to python #2006

yuvipanda · 2023-10-17T06:18:09Z

Based on

Stop using bash, haha 👍

If there's more apetite for this, I'll try to migrate start.sh and start-singleuser.sh as well - I think they should all be merged together. We can remove the .sh suffixes for accuracy, and keep symlinks in so old config still works. Since the shebang is what is used to launch the correct interpreter, the .sh doesn't matter.

Will help fix #1532, as I believe all those things are going to be easier to do from python than bash

TODO

Add .py extensions to the script
Add shim .sh files that just redirect us back
Document these .sh scripts

yuvipanda · 2023-10-17T06:18:33Z

@mathbunnyru I wanted to ask if there's apetite for this kinda change before I put more work into this :)

yuvipanda · 2023-10-17T08:38:51Z

I'll take a look at the test failures if this kinda migration is acceptable! I'm sure I can sort those out if needed.

mathbunnyru · 2023-10-17T09:48:09Z

I like this PR a lot, just a few thoughts about the implementation:

Let's add the .py extension to the added start-like scripts. It doesn't change much, but some utils (maybe pre-commit hooks) and editors definitely work better (at least not worse).
Instead of symlinking to keep compatibility, I propose having simple shell files, which first echo a warning that calling is file is deprecated and then simply pass all the params to the corresponding .py file. This way we will be able to first replace a warning with a hard error in a few months and then completely delete the .sh files.
Let's also document the warning about calling old .sh start scripts.
I really hope you will have the courage to deal with start.sh when we merge this PR 🙂

Overall, great PR, thank you!

images/base-notebook/start-notebook

mathbunnyru · 2023-10-17T11:04:24Z

I updated your branch (to make sure we test latest versions of everything).

yuvipanda · 2023-10-17T11:05:35Z

Glad to hear you like it, @mathbunnyru! I've addressed all the points you mentioned and I think this is ready for review now.

I do personally prefer leaving out the .py suffix, but am alright leaving it in. I find that most tools are ok looking at the shebang for detection in those cases, but I'm sure that's not 100% of them.

I'm definitely interested in tackling start.sh as well! I actually wrote https://github.com/yuvipanda/jupyterhub-roothooks/ a while ago that provides some part of that functionality, so there's gonna be stuff for me to steal from there. I also spent today picking up a plan to replicate as exactly as possible bash's source semantics in python and I think I've a path there.

My goal would be to do this in such a way that all the existing tests pass, so no new functionality.

Based on > Stop using bash, haha 👍 from jupyter#1532. If there's more apetite for this, I'll try to migrate `start.sh` and `start-singleuser.sh` as well - I think they should all be merged together. We can remove the `.sh` suffixes for accuracy, and keep symlinks in so old config still works. Since the shebang is what is used to launch the correct interpreter, the `.sh` doesn't matter. Will help fix jupyter#1532, as I believe all those things are going to be easier to do from python than bash

-u can not be set by shebang, we must set the env var instead

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

yuvipanda · 2023-10-17T11:06:09Z

@mathbunnyru hah, we clashed as I was just about to rebase :)

docs/using/selecting.md

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

mathbunnyru · 2023-10-17T11:10:55Z

images/base-notebook/start-notebook.py

+import shlex
+import sys
+
+# If we are in a JupyterHub, we pass on to `start-singleuser.py` instead so it does the right thing


I appreciate it that you added comments to this file! I'm sure it will help future contributors to understand this file faster.

@mathbunnyru you're welcome :)

manics · 2023-10-17T11:12:48Z

I'm definitely interested in tackling start.sh as well! I actually wrote https://github.com/yuvipanda/jupyterhub-roothooks/ a while ago that provides some part of that functionality, so there's gonna be stuff for me to steal from there. I also spent today picking up a plan to replicate as exactly as possible bash's source semantics in python and I think I've a path there.

If you're modifying the behaviour of the start scripts #1528 might fit in too.

yuvipanda · 2023-10-17T12:18:27Z

@manics I'm probably going to just try do a verbatim port, and leave the behavior modification to others in the future. Hopefully the fact that it's in python and not bash will make that easier.

yuvipanda · 2023-10-17T12:48:56Z

Yay, thank you for the quick review and merge, @mathbunnyru. And for the long set of tests that made this easy

yuvipanda changed the title ~~Migrate start-notebook.sh to bash~~ Migrate start-notebook & start-singleuser to python Oct 17, 2023

yuvipanda force-pushed the start-notebook-python branch from 6ac1519 to 0cabfb7 Compare October 17, 2023 06:42

mathbunnyru reviewed Oct 17, 2023

View reviewed changes

images/base-notebook/start-notebook Outdated Show resolved Hide resolved

mathbunnyru reviewed Oct 17, 2023

View reviewed changes

images/base-notebook/start-notebook Outdated Show resolved Hide resolved

yuvipanda force-pushed the start-notebook-python branch 2 times, most recently from c4f46de to 15f6833 Compare October 17, 2023 10:46

yuvipanda marked this pull request as ready for review October 17, 2023 11:01

yuvipanda and others added 17 commits October 17, 2023 16:35

Rename start-notebook.sh to start-notebook

ed70463

Cleanup start-notebook a little

00e751e

Fix typo

ea06768

Migrate start-singleuser as well

14a1e8c

Remove unused import

40f37d7

Run symlink commands as root

3364294

Combine repetitive RUN commands

bb98d5e

Remove multiple args to env

9e71839

-u can not be set by shebang, we must set the env var instead

Fix conditional inversion

67302e3

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

Fix how start-singleuser is exec'd

1804a8c

Actually call jupyterhub-singleuser in start-singleuser

527b756

Pass through any additional args we get

9f7bdb3

Put .py suffix on the start-* scripts

37d5b15

Add .sh shims for the start-* scripts

81c67ef

Document start-notebook.sh and start-singleuser.sh

219cf36

Partially test start-notebook.sh

d6519ac

yuvipanda force-pushed the start-notebook-python branch from b9334f9 to d6519ac Compare October 17, 2023 11:05

mathbunnyru reviewed Oct 17, 2023

View reviewed changes

docs/using/selecting.md Outdated Show resolved Hide resolved

Reflow warning docs

99410dc

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

mathbunnyru reviewed Oct 17, 2023

View reviewed changes

mathbunnyru approved these changes Oct 17, 2023

View reviewed changes

yuvipanda mentioned this pull request Oct 17, 2023

[FAST_BUILD] Migrate runhooks to python #2007

Draft

mathbunnyru merged commit bceaead into jupyter:main Oct 17, 2023
56 checks passed

benz0li mentioned this pull request Oct 23, 2023

"CONDA_DIR" b-data/jupyterlab-python-docker-stack#3

Closed

chris-bateman mentioned this pull request Oct 31, 2023

Move from dockerhub to quay.io (and consider we got an 800$ charge!) jupyterhub/team-compass#688

Open

9 tasks

mathbunnyru mentioned this pull request Dec 2, 2023

Error to start docker service using older image #2045

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate start-notebook & start-singleuser to python #2006

Migrate start-notebook & start-singleuser to python #2006

yuvipanda commented Oct 17, 2023 •

edited

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

mathbunnyru commented Oct 17, 2023

mathbunnyru commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

mathbunnyru Oct 17, 2023

yuvipanda Oct 17, 2023

manics commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

Migrate start-notebook & start-singleuser to python #2006

Migrate start-notebook & start-singleuser to python #2006

Conversation

yuvipanda commented Oct 17, 2023 • edited

TODO

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

mathbunnyru commented Oct 17, 2023

mathbunnyru commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

mathbunnyru Oct 17, 2023

Choose a reason for hiding this comment

yuvipanda Oct 17, 2023

Choose a reason for hiding this comment

manics commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023

yuvipanda commented Oct 17, 2023 •

edited