Skip to content

feat: Jupyterhub with keycloak, spark and s3 #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 39 commits into from
Mar 4, 2025
Merged

feat: Jupyterhub with keycloak, spark and s3 #155

merged 39 commits into from
Mar 4, 2025

Conversation

adwk67
Copy link
Member

@adwk67 adwk67 commented Feb 26, 2025 •
edited
Loading

This demo illustrates JupyterHub usage with Keycloak. Various configuration settings will be explained in more detail in the main documentation. See stackabletech/documentation#715.

adwk67 and others added 25 commits November 19, 2024 10:31
@adwk67
initial keycloak setup
8f41ef3
@adwk67
wip: jupyterhub + keycloak
eaf555c
@adwk67
wip
34fb370
@adwk67
wip: certificates work but callback does not
54fc0dc
@adwk67
wip: various tweaks
ae34b6a
@adwk67
added some temp docs
34c138d
@adwk67
add login info
05ad6d6
@adwk67
added some readme info
e16da8c
@adwk67
corrected ingress secret, set python cacert explicitly
0f5dce1
@adwk67
Merge branch 'main' into feat/keycloak-jupyterhub
0e3a28c
@adwk67
wip: working version
ca0c492
@adwk67
clean-up realm-config
f046dd8
@adwk67
delegate user check to Keycloak
e8eb2f9
@adwk67
use demo-specific keycloak
c1274e6
@adwk67
removed unnecessary settings
c41f309
@adwk67
specify ports
f6d22a9
@adwk67
add jupyterhub.yaml to stack
697a0a8
@adwk67
wip: working nb/spark combo
396705f
@adwk67
read/write from s3
bc94e33
@adwk67
remove driver service resource in favour of the ones produced dynamic…
53132a8 
…ally
@adwk67
use secret for minio credentials, add demo entry
803e520
@adwk67
set endpoints via extra config
bcfa3ae
@adwk67
mount notebook
0ff07da
@adwk67
user-specific job name
9d431b5
@adwk67
add some notebook comments
79fdb3b
@adwk67 adwk67 linked an issue Feb 26, 2025 that may be closed by this pull request
SDP support for JupyterHub stackabletech/issues#191
Closed
@adwk67 adwk67 marked this pull request as ready for review February 28, 2025 15:16
@adwk67 adwk67 moved this to Development: Waiting for Review in Stackable Engineering Feb 28, 2025
@adwk67 adwk67 changed the title feat: jupyterhub with keycloak, spark and s3 feat: Jupyterhub with keycloak, spark and s3 Feb 28, 2025
@adwk67 adwk67 requested a review from a team February 28, 2025 15:22
@maltesander maltesander self-requested a review March 3, 2025 11:32
@maltesander maltesander moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Mar 3, 2025
maltesander
maltesander requested changes Mar 3, 2025
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worked smoothly, first batch of comments, will do some more testing!

sbernauer
sbernauer reviewed Mar 3, 2025
View reviewed changes
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the nice demo!
One general remark, what is the reason that HDFS was added? I feel like this over-complicates the demo a bit, S3 would be sufficient from my point of view.

@adwk67 @maltesander
Apply suggestions from code review
3d4484c 
Review comments.

Co-authored-by: Malte Sander <malte.sander.it@gmail.com>
@adwk67
Copy link
Member Author

adwk67 commented Mar 3, 2025

Thanks for the nice demo! One general remark, what is the reason that HDFS was added? I feel like this over-complicates the demo a bit, S3 would be sufficient from my point of view.

Yes, I think I agree - I'll take it out (we have it included in one of the other jupyterhub demos anyway).

@sbernauer
Copy link
Member

Thanks!

sbernauer
sbernauer reviewed Mar 3, 2025
View reviewed changes
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the feedback!

@adwk67
Copy link
Member Author

adwk67 commented Mar 3, 2025

Thanks!

Done: 80bd2c6

adwk67 and others added 2 commits March 3, 2025 16:20
@adwk67 @sbernauer
Update docs/modules/demos/pages/jupyterhub-keycloak.adoc
44b0ecf 
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
@adwk67 @sbernauer
Update docs/modules/demos/pages/jupyterhub-keycloak.adoc
49d47e0 
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
@adwk67 adwk67 requested a review from maltesander March 3, 2025 15:33
adwk67 and others added 3 commits March 3, 2025 18:07
@adwk67 @maltesander
Update docs/modules/demos/pages/jupyterhub-keycloak.adoc
5a2c6cf 
Co-authored-by: Malte Sander <malte.sander.it@gmail.com>
@adwk67 @maltesander
Update docs/modules/demos/pages/jupyterhub-keycloak.adoc
0eb1ac7 
Co-authored-by: Malte Sander <malte.sander.it@gmail.com>
@adwk67
added a note about proxy reachability
7be5288
maltesander
maltesander approved these changes Mar 4, 2025
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM!

@adwk67 adwk67 added this pull request to the merge queue Mar 4, 2025
Merged via the queue into main with commit 96c04ac Mar 4, 2025
2 checks passed
@adwk67 adwk67 deleted the feat/keycloak-jupyterhub branch March 4, 2025 09:31
@adwk67 adwk67 moved this from Development: In Review to Development: Done in Stackable Engineering Mar 4, 2025
@lfrancke lfrancke moved this from Development: Done to Done in Stackable Engineering Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbernauer sbernauer sbernauer left review comments

@maltesander maltesander maltesander approved these changes

Assignees

@adwk67 adwk67

Labels
release/25.3.0
Projects
Stackable Engineering
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

SDP support for JupyterHub
4 participants
@adwk67 @sbernauer @maltesander @lfrancke