Has anyone succeeded in finding a good Scala/Spark kernel for Jupyter?
I am using both almond and toree. For executing only scala code, I am using almond. If you want to learn new features in scala (ex: 2.13), then use almond(Add Scala 2.13.4 support in almond v0.11.0). Almond might start supporting scala 3.0 once it is released.
For spark, I am using toree. Toree works fine for me. I am using toree 0.5.0 with spark 3.0.0. Toree uses scala 2.12.
First install spark 3.0.0 by following the steps provided here. After installation check if you are able to launch spark shell from the terminal.
#toree installation
#I used poetry for managing deps in virtual environment
pip install https://dist.apache.org/repos/dist/dev/incubator/toree/0.5.0-incubating-rc1/toree-pip/toree-0.5.0.tar.gz

# I installed/copied extracted spark binaries to /opt/spark
jupyter toree install --user --kernel_name=toree --spark_home=/opt/spark

With these steps, you should find the new kernel added to ~$HOME/.local/share/jupyter/kernels/. Launch jupyter notebook and you should be able to find the toree kernel listed when trying to create a new notebook. You could also execute jupyter kernelspec list to see if the new kernel is listed.
If you are able to create a new notebook, then you should have the kernel initialized without any issues. Try executing println(&quot;Hello world&quot;) and it should be successful.
I have captured installation summary here