Customize or update environment
You can build a custom Docker agent image on top of the default one to customize your environment (install some package from apt to be available in your notebooks, or set up a custom Python environment by pre-installing the required libraries).
- Generate indexes to ensure the editor's proper work
After you create a new custom environment or update one of our environments, run the following command in the same file:
RUN /opt/datalore/build_code_insight_data.sh /[environment_path]where
[environment_path]
is the path to the environment you created or updated.If not performed, these indexes will be updated every time the agent starts, which slows down the editor.
- Add the created environment file to the main datalore pod
Add the file to following directory:
/opt/datalore/configs/environment_infoMake sure you follow these instructions:
For a conda environment, specify the respective .yml file matching the envrionment name. For example, if you created a conda environment called somename, add an environment_somename.yml file.
For a pip environment, specify the respective .txt file matching the envrionment name. For example, if you created a pip environment called somename, add a requirements_somename.txt file.
Enable arbitrary PyPi server configuration
The procedure sets up a custom PyPI server configuration. This will allow you to explore your custom repositories in the editor's Environment tool and directly install packages from outside the default pypi.org.
Run the following in RUN command when building the custom agent image:
pip config set global.trusted-host "$(pip config get global.trusted-host) <host>" pip config set global.extra-index-url "url>"To enable it for a specific notebook, add the above commands to the init.sh file.
To make a custom environment available for new notebooks, use the Admin panel as described in this topic.
Examples
Modify the pip/minimal environment
Use the additional_packages.txt file as a list of packages and their versions as shown below:
Create a new environment
Assuming the name of the new environment is myenv
:
- Custom Python environment with version 3.10
This custom environment sets Python 3.10 as the default version.
FROM jetbrains/datalore-agent:2024.4 USER root ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && apt-get install -y python3.10-venv && rm -rf /var/lib/apt/lists/* && apt-get clean USER datalore RUN /usr/bin/python3.10 -m venv /opt/python/envs/myenv RUN /opt/python/envs/myenv/bin/python -m pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 aiohttp==3.8.3 lets-plot pandas RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/myenv- Custom Python environment with version 3.11
This custom environment sets Python 3.11 as the default version.
FROM jetbrains/datalore-agent:2024.4 USER root RUN apt-get update && apt-get install -y python3.11 python3.11-venv && rm -rf /var/lib/apt/lists/* && apt-get clean USER datalore RUN /usr/bin/python3.11 -m venv /opt/python/envs/myenv RUN /opt/python/envs/myenv/bin/python -m pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 aiohttp==3.8.3 lets-plot pandas RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/myenv- Custom R environment without Anaconda
This custom environment sets a non-conda default environment with the R kernel.
FROM jetbrains/datalore-agent:2024.4 ENV CUSTOM_ENV_NAME myenv USER root RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y -q --no-install-recommends libzmq3-dev libcurl4-openssl-dev libssl-dev r-base make g++ libharfbuzz-dev libfribidi-dev libtiff-dev apt-file && \ Rscript -e "install.packages(c('repr', 'IRdisplay', 'IRkernel'), type = 'source')" && \ rm -rf /var/lib/apt/lists/* && apt-get clean RUN sudo chown -R datalore:datalore /home/datalore USER datalore RUN mkdir -p /opt/anaconda3/envs/$CUSTOM_ENV_NAME RUN /opt/python/bin/python -m venv /opt/anaconda3/envs/$CUSTOM_ENV_NAME RUN /opt/anaconda3/envs/$CUSTOM_ENV_NAME/bin/pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 RUN PATH=/opt/anaconda3/envs/$CUSTOM_ENV_NAME/bin/:$PATH Rscript -e "IRkernel::installspec(sys_prefix=TRUE)" RUN /opt/datalore/build_code_insight_data.sh /opt/anaconda3/envs/$CUSTOM_ENV_NAME