Set up a working environment
Python interpreters in DataSpell
To execute notebooks or Python code in DataSpell, you need to configure at least one virtual environment based on a Python interpreter. You can use a system interpreter that is available with your Python installation. You can also create a Virtualenv, pipenv, Poetry, or conda virtual environment. A virtual environment consists of a base interpreter and the installed packages.
With DataSpell, you can also configure interpreters to execute your notebooks on remote managed Jupyter servers by using SSH, or WSL (only for Windows).
When you attach a directory to the workspace or open a project, the IDE creates a virtual environment for you. In most cases, it is a conda environment based on your Anaconda installation. You can select any other Python environment on your machine or create a new one.
To preview a configured Python environment, modify its settings, and update packages, press Ctrl+Alt+S and go to .
Configuring an existing virtual environment
At any time, you can switch the virtual environment either by using the Python Interpreter selector or in Settings.
Switch the virtual environment using the Python Interpreter selector
The Python Interpreter selector is located on the status bar. It is the most convenient and quickest way to switch the Python interpreter. Just click it and select the target interpreter:
Switch the virtual environment in the IDE settings
Press Ctrl+Alt+S to open settings and then select .
If you want to change the Python interpreter for a specific project or workspace directory, select it. Otherwise, select workspace.
Click the drop-down and select the desired Python interpreter:
If it's not on the list, click Show All. Then select the desired interpreter in the left pane and click OK.
When DataSpell stops supporting any of the outdated Python versions, the corresponding Python interpreter is marked as unsupported.
Use Anaconda CLI to switch conda environments
In the Terminal window, run the
lscommand in the <CONDA_HOME> /envs directory (for example, /Users/jetbrains/.conda/envs) and select the target environment.Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin).
Execute the
conda activate <env name>command (for example,conda activate my-conda-env).
Modify a virtual environment
Press Ctrl+Alt+S to open settings and then select .
Expand the list of the available interpreters and click Show All.
You can modify the path to the Python executable in the Interpreter path field.
When the Associate this virtual environment with the current project checkbox is enabled, the interpeter is available only in the current DataSpell project.
To change the interpreter name, select the target interpreter and click
.
The Python interpreter name specified in the Name field, becomes visible in the list of available interpreters. Click OK to apply the changes.
Remove a virtual environment
If you no longer need a Python interpreter for a project, you can remove it from the project settings.
Do one of the following:
Press Ctrl+Alt+S to open settings and then select .
Click the Python Interpreter selector and choose Interpreter Settings.
Expand the list of the available interpreters and click Show All.
Choose the interpreter that you want to remove and click
.
Creating a new virtual environment
Configuring local virtual environments
To configure a local virtual environment follow one of the procedures below:
Create a virtualenv environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
The following actions depend on whether you want to generate a new virtual environment or to use an existing one.
- New virtualenv environment
Select Virtualenv from the list of environment types.
Select the base interpreter from the list, or click
and find the Python executable in your file system.Specify the location of the new virtual environment in the Location field, or click
and browse for the location in your file system. The directory for the new virtual environment should be empty.Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the
--system-site-packagesoption of the virtualenv tool.Select the Make available to all projects checkbox if you want to reuse this environment when creating Python interpreters in DataSpell.
- Existing virtualenv environment
Select the required interpreter from the list.
If the required interpreter is not on the list, click
, and then browse for the required Python executable (for example, venv/bin/python on macOS or venv\Scripts\python.exe on Windows).The selected virtual environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a virtualenv environment.
Create a conda environment
Ensure that Anaconda or Miniconda is downloaded and installed on your computer, and you're aware of a path to its executable file.
For more information, refer to the installation instructions.
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
The following actions depend on whether you want to create a new conda environment or to use an existing one.
- New conda environment
Select Conda from the list of environment types.
Select the Python version from the list.
Specify the environment name.
Normally, DataSpell will detect conda installation.
Otherwise, specify the location of the conda executable, or click
to browse for it.
- Existing conda environment
Select Conda from the list of environment types.
Normally, DataSpell will detect conda installation.
Otherwise, specify the location of the conda executable, or click
to browse for it.Select the environment from the list.
The selected conda environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a conda virtual environment.
Create a pipenv environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
The following actions depend on whether you want to generate a new virtual environment or to use an existing one.
- New pipenv environment
Select Pipenv from the list of environment types.
Select the base interpreter from the list, or click
and find the Python executable in your file system.If you have added the base binary directory to your
PATHenvironmental variable, you don't need to set any additional options: the path to the pipenv executable will be autodetected.If DataSpell doesn't detect the pipenv executable, click Install pipenv via pip to allow DataSpell to install it for you automatically.
Alternatively, follow the pipenv installation procedure to discover the executable path, and then specify it in the dialog.
- Existing pipenv environment
Select the interpreter from the list.
If the required interpreter is not on the list, click
, and then browse for the required Python executable.The selected virtual environment will be reused for the current project.
Click OK to complete the task.
When you have set the pipenv virtual environment as a Python interpreter, all available packages are added from the source defined in Pipfile. The packages are installed, removed, and updated in the list of the packages through pipenv rather than through pip.
For more information, refer to Configure a pipenv environment.
Create a Poetry environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
The following actions depend on whether you want to create a new Poetry environment or to use an existing one.
- New Poetry environment
Select Poetry from the list of environment types.
Select the base interpreter from the list or click
and find the Python executable in your file system.
- Existing Poetry environment
Make sure that the project directory contains a pyproject.toml file.
Select the interpreter from the list.
If the required interpreter is not on the list, click
, and then browse for the Python executable within the previously configured Poetry environment.
The selected Poetry environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a Poetry environment.
Configuring remote virtual environments
When a remote Python interpreter is added, at first the DataSpell helpers are copied to the remote host. DataSpell helpers are needed to run remotely the packaging tasks, debugger, tests and other DataSpell features.
Next, the skeletons for binary libraries are generated and copied locally. Also, all the Python library sources are collected from the Python paths on a remote host and copied locally along with the generated skeletons. Storing skeletons and all Python library sources locally is required for resolve and completion to work correctly.
DataSpell checks the remote helpers version on every remote run, so if you update your DataSpell version, the new helpers will be uploaded automatically, and you don't need to recreate remote interpreter.
Configure a WSL interpreter
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select On WSL.
Wait until DataSpell detects Linux on your machine and completes introspection. Click Next to proceed:
In the left-hand pane of the dialog, select the type of the WSL interpreter you want to create: Virtual Environment, Conda Environment, or System Interpreter.
For a system interpreter, just provide the path to the Python executable in the selected Linux distribution.
For virtual and conda environments, you can provide a path to a Python executable of an existing environment in the selected Linux distribution or create a new environment based on the specified Python.
Once done, the new interpreter will be added to your project, and the default mnt mappings will be set.
For more information, refer to Configure an interpreter using WSL.
Configure an interpreter using SSH
Ensure that there is an SSH server running on a remote host, since DataSpell runs remote interpreters via ssh-sessions.
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select On SSH.
Select an option to create a new SSH connection, then specify server information (host, port, and username).
Alternatively, you can select Existing and choose any available SSH configuration from the list. To create a new SSH configuration, follow the steps below:
- Creating an SSH configuration
Click
next to the list of configurations:
Click
, disable the Visible only for this project checkbox, and fill in the required fields:
Once done, the newly created SSH configuration will appear in the list of available configurations. Click Next to proceed:
In the next dialog window, provide the authentication details to connect to the target server.
Select Password or Key pair (OpenSSH or PuTTY) and enter your password or passphrase. If Key pair (OpenSSH or PuTTY) is selected, specify:
Private key: location of the file with a private key
Passphrase: similar to a password, it serves to encrypt the private key.
Click Next to proceed.
Wait until DataSpell completes the introspection of the SSH server.
In the next dialog, select a type of Python environment to configure on the SSH server.
You can create a new virtual environment or сonda environment, select an existing one, or use a system interpreter.
Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the
--system-site-packagesoption of the virtualenv tool.If you need to execute your Python code on the SSH server as a sudo user, enable the Execute code with root privileges via sudo checkbox.
DataSpell can automatically synchronize files with the remote host. To do that, select the Automatically upload files to remote host checkbox, and specify the root directory for synchronization in Root directory. If the directory doesn't exist, it will be created.
When this option is enabled, the attached directories that have been configured to use the SSH interpreter are uploaded to the root directory on the remote host. Then they are synced upon any of the following events:
Applying changes to files
Switching to this interpreter
IDE start-up
You can exclude files that do need to be synchronized. To do that, specify the glob patterns in the Excluded paths field. By default, the field contains some patterns for directories and files that are not required for running a remote interpreter.
Click Create to complete adding the interpreter.
For more information, refer to Configure an interpreter using SSH.
Manage packages
The reason for creating various Conda environments based on the same Anaconda installation is obvious - you can install specific packages for each environment and use them for specific tasks and projects.
You can install packages for an environment using pip and the Conda package manager. You can do it using the IDE UI in the environment settings or in the Terminal window.
Install packages using the environment settings
In the Python interpreter selector, select Interpreter Settings.
Select the target environment from the Python Interpreter list.
Click + to add a new package. Click the Conda package manager button (
) to manage packages from the Conda repository. Otherwise, DataSpell will be using pip.Type a package name in the search field and locate the target package. If needed, specify a package version. Then press Install
Close the window on the task completion. If needed, press the Manage Repositories button and add a custom repository from which you install packages.
Install packages in the Terminal window
Open the Terminal window.
To install a package from pip, execute:
pip install <package name>(for example,pip install plotly).To install packages from the Conda repository:
Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin)
Execute the following command:
conda install <package name>(for example,conda install tensorflow).
Sometimes, you want to upgrade packages in your environment. The quickest way is to use the environment settings.
Upgrade packages
In the Python interpreter selector, select Interpreter Settings.
Mind the
in the Latest version column. Select the corresponding package and click the Upgrade button ().