Datalore 2024.4 Help

Connect to an Amazon S3 cloud storage

The article explains how to create an Amazon S3 cloud storage connection.

Attach an Amazon S3 bucket

  1. Open the New cloud storage connection dialog. The step depends on whether you want to add a new data source to workspace resources or attach it to a notebook.

    1. On the Home page, select the workspace to which you want to add a cloud storage connection.

    2. From the left-hand menu of the selected workspace, select Data and switch to the Cloud storages tab. This will open the list of all workspace cloud storage connections.

    3. On the Cloud storages tab, click the Add button in the upper right corner.

      Cloud storages menu
    1. Open the Attached data tool from the left-hand sidebar.

    2. Switch to the Cloud storages tab. You will see the list of all cloud storage connections available from the respective workspace.

    3. At the bottom of the tab, click New cloud storage.

      Creating cloud storage connection from notebook
  2. In the New cloud storage connection dialog, select Amazon S3.

  3. In the New Amazon S3 cloud storage connection dialog, fill in the following fields:

    • Display name: to specify the name for this data source in your system

    • AWS access key and AWS secret access key: to access your AWS account (details here)

    • Region: to specify your AWS region

    • Amazon Bucket name: to specify the name of the bucket you want to mount

    • Custom options: to specify additional parameters. See the example below

    • Endpoint URL: to specify the website of the bucket you want to mount

    New Amazon S3 cloud storage connection dialog
  4. (Optional) Click Test connection to make sure the provided parameters are correct.

  5. Click Create and close to finish the procedure.

Configure optional parameters for Amazon S3 data sources

Use the Custom_options field for optional parameters when creating an Amazon S3 data source. Below are two examples of how it can be used.

  • To enable SSE-C for S3 data sources, specify the following in the Custom_options: In the Custom_options field, specify the following:

    use_sse=c:/path/to/keys/file

    where:

    /path/to/keys/file is the file that contain keys. Make sure permissions are 600.

  • (For Datalore On-Premises only) To provide access based on a role associated with that of an EC2 instance profile, add public_bucket=0,iam_role into the Custom_options field.

Results and further actions

  • If created for a specific notebook, the new connection will be automatically added to the workspace. You can later attach this data source to any other notebook from this workspace.

  • If created for a workspace, this connection is added to the workspace cloud storages and can be attached to any notebook from this workspace.

Keywords

cloud storages, cloud storage connection, attach data, data sources, Amazon S3

Last modified: 30 October 2024