Refresh S3 credentials in a service

In EDITO services or processes, you can automatically integrate the environment variables needed to access S3 storage, directly from the information available in your user account.

These S3 credentials allow you to interact directly with the content of your EDITO buckets, which you can view in File Explorer. They provide authenticated access to your storage space (read, write, delete) and make it easy to use S3 in your Python scripts, applications, or notebooks without having to manually manage your credentials.

The S3 credentials generated by the platform are temporary tokens valid for 24 hours. After expiration, you can renew them either by deleting and restarting your service, or by running a refresh script directly in your environment, like this:

source /opt/refreshS3Credentials.sh

Or, to avoid prompting:

EDITO_USERNAME=<USERNAME> EDITO_PASSWORD=<PASSWORD> && source /opt/refreshS3Credentials.sh

In python, you can use this snippet of code instead:

import requests
import os
from xml.etree import ElementTree

DATALAB_USERNAME = "<USERNAME>"  # To change with your username
DATALAB_PASSWORD = "<PASSWORD>"  # To change with your password

url = "https://auth.dive.edito.eu/auth/realms/datalab/protocol/openid-connect/token"
headers = {
    'Content-Type': 'application/x-www-form-urlencoded'
}
data = {
    'client_id': 'onyxia-minio',
    'username': DATALAB_USERNAME,
    'password': DATALAB_PASSWORD,
    'grant_type': 'password',
    'scope': 'openid email profile'
}

response = requests.post(url, headers=headers, data=data)
json_response = response.json()
access_token = json_response["access_token"]

params = {
    "Action": "AssumeRoleWithWebIdentity",
    "WebIdentityToken": access_token,
    "DurationSeconds": "86400",
    "Version": "2011-06-15"
}

response = requests.post(os.environ["S3_ENDPOINT"], params=params)

root = ElementTree.fromstring(response.content)
namespace_as_text = root.tag[root.tag.find("{")+1:root.tag.find("}")]
namespace = {'ns': namespace_as_text}
access_key_id = root.find('.//ns:AccessKeyId', namespace).text
secret_access_key = root.find('.//ns:SecretAccessKey', namespace).text
session_token = root.find('.//ns:SessionToken', namespace).text

os.environ["AWS_ACCESS_KEY_ID"] = access_key_id
os.environ["AWS_SECRET_ACCESS_KEY"] = secret_access_key
os.environ["AWS_SESSION_TOKEN"] = session_token

NOTE: Use the credentials that you used to connect to the datalab.

Create a token with customize expiration (or no expiration)

Connect to the minio-console and use “Login with SSO” to access it. You will have access to your personal MinIO S3 account. On the navigation bar, click on Access Keys under the User section and then create a new access key without expiration (or with something that fit your needs).

From there, you are also strongly encourage to customize the S3 policy you give to those credentials. Indeed, you should restrict to the minimum needed rights you want. You can learn more about what you can do and how S3 policies are working by reading this.

Once created, you can use those credentials to access you storage wherever you want (locally, from an EDITO service or process), or using the library you want (minio client, aws, boto3…).

Notes about how to configure your EDITO service or process with token generated from the MinIO Console

In the configuration of an EDITO service or process that is configured to allow S3 configuration, the fields in the “S3 configuration” section are automatically configured with your project settings. If you want to use the credentials you generated, you can either change the project configuration, or directly edit the service or process “S3 configuration”. In both case, the “SessionToken” field is not needed (leave it or make it empty).

The S3 endpoint should be minio.dive.edito.eu, the region should be waw3-1, and the Secret Access Key and Access Key ID should be the one generated with the MinIO console.