Skip to main content

Set up AWS Python SDK (boto3) to assume roles with MFA and interact with GDS AWS

Here we go through how to use Python boto3 to interact with GDS AWS by assuming an AWS IAM Role that has permissions your AWS user account does not have (e.g., accessing S3). It assumes that MFA is also required.

Assuming a role means that the AWS token service will give you temporary credentials to access the (GDS) AWS account with an assumed role.

GDS AWS Requirements

  1. Ensure you have access to the GDS AWS Account. Follow these instructions if you have not already done so as part of your onboarding: GDS - Get AWS Access. At the end, you will have created your AWS user account, and also received an AWS access key ID and secret access key.

  2. Get STS Permission to AssumeRole with MFA for the role you want to assume.

    For instance, if you are a Data Scientist in CPTO, you may want to assume the govuk-datascienceusers IAM Role. Ask on the #data-engineering Slack channel to get this permission.

boto3 Requirements

Ensure you have a python environment activated.

Install the python package boto3.

Configure aws credentials:

```shell
aws configure
```

and follow the prompts. NOTE: you will be asked to provide your AWS access key ID and secret access key so have them ready (see AWS SDKs Configurations for more info. Please also provide eu-west-1 when asked for region.

Alternatively, you can set up your configurations and credentials as (secret) environment variables.

Set up boto3

These are the basic steps to generate temporary credentials in boto3 via AssumeRole-with-MFA and use them to make a connection to Amazon S3.

  1. Create an STS client object, representing a live connection to the STS service

    sts_client = boto3.client("sts")

  2. Define the ARN of the role we want to assume (e.g., govuk-datascienceusers), substituting appropriate values for ROLE_ACCOUNT_ID and ROLE_NAME:

    role_arn = f"arn:aws:iam::{ROLE_ACCOUNT_ID}:role/{ROLE_NAME}"

  3. Ask for the MFA token:

    MFA_OPT = input("Enter the MFA code: ")

  4. Call the assume_role method of the STSConnection object and pass the role ARN and a chosen role session name:

        assumedRoleObject = sts_client.assume_role(
    RoleArn=role_arn,
    RoleSessionName=f"{SESSION_X}",
    SerialNumber=f"arn:aws:iam::{USER_ACCOUNT_ID}:mfa/{USERNAME}@digital.cabinet-office.gov.uk",
    DurationSeconds=3600,
    TokenCode=MFA_OPT,
    )

    providing appropriate values for SESSION_X (your choice here, could be for instance Session_Alessia), USER_ACCOUNT_ID and USERNAME. If you are unfamiliar with what your USER_ACCOUNT_ID and USERNAME are, sign in to the GDS AWS Management Console. In the navigation bar on the upper right, click on your email/username and then go to “My Security Credentials”, you will find them in there.

  5. From the response that contains the assumed role, get the temporary credentials:

    temp_credentials = assumedRoleObject["Credentials"]

  6. You can now use the temporary credentials to, for instance, connect to S3:

    s3_resource = boto3.resource(
"s3",
aws_access_key_id=temp_credentials["AccessKeyId"],
aws_secret_access_key=temp_credentials["SecretAccessKey"],
aws_session_token=temp_credentials["SessionToken"],
)

    And for instance list all S3 buckets:

    buckets = [bucket.name for bucket in s3_resource.buckets.all()]
print(buckets)

Helper functions for boto3

You can find examples of helpers functions (wrappers for the steps above) in github.com/alphagov/govuk-content-metadata/src/utils/helpers_aws.py.

This page was last reviewed on 7 September 2022. It needs to be reviewed again on 7 September 2023 .
This page was set to be reviewed before 7 September 2023. This might mean the content is out of date.