Managing Usage Groups via Terraform

Learn how to manage your Usage Groups via Infrastructure as code with Terraform

SELECT offers both an API and a Terraform provider, which can be utilized to manage usage groups within your organization.

This guide presents a way to migrate from usage groups configured in the SELECT UI to managing those resources in Terraform. We provide a Python script which will generate Terraform resource blocks based on what is already configured in the SELECT UI, simplifying this process greatly.

Here's an overview of the process:

  1. Obtain an API key from the SELECT UI
  2. Download the helper script and set up its dependencies
  3. Run the script with your API key to generate Terraform configuration for all usage groups in your organization
  4. (Optional) Move the generated Terraform resources into your existing Terraform repository
  5. (IMPORTANT) Move the API key to secure storage
  6. Run the generated import.sh which will pull all your usage groups into your Terraform state.
  7. Iteratively execute terraform plan and adjust the generated resources until they match what's configured in SELECT

Step by step guide

Prerequisites

Before getting started, you'll need:

1. Obtain an API Key from the SELECT UI

Please see the API quickstart for guidance on how to obtain this from the SELECT UI. The API key you generate will need read/write access to any resources you want to manage in Terraform. For this guide, at a minimum, the usage group and the usage group set read/write access are required.

2. Download and setup the script

# Clone the repository or download the migration folder

git clone https://github.com/get-select/terraform-provider-select.git

cd terraform-provider-select/scripts/usage_groups_migration



# setup and activate a virtual environment

python3 -m venv .venv

source .venv/bin/activate



# Install dependencies

pip install -r requirements.txt

3. Generate terraform configuration

Replace YOUR_API_TOKEN and YOUR_ORG_ID with your actual credentials:

python generate_usage_group_resources.py --token YOUR_API_TOKEN --org-id YOUR_ORG_ID

This creates a select_usage_groups directory with:

  • main.tf - Provider configuration
  • {usage_group_set}.tf - One file per usage group set
  • import.sh - Script to import existing resources

4. Move the generated Terraform resource files

If you would like to manage SELECT resources alongside your other infrastructure, please move this directory into the corresponding repository now.

If you are setting up a new repository for managing SELECT resources, you can move this directory to the new repository now.

Once you've moved the generated resource files, you can remove the script or the entire terraform repository if you cloned it earlier.

5. Secure your API key

⚠️ IMPORTANT: The generated main.tf contains your API key in plaintext.

If you already manage resources in Terraform it's likely that you have a strategy for securing sensitive variables. Otherwise, a simple way to secure this is to use environment variables.

cd select_usage_groups



# Move API key to environment variable

export TF_VAR_select_api_key="YOUR_API_TOKEN"



# Edit main.tf to use the variable instead of hardcoded key

# Change: api_key = "YOUR_API_TOKEN"

# To: api_key = var.select_api_key

There are alternatives and Hashicorp has extensive documentation on this topic.

6. Initialize and import

The python script generated a command to import the resources into your Terraform state.

# Initialize terraform

terraform init



# Import existing resources into terraform state

# You may need to make the script executable first: chmod +x ./import.sh

./import.sh



# Check for configuration drift

terraform plan

7. Fix any configuration drift

If terraform plan shows differences, update your .tf files to match your existing resources until the plan shows no changes.

# After making changes, verify

terraform plan

# Should show: "No changes. Your infrastructure matches the configuration."

Examples


The script above will generate a module per snowflake account or organization that API key used has access to, and create a file per Usage Group Set.

To see some examples of what the Terraform resources for Usage Groups looks like, please visit the Terraform provider docs.