# SELECT Documentation

> Documentation for SELECT, a cost and performance platform for Snowflake, Databricks and BigQuery. This file has the full text of every docs page. The index of all pages is at https://select.dev/docs/llms.txt.

---

# Welcome to SELECT

URL: https://select.dev/docs/reference/setup-guide/welcome-to-select

> Learn about how to get started with SELECT

## Why SELECT?

Snowflake is an incredibly powerful platform, easily scaling to handle ever-larger data volumes without compromising on performance. But, if not controlled, costs associated with this scaling quickly climb. Whether your goal is to reduce the price of an upcoming renewal, extend your existing contract's runway, or reduce on-demand costs, SELECT can help.

SELECT is a SaaS web application that helps Snowflake users maximize ROI by providing visibility into spend, optimizing performance and automatically lowering costs. You can [explore all of our features here](https://select.dev/docs/reference/using-select), or learn more about our company and frequently asked questions [here](https://select.dev/about).

## Get Started

Snowflake customers can get up and running with SELECT in less than 15 minutes.

1. [Sign up](https://select.dev/signup?source=%2Fdocs&source_location=get_started) for immediate access, or [book a demo](https://select.dev/demo) if you'd like to see the product in action first.
2. [Create a new Snowflake user with metadata access](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-1-create-snowflake-service-user)
3. [Connect your Snowflake account in the SELECT web application UI](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-2-connect-your-snowflake-account)

That's it!

> [!NOTE]
> Reach out to our team at any point if you have questions or need help using the chat button in the bottom right.

---

# Connect your Snowflake Account

URL: https://select.dev/docs/reference/setup-guide/connect-snowflake-account

> Create a new Snowflake User and connect your Snowflake account to SELECT

[YouTube video](https://www.youtube.com/embed/BYVPZS4QFkk?rel=0&controls=0)

Connecting your Snowflake account to SELECT is an easy process:

1. **Step 1:** You'll create a new Snowflake service user with read only access to your Snowflake metadata and a few other permissions.
2. **Step 2:** You'll connect your Snowflake account to SELECT using the user's credentials.

## Step 1: Create Snowflake Service User

As outlined in our [security documentation](https://select.dev/security), SELECT requires a new Snowflake service user to be created with read-only access to your [Snowflake account metadata database](https://docs.snowflake.com/en/sql-reference/snowflake-db.html). 

### 1. Create key pair for service user

First, create a new [key pair](https://select.dev/docs/resources/snowflake-developer-guide/snowflake-key-pair) that will be used as the authentication method for the new user created by the script.  See our [guide](https://select.dev/docs/resources/snowflake-developer-guide/snowflake-key-pair) on how to create the key pair.

### 2. Run the setup script

Now that we have created the user's key pair credentials, we can run the setup script.

The script below handles creating the new user, role and associated permissions used by SELECT. 

The script has plenty of code comments to help you understand what is happening. You can refer to the setup script explanation [section below](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#setup-script-explanation) for more details.

Please fill out the variable values in lines 1-22 and **provide the public key on line 26.**  

```sql
-- Attention users:  Edit variable values up to line 26! Do not change variable names, only change the values to the right of the equals sign.
SET select_role = 'SELECT_ROLE';
SET select_warehouse = 'SELECT_WH'; -- you can use an existing warehouse if you'd like.
SET select_user_name = 'SELECT_USER';

/* 
SELECT uses a storage integration and a stage to access your Snowflake usage data.
For more info, see docs:
https://select.dev/security/model#select-architecture

Also, SELECT uses a stored procedure which tells you avg depth and cluster scores of each table.

If you would like the storage integration, stage, and procudure in an isolated database, type a new database and schema name below.
If you'd like this in an existing database, type the existing database and schema name.

*/

SET select_database = 'SELECT_DEV';
SET select_schema = 'UTILS';
SET storage_integration_name = 'SELECT_STORAGE_INTEGRATION';
SET describe_object_procedure = 'SPROC_DESCRIBE_OBJECT';
SET stage_name = 'SELECT_EXPORT';

use role useradmin;

-- ‼️ edit the public key line below ‼️
CREATE USER IF NOT EXISTS identifier($select_user_name)
  RSA_PUBLIC_KEY = '‼️ your public key here‼️ '
  DEFAULT_ROLE = $select_role
  DEFAULT_WAREHOUSE = $select_warehouse
  COMMENT = 'Used by select.dev'
  type = 'SERVICE';

-- No edits are required below this line.
--------------------------------------------

-- This is setting the bucket name, which is used in the bucket name for the stage.
-- CRITICAL: Do not modify this gcs_path variable!! Leave GCS even if you use Azure or AWS.
-- You do not need to create the bucket, it will be created for you.
SET gcs_path = (
  SELECT 
    'gcs://' || 
    CASE 
      WHEN RIGHT(bucket_base, 1) = '_' 
      THEN LEFT(bucket_base, LENGTH(bucket_base) - 1)
      ELSE bucket_base
    END || '/'
  FROM (
    SELECT 
      LEFT(
        'select_ingest_' || 
        regexp_replace(lower(CURRENT_ORGANIZATION_NAME() || '_' || CURRENT_ACCOUNT_NAME()), '\\W', '_'),
        63
      ) AS bucket_base
  )
);


USE ROLE ACCOUNTADMIN;

------- create role and warehouse used by SELECT --------------
CREATE ROLE IF NOT EXISTS identifier($select_role) COMMENT = 'Used by select.dev';

CREATE WAREHOUSE IF NOT EXISTS identifier($select_warehouse)
  WAREHOUSE_SIZE = XSMALL
  AUTO_SUSPEND = 60
  INITIALLY_SUSPENDED = TRUE
  COMMENT = 'Used by select.dev';


GRANT ROLE identifier($select_role) TO USER identifier($select_user_name);

GRANT MONITOR, OPERATE, USAGE, MODIFY
  ON WAREHOUSE identifier($select_warehouse) TO ROLE identifier($select_role);

-- Grant additional privileges needed by SELECT
-- Grant read-only access to the Snowflake metadata database
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE identifier($select_role);
-- Grants the ability to monitor account-level usage and historical information for databases and warehouses
GRANT MONITOR USAGE ON ACCOUNT TO ROLE identifier($select_role);
-- Grant the ability ability to monitor warehouse settings through the manage warehouses permission
-- This permission is also required for SELECT to retrieve query statistics using get_query_operator_stats, and for the Automated Savings feature
GRANT MANAGE WAREHOUSES ON ACCOUNT TO ROLE identifier($select_role); 

-------- Create DB and schema to house SELECT objects --------------
CREATE DATABASE IF NOT EXISTS identifier($select_database);
GRANT USAGE ON DATABASE identifier($select_database) TO ROLE identifier($select_role);

USE DATABASE identifier($select_database);

CREATE SCHEMA IF NOT EXISTS identifier($select_schema);

GRANT USAGE ON SCHEMA identifier($select_schema) TO ROLE identifier($select_role);

USE SCHEMA identifier($select_schema);

-- Create the GCS storage integration
-- Do not change to AWS or Azure, SELECT's Snowflake account is in GCS and this enables copying data to SELECT's account
CREATE STORAGE INTEGRATION IF NOT EXISTS identifier($storage_integration_name)
  TYPE = EXTERNAL_STAGE
  STORAGE_PROVIDER = 'GCS' -- do not change!
  ENABLED = TRUE
  STORAGE_ALLOWED_LOCATIONS = ($gcs_path); -- the storage integration will only be used for the dedicated GCS bucket created on your behalf

GRANT USAGE ON INTEGRATION identifier($storage_integration_name) TO ROLE identifier($select_role);

--Create stage
CREATE OR REPLACE STAGE identifier($stage_name)
  URL = $gcs_path
  STORAGE_INTEGRATION = $storage_integration_name;

GRANT USAGE ON STAGE identifier($stage_name) TO ROLE identifier($select_role);

-- The stored procedure below used to analyze clustering statistics for your table
-- It is optional, but helpful to have
CREATE OR REPLACE PROCEDURE identifier($describe_object_procedure) (
        object_type varchar, -- Generic, so SELECT can describe tables, dynamic tables and materialized views
        object_fqn varchar, -- The full path of the object to be described (database_name.schema_name.object_name)
        run_describe_object boolean, -- Whether to call "describe table" or not
        add_clustering_information boolean, -- Whether to include clustering information in the results or not
        columns_filter_clustering_information array -- Sometimes SELECT only needs clustering information on certain columns
    )
    RETURNS variant
    LANGUAGE javascript
    EXECUTE AS OWNER
AS
$$
    /**
    * Function to call "describe table" and parse the results.
    */
    function describeObject(objectType, objectFqn) {
        const sql = `describe ${objectType} ${objectFqn}`

        const statement = snowflake.createStatement({ sqlText: sql })
        const resultSet = statement.execute()

        const columnsInResult = []
        for (let columnIndex = 0; columnIndex < statement.getColumnCount(); columnIndex++) {
            columnsInResult.push(statement.getColumnName(columnIndex + 1)) // Starts from 1
        }

        const output = [];

        for (let rowIndex = 0; resultSet.next(); rowIndex++) {
            let row = { _index: rowIndex }
            columnsInResult.forEach(column => row[column] = resultSet.getColumnValue(column))
            output.push(row)
        }

        return output
    }

    /**
    * Function to call "system$clustering_information" and parse the results.
    * Snowflake requires one call per column, but we combine all calls into one SQL statement
    * using UNION ALL.
    */
    function extendColumnsWithClusteringInformation(tableFqn, columns, runDescribeObject, columnsFilterClusteringInformation) {
        let filteredColumns

        if (runDescribeObject) { // If describe object has been called
            // Filter out unsupported column types (causes errors on Snowflake)
            filteredColumns = columns.filter(column => !['OBJECT', 'VARIANT', 'ARRAY'].includes(column.type))

            // If a list of columns to filter on is provided, use it
            if (columnsFilterClusteringInformation)
                filteredColumns = filteredColumns.filter(column => columnsFilterClusteringInformation.includes(column.name))
        } else { // If describe object hasn't been called, use the columns provided in the filter
            // Convert it to the expected format
            filteredColumns = columnsFilterClusteringInformation.map(columnName => ({ name: columnName }))
        }

        if (!filteredColumns.length) return

        const sql = filteredColumns.map(column =>
            `select ${column?._index || 0} as column_index, '${column.name}' as column_name, system$clustering_information('${tableFqn}', '("${column.name}")') as clustering_information`
        ).join("\nunion all\n")
        const resultSet = snowflake.execute({ sqlText: sql })

        while (resultSet.next()) {
            let colIndex = resultSet.getColumnValue(1)
            let clusteringInformation = JSON.parse(resultSet.getColumnValue(3))

            if (runDescribeObject) { // If there's already data available from describe object, extend it
                columns[colIndex] = { ...columns[colIndex], clustering_information: clusteringInformation }
            } else { // Otherwise, append to the array
                columns.push({ clustering_information: clusteringInformation })
            }

        }
    }

    /**
    * Function to call "system$clustering_information" without any expression, which defaults
    * to the existing clustering key if it's defined in the table.
    */
    function fetchClusteringKeyInformation(tableFqn) {
        try {
            const resultSet = snowflake.execute({ sqlText: `select system$clustering_information('${tableFqn}')` })
            resultSet.next()
            return JSON.parse(resultSet.getColumnValue(1))
        } catch (err)  {
            // "Invalid clustering keys or table ... is not clustered"
            return null;
        }
    }

    let columns = null
    let clusteringKeyClusteringInformation = null

    if (RUN_DESCRIBE_OBJECT) {
        columns = describeObject(OBJECT_TYPE, OBJECT_FQN)

        // If the ADD_CLUSTERING_INFORMATION argument is true, append the output with
        // the results from system$clustering_information
        if (ADD_CLUSTERING_INFORMATION) {
            extendColumnsWithClusteringInformation(OBJECT_FQN, columns, true, COLUMNS_FILTER_CLUSTERING_INFORMATION)
            clusteringKeyClusteringInformation = fetchClusteringKeyInformation(OBJECT_FQN)
        }
    } else if (ADD_CLUSTERING_INFORMATION && COLUMNS_FILTER_CLUSTERING_INFORMATION) {
        // If only one column is passed as a filter and it is an empty string, fetch clustering data about
        // the clustering key
        if (COLUMNS_FILTER_CLUSTERING_INFORMATION?.length === 1 && COLUMNS_FILTER_CLUSTERING_INFORMATION[0] === '') {
            clusteringKeyClusteringInformation = fetchClusteringKeyInformation(OBJECT_FQN)
        } else {
            columns = []
            extendColumnsWithClusteringInformation(OBJECT_FQN, columns, false, COLUMNS_FILTER_CLUSTERING_INFORMATION)
        }
    }

    return { clusteringKey: clusteringKeyClusteringInformation, columns: columns, version: 'v1' }
$$;


-- Grant the procedure to the role used by SELECT
SET full_proc = $select_database || '.' || $select_schema || '.' || $describe_object_procedure;
SET grant_sql = 'GRANT USAGE ON PROCEDURE ' || $full_proc || '(VARCHAR, VARCHAR, BOOLEAN, BOOLEAN, ARRAY) TO ROLE ' || $select_role;
select $grant_sql; -- optional, just so you can see what it will do.
EXECUTE IMMEDIATE $grant_sql;


-------------------------------------------------------------------
-------------------------------------------------------------------
--------------- END OF SCRIPT ---------------------------------------


-- Run the select statment below to assist in filling out the connect account form in the SELECT UI:

SELECT {
    'account_identifier': CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME(),
    'user': $select_user_name,
    'stage_name': $select_database || '.' || $select_schema || '.' || $stage_name,
    'procedure_name': $full_proc
} as select_form_data; 
```

### 3. Verify your account has access to organization_usage views

SELECT relies on a few views from the `snowflake.organization_usage` schema. For example, `snowflake.organization_usage.rate_sheet_daily` is used to look up your daily rate for each Snowflake service (i.e. your cost per credit for compute, cost per terabyte for storage, etc).

Please run the following SQL statement to check if your account has these views populated:

```sql
use role accountadmin;

select *
from snowflake.organization_usage.rate_sheet_daily;
```

If records are returned, you're all set.

If no records are returned, the ORGADMIN role likely isn't enabled for the account. To enable it, log into a Snowflake account in your organization which has ORGADMIN enabled (at least one account will), then head to Admin -> Accounts in the sidebar. Then, enable ORGADMIN for the account you're connecting to SELECT.

![Snowflake organization admin role](https://cdn.sanity.io/images/wn0ylp2c/production/6e3b847271c5959812717ff11b63f098c7fe7d0b-1270x793.png)

It can take up to 1 day for these views to become populated with data after running this statement. As a result, you may not see data in your Snowflake account until the morning after connecting your account. Reach out to our team using the Intercom chat on the website if you're still not seeing data after 1 day.

> [!TIP]
> Users who purchased their Snowflake contract/credits through a reseller (i.e. AWS Marketplace) will not have access to the `organization_usage` views even after running the command above. In this case, you can manually add your cost per credit and storage cost in the "Advanced" section of the "[Add New Snowflake Account](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-2-connect-your-snowflake-account)" screen.

### 4. [Optional] Update Network Policies

If your Snowflake account has configured a network policy to restrict access based on IP address, you'll need to update the network policy to allow SELECT to connect.

In Snowsight, go to **Admin » Security » Network Policies**.  From there, add the following IP addresses to your **Allowed IP Addresses** list:

```sql
34.23.79.180
34.139.189.198
35.229.105.183
35.243.245.125
34.138.184.161
34.73.135.174
34.148.143.80
34.23.67.112
```

You can also apply the changes via SQL instead. The example below shows how to create the policy and assign it to the SELECT service user. Be sure to update the policy and user names as needed:

```sql
CREATE OR REPLACE NETWORK POLICY select_network_policy
  ALLOWED_IP_LIST = (
    '34.23.79.180',
    '34.139.189.198',
    '35.229.105.183',
    '35.243.245.125',
    '34.138.184.161',
    '34.73.135.174',
    '34.148.143.80',
    '34.23.67.112'
  );

ALTER USER select_user SET NETWORK_POLICY = select_network_policy;

```

#### Updating an account wide network policy

If you're updating a network policy that is already active for all users in your Snowflake account, **you do not need to reassign the policy** to the SELECT user. Simply adding the new IPs is enough.

For more details, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/network-policies.html) on Network Policies, or reach out using the chat button on the bottom right if you run into any issues.

### Setup Script Explanation

In the sections below, we describe the setup script in more detail.

#### Create User, Role, and Warehouse

The first action taken by the script (after you set your variables) is to create a new user and role that will only be used by SELECT. 

If you'd like SELECT to use an existing warehouse, set line 3 to an existing warehouse name.  Otherwise, the warehouse indicated on line 3 will be created as an extra-small warehouse. 

After the user, role, and warehouse are created:

- The role is granted read-only access to the Snowflake metadata database.
- The role is also grant ability to monitor warehouse settings through the [manage warehouses permission](https://docs.snowflake.com/en/user-guide/warehouses-tasks#label-grant-manage-warehouses-privilege). This permission is also required for SELECT to retrieve query statistics using [get_query_operator_stats](https://docs.snowflake.com/en/sql-reference/functions/get_query_operator_stats) and for the [Automated Savings](https://select.dev/features/automated-savings) feature.

#### Create a Storage Integration and Stage used by SELECT

The script will create a `stage` and `storage integration` in your Snowflake account that will be used to give SELECT access to your Snowflake spend metadata.  The bucket used in Google Cloud Storage will be created on the fly by SELECT when you connect your Snowflake account.  There is nothing else you need to do to provide access.

Google Cloud Storage is used as an intermediary between SELECT's Snowflake account and the customer's. This is done because the [Snowflake account usage views](https://select.dev/security#what-snowflake-metadata-do-we-access) used by SELECT cannot be data shared directly, and the customer accounts may be in different regions which would require setting up data replication. Using Google Cloud Storage makes setup a lot simpler and more cost-effective for the customer.

As per our [security documentation](https://select.dev/security/model), each customer's Snowflake metadata is stored in a dedicated Google Cloud Storage bucket. Data is stored in the us-east-1 Google Cloud region. There is a [data egress cost](https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf) charged by Snowflake, but it's incredibly small for the metadata volumes we're processing (the highest we've seen across our customer base is $20 per year).

The image below shows how the Snowflake Metadata is unloaded to GCS in SELECT's infrastructure using this `storage integration`.

![SELECT security architecture](https://cdn.sanity.io/images/wn0ylp2c/production/318e36376593d859341a5efa46b829a04ba1f6e7-2332x881.png)

These objects will get created in the database and schema you specified on lines 18 & 19.  If you used a new database / schema name, the database and Schema will be created at this time.  If you used an existing database and Schema name, the SELECT role will be granted usage in the existing database and only the new objects will be created.

#### Create a Stored Procedure used by SELECT

Some features in SELECT require information about objects in your account by:

- Running a [Describe Table](https://docs.snowflake.com/en/sql-reference/sql/desc-table) statement in your Snowflake account to fetch additional metadata from columns in a particular object (table, view, materialized table, etc).
- Running Snowflake's system function [system$clustering_information](https://docs.snowflake.com/en/sql-reference/functions/system_clustering_information) to fetch metadata about the clustering health status of columns in a table.

Instead of granting the necessary permissions to the `select_user` on all your existing objects and using future grants to also include new objects (which can conflict with your existing future grants), we ask our customers to create a custom stored procedure for when SELECT needs to run the commands above.

Just like above, these objects will be created in the database and schema specified on lines 18 and 19, but it needs to be created with the `accountadmin` role, as SELECT relies on the `execute as owner` property.

## Step 2: Connect your Snowflake Account

Now that we've created the new Snowflake user and associated resources, we can plug those details into the SELECT UI under Settings -> Accounts -> Add Snowflake account.

![SELECT connect Snowflake account](https://cdn.sanity.io/images/wn0ylp2c/production/a6f9ba24eced8d574249d0d2c946a04a64e87bf3-1890x1647.png)

The last few lines in the script above contain a `select` statement you can run to assist you in copy / pasting into the form in SELECT.  When me mention "script output" below, we are referring to this:

![Details needed for connecting SELECT app to Snowflake](https://cdn.sanity.io/images/wn0ylp2c/production/f7d49b4ee6efb5bbab555982114c3f2e00386878-1311x352.png)

Here are some more details on each field:

- **Friendly Account Name:** You can put whatever you want here. If you only have one Snowflake account, then it’s common to just put your company name, for example: “Cyberco”. If you plan to connect multiple Snowflake account to SELECT, name them in a way you’ll be able to differentiate them using the account switcher on the top left. For example, you could have “Cyberco Prod” and “Cyberco Dev”.
- **Account Identifier:** An account identifier uniquely identifies a Snowflake accounts within your organization. There are two ways to find it, [outlined below](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#locating-your-snowflake-account-identifier).  Or, from the worksheet you ran the onboarding script, copy / paste the account identifier from the script output, shown in the screenshot above.  (Near line 241 of the script.)
- **Username:** This will be the name of the new Snowflake user you created for SELECT. For example, `SELECT_USER`. This is the variable you defined on line 4 and can be fetched using the final `select` statement in the script.
- **Private Key:** Paste the entire Secret Key you created in step 1.  You should copy / paste the entire thing as-is, including the Begin and End tags as well as all of the line breaks.
- **Passphrase:** If you created an encrypted private key, put the passphrase here.  Leave blank if you opted for an unencrypted private key.
- **[OPTIONAL] Role:** The script applies a default role to the user, so you can leave this blank if you ran the script as-is.  Or, fill this out to meet your requirements.
- **[OPTIONAL] Warehouse:** The script applies a default warehouse to the user, so you can leave this blank if you ran the script as-is.  Or, fill this out to meet your requirements.
- **Stage:** In the drop down for Export Method, leave it as **Stage**, and pass the stage name returned from the script output on line 241.
   - The stage name must be fully qualified.

![](https://cdn.sanity.io/images/wn0ylp2c/production/a9e95da6a2d6c70479abdd9d62ec7870c133d4d6-595x186.png)

- **Procedure Name:** To get the Procedure Name, copy and paste the result from the script output.
- **Advanced:** Only fill out the "Advanced" section if Snowflake was purchased through a reseller (i.e. AWS Marketplace) and you do not have access to the `organization_usage` views, mentioned [above](https://select.dev/docs/reference/setup-guide/account-setup-guide#3-verify-your-account-has-access-to-organization_usage-views).

![](https://cdn.sanity.io/images/wn0ylp2c/production/756d72ea1422b638d501d0dae5459f2a6bb339d1-560x299.png)

Once you click Add, the app will let you know if there were any errors.  In most cases the error message should be clear, but reach out to us via the in-app chat if there are any problems you cannot clear up on your own.

## Additional resources

### Workload identity federation

SELECT supports Snowflake’s [Workload identity federation](https://docs.snowflake.com/en/user-guide/workload-identity-federation) mechanism which is more secure than traditional methods like username/password or key-pair auth since it doesn’t involve sharing any secrets with SELECT.

To leverage this method, select “Workload Identity Federation” under the authentication method section. You will then be shown a SELECT Service Account ID (these are specific to each organization).

![](https://cdn.sanity.io/images/wn0ylp2c/production/dc70892cd6e2946d69b66ce4eb495071cad83abc-3334x2142.png)

On the Snowflake side, rather than setting a password or public key, configure the Snowflake service user you created for SELECT with the workload_identity attribute set as follows:

```sql
create user select_user
  workload_identity = (
    type = gcp
    subject = 'REPLACE_ME' -- Replace with the SELECT Service Account ID from the form
  )
  type = service;

```

No additional action is required outside of changing the way the Snowflake user is initially setup. Everything else in the setup script can be followed as normal.

To switch an existing password/key-pair user to workload identity, you can run:

```sql
alter user select_user set type = 'SERVICE';
alter user select_user set workload_identity = (
    type = gcp
    subject = 'REPLACE_ME' -- Replace with the SELECT Service Account ID from the form
  )
```

### Locating your Snowflake Account Identifier

In the bottom left corner if the Snowsight UI, click your User Name / Profile, then choose "Connect a tool to Snowflake". On the subsequent screen you will find your account identifier.

![](https://cdn.sanity.io/images/wn0ylp2c/production/27537e6dc7394916482ee42ebc3431bd04c66b6e-287x619.png)

![](https://cdn.sanity.io/images/wn0ylp2c/production/d96cc28ae69f7aa5f87f86fe5063643fde2405b4-901x649.png)

### Notes

  We recommend using a tool like 1Password to both randomly generate and save the password. Alternatively, you can use a [password generator website](https://passwordsgenerator.net/) and store the password securely elsewhere.

![](https://cdn.sanity.io/images/wn0ylp2c/production/d6834e348775868b2d078411bf9456d164136515-1860x474.png)

---

# Connect your Databricks Workspace

URL: https://select.dev/docs/reference/setup-guide/connect-your-databricks-workspace

> Create a new Databricks Service Principal and connect your Databricks workspace to SELECT

## Prerequisites

- You need **Account Admin** privileges and **Metastore Admin** privileges for your Databricks workspace
   - [https://docs.databricks.com/en/administration-guide/index.html](https://docs.databricks.com/en/administration-guide/index.html)

![](https://cdn.sanity.io/images/wn0ylp2c/production/6bab78cffcf65efc9a9079e021ef64c18642876c-1133x687.png)

- A Unity Catalog-enabled workspace
   - [https://docs.databricks.com/aws/en/data-governance/unity-catalog/get-started#how-do-i-know-if-my-workspace-was-enabled-for-unity-catalog](https://docs.databricks.com/aws/en/data-governance/unity-catalog/get-started#how-do-i-know-if-my-workspace-was-enabled-for-unity-catalog)

## Connecting the workspace

### Manual Setup

#### Step 1: Get account and workspace identifiers

- Head to the Databricks console
   - [https://accounts.cloud.databricks.com/](https://accounts.cloud.databricks.com/)
- Note down your Databricks **Account ID** from the menu on the top right profile icon
   - [https://docs.databricks.com/aws/en/admin/account-settings/#locate-your-account-id](https://docs.databricks.com/aws/en/admin/account-settings/#locate-your-account-id)
- Click **Workspaces**, select a Unity Catalog-enabled workspace
- Note down your **Workspace URL**

#### Step 2: Create a service principal

- In the workspace, select **Settings** from the top right Profile icon
- Under **Workspace admin**, select **Identity and access**
- Next to **Service Principals**, click **Manage**
- Click **Add service principal** 
   - An **Add new service principal** modal pops up, click **Add new**
   - Give the service principal a memorable name (e.g. `select-service-principal`)
- Open the newly created service principal, select **Secrets** tab, then click **Generate secret**
- Enter lifetime of 730 days, then click **Generate**
   - you will need to create a new secret and update it in SELECT when it expires
- Note down your **Secret** and **Client ID**

#### Step 3: Add the service principal to the `databricks_pii_access` group

By default, Databricks masks SQL statement text in system tables (`system.query.history.statement_text` returns `<REDACTED>`) for any principal outside the account-level `databricks_pii_access` group. Adding SELECT’s service principal to this group lets SELECT read your query text, which powers query patterns, query insights, and AI summaries.

> [!NOTE]
> **This is done in the Databricks** **_account console_, not workspace settings — you must be an account admin.** The group is account-level; creating it under Workspace admin settings will not work, and the group name is case-sensitive.

- Go to the Databricks account console ([`https://accounts.cloud.databricks.com`](https://accounts.cloud.databricks.com/), or `accounts.azuredatabricks.net` / `accounts.gcp.databricks.com` for Azure/GCP) and sign in as an **account admin**
- In the sidebar, select **User management**
- On the **Groups** tab, click **Add group**
- Enter the name exactly as `databricks_pii_access` (case-sensitive), then click **Confirm**
   - If a `databricks_pii_access` group already exists (e.g. created for another connection), skip creating it and open the existing group instead
- Open the group, click **Add members**, search for the service principal you created in Step 2 (e.g. `select-service-principal`), select it, and click **Add**
- Membership takes about **10 minutes** to propagate before query text becomes readable

#### Step 4: Create a Serverless SQL Warehouse

- Navigate to **SQL**→**SQL Warehouse** from the navigation menu
- Click **Create SQL warehouse** with the following config 
   - Name: pick a memorable name (e.g. `select-sql-warehouse`)
   - Cluster size: **2X-Small**
   - Type: **Serverless**
   - Auto-stop: set to **5 minutes** to minimize idle costs (can be reduced to 1 minute via the [Databricks Warehouses API](https://docs.databricks.com/api/workspace/warehouses/edit) after creation — the UI minimum is 5 minutes)

![](https://cdn.sanity.io/images/wn0ylp2c/production/e5e15528379e18e55639b078b3791aa9bc238e2e-1786x994.png)

- Click **Create**
- After the warehouse is created, the **Manage permissions** modal window is shown
   - You can also access this modal by clicking **Permissions**
- Search for and select the service principal you created (using the name or client id).
- Select **Can Use** permission, click **Add** and close the modal
- Note down the **Warehouse ID**

![](https://cdn.sanity.io/images/wn0ylp2c/production/e43954db2bcb3d5ac6cfe7712ad78f0636d355a9-782x592.png)

#### Step 5: Grant Read Access to `system` catalog tables

Grant the service principal the exact privileges SELECT needs:

- `USE CATALOG` on the `system` catalog
- `USE SCHEMA` on each schema we read from
- `SELECT` on each table we read from

Open a SQL editor in your Databricks workspace (attached to the warehouse created in Step 3), find/replace `<service-principal-client-id>` with the \*\*Client ID\*\* noted in Step 2, then run:

```sql
-- Catalog
grant use catalog on catalog system to `<service-principal-client-id>`;

-- Schemas
grant use schema on schema system.query    to `<service-principal-client-id>`;
grant use schema on schema system.billing  to `<service-principal-client-id>`;
grant use schema on schema system.compute  to `<service-principal-client-id>`;
grant use schema on schema system.access   to `<service-principal-client-id>`;
grant use schema on schema system.lakeflow to `<service-principal-client-id>`;

-- Tables
grant select on table system.query.history                  to `<service-principal-client-id>`;

grant select on table system.billing.usage                  to `<service-principal-client-id>`;
grant select on table system.billing.account_prices         to `<service-principal-client-id>`;  -- skip on Azure
grant select on table system.billing.list_prices            to `<service-principal-client-id>`;

grant select on table system.compute.warehouses             to `<service-principal-client-id>`;
grant select on table system.compute.warehouse_events       to `<service-principal-client-id>`;
grant select on table system.compute.clusters               to `<service-principal-client-id>`;
grant select on table system.compute.node_timeline          to `<service-principal-client-id>`;

grant select on table system.access.table_lineage           to `<service-principal-client-id>`;
grant select on table system.access.column_lineage          to `<service-principal-client-id>`;
grant select on table system.access.workspaces_latest       to `<service-principal-client-id>`;

grant select on table system.lakeflow.job_run_timeline      to `<service-principal-client-id>`;
grant select on table system.lakeflow.job_task_run_timeline to `<service-principal-client-id>`;
grant select on table system.lakeflow.jobs                  to `<service-principal-client-id>`;
grant select on table system.lakeflow.job_tasks             to `<service-principal-client-id>`;
grant select on table system.lakeflow.pipelines             to `<service-principal-client-id>`;

```

#### Step 6: Allow SELECT IP Addresses

If your Databricks workspace has configured a network policy to restrict access based on IP address, you'll need to update the network policy to allow SELECT to connect. Refer to the following documentation for more information:

- [Workspace IP Access list](https://docs.databricks.com/aws/en/security/network/front-end/ip-access-list-workspace)
- [Context-based Ingress Control](https://docs.databricks.com/aws/en/security/network/front-end/context-based-ingress)

```markdown
34.23.79.180,
34.139.189.198,
35.229.105.183,
35.243.245.125,
34.138.184.161,
34.73.135.174,
34.148.143.80,
34.23.67.112,
```

The following python code can be run in a notebook if you would like to configure this at a workspace level:

```python
from databricks.sdk import WorkspaceClient
from databricks.sdk.service.settings import ListType

w = WorkspaceClient()

# Create a new IP allow list
created_list = w.ip_access_lists.create(
    label="SELECT-allowed-ips",
    list_type=ListType.ALLOW,
    ip_addresses=[
        "34.23.79.180",
        "34.139.189.198",
        "35.229.105.183",
        "35.243.245.125",
        "34.138.184.161",
        "34.73.135.174",
        "34.148.143.80",
        "34.23.67.112",
    ],
)
```

#### Step 7: Enabling Automated Savings

> [!NOTE]
> Automated Savings for Jobs Compute is currently in development. In the meantime, you can set up the required permissions now so you're ready when it launches.

The Automated Savings feature for All-Purpose Compute, Jobs Compute and SQL Warehouses requires granting the `CAN_MANAGE` permission to the SELECT Service Principal on each of the relevant resources. For Jobs Compute, the permissions should be granted on the Job itself rather than the compute resources which it creates. 

For Warehouses, the Service Principal also needs the `Databricks SQL access` entitlement.

Repeat this for all Clusters, Warehouses and Jobs in your workspace. For connecting to Clusters, Warehouses and Jobs in other workspaces, ensure that the service principal is added to those workspaces as well (see instructions [here](https://docs.databricks.com/aws/en/admin/users-groups/manage-service-principals?language=Account%C2%A0console#assign-a-service-principal-to-a-workspace))

For an automated way to assign permissions, see: [https://select.dev/docs/reference/faq/automated-savings/how-do-i-assign-can_manage-to-all-my-compute-resources-in-databricks](https://select.dev/docs/reference/faq/automated-savings/how-do-i-assign-can_manage-to-all-my-compute-resources-in-databricks).

**All-Purpose Compute**

![Editing permissions on All-purpose Compute](https://cdn.sanity.io/images/wn0ylp2c/production/089a79d54cd956a68a96a4550e05a976acb52ddb-800x362.gif)

**Jobs**

![Editing permissions on Jobs](https://cdn.sanity.io/images/wn0ylp2c/production/6c930827a6e59f961fe8a5be617d5a702f77fe9f-800x560.gif)

**SQL Warehouses**

![Editing permissions on SQL Warehouses](https://cdn.sanity.io/images/wn0ylp2c/production/1fda725c258a8fd20fa6c3ee52032b2b7979c940-800x429.gif)

If, as part of evaluating an All-Purpose Compute cluster for potential savings, granting the `CAN_ATTACH_TO` permission allows access to Spark metrics and better estimates of savings potential, before full management access is granted to enable the feature. The SELECT Automated Savings agent will begin monitoring all clusters automatically where this permission is granted.

### Automated setup

Running the following script will do all of the manual steps above (apart from creating the secret). This script provisions a Databricks service principal for select.dev by creating/finding the SP, assigning it to the target workspace, granting system catalog access, and setting up a SQL warehouse. If `USE_AUTOMATED_SAVINGS` is enabled, it also iterates all workspaces in the same region and grants the SP `CAN_MANAGE` permissions on all jobs, clusters, and warehouses to enable automated savings features. If `IP_ALLOW_LIST_ENABLED` is `True`, it will create an allowed list of IPs for all the workspaces as well. Copy and save to a file locally (e.g. `setup-select.py`), edit the first 10 lines with your details, and then just run `python setup-select.py`:

```python
# Dependencies: pip install databricks-sdk

# ── Config — edit these before running ───────────────────────────────────────

WORKSPACE_URL = "<https://dbc-xxxxx.cloud.databricks.com>"
ACCOUNT_ID = "<databricks-account-id>"
SERVICE_PRINCIPAL_NAME = "<service-principal-name>"
WAREHOUSE_NAME = "<sql-warehouse-name>"
IP_ALLOW_LIST_ENABLED = False
IP_ALLOW_LIST_LABEL = "select-ip-allow-list"
USE_AUTOMATED_SAVINGS = True
GRANT_PII_ACCESS = (
    True  # add SP to databricks_pii_access so SELECT reads unmasked query text
)
PII_ACCESS_GROUP_NAME = "databricks_pii_access"  # account-level, case-sensitive

# ─────────────────────────────────────────────────────────────────────────────

from urllib.parse import urlparse

from databricks.sdk import AccountClient
from databricks.sdk.errors.platform import ResourceConflict
from databricks.sdk.service import catalog as cat
from databricks.sdk.service import sql
from databricks.sdk.service.compute import ClusterSource
from databricks.sdk.service.iam import (
    AccessControlRequest,
    Patch,
    PatchOp,
    PatchSchema,
    PermissionLevel,
    WorkspacePermission,
)
from databricks.sdk.service.settings import ListType

IP_ADDRESSES = [
    "34.23.79.180",
    "34.139.189.198",
    "35.229.105.183",
    "35.243.245.125",
    "34.138.184.161",
    "34.73.135.174",
    "34.148.143.80",
    "34.23.67.112",
    "70.31.179.9",
    "216.209.96.31",
]


def main():
    parsed = urlparse(WORKSPACE_URL)
    workspace_host = f"{parsed.scheme}://{parsed.netloc}"
    account_host = detect_account_host(workspace_host)
    account_client = AccountClient(
        host=account_host, account_id=ACCOUNT_ID, auth_type="external-browser"
    )

    ws_info = find_current_workspace(account_client, workspace_host)
    ws_client = account_client.get_workspace_client(ws_info)
    region = ws_info.location or ws_info.aws_region
    print(f"Workspace : {ws_info.workspace_name}")
    print(f"Region    : {region}")

    sp = get_or_create_service_principal(account_client, SERVICE_PRINCIPAL_NAME)
    if GRANT_PII_ACCESS:
        ensure_sp_in_pii_access_group(account_client, sp)
    ensure_sp_assigned_to_workspace(account_client, ws_client, ws_info, sp)
    ensure_system_catalog_access(ws_client, sp, account_host)
    warehouse = get_or_create_warehouse(ws_client, WAREHOUSE_NAME)
    grant_permission(ws_client, "warehouses", warehouse.id, sp, PermissionLevel.CAN_USE)

    workspaces_in_region = get_workspaces_in_region(account_client, region)
    print(f"\n{len(workspaces_in_region)} workspace(s) in region '{region}'")

    for ws in workspaces_in_region:
        print(f"\n  Workspace: {ws.workspace_name}")
        ws_client = account_client.get_workspace_client(ws)

        ensure_sp_assigned_to_workspace(account_client, ws_client, ws, sp)

        if IP_ALLOW_LIST_ENABLED:
            ensure_ip_access_list(ws_client, IP_ALLOW_LIST_LABEL, IP_ADDRESSES)
        else:
            print("    Skipping IP allow list (IP_ALLOW_LIST_ENABLED=False)")

        if USE_AUTOMATED_SAVINGS:
            print(f"  Granting CAN_MANAGE to SP on jobs in {ws.workspace_name}")
            for job in ws_client.jobs.list():
                grant_permission(
                    ws_client, "jobs", job.job_id, sp, PermissionLevel.CAN_MANAGE
                )

            print(f"  Granting CAN_MANAGE to SP on clusters in {ws.workspace_name}")
            for cluster in ws_client.clusters.list():
                if cluster.cluster_source not in (ClusterSource.API, ClusterSource.UI):
                    continue
                grant_permission(
                    ws_client,
                    "clusters",
                    cluster.cluster_id,
                    sp,
                    PermissionLevel.CAN_MANAGE,
                )

            print(f"  Granting CAN_MANAGE to SP on warehouses in {ws.workspace_name}")
            for wh in ws_client.warehouses.list():
                grant_permission(
                    ws_client, "warehouses", wh.id, sp, PermissionLevel.CAN_MANAGE
                )
        else:
            print(
                "    Skipping automated savings permissions (USE_AUTOMATED_SAVINGS=False)"
            )

    secrets_url = (
        f"{account_host}/user-management/serviceprincipals/{sp.id}"
        f"/credentials-and-secrets?account_id={ACCOUNT_ID}"
    )
    print(
        "\n── Done ─────────────────────────────────────────────────────────────────────"
    )
    print(f"  Account ID   : {ACCOUNT_ID}")
    print(f"  Workspace URL: {workspace_host}")
    print(f"  Warehouse ID : {warehouse.id}")
    print(f"  SP Client ID : {sp.application_id}")
    print(f"\n  Create a client secret at:\n  {secrets_url}")
    print(
        "─────────────────────────────────────────────────────────────────────────────"
    )


# ── Helpers ───────────────────────────────────────────────────────────────────


def detect_account_host(workspace_host):
    if "azuredatabricks.net" in workspace_host:
        return "https://accounts.azuredatabricks.net"
    elif "gcp.databricks.com" in workspace_host:
        return "https://accounts.gcp.databricks.com"
    else:
        return "https://accounts.cloud.databricks.com"


def find_current_workspace(account_client, workspace_host):
    for ws in account_client.workspaces.list():
        if ws.deployment_name and ws.deployment_name in workspace_host:
            return ws
    raise ValueError(
        f"No workspace found matching '{workspace_host}'. "
        "Ensure WORKSPACE_URL contains a valid Databricks workspace URL."
    )


def get_or_create_service_principal(account_client, name):
    for s in account_client.service_principals.list():
        if s.display_name == name:
            print(f"\nFound SP '{s.display_name}' (app_id={s.application_id})")
            return s
    print(f"\nCreating service principal '{name}' …")
    sp = account_client.service_principals.create(display_name=name)
    print(f"  Created (app_id={sp.application_id})")
    return sp


def ensure_sp_in_pii_access_group(account_client, sp):
    """Add the SP to the account-level databricks_pii_access group.

    Databricks masks statement_text to <REDACTED> in system.query.history for
    principals outside this group; membership lets SELECT read the real text.

    Args:
        account_client: An authenticated Databricks AccountClient.
        sp: The service principal to add to the group.

    Returns:
        None. Creates the group if absent and adds the SP if not already a
        member; both steps are skipped when already satisfied.
    """
    group = _get_or_create_account_group(account_client, PII_ACCESS_GROUP_NAME)
    if any(str(m.value) == str(sp.id) for m in (group.members or [])):
        print(f"\nSP already in '{PII_ACCESS_GROUP_NAME}', skipping.")
        return
    print(f"\nAdding SP to '{PII_ACCESS_GROUP_NAME}' …")
    account_client.groups.patch(
        id=group.id,
        operations=[
            Patch(op=PatchOp.ADD, path="members", value=[{"value": str(sp.id)}])
        ],
        schemas=[PatchSchema.URN_IETF_PARAMS_SCIM_API_MESSAGES_2_0_PATCH_OP],
    )


def _get_or_create_account_group(account_client, name):
    """Return the account group named ``name``, creating it if absent.

    Args:
        account_client: An authenticated Databricks AccountClient.
        name: The case-sensitive account-level group display name.

    Returns:
        The group, re-fetched so its ``members`` are populated.
    """
    found = _find_account_group(account_client, name)
    if found is not None:
        return found
    print(f"\nCreating account group '{name}' …")
    try:
        return account_client.groups.create(display_name=name)
    except ResourceConflict:
        # Already exists (raced, or the server-side filter missed it) — find it.
        found = _find_account_group(account_client, name)
        if found is None:
            raise
        return found


def _find_account_group(account_client, name):
    # List and match in Python; the server-side displayName filter can miss it.
    for g in account_client.groups.list():
        if g.display_name == name:
            return account_client.groups.get(g.id)  # re-fetch to include members
    return None


# Tables SELECT replicates from the system catalog. Mirrors
SYSTEM_TABLES_BY_SCHEMA = {
    "query": ["history"],
    "billing": ["usage", "account_prices", "list_prices"],
    "serving": ["served_entities", "endpoint_usage"],
    "compute": ["warehouses", "warehouse_events", "clusters", "node_timeline"],
    "access": ["table_lineage", "column_lineage", "workspaces_latest"],
    "lakeflow": [
        "job_run_timeline",
        "job_task_run_timeline",
        "jobs",
        "job_tasks",
        "pipelines",
    ],
}
# system.billing.account_prices is only available on AWS/GCP.
AZURE_UNAVAILABLE_TABLES = {("billing", "account_prices")}


def ensure_system_catalog_access(ws_client, sp, account_host):
    principal = str(sp.application_id)
    is_azure = "azuredatabricks.net" in account_host

    print("\nGranting USE_CATALOG on 'system' …")
    _grant_unity_catalog(
        ws_client, "catalog", "system", principal, [cat.Privilege.USE_CATALOG]
    )

    for schema, tables in SYSTEM_TABLES_BY_SCHEMA.items():
        full_schema = f"system.{schema}"
        print(f"Granting USE_SCHEMA on '{full_schema}' …")
        _grant_unity_catalog(
            ws_client, "schema", full_schema, principal, [cat.Privilege.USE_SCHEMA]
        )

        for table in tables:
            if is_azure and (schema, table) in AZURE_UNAVAILABLE_TABLES:
                print(f"  Skipping system.{schema}.{table} (not available on Azure)")
                continue
            full_table = f"{full_schema}.{table}"
            print(f"  Granting SELECT on '{full_table}' …")
            _grant_unity_catalog(
                ws_client, "table", full_table, principal, [cat.Privilege.SELECT]
            )


def _grant_unity_catalog(ws_client, securable_type, full_name, principal, privileges):
    required = set(privileges)
    for pa in (
        ws_client.grants.get(securable_type, full_name).privilege_assignments or []
    ):
        if pa.principal == principal and required.issubset(set(pa.privileges or [])):
            print("    already granted, skipping.")
            return
    ws_client.grants.update(
        securable_type=securable_type,
        full_name=full_name,
        changes=[cat.PermissionsChange(principal=principal, add=list(privileges))],
    )


def get_or_create_warehouse(ws_client, name):
    for w in ws_client.warehouses.list():
        if w.name == name:
            print(f"\nFound existing warehouse '{name}' (ID: {w.id})")
            return w
    print(f"\nCreating serverless SQL warehouse '{name}' …")
    warehouse = ws_client.warehouses.create(
        name=name,
        cluster_size="2X-Small",
        enable_serverless_compute=True,
        auto_stop_mins=1,
        max_num_clusters=1,
        warehouse_type=sql.CreateWarehouseRequestWarehouseType.PRO,
    ).result()
    print(f"  Created (ID: {warehouse.id})")
    return warehouse


def get_workspaces_in_region(account_client, region):
    workspaces = []
    for ws in account_client.workspaces.list():
        if (ws.location or ws.aws_region) == region:
            workspaces.append(ws)
    return workspaces


def ensure_sp_assigned_to_workspace(account_client, ws_client, ws, sp):
    for s in ws_client.service_principals.list():
        if s.application_id == sp.application_id:
            return
    print("    Assigning SP to workspace …")
    account_client.workspace_assignment.update(
        workspace_id=ws.workspace_id,
        principal_id=int(sp.id),
        permissions=[WorkspacePermission.USER],
    )


def ensure_ip_access_list(ws_client, label, ip_addresses):
    desired = set(ip_addresses)
    for existing in ws_client.ip_access_lists.list():
        if existing.label != label:
            continue
        current = set(existing.ip_addresses or [])
        if desired.issubset(current) and existing.enabled:
            print(f"    IP allow list '{label}' already configured, skipping.")
            return
        merged = sorted(current | desired)
        print(f"    Updating IP allow list '{label}' …")
        ws_client.ip_access_lists.update(
            ip_access_list_id=existing.list_id,
            label=label,
            list_type=ListType.ALLOW,
            ip_addresses=merged,
            enabled=True,
        )
        return
    print(f"    Creating IP allow list '{label}' …")
    ws_client.ip_access_lists.create(
        label=label,
        list_type=ListType.ALLOW,
        ip_addresses=ip_addresses,
        enabled=True,
    )


def grant_permission(ws_client, resource_type, resource_id, sp, level):
    sp_app_id = str(sp.application_id)
    perms = ws_client.permissions.get(resource_type, resource_id)
    already_granted = any(
        entry.service_principal_name == sp_app_id
        and any(p.permission_level == level for p in (entry.all_permissions or []))
        for entry in (perms.access_control_list or [])
    )
    if already_granted:
        print(
            f"      {level.value} on {resource_type}/{resource_id} already granted, skipping."
        )
        return
    print(f"      Granting {level.value} on {resource_type}/{resource_id} …")
    ws_client.permissions.update(
        resource_type,
        resource_id,
        access_control_list=[
            AccessControlRequest(
                service_principal_name=sp_app_id, permission_level=level
            )
        ],
    )


if __name__ == "__main__":
    main()
```

### Add workspace to SELECT

- Navigate to settings: [https://select.dev/app/settings](https://select.dev/app/settings)
- Go to the **Databricks** tab and click the **Add Workspace** button
- Enter a memorable workspace name, and fill out all the details you have noted in the previous steps:  
   - Account ID
   - Workspace URL
   - Warehouse ID
   - Client ID
   - Client Secret
- Click **Add** and you are good to go!

---

# Connect your BigQuery Organization

URL: https://select.dev/docs/reference/setup-guide/connect-bigquery-organization

> A walkt though on how to connect you BQ org to SELECT

> [!NOTE]
> ### **Which path applies to you?**
>
> This guide covers two setups, and Steps 1–4 are identical for both.
>
> - **DoiT resells your Google Cloud and invoices you.** Your billing data lives in DoiT's project rather than your own, so skip Steps 5–6 and choose the DoiT-managed tab on the connection form.
> - **Google invoices you directly.** Follow every step, and choose the Direct billing tab on the connection form.
>
> If you work with DoiT for support or tooling but Google still sends your bill, you're on the direct billing path.

## Prerequisites

#### Permissions

A user with the following roles is needed to complete this setup. All of them apply to both paths except the last.

- **Organization Admin** — for the organization-level binding in Step 3
- **Role Administrator** — for the custom role in Step 1
- **Project IAM Admin** — for the project-level bindings in Step 3
- **Service Usage Admin** — to enable the APIs listed below
- **Billing Account Costs Manager or Billing Account Administrator.** Not needed if DoiT resells your Google Cloud, since you won't be setting up billing exports.

#### Enabled APIs

Both paths. Ensure the following APIs are enabled on the project that will host the service account:

- `bigquery.googleapis.com`
- `iam.googleapis.com`
- `iamcredentials.googleapis.com`
- `cloudresourcemanager.googleapis.com`

#### Enable TABLE_STORAGE_BY_ORGANIZATION

This step applies to both paths. SELECT uses INFORMATION_SCHEMA.TABLE_STORAGE_BY_ORGANIZATION to provide storage visibility across your org, and the view must be explicitly enabled for each region where your BigQuery data resides. Run the following in Cloud Shell to discover all regions in use and enable the view for each automatically.

```bash
gcloud asset search-all-resources \
  --scope=organizations/${ORG_ID} \
  --asset-types=bigquery.googleapis.com/Dataset \
  --format="value(location)" | sort -u | tr '[:upper:]' '[:lower:]' | \
while read region; do
  # bq CLI requires uppercase for multi-regions (US, EU)
  if [ "$region" = "us" ] || [ "$region" = "eu" ]; then
    location=$(echo $region | tr '[:lower:]' '[:upper:]')
  else
    location=$region
  fi
  echo "Enabling for region: ${region}"
  bq query \
    --location=${location} \
    --nouse_legacy_sql \
    "ALTER ORGANIZATION SET OPTIONS (\`region-${region}.enable_info_schema_storage\` = TRUE)"
done
```

> [!NOTE]
> Allow up to 24 hours for historical data to become available after enabling. SELECT cannot display storage data until this step is complete.

## Connecting your account

Steps 1 through 4 are identical for both paths. Throughout, CUSTOMER_PROJECT refers to a project in your organization where SELECT runs its queries — you can use an existing project or create one for this purpose.

```bash
export ORG_ID="<ORG_ID>"
export CUSTOMER_PROJECT="<CUSTOMER_PROJECT>"
export SELECT_SERVICE_ACCOUNT="<SELECT_SERVICE_ACCOUNT>"
```

### Step 1: Create the custom org role

Rather than binding several predefined roles separately, create one custom role at the org level. This gives you a single binding to manage and makes it easy to audit exactly what access SELECT has. It combines the permissions of roles/bigquery.resourceViewer, roles/bigquery.metadataViewer and roles/browser, plus bigquery.config.get, which is required to detect fluid scaling on reservations and is not included in any predefined role.

```bash
gcloud iam roles create SelectOrgViewer \
  --organization="${ORG_ID}" \
  --title="SELECT Org Viewer" \
  --description="Grants SELECT read-only metadata access across the org" \
  --permissions=bigquery.bireservations.get,bigquery.capacityCommitments.get,bigquery.capacityCommitments.list,bigquery.config.get,bigquery.datasets.get,bigquery.datasets.getIamPolicy,bigquery.jobs.get,bigquery.jobs.list,bigquery.jobs.listAll,bigquery.jobs.listExecutionMetadata,bigquery.models.getMetadata,bigquery.models.list,bigquery.propertyGraphs.get,bigquery.propertyGraphs.list,bigquery.reservationAssignments.list,bigquery.reservationAssignments.search,bigquery.reservationGroups.get,bigquery.reservationGroups.list,bigquery.reservations.get,bigquery.reservations.list,bigquery.reservations.listFailoverDatasets,bigquery.routines.get,bigquery.routines.list,bigquery.tables.get,bigquery.tables.getIamPolicy,bigquery.tables.list,dataplex.projects.search,resourcemanager.organizations.get,resourcemanager.folders.get,resourcemanager.folders.list,resourcemanager.projects.get,resourcemanager.projects.getIamPolicy,resourcemanager.projects.list
```

> [!NOTE]
> If you see "role SelectOrgViewer already exists", that's fine — continue to the next step.

### Step 2: Create the service account

In your project UI navigate to IAM → Service Accounts → Create Service Account and give it a name, e.g. select-viewer. Click Create and continue, then skip the optional role and group steps and click Done. Alternatively, in Cloud Shell or any environment with gcloud authenticated:

```bash
gcloud iam service-accounts create select-viewer \
  --display-name="SELECT replicator viewer" \
  --project="${CUSTOMER_PROJECT}"
```

This creates select-viewer@${CUSTOMER_PROJECT}.iam.gserviceaccount.com. Note this email down — you'll need it in the final step.

### Step 3: Grant read and query access

First, bind the custom role at the organization level so SELECT can see every project's usage:

```bash
gcloud organizations add-iam-policy-binding "${ORG_ID}" \
  --member="serviceAccount:select-viewer@${CUSTOMER_PROJECT}.iam.gserviceaccount.com" \
  --role="organizations/${ORG_ID}/roles/SelectOrgViewer"
```

Then grant the project-level roles that let SELECT actually run queries and read sessions. These are intentionally scoped to the project only, not the org, to limit the blast radius:

```bash
for ROLE in roles/bigquery.jobUser roles/bigquery.readSessionUser; do
  gcloud projects add-iam-policy-binding "${CUSTOMER_PROJECT}" \
    --member="serviceAccount:select-viewer@${CUSTOMER_PROJECT}.iam.gserviceaccount.com" \
    --role=$ROLE
done
```

> [!NOTE]
> If you see a 403 "User does not have bigquery.jobs.create permission" error when testing the connection, this step was likely skipped or run against the wrong project. Confirm CUSTOMER_PROJECT matches the project ID you entered in SELECT's connection form.

### Step 4: Allow SELECT to impersonate the service account

SELECT authenticates by impersonating the service account you just created — nothing leaves your organization. You can find your org-specific SELECT service account email on the Add BigQuery connection form in the SELECT app.

```bash
gcloud iam service-accounts add-iam-policy-binding \
  select-viewer@"${CUSTOMER_PROJECT}".iam.gserviceaccount.com \
  --member="serviceAccount:${SELECT_SERVICE_ACCOUNT}" \
  --role="roles/iam.serviceAccountTokenCreator" \
  --project="${CUSTOMER_PROJECT}"
```

> [!WARNING]
> **Org policy restrictions:** Some organizations enforce constraints/iam.managed.allowedPolicyMembers, which blocks binding external service accounts directly to resources in your org. If you hit an error mentioning this constraint, add SELECT's service account to a Google Group in your organization, then grant roles/iam.serviceAccountTokenCreator to that group on the select-viewer SA instead of binding SELECT's SA directly.

> [!TIP]
> **DoiT-managed customers: you're done with the GCP-side setup. Skip Steps 5 and 6 and go straight to** [**Add connection to SELECT**](https://select.dev/docs/reference/setup-guide/connect-bigquery-organization#add-connection-to-select) **— DoiT handles your billing export, so there is no billing setup on your side.**

### Step 5: Enable billing exports 

SELECT needs three billing exports — detailed usage cost, pricing data, and committed use discount data — to give you complete cost visibility. Each is written to BigQuery and all three should land in the same dataset. In the Google Cloud Console, navigate to Billing, then click Billing export in the left pane.

#### 5a. Detailed usage cost export

- On the BigQuery export tab, find Detailed usage cost and click Enable detailed export.
- For Project, select your billing project.
- For the dataset, click create new dataset. Creating a dataset specific to SELECT lets you limit SELECT's access to just the billing data. Enter a dataset ID of your choosing (e.g. billing_export) and choose a multi-region location (US or EU). Note the ID down.

#### 5b. Pricing data export

Back on the Billing export page, find Pricing data, click Enable pricing export, select the same project and dataset as in 5a, and save.

#### 5c. Committed use discount (CUD) data export

Find Committed use discount data, click Enable CUD export, select the same project and dataset, and save. It can take up to 24 hours for the first batch of billing data to appear.

### Step 6: Grant read access to the billing dataset 

Grant bigquery.dataViewer scoped to only the billing export dataset, so SELECT has no access to your company's data. In BigQuery Studio, navigate to the billing export dataset, click Share → Manage Permissions → Add principal, add the select-viewer service account and assign BigQuery Data Viewer. Alternatively:

```bash
gcloud alpha bq datasets add-iam-policy-binding \
  ${BILLING_DATASET} \
  --project="${CUSTOMER_PROJECT}" \
  --member="serviceAccount:select-viewer@${CUSTOMER_PROJECT}.iam.gserviceaccount.com" \
  --role=roles/bigquery.dataViewer
```

## Add connection to SELECT

Navigate to the connections tab in SELECT's settings and click Add Connection. The fields differ depending on your billing path.

### Direct billing

Enter a connection name (e.g. your organization's name) and fill in:

- **Project ID** — your CUSTOMER_PROJECT
- **Billing export dataset name** — e.g. billing_export
- **Billing Account ID** — e.g. ABCDEF-12345-67810
- Leave Managed by DoiT unchecked.

### DoiT-managed

Turn on DoiT-managed connection, then fill in:

- **Friendly Connection Name** — any label you like
- **Export Project ID** — your CUSTOMER_PROJECT, where queries run. Not a DoiT project.
- **Service Account** — select-viewer@CUSTOMER_PROJECT.iam.gserviceaccount.com

You don't enter a billing account or dataset — SELECT pulls your cost data from DoiT's billing export automatically.

Click Add. SELECT runs a test query to verify the connection. You'll receive an email when the initial sync completes, usually within a couple of hours.

## Troubleshooting

#### Test fails with a "jobs.create" 403

The project ID you entered must be your own CUSTOMER_PROJECT, and that project needs the bigquery.jobUser grant from Step 3. If you are on the DoiT-managed path, don't enter a DoiT project such as doitintl-cmp-gcp-data — the service account can't run jobs there.

#### "Permission denied" creating the role or binding

Steps 1 and 3 act at the organization level and need Role Administrator and Organization Admin. Ask whoever owns your org's IAM to run those commands, or to grant you those roles temporarily.

#### "Service account does not exist"

This comes from Step 4 when SELECT_SERVICE_ACCOUNT doesn't exactly match the email on your connection form. Re-copy it from the top of the connection screen and rerun that command.

#### Can't find your organization ID?

Your Cloud Console org picker (top-left) shows it. DoiT resale customers who are still stuck can reach out to their DoiT account team, who can look it up from your DoiT billing data.

## VPC Service Controls (optional)

Applies to both paths. If your GCP organization has VPC Service Controls enabled with a service perimeter around BigQuery, SELECT's queries will be blocked by default and you'll see "Request is prohibited by organization's policy" when adding the connection. Since SELECT authenticates by impersonating your select-viewer service account, the most reliable fix is an ingress rule scoped to SELECT's GCP project.

```yaml
- ingressFrom:
    identityType: ANY_IDENTITY
    sources:
    - resource: "projects/813120990659"
  ingressTo:
    operations:
    - serviceName: bigquery.googleapis.com
      methodSelectors:
      - method: "*"
    resources:
    - "*"
```

ANY_IDENTITY is required because SELECT's service account lives in a different GCP organization than your perimeter, and ANY_SERVICE_ACCOUNT only matches service accounts within the same org. This remains secure: the source is locked to SELECT's project, IAM enforces that only SELECT's service account can impersonate your select-viewer SA, and that SA has read-only BigQuery permissions. Apply it with:

```bash
gcloud access-context-manager perimeters update YOUR_PERIMETER_NAME \
  --add-ingress-policies=ingress-select.yaml \
  --policy=YOUR_POLICY_ID
```

> [!WARNING]
> IP allowlisting will not work for most customers. SELECT runs on Google Cloud Run, which routes traffic through Google's internal network and redacts the source IP, so IP-based access levels never match. Always test perimeter changes in dry-run mode before enforcing them.

---

# PrivateLink

URL: https://select.dev/docs/reference/setup-guide/privatelink

> Connect SELECT to your Snowflake or Databricks instance through PrivateLink connection to avoid public internet

- [PrivateLink for Snowflake Account on AWS](https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-snowflake-account-on-aws.md): How to create a PrivateLink connection from SELECT to your Snowflake account running on AWS
- [PrivateLink for Snowflake Account on Azure](https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-snowflake-account-on-azure.md): How to create a PrivateLink connection from SELECT to your Snowflake account running on Azure
- [PrivateLink for Databricks Workspace on AWS](https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-databricks-workspace-on-aws.md): How to create a PrivateLink connection from SELECT to your Databricks workspace running on AWS
- [PrivateLink for Databricks Workspace on Azure](https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-databricks-workspace-on-azure.md): How to create a PrivateLink connection from SELECT to your Databricks workspace running on Azure

---

# PrivateLink for Snowflake Account on AWS

URL: https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-snowflake-account-on-aws

> How to create a PrivateLink connection from SELECT to your Snowflake account running on AWS

Organizations using Snowflake with AWS PrivateLink can still use connect SELECT to their Snowflake Account with a few extra steps. Please reach out to your SELECT account representative to start the process.

This guide walks you through enabling AWS PrivateLink between your Snowflake account and SELECT.

## Prerequisites

- Snowflake account running on AWS
- Snowflake `ACCOUNTADMIN` role (required to run the authorization and config commands)
- The AWS region of your Snowflake account (for example, `us-east-1`)

## Step 1 — Authorize SELECT's AWS account

You need to grant SELECT's AWS account permission to connect to your Snowflake instance via PrivateLink. There are two ways to do this.

### Option A — Support ticket (slower)

1. Open a Snowflake support ticket requesting third-party PrivateLink access to your Snowflake account.
2. Include SELECT's AWS account ID in the ticket — contact your SELECT representative to get it.

> Snowflake support typically processes the request within 1–2 business days.

### Option B — AUTHORIZE_PRIVATELINK command (faster)

SELECT can generate a short-lived federation token that lets you authorize the connection yourself in seconds — no support ticket needed.

Contact your SELECT representative and ask for the SYSTEM$AUTHORIZE_PRIVATELINK command. Once you receive it, run it in your Snowflake account as ACCOUNTADMIN:

```sql
SELECT SYSTEM$AUTHORIZE_PRIVATELINK('<select_aws_account_id>', '<federation_token>');
```

> The token expires after a fixed window (your SELECT representative will tell you how long). If it has expired, ask for a new one.

## Step 2 — Retrieve your PrivateLink configuration

Run the following command in your Snowflake account as ACCOUNTADMIN, then share the output with your SELECT representative:

```sql
SELECT SYSTEM$GET_PRIVATELINK_CONFIG();
```

The output is a JSON object. SELECT needs the following two fields from it:

| Field | Description | Example |
| --- | --- | --- |
| privatelink-account-url | Your account locator and AWS region | abc12345.us-east-1.privatelink.snowflakecomputing.com |
| privatelink-vpce-id | The AWS VPC endpoint service name for your account | com.amazonaws.vpce.us-east-1.vpce-svc-... |

## Step 3 — SELECT creates the VPC endpoint

SELECT uses the information from Step 2 to provision a VPC endpoint on the backend. This typically completes within minutes once SELECT has the config. SELECT will confirm when it is ready.

## Step 4 — Add your Snowflake account connection

Once SELECT confirms the endpoint is live, add your Snowflake account connection in SELECT. In the Account Identifier field, use the PrivateLink account name format below — instead of the standard Snowflake account URL:

```markdown
<account_locator>.<region>.privatelink
```

For example, if your account locator is `abc12345` and your region is `us-east-1`:

```markdown
abc12345.us-east-1.privatelink
```

## Step 5 — Network policy (if applicable)

If your Snowflake account has a network policy restricting inbound connections, add SELECT's private IP ranges to the allowlist:

| IP range | Purpose |
| --- | --- |
| `10.124.0.0/28` | SELECT backend |
| `10.1.128.0/17` | The periodic job that reads your Snowflake metadata |

---

# PrivateLink for Snowflake Account on Azure

URL: https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-snowflake-account-on-azure

> How to create a PrivateLink connection from SELECT to your Snowflake account running on Azure

Organizations using Snowflake with Azure PrivateLink can still use connect SELECT to their Snowflake Account with a few extra steps. Please reach out to your SELECT account representative to start the process.

This guide walks you through enabling Azure PrivateLink between your Snowflake account and SELECT.

## Prerequisites

- Snowflake account running on Azure
- Snowflake `ACCOUNTADMIN` role (required to run the authorization commands)
- The Azure region of your Snowflake account (for example, `eastus`)

## Step 1 — Retrieve your Private Link configuration

Run the following command in your Snowflake account as `ACCOUNTADMIN`, then share the output with your SELECT representative:

```sql
SELECT SYSTEM$GET_PRIVATELINK_CONFIG();
```

The output is a JSON object. SELECT needs the following two fields from it to create a private endpoint on their side before authorization can proceed:

| Field | Description | Example |
| --- | --- | --- |
| privatelink-account-url | Your account locator and Azure region | eua49794.east-us-2.privatelink.snowflakecomputing.com |
| privatelink-pls-id | Your Private Link Service alias | sf-pvlinksvc-azeastus2.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.eastus2.azure.privatelinkservice |

## Step 2 — Wait for SELECT to create a private endpoint

SELECT uses the configuration from Step 1 to provision a private endpoint in their Azure environment, connecting to your Snowflake instance. **SELECT will contact you once the endpoint is ready** and provide the information needed for Step 3.

## Step 3 — Authorize SELECT's private endpoint

Once SELECT provides the `SYSTEM$AUTHORIZE_PRIVATELINK` command, run it in your Snowflake account as `ACCOUNTADMIN`:

```sql
SELECT SYSTEM$AUTHORIZE_PRIVATELINK('<private_endpoint_resource_id>', '<azure_access_token>');
```

## Step 4 — Add your Snowflake account connection

Add your Snowflake account connection in SELECT. In the **Account Identifier** field, use the Private Link account name format below — instead of the standard Snowflake account URL:

```markdown
<account_locator>.<region>.privatelink
```

For example, if your account locator is `abc12345` and your region is `eastus`:

```markdown
abc12345.eastus.privatelink
```

## Step 5 — Network policy (if applicable)

If your Snowflake account has a network policy restricting inbound connections, add SELECT's private IP ranges to the allowlist:

| IP range | Purpose |
| --- | --- |
| `10.124.0.0/28` | SELECT backend |
| `10.1.128.0/17` | The periodic job that reads your Snowflake metadata |

---

# PrivateLink for Databricks Workspace on AWS

URL: https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-databricks-workspace-on-aws

> How to create a PrivateLink connection from SELECT to your Databricks workspace running on AWS

Organizations using Databricks with AWS PrivateLink can still use connect SELECT to their Databricks Workspace with a few extra steps. Please reach out to your SELECT account representative to start the process.

This guide walks you through enabling AWS PrivateLink between your Databricks workspace and SELECT.

## Prerequisites

- Databricks account on the Enterprise plan
- Databricks account admin role
- The AWS region of your Databricks workspace (for example, `us-east-1`)

## Step 1 — Share your region and workspace URL with SELECT

Contact your SELECT representative and provide:

- Your Databricks workspace's AWS region (for example, `us-east-1`)
- Your Databricks workspace URL (for example, `dbc-xxxx-xxxx.cloud.databricks.com`)

In return, SELECT will give you the VPCE ID for your region, and will use your workspace URL to add a DNS entry so that traffic from SELECT's infrastructure resolves to your workspace via PrivateLink once setup is complete.

## Step 2 — Register the VPC endpoint in Databricks

In your Databricks account console:

1. Go to [**Security → Networking → VPC endpoints**](https://accounts.cloud.databricks.com/security/networking/vpc-endpoints)
2. Click **Register a VPC endpoint**
3. Enter a name (for example, `select-privatelink-<region>`) and select your AWS region
4. Paste SELECT's VPCE ID from Step 1 into the **AWS VPC endpoint ID** field
5. Click **Register new VPC endpoint**

## Step 3 — Configure Private Access Settings

Go to [**Security → Networking → Private access settings**](https://accounts.cloud.databricks.com/security/networking/private-access-settings).

**If your workspace already has a private access setting**, edit it and add the VPCE registered in Step 2 to the allowed endpoints.

**If you don't have one yet**, create a new one:

1. Click **Add private access setting**
2. Enter a name (for example, `select-privatelink-<region>`) and set the region to match your workspace's AWS region
3. Set the **Private access level** to include the VPCE registered in Step 2
4. Optionally keep public access enabled initially for testing, then disable it once confirmed working

## Step 4 — Attach Private Access Settings to your workspace

1. Go to [**Workspaces**](https://accounts.cloud.databricks.com/workspaces) in the Databricks account console and click on your workspace
2. Scroll to the **Networking** section and click **Edit**
3. Under **Private access setting**, select the private access settings object from Step 3
4. Save and wait for the workspace to update

> **Note:** Once a private access setting is attached to a workspace, it cannot be removed — you can only change it to a different private access setting or modify the existing one. See [Update a running or failed workspace](https://docs.databricks.com/administration-guide/workspace/update-workspace.html).

## Step 5 — Add your Databricks connection to SELECT

Once the workspace update completes, add your Databricks workspace to SELECT using your standard workspace URL:

```markdown
dbc-xxxx-xxxx.cloud.databricks.com
```

Use this as the **Workspace URL** in your SELECT connection settings. SELECT's infrastructure will route traffic to your workspace privately via the endpoint registered in Step 2.

## Step 6 — Network policy (if applicable)

If your Databricks workspace has network policies restricting inbound connections, add SELECT's private IP ranges to the allowlist:

| IP range | Purpose |
| --- | --- |
| `10.124.0.0/28` | SELECT backend |
| `10.1.128.0/17` | The periodic job that reads your Snowflake metadata |

---

# PrivateLink for Databricks Workspace on Azure

URL: https://select.dev/docs/reference/setup-guide/privatelink/privatelink-for-databricks-workspace-on-azure

> How to create a PrivateLink connection from SELECT to your Databricks workspace running on Azure

Organizations using Databricks with Azure PrivateLink can still use connect SELECT to their Databricks Workspace with a few extra steps. Please reach out to your SELECT account representative to start the process.

This guide walks you through enabling Azure PrivateLink between your Databricks workspace and SELECT.

## Prerequisites

- Databricks workspace on Azure with the Premium plan
- Azure Contributor role (or higher) on the workspace's resource group
- Access to the Azure Portal

## Step 1 — Send SELECT your workspace details

SELECT needs two pieces of information to provision the private endpoint on their side. Send both to your SELECT representative:

**1\. Workspace URL**

Your workspace URL (for example, `adb-1234567890123456.1.azuredatabricks.net`).

**2\. Workspace Resource ID**

The ARM resource ID looks like:

`/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-workspace`

Find it in the Azure Portal under **Databricks workspace → Settings blade → Properties → Id**.

## Step 2 — Wait for SELECT to provision the private endpoint

SELECT uses the details from Step 1 to create a private endpoint in their Azure environment, connecting to your workspace. They will contact you once the endpoint is ready — this typically takes a few minutes after their deployment.

## Step 3 — Approve the private endpoint connection

Once SELECT deploys the endpoint, a pending connection request will appear in your workspace's networking settings.

In the Azure Portal:

1. Open your **Databricks workspace**
2. In the left menu, go to **Networking** (under Settings)
3. Click the **Private endpoint connections** tab
4. Find the connection named **`select-<your-workspace-id>`**
5. Select it and click **Approve**
6. Add a note if prompted (for example, "Approved for SELECT PrivateLink") and confirm

The connection state will change from **Pending** to **Approved** within a few seconds.

> **Note:** If you don't see the connection, SELECT's deployment may still be in progress. Wait a few minutes and refresh the page.

## Step 4 — Add your Databricks connection to SELECT

Once the connection is approved, add your Databricks workspace to SELECT using your standard workspace URL:

```markdown
adb-1234567890123456.1.azuredatabricks.net
```

Use this as the **Workspace URL** in your SELECT connection settings. SELECT's infrastructure will route traffic to your workspace privately via the endpoint approved in Step 3.

## Network policy (if applicable)

If your Databricks workspace has IP access lists restricting inbound connections, you may need to add SELECT's private IP ranges to the allowlist:

| IP range | Purpose |
| --- | --- |
| `10.124.0.0/28` | SELECT backend |
| `10.1.128.0/17` | The periodic job that reads your Snowflake metadata |

---

# Connect your AWS bill to SELECT (Private Preview)

URL: https://select.dev/docs/reference/setup-guide/connect-your-aws-bill-to-select

> Create a CUR 2.0 data export and a read-only IAM user, then connect your AWS bill to SELECT

> [!WARNING]
> This feature is currently in Private Preview, and only available to customers of SELECT for Databricks.

When Databricks runs on classic compute, AWS bills you for the EC2 instances, EBS volumes and networking underneath. Once your AWS cost data is connected, SELECT shows that AWS cost next to the DBU cost of each workspace, cluster, SQL warehouse, job and query, so you can see the full cost of running Databricks.

## Prerequisites

- Admin access to your AWS **payer (management) account**, or at least permission to manage Billing, Data Exports, S3 and IAM
   - Only the payer account can manage cost allocation tags, and an export created there includes every member account in your AWS organization, so one connection covers all of your Databricks workspaces
- Your Databricks workspace already connected to SELECT
   - [Connect your Databricks Workspace](https://select.dev/docs/reference/setup-guide/connect-your-databricks-workspace)

## Connecting your AWS bill

#### Step 1: Activate the Databricks cost allocation tags

Databricks automatically tags the EC2 instances and EBS volumes it launches. SELECT uses those tags to tie AWS cost back to a cluster, SQL warehouse or job. AWS only includes the tags in your cost data if they're activated as **cost allocation tags**.

- In the AWS console, open **Billing and Cost Management → Cost allocation tags**
   - [AWS docs: Activating user-defined cost allocation tags](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html)
- Open the **User-defined cost allocation tags** tab, then find and select these tags:
   - `Vendor`: set on all compute. Identifies resources Databricks launched (value `Databricks`)
   - `ClusterId`: set on clusters and SQL warehouses. Ties instances and volumes to a cluster
   - `SqlEndpointId`: set on SQL warehouses. Ties instances to a SQL warehouse
   - `JobId`, `RunName`: set on job clusters. Ties job clusters to a job and run
   - `DatabricksInstancePoolId`: set on instance pools. Ties pool instances to a pool
   - `DatabricksInstancePoolCreatorId`: set on instance pools. Display only
   - `ClusterName`, `Creator`: set on clusters. Display only
- Click **Activate**
   - [Databricks docs: Default tags on AWS resources](https://docs.databricks.com/aws/en/admin/account-settings/usage-detail-tags)

> [!NOTE]
> **Timing:** a tag only appears in this list after AWS has seen it on a resource. Once it's activated, AWS can take up to another 24 hours to apply it. So allow up to two days in total before tags show up in your cost data.

> [!NOTE]
> **Instance pools:** instances that come from an instance pool only carry the pool's tags (plus your workspace's custom tags), not the cluster's. SELECT handles this by matching each instance to the clusters it served, using Databricks' own node timeline, so pool cost still reaches the right clusters and jobs. Time an instance spent idle in the pool shows up as idle pool cost. Activate the pool tags above either way.

Optionally, but recommended, backfill the tags so earlier months are included. By default, tags only apply to cost data from the moment you activate them. You can backfill them for **up to the last 12 months**:

- On the same **Cost allocation tags** page, click **Backfill tags** (top right)
- Pick the month to start from, then click **Confirm**
   - You can make one backfill request every 24 hours. AWS only fills in a tag value where the resource actually had that tag at the time, and the updated history reaches your data export within about 24 hours
   - [AWS docs: Backfill cost allocation tags](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-allocation-backfill.html)

#### Step 2: Create a CUR 2.0 data export

- Open **Billing and Cost Management → Data Exports**, then click **Create export**
   - [AWS docs: Creating data exports](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-create-standard.html)
- **Export type:** choose **Standard data export**
- **Export name:** pick a memorable name (e.g. `select-aws-cost-export`), and note it down
- **Data table configurations:**
   - **Data table:** CUR 2.0
   - Check **Include resource IDs**
   - **Time granularity:** Hourly
   - Leave the other options (such as **Split cost allocation data** and **Include Capacity reservation data**) at their defaults
- **Column selection:** select all columns, using the checkbox in the table header
- **Data table delivery options:**
   - **Compression type and file format:** Parquet
   - **File versioning:** Overwrite existing data export file
   - **Data export refresh cadence:** Daily (the default)
   - **Report data integration:** leave unset
- **Data export storage settings:** choose **This account**, then **Configure**
   - We recommend creating a new bucket for the export (**Create a bucket**, then pick a name and a **Region**). Note down the bucket name and its region
   - **S3 path prefix:** pick a prefix (e.g. `cur`), and note it down
- Click **Create**

> [!WARNING]
> AWS writes the bucket policy it needs to deliver the export. If you select an existing bucket instead of creating one, Data Exports **overwrites that bucket's policy**, and the console asks you to confirm it. Don't edit the policy later: changing it can stop deliveries. That includes the `us-east-1` region in its `SourceArn`, which is correct whatever region your bucket is in.

AWS delivers the first export within 24 hours, then refreshes it at least once a day. The files land in:

```markdown
s3://<your-bucket>/<your-prefix>/<export-name>/
    data/BILLING_PERIOD=YYYY-MM/<export-name>-00001.snappy.parquet
    metadata/BILLING_PERIOD=YYYY-MM/<export-name>-Manifest.json

```

> [!TIP]
> **Want older history?** A new export starts from the current month. To get earlier months, open a case with AWS Support from the payer account and ask them to backfill the export. SELECT loads up to 180 days of history.

#### Step 3: Create a read-only IAM policy

SELECT needs **read-only access to the export files**, and nothing else.

- Open **IAM → Policies → Create policy**, then switch to the **JSON** editor
- Paste the policy below, and find/replace the three placeholders:
   - `<BUCKET>`: the bucket name from Step 2, without `s3://`
   - `<PREFIX>`: the S3 path prefix from Step 2 (e.g. `cur`)
   - `<EXPORT_NAME>`: the export name from Step 2 (e.g. `select-aws-cost-export`)

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListCURBucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::<BUCKET>"
        },
        {
            "Sid": "ReadCURObjects",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::<BUCKET>/<PREFIX>/<EXPORT_NAME>/*"
        }
    ]
}

```

- Click **Next**, give the policy a memorable name (e.g. `SelectCURReadOnly`), and click **Create policy**

#### Step 4: Create an IAM user

- Open **IAM → Users → Create user**
- **User name:** pick a memorable name (e.g. `select-cur-reader`)
   - Leave **Provide user access to the AWS Management Console** unchecked
- On **Set permissions**, choose **Attach policies directly**, then select the policy from Step 3
- Click **Create user**

#### Step 5: Create an access key

- Open the new user's **Security credentials** tab. Under **Access keys**, click **Create access key**
   - [AWS docs: Managing access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-keys-admin-managed.html)
- On **Access key best practices & alternatives**, choose **Third-party service** (or **Other**), confirm, and click **Next**
- Optionally add a description tag, then click **Create access key**
- On **Retrieve access keys**, note down the **Access key ID** and **Secret access key** (or click **Download .csv file**)
   - AWS shows the secret only once
   - You can rotate the key at any time: create a new one, update it in SELECT, then deactivate the old one

### Add AWS account to SELECT

- Navigate to: [https://select.dev/app/settings/aws/add](https://select.dev/app/settings/aws/add)
- Fill out the details you noted in the previous steps:
   - **Display Name**: any name you'll recognize (e.g. `Production AWS billing`)
   - **Payer Account ID**: your 12-digit payer account ID, shown in the top-right account menu of the AWS console (e.g. `123456789012`)
   - **S3 Bucket**: the bucket name only, without `s3://` (e.g. `my-cur-bucket`)
   - **S3 Prefix**: `<PREFIX>/<EXPORT_NAME>`, the folder that contains `data/` and `metadata/` (e.g. `cur/select-aws-cost-export`)
   - **AWS Region**: the region your S3 bucket is in (e.g. `us-east-1`)
   - **Access Key ID**: from Step 5
   - **Secret Access Key**: from Step 5
- Click **Save**. SELECT runs three checks and shows the result of each:
   - **Reach the S3 bucket**
   - **Find a CUR manifest under the prefix**
   - **Read the first manifest object**
- When all three pass, the connection is saved and the first sync starts. You are good to go!

## What happens next

- **The first sync** loads every billing period in the export, up to 180 days back. After that, SELECT picks up each new delivery automatically. AWS keeps restating the current month, and can update the previous month for up to two weeks after it ends, so SELECT reloads both months on every sync
- **Where you'll see AWS cost:**
   - On Databricks pages, next to DBU spend for each warehouse, cluster, job and query, as **Cloud spend**
   - In **Total spend**, which is DBU spend plus Cloud spend
   - In **Cost Explorer**, under the AWS platform
- **Serverless compute** shows `-` for Cloud spend. Databricks runs serverless on its own AWS account and includes the infrastructure in the DBU price, so there's no separate AWS charge on your bill
- **AWS cost that doesn't belong to Databricks** (other workloads, shared networking, tax) still appears in Cost Explorer under AWS, so the full bill adds up

## Troubleshooting

#### Reach the S3 bucket

- **Likely cause:** wrong bucket name or region, or the key is wrong or deactivated
- **Fix:** check the bucket name has no `s3://` or trailing `/`. Check the region matches the bucket's region in S3. Re-copy both keys.

#### Reach the S3 bucket (AccessDenied)

- **Likely cause:** the policy doesn't cover the bucket, or isn't attached to the user
- **Fix:** check the first statement's `Resource` is `arn:aws:s3:::<BUCKET>` and the policy is attached to the user. If you've set the bucket to encrypt with a KMS key you manage, also allow `kms:Decrypt` on that key.

#### Find a CUR manifest under the prefix

- **Likely cause:** the first export hasn't arrived yet, or the prefix is wrong
- **Fix:** AWS can take up to 24 hours to deliver. In S3, check that `<PREFIX>/<EXPORT_NAME>/metadata/` exists and contains a file ending in `Manifest.json`. Enter the prefix without a leading `/`.

#### Read the first manifest object

- **Likely cause:** the policy's object path doesn't match where the files are
- **Fix:** check the second statement's `Resource` is `arn:aws:s3:::<BUCKET>/<PREFIX>/<EXPORT_NAME>/*`, matching the folders in S3 exactly, including case.

#### Exports stopped arriving (an Invalid bucket error in Data Exports)

- **Likely cause:** the bucket policy or bucket owner changed after setup
- **Fix:** restore the bucket policy Data Exports created, or recreate the export.

#### Connected, but Cloud spend is - for classic compute

- **Likely cause:** tags not activated (Step 1), or not yet in the data
- **Fix:** activate the tags, and backfill them for earlier months. Allow up to two days for the tags to reach your data.

Stuck? Reach out to your SELECT contact with the failed check's message. It includes the AWS error code, which is usually all we need.

---

# Workloads

URL: https://select.dev/docs/reference/using-select/workloads

> Categorize SQL queries into workloads

## Overview

SELECT helps Snowflake users understand their usage by categorizing SQL queries into **workloads**. By calculating the [cost of each query](https://select.dev/posts/cost-per-query) in a workload, SELECT can surface your top consumption drivers. This helps customers prioritize optimization efforts and ensure they are effectively using Snowflake.

![SELECT workloads](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/6e56ffe9-d474-4a86-4df4-57445bfed000/width=3024,quality=75)

## Query Patterns

By default, SELECT categorizes all queries into individual "Query Pattern" workloads. All queries are run through a custom query parser which standardizes the SQL into a recurring Query Pattern. In the example below, literals in the where filters are stripped out in each query, resulting in both queries being matched to the same pattern.

![SELECT Snowflake query parser](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/marketing/query-parser.png/width=3194,quality=75)

By analyzing the cost of **Query Patterns** instead of individual queries, users can spot frequently running queries that add up in cost.

![SELECT Snowflake Query Patterns](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/5f0faa17-1f63-4499-84dd-d109a2d40900/width=3024,quality=75)

Each Query Pattern has a dedicated page where SELECT users can see an overview, costs and performance statistics over time, and the full query history.

![SELECT Snowflake Query Pattern page](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/18a7384b-6013-4bd6-ceb1-57846f0eba00/width=3024,quality=75)

## Metadata-Based Workloads

One of the downsides of Query Patterns is that structural changes to the SQL text will result in a new Query Pattern being created. This can make it difficult to understand the impact of changes to a query that is associated with a business process. Additionally, users must analyze the query text, username and warehouse of a Query Pattern to understand what system or business process it is a part of.

To support this, SELECT has created several different **metadata-based** workloads. Instead of grouping queries that have the same underlying SQL pattern, SELECT leverages query comments and [query tags](https://select.dev/posts/snowflake-query-tags). This allows for a more natural and flexible grouping of queries. For example, a given [Looker dashboard](https://select.dev/docs/looker) can consist of 50 different queries that run every time the dashboard is refreshed. The individual queries can be cheap, but, when grouped, quite expensive.

![SELECT Looker integration dashboard overview](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/looker.png/width=2048,quality=75)

Similar to Query Patterns, each workload has a dedicated page with cost & performance trends, and [optimization insights](https://select.dev/docs/insights).

The following is a list of metadata-based workloads supported by SELECT:

- [Custom Workloads](https://select.dev/docs/custom-workloads): add your own tags / comments to queries running in your account
- [dbt](https://select.dev/docs/dbt): understand the cost of all dbt resources (models, tests) and analyze your run performance
- [Fivetran](https://select.dev/docs/fivetran): analyze the compute costs from Fivetran loading each table into your account
- Analyze the cost of each dashboard or business user in your BI Tools:
   - [Looker](https://select.dev/docs/looker)
   - [Mode](https://select.dev/docs/mode)
   - [Periscope](https://select.dev/docs/periscope)
   - [Hex](https://select.dev/docs/hex)

## Tasks

Customers often use [Tasks](https://select.dev/posts/snowflake-tasks) to orchestrate their various workloads in Snowflake. SELECT provides native cost and performance monitoring support for Snowflake Tasks. To view details about your Tasks, click on Workloads -> Tasks from the sidebar. Similar to other workloads, you'll see an overview of all Tasks in your account:

![SELECT cost per Snowflake task](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/cf2f86d9-7244-4854-5232-ed77bd21a200/width=3024,quality=75)

Click on any task to see details around its configuration, along with costs, performance and any related [insights](https://select.dev/docs/insights):

![SELECT Snowflake task page](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/33e095e0-2ffa-4f73-b8ed-bba61ee8bc00/width=3024,quality=75)

In this example above, you'll see the task calls a stored procedure `call dev_select_ingest.fernando.sproc_root(0)` each time it runs. From the Run History tab, you can drill into individual runs of a task to see all the queries kicked off as part of that stored procedure call:

![SELECT Snowflake task run timeline](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/220fdf80-5d7d-4bfc-8f4f-b4038ee6b200/width=3024,quality=75)

> [!NOTE]
> Serverless tasks are included on the Tasks page. Our [cost-per-query](https://select.dev/posts/cost-per-query) algorithm has been updated to account for queries that run on serverless compute.
>
> You can see the type of task using the "Task Compute Type" column, and filter on it accordingly.

## Stored Procedures

[Stored Procedures](https://select.dev/posts/snowflake-stored-procedures) are often created to run specific "jobs" or "workloads" in Snowflake, like loading a table or triggering an ETL process. SELECT provides native cost & performance tracking for Stored Procedures, by grouping together the `CALL` query triggered by a Stored Procedure, along with all the child queries it may kick off in the same session. To view details about your Stored Procedures, click on Workloads -> Stored Procedures from the sidebar:

![SELECT Snowflake cost per stored procedure](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/31ada12f-0572-44c0-b0d3-cdfa135d0a00/width=3024,quality=75)

Click on any procedure to see details around its configuration, along with costs, performance and any related [insights](https://select.dev/docs/insights):

![SELECT Snowflake stored procedure page](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/a93052b4-cbc5-4bc8-9baf-9ab7789bb300/width=3024,quality=75)

From the Run History tab, you can drill into individual queries kicked off as part of that stored procedure call:

![SELECT Snowflake stored procedure run timeline](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/60833b20-f52d-4e3c-e374-42a65a13bb00/width=3024,quality=75)

## Dynamic Tables

Snowflake Dynamic Tables are a feature that allows you to define tables that are automatically and incrementally updated as the underlying data changes, based on a query definition. SELECT provides native cost and performance insight for Dynamic Tables by tracking execution of both the queries that monitor data freshness of upstream sources and the queries that refresh the table.

To view the details about your Dynamic Tables click on Workloads -> Dynamic Tables in the sidebar:

![SELECT Snowflake Dynamic Tables](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/b6b7ee8a-b713-4d70-1cc7-d68c67d41000/width=3024,quality=75)

Click on any Dynamic Table to see details it's configuration along with costs, performance, and related [insights](https://select.dev/docs/insights):

![SELECT Snowflake Dynamic Table Overview](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/161d1bb6-45c7-4096-acf7-055ea6e65b00/width=3024,quality=75)

In the Run History tab, you can drill into both the freshness monitoring and refresh queries for a Dynamic Table:

![SELECT Snowflake Dynamic Table Run History](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/a23c5502-3c4a-45a1-a2a8-463e92b39b00/width=3024,quality=75)

## Serverless Workloads

SELECT also surfaces the cost of different **serverless** workloads running in Snowflake:

- Automatic Clustering
- Snowpipe
- Serverless Tasks
- Search Optimization
- Materialized Views (coming soon)
- Database Replication (coming soon)

---

# Lineage

URL: https://select.dev/docs/reference/using-select/lineage

> Use data lineage to understand total data asset cost and ensure ROI

[YouTube video](https://youtu.be/9LhNRXRVUPg)

SELECT’s end-to-end data lineage feature helps users:

1. Easily understand the upstream and downstream dependencies of any table, view, or data asset — across **Snowflake, Databricks, and BigQuery**.
2. Quickly understand resource usage and costs from the Lineage sidebar
3. Understand total data asset costs & identify optimization opportunities

## Understand data dependencies

SELECT’s lineage feature can be accessed from any [workload](https://select.dev/docs/workloads) or table page in the UI. By default, SELECT will load all direct upstream and downstream dependencies.

![SELECT Total cost lineage](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/d433e7b6-4930-4a03-de2d-55b41e3bf900/width=3024,quality=75)

Users can expand the lineage graph by clicking the buttons on each node.

[Video](https://storage.googleapis.com/prod_select_images/docs/expand_lineage.mp4)

SELECT’s lineage features shows both datasets (tables) and workloads (queries, dbt models, tasks, etc.). Tables are shown as rectangles, whereas workloads are shown as ovals.

![SELECT lineage workload versus table](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/83903e4b-642c-46d9-f5de-b3082b43ac00/width=3024,quality=75)

If you just want to analyze data dependencies and not see workload details, you can select the “Collapse Workloads” option to minimize the workload nodes.

![SELECT lineage collapsed workloads](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/7740d550-3a87-4cef-a8bc-880c33296b00/width=3024,quality=75)

## Understand resource details with the sidebar

Users can quickly get more context without navigating away from the lineage view by clicking on any of the nodes. A sidebar will automatically open containing information about the asset, its cost trends, and associated users.

![SELECT lineage sidebar](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/af69f913-cf62-47bd-967d-a5af6af27100/width=3024,quality=75)

## Understand total data asset cost

When looking at costs from tables and queries in isolation, opportunities to save costs from removing unnecessary resources are often disregarded since the cost savings from those individual resources alone are often not significant.

With SELECT’s data lineage feature, users can now easily see the total cost of all these interconnected processes:

![SELECT lineage cost view](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/d06797bc-d015-4e92-982d-c89755a2cc00/width=3024,quality=75)

Using this view, users can spot new optimization opportunities like:

1. Removing an entire group of related resources which aren’t driving enough business value relative to their combined cost
2. Identifying resources that are updating more frequently than required by downstream users
3. Spotting unnecessary costs they were not aware about.

## Refine the lineage view

Users can refine the lineage view to their needs using the following options:

1. Collapse the rounded workloads nodes to focus purely on the data dependencies and flows
2. View lineage at a different point in time using the date range selector
3. Exclude development databases/schemas or workloads run by certain users using the available filters
4. Hide the cost statistics shown on each node for a simplified dependency view

## Limitations

There are currently a few scenarios where data lineage will not be generated for a given resource in SELECT:

- If there are no downstream/upstream dependencies for a given workload. Examples of this would be a query which writes data to an external stage.
- Dynamic Tables have only partial support due to Snowflake reporting limitations.

> [!WARNING]
> Our lineage feature relies on the [Snowflake access history view](https://select.dev/posts/snowflake-access-history) which is not available for customers on the [Standard Edition of Snowflake](https://select.dev/posts/snowflake-pricing#snowflake-editions).

---

# Insights

URL: https://select.dev/docs/reference/using-select/insights

> An overview of key optimization opportunities

## Overview

Teams want to know whether they're using their data platform efficiently. Every day, SELECT intelligently scans all the different resources and workloads running in your account and identifies the potential savings associated with a variety of different optimization opportunities. These optimization opportunities are surfaced as insights through the SELECT web application.

### Individual Resource Pages

When applicable, optimization opportunities are highlighted on the individual resource pages throughout SELECT. Here's an example shown for a dbt model with an exploding join resulting in excessive costs:

![SELECT Snowpipe recommendation](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog-posts-0-1582c36404ccd71cb1f0fa067e49c2406d6c08545bafd2fd7ed844b6533c267e/width=3024,quality=75)

### Central Insights

To help users instantly discover all the different optimization opportunities in their account, we also surface Insights on a centralized page, accessible from the sidebar.

The "Overview" tab highlights your top optimization opportunities, showing the total savings potential, the estimated effort to implement this optimization, and the number of resources the insight applies to.

![SELECT Snowflake Insights feature part 1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog-posts-1-a1d6dd0ace41c8df804011cc78bebd2231fe42dbeda403ca0535c7b34c1dbf1f/width=3024,quality=75)

On the Insights "Explorer" tab, you can browse, filter and sort through all your optimization opportunities.

![SELECT Snowflake Insights feature part 2](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/insights3.png/width=3138,quality=75)

Each row is clickable and reveals an Insight detail card showing additional information about the insight, how the potential savings were calculated, and the estimated effort required to action this opportunity.

![SELECT Snowflake Insights feature part 3](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/insights4.png/width=3138,quality=75)

### Dismissing Insights

SELECT's insights engine generates a broad range of insights for your account. Some insights won't be actionable, due to the nature of the data or latency requirements as examples.

Insights can be dismissed either from the insight detail card, or using the bulk dismissal action. To dismiss multiple insights, click the ellipsis button to the right of 'All Insights' and then 'Bulk dismiss'.

![Dismissing insights in SELECT](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/8a27df0e-a3ea-44a1-947c-da86dda6a900/width=3024,quality=75)

A note explaining why the insights were dismissed can optionally be added for future reference.

![Dismissing insights in SELECT](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/fa927239-514e-4015-95c5-3f51d027cd00/width=3024,quality=75)

To see dismissed insights, use the dismissed status filter. Dismissed insights can be easily un-dismissed again through the insights detail card or in bulk. Dismissed insights show when the insight was dismissed, by whom, and the comment if supplied.

![Dismissing insights in SELECT](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/1208e70b-1afc-4d82-7426-3c1ee4745e00/width=3024,quality=75)

Note that dismissing insights requires the editor or admin role within SELECT.

---

# Usage Groups

URL: https://select.dev/docs/reference/using-select/usage-groups

> Create flexible cost categories in SELECT

## Feature Overview

Usage Groups provide a flexible way of creating cost categories within SELECT. They can be used to create showbacks or chargebacks for teams in your organization. That helps:

1. Reduce the burden on central teams playing the role of 'query cop' 🚓
2. Distribute costs across organizational units
3. Encourage efficient use of Snowflake
4. Flexibly allocate costs for shared warehouses using query metadata, [user roles](https://select.dev/posts/snowflake-roles), and more

[Video](https://storage.googleapis.com/prod_select_images/docs/usage_groups.mp4)

## Managing Usage Groups

Usage Groups are managed from the Usage Groups page, accessible from the sidebar.

### Usage Group Sets

Usage Group Sets are containers for Usage Groups. Typically, an organization will create a Usage Group Set called 'Teams' to categorize spend by team. Additional Usage Group Sets can be created for different spend categorizations, such as 'Projects'.

Create your first usage group set by heading to the Usage Groups tab in the sidebar, and then clicking 'Manage Sets' from the 'Usage Group Set' dropdown.

[Video](https://storage.googleapis.com/prod_select_images/docs/usage_group_sets.mp4)

### Creating Usage Groups

Usage Groups consist of a name and set of rules. To create a new Usage Group, click the 'New Group' button. An empty Usage Group will be created:

![Creating a new Usage Group](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/8b888e46-f472-4bea-87fa-b9d88c4f1e00/width=2044,quality=75)

Start typing in the input to get suggestions on corresponding values, along with their cost. Rules can be added and deleted using the buttons on the right.

### Usage Group Order

Usage Groups can have overlapping rules. The order of the Usage Groups determines the precedence when assigning workloads. To adjust the order, click and drag using the triple line buttons.

### Options for Account-Level Usage Groups

You can create Usage Groups for **individual Snowflake accounts** based on **Snowflake resources**:

- Warehouse
- Database
- Automatic Clustering
- Snowpipe

Or more flexibly using **query metadata**:

- User Name
- Role Name
- dbt metadata
   - [dbt meta keys](https://select.dev/docs/dbt#leveraging-dbt-meta-and-tags)
   - dbt target name
   - dbt target database & schema
   - dbt resource database & schema
   - dbt resource type
   - custom top-level keys
- [Custom metadata fields](https://select.dev/docs/custom-workloads#leveraging-custom-metadata-fields)
- Looker metadata
   - Dashboard title
   - Looker user name
- Fivetran schema name

### Using Snowflake Object Tags

Rather than specifying the names of Snowflake resources, SELECT users can assign their Snowflake users, roles, warehouses, databases to Usage Groups based on any Snowflake object tag associated with that resource.

![To help users more easily manage their Usage Groups, we’ve added support for object tags. Automatically assign users, roles, warehouses, databases to Usage Groups based on any Snowflake object tag associated with that resource](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/1ae8ed3a-5f35-4719-9aa1-993da1cece00/width=1920,quality=75)

### Options for Organization-Level Usage Groups

When creating Usage Groups for your entire **Snowflake organization** (from the [Organization Section](https://select.dev/docs/snowflake-organization) of SELECT), you have the following options:

- Snowflake Account Name
- Service (Compute, Storage, Data Transfer, Serverless Tasks, etc.)
- Snowflake Region (AWS_US_EAST_1, GCP_US_EAST4, AZURE_WESTEUROPE, etc.)
- Warehouse Name

### Usage Group Rules

When assigning Snowflake resources or query metadata fields to a Usage Group, you have three options:

1. **in**: The 'in' operator matches to exact values, and is case sensitive.
2. **includes**: The 'includes' operator matches to sub-values, and is case insensitive. For example, if you wanted to flexibly assign all databases that include `_dev` in the name, you would use this option.
3. **starts with**: The 'starts with' operator matches only on the start of strings. For example, to match on all warehouses starting with `BI_`, you could use this option.

### Usage Group Version History

Usage Group updates are tracked and can be restored from the version history. To view the version history, click on the "Version History" button on the usage groups page. This will show a list of all previous versions of the usage groups. To view the definitions for a version, simply click on the version, and it will be displayed in the usage group definitions UI. To restore a version, click on the "Restore" button. This will create a new version from the historic version.

![Usage group versions](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/8dd1deb3-8ee7-4a50-f21c-f9ec30775400/width=3024,quality=75)

## Usage Groups throughout SELECT

Usage Groups can be utilized throughout SELECT, including insights. The all workloads page provides the ability to filter to specific Usage Groups, in addition to slicing.

![Using Usage Groups on the all workloads page](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/c6a0bd8a-1723-45db-cf62-7634a88b7700/width=3456,quality=75)

## Sending automated reports for Usage Groups

In order to help raise Snowflake cost awareness in your company, we recommend setting up a [Monitor](https://select.dev/docs/monitors) so that each Usage Group can receive a scheduled [Spend Digest](https://select.dev/docs/monitors#spend-digests). Here's an example Weekly Spend Digest report sent to Slack:

![SELECT usage group weekly spend digest](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/2b25f42d-0bab-42d0-93f7-375a8ff4ba00/width=1198,quality=75)

## Upcoming improvements

In future releases, we will provide versioning for usage groups, and the ability to import and export via JSON and YAML. We are continually adding support for more Snowflake services to usage groups. If you have a specific attribute or Snowflake resource you'd like to use with Usage Groups, please reach out to the SELECT team.

---

# Budgets

URL: https://select.dev/docs/reference/using-select/budgets

> Set and manage budgets across different areas

## Feature Overview

Take control of your cloud costs by setting budgets for teams, projects, or any combination of resources. SELECT tracks your spend and uses forecasting to predict whether you'll stay within budget—so you can take action before overruns happen.

## Setting a Budget

To create a new budget, navigate to the Budgets page from the sidebar and click **Create Budget**. The configuration form guides you through:

1. **Define Scope**: Choose organization-wide or team-owned, and optionally add filters for specific warehouses, databases, users, roles, or query metadata (dbt, Looker, Fivetran).

![Budget Name + Scope](https://cdn.sanity.io/images/wn0ylp2c/production/f99c3c1cdb5cda1cdf7a138748064e1c247a2475-1678x822.png)

**2\. Set Amount & Period**: Enter the budget amount and select a schedule (weekly, monthly, quarterly, yearly, or custom)

![Budget Frequency + Period](https://cdn.sanity.io/images/wn0ylp2c/production/b282fa6b001b3e5153fdedb01c97fd30f09c9aa0-1684x1200.png)

**3\. Set Expiration** (optional): Configure when the budget should stop tracking

![Budget Expiration](https://cdn.sanity.io/images/wn0ylp2c/production/ec1e72f41f2513f8b73d2b9721abe1213c81fd6a-1662x856.png)

As you configure your budget, a real-time preview shows historical spend and forecast projections.

![Configure Tab](https://cdn.sanity.io/images/wn0ylp2c/production/8da3a09799df02e0c8b42bc4d813fa1ed0e8cfaa-3024x1656.png)

### Budget Periods

Budgets support flexible scheduling

- **Fixed Range:** Single period with explicit start and end dates
- **Weekly / Monthly / Quarterly / Yearly:** Resets automatically on the chosen cadence
- **Custom:** Custom repeat interval (e.g., every 2 weeks)

## Monitoring Budgets

### **Budget Status**

Each budget displays a status based on forecasted spend:

- **On Track**: Forecasted spend is within budget
- **At Risk**: Forecasted spend is within 5% of the budget
- **Overspending**: Forecasted spend exceeds the budget by 5% or more
- **Expired**: The budget period has ended

### **Overview Dashboard**

The Budgets page shows summary statistics at the top:

- Total active budgets
- Number of budgets overspending or at risk
- Budgets expiring soon

Use the **Grid View** for visual cards with spend charts, or **Table View** for a sortable list with detailed columns.

![Grid View](https://cdn.sanity.io/images/wn0ylp2c/production/5780fc7a6e9d0c7ee4a64c0ff31c62552903895d-3024x1652.png)

![Table View](https://cdn.sanity.io/images/wn0ylp2c/production/c1373d32f8224acfa3937df07a3d2e54bdad3268-3024x1654.png)

## **Checking Individual Budgets**

Click any budget to see its detail page with:

- Spend chart showing actual vs. budgeted spend with forecast
- Current period dates and progress percentage
- Budget configuration and filters

![Budget Overview Tab](https://cdn.sanity.io/images/wn0ylp2c/production/4060507f06119ed98d4dfd82726824b2a390a3ee-3024x1654.png)

## Upcoming Improvements

- Budget monitors and alerts
- Support viewing previous budget periods

---

# Monitors

URL: https://select.dev/docs/reference/using-select/monitors

> SELECT’s Monitors feature helps you get timely notifications about your data platform.

- [Overview](https://select.dev/docs/reference/using-select/monitors/overview.md): Overview of SELECT's Monitors feature
- [Configure Destinations](https://select.dev/docs/reference/using-select/monitors/configure-destinations.md): Choose where you will be notified for alerts from your monitors
- [Using Monitors](https://select.dev/docs/reference/using-select/monitors/using-monitors.md): Learn how to view and use monitors within SELECT's App
- [Custom SQL Monitors](https://select.dev/docs/reference/using-select/monitors/custom-sql.md): Learn how to create your own monitors for a SQL query
- [Best Practices and Troubleshooting](https://select.dev/docs/reference/using-select/monitors/best-practices-and-troubleshooting.md): Learn best practices and troubleshooting for Monitors

---

# Overview

URL: https://select.dev/docs/reference/using-select/monitors/overview

> Overview of SELECT's Monitors feature

Staying on top of a Snowflake environment is challenging. With thousands of active users, new workloads added every day, and complex data pipelines, it’s hard for platform teams to see everything happening in their account.

SELECT’s **Monitors** feature makes it easier. Monitors send timely notifications about critical events in your Snowflake account so you can respond quickly.

## Monitor Types

A Monitor type is defined by two parts: a **data source** and a **trigger condition**.

**Data sources available in SELECT:**

- **Account Spend** – Track spend across an entire account.
- **Warehouses** – Monitor metrics such as spend, credits, query executions, and more, for one or multiple warehouses.
- **Usage Groups** – Track spend by Usage Group.
- **Workloads** – Monitor high-cost workloads, slow queries, and related activity.
- **Queries -** Your Snowflake query history enhanced with SELECT data, such as Usage Groups, Workloads, etc.
- **Storage -** Table and Stage storage billed by Snowflake.
- **Insights** - Our custom insight categories as seen on the Insights page.
- **Other Data Sources** – dbt, Looker, Tableau
- **Custom SQL** – Define your own query as the source.

**Trigger Conditions:**

- **Scheduled Report** – Send results on a set schedule.
- **Conditional Monitors:** 
   - **Results Found** – Only notify when the query returns records.
   - **Threshold Monitors** – Notify when a metric exceeds a specific threshold within a time period (ideal for spend spikes, performance regressions, and more).
   - **Anomaly Monitors** – Use machine learning to detect abnormal values automatically, factoring in seasonal patterns. Compared to Threshold Monitors, they don’t require manual tuning, though you can adjust sensitivity to balance detection speed with false positives.

See the Managing Monitors page for details on creating and managing each type of monitor and trigger conditions.

## Example Monitor Results

Here’s how a Monitor notification looks in Slack (with similar previews in Teams and Email):

![Example monitor output in from SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/0dea165e16f6b41648ba0b808f8d187472fc736e-1378x728.png)

---

# Configure Destinations

URL: https://select.dev/docs/reference/using-select/monitors/configure-destinations

> Choose where you will be notified for alerts from your monitors

SELECT supports sending Monitors to the following destinations:

- Email
- Slack
- Teams
- [OpsGenie](https://select.dev/docs/opsgenie)

You can configure destinations directly when creating a Monitor, or from **Settings → Notifications**.

From the **Configure Monitor** page, scroll to the [**Configure destinations and schedule**](https://select.dev/docs/reference/using-select/monitors/using-monitors#step-4-configure-destination-and-schedule) section. Use the **Select destinations** dropdown. If you don’t have any existing integrations, you can create one directly from this screen.

![Add monitor destination in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/f393056193efec3971cea18893fdfe1585f6f9c7-467x342.png)

Some customers prefer to manage integrations centrally in **Settings → Notifications**. Here you can view all existing destinations, manage integrations, and add new ones.

![Notifications settings in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cedfefa0fd2107ed13121bd7fd94010793e87f6d-1699x847.png)

The steps below use **Settings → Notifications** as the example workflow. The same process applies when configuring destinations during the Create Monitor flow.

## Step 1: Connect a Notification Integration

First, enable the notification integrations you want to use. Go to **Settings → Notifications** in the SELECT web app. From here you can connect Email, Slack, or Teams.

### Email

Click **Connect** to enable email. You’ll be prompted to provide an address when adding a destination.

### Teams

To connect Teams, click the "Connect" button. You'll be prompted to enter a webhook in the next step when adding a destination.

### Slack

To connect Slack, install SELECT’s Slack application by clicking **Connect**. You’ll see a screen like this:

![Slack / SELECT integration](https://cdn.sanity.io/images/wn0ylp2c/production/f8610adbad41222ee3c1bf37ddef1f6d3fe78365-1440x1158.png)

The SELECT Slack app only requests access to:

- List channels in your workspace
- Post messages

Once installed, you’ll be redirected back to SELECT.

To add another Slack Workspace, click the Slack logo in the **Integrations** section of the Notifications page, or use the Slack option in the sidebar. This opens a page with an **Add Workspace** button.

Our Slack app only requires access to list channels in your workspace and post messages. That's it! Once the installation is complete, you'll be directed back to this web page.

If you want to add another Slack Workspace, you can click the Slack logo in the Integrations section of the Notifications page, or you can click Slack in the left sidebar. This will bring you to another page with the Add Workspace button.

![Add additional Slack Workspace](https://cdn.sanity.io/images/wn0ylp2c/production/feb7f48bdff56f53960afaac3054cebd3ba436a6-616x256.png)

## Step 2: Add a Destination

After connecting an integration, the next step is to add a destination where Monitors will be delivered.

- **Email** → add an address
- **Slack** → select a channel
- **Teams** → select a channel
- Opsgeenie -> use email integration

> [!WARNING]
> Slack USERS:
>
> In order to send alerts to a private Slack channel, the SELECT app must be added to the channel. You can do this by typing `/invite` in Slack, or by typing `@SELECT` in the channel and hitting enter. You'll then see a prompt to invite the app to the channel.

Adding a destination can be done directly in the [Create Monitor workflow](https://select.dev/docs/reference/using-select/monitors/using-monitors#step-4-configure-destination-and-schedule), Step 4, shown below. If you don’t have any existing destinations, one can be added here.

![Add a distination for monitors in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/821352ff22704fcfd50b3af30500ffe8e8195385-1234x329.png)

## Troubleshooting:

#### I’ve updated the channel in my Teams destination but are digests are still going to the old one

- Microsoft Teams webhooks are created for specific channels and are only authorised to send to that channel. The channel name is just to help remember where digests will go.
- If you’d like to change the channel a destination sends alerts to, you can [create a webhook](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook?tabs=dotnet#create-incoming-webhooks-1) for the new channel and update your destination
- If you’d like to change the channel for one account without affecting other accounts, you [create a new webhook](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook?tabs=dotnet#create-incoming-webhooks-1) for a new destination then set the account you wish to change to use that destination

#### I’ve set up a Teams destination but there aren’t any digests coming through

- Make sure the account for which you want to receive spend digests has a destination and at least one period set (as per the last screenshot)
- When you configure spend digests for the first time, you should receive your first one within 24 hours. Reach out to the SELECT team if you do not.

---

# Using Monitors

URL: https://select.dev/docs/reference/using-select/monitors/using-monitors

> Learn how to view and use monitors within SELECT's App

## Creating a new Monitor

To create a monitor, you must have at least “Editor” role in SELECT; viewers cannot create or edit monitors.

From the sidebar, click **Monitors → + Create Monitor**. Then either choose a template or click **Continue without Template**.

![Monitor templates in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/00bd06ad5ad18a4f2eb352d2db26f5e86bfb60a0-898x581.png)

**Templates** help you get started quickly by pre-filling the form with common defaults. Templates are organized by category (e.g., Account Spend, Warehouses) as shown in the image above. They are just starting points; everything remains fully editable.

## Monitor Configuration

After selecting a template (or continuing without one), you’ll configure the Monitor details. The steps below use the **Warehouse Spend Monitor** template as an example, but the workflow is the same in all cases.

### **Step 1: Data Sources**

The first step is to choose a data source for the Monitor. Data sources align with the main pages in the SELECT app. In this example, the template has already pre-selected a data source, so we’ll leave it as is. Notice that the Monitor preview is already rendering on the right side of the page.

If you are creating a monitor from scratch, choose a data source, then continue.

![Data sources in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/9fba4b0103a898af2682e6ef2375a897cd95dd51-1685x998.png)

#### **Select a Partition Column (Optional)**

Partition monitors help detect more granular cost issues that would otherwise go unnoticed. They allow you to detect spend anomalies across individual Snowflake users, warehouses, databases, workloads, or any other dimension in SELECT! We will keep track of each segment's historical baseline and notify you if it deviates from that.

Example use cases:

- Notify me if any warehouse (new or existing) spikes above it's historical baseline
- Notify me if any users exceeds $100 in spend per day
- Notify me if any dbt model or dynamic table spikes abnormally in cost OR performance

![Partition monitors](https://cdn.sanity.io/images/wn0ylp2c/production/dfb931874727f750a1f19aef3a1249b88f4f666f-3014x1956.png)

### Step 2: Choose a metric and Filter

**Metric Selection**

For most data sources, you can monitor by any metric available in SELECT. Some data sources have fewer available metrics, for example, Account Spend can only be monitored by cost.

![Change metric in monitor in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/48db5c0f9f929115ad8dcd9655e62aa6d21b7faa-1045x454.png)

If you want to monitor failed query executions instead of spend, just change the metric; the preview updates instantly.

**Filtering**

The same filtering functionality available throughout SELECT is also available in Monitors. If you need grouped criteria with OR logic or more complex conditions, you can convert the filter into an advanced filter.

![Add filter in monitor in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/eea453f98109d38f5401b2f23aadcd30c30cfa06-439x307.png)

The preview continues to update with every change you make.

### **Step 3: Choose Your Trigger Type**

Now choose the conditions under which the Monitor will run.

- **Scheduled Reports**: Send results on a fixed cadence that you specify in the next step. These send every time, even if no rows are returned. Use _Results Found_ if you only want notifications for non-empty results.
- **Conditional**: 
   - **Results Found**: Notify only when results are returned.
   - **Threshold**: Notify when a metric crosses a threshold. After entering the threshold, the value is automatically drawn on the chart in the preview.
   - **Anomaly**: Detect abnormal highs or lows using machine learning. When you select this option, you must also configure threshold bands, minimums, and sensitivity.

Here is an example where the alert sends only if there are more than 1,000 failed queries and the number also crosses the anomaly band.

![Anomaly monitors in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/55923302764c774626508c585f556476fe26a3e1-1670x624.png)

### Step 4: Configure Destination and Schedule

In Step 4 you will find the following fields to be filled out.

- **Team Assignment**: Monitors assigned to a team run with that team’s permissions. If the team only has access to certain usage groups, results will be filtered accordingly.
- **Choose where to be notified:** Use the picker to choose from existing destinations or create new ones.
- **Choose run frequency**: Monitors run after your daily sync is complete. You can schedule them daily, weekly, or monthly. For core data sources, run them immediately after SELECT updates the data. Custom SQL Monitors run live in your Snowflake account and can use a cron expression.
- Choose when to send the alert: Here you will find 3 options: 
   - **Every time it is triggered**: Runs on the chosen schedule, following your thresholds and conditional logic.
   - **Only once per status change**: Sends the first time the threshold is crossed. Dropping back below the threshold resets the status, so you will only get one alert per change. This reduces repeat alerts for the same issue.
   - **At most (x) per (frequency)**: For example, once per day. Another way to limit how often alerts are sent.

![Configure destinations in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/368514f824e8008ea42a50898c3ec276a18cea1c-1025x616.png)

### Step 5: Configure the Message

This step has several sections:

- **Title**: The name of the Monitor. This appears in the alert message and on the Monitors list page.
- **Include Message**: Toggle off if you do not want to include custom text in the alert.
- **Configure your message**: Use Markdown and Jinja to customize the body. By default this is pre-filled with Jinja to reference columns in your data.
   - In custom SQL, reference columns directly with `{{ row.column_name }}`.
   - Custom SQL Monitors are pre-filled to loop through all rows and columns returned.
- **Include Chart** and **Include Table**: Disable these if you do not want a chart or table in the alert.
- **Table Lookback**: Defines the data window for non-SQL Monitors. The default is yesterday. Changing this automatically updates the table title so readers know the time range.
- **Select Table Columns**: Choose which columns to display. Click a column to remove it, or use **Add Column** to include another.

The tables will always return the top 10 results, so as you change metrics, be sure to update the sorting of the columns so the correct results are included.

Continuing the example of “failed query executions”, it makes sense to remove all Spend columns and instead group by Warehouse and sum the Failed Executions. In order to ensure that my top 10 results are shown, I must sort by “failed executions descending”.

![Change sorting of monitor table in SELECT to ensure you get the correct rows.](https://cdn.sanity.io/images/wn0ylp2c/production/4da82adf1dd32db7671f2cc3af5922bf8ccd0308-610x307.png)

This produces a finished preview:

![SELECT monitor preview](https://cdn.sanity.io/images/wn0ylp2c/production/f60b12721caec3cef0b820d689bfd2077aae02d3-619x824.png)

### Send Preview and Save

Before saving the monitor, it is a good idea to send a test to the destinations. You can do this by clicking the Send button in the top right corner, also shown in the screenshot above.

Once you are satisfied, click the blue “Save Monitor” button.

## Managing Existing Monitors

All of your existing monitors can be found on the Monitors page by clicking Monitors on the sidebar.

- **Filter**: Use the filter bar at the top to search for specific Monitors. You can also create a saved views and favorite here.
- **Side Peek**: Open the drawer to view details such as the preview, status, and run history. From here you can also click **View Monitor** to edit.
- **Monitor name**: Clicking the name in the main table takes you to the edit screen.

![Manage existing monitors in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/c4340ad60e8037d6c7fe96c23eaf9d819775e05c-1453x1035.png)

**Enable / Disable monitors**

Monitors can be enabled or disabled using the toggle in the main table on the monitors page or the side peek, or in the Edit Monitor screen.

**Editing existing monitors**

To edit a monitor, click the Monitor name in the table. The next page is broken up into 3 tabs:

- **Overview:** shows you a preview of the monitor, the recent runs, plus other stats about the monitor.
- Run History: Provides a full run history of the monitor, including the thresholds and observed values.
- Configuration: This is where you will edit your existing monitors using the same interface that is used to create monitors.

---

# Custom SQL Monitors

URL: https://select.dev/docs/reference/using-select/monitors/custom-sql

> Learn how to create your own monitors for a SQL query

## What are Custom SQL Monitors?

While SELECT offers many types of monitors to help you stay on top of your Snowflake usage, these one size fits all solutions may not suit your needs. When our out-of-the-box monitors fall short, you can use Custom SQL Monitors to get alerted on anything in your Snowflake account that can be returned by a SQL query.

Configuring Custom SQL monitors is simple:

1. Configure the monitors schedule and where alerts will be sent
2. Define a SQL query
3. Customize the monitors message using the results of the SQL query

## Configure the monitors schedule

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/b317d251-1749-4db0-6a8f-e694b24bc400/width=932,quality=75)

### Scheduling frequency

There are a number of options for how often your monitor will run. SELECT syncs with your Snowflake account daily at around 3-5AM UTC, this will occur before any daily, weekly, or monthly monitors run.

- Hourly: Your monitor will run every hour
- Daily: Your monitor will run every day
- Weekly: Your monitor will run once a week on Monday
- Monthly: Your monitor will run on the first of the month
- Cron: Your monitor will run on a customizable schedule defined by a cron expression

### Destinations

Destinations determine where you will be alerted when your monitor runs, see: [configuring destinations](https://select.dev/docs/monitors/destinations) to learn more about setting these up.

### Sending Options

You can optionally choose to only have your monitor send a message if the SQL statement returns some records.

This is helpful when you only want to get alerted if results are returned from the query.

## Define a SQL query

You can define a single SQL expression which will run under the Snowflake user you’ve created for SELECT. By default, this means you’ll only be able to access data in the [Snowflake metadata database](https://docs.snowflake.com/en/sql-reference/snowflake-db). Grant the SELECT Snowflake service user access to additional databases if you want to build alerts off of other datasets.

It’s worth noting that we will automatically limit the results of your query to at most 50 rows, but we suggest putting a more aggressive limit than this on your query as some destinations may have character limits and your message will be truncated if it exceeds this limit.

### Limit Query to data since last run

SELECT exposes an optional system for limiting a monitor run to only the data since the last run that returned values.

If your query returns a value called **`select__incremental_key_ts`** the first row with this value will be stored and provided to the next run of the query. Here is the intended usage:

```sql
select
    query_id,
    start_time,
    max(start_time) over () as select__incremental_key_ts
from select_ingest.task_queries
where start_time >
    coalesce(
      try_to_timestamp('{{ last_run.select__incremental_key_ts | default("") }}'),
      to_timestamp_ntz('1970-01-01 00:00:00')
    )
limit 1;
```

#### Let's break it down

The `select` clause contains:

`max(start_time) over () as select__incremental_key_ts`

- Calculates and returns the maximum start_time across all rows in the result set
   - Every row will have the same value for this column
- Names it `select__incremental_key_ts` which is the special column name that triggers the incremental mechanism
- Uses a window function (`over ()`) to return this max value in every row

The `where` clause contains:

`start_time > TO_TIMESTAMP('{{last_run.select__incremental_key_ts | default("1970-1-1")}}')`

- Uses a template variable `{{last_run.select__incremental_key_ts}}` which contains the timestamp from the previous run
- The `| default("1970-1-1")` part provides a fallback value (Jan 1, 1970) if no previous monitor runs have returned a value
   - This isn’t obviously isn’t standard SQL, we’re using jinja to inject the string value into your query before running it
- Converts this string to a timestamp with `TO_TIMESTAMP()`
- Only returns rows with a start_time newer than the last run

## Customize the monitors message

After running your query you can edit a message template. A reasonable default will be generated to get you started.

This message template uses markdown for formatting and Jinja for powerful templating based on your query results.

Your markdown will be converted to a format that is compatible with your destination, and as such only markdown that is supported by your destination is applicable here.

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/6bea2f29-c111-4ac3-9402-dd5411800000/width=944,quality=75)

### Message structure

The area highlighted in purple is the name of the monitor, this can be configured at the top of the page.

The area highlighted in orange is fully customizable using the message template.

SELECT will also include a link to the monitor as a footer on your message, not shown here.

### Formatting Options

The top bar of the editor provides shortcuts for a variety of common formatting options, in order they are:

- Bolded text
- Italicized text
- Strikethrough text
- A link
- A bullet point list
- A numbered list
- A quote block
- An inline code snippet
- A code block

The preview is a close approximation of what the message will look like in emails, but the exact format of each style is dependent on the destination that you choose.

It's recommended to use the ‘Send preview’ button at the bottom of the page to double check formatting in your target destination is what you expect.

More advanced markdown features such as headers or image insertion will also work, but may yield unexpected results in some destinations. Notable examples include:

- Slack does not support headers, and will render bolded text instead.
- Teams does not support code blocks.

### Tag a user in Slack

You can alert a specific person in slack by including a `<@slack_member_id>` tag in your message. A user’s Slack ID can be copied from their profile in slack here:

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/b6b696a1-c103-48e6-3c80-cec1aa174100/width=527,quality=75)

If your query returns Slack ID’s, you might include them like this `<@{{row.slack_user_id}}>`.

### Adding in data from the query results with Jinja Templating

Your message can also use Jinja to inject information from your query or metadata about the monitor into your message.

We suggest exploring the options by clicking the ‘Insert variable’ variable button to see what’s available, as well as some useful snippets for common patterns.

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/258b2dc6-df13-48e7-5579-53135bf44300/width=1942,quality=75)

Let's take a deeper look at the suggested default message

```sql
A total of {{count}} rows were returned:
# Loop over the query results here #
{% for row in rows[:10] %}
- {{ row.sum_serverless_task_cost }}, {{ row.sum_query_cost }}, {{ row.start_day }}, {{ row.first_ever_start_time }}, {{ row.identifying_metadata }}
{% endfor %}
# Include any other useful information here, using markdown to format it #

# Next Steps
Optionally describe any action users should take off of this alert.
```

On line 1 we’re using `{{count}}` to inject the number of rows returned, a variety of other variables, such as the name of the monitor or when it ran are also available from the ‘Insert Variable’ menu.

Line 2 contains a comment that will not be included in the final message.

On lines 3-5, we loop over the first 10 rows returned by the query and include some of the results. The snippet generator will pick out the first few values for you, but any value can be displayed with `{{row.<column_name>}}`. If you would like to render a row but the column name isn’t a valid identifier, for example:

```sql
select sum(foo) from bar
```

You can also use the alternative `[]` syntax to render this value like this: `{{row['sum(foo)']}}`

To discover other row values, click ‘Insert Variable’ → ‘Row Values’ These snippets must be used within a for `{% for row in rows %}` style loop, as the `row` variable must be defined.

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/908556b7-121e-4f8a-d948-860185540b00/width=1932,quality=75)

If you only care about the first row, you can also insert these values like this `{{rows[0].some_value}}`!

### Using Jinja filters to format values

All [default Jinja filters](https://tedboy.github.io/jinja2/templ14.html) are available, here are some potentially useful examples:

1\. Truncate a long uuid to only 10 characters

`{{row.query_id | truncate(10)}}`

2\. Handle rendering a nullable column

`{{ row.maybe_this_is_null | default("N/A", true) }}`

3\. Format a float to 2 decimal places: `1234.5678` → `1234.57`

`{{ "%0.2f" | format(row.float_value | float) }}`

4\. Format a timestamp to a human readable date and time: `2024-01-01T12:00:00.000Z` → `Jan 1, 2024 at 12:00 PM`

`{{ row.timestamp_column | format_timestamp("%b %d, %Y at %I:%M %p") }}`

### Whitespace control

Markdown can often seem somewhat verbose when it comes to white space. In this example you might expect that bold and italics render on separate lines, but an empty new line is needed between them

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/8d1a3024-4dce-41a3-7061-9fd723000300/width=936,quality=75)

Markdown rendering occasionally requires a newline when you might not expect.

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/2d2f08b4-cbf6-4914-855b-6dfa97ea2300/width=962,quality=75)

Additionally Jinja provides a shorthand for removing whitespace on either side of a Jinja block, for example, here is a list of all column names returned by a query:

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/36252a41-8697-4b0d-f3d4-c889fb1f0b00/width=942,quality=75)

The `-` in `{%- endfor %}` tells Jinja to remove whitespace before the end of the block, without this, each column renders on a new line. SELECT runs Jinja with the default settings described [here](https://jinja.palletsprojects.com/en/stable/templates/#whitespace-control).

## Error Handling

If either your SQL query or the templated expression fails to send, you will receive an error message in your destination in its place.

## Example Custom SQL Monitors Use Cases

### Long running queries

This topic has some special nuances and deserves its own page. The details can be found [here in our FAQ](https://help.select.dev/articles/7345399580-how-can-i-monitor-long-running-queries-in-my-snowflake-account).

### Snowflake Services that Increased More than 50% in costs

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/7c633fbb-d0f3-4d10-e4f8-814321adbe00/width=1208,quality=75)

**SQL Query**

```sql
A total of {{count}} rows were returned:
{% for row in rows[:5] %}
* Query run by `{{ row.user_name }}` took `{{ "%0.1f" | format(row.total_elapsed_time_s | float) }}`s on {{ row.warehouse_size }} WH [→](https://app.snowflake.com/scwxhob/ad38017/#/compute/history/queries/{{row.query_id}}/detail)
{% endfor %}
```

****

**Message Template**

```markdown
The following resources had a usage increase of than 50%:

{% for row in rows %}
- {{ row.start_time }}: `{{ row.name }}` ({{ row.service_type }}) had credit usage of {{ row.credits_used }} (+{{ row.pct_change }}%)
{% endfor %}
```

### Failing Tasks

![SELECT Custom SQL Monitors](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/bd0b4f18-733d-4760-c704-04985af1fa00/width=2400,quality=75)

**SQL Query**

```sql
with
hourly_spend_by_service as (
    select
        start_time,
        service_type,
        name,
        credits_used,
        lag(credits_used) over (partition by service_type, name order by start_time) as prev_credits_used
    from snowflake.account_usage.metering_history
    where true
        and start_time > current_date - 7
)
select 
  *,
  round((credits_used - prev_credits_used)/prev_credits_used*100, 2) as pct_change
from hourly_spend_by_service
where true
		-- increased by more than 50% compared to previous hour
    and credits_used > prev_credits_used*1.5
    -- Set a minimum threshold, don't care about tiny spikes
    and credits_used > 1
    and start_time between current_date - 1 and current_date
order by start_time desc
```

**Message Template**

```markdown
The following resources had a usage increase of than 50%:

{% for row in rows %}
- {{ row.start_time }}: `{{ row.name }}` ({{ row.service_type }}) had credit usage of {{ row.credits_used }} (+{{ row.pct_change }}%)
{% endfor %}
```

### High warehouse queueing

For warehouses which back customer facing services, BI tools, or other workloads where fast response time is desirable, it may be beneficial to implement a queueing monitor for that warehouse. We can use the built in [warehouse_load_history](https://docs.snowflake.com/en/sql-reference/functions/warehouse_load_history) function in Snowflake to query relatively live load information for warehouses which the user has `MONITOR` privileges on.

It's worth noting that this monitor would be inappropriate for an ETL (or similar background task) warehouse, where query queueing is a normal and desirable feature for efficient resource utilization.

This monitor is designed to be used for **one warehouse at a time**, as it is unusual for the trigger parameters to be identical for each warehouse given their potentially diverse workloads.

**SQL Query**

When considering excessive queueing, there are three sensitivity parameters which we recommend you consider to minimize false positives without missing any true positives. If you're unsure how to determine appropriate values for these, we provide a _tuning query_ below which can be used to give you the right context. The parameters are:

1. The _time window_ over which we should evaluate the monitor (set to 1 minute below). Shorter windows allow a faster response to a spike in load, but they can create false positives during short transient spikes. In general we'd recommend setting this to ~3-4x the length of your _normal load_ spikes. This gives you some robustness against slightly longer spikes in normal operation, but would detect issues before they're 10x longer than normal.
2. The _intensity_ of queueing which you deem unacceptable. This depends entirely on your use case. If your warehouse serves customer queries directly, then you may set a lower threshold than for an internal BI tool warehouse where a small amount of queueing is more tolerable to users.
3. The number of time periods in your window which are over your intensity threshold. A blunt solution is to ignore this value (i.e. remove the `HAVING` clause below), which simply means the alert will fire if _any_ five second period in your time window is over the intensity threshold. In general, this can lead to unacceptable false positives however, and so we would recommend that roughly 50% is a good starting point (i.e. your _time window_ in seconds, divided by ten).

To tune your parameters, you can use the following query to see what data your target warehouse would currently give you. Plotting this as a chart in snowsight, can give you insight into how intense, and how long your normal spikes in queueing are. You'll also be able to examine the approximate latency of the data (by looking at the timestamp of the freshest value).

```sql
select warehouse_name, start_time, avg_queued_load
from table(
    -- See: https://docs.snowflake.com/en/sql-reference/functions/warehouse_load_history
    snowflake.information_schema.warehouse_load_history(
        -- For diagnosis, choose a period of high load, ideally a recent
        -- time users have noticed slow responses.
        -- The data is returned in five second time slices.
        date_range_start=>dateadd('minute',-60, current_timestamp()),
        -- Replace this with the name of the warehouse you
        -- want to monitor
        warehouse_name=>'<TARGET_WAREHOUSE>'
    )
)
order by start_time desc
```

Using the query above, you can use the query below as a template for your monitor. Adjust the provided defaults as appropriate for your use case.

```sql
-- Warhouse Queueing Monitor Query
-- -------------------------------
-- The data is returned in five second time slices.
-- By counting the returned rows, we look for a certain
-- number of periods in the last minute that were over
-- the queueing threshold.
select warehouse_name, count(*) as number_of_periods
from table(
    -- See: https://docs.snowflake.com/en/sql-reference/functions/warehouse_load_history
    snowflake.information_schema.warehouse_load_history(
        -- Parameter #1: --
        -- Here we define how far back we look. In this case
        -- we're fetching the last minute of data, and so (once
        -- we account for the lag in the datasource) will get
        -- rougly 13 rows of raw data.
        date_range_start => dateadd('minute', -1, current_timestamp()),
        -- Replace this with the name of the warehouse you
        -- want to monitor
        warehouse_name => '<TARGET_WAREHOUSE>'
    )
)
-- Parameter #2: --
-- Adjust this queued load number as appropriate for your
-- warehouse. In this example, to be more than 0.5 would mean,
-- on average there was at least one query queued for 50% of
-- the time in that five second period.
where avg_queued_load >= 0.5
-- NOTE: The GROUP BY here is so that we only get one consolidated
-- row in the results. The query only returns results for a single
-- warehouse, but there will be potentially many rows for that warehouse.
group by warehouse_name
-- Parameter #3: --
-- By including a HAVING clause we only get results when this
-- warehouse has more than a certain number of periods of high
-- queueing in the window. Adjusting this can help avoid false
-- positives on transient spikes.
having number_of_periods > 4
```

**Message Template**

```markdown
🚨 Warehouse `{{rows[0].warehouse_name}}` is currently experiencing high queueing. There have been {{rows[0].number_of_periods}} time slices in the last minute where queuing has been unacceptably high. Any Users or Workloads querying this warehouse will be experiencing slower response times and potentially timeouts.

### Next Steps
Urgently review warehouse size and max cluster count to assess whether either should be increased, or whether a workload is generating anomalous query load.
```

---

# Best Practices and Troubleshooting

URL: https://select.dev/docs/reference/using-select/monitors/best-practices-and-troubleshooting

> Learn best practices and troubleshooting for Monitors

## Monitor Design

- **Choose frequency to reduce alert fatigue**: Weekly Monitors often provide enough visibility without overwhelming your team. Reserve high-frequency schedules (daily, hourly, or more) for critical use cases.
- **Use Conditional Monitors or Anomaly Monitors** for anything that runs daily.
   - Daily alerts for cost spikes if they exist are great.
   - Daily FYI will cause you to never read the alert.
- **Use “once per status change” on high-frequency Monitors**: For Monitors that run frequently, this setting prevents duplicate alerts and ensures you only see a notification when something meaningfully changes.
- **Use Jinja filtering on SQL Monitors**: Jinja allows you to parameterize filters and reuse Monitor definitions. For example, you can filter by the current date or pass in environment variables, making your Monitors flexible and easier to maintain.
- **Set effective thresholds**: Thresholds should be tight enough to catch meaningful changes but not so sensitive that they generate noise. Review historical data to understand normal ranges before defining your thresholds.

## Troubleshooting

### Common Issues

- Failed monitor runs: 
   - If no destination is configured, the monitor will fail. This could happen if a Slack channel was deleted, for example.
   - For standard monitor types, failures are rare. Please contact us if a standard monitor is failing with a message other than “No destination is configured”.
   - For Custom SQL Monitors, check the following:
      - Snowflake user access, role, connection, etc.
      - Check the Snowflake query history to try to replicate.
- Permission issues: 
   - Can’t edit a monitor: check your role or ask your Org Admin.
   - Monitor result is empty: verify your role has access to the data produced by the monitor.
   - Monitor did not trigger: This usually means no data was produced by the monitor, and it is configured not send when empty.

### Monitor Status Types

**OK:** The last execution was successful.

**Triggered:** A conditional monitor exceeded the threshold and was triggered.

**Failing:** The last run of a monitor failed.

**Skipped:** The last run of the monitor was skipped because the result set was empty or a threshold was not met.

**New:** The monitor is new and has not been triggered yet.

---

# Jira

URL: https://select.dev/docs/reference/using-select/jira

> SELECT's Jira integration is here. Connect your Jira workspace to SELECT and seamlessly link tickets to track and resolve cost optimization issues — all without leaving your workflow.

## Feature Overview

SELECT's Jira integration is here. Connect your Jira workspace to SELECT and seamlessly link tickets to track and resolve cost optimization issues — all without leaving your workflow.

## Linking a Jira Workspace

Navigate over to `Settings` → `Integrations` . Here is where you can link external applications to SELECT. Click on Jira Cloud to manage the list of active connections.

![](https://cdn.sanity.io/images/wn0ylp2c/production/240fa9df20019c9bef0e883d3b1fd3d5a1dfc112-3024x1652.png)

![](https://cdn.sanity.io/images/wn0ylp2c/production/20da1317392587bb95a7454c9423353f774cb6ff-3024x1652.png)

Clicking on connect will navigate you to Jira where you can select which Jira Site to connect with. Once successful, you will be navigated back to SELECT with a new active connection.

![](https://cdn.sanity.io/images/wn0ylp2c/production/f3aa142c7aeb616f46bdea8c55216d3faa7d4a2b-3024x1650.png)

![](https://cdn.sanity.io/images/wn0ylp2c/production/13fd2140907f09ee30a30d04185b66456bc75bf0-3024x1652.png)

## Creating a ticket

To create a new ticket, navigate to any page from the sidebar (i.e. Budgets, Monitors, Insights, etc). Clicking onto the sidebar view, you will find the “Add a Linked Issue” button.

![](https://cdn.sanity.io/images/wn0ylp2c/production/5f6821a0a0462bf8d3ab31a54582ce5c68928832-3024x1652.png)

The configuration form guides you through how to create a ticket, what fields to provide, etc. By default, the title and description come pre-generated based on the type, assisting you in making the best ticket.

![](https://cdn.sanity.io/images/wn0ylp2c/production/5a7337ac3e0f1b6717f2acfe1e1fd9a253001ace-3024x1650.png)

There is also an option to link pre-existing tickets from Jira. Simply navigate over to Jira and copy any URL of an existing ticket and paste that back in SELECT to seamlessly link the ticket.

![](https://cdn.sanity.io/images/wn0ylp2c/production/7534487e3b6fe99e428d898007575d3657217ccc-3024x1652.png)

## Viewing ticket details

Switching over to the activity feed, here is where ticket information will be displayed. Details such as assignee, status, name change will be shown.

![](https://cdn.sanity.io/images/wn0ylp2c/production/b8e734299251c322d00cf51accdf09d23f38c9cd-2438x1330.png)

## Upcoming Improvements

- Comment syncing
- Auto resolving an insight when a Jira Ticket is closed

---

# Organization Dashboard

URL: https://select.dev/docs/reference/using-select/snowflake-organization

> An annual view of your organization's spend, broken down into components

## Organization Cost Dashboard

To navigate to the Snowflake Organization Cost Dashboard, open the account switcher and click on your Snowflake organization.

![SELECT Snowflake account switcher](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/f316b6c9-9dc1-4919-da3d-2d253173e600/width=1476,quality=75)

In the Snowflake Organization section of SELECT, you'll see a dashboard containing:

1. A snapshot of your annual Snowflake organization spend
2. A breakdown of this spend by account, service, warehouse and [Usage Group](https://select.dev/docs/usage-groups)
3. A variety of filter options to help you further explore and analyze your usage patterns

[Video](https://storage.googleapis.com/prod_select_images/docs/organization_dashboard_overview.mp4)

In the sidebar, you'll see a different set of options than the usual "single" Snowflake Account view in SELECT:

1. The [Organization Cost Explorer](https://select.dev/docs/snowflake-organization#organization-cost-explorer)
2. Organization level [Usage Groups](https://select.dev/docs/usage-groups)
3. The [Snowflake Contract Utilization](https://select.dev/docs/snowflake-organization#snowflake-contract-utilization) section
4. Organization level [Monitors](https://select.dev/docs/monitors)
5. Organization level settings

## Organization Cost Explorer

In conjunction with the main dashboard, you can use the **Cost Explorer** to quickly analyze usage patterns in any Snowflake account that is part of your organization. At a glance, you can see:

1. How much each Snowflake account contribute to your overall spend
2. How each account is trending in the given time period (% Change)
3. Key information about each account, including the region, [Snowflake Edition](https://select.dev/posts/snowflake-pricing#snowflake-editions), and [Snowflake credit pricing](https://select.dev/posts/snowflake-pricing#snowflake-credit-pricing)
4. Which accounts you have [connected to SELECT](https://select.dev/docs/snowflake-organization#connecting-additional-accounts-to-select)

![Snowflake organization spend dashboard by account](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/8f1dc16d-a325-4d02-4447-992efdd3ab00/width=3024,quality=75)

You can also easily understand spend drivers in each account using the "Slice By" menu at the top right of the chart. In this view, we can quickly discover that the majority of the spend in the additional accounts in our Snowflake organization is coming from data transfer costs.

![Snowflake organization spend dashboard by service](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/ed5f6140-8859-43c5-b2c7-efda39705100/width=3024,quality=75)

Leverage the filters at the top to zoom into a subset of accounts or look at specific services across your accounts.

## Snowflake Contract Utilization

To help users manage their Snowflake spend more effectively, we have a “Contract Overview” tab in the Organization dashboard. This page is packed with helpful information to ensure you get the most value out of your credits:

- Understand your active contract capacity and when your contract expires. The tooltip reveals more information about how much capacity was purchased vs rolled over.
- Understand how much capacity you’ve used, and the forecasted total usage at the end of the contract.
- Understand whether you’ll have to renew your contract early and the savings necessary to avoid an early renewal. Alternatively, budget surplus capacity to new workloads that you could bring into Snowflake or other performance optimizations.

![SELECT Snowflake contract capacity monitoring](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/contract_overview_20230802.png/width=2048,quality=75)

The forecasted total usage is based on a linear regression forecast. Users can adjust the number of days used to create the forecast.

[Video](https://storage.googleapis.com/prod_select_images/changelog/contract_overview_20230802.mp4)

> [!WARNING]
> The "Contract Overview" tab in the sidebar is now called "Contract Utilization". This section has moved to the "Contract Utilization" tab within the new [Snowflake Organization section](https://select.dev/docs/snowflake-organization#organization-cost-dashboard) of SELECT.

## Connecting additional accounts to SELECT

### Step 1: Connect the account

To connect additional accounts to SELECT, click the "Connect" button in the table. This will take you to the "Add Account" form, with the information pre-populated based on the account you selected.

Refer to the instructions [here](https://select.dev/docs/connect-snowflake-account) for more details on the "Add Account" form.

### Step 2: Create new Snowflake user

In order to connect additional Snowflake accounts to SELECT, a new user with metadata access must be created by following the steps [here](https://select.dev/docs/snowflake-metadata-access).

Once created, add those user details to the form.

---

# Views

URL: https://select.dev/docs/reference/using-select/views

> Save customized pages in SELECT

## What are Views?

If you find yourself repeatedly applying a certain set of filters, columns, or looking at a specific slice of data in SELECT, you can save this selection as a View for easier reuse and sharing.

![SELECT Views overview](https://cdn.sanity.io/images/wn0ylp2c/production/2d864147368239831d64bf66b74440b5ed835116-1900x870.gif)

Views will remember the following settings for a given page:

- Filters
- Columns
- Names
- Chart settings

Views work any page in SELECT where a filter can be applied or a table can be modified.  This includes:

- Home
- Explorer
- Insights
- Monitors
- Query History
- Warehouses
- Storage

## Create a View

Once you have selected the desired filters, columns and chart settings, click "Save View" in the top right.

In the pop up box:

1. Decide if the view should be Private, Public, or shared to a Team.
2. Give the view a Name.
3. Choose a custom icon and icon color for the view.
4. Hit Save

![How to save a view in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/ef8842f79898bd333a367b8b56ae33485ebb8d4b-1900x870.gif)

## Public vs. Private vs. Team Views

When creating a view, you'll need to choose how accessible the view will be. The view type will determine who else within SELECT will see the view on a given page in the view selector.

1. **Private** views will only be visible to you. They can still be shared by sending them the URL, as long as that person has access to the underlying page/resources. No one else can edit or modify your view.
2. **Public** views are visible to everyone in your SELECT organization. Other users with an editor or higher privilege will be able to modify this view.
3. **Team** views are only visible to other members of your [team](https://select.dev/docs/reference/using-select/access-management/teams). Other users in your team with an editor privilege will be able to modify this view.

## Update a View

To update the filters or the columns in a View, simply make the change while the view is active, then click Update View in the top right.

![How to update filters or column in a view](https://cdn.sanity.io/images/wn0ylp2c/production/d35cca9493ed71b3d7ce3fc1a6000ca2db7aae72-1900x870.gif)

To edit the View name, who the view is shared to, the icon or color, click the view name in the top right, then click Edit.  After making your changes, click Save.

![Edit a view](https://cdn.sanity.io/images/wn0ylp2c/production/1cd3bd96233c3cea96b15fe1b2b1e8645d496816-1900x870.gif)

## Delete a View

To delete a view, click the view name drop down in the top right.  Then click the 3 dots next to the view you want to delete.  Then click "Delete".

![delete a view](https://cdn.sanity.io/images/wn0ylp2c/production/280c2afca7d7520f9861ee5c034fa641b199e6b2-337x298.png)

## Add View to Favorites

Favorites are a list of saved Pages and Views that are pinned on your left sidebar.

To Favorite a View, click the View name in the top right corner, click the 3 dots, and find the "Favorite" option there.

![Favourite view](https://cdn.sanity.io/images/wn0ylp2c/production/0681f02f9b9c69dca31c8a692961c6102307655b-453x401.png)

Now the View will appear under Favorites in your left sidebar, and can be organized into folders by using the +folder icon.  Unfavorite an item by using the "unpin" icon next to the View name in the Favorites menu.

![Remove favourite](https://cdn.sanity.io/images/wn0ylp2c/production/df70f2897d7d4d7f0e09e33a4210e69e1fc69569-241x673.png)

## Listing all Views

Since almost every page in SELECT can become a view, we need one place to list all the views we have access to.  Head over to the Views page in the left side bar.

![Managing views](https://cdn.sanity.io/images/wn0ylp2c/production/851763343d1d13f45ff42c3aa5a34cab8d227806-1663x497.png)

Here you will find a list of every view you have access to, along with plenty of metadata about the view.  The table can be sorted by clicking in the headers.  Also, be sure to note you can click "Add Filter", which can help you sift through your pile of views.

## Share a View

As mentioned earlier, you can make a View public or share it to a team.  But sometimes you may want to keep the view private and have a teammate review it once.  In this case, you can share a view via URL.

To share a View via URL, just copy the URL from the address bar when the View is active. The URL will look something like this: `https://select.dev/app/idfshoz-le02430/workloads/dbt/resources?savedView=7477bf38-49e3-49be-980f-52b099573668`

A user in your SELECT organization can use this link to see the saved View, as long as they have access to the underlying Snowflake account and resources.

---

# Actions & Savings

URL: https://select.dev/docs/reference/using-select/actions-and-savings

> Log cost-saving changes as Actions and let SELECT measure realized savings on your warehouses, including automated before/after spend calculations.

## Actions & Savings

Actions let you record every cost-saving change you make and measure its impact on your bill over time. They are the system of record for optimization work in SELECT — similar to tickets in Linear or Jira, but tied directly to your spend, so you can **prove** impact instead of estimating it.

Actions come from two sources: ones **you log** by hand, and ones **SELECT logs automatically** for its automated-savings features. In most accounts the automated ones are the majority of what you'll see.

## Where to find Actions

Actions live under the **Savings** section of the sidebar.

![The Actions tab on the savings page](https://cdn.sanity.io/images/wn0ylp2c/production/dba8d62e1cc31c660076b8ffa5c32ffaf9227554-1555x783.png)

Savings has three tabs:

- **Overview** — a dashboard showing total realized savings, projected savings, and a leaderboard of who's logging the most actions.
- **Actions** — the full list of logged actions with filtering, grouping, and summary metrics.
- **Insights** — cost-optimization recommendations that you resolve by logging an action.

Actions also appear in a few other places:

- **Insight detail drawers** — each insight shows related actions and a quick **Log Action** button.
- **Spend charts** — top-level charts show actions for all resources as circles on the x-axis; resource-specific charts show just that resource's actions. By default a chart shows only the relevant action types, but you can adjust any chart through the chart menu (a simple show/hide on resource charts, or a multi-select of action types on top-level pages).

![Actions are visible in charts throughout SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/f74584e9a25edf0dd1bf7a997a0d1cd20131ad96-1667x810.png)

Hover over an action marker to open a panel. The panel lists the actions on that date and their combined annualized savings. This value is the per-action annualized savings (see Annualized savings), so the chart's date range does not change it. Click **Add** to log a new action with that resource and date already filled in.

![Take a closer look at an action by clicking it's orb](https://cdn.sanity.io/images/wn0ylp2c/production/03cdefc56ba13ffa7471d8d8ee5fdd3974471bf8-1428x518.png)

## Logging a new Action

Click **Log Action** in the Actions tab header to open the creation modal. You can also click the chart to pin the tooltip and add an action for that day, which pre-fills the date.

![Fill out the 'Log action' modal with the details of what you changed](https://cdn.sanity.io/images/wn0ylp2c/production/afda4a9ad3325e1709bdcbab3aae180ca9415d85-899x410.png)

### Action fields

- **Title** — a short name for what you did (e.g. "Downsize ANALYTICS_WH to Medium").
- **Scope** — whether the action applies to a specific resource or globally (see below).
- **Actioned by** — the team member who took the action.
- **Date of action** — when the change took effect. SELECT uses this as the before/after boundary for measuring impact. Can't be in the future.
- **End date** _(optional)_ — when the action stopped being in effect. Leave blank for an ongoing change; set it to bound a finished experiment (see Start and end dates below).
- **Description** _(optional)_ — any additional context about the change. Supports Markdown.
- **Savings type** — how savings are tracked: Automated or Manual (see below).
- **Savings amount** _(manual only)_ — your savings estimate, entered as either a flat dollars-per-day or a percentage of the resource's spend.
- **Team** _(optional)_ — the team the action belongs to; controls who can see it. Required if you don't have org-level write permission.

### Start and end dates

The **date of action** marks when the change took effect — the start of the action's effective period. The **end date** is optional and marks when it stopped.

Set an end date when you run an experiment and want a clear picture of what the change cost or saved during a fixed window. After an action ends, SELECT stops adding daily savings for it and keeps its realized total. The action's daily and annualized savings become $0. Leave the end date blank for changes that are still in effect.

### Scope: Resource vs Global

**Resource scope** ties the action to a specific resource (warehouse, query pattern, storage, etc.). This is what enables **automated** savings — SELECT compares the resource's spend before and after the action date.

![The resource selector](https://cdn.sanity.io/images/wn0ylp2c/production/e52dbcdbce8dd1414130e5e9885f9485a2d022b7-482x198.png)

**Global scope** is for changes that affect overall spend but aren't tied to a single trackable resource — e.g. "Implemented org-wide query governance," sunsetting an untracked account, or renegotiating your contract. Because there's no single resource to measure against, **global actions require manual savings**: when you pick Global, the savings type switches to Manual and Automated is disabled, and you must enter a flat dollars-per-day amount (percentage savings aren't available for global actions).

### Savings type: Automated vs Manual

![Manual Savings can be auto or a flat $/day](https://cdn.sanity.io/images/wn0ylp2c/production/e2b1c307ceba8b9483c5076b936fcb2aee560910-359x324.png)

- **Automated** — SELECT compares your resource's average daily spend before and after the action date to estimate savings. Requires a resource-scoped action and about a week of post-action data before a result appears (see below). Default for resource-scoped actions.
- **Manual** — you enter the savings yourself, as either a **flat dollars-per-day** or a **percentage of the resource's spend**. Use this to override the automated estimate when you have a better number, or when the resource's usage is too irregular for the before/after comparison to be trustworthy.

> Global-scope actions always use manual flat-dollar savings, since there's no specific resource to measure.

### Who can see an action

Anyone with write permission can log an action against any resource. Visibility depends on team scoping:

- **Scoped to a team** — visible to members of that team, plus organization-wide viewers.
- **Not attributed to a team** — visible only to organization-wide viewers.

SELECT picks a default team for you in the UI, which you can change. Note that SELECT doesn't have a strict notion of teams _owning_ resources, so team scoping is a manual choice rather than something inferred from the resource — pick the team that should see the action.

## Automated savings actions

When **automated savings** is enabled on a Snowflake warehouse or Databricks resource, SELECT automatically logs an action whenever the automation makes a change, so savings from automation are tracked alongside the changes you log by hand. These auto-generated actions behave like any other and appear in the same lists and charts.

**Who they're attributed to.** Historical automated-savings actions were backfilled and are attributed to an automated-savings system user. Going forward, a new automated-savings action is attributed to the actual user who toggled the feature on.

**A different savings methodology.** Because SELECT operates these features, it doesn't rely on the before/after comparison alone. The automated-savings service reports the savings it actually achieved — and that report is often conservative and _under_-counts. So the savings shown for an automated-savings action can be a blend: a floor from what the service reports, combined with the before/after percentage when that's higher.

A practical consequence: an automated-savings action can still show savings even when the before/after comparison would land at **0%** (e.g. because spend fell under the minimum-spend floor described below). The service's own conservative day-to-day estimate carries through instead of being zeroed out.

## Savings Overview dashboard

The **Overview** tab gives a high-level view of your team's optimization impact.

![Savings Overview Tab](https://cdn.sanity.io/images/wn0ylp2c/production/2c6e40b9827f6616affe0ff2c0f9caa13a9dbdf8-1613x885.png)

## Actions tab

The Actions tab shows all logged actions in a filterable table; each action can also be opened in a side pane.

### Summary cards and Daily Savings chart

Three summary cards show at the top of the Actions tab:

- **Total Actions** — the number of actions that match your filters.
- **Savings in Period** — the total savings from the matching actions in the selected date range.
- **Annualized Savings** — Savings in Period, scaled to a full year. See Annualized savings below.

The **Daily Savings** chart shows one bar for each day. Each bar is the total savings from the matching actions on that day. Drag across the chart to zoom to a shorter date range.

The date range and the filters change the cards and the chart. They do not change the savings columns in the actions table.

![The Actions tab with summary cards, the Daily Savings chart, and the leaderboard](https://cdn.sanity.io/images/wn0ylp2c/production/e8d3856021aa335fe81ab2eb8101de188e2b6540-860x379.png)

### Logged Actions Leaderboard

A ranked list of team members by the number of actions they logged and the savings from those actions in the selected date range.

![The leaderboard showcases whose driving the most savings](https://cdn.sanity.io/images/wn0ylp2c/production/3f5dc90304db01ce3c08e7b627cf73901267dbeb-860x378.png)

**Filtering.** Use the inline filters above the table to narrow by title, resource, user, insight type, savings type, or date range.

**Grouping.** Use the **Group by** dropdown to organize actions (by savings type, resource type, team, and more).

### Action details

Click any action row to open the detail drawer on the right. The drawer shows the action's details, a breakdown of the savings calculation, any linked Jira issues (see Linking actions to Jira below), and an activity log of changes.

![The actions drawer](https://cdn.sanity.io/images/wn0ylp2c/production/777000da02d178c9cb0d3185d853fe994bf5a47c-574x856.png)

### Editing & deleting actions

In the detail drawer, click any field to enter edit mode; changes save when you confirm. To delete, open the drawer menu (three dots) in the top-right and select **Delete**. A confirmation dialog appears — deletion can't be undone, and removes the action's savings from your aggregate numbers.

![The delete option](https://cdn.sanity.io/images/wn0ylp2c/production/e0ad50e519cd3e8c4c6ef6ed6a197d1a9bf58599-585x239.png)

## How savings are calculated

### Automated savings

When you log a resource-scoped action with automated savings, SELECT compares the resource's average daily spend **before** the action to the average daily spend **after**. The difference is your daily savings.

- **Before window**: the **7 days before** the action date.
- **After window**: the **7 days after** the action date.
- The difference in average daily spend becomes the daily savings, expressed as a percentage of the before-period spend.

![Before /After analysis breaks down the savings calculation](https://cdn.sanity.io/images/wn0ylp2c/production/72a64b18ca08584827d6b01415e21b570976a5a8-564x367.png)

A few things to know about the method:

- **The action day itself is excluded.** Spend is day-level, so the action day holds partial/mixed before-and-after data that doesn't cleanly belong to either window.
- **It needs about a week of post-action data.** SELECT can't produce a result until the full 7-day after-window has elapsed — roughly 8 days after the action once nightly processing is included.
- **It runs nightly.** The calculation runs once per day, after SELECT pulls the latest spend from your account — so savings populate on the next daily refresh, not instantly.
- **Minimum-spend floor.** SELECT only trusts the comparison when there's at least **$1/day of average spend in both windows**. Below that, the automated rate defaults to **0%** — too small to attribute reliably — and you can eyeball the impact from the before/after chart and switch to a manual figure if warranted.
- **Multiple actions in the window.** If more than one action was recorded for the resource within the comparison window, the breakdown includes a disclaimer, since their effects can't be cleanly separated.

> ⚠️ **When to override.** The before/after comparison assumes usage is roughly stable across the window, so it's sensitive to changes in usage patterns. If the actioned resource has seasonal or irregular usage, the automated number can misattribute normal variance to your change. When the chart looks distorted by usage swings rather than your change, switch to a manual percentage you can defend from the numbers you see.
>
> _Automated rates are also capped — generally 95%, and 50% for idle-time automations like smart suspend (which can only recapture idle time, not active query time). A computed rate above the cap usually signals a workload change rather than the action itself._

### Manual savings

For manual savings you provide either a flat dollars-per-day amount or a percentage of the resource's spend. A flat amount annualizes simply as that amount × 365.

### Annualized savings

SELECT shows annualized savings in two places on the Actions tab. Each place uses a different time window, so the two numbers usually do not match.

#### Annualized Savings card

The Annualized Savings card uses the same method as annualized cost and savings in other parts of SELECT:

**Annualized Savings = (Savings in Period ÷ days in the date range) × 365**

- The date range and the filters change this number.
- SELECT counts only days with synced data. Days after the last sync show as empty on the chart, and SELECT does not include them in the average.
- The **All time** range starts 7 days before your first action.

#### Per-action savings

Each action has its own savings values. SELECT calculates them every night from the action's own data. **The date range and the filters on the Actions tab do not change them.**

- **$ Daily Savings** — the average daily savings of the action over the last 7 complete days of synced data. For a manual flat-amount action, this is the amount you entered.
- **$ Annualized Savings** — $ Daily Savings × 365.
- **$ Realized Savings** — the total savings of the action from its action date to today, or to its end date.

The actions table, the action detail drawer, and the action markers on charts all show these per-action values.

#### Why the two numbers are different

The Annualized Savings card averages the full date range. The per-action values use only the last 7 days. If you add up the $ Annualized Savings column, the total usually does not equal the card.

A longer date range can also show a lower annualized number than a shorter one. For example:

- **Last 60 days:** $3,000 in savings. Annualized: $3,000 ÷ 60 × 365 = **$18,250**.
- **All time (240 days):** $3,300 in savings. Annualized: $3,300 ÷ 240 × 365 = **$5,019**.

The All time range includes early days when fewer actions were active and savings were low. Those days pull the daily average down. To see your current savings rate, select a short, recent date range, or look at the $ Annualized Savings column.

## Logging actions from Insights

Insights are cost-optimization recommendations generated by SELECT. You can log an action directly from an insight to record that you've acted on the recommendation.

From the insight detail drawer, click **Log Action**.

![Log action appears when you hover over the 'Related actions' row](https://cdn.sanity.io/images/wn0ylp2c/production/4dc8e6a2409eb22f080d78930b9464f320c57442-576x1033.png)

The Log Action modal opens with fields pre-filled from the insight:

- The resource is locked to the insight's resource.
- The title is pre-populated with the insight type.

![Some fields will be locked, such as resource related to the insight](https://cdn.sanity.io/images/wn0ylp2c/production/45f7cbabe6ed634f2f842cc3c348aa8dfb79ba47-895x401.png)

After logging, the insight shows its related actions.

## Linking actions to Jira

Because actions mirror issue-tracking workflows, you can link a (resource-scoped) action to a Jira ticket. Open the action's detail drawer and use the **Linked Issues** section, or use **Link Jira issue** in the action's menu. This keeps optimization work tracked in SELECT connected to the engineering work tracked in your issue tracker. 

![Actions can also be linked to issues in Jira](https://cdn.sanity.io/images/wn0ylp2c/production/6288206dca188f4ae0a97ca0742bd237a71d4ddf-575x696.png)

---

# Access Management

URL: https://select.dev/docs/reference/using-select/access-management

> Authenticate through SSO, or grant specific permissions to users

- [User Management](https://select.dev/docs/reference/using-select/access-management/user-management.md): Learn how to invite teammates and manage users in SELECT.
- [Role Based Access Control](https://select.dev/docs/reference/using-select/access-management/role-based-access-control.md): Understand SELECT's permission system and how to configure granular access controls.
- [Teams](https://select.dev/docs/reference/using-select/access-management/teams.md): Learn how to create and manage teams for simplified permissions and collaborative resource ownership.
- [Authentication Management](https://select.dev/docs/reference/using-select/access-management/auth-management.md): Learn how to configure authentication preferences and methods to suit your security needs.
- [Setup Okta SSO](https://select.dev/docs/reference/using-select/access-management/okta-sso.md): Learn how to setup Okta SSO with SELECT
- [Setup Entra ID (Azure AD) SSO](https://select.dev/docs/reference/using-select/access-management/azure-ad-sso.md): Learn how to setup Entra ID (Azure AD) SSO with SELECT
- [Setup SAML Identity Provider SSO](https://select.dev/docs/reference/using-select/access-management/saml-sso.md): Learn how to setup a SAML identity provider SSO connection with SELECT.
- [SCIM](https://select.dev/docs/reference/using-select/access-management/scim.md): SCIM provisioning is available as an add-on. Contact support for configuration instructions.

---

# User Management

URL: https://select.dev/docs/reference/using-select/access-management/user-management

> Learn how to invite teammates and manage users in SELECT.

## User Management

Note that inviting and deleting users can only be performed by users with `Admin` access on the **SELECT Organization**.

**To invite a team member** to SELECT, click Settings (gear icon in lower left) → Users (left sidebar), then click Invite User.  (If you are using SSO, see [below](https://select.dev/docs/reference/using-select/access-management/user-management#single-sign-on-sso-configuration).)

![Invite a user to SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/715c093a063394c28844e8a172189bc2d8ba4612-1882x963.png)

In the Add User side drawer, type the user’s email address then determine their Roles.

![Add User Side bar.  How to add a user in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/070096a9c49892861aa1d5581c7e209c4a08a136-1899x959.png)

Roles consist of Entities and Levels. Entities can be a specific Snowflake account, an entire Snowflake Org, or a SELECT [Usage Group](https://select.dev/docs/reference/using-select/usage-groups). Level is one of: Admin, Editor, or Viewer. See more info [here](https://select.dev/docs/reference/using-select/access-management/role-based-access-control#roles-and-role-entities).

![](https://cdn.sanity.io/images/wn0ylp2c/production/859857178bf0270489473c6cdfc1906d42cd09e1-580x550.gif)

Also on the _Settings -> Users_ you will see a list of current users with access to SELECT and any outstanding invitations.

To remove a user, click the sidepane button next to their name, and then delete the user.

![Delete a user in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/5b61dfb7316ac6968561192a7d6b27c800b6ee59-1903x958.png)

> [!NOTE]
> If you are using one of our SSO integrations like [Okta](https://select.dev/docs/okta-sso) or [Azure AD](https://select.dev/docs/azure-ad-sso), manually inviting users will be disabled.

## Single Sign-On (SSO) Configuration

Rather than manually inviting users to SELECT by email, you can provision access through your company's SSO provider. We support most common providers including:

- [Okta](https://select.dev/docs/okta-sso)
- [Azure AD / Entra ID](https://select.dev/docs/azure-ad-sso)
- PingFed
- Google SSO
- OneLogin
- JumpCloud
- and more...

Once configured, the ability to invite teammates and remove users through the SELECT UI will be disabled.

> [!NOTE]
> SELECT's SSO integrations are an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

---

# Role Based Access Control

URL: https://select.dev/docs/reference/using-select/access-management/role-based-access-control

> Understand SELECT's permission system and how to configure granular access controls.

SELECT uses Role-Based Access Control (RBAC) to manage access to the platform. Permissions are granted by assigning the appropriate role to a user. To simplify the process of managing individual users, users can also be assigned to [teams](https://select.dev/docs/reference/using-select/access-management/teams), so that roles can be granted to a team as a whole rather than to each individual user.

Note that roles in SELECT are always additive.

## Assigning Roles to Users

There are four ways to assign roles to users in SELECT, all of which are configured within settings under the Users tab, with the exception of Team Memberships, which are configured under the Teams tab.

### Direct Assignment

Roles can be assigned directly to individual users. This is useful for specific users who need customized access permissions. Direct assignments can be made when inviting a new user or by updating an existing user's roles in the user settings page.

To edit a user's directly assigned roles, click the side pane button next to their email address.

![](https://cdn.sanity.io/images/wn0ylp2c/production/345c9ffebeb74b20c64c779d58bdd2dbac9a328d-3024x1964.png)

From the side pane, roles can be added, edited, or deleted. An admin role is required on the associated entity to manage roles granted on it.

![](https://cdn.sanity.io/images/wn0ylp2c/production/5b817307ca3b7f1ffa26fdf1bf53a95a5c0abab2-3024x1964.png)

### Default Roles

Default roles provide a baseline level of access for all users in SELECT. These roles are applied universally without requiring manual assignment for each user.

To edit default roles, head to the Users tab, and scroll down to default roles.

![](https://cdn.sanity.io/images/wn0ylp2c/production/52dd6644b0394893677891f9a3076196c7a5dd78-3024x1964.png)

### SSO Group Mapped Roles

Roles can be automatically assigned based on a user's SSO group membership. After configuring your SSO provider to pass the user's group information to SELECT, you can create mappings between SSO groups and SELECT roles in the settings page. This allows for role assignments to be managed through your existing identity provider.

This feature is only enabled for customers using SSO to log in to SELECT.

If enabled, SSO Group Mapped Roles can be configured from the Users tab, after scrolling down the page after the users table.

![](https://cdn.sanity.io/images/wn0ylp2c/production/52240b5e0069b2fc169c7fa219aea2599ad03dc8-3024x1964.png)

### Team Membership

Roles can be assigned to [teams](https://select.dev/docs/reference/using-select/access-management/teams), and these roles will be inherited by all members of that team. This is the recommended approach for managing access for groups of users who need the same permissions. When a user is added to a team, they automatically receive all roles assigned to that team.

The other benefit of this approach is that team members can share resources with each other which are isolated from other teams, such as views, and monitors.

See the [teams](https://select.dev/docs/reference/using-select/access-management/teams) documentation for more information.

## Roles and Role Entities

![SELECT roles and entities](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/adf5d920-7b94-49f3-e2a4-a493adefc400/width=2100,quality=75)

Roles have a **level** and an **entity**. The **level** defines the permissions associated with the role, with three levels available: `Admin`, `Editor`, `Viewer`. The **entity** defines the accounts and organizations within SELECT that the role can access. For example, the `Viewer` role on a Snowflake Account will have read-only access to all objects in the account, but will not have access to other accounts, and will not be able to _edit_ any objects in the account. There are seven entities: **SELECT Organization**, **Snowflake Organization**, **Snowflake Account**, **Databricks Account**, **Databricks Workspace**, **Tableau Site**, and **Usage Group**.

Multiple roles can be granted to a user or team, allowing for a granular approach to access control. Where multiple roles are assigned, the permissions are additive.

Each level always has access to the permissions of the level below it, likewise a role on an entity always applies to all the entities below it. For example, a user with the `Admin` role on the **SELECT Organization** entity will also have `Admin` access to all **Snowflake Organizations,** **Snowflake Accounts**, **Databricks Accounts**, **Databricks Workspaces**, and **Tableau Sites** within it.

## Roles and Permissions

### The SELECT Organization Entity

Roles on the **SELECT Organization** entity provide the ability to perform user management actions such as inviting new users, removing existing users and managing teams. The `Admin` role grants the ability to perform these actions.

The user who initially configures SELECT is granted the `Admin` role on the **SELECT Organization** entity automatically.

#### SELECT Organization Roles and Permissions

| Action | Admin | Editor | Viewer | Team Creator |
| --- | --- | --- | --- | --- |
| Create & Edit API Keys | ✅ | ❌ | ❌ | ❌ |
| Create & Edit Teams | ✅ | ❌ | ❌ | ✅ |
| Invite & Delete Users | ✅ | ❌ | ❌ | ℹ️ |
| Manage SSO Settings | ✅ | ❌ | ❌ | ℹ️ |
| Edit Usage Group Definitions | ✅ | ✅ | ❌ | ❌ |
| View Usage Group Definitions | ✅ | ✅ | ✅ | ❌ |

ℹ️ _Team Creators are granted Team Admin on teams they create, which allows them to invite users to those teams. They can also asign SSO groups to those teams but cannot change Organization wide SSO settings._

> [!NOTE]
> Only SELECT Organization Entity Admins can create and edit API Keys. Admins of other Entities that aren't also assigned to the Admin role on the SELECT Organization Entity will not be able to manage API Keys.

#### Example: Data Platform Team

A central data platform team manages several Snowflake Accounts for different business units. They want to be able to restrict security and team management to only a few select users, but also allow everyone in the central team to be able to view usage data within all of the business unit accounts. In this scenario we would recommend:

- Create a **Team** called _Data Platform Team_. Use an SSO group mapping to automatically add users to the team.
- Grant the **Data Platform Team** the **Editor** role on the **SELECT Organization**. This role will grant them the ability to view usage data across all accounts and create any resources within SELECT such as monitors and usage groups without having to manage individual grants on each account.
- Grant the few specific users who need to manage users and teams the **Admin** role on the **SELECT Organization**. You could do this directly, or again use an SSO group mapping to automatically add users to the team.

### The Snowflake Organization & Account Entities

The **Snowflake Organization** and **Snowflake Account** entities have the same permissions and roles, with the exception of insights which are only generated at the account level.

- The `Admin` role additionally grants the ability to edit the configuration settings for the account or organization.
- The `Editor` role additionally grants edit access to any usage groups, monitors and views within that account or organization. It also grants the ability to enable or disable the [Automated Savings](https://select.dev/features/automated-savings) feature for associated accounts.
- The `Viewer` role grants read access to the usage data within that account or organization.

#### Snowflake Organization & Account Roles and Permissions

| Action | Admin | Editor | Viewer |
| --- | --- | --- | --- |
| Edit Usage Group Definitions | ❌ | ❌ | ❌ |
| View Usage Group Definitions | ❌ | ❌ | ❌ |
| Edit Settings | ✅ | ❌ | ❌ |
| Update User Roles on this Entity | ✅ | ❌ | ❌ |
| Enable/Disable Automated Savings | ✅ | ✅ | ❌ |
| Dismiss Insights | ✅ | ✅ | ❌ |
| Edit Public Saved Views | ✅ | ✅ | ❌ |
| Create & Edit Monitors | ✅ | ✅ | ❌ |
| View Settings | ✅ | ✅ | ✅ |
| View Monitors | ✅ | ✅ | ✅ |
| View User Roles | ✅ | ✅ | ✅ |
| View Dashboards | ✅ | ✅ | ✅ |
| View Public Saved Views | ✅ | ✅ | ✅ |
| View & Edit Personal Saved Views | ✅ | ✅ | ✅ |

In addition to roles on **Snowflake Accounts** and **Snowflake Organizations**, some of the same permissions are also available on **Usage Groups**, which are detailed in the next section.

### Databricks Account & Workspace Roles

The actions available on **Databricks Accounts** and **Databricks Workspaces** are the same as the actions available on **Snowflake Accounts** and **Snowflake Organizations**, which are detailed in the previous section.

### Tableau Site Roles

The actions available on **Tableau Sites** are the same as the actions available on **Snowflake Accounts** and **Snowflake Organizations**, which are detailed in the previous section.

### The Usage Group Entity

> [!NOTE]
> SELECT's Usage Group level roles are an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

The **Usage Group** entity is the lowest level of entity and only grants access to usage data assigned to that usage group. There is only a `Viewer` role on the **Usage Group** entity as there are no associated editable objects.

Where multiple teams operate within the same **Snowflake Account**, the **Usage Group** entity allows for the ability to restrict access for those teams to only the usage data associated with their own Usage Group.

#### Usage Group Roles and Permissions

| Action | Viewer |
| --- | --- |
| View Dashboards | Only usage data for the granted usage group |
| Create Monitors | Only owned by teams the user has Editor resource access on |
| Edit Monitors | Only monitors owned by teams the user has Editor resource access on |
| View Monitors | Only monitors owned by teams the user is a member of |
| Edit Usage Group Definitions | ❌ |
| View Usage Group Definitions | ❌ |

The only role available on **Usage Groups** is the **Viewer** role. This acts as a more restricted version of the **Viewer** role on the parent **Snowflake Account** or **Databricks Workspace**.

For information about creating and managing teams, see [Teams](https://select.dev/docs/reference/using-select/access-management/teams).

---

# Teams

URL: https://select.dev/docs/reference/using-select/access-management/teams

> Learn how to create and manage teams for simplified permissions and collaborative resource ownership.

SELECT uses teams to simplify permission management, enable collaborative ownership of resources, and integrate with SSO group mappings.

## Teams Overview

![SELECT team settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/f8eeedfa-919b-454d-d8fe-9d1518ad2a00/width=3456,quality=75)

**Teams** play three key roles in SELECT:

1. They are an abstraction which streamline permissions management for a group of users. Roles can be granted to a team, and then users added to that team to inherit the roles.
2. They act as owners for editable objects, currently just [Monitors](https://select.dev/docs/monitors).
3. For users with SSO enabled, they are the mechanism by which SSO groups are mapped to SELECT users.

## Team Membership Types

When adding a user to a **team**, there are three kinds of access: `Viewer`, `Editor`,  and `Admin`. `Viewer` membership will grant the user only read access to any editable objects owned by the team. `Editor` membership will grant the user edit access to any editable objects owned by the team. `Admin` membership will grant the user everything Editor does plus managing the teams settings and other team members.

#### Team Admins

A Team Admin administers one team: the team the role was granted on. On that team they can:

- Rename it, and change the role new members receive by default.
- Delete it.
- Add and remove members, and add and remove the SSO groups mapped to it.
- Change the role a member holds, including promoting another member to Admin.
- Grant and revoke roles on the team, within the limits described below.

A Team Admin cannot:

- Do any of the above on a team they do not administer.
- Create a new team. That requires the Team Creator role, or the Admin role on the SELECT Organization.
- Mark a team as the All Users team, or remove that marking. That is a statement about the whole organization, so it requires the Admin role on the SELECT Organization.
- Grant the team a role that gives more access than the Team Admin already holds. In practice this means a Team Admin cannot widen their own team’s access to data. Granting the team an account role, a usage group role, or a role on the SELECT Organization is rejected.

Team membership grants no access to usage data on its own. A team’s access to data comes from the roles granted to the team, which every member inherits.

![SELECT team settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/abeb089d-5f1b-49ab-4e65-bd7a72f12100/width=3456,quality=75)

## Team Ownership of Monitors and Other Editable Objects

By default, newly created [Monitors](https://select.dev/docs/monitors) are owned by the SELECT organization. This means they can only be seen by users with `Viewer` access (or higher) on the SELECT Organization, and they can only be edited by users with `Editor` access (or higher) on SELECT Organization.

To facilitate the creation of monitors by users who may not have full access to the whole SELECT Organization, monitors can also be owned by a **team**. Where monitors are owned by a team, they are subject to the same read permissions as that team. For example, if a team has only `Viewer` access to the _Sales_ Usage Group, then any monitors owned by that team will also only be able to monitor usage data tagged with the _Sales_ Usage Group.

![SELECT team settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/c700553a-8863-482c-8384-2e863a7dc200/width=3456,quality=75)

## Example: Data Platform Team

A central data platform team manages several Snowflake Accounts for different business units. They want to be able to restrict security and team management to only a few select users, but also allow everyone in the central team to be able to view usage data within all of the business unit accounts.

In this scenario we would recommend:

- Create a **Team** called _Data Platform Team_. Use an SSO group mapping to automatically add users to the team.
- Grant the **Data Platform Team** the **Editor** role on the **SELECT Organization**. This role will grant them the ability to view usage data across all accounts and create any resources within SELECT such as monitors and usage groups without having to manage individual grants on each account.
- Grant the few specific users who need to manage users and teams the **Admin** role on the **SELECT Organization**. You could do this directly, or again use an SSO group mapping to automatically add users to the team.

## Example: Embedded Teams Sharing an Account

If multiple teams share a single **Snowflake Account**, we recommend:

1. Create corresponding teams in SELECT and add users directly, or optionally via SSO mappings
2. Create usage groups for each team
3. Grant teams the **Viewer** role on their usage groups

Users will only see their team's usage data and any monitors they create will be scoped to their team, providing clear isolation between teams.

---

# Authentication Management

URL: https://select.dev/docs/reference/using-select/access-management/auth-management

> Learn how to configure authentication preferences and methods to suit your security needs.

Modifying authentication settings is only permitted for users with `Admin` access on the SELECT Organization.

## Authentication Management

Authentication settings allow admins to configure how users sign in to SELECT and control org-wide access policies. They can be accessed by navigating to **Settings** → **Authentication**.

**Email & password** and **Google OAuth** authentication methods may be disabled (and re-enabled), which will prevent users from being able to login using that method. At least one method must remain active at all times to prevent lockout.

**Manual user invitations** via email may also be configured. They will be automatically disabled when SSO connections are first created and re-enabled if all SSO connections are removed.

When **MFA enforcement** is enabled, users who don't have it set up yet will be prompted to do so on their next login.

![Authentication settings page](https://cdn.sanity.io/images/wn0ylp2c/production/e51631a7f707757a65bc450ebd80193331edb685-3024x1963.png)

## Enterprise SSO

> [!NOTE]
> Enterprise SSO is an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

All enabled enterprise SSO connections are listed under the Authentication Methods section along side the default methods.

**To set up a new SSO connection**, click Add SSO and select your provider: 

- [Okta](https://select.dev/docs/reference/using-select/access-management/okta-sso)
- [Microsoft Entra ID (Azure AD)](https://select.dev/docs/reference/using-select/access-management/azure-ad-sso)
- [SAML Identity Provider](https://select.dev/docs/reference/using-select/access-management/saml-sso)

![Add SSO](https://cdn.sanity.io/images/wn0ylp2c/production/89da4e3af4d3c7a819fee3e58ca39d8f4d429615-3024x1963.png)

To **update or remove an SSO connection**, click the edit icon next to it.

![Edit SSO connection](https://cdn.sanity.io/images/wn0ylp2c/production/0cd677ab33fb66c9a8b457b8f130af978f2dc75c-3024x1963.png)

![Delete SSO connection](https://cdn.sanity.io/images/wn0ylp2c/production/e2b4f57fdf72740ee5cffe5a2dbf5398fad25cf9-1512x982.png)

---

# Setup Okta SSO

URL: https://select.dev/docs/reference/using-select/access-management/okta-sso

> Learn how to setup Okta SSO with SELECT

## Connecting Okta

> [!NOTE]
> SELECT's Okta integration is an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

To manage access and authentication to SELECT using your Okta account, please follow the steps below.

### Step 1: Create App Integration

Launch the create app integration flow in the Okta console.

![SELECT Okta integration setup step #1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/okta1.png/width=2640,quality=75)

Select OIDC for the sign-in method, and web application as the application type.

![SELECT Okta integration setup step #2](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/okta2.png/width=1980,quality=75)

On the next page, set the app name to SELECT. Check all the grant type boxes so that they match the screenshot below. Set the sign-in redirect URI to `https://auth.select.dev/login/callback`.

![SELECT Okta integration setup step #3](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/ec724c42-a209-48e5-89b9-61e5b5984900/width=1280,quality=75)

On the next page, scroll to ‘General Settings’ and click ‘Edit’. Under ‘Login’, set the ‘Login initiated by’ dropdown to ‘Either Okta or app’. Under ‘Application visibility’, check the box next to ‘Display application icon to users’. You can find your `ORGANIZATION_ID` value in [settings](https://select.dev/app/settings).

![Find your organization ID SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4e7878f4fc488065620d278b6e1fdfd5a07aef64-2214x1304.png)

Set the Initiate Login URI value to `https://select.dev/api/auth/login?organization=ORGANIZATION_ID`, replacing `ORGANIZATION_ID` with the value you have.

![SELECT Okta integration setup step #4](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/17ebb209-96d2-4468-b1a3-d557a8e2e500/width=1506,quality=75)

Set Federation Broker Mode to Disabled to ensure that the app tile shows up for users. Add the required assignments under the ‘Assignments’ tab at the top of the page.

![SELECT Okta integration setup step #5](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/okta5.png/width=1516,quality=75)

### Step 2: Add SSO Connection in SELECT

Once complete, head over to SELECT and navigate to **Settings** → **Authentication**. In the Custom Enterprise SSO section, click Add SSO and select Okta from the dropdown.

![Add Okta SSO Flow](https://cdn.sanity.io/images/wn0ylp2c/production/a8f356b07aeb0a88b31b8c01d8c43446c23d2604-3024x1963.png)

Next, fill out the Add Okta SSO form. Details about where to find the domain, client ID, and client secret on the Okta console are listed below:

- Client ID and secret for the newly created application integration

![SELECT Okta integration setup step #6](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/c342d15f-1f55-4534-03cc-7208308a0700/width=1534,quality=75)

- The Okta domain. This is usually the domain found by clicking your email address in the top right, as shown in the below screenshot. If you've configured the app to use a custom domain, however, please specify that instead.

![SELECT Okta integration setup step #7](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/okta7.png/width=592,quality=75)

Click Add SSO to complete. You should then see your newly created SSO connection under Authentication Methods.

![Add Okta SSO form](https://cdn.sanity.io/images/wn0ylp2c/production/791d57b061c46d8f3a94398215e7a981b61b000f-3024x1963.png)

![Created Okta SSO connection](https://cdn.sanity.io/images/wn0ylp2c/production/ef51dfdf2ba23adfaa073f9972d049350f442686-3024x1963.png)

> [!NOTE]
> Creating an SSO connection will automatically disable manual user invitations. 

## Mapping SSO Groups to Roles in SELECT

Through the User Roles & Permissions in SELECT, users can enable fine-grained access control in SELECT.

Rather than manually [assigning each user a role](https://select.dev/docs/reference/using-select/access-management/role-based-access-control#direct-assignment) in SELECT, you can configure roles to be automatically assigned based on the user's Okta SSO group.

To do this, the SELECT integration you set up in Okta needs to be configured to pass your employee's Okta groups to SELECT. Follow the steps below to add the necessary group claims.

1. In Okta, click **Applications** in the sidebar
2. Click the SELECT app

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/sso-group-mappings-2-5f3cb51d7bcaedbe1b261625d064aad40ef92322a802907a0535b827e85b29ca/width=2626,quality=75)

3\.  Click the **Sign On** tab

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/sso-group-mappings-4-a94c684018cc38af3d0c3fe51eebdbeb9314072a894b2f309fadd52554b119e1/width=2624,quality=75)

4\. Scroll down to **OpenID Connect ID Token** section and click **Edit**.

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/sso-group-mappings-0-d8ecb062c24e703a588e24f7a24e7d1ccb20363c27ad6cd9af433fb419d57952/width=2626,quality=75)

5\. Set the **Group claims type** to `Filter`

6\. Set **Groups claim filter** to `groups` (which is pre-filled) and for pattern pick matches regex and enter `.*` if you want to send along all the groups. If you want to restrict the groups that are made available to select choose another option. You can learn more about the [Groups Claim here](https://developer.okta.com/docs/guides/customize-tokens-groups-claim/main/).

7\. Click Save

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/sso-group-mappings-1-b7c9ccfbf4dc872e550b688a753197b22207af4ab3efb373d6da8a7a71cbf32c/width=2624,quality=75)

8\. Log in / out of SELECT, and validate that the SSO groups are being exposed to SELECT by checking the user's roles

![SELECT User Sidebar Roles](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/230dc412-57fc-4fd5-91b1-1357ec3ddc00/width=1008,quality=75)

9\. Head back to our [Roles documentation](https://select.dev/docs/reference/using-select/access-management/role-based-access-control#sso-group-mapped-roles) and add the group mappings in SELECT.

---

# Setup Entra ID (Azure AD) SSO

URL: https://select.dev/docs/reference/using-select/access-management/azure-ad-sso

> Learn how to setup Entra ID (Azure AD) SSO with SELECT

> [!NOTE]
> SELECT's Azure AD integration is an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

## Connecting Entra ID

To manage access and authentication to SELECT using your Microsoft Entra ID (formerly Azure AD) account, please follow the steps below.

### Step 1: Register Application

Launch the new app registration flow in the Microsoft Entra admin center.

![SELECT Entra ID integration setup step #1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/3bf0527f-47b5-492d-cc61-c736edb58500/width=3456,quality=75)

Set the name to SELECT, and restrict the access to accounts in your organizational directory only. Set the Redirect URI type to 'Web' and the URI value to `https://auth.select.dev/login/callback`, and click 'Register'.

![SELECT Entra ID integration setup step #3](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/b1512e1c-fb01-4de7-3ae0-0d1b96120300/width=3456,quality=75)

On the app's registration overview page, make a note of the 'Application (client) ID' value for configuration in SELECT.

![SELECT Entra ID integration setup step #3](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/d004971d-1219-4a98-749d-752f1805bc00/width=3456,quality=75)

Next, click the 'Add a certificate or secret' link from the same overview page.

![SELECT Entra ID integration setup step #4](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/fca00801-9728-4482-8215-1abd4a816b00/width=3456,quality=75)

Then, create a 'New client secret'.

![SELECT Entra ID integration setup step #5](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/6acb22a4-5a9b-46eb-d25c-f05ddbd4e000/width=3456,quality=75)

Set the expiration to the maximum allowed duration. Click 'Add'.

![SELECT Entra ID integration setup step #6](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/d5802e4d-c7f2-48e3-6914-b6573a7df000/width=3456,quality=75)

Make a note of the secret value for configuration in SELECT. We recommend creating a reminder to update the connection on SELECT with a new secret before it expires.

![SELECT Entra ID integration setup step #7](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/08bbc3ea-2b1d-4ca5-1002-eceacf413900/width=3456,quality=75)

Next, click the 'Branding & properties' tab. You can find your `ORGANIZATION_ID` value in [settings](https://select.dev/app/settings). Then set the Home page URL to `https://select.dev/api/auth/login?organization=ORGANIZATION_ID`, replacing `ORGANIZATION_ID` with the value you have. Then click save.

![Find your organization ID SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4e7878f4fc488065620d278b6e1fdfd5a07aef64-2214x1304.png)

![SELECT Entra ID integration setup step #8](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/3ccd8eeb-247a-4ff1-673e-5977db5d4c00/width=3456,quality=75)

SELECT needs to be able to retrieve the group memberships of users for [SSO Group role mappings](https://select.dev/docs/reference/using-select/access-management/role-based-access-control#sso-group-mapped-roles). To enable this, head to the 'API permissions' tab. SELECT requires the `User.Read` and `Directory.Read.All` permissions.

![SELECT Azure AD Permissions](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/2d5cae31-239c-4e5c-2890-b1ac1ebd5600/width=3456,quality=75)

Finally, click the 'Enterprise applications' section of the sidebar, and then click the 'SELECT' application you created. Under properties, ensure the app is set to be visible to users. Access to SELECT is governed by the settings under the 'Users and groups' tab.

![SELECT Entra ID integration setup step #9](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/7a11d815-1af3-452d-1907-c1a4709bcc00/width=3456,quality=75)

### Step 2: Add SSO Connection in SELECT

Head over to SELECT and navigate to **Settings** → **Authentication**. In the Custom Enterprise SSO section, click Add SSO and select Microsoft Entra ID from the dropdown.

![Add Azure AD (Entra ID) SSO flow](https://cdn.sanity.io/images/wn0ylp2c/production/61720239d88288c3b3782b62f33a17d34c5c35a3-3024x1963.png)

Next, fill out the Add Microsoft Entra ID SSO form. Enter the client ID and client secret that we made note of earlier. You may also enable Include SSO Groups for [SSO group role mappings](https://select.dev/docs/reference/using-select/access-management/role-based-access-control#sso-group-mapped-roles). To find your Entra ID Domain, [see your directory page](https://entra.microsoft.com/#settings/directory):

![SELECT Entra ID integration setup step #10](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/46c0e71f-7270-4bdf-e478-bfc4dcb62000/width=3456,quality=75)

Click Add SSO to complete. You should then see your newly created SSO connection under Authentication Methods.

![Add Microsoft Entra ID SSO form](https://cdn.sanity.io/images/wn0ylp2c/production/71a3c19c3c500ac023ef580b1e80fe86be9d2eb6-3024x1963.png)

![Newly added Azure AD (Entra ID) SSO connection](https://cdn.sanity.io/images/wn0ylp2c/production/b8a64ae6f1037e0cd2f560ee753ab6d0439f251d-3024x1963.png)

> [!NOTE]
> Creating an SSO connection will automatically disable manual user invitations. 

---

# Setup SAML Identity Provider SSO

URL: https://select.dev/docs/reference/using-select/access-management/saml-sso

> Learn how to setup a SAML identity provider SSO connection with SELECT.

## Connecting SAML Identity Provider

> [!NOTE]
> SELECT's SAML integration is an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

To manage access and authentication to SELECT using a SAML identity provider (IdP), please follow the steps below.

### Step 1: Register SELECT with IdP

Navigate to **Settings** → **Authentication**. Click Add SSO and then select SAML in the dropdown.

![Add SAML IdP flow](https://cdn.sanity.io/images/wn0ylp2c/production/b3bd8cce8893339765e07bc35ac7c132e011fe24-3024x1963.png)

The SAML Connection Details section will display the values ACS URL and entity ID. Please enter both at your IdP. The specific methods for doing so will vary, so please see your IdP's documentation if you need additional assistance.

![Add SAML connection details](https://cdn.sanity.io/images/wn0ylp2c/production/7e75e21a6f145a940677523a6010be43bc841752-3024x1964.png)

### Step 2: Add SSO Connection in SELECT

Once your IdP is configured, complete the form using the following values obtained from the SAML IdP:

- Sign in URL
- X.509 signing certificate (in either PEM or CER format). You will need to convert the contents of the file to Base64 before entering it, which can be done using an [online tool](https://www.base64encode.org/) or with the following Bash command: `cat signing-cert.crt | base64`

![Add SAML connection form filled](https://cdn.sanity.io/images/wn0ylp2c/production/d6073cc758b491655071ce8552be348c08e8d56b-3024x1964.png)

Click Add SSO to complete. You should then see your newly created SSO connection under Authentication Methods.

![Added SAML IdP connection](https://cdn.sanity.io/images/wn0ylp2c/production/8fc262487eeb5fda25bf5929a23506c46cb54350-3024x1964.png)

> [!NOTE]
> Creating an SSO connection will automatically disable manual user invitations. 

### Step 3: Enable login from your Identity Provider portal

In the **SAML Connection Details** section, copy the **IdP-Initiated Login URL** and add it to your identity provider as a bookmark or linked app - for example, a Bookmark App in Okta or a linked app in Entra ID. Do not add it as a SAML application.

![](https://cdn.sanity.io/images/wn0ylp2c/production/0a3b521597d71fcaac2598cb69973f3226f1f6bc-3024x1964.png)

Note that SELECT rejects identity-provider-initiated assertions as a security best practice, as the app cannot verify logins it did not initiate, and accepting unsolicited sign-ins would expose users to login CSRF (an attacker forcing a user into a session). Using the provided login URL ensures a SELECT-initiated login flow that is verified.

---

# SCIM

URL: https://select.dev/docs/reference/using-select/access-management/scim

> SCIM provisioning is available as an add-on. Contact support for configuration instructions.

## SCIM

SCIM (System for Cross-domain Identity Management) provisioning is available as an add-on feature in SELECT. SCIM enables automated user provisioning and deprovisioning through your identity provider, keeping your SELECT user base in sync with your organization's directory.

Please contact support for configuration instructions.

---

# Copilot

URL: https://select.dev/docs/reference/using-select/copilot

> Learn how to use SELECT Copilot, our AI assistant for cost and performance analysis.

- [Overview](https://select.dev/docs/reference/using-select/copilot/copilot-overview.md): Learn how to use SELECT Copilot, our AI assistant for cost and performance analysis across your data platform.
- [MCP Server](https://select.dev/docs/reference/using-select/copilot/mcp-server.md): Connect Select Copilot to Claude Code, Cursor, Codex, and other AI tools using the Model Context Protocol.

---

# Overview

URL: https://select.dev/docs/reference/using-select/copilot/copilot-overview

> Learn how to use SELECT Copilot, our AI assistant for cost and performance analysis across your data platform.

SELECT Copilot is an AI assistant that helps you understand and optimize your usage across your data platform. It can investigate cost spikes, analyze individual queries, and recommend optimizations using the same data you see in SELECT.

[Video: SELECT Copilot quick demo](https://player.mux.com/23ROyU4Auanm3Vuf3UtX102xNBcIGHKmaU02vuTR6FJzM)

Here are some of the key features

- Investigate cost spikes and spend changes granularly
- Analyze any query end-to-end — what it does, why it's slow, what it costs, and how to fix it.
- Talk to Copilot from inside SELECT, from Slack, or from any MCP-compatible AI client.
- Search SELECT's documentation directly through Copilot when you need a how-to.

![SELECT Copilot in action](https://cdn.sanity.io/images/wn0ylp2c/production/c77416d398f07762ab0bde013ddba625763ef3c5-2926x2020.png)

## Where to find Copilot

Inside the SELECT app, you can launch Copilot in two ways:

- **Ask Copilot bubble** — the floating chat bubble in the corner of every page. Open it from anywhere to start a new conversation.
- **Analyze with Copilot** — click any chart in SELECT to send its data and context straight to Copilot. This works on time-series, breakdowns, and stacked or multi-series charts.

![Analyze with Copilot tool tip](https://cdn.sanity.io/images/wn0ylp2c/production/c626f6caa0873358b5b6bdc5884f44715b24a78f-1512x982.png)

## Cost and performance investigations

Ask Copilot questions about your usage. For example:

- "Why did our spend spike yesterday?"
- "Which workload drove the cost increase this week?"
- "What changed in our costs month-over-month?"
- "Show me the most expensive queries on this warehouse over the last 24 hours."

## Query analysis

Paste a query ID, or ask Copilot to find one for you, and it'll tell you:

- **What the query does.** A plain-English summary so you can quickly understand unfamiliar SQL without reading hundreds of lines.
- **Why it's slow.** Performance bottlenecks like spillage, exploding joins, or full column scans, with an explanation of what's happening.
- **What it costs.** Per-execution cost plus annualized cost projections at hourly, daily, and weekly run rates.
- **How to fix it.** Actionable recommendations from SELECT's query-level insights, sorted by effort so you know where to start.

## Copilot in Slack

Tag `@SELECT` in any Slack channel with a question to start a chat. Copilot can read the thread it's tagged in, but not the rest of the channel. You can pick up a Slack conversation in the SELECT app at any time.

![SELECT Copilot in Slack](https://cdn.sanity.io/images/wn0ylp2c/production/6ac319fcfb449b2872053f2039b37b7a49b49851-887x943.png)

To enable Copilot in Slack, install (or re-install) the SELECT Slack app from [select.dev/app/settings/slack](https://select.dev/app/settings/slack).

![Re-installing the SELECT Slack app](https://cdn.sanity.io/images/wn0ylp2c/production/26e5e954b68a15e9b24e2e8ef93b8894b3b91707-1201x749.png)

## Copilot via MCP

You can also talk to Copilot from any MCP-compatible AI client — Claude Code, Cursor, Codex, and more. See the [MCP Server documentation](https://select.dev/docs/reference/using-select/copilot/mcp-server) for setup instructions.

## Permissions and data access

- You'll only see data you have permission to view in SELECT.
- Copilot can read your data, but can't change anything in SELECT or your data platform.
- In Slack, Copilot can only see threads it has been tagged in, not the full channel history.

## Tips for getting good answers

- **Start broad, then narrow.** Ask "why did spend go up last week?" before drilling into a specific warehouse or workload — Copilot will guide the investigation.
- **Use Analyze with Copilot.** Clicking a chart sends rich context to Copilot — the time range, dimensions, and filters — so you don't have to describe what you're looking at.
- **Paste query IDs directly.** If you already have a query ID from an alert, monitor, or your own investigation, paste it in and Copilot will analyze it immediately.
- **Continue across surfaces.** A conversation started in Slack can be picked up in the SELECT app, and vice versa.

---

# MCP Server

URL: https://select.dev/docs/reference/using-select/copilot/mcp-server

> Connect Select Copilot to Claude Code, Cursor, Codex, and other AI tools using the Model Context Protocol.

## Feature overview

Connect [Select Copilot](https://select.dev/docs/reference/using-select/copilot/copilot-overview) to your favourite AI tools — Claude Code, Cursor, Codex, and more — using the Model Context Protocol (MCP). Ask the same questions you'd ask Copilot in the SELECT app or in Slack, but right from your terminal, editor, or chat client.

## Install

### CoCo CLI

Add this to `~/.snowflake/cortex/mcp.json` (create the file if it doesn't exist):

```json
{
  "mcpServers": {
    "select-mcp": {
      "type": "http",
      "url": "https://api.select.dev/mcp/v1",
      "oauth": {
        "client_id": "NGqfslunrY4oHlVx392FvuFGLVuiG1ky",
        "client_name": "Cortex Code",
        "redirect_port": 3333
      }
    }
  }
}
```

### Claude Code

Run this in your terminal:

```bash
claude mcp add --transport http \
  --client-id NGqfslunrY4oHlVx392FvuFGLVuiG1ky \
  --callback-port 3333 \
  select-mcp https://api.select.dev/mcp/v1
```

### Cursor

Add this to `~/.cursor/mcp.json` (create the file if it doesn't exist):

```json
{
  "mcpServers": {
    "select-mcp": {
      "url": "https://api.select.dev/mcp/v1",
      "auth": {
        "CLIENT_ID": "NGqfslunrY4oHlVx392FvuFGLVuiG1ky",
        "scopes": ["openid", "profile", "email", "mcp:tools"]
      }
    }
  }
}
```

### Codex CLI

Codex doesn't yet support fixed-client OAuth for remote HTTP MCP servers, so the connection is wrapped through `mcp-remote` (requires Node.js). Add this to `~/.codex/config.toml`:

```toml
[mcp_servers.select-mcp]
command = "npx"
args = [
  "-y",
  "mcp-remote@0.1.37",
  "https://api.select.dev/mcp/v1",
  "3333",
  "--host",
  "localhost",
  "--static-oauth-client-info",
  "{\"client_id\":\"NGqfslunrY4oHlVx392FvuFGLVuiG1ky\",\"token_endpoint_auth_method\":\"none\"}",
  "--static-oauth-client-metadata",
  "{\"scope\":\"mcp:tools\"}",
  "--resource",
  "https://api.select.dev/mcp/v1",
  "--transport",
  "http-only"
]
```

### VS Code

Add this to `.vscode/mcp.json` in your workspace (create the file if it doesn't exist):

```json
{
  "servers": {
    "select-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote@0.1.37",
        "https://api.select.dev/mcp/v1",
        "3333",
        "--host",
        "localhost",
        "--static-oauth-client-info",
        "{\"client_id\":\"NGqfslunrY4oHlVx392FvuFGLVuiG1ky\",\"token_endpoint_auth_method\":\"none\"}",
        "--static-oauth-client-metadata",
        "{\"scope\":\"mcp:tools\"}",
        "--resource",
        "https://api.select.dev/mcp/v1",
        "--transport",
        "http-only"
      ]
    }
  }
}
```

### CoCo Desktop

Add this to `~/.snowflake/cortex/mcp.json` (create the file if it doesn't exist):

```json
{
  "mcpServers": {
    "select-mcp": {
      "type": "http",
      "url": "https://api.select.dev/mcp/v1",
      "oauth": {
        "client_id": "NGqfslunrY4oHlVx392FvuFGLVuiG1ky",
        "client_name": "Cortex Code",
        "redirect_port": 33148
      }
    }
  }
}
```

After adding the entry to your mcp.json file, open CoCo Desktop and go to Settings > MCP Settings > find the select-mcp entry. Ensure it is turned on, and click connect. A browser window will open for you to sign into SELECT. After that you're good to go.

### Other clients

Any MCP client that supports remote HTTP servers can connect. Use these settings:

- **URL:** `https://api.select.dev/mcp/v1`
- **Client ID:** `NGqfslunrY4oHlVx392FvuFGLVuiG1ky`
- **Callback port:** `3333`

## Signing in

The first time you use Copilot from your client, a browser window will open for you to sign in to SELECT. After that you're good to go.

---

# Custom Dashboards

URL: https://select.dev/docs/reference/using-select/dashboards

> Custom dashboards let you build your own views across any combination of resources, metrics, and time ranges

## Custom Dashboards

Custom dashboards let you build your own views across any combination of resources, metrics, and time ranges. Use them to monitor spend, track key signals for your team, and share progress with stakeholders.

![populated dashboard with several block types](https://cdn.sanity.io/images/wn0ylp2c/production/2a5b9abb70f58702ffa6dc52f2c99880e50fa2b2-2850x1722.png)

## Where to find dashboards

Open **Dashboards** from the left navigation. This lists every dashboard you can see, grouped by scope:

- **Personal** — dashboards only you can see.
- **Team** — dashboards shared with a team you belong to.
- **Public** — dashboards shared across your entire organization.

Starred dashboards appear at the top for quick access. Use the search field to filter by name.

## Creating a dashboard

Click **New dashboard** in the top right. You'll see three ways to start:

### Starting from a template

Ready-made templates cover common use cases. Pick one and a new dashboard will be created with the template's blocks and layout, ready to customize.

Dashboards your organization has already created also appear as templates, so teams can build on each other's work.  


![template picker modal with preset tiles visible](https://cdn.sanity.io/images/wn0ylp2c/production/2c670942d22ca755eee0a97fa8e006f432faae8b-2862x1714.png)

### Starting blank

Choose **Continue from Scratch** to start with an empty grid.

![empty new dashboard](https://cdn.sanity.io/images/wn0ylp2c/production/4a40748ba149ed96f21cce5086250248cc9461af-2868x1710.png)

### Duplicating an existing dashboard

Any dashboard you can see can be used as a starting point. From the dashboards list, you can duplicate any dashboard via the options menu at the right.

New dashboards start private to you. When you're ready to hand one off, change the scope from the **Share** button

## Block types

Every dashboard is built from blocks. There are two types: **data blocks** (backed by your Snowflake, BigQuery, or Databricks data) and **text blocks** (markdown for headings, notes, and context).

Click **Add block** in the dashboard editor to choose between them.

![add data block or text block](https://cdn.sanity.io/images/wn0ylp2c/production/d59e9e4f1e384f4f4c72b8e229015fc4e04d5b5c-744x678.png)

### Chart

Line, bar, stacked bar, percent bar, cumulative line, and forecast charts. Group by any dimension — warehouse, workload, user, cost type, and more. Optionally overlay logged savings actions to correlate cost changes with team activity.

![line chart showing bob's spend over time](https://cdn.sanity.io/images/wn0ylp2c/production/239ca003c98c7f49419bc9f99d60f7bb4baf1ccf-2864x1698.png)

### Big Number

A single KPI, prominently displayed. Optionally show a previous-period comparison as an absolute value, a percent delta, or both.

![total spend in period big number](https://cdn.sanity.io/images/wn0ylp2c/production/54d80432b0fc2bafe3845fa180d78d27e296eca4-2858x1710.png)

### Bar List

A top-N horizontal ranking. Pick a metric and a group-by dimension; set the row limit. Ideal for "top 10 warehouses by spend" style views.

![top ten warehouse spend](https://cdn.sanity.io/images/wn0ylp2c/production/0d161c334b800211f53086d8f210352b7a522e52-2858x1700.png)

### Table

Rows of data with sortable, rearrangeable columns. Configure which columns appear, the default sort, and the row limit (up to 1,000).

![Insights table](https://cdn.sanity.io/images/wn0ylp2c/production/df07f931149ce6e9099b14904afa6e80a7e7c18f-2858x1708.png)

### Text

Rich markdown for headings, section dividers, narrative, and links. Use text blocks to explain what a dashboard is for, call out important context, or link to related resources.

![Text Block](https://cdn.sanity.io/images/wn0ylp2c/production/39d1e55907cc3a8ae6c73c713ecfe96c3ef0ffae-2854x1726.png)

### Arranging the dashboard

Use resize handles and drag and drop to layout out the dashboard as you like

![rearranging the dashboard](https://cdn.sanity.io/images/wn0ylp2c/production/8e1e5b0b20e015112a7b721c787805c9aaec2f23-800x479.gif)

## Configuring blocks using Build with Copilot

Every data block can be configured by hand — pick a data source, a visualization type, group-by, filters — or you can describe what you want in plain language to Copilot and and let it configure the block for you.

Open the block editor and expand the **Copilot** sidebar. Ask questions like:

- _"Show me daily Snowflake spend by warehouse for the last 30 days."_
- _"What are my most expensive queries this month?"_
- _"Compare this quarter's compute cost to last quarter, by workload."_

Copilot picks the data source, visualization, group by, and filters, then generates a live preview in the block editor. Every field it sets remains editable — treat its output as a starting point, refine as needed, and iterate by asking follow-up questions in the same conversation.

![copilot adds a warehouse block](https://cdn.sanity.io/images/wn0ylp2c/production/6575575858edcc98a48c8fa7576a2f131d95c416-2986x1694.png)

## Combined filters

Dashboards support filters at two levels: **dashboard-level** filters that apply to every applicable block, and **block-level** filters that narrow a single block's data.

### Dashboard-level filters

The filter bar at the top of the dashboard applies to every block. Use it to scope the entire view to a date range, a warehouse, a workload, or any other dimension. Dashboard filters combine with each block's own scope — the block will see the intersection.

![a dashboard filtered to snowflake account and user bob](https://cdn.sanity.io/images/wn0ylp2c/production/b89abb27709cf417bc71738d829a56a54e10f8fe-2840x1696.png)

Note here that filter `User is Bob` does not apply to the Insights table. The warning icon in the filter pill indicates that the filter does not apply to all data sources.

### Block-level filters

Inside the block editor, each block has its own **Filters** panel. Use these to pin a single block to a specific slice — for example, keep a block scoped to "prod warehouses only" even as the surrounding dashboard filter changes.

![Workload Spend on warehouse in addition to dashboard-level filters](https://cdn.sanity.io/images/wn0ylp2c/production/9c2bc05e356e5b24f9cbd29127554ef2a786bfd4-2842x1712.png)

### Date ranges and time grain

Filter by a relative window (**Last 7 days**, **Last 30 days**, **Month to date**, **Year to date**) or an absolute start/end date. Set the time grain to **day**, **week**, or **month** to control how data is bucketed on charts and tables.

![date range filter overrides the chart default configuration of last 30 days](https://cdn.sanity.io/images/wn0ylp2c/production/49b7dd84bf9b08f5c429f78eb2514e61ce0a6304-2836x1718.png)

## Sharing and visibility

Every dashboard has one of three scopes:

- **Personal** — only you can see it.
- **Team** — everyone in a team you choose can see it; other Select users cannot.
- **Public** — everyone in your organization can see it.

Click **Share** in the top right to change the scope.

Who can edit a shared dashboard depends on the viewer's role in Select — for example, Org Viewers can see Public dashboards but can't edit them, while Org Admins can edit anything shared to the organization. Team dashboards require team membership.

> ⚠️ Changing a dashboard's visibility away from **Public** will unset it as your organization's home dashboard if it was configured as such.

![sharing dropdown modal](https://cdn.sanity.io/images/wn0ylp2c/production/91981c15eeef6e9ba5dad5f09ab4acd894692ea2-2840x1716.png)

## Setting a dashboard as your homepage

You can make any dashboard the first thing you see when you open Select.

### Personal home

From the dashboard's options menu, choose **Set as personal home**. That dashboard now loads whenever you sign in or click the Select logo or click Home on the sidebar. Only you are affected.

![set as home options](https://cdn.sanity.io/images/wn0ylp2c/production/9047461e9a57a5000357619048829b32058f1d17-2872x1710.png)

![add dashboard as home from dashboard detail page](https://cdn.sanity.io/images/wn0ylp2c/production/e992dab21204279615e8312eecc12299f1454d6c-2842x1714.png)

### Organization home

Org Admins can set a default landing dashboard for everyone in the organization. From the dashboard's options menu, choose **Set as organization home**. That dashboard now loads whenever you sign in or click the Select logo or click Home on the sidebar. Everyone in your organization will see this as their home unless they have a personal home.

If a user has set a personal home, their personal choice takes precedence over the organization home.

## Scheduled send

Scheduled sends/digests deliver a snapshot of a dashboard to email, Slack, or Teams on a schedule. Use them for reporting such as weekly cost reviews, monthly stakeholder updates.

Open a dashboard and click the paper airplane icon in the top right to manage scheduled sends.

![scheduled digest sends](https://cdn.sanity.io/images/wn0ylp2c/production/633359c8defa37b02561ced8c1e643175f46e8b5-3020x1734.png)

### Creating a scheduled send

From the paper airplane icon in the top right, click **New scheduled send** and configure:

- **Cadence** — Daily, Weekly (pick day of week), or Monthly (pick day of month; months without that day fall back to the last day).
- **Destinations** — one or more email addresses, Slack channels, or Microsoft Teams. Destinations are managed in the same place as your monitor alert destinations.
- **Format** — **PDF**, **PNG**, or a **URL link** back to the live dashboard.
- **Filters** _(optional)_ — apply dashboard filters that will be baked into every send. Useful for team-specific digests that scope the same dashboard to different slices.
- **Custom message** _(optional)_ — a note prepended to the digest body.

![send dashboard options](https://cdn.sanity.io/images/wn0ylp2c/production/03f9f0fd1b5249f79eeb8dafd1d8794700158913-2850x1714.png)

![scheduled send config options](https://cdn.sanity.io/images/wn0ylp2c/production/2d9645001ebc26303bcd8ba4a5166cb595794715-2946x1756.png)

You can duplicate an existing digest to create a variant (e.g., a weekly PDF for leadership, a daily URL link for the team).

### Send now

Need to fire off a one-time send? Use **Send dashboard** — same picker, no schedule. The dashboard is captured with the destinations, format, and filters you choose, and delivered immediately.

### Run history

Each digest shows a history of its recent sends, including status and a link to the delivered artifact. Failed runs surface the error so you can retry or fix the destination.

## Exporting

Export a dashboard for use outside of Select — dropping into a slide deck, attaching to a ticket, or archiving.

From the dashboard's options menu, choose **Export** and pick a format:

- **PDF** — best for reports and email.
- **PNG** — a single high-resolution image.
- **JPEG** — smaller image file, lossy compression.
- **Print** — opens your browser's print dialog for direct printing or "save as PDF."

![export options dropdown](https://cdn.sanity.io/images/wn0ylp2c/production/d9d2813dfa40f07e511ccbb8397680019ff53c64-2850x1724.png)

Exports capture the dashboard as you're currently viewing it, including any active filters. Apply the filters you want first, then export.

## Managing dashboards

![dashboard management](https://cdn.sanity.io/images/wn0ylp2c/production/46a03ac9815b4f9ed0a6cf2dc68f510e86bd2189-2882x1726.png)

### Duplicate

The **Duplicate** action creates a full copy of a dashboard — blocks, layout, and filters — as a new dashboard that you own.

### Favorite

Click the star icon to add a dashboard to your favorites. Favorites are pinned to the top of the dashboards list and easy to jump to.

### Rename, icon, and color

Click the dashboard title or its icon to edit the name, choose an icon, and pick an accent color. These help you tell dashboards apart at a glance.

### Delete

The **Delete** action removes a dashboard. You'll be asked to confirm, and if the dashboard is set as anyone's personal home or the organization home, the confirmation calls that out.

---

# SELECT Free Trial Guide

URL: https://select.dev/docs/reference/usage-guide/select-free-trial-guide

> A guide to make the most out of your free trial with SELECT

Welcome to SELECT! This guide will walk you through everything you should do to make the most out of your free trial of SELECT. Let’s get started!

## Before the Free Trial

### Create Snowflake Service User

The one mandatory step before starting your free trial in SELECT is to [create a new Snowflake service user](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-1-create-snowflake-service-user) with read only access to your Snowflake metadata. Once this is completed, you’ll be ready to connect your Snowflake account to SELECT once the trial begins.

While creating the new Snowflake service user is the only required pre-trial step, there are few other optional things you can do in advance to make the most out of your evaluation period.

### Explore Relevant Integrations

SELECT helps attribute spend to the different tools in your data stack which connect to Snowflake. To get better visibility into your Snowflake spend, we recommend exploring our relevant [integrations](https://select.dev/docs/reference/integrations).

If you’re using dbt, we will require you to setup our open source dbt query tagging package. Instructions for the [SELECT dbt integration](https://select.dev/docs/reference/integrations/dbt) can be found [here](https://select.dev/docs/reference/integrations/dbt). We recommend setting this up in advance of the free trial so that you have some historical data you can analyze once you connect your account.

If you’re using Looker, we require a new Looker service user to be created. Instructions for the [SELECT Looker Integration](https://select.dev/docs/reference/integrations/looker) can be found [here](https://select.dev/docs/reference/integrations/looker). Other BI tools like Sigma, Hex and Periscope work out of the box based on the query comments automatically added by these tools.

### Prepare Query Tags

For any tools or custom applications we don’t have an integration with, SELECT can analyze your [Snowflake query tags](https://select.dev/posts/snowflake-query-tags) using our [Custom Workloads feature](https://select.dev/docs/reference/integrations/custom-workloads).

To take advantage of this feature, you’ll need to add a query tag or query comment to your queries run against Snowflake. Refer to [our guide here](https://select.dev/docs/reference/integrations/custom-workloads#integrating-with-custom-workloads) on how to integrate with custom workloads. Similar to the dbt integration, we recommend setting this up in advance of the free trial so you have some historical data to analyze.

### Apply Object Tags in Snowflake

[Snowflake object tags](https://select.dev/posts/snowflake-object-tags-guide) are key-value pairs that can be applied to any Snowflake object. This allows you to categorize Warehouses, Users, Roles, etc., by department, team, cost center, business purpose and so on. We wrote a [detailed article](https://select.dev/posts/snowflake-object-tags-guide) on how object tags work, how to create them, and how to observe existing tags in your account.

First, think through how you want to allocate spend to different teams. Is it by role? Database / schema? Once you have an idea of how to organize the budgets, implement the object tagging in Snowflake. Tagging objects is a powerful way to create [Usage Groups](https://select.dev/docs/reference/using-select/usage-groups) in SELECT, which we’ll do later on.

## During the Free Trial

Now that the trial prep is done, let’s get into how to make the most of your free trial. Here

### Log into SELECT Application

Head to [https://select.dev/signup](https://select.dev/signup) and enter your details. You’ll be emailed an invitation to the product within 24 hours.

### Connect Your Snowflake Account

After [creating the new Snowflake service user](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-1-create-snowflake-service-user) for your account, you can connect it to SELECT by following the instructions listed [here](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#step-2-connect-your-snowflake-account).

To access the add Snowflake account form, you can navigate to [https://select.dev/app/add-snowflake-account](https://select.dev/app/add-snowflake-account) or go to **Settings → Snowflake → Add New.**

When you successfully submit the form, it will take a few hours for the initial metadata sync. You will receive an email when the load is complete.

### Invite Teammates to SELECT

During your free trial, you can invite any of your team members to SELECT. Our instructions for adding new team members can be found [here](https://select.dev/docs/reference/using-select/access-management/user-management).

Invited users access can be restricted using SELECT built-in RBAC. You can grant users access to specific Snowflake accounts, or the entire Snowflake organization. See our [RBAC docs](https://select.dev/docs/reference/using-select/access-management/role-based-access-control) for more info. If you are interested in using Usage Groups to create RBAC, see our [usage guide](https://select.dev/docs/reference/usage-guide/usage-group-rbac-guide) on that topic.

### Enable Automated Savings

SELECT’s Automated Savings feature ensures your warehouses are fully utilized by adjusting warehouse configurations in real time. To enable Automated Savings, navigate to Warehouses page. You can use the check boxes to enable Automated Savings for multiple warehouses at once, or you can enable one Warehouse at a time using the toggle, as shown below.

![Enable Automated Savings in SELECT to save 10% on Snowflake right away.](https://cdn.sanity.io/images/wn0ylp2c/production/b3a864aadd66255c53084064f457e80492855848-800x401.gif)

In the table on the Warehouses page you can see your Annual Savings and Annual Estimated Savings.

![Show estimated savings in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/0770db1bd87963e49a0d2143b704a1a26ed8baa5-802x159.png)

Also on the Warehouses page, you can set the bar chart metric to Estimated Savings or Savings to see a time series of those metrics.

![SELECT, group choose a different metric in the chart](https://cdn.sanity.io/images/wn0ylp2c/production/104ed2dc3c283de6b8a09bc83443f9b4438e32d4-1675x577.png)

You can also see estimated savings for your account on the Insights page.

![SELECT Snowflake Savings Insights](https://cdn.sanity.io/images/wn0ylp2c/production/2e068f813ba330c9037f46ebcf4b4126a661cb03-1889x712.png)

Click “View n Findings” to see the estimated savings by Warehouse.

### Exploring Costs

![Exploring Snowflake costs in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/630f12dddb9c47c2d2131ac47433b5550ad06bfb-1893x948.png)

One way to explore cost in SELECT is from the Workloads pages. Decide if you’d like to explore All Workloads or a specific Workload category. The Workloads page will be sorted by highest annual cost. The default Workload (for uncategorized workloads) in SELECT is called [Query Patterns](https://select.dev/docs/reference/using-select/workloads#query-patterns). Query Patterns will group similar queries together so you can see the total annualized cost of running those similar queries. For more on [Workloads](https://select.dev/docs/reference/using-select/workloads), see our docs.

For more information, see our [usage guide on exploring costs in SELECT](https://select.dev/docs/reference/usage-guide/exploring-costs-in-select).

### Review Optimization Opportunities

The easiest way to find optimization opportunities in SELECT is from the Insights page. Here you’ll get an overview of the 6 insight categories with the ability to drill into each category by clicking the orange ‘View Findings’ button. Or, you can view all findings in the Explorer tab at the top.

![Review optimization opportunities in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/89f01fc5c40f1cf3a1a01bb5e1992fe066104e48-800x397.gif)

To learn more about finding optimization opportunities, please refer to our [usage guide](https://select.dev/docs/reference/usage-guide/identifying-optimization-opportunities).

### Configure Usage Groups

Usage Groups are a flexible way of creating cost categories in SELECT. This feature allows you to allocate cost to teams or projects using a variety of Snowflake attributes such as User, Role, Tags, Database, or Workload, to name a few. For more on creating [usage groups](https://select.dev/docs/reference/using-select/usage-groups), see our docs or our [usage guide](https://select.dev/docs/reference/usage-guide/allocating-costs-with-usage-groups) (coming soon).

### Configure Monitors

In SELECT, Monitors are a feature that allow you to monitor your Snowflake account spend by sending a digest at recurring intervals (we suggest weekly). You can also create alerts for spend anomalies or monitor your account for any condition using our custom SQL Monitors.

The alerts can be sent to a variety of destinations including Slack, MS Teams, or email.

You can monitor spend at the account level, or by Usage Group, Warehouse, or specific Workload.

For more on Monitors, see our [docs](https://select.dev/docs/reference/using-select/monitors) and our best-practices guide (coming soon).

### Create Saved Views and Favorites

Many screens in SELECT have filterable and configurable tables and charts. Instead of repeating the config and filters each time you visit the app, you can save the views.

![Save a view in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/ef8842f79898bd333a367b8b56ae33485ebb8d4b-1900x870.gif)

Saved views can be favorited, which pins them to the left sidebar for easy access.

We suggest saving and favoriting views to support your weekly cost review ritual. Companies who review important views weekly are more likely to keep their Snowflake spend under control!

For more info, see our docs for [Saved Views](https://select.dev/docs/reference/using-select/views) and [Favorites](https://select.dev/docs/reference/using-select/views#add-view-to-favorites) or our [usage guide](https://select.dev/docs/reference/usage-guide/using-views-for-ongoing-cost-reviews).

### Contact us at any point!

At any point during your free trial, you can reach out team via the in-app support by clicking the chat bubble at the bottom of the sidebar. Alternatively, reach us via email at [support@select.dev](mailto:support@select.dev).

![Contact SELECT: hello@select.dev](https://cdn.sanity.io/images/wn0ylp2c/production/f510863b29c904590b4efff9ada96280cf4029b5-3004x2004.png)

You can also book a meeting with our team at any point by reaching out to [hello@select.dev](mailto:hello@select.dev).

---

# Using Views for Ongoing Cost Reviews

URL: https://select.dev/docs/reference/usage-guide/using-views-for-ongoing-cost-reviews

> Let's review how to use Saved Views and Favorites in SELECT to establish a cost review ritual.

[YouTube video](https://youtu.be/UkAWztNqbPM?si=op6F57emcBbKmBTh)

In SELECT, it's common to apply the same filters or add the same columns each time you use the app. Views are a powerful feature that lets you save your filter and column configurations. They make it easy to build a repeatable cost review ritual by creating and reusing Views that highlight what matters most.

## Explore default views

SELECT comes with a few default Views. Click “Views” in the sidebar. On this page, any View created by SELECT is considered a default View. In this screenshot, I’ve filtered on the “Created By” field to show the default views:

![Default Views in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/f95797e78aff8f79a3d40d87e34cb5355c63c66d-1681x524.png)

This can serve as a starting point for cost reviews. You can easily modify these default Views by clicking into one, changing the filters (or column set), then clicking “Update For Everyone” in the top right.

For example, maybe you want to reduce the number of rows returned in the “Failing Workloads” view by filtering to a higher number of Failed Executions. Simply add a filter for “Failed Executions > 10” and the noise will be reduced.

![Edit a Saved View in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cf185e84643a5a03a427b55bcc58c68994116117-800x401.gif)

## Create a weekly cost review folder

You can pin your favorite Views to the sidebar using the Favorites feature and organize them into folders for easy access. Let’s create a “Weekly Cost Review” folder and add these default views to the folder:

- Top New Workloads
- Failing Workloads
- New Insights
- High Cost Increase Workloads

Here are the steps:

1. First, hover near “Favorites” in the sidebar until a + Folder icon appears. Click it to create a new folder named “Weekly Cost Review”.
2. Go to the Views page (sidebar).
3. Click “Top New Workloads”.
4. Now in the top right you will see the “✮ Top New Workloads” displayed in a drop down menu. Click that drop down, then click the ellipsis (three dots) next to “✮ Top New Workloads” and select Favorite. From this context menu you can add it directly to the folder.
5. Observe the new Favorite item in your new folder.
6. Repeat steps 2-6 for the other three views we want to add to this folder.

The end result should look like this:

![SELECT Favorites in Sidebar](https://cdn.sanity.io/images/wn0ylp2c/production/615c3ec9cf64968786527eb867a59c5b5a7c86f0-247x763.png)

## Create New Views

Let’s create two new Views and add them to our Weekly Cost Review folder.

### Workloads with Poor Pruning Efficiency

To find queries that execute full table scans, click Insights on the sidebar, then click View Findings on the Poor Pruning Workloads card.

![Snowflake poor pruning workloads](https://cdn.sanity.io/images/wn0ylp2c/production/ff3d61221098cd2553caef7383b1da70adae8d68-1897x744.png)

On the subsequent page you will find the “Save View” button in the top right. Click Save View, name it “Poor Pruning Workloads,” make it public, and add it to the Weekly Cost Review folder.

![Saved View in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4491a62f669416f61dedf67bda51a30873c80db7-580x358.png)

![Favorite a View in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/931273011ae28447824ddbc02624b64983dd9db3-1673x443.png)

To drill into any of these workloads, click the Side Peek button, then click View Resource in the side drawer.

![Drill into a resource to get more info in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cfd32de189173b7b503d1b5e2f79e29f0f6c4a95-1672x932.png)

### Queries Accessing a Specific Table

Let’s say we want to monitor all workloads accessing a specific table. This is commonly done to monitor performance when making changes to clustering keys. SELECT makes it easy to monitor this behavior, but requires Enterprise Edition of Snowflake. Here are the steps.

1. Go to Query History in the sidebar.
2. Optional: Hide the chart by clicking the Chart Toggle.
3. Add a filter using the + filter button: Tables Access → is → then search for a specific table. I’ll use our most popular table, Orders.
4. Save the view and add it to the Weekly Cost Review folder.

Here is the final result, including the updated sidebar:

![Favorites in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/e05d7f088177d291d510687ddde80f770ca38c6b-1897x955.png)

---

# Identifying Optimization Opportunities

URL: https://select.dev/docs/reference/usage-guide/identifying-optimization-opportunities

> Learn how to use SELECT to find cost optimization opportunities.

[YouTube video](https://youtu.be/rSLA9gFbdfc)

In this guide, we will explain the primary ways users can find optimization opportunities in SELECT.

## Method 1: Use Insights

The easiest way to find optimization opportunities in SELECT is from the Insights page found in the left sidebar. Here you’ll get an overview of 6 highlighted insight categories with the ability to drill into each category by clicking the reddish-orange ‘View Findings’ button. Or, you can view all findings in the Explorer tab at the top. Let’s walk through how to explore these.

![Insights page in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/89f01fc5c40f1cf3a1a01bb5e1992fe066104e48-800x397.gif)

### Exploring Insights by Category

On the main Insights page, you’ll find 6 categories of insights that you can explore one at a time. For example, let’s click the “View Findings” button on the “Enable Automated Savings” card.

![View insight findings in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/2172c0b5e4aa07f1438e77c27267ed38c838d0bd-1668x708.png)

The subsequent page will show each insight. In this case, it highlights warehouses where the Automated Savings feature can be enabled. The Side Peek button will give more info.

![Insights Explorer in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/037543c893b12ce57b8f570bd4e3bdcc750599cc-1676x555.png)

Let’s walk through one more insight category. Click “High Run Frequency Workloads” insight category:

![SELECT Insights](https://cdn.sanity.io/images/wn0ylp2c/production/b0fc646a91291ca346d5b82155e3ff8b67ad66a4-1660x706.png)

On the subsequent page you’ll find each workload that is running frequently, and in some cases you’ll get a tool-tip that compares the write frequency vs the read frequency of a table.

![SELECT Insights](https://cdn.sanity.io/images/wn0ylp2c/production/be4136e699ba4f79103a6027398084b90578d429-1669x370.png)

Let’s use the side peek button to open up the side drawer. Here we can use the slider to see the savings by reducing the run frequency of the job.

![SELECT Insight: Reduce run frequency of Snowflake job to save](https://cdn.sanity.io/images/wn0ylp2c/production/36ae7a7ada9647646c8b36adc7b230c11a11d71c-2048x1372.png)

### Find Low Effort Opportunities

To find low effort opportunities, click the Explorer tab near the top of the Insights Page. On the Explorer page, click the “+” Filter button, then select Effort → is → Low.

![Find low effort Snowflake savings in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/bc5e0bd615a34c63437e737f3bf7badc5cd8351f-720x442.png)

Now you have a list of all your low effort optimizations!

![SELECT Insights: Low effort snowflake savings](https://cdn.sanity.io/images/wn0ylp2c/production/4fa0556b30d7c2f323a2de33cfcb046956ef1430-1657x799.png)

### More on the Insight Explorer

It is worth noting that we have more Insight Categories than are shown on the Insights home page. On the Explorer page, you can use “Insight is” filter to see a list of all the insight types.

![All insight (Snowflake Savings) categories in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/f93fb2aa79150c4ea731abbeb4a773e4aab15b72-850x674.png)

When you hover on an Insight Name, you’ll get a nicely formatted tooltip explaining the Insight:

![SELECT Insight explanation](https://cdn.sanity.io/images/wn0ylp2c/production/5cee4ae892071cd8b357aa31e1bd2ac76469b8d3-1680x336.png)

### Click into a Resource to Learn More

In any Insight side drawer, you can click the View Resource button to learn more.

![Drill down on insights in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/59f7259b711469d3fce59cfa182ed9ab3b39c9ea-598x269.png)

This will navigate you to the page for that specific resource. In this case it is the dbt Resources page for the Orders model. From here, I can click through all of the tabs to learn more.

![SELECT Snowlake cost savings insights drill down](https://cdn.sanity.io/images/wn0ylp2c/production/d9ffc793c723060a0b4605239d3cb69f296b15ca-800x454.gif)

### Dismiss an Insight if You Want to Ignore it

If you decide an insight isn’t applicable, click the Dismiss Insight button for the insight. You can then add a dismissal reason, i.e. “Cannot reduce run frequency of this workload due to business requirements”.

![SELECT dismiss insights](https://cdn.sanity.io/images/wn0ylp2c/production/a9e25af4cde5383893fdc100c2dfb99f98a3eb08-2048x1379.png)

If you’d like to review your Dismissed Insights, you can do so by filtering the Explorer page to “Dismissed = true” and revisit previously dismissed Insights.

![SELECT review dismissed insights](https://cdn.sanity.io/images/wn0ylp2c/production/c429aa710c95979a12acfc4e7086516357774dfb-2048x1375.png)

### Filter the Insights Explorer by Usage Group

Don’t forget that all tables in SELECT, including Insights, are filterable by Usage Group Sets. This is an easy way to get Insights that apply to your specific team. See our [guide on Usage Groups](https://select.dev/docs/reference/usage-guide/allocating-costs-with-usage-groups) to learn more.

![SELECT filter insights](https://cdn.sanity.io/images/wn0ylp2c/production/85f7405b469171906d99583f6b9510e0d68b3a12-1669x442.png)

## Method 2: Explore Top Workloads

The next step is to explore the Workloads page(s), either from the All Workloads page or from a specific Workload type. The tables will automatically be sorted by your highest cost Workloads.

For each workload, ask yourself the following questions:

1. Evaluate if you can run things less frequently. Changing an hourly job to run only during business hours saves 67%. Changing a daily job to run on weekdays only, you save 29%. Running workloads less frequently is one of the most powerful cost optimization strategies, and takes very little effort! 
2. Evaluate if you can run Workloads on a smaller warehouse size. If using dbt, consider using an Extra-Small as the default warehouse size and use a larger warehouse for particular workloads that require more resources. This is another powerful cost savings technique that takes less than a day to fine tune. Be sure to monitor cost and performance of any adjustments you make using the workloads page → cost and performance tab.

![Save money in Snowflake by running jobs less frequently](https://cdn.sanity.io/images/wn0ylp2c/production/6823507977e7446876e0ae698ae120e16c5a4266-949x787.png)

When you drill into a Workload by clicking it (click the leftmost column in the Workloads table), you can see if that workload has any specific insights.

![Workload insights in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/0cb88069a9e3228487623549b28bea6f4045d8ed-800x400.gif)

## Method 3: Use Lineage to spot additional opportunities

[YouTube video](https://youtu.be/9LhNRXRVUPg)

Using the Workloads → All Workloads page, we can find our most expensive workloads. Then we can drill into any interesting Workload by clicking on the Workload name. To analyze the lineage of that workload, click the Lineage tab.

In this example, I’ll analyze the lineage of my most expensive dbt asset, Orders.

You can expand the graph by clicking the + or “expand all” icons.

![Using SELECT's lineage page to identify Snowflake optimization opportunities](https://cdn.sanity.io/images/wn0ylp2c/production/2764d49ad9890776234e9897c146e36cf9968d4b-800x401.gif)

Hover on the node KPIs to get informative tool tips.

Click on a node to open the side drawer for more details on that node, or click the “View Resource” button to get more detail on that node. From here, you can see a chart of cost over time under cost trends, a run frequency chart, and a list of all users access that Snowflake resource of BI dashboard.

![Navigate to details from Lineage page in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/242b9aa129cdefc957c8ac09c67ee4bff4dcaad2-1721x879.png)

The KPI tile in the upper right of the Lineage page shows the total annualized cost of all of the expanded nodes in your graph. As you expand more nodes, this tile will update.

![See total cost of DAG in SELECT's lneage page](https://cdn.sanity.io/images/wn0ylp2c/production/78d9c9cc897a51c25ba53eb797baddd3702433fd-1673x846.png)

---

# Exploring Costs in SELECT

URL: https://select.dev/docs/reference/usage-guide/exploring-costs-in-select

> Let's look at the most common ways to explore costs in SELECT.

[YouTube video](https://youtu.be/zHgcqHTLmpI)

SELECT offers a variety of tools to help you explore and understand your Snowflake spend. In this guide, we’ll walk through key areas you can investigate, including compute, storage, serverless usage, LLM costs, and more.

## Scan the Home Page

The Home page gives you a high-level overview of your entire account, breaking down spend across several dimensions, including:

- Total spend by service
- Potential savings by Insight category
- Compute spend by warehouse and user
- Most expensive workloads
- Usage Groups (if configured)
- Storage and serverless usage
- A summary of your automated savings

The bar charts and KPI tiles on the right side of the page offer helpful at-a-glance overviews of the various categories.

![Snowflake cost by Service in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4e7444a6698b1bcaeddefbf4511a927ab8b910c6-664x257.png)

![Snowflake cost by Warehouse and User in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/655ecc0e057c8281827443cd543025cd3d079540-668x727.png)

![Snowflake cost by Database in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/b2418d0064830b4405ded325f5dd39108814b206-667x399.png)

We recommend spending a few minutes exploring the home page, then move on to the more detailed pages such as Explorer and Workloads.

## Using the Cost Explorer

The Cost Explorer is a great starting point for deeper analysis.

In the left sidebar, click **Explorer**. The default page shows you cost by Service, but you can easily change the primary dimension by clicking the button in the top right and change the “group by” using the ellipsis on the chart.

![Cost Explorer in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4ec74f21d030819816a90e86ce45b203042c67b8-800x411.gif)

If you find specific explorer views helpful, you can [save them as a view and pin to your sidebar](https://select.dev/docs/reference/usage-guide/using-views-for-ongoing-cost-reviews) for later reuse.

### General Usage of Charts and Tables in SELECT

On the Explorer page, or any page with a chart or table, keep these tips in mind:

- Add filters in the top section.
- Click and drag in the chart to filter by date range.
- Clicking a bar chart item below the trend chart will temporarily filter the trend chart.
- The date grain (hour, day, month, etc.) and the “group by” (chart color breakout) can be changed using the ellipsis in the top right of the chart.
- Settings can be saved and favorited using the Save View button, then Favorite.
- Columns can be added to tables using the blue + sign on the right of any table.
- Columns can be removed by clicking the column header, then Hide.
- Columns can be re-ordered by dragging.

### Drilling Down in Cost Explorer

In the table in Cost Explorer (and many tables on other pages), click any name in the left column to drill into that resource. For example, drill into **Compute** to view warehouses, then into a warehouse to see workloads.

![SELECT Cost Explorer drill down](https://cdn.sanity.io/images/wn0ylp2c/production/7ae2f2eee7e9413c48af2a677aed6f0b091ca540-800x400.gif)

## Deep Dive on Compute Costs

A good way to focus on compute costs is to study the most expensive workloads in your account. Click Workloads → All Workloads (or pick a specific Workload Type) in the sidebar. The default chart layout will display by Workload Type, for example [Query Pattern](https://select.dev/docs/reference/using-select/workloads#query-patterns), [dbt](https://select.dev/docs/reference/integrations/dbt), Task, [Custom Workload](https://select.dev/docs/reference/integrations/custom-workloads), etc. If you’d like, you can change this to show by Warehouse, [Usage Group](https://select.dev/docs/reference/usage-guide/allocating-costs-with-usage-groups), Role, User, etc.

Like most pages in SELECT, you can filter by nearly any attribute, including query type, cost category, timing, resources, and more.

The table will be sorted with your most expensive Workloads on top. Drill into these Workloads by clicking on the name in the left column. Here’s an example of how I can explore our most expensive dbt model to gain some insights:

![Explore compute costs for top workload in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/fe02aee24bc7377d3851363b81ce4a77d980c31c-800x456.gif)

## Exploring Storage Costs

To explore Storage costs in SELECT, use the Storage page in the sidebar. The navigation and exploration pattern is similar to other pages. By default, you'll see data by database, and you can switch to Schema or Table using the tabs at the top.

### Finding Unused Tables

One common thing we like to look at is unused tables. To find unused tables, you can add a filter: Data Access → Last Query Date → is before → pick a date.

You can also add a few helpful columns to your table to analyze the impact of dropping a table or eliminating an ETL process:

- Annual Write Query Cost
- Annual Read Query Cost
- Number of Users
- Users

SELECT comes with a default View for Unused Tables, but we thought it would be helpful for you to see how to create a similar from scratch.

## Exploring Cortex LLM Costs

The best place to find Cortex Cost is also on the Workloads page.

1. Click Workloads → All Workloads.
2. Change the chart metric to Cortex Function Cost (type Cortex in the search).
3. Change the Group By to Cortex Function Name or Cortex Model.
4. Add a column to the table: Cortex Function Cost.
5. Sort by that column.
6. Drill down on the workloads (click the name in the left column) to learn more.

![Explore Cortex cost in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/1600bbed2a7d6eed61aef3b973876fe9746b6dc0-800x397.gif)

Don’t forget to [save this View](https://select.dev/docs/reference/using-select/views) and favorite it!

At this time SELECT doesn’t support cost exploration for Cortex Analyst or other AI services, but we will in the future.

## Explore Serverless Costs

To explore Serverless costs, click Workloads → Serverless in the sidebar. The default chart grouping is by “Type”, where you can find Automatic Clustering, Snowpipe, Serverless Tasks, QAS, Search Optimization Services, to name a few.

The exploration pattern here is much the same. Apply any filters / add any columns you’d like. Then drill down by clicking the item in the Name column.

![Explore Serverless cost in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/5d74275d69dd065b70ff62bdafb1eb148fd95e13-1887x1125.png)

## Lineage

[YouTube video](https://youtu.be/9LhNRXRVUPg)

Another powerful way to explore cost in SELECT is via the Lineage tab. [Our Usage Guide on Identifying Optimization Opportunities](https://select.dev/docs/reference/usage-guide/identifying-optimization-opportunities#method-3-use-lineage-to-spot-additional-opportunities) covers using lineage, so please refer to that article.

---

# Understanding Cost Spikes

URL: https://select.dev/docs/reference/usage-guide/understanding-cost-spikes

> We will walk through how to use SELECT to understand cost spikes in your Snowflake account

[YouTube video](https://youtu.be/6yZTSzGCF0s)

SELECT makes it easy to spot and investigate unexpected cost spikes in your Snowflake account. The high level process is:

1. Use the trend charts on the Home → Dashboard to identify dates with a spike in spend.
2. Pinpoint which services, warehouses, users, or databases contributed to the spike.
3. Use those details to filter down to specific workloads. For example, you can navigate to the Workloads page filtered by date range and warehouse to isolate the source of the spike.

## Identify Spikes Using the Trend Charts

Navigate to **Home → Dashboard** (tab at the top of the screen). In this example, I’ll set the date filter to **Last 90 Days** to surface spikes across a longer timeframe. Here, there’s a clear cost spike from late May to early June:

![SELECT dashboard page reveals a spike in Snowflake cost](https://cdn.sanity.io/images/wn0ylp2c/production/cfb1762bb4f8d9f7b8539ef758ba61e5b23eeba4-1660x729.png)

As I scroll down, I can see spend broken out by category. The first category is **Snowflake Service**, where it becomes immediately clear the spike is in **Compute** spend. Interestingly, once compute spend returned to normal, there was a noticeable spike in **Storage** spend a few days later.

![Analyze cost spikes in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cfb1762bb4f8d9f7b8539ef758ba61e5b23eeba4-1660x729.png)

Continuing down the page, we see the spike was entirely driven by the **SYSTEM** user. That suggests the cause could be a task or dynamic table, but let’s keep exploring.

![Analyze cost spikes in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/45cae020cf769436b70bd83c9cf62981e8fdf9a6-800x392.gif)

We can see from the final chart that TUTORIAL_DB was responsible for the Storage spike.

![Snowflake storage cost by Database in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4603db59426fa7d8631197ee6c01bb467bcc01cd-1660x499.png)

Now, let’s find out what happened.

## Drill Down to Resources with the Spike

Let’s figure out what caused the spike in Compute cost. First, filter to the date range where the spike occurred. Then, click the warehouse name in the Warehouse bar chart to jump to the Workloads page filtered by that warehouse and date range.

From there, it quickly becomes clear that a Task called `WEEKLY_REFRESH_DEMO_ORDERS` was responsible for about 90% of the spend on those days.

![Drill down on Snowflake costs spikes in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/13aee8b17b00d05fb517c199d6254bddfdd61293-800x397.gif)

Click the workload name in the table, then go to the **Cost & Performance** tab. Here, you’ll see the task was running about 300 times per day for a few days, then dropped to once per day. That explains the spike!

![In SELECT, use Cost and Performance tab to analyze any asset that drove a Snowflake cost spike](https://cdn.sanity.io/images/wn0ylp2c/production/b547496cc6f654f92665a1b3e020bca95b0c14c2-1655x929.png)

## Generalizing the Approach

You can take the same approach from any of the “Resources” pages in SELECT: Workloads, Query History, Warehouses, or Storage. You can also follow this process from the Explorer page. The steps are:

1. Identify the spike on a trend chart.
2. Filter to the spike.
3. Drill into the detailed pages by clicking the left column.
4. Analyze the _Cost & Performance_ tab, _Lineage_, and _Insights_.

---

# Allocating Costs with Usage Groups

URL: https://select.dev/docs/reference/usage-guide/allocating-costs-with-usage-groups

> Usage Groups are a flexible way to attribute Snowflake spend in SELECT. Let's dive in.

[YouTube video](https://youtu.be/6f2fFBDaUfs)

[Usage Groups](https://select.dev/docs/reference/using-select/usage-groups) are a flexible way to allocate costs in SELECT. With Usage Groups, you can flexibly allocate cost to teams, departments, project, or any bucket of your choice using a variety of Snowflake attributes such as User, Role, Warehouse, Object Tags, Database, or other query metadata.

## Usage Group Sets vs. Usage Groups

Usage Group Sets are collections of Usage Groups. For example, we can create a Usage Group Set called “Team” and the individual Usage Groups can be “Finance”, “Accounting”, “HR”, etc. Other common examples of Usage Group Sets include “Department”, “Cost Center”, “Customer” or “Product”.

## Creating Usage Group Sets

Before creating any Usage Groups, you need to first create a new Usage Group Set.

In the region above the chart, click “Usage Group Set”, then “Manage Sets”. Add a new name, then click “Done” and “Save”.

This gif demonstrates creating a new Usage Group Set. If you have not created one before, your page will look a little different, but the overall steps are similar.

![How to create a Usage Group Set in SELECT to attribute Snowflake spend](https://cdn.sanity.io/images/wn0ylp2c/production/cfff938a696752dfdb780cc42d3ab71d3b9c487d-800x387.gif)

## Allocate Using dbt Metadata

Let’s say we want to attribute every dbt model and test executed in our environment to a specific business owners. The first step would be to follow our docs on [leveraging dbt meta](https://select.dev/docs/reference/integrations/dbt#leveraging-dbt-meta-tags) fields. Once the meta fields are set up in your dbt project, we are ready to create a Usage Group Set and Usage Groups.

Create the Set as shown above, then let’s create the Usage Groups. In this example we’ll create one Group for each Business Owner found in the dbt Meta, as shown here:

![Create a Usage Group in SELECT to attribute Snowflake spend to teams](https://cdn.sanity.io/images/wn0ylp2c/production/4374447d526ee4a83d88f8438dc5201fd1224b42-1672x802.png)

After the Usage Groups are created, visit the Workloads → dbt page, and change the “Group By” dimension on the chart to “Usage Group Set - dbt team”. Now the chart clearly shows which business owner is spending the most in Snowflake!

![View trend chart of Snowflake Spend by flexible Usage Group in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/69e89c2546ed4c8c4286ea0c7249ebb0822bbfe7-1668x590.png)

In the table below, I can add a column for “Usage Group Set > dbt teams” to show each model along side the team name.

![Add Usage Group column to a table in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/096aab154feb87f12d4acf5dff5b689c042b20ef-1664x364.png)

## Allocating Costs Using Snowflake Objects

One of the most popular ways of attributing cost is by using Snowflake Objects such as Warehouse, User, and Role for Compute and Database / Schema for Storage. In SELECT you can easily create usage groups based on any Snowflake Object.

Here’s an example of using several Snowflake Objects allocate cost to departments.

![Allocate Snowflake costs by Object in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/7a5ad72205d6633af66d6485bb1db5d0236f2c21-1680x962.png)

## Allocating Costs Using Snowflake Object Tags

Another popular way of attributing costs is by Snowflake Object Tags. Assuming you have [configured object tags](https://select.dev/posts/snowflake-object-tags-guide) in your account, search for the word “tag” in the Usage Group configuration `where` dropdown search. Here you can see that we can report on Warehouse, User, Role, or Database Tags.

![Allocate Snowflake Cost by Object Tag in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/4c3166411646a66a5619953293368a1579982a15-337x223.png)

All other aspects of creating the usage group remains the same.

## Allocate Spend Using Other SELECT Workloads

**Custom Workloads / Query Tag Metadata**

If you’re not using dbt, you may want to allocate specific workloads by Query Tag or Query Comments using our Custom Workloads feature. Once you’ve [configured the Custom Workloads](https://select.dev/docs/reference/integrations/custom-workloads), you can find them in the Usage Group configuration by typing the word “custom” in the `where` dropdown search.

![Allocate Snowflake cost by Custom Workload or Query Tag Metadata in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/3254e0abd1fadf3dccc74c3a2c3b49a278dee1aa-379x214.png)

**Other Supported Workloads**

You can also allocated spend to a Usage Group based on Tasks, Snowpipe, Dynamic Tables, Stored Procedures, Serverless Tasks, Looker User, Hex User, etc.

## Reviewing Uncategorized Spend

In the chart screenshot above, you can see we have a small amount of uncategorized spend in our “dbt team” Usage Group Set, which means some model or test is missing the meta tag. Let’s isolate the problem by filtering to the "Uncategorized" data:

![Review uncategorized (usage group) Spend in SELECT.](https://cdn.sanity.io/images/wn0ylp2c/production/38283f6bbb3b026d8eaa3a4c503aac5bad7a976f-800x399.gif)

After getting the list of dbt assets that are missing the “business_owner” Meta in dbt, I update my dbt project to include the meta for those assets, then update the usage groups in SELECT. Don’t forget to create a [Saved View](https://select.dev/docs/reference/using-select/views) and add a [favorite](https://select.dev/docs/reference/using-select/views#add-view-to-favorites) to track uncategorized spend! The best practice is to make a habit of reviewing and eliminating uncategorized spend in Usage Groups.

## Other Callouts with Usage Groups

There are a few things to keep in mind and explore about Usage Groups:

- As seen in the demonstration above, Usage Groups can be added to the “group by” of any chart in SELECT or as a column in any table.
- It is easy to monitor spend by Usage Groups by [creating Monitors](https://select.dev/docs/reference/using-select/monitors) for each Usage Group Set.
- Usage Groups can also be used for Role Based Access Control, as demonstrated in [this usage guide](https://select.dev/docs/reference/usage-guide/usage-group-rbac-guide).
- For simpler ongoing management and version control, Usage Groups can be created and [managed via Terraform](https://select.dev/docs/reference/api-reference/managing-usage-groups-terraform) or through our [Usage Groups API](https://select.dev/docs/reference/api-reference/working-with-the-usage-groups-api).

---

# Resource Based Access Control in SELECT

URL: https://select.dev/docs/reference/usage-guide/usage-group-rbac-guide

> This guide demonstrates how to control access to specific resources in SELECT

Most SELECT customers will provide SELECT access to their users using our out of the box [roles](https://select.dev/docs/reference/using-select/access-management/role-based-access-control), typically giving them a viewer or editor role on the Snowflake account.

However, certain customers may wish to further restrict access within a single Snowflake account. For example, you may want to only allow a user to see specific warehouses/databases that are owned by their team.

In SELECT, you can accomplish restrict a user’s access within a Snowflake account through [Usage Groups](https://select.dev/docs/reference/using-select/usage-groups). 

## Create thoughtful Usage Groups

To limit a user to only see specific resources in a Snowflake account, you need to first create Usage Groups. The user's access will then be restricted to one or more Usage Groups.

The first step is define clear and thoughtful Usage Groups.

One approach would be to assign Usage Groups to teams within your org. In this case we can create a Usage Group Set called “RBAC Teams” or just “Teams”, and the individual groups within the set can be something like “HR”, “Finance”, “Accounting”, “Product Supply”, etc. 

Warehouses, databases and other resources can then be assigned to each Usage Group. For details on the mechanics of creating Usage Groups and Sets, see our [docs page](https://select.dev/docs/reference/using-select/usage-groups).

## Create a Team and assign Roles to the Team

The next step is to create a Team.

1. Click the gear icon in the bottom left
2. Click Teams in the sidebar
3. In the top right, click “Create Team”.

![Creating a Team in the SELECT app](https://cdn.sanity.io/images/wn0ylp2c/production/3b90a6119c94d25b8e260ce26f36f8b90592141c-1876x953.png)

After you click “Create Team”, a right side panel will pop out. Here, you can give the Team a name and assign roles.

> [!NOTE]
> When you use Usage Groups as the entity for access control, the only role that can be assigned at the Team level is Viewer. This means a member of the team cannot edit the Usage Group itself, which would widen their own access.

![Creating a Team in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/38588a784f3389ad25e24f5ef069493d32e73e9f-528x950.gif)

## Add members to Teams and set the permission level of Team members

### Manually add Team members

After you save the team, click Add Members. Select a user name and give them Editor or Viewer “Team Resource Access”.

Assigning “Editor” will allow the user to do things like create and edit monitors, save views, and edit views **belonging to that Team**. Assigning “Viewer” will make the user read-only across the SELECT platform, assuming no other roles are applied to the user.

![Add members to a team in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/7deb7847654ea66dcf28fd31b943740b35d87de1-1588x774.gif)

### Automatically assign users to Team through SSO Groups

To avoid manually adding members to a team, you can automatically assign them to teams by setting up an SSO Group mapping.  
  
SSO Group mappings can be configured on the Team side panel, which can be accessed by clicking the Edit button beside the Team.

Under the **Team Memberships via SSO Groups** section, click **Add SSO group.** In the screenshot below, any user who is part of the **DATAENG** group in your SSO provider (Okta, Entra ID, etc.), will automatically get assigned to the **Data Engineers** team when they log in, and get the associated roles/permissions.

![SELECT SSO group mapping for team membership](https://cdn.sanity.io/images/wn0ylp2c/production/72fc5e030560fc70b84ffed037fa81b5392efc3e-2992x2104.png)

## What the user sees

For a user who only has view access to a specific Usage Group every page in SELECT will be filtered based on the Usage Group they’ve been assigned.

![Usage Group RBAC in SELECT; end users sees filtered data on every page](https://cdn.sanity.io/images/wn0ylp2c/production/bf46da88a42f1bd482f23f89691f11d2af6dc706-1889x805.png)

If a user has **Viewer** “Team Resource Access” assigned (and not **Editor**), things like “Create Monitor” will be greyed out. They also won’t be able to edit or disable an existing monitor.

![Create Monitors is greyed out in SELECT if a user has Viewer Access on a Team.](https://cdn.sanity.io/images/wn0ylp2c/production/f0cf9d09e93222c6815a7b1cba03c6f2b518b2b4-1678x414.png)

![User cannot disable Monitors in SELECT if they are Viewer on a Team](https://cdn.sanity.io/images/wn0ylp2c/production/6709454181e394d8ca4026fb1ce9986e12b548b4-329x107.png)

For saving Views, the user with Viewer role cannot save a public or team view:

![Viewer on team cannot save Public Views in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/3fe38b09057be95ae7cb223acf4f215bdacbdf47-583x457.png)

But they can save a Personal View.

Switching a user’s Team Resource Access to Editor will immediately enable them to create Monitors and save team Views.

## Editing user’s access

Editing a user’s access can be done in two places. The first place we’ve already seen: Settings → Teams → then hit the expand arrow icon for the team you want to edit. From this screen you can Add Members, edit a member’s permission, or delete a member.

![Editing a Team Member in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/3001b95abad3274c4cc479463545664ad3cc6c9d-1648x214.png)

You can also edit a user’s permissions from the Settings → Users page. Click the “side peek” icon next to the user name.

![Editing a User in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/03f7f879b33f6f5398c2c3d7ea2c98b1e60bb7e6-1423x98.png)

Then edit the permission in the sidebar.

![Edit User in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cf4df8f4014f35986547f6fc64e515cd71207016-580x952.png)

It is important to note that on the Edit User page, you can assign the user roles outside of Team Membership. This means a user doesn’t have to be a member of a Team to get Viewer access on a Usage Group (or Account). However, we recommend using the Teams feature because it keeps things organized and this is the only place you can give the “Edit” permission.

---

# Accessing Data in SELECT

URL: https://select.dev/docs/reference/usage-guide/accessing-select-data

> Export your SELECT data

## Export to CSV

Users can easily export data from any chart or table they see in the SELECT UI. Just hover over the data you want to export and you'll see a menu with 4 options:

1. Download to CSV
2. Copy CSV to Clipboard
3. Copy TSV to Clipboard (use this option to paste directly in Google Sheets!)
4. View SQL (if you have a SELECT data share setup, learn more [here](https://select.dev/docs/reference/usage-guide/accessing-select-data#data-sharing-feature-overview))

[Video](https://storage.googleapis.com/prod_select_images/docs/download_csv.mp4)

## Data Sharing Feature Overview

> [!NOTE]
> Enabling data sharing for your SELECT account is an add-on feature. Please contact SELECT to determine your eligibility and receive a quote.

Once the data share has been enabled, you can hover over any chart or table in SELECT and you will see a little code icon appear in the top left. Copy the SQL code and run it directly in your Snowflake account. You’ll see the exact data used to populate the UI in SELECT!

[Video](https://storage.googleapis.com/prod_select_images/docs/view_sql.mp4)

### Setup Instructions

Using the `accountadmin` role, run the `show shares` SQL command in your account:

```sql
use role accountadmin;
show shares;
```

You'll see an output that looks like this:

![SELECT data share setup part 1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/dataShare1.png/width=2030,quality=75)

Look for the row where `owner_account` is something like `SCWXHOB.SELECT_AWS_US_WEST_2_PROVIDER` (the name will vary depending on the cloud/region you are using, but it will have the similar structure of `SCWXHOB.SELECT_<cloud provider>_<cloud region>_PROVIDER`). The share name will look like `SELECT_DATA_SHARE_ORG_ABC123` i.e. `SELECT_DATA_SHARE_<your org id>`.

Copy and paste the account name and share name from the output above, then create a database in your account for the share:

```sql
-- modify using the owner_account and name from the output of the show shares query
-- keep the database named select_datashare if possible
create database select_datashare 
from share scwxhob.select_aws_us_west_2_provider.select_data_share_org_abc123
;
```

> [!WARNING]
> The generated SQL shown in the demo video above assumes that the database you have created is named `select_datashare`. You can name it something else, but the SQL code we generate will not work directly as a result.

By default, this shared database is only accessible to users with the [`accountadmin`](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles) [role](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles). You can grant access to other users using the following command:

```sql
grant imported privileges on database select_datashare to role <enter_role_name_here>;
```

### Querying the data with Python

If you are looking to automate spend reporting within your organization using the data available in SELECT, you can easily do so once the data share has been setup. Once you have copied & modified the relevant SQL query you want to run from the SELECT UI, you can programmatically execute that query and retrieve the results using the [Snowflake Python connector](https://select.dev/posts/snowflake-python-connector).

#### Example Python Script

Here is an example Python script that shows the end-to-end flow for connecting to Snowflake, executing the query, and retrieving the results as a list of dictionaries:

```python
import os

import snowflake.connector

# Replace this with your Snowflake account identifier. 
# See our docs on how to find this if you don't know it.
# https://select.dev/docs/connect-snowflake-account#using-snowsight-recommended
account_identifier = 'your_account_identifier'

# Ensure this is a service user account that doesn't require MFA
user = os.environ['SNOWFLAKE_USERNAME']
password = os.environ['SNOWFLAKE_PASSWORD']

# Example SQL query. Update this!
sql = """
select
    query_id
from snowflake.account_usage.query_history
where
    start_time > current_date - 1
limit 10
"""

# Establish a connection to Snowflake
conn = snowflake.connector.connect(
    user=user,
    password=password,
    account=account_identifier,
)
# Create a cursor object
cursor = conn.cursor()

try:
    # Execute the SQL query
    cursor.execute(sql)
    # Fetch the results
    results = cursor.fetchall()
    # Get column names
    column_names = [desc[0] for desc in cursor.description]
    # Convert the results to a list of dictionaries
    result_list = [dict(zip(column_names, row)) for row in results]
finally:
    # Close the connection
    conn.close()
```

Here's an example of the `result_list` output from above:

```markdown
>>> result_list
[{'QUERY_ID': '01b17a56-0000-48b5-0000-3de90684bc82'}, {'QUERY_ID': '01b17a56-0000-48b5-0000-3de90684bc86'}, {'QUERY_ID': '01b17a56-0000-48b5-0000-3de90684bc8a'}, {'QUERY_ID': '01b17a56-0000-4892-0000-3de90684c922'}, {'QUERY_ID': '01b17a56-0000-48c1-0000-3de90684e48a'}, {'QUERY_ID': '01b17a56-0000-48c1-0000-3de90684e48e'}, {'QUERY_ID': '01b17a54-0000-4892-0000-3de90684c91a'}, {'QUERY_ID': '01b17a54-0000-48c1-0000-3de90684e482'}, {'QUERY_ID': '01b17a53-0000-48b5-0000-3de90684bc7e'}, {'QUERY_ID': '01b17a53-0000-4892-0000-3de90684c916'}]
>>>
```

#### Generating a custom date range

Let's imagine you want to report on spend by Snowflake service, and you grab the corresponding query from SELECT's UI:

![SELECT data share example query](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/adfdb401-afe4-4c3f-3af0-4909db5f8b00/width=2598,quality=75)

In the generated SQL, a date range is shown for the period selected in the web UI. For your custom reporting, you'll likely want to change this.

```sql
select
    to_char(date, 'YYYY-MM-DD') as day,
    service as service,
    sum(spend) as sum_spend
from select_datashare.consolidated.consolidated_daily_spend
where
    to_char(date, 'YYYY-MM-DD') >= '2023-11-07' -- dates from the SELECT web UI
    and to_char(date, 'YYYY-MM-DD') <= '2024-01-05'
group by all
```

We can accomplish this using the `datetime` library from Python

```python
from datetime import datetime, timedelta

period_end = datetime.now() - timedelta(days=1)
period_start = period_end - timedelta(days=30)

sql = f"""
select
    to_char(date, 'YYYY-MM-DD') as day,
    service as service,
    sum(spend) as sum_spend
from select_datashare.consolidated.consolidated_daily_spend
where
    to_char(date, 'YYYY-MM-DD') >= '{period_start.strftime('%Y-%m-%d')}'
    and to_char(date, 'YYYY-MM-DD') <= '{period_end.strftime('%Y-%m-%d')}'
group by all
"""
```

### Limitations

- Not all SELECT datasets have been exposed to the data share. Please reach out if you see a dataset that is not present and we will expedite the process of adding it.
- Not all UI elements have the code snippet available as shown in the demo above. If you see a chart/table and would like to access to underlying data but cannot see the corresponding query, please reach out to us.

---

# Using Monitors to Stay on Top of Cost

URL: https://select.dev/docs/reference/usage-guide/monitors-guide

> Learn how to use monitors to get alerted of spikes or anomilies in your Snowflake account.

[YouTube video](https://youtu.be/aY3Y2bQn-Qo)

Monitors in SELECT allow you to set conditional alerts or scheduled reports sent to Slack, Teams, or Email. Our [docs page](https://select.dev/docs/reference/using-select/monitors) walks through an entire example of creating a monitor and configuring destinations. Start there to get an idea of how to use monitors. This page will focus more on when to use monitors, what use cases are covered, best practices, etc.

## When To Use Monitors

Monitors have two general uses:

1. Proactive alerts for spikes, anomalies, or spend thresholds exceeded.
2. Scheduled reports that are more FYI in nature.

You can create a Monitor for just about anything you’d want to know about your Snowflake account. If our out-of-the-box monitors don’t cover your use case, you can use Custom SQL Monitors to meet your needs.

Examples include:

- Weekly account spend report
- Monitor spikes for a usage group, warehouse, workload, etc.
- Report long running or expensive queries from a previous time period
- Report long running queries still running
- Monitor any measure: Cortex spend, credits, partitions scanned, pruning efficiency, rows inserted, etc. Any measure available in SELECT is available in Monitors.
- Get notified about new insights
- … and many other conditions you may want to track

If you have a use case for an alert or scheduled report, Monitors has you covered.

## Types of Templates

The easiest way to get started with Monitors is by using a template. Clicking “Create Monitor” will bring up a long list of templates to choose from. The templates are organized by category, so click through each category to see what types of templates are available.

> [!TIP]
> Remember, templates are just a starting point. The idea is to customize a template to your exact needs.

![Monitor templates in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/f0a0f06e4573b8225f462e508d40c0412452a5b0-898x581.png)

- Account Spend templates: 
   - Use this to monitor spend on your entire account.
   - The results are grouped by Warehouse by default but can be changed.
   - This is the only type of template where the measure cannot be changed from “Spend” to some other measure.
- Warehouse templates:
   - Use these when you want to filter by warehouse and get flexible measures.
- Usage Group templates:
   - Use these to report spend or alert anomalies by [Usage Groups](https://select.dev/docs/reference/usage-guide/allocating-costs-with-usage-groups).
- Workload templates:
   - Stay on top of spend for various workload types: dbt, stored procedures, dynamic tables, BI tools, etc.
- Storage templates:
   - Used to monitor total storage cost or volume.
- Insights templates:
   - Get alerted about new insights in SELECT.
- Custom SQL:
   - Use this when editing one of our primary data source templates does not work for you.

### When to use Templates vs Create From Scratch

Since all templates are editable, starting with one is always a safe choice. It is possible to completely diverge from the template by selecting a different data source or editing any of the configuration. You would never need to close a template and start over if you choose the wrong one!

Use the “Continue without Template” option if you know exactly what you are after and are already familiar with SELECT’s monitor creation workflow.

## Custom SQL Monitors

Custom SQL Monitors are a powerful option to monitor any condition in your Snowflake account not covered by one of our standard data sources. Custom SQL Monitors have an important distinction from other monitors: they run directly in your Snowflake account with the permissions of your service user. Therefore, if you want, you can expand the permissions of that user to access log tables (or anything you’d like) outside of the Snowflake metadata database.

Since Custom SQL Monitors run in your account, this is the only way you can monitor activity in real-time. All other standard Monitors run on SELECT’s backend, which only updates your account once per day.

Here are some examples of what you can do with Custom SQL monitors that is not possible with our standard monitors.

- Alert on long running queries still running now.
- Alert on intra-day anomalies such as hourly warehouse spikes, data transferred spikes.
- Report failed tasks every hour.
- Report user metadata, such as users without MFA or users with RSA keys.
- Monitor custom logging tables by expanding the permissions of the SELECT user.

Custom SQL Monitors are the only monitor type that can be scheduled with Cron. Standard monitor types are scheduled Daily, Weekly, or Monthly, and trigger after your Snowflake data is synced to SELECT for that day.

## Understanding the Flow of Creating a Monitor

Our [docs page](https://select.dev/docs/reference/using-select/monitors/using-monitors) walks through the monitor creation process in detail. Here’s a screenshot with each configuration section collapsed, so you can get an idea for the steps involved in creating a monitor:

![SELECT Monitor creation flow](https://cdn.sanity.io/images/wn0ylp2c/production/f7af86824fb7a9ccbdb7092a98b4daab5697af2c-1280x349.png)

You can see at a high level the creation process is as simple as filling out the form. Consider each section carefully as you design a effective monitor.

## Best Practices for Monitor Schedule and Trigger Conditions

![Monitor types in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/cb297866a8bef6017742cfd516c1af9c5dd73a8a-1243x283.png)

There is a bit of an art to selecting a monitor schedule. From a high level, you want to avoid alert fatigue while staying informed. If notifications come too often, you learn to ignore them. Here are some guidelines to follow:

- **Scheduled Reports** are sent every time on schedule, without checking thresholds or whether results exist. These serve as simple FYI updates on your account activity.
   - Schedule these **weekly** at most.
   - Use **monthly** reports to update stakeholders on Snowflake activity.
- **Results Found** is commonly used when you filter a measure or in Custom SQL Monitors. It only triggers if the SQL statement produces at least 1 row. Since these monitors typically filter on a measure, the custom message and trend chart are unavailable. Common use cases for “Results Found” monitors are: 
   - Alert if a failure log exists.
   - Alert if a long running query exists.
   - Alert when any condition exists.

![Results Found monitors in SELECT only send when filtered data has results.](https://cdn.sanity.io/images/wn0ylp2c/production/45d81f2dc83d936396b4dbb1784741ede24ee0be-1240x285.png)

- **Threshold** monitors are great when you only want to get alerted if the data exceeds a certain threshold. This option allows you to still see the trend chart and custom message when a threshold is exceeded. Example: Daily spend exceeds $1,000.
- **Anomaly** monitors are great for alerting on changes or sudden spikes in your account.

Results Found, Threshold, and Anomaly monitors execute on a schedule (daily, weekly, monthly) after your Snowflake data is synced to SELECT for that day but only send if the conditions are met.

Best practice is to become familiar with each type of monitor and carefully choose the trigger condition and schedule.

---

# dbt

URL: https://select.dev/docs/reference/integrations/dbt

> Learn how to use SELECT dbt integration

## Feature overview

[Video](https://storage.googleapis.com/prod_select_images/marketing/dbt-overview.mp4)

The SELECT dbt integration allows you better understand and optimize Snowflake spend attributed to dbt.

- Dissect your dbt spend across a variety of different dimensions such as project, target, materialization and resource type.
- Identify expensive models and tests, and understand how their cost and performance varies over time.
- Drill into the execution history of any dbt resource and view query profile diagnostics for each run.
- Receive tailored optimization recommendations to improve model cost and performance

## dbt model timing chart

With SELECT's dbt integration, you can also understand the performance of each dbt run with our model timing chart, located on the **Timeline** tab of each dbt run page:

![SELECT Snowflake dbt model run timing chart](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/66955772-4fdc-4614-7315-88a662b44c00/width=2526,quality=75)

A great feature of this chart is you can click straight into a given model's run to understand why it took the time it did using our version of the [Snowflake query profile](https://select.dev/posts/snowflake-query-profile):

[Video](https://storage.googleapis.com/prod_select_images/changelog/dbt_model_timeline_20230712.mp4)

## dbt runs view

Understand the performance and cost of your dbt project over time using the dbt runs view. Filter for specific commands, targets, users and more. Click into individual runs in the table or chart to view more detail and access the timing chart.

[Video](https://storage.googleapis.com/prod_select_images/changelog/dbt_runs_chart_20240229.mp4)

## Setup

SELECT uses query tags and comment metadata to power this integration. To enable this integration, follow the configuration steps below.

### Step 1: Update packages.yml

Add the following entry to your dbt project's `packages.yml` file in the root of the dbt project, creating the file if it doesn't exist:

```yaml
packages:
  - package: get-select/dbt_query_tags
    version: [">=3.0.0", "<4.0.0"]
```

### Step 2: Adding the query tags

Configure the dispatch search order in `dbt_project.yml`.

```yaml
dispatch:
  - macro_namespace: dbt
    search_order:
      - <YOUR_PROJECT_NAME>
      - dbt_query_tags
      - dbt
```

### Step 3: Configure the query comments

Add the following config to `dbt_project.yml`.

```yaml
query-comment:
  comment: '{{ dbt_query_tags.get_query_comment(node) }}'
  append: true
```

That's it! All dbt-issued queries will now be tagged.

#### Example dbt query metadata

This configuration adds metadata describing each dbt-issued query to a tag on the query. Once enabled, SELECT will begin to provide insights, analytics and recommendations on your dbt project. An example of the metadata added to each query is shown below:

```json
{
  "dbt_query_tags_version": "3.0.1",
  "app": "dbt",
  "dbt_version": "1.4.0",
  "project_name": "my_project",
  "target_name": "dev",
  "target_database": "dev",
  "target_schema": "larry_goldings",
  "invocation_id": "c784c7d0-5c3f-4765-805c-0a377fefcaa0",
  "node_name": "orders",
  "node_alias": "orders",
  "node_package_name": "my_project",
  "node_original_file_path": "models/staging/orders.sql",
  "node_database": "dev",
  "node_schema": "mart",
  "node_id": "model.my_project.orders",
  "node_resource_type": "model",
  "materialized": "incremental",
  "is_incremental": true
}
```

### Step 4: All done!

That's all that's required for setup. The next time your dbt project runs, the generated queries will automatically be tagged with this metadata. SELECT will then process this query metadata and present it back to you in the web application.

## Leveraging dbt meta & tags

Consider the example `dbt_project.yml` below, which implements a combination of dbt [meta fields](https://docs.getdbt.com/reference/resource-configs/meta) and [tags](https://docs.getdbt.com/reference/resource-configs/tags).

```yaml
# dbt_project.yml
name: 'select_dbt'
version: '1.0.0'
config-version: 2

models:
  +meta:
    # Set defaults for all models
    team: 'data'
    owner: "@james.fischer"
    business_owner: "sales_team"
    model_maturity: "production"
    refresh_frequency: "daily"
    domain: "customer_analytics"
    contains_pii: true
  select_dbt: # when setting meta for specific paths, make sure to namespace with the project name
    engineering:
      # applies to models in models/engineering folder
      +meta:
        team: 'engineering'
        business_owner: "engineering"
        individual_owner: "@james.fischer"
        model_maturity: "production"
        refresh_frequency: "hourly"
        domain: "equipment_analytics"
    marketing:
      # applies to models in models/marketing folder
      +meta:
        team: 'marketing'
        refresh_frequency: "hourly"
        contains_pii: "true"
    forecasts:
      # applies to models in models/forecast folder
      +meta:
        team: 'finance'
        refresh_frequency: "daily"
      +tags:
        - "sensitive"
        - "critical"

tests: # setting meta for singular tests (ones in .sql files)
  +meta:
    team: 'data' # configures all singular tests with team = 'data' by default
  select_dbt: # when setting meta for specific paths, make sure to namespace with the project name
    engineering:
      +meta:
        team: 'engineering'
    ...
```

The file structure for this dbt project would look something like what is shown below. All resources under the engineering directory would be get a `team` meta field added with the value `engineering`. Similarly, the one model in the `forecasts` directory would get a `team` field of `finance` added. It would also have tags: `["sensitive", "critical"]`.

```markdown
.
├── dbt_project.yml
├── packages.yml
└── models/
    ├── engineering/
    |   ├── syncs.sql
    |   └── ai.sql
    ├── marketing/
    |   ├── attribition.sql
    |   └── page_clicks.sql
    └── forecasts/
        └── forecast.sql
```

If you would like to set meta per model instead of per folder or "resource path", here is how you would do that:

```yaml
models:
  - name: customers
    description: 1 row per customer.
    config:
      meta:
        team: "data_team"
        business_owner: "sales_team"
        domain: "customer_analytics"
        model_maturity: "production"
        refresh_frequency: "daily"
        contains_pii: true
    columns:
      - name: customer_id
        description: The unique key.
        data_tests:
          - not_null
          - unique
        config:
          meta:
            data_type: "integer"
            is_primary_key: true
            is_sensitive: false
```

To expose dbt meta fields in SELECT, head to the dbt settings and specify which fields you'd like to filter on. No such specification is required for dbt tags.

![SELECT dbt meta keys settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/dbtMetaKeysSettings.png/width=2698,quality=75)

Once added, you can filter and slice by this metadata throught the UI!

[Video](https://storage.googleapis.com/prod_select_images/docs/dbt_custom_metadata.mp4)

> [!NOTE]
> Looking to do cost chargebacks or showbacks using dbt metadata? You can now flexibly allocate costs using dbt [meta fields](https://docs.getdbt.com/reference/resource-configs/meta) and [tags](https://docs.getdbt.com/reference/resource-configs/tags) you've specified using SELECT's [Usage Groups](https://select.dev/docs/usage-groups) feature!

## Accessing the dbt query metadata

If you would like to access the dbt query metadata created by our dbt package, you have two options.

### Option 1: Snowflake Query History

You can access the dbt query metadata by parsing the query comments/tags our package adds by querying the Snowflake query history. Here is an example SQL script you can run in your account:

```sql
with
dbt_queries as (
    select 
        start_time,
        query_text,
        execution_time/1000 as execution_time_s,
        try_parse_json(query_tag) as dbt_query_tag_meta,
        try_parse_json(regexp_substr(query_text, $$\/\*\s({"app":\s"dbt".*})\s\*\/$$, 1, 1, 'ie')) as dbt_comment_meta
    from snowflake.account_usage.query_history
    where
        dbt_query_tag_meta is not null
        and dbt_comment_meta is not null
        and start_time > current_date - 7
)
select
    start_time,
    query_text,
    execution_time_s,
    dbt_comment_meta['invocation_id']::string as invocation_id,
    dbt_comment_meta['node_id']::string as node_id,
    dbt_comment_meta['node_name']::string as node_name,
    dbt_comment_meta['node_resource_type']::string as resource_type,
    dbt_comment_meta['project_name']::string as project_name,
    dbt_comment_meta['target_name']::string as target_name,
    dbt_comment_meta['target_database']::string as target_database,
    dbt_comment_meta['target_schema']::string as target_schema,
    dbt_comment_meta['node_package_name']::string as package_name,
    dbt_comment_meta['materialized']::string as materialization,
    dbt_query_tag_meta['is_incremental']::boolean as is_incremental
from dbt_queries
limit 50
```

### Option 2: Via a data share

All data visible in the SELECT web application can be accessed through a secure data share, including a table named `dbt_queries` which includes all of the metadata (and more) shown above. See our [data sharing documentation](https://select.dev/docs/reference/usage-guide/accessing-select-data#data-sharing-feature-overview) for more details.

---

# Custom Workloads

URL: https://select.dev/docs/reference/integrations/custom-workloads

> Observe any Snowflake or BigQuery workload with query tags or comments.

## What are custom workloads?

SELECT supports custom workloads for Snowflake and BigQuery, but currently not for Databricks.

Many users leverage their database for unique workloads, such as custom ETL pipelines, machine learning applications, or analytics applications. Users need to monitor these workloads to understand their cost and performance over time. By adding query comments or [query tags](https://select.dev/posts/snowflake-query-tags) to the queries run by these workloads, SELECT users can benefit from enhanced monitoring capabilities compared to the default [Query Patterns](https://select.dev/docs/workloads#query-patterns) workload type.

## Feature Overview

The SELECT custom workload integration allows you to better understand and optimize spend associated with these unique workloads.

- Quickly identify expensive workloads by **name** or **application** (instead of by [Query Pattern](https://select.dev/docs/workloads#query-patterns)), and understand how their cost and performance vary over time
- Easily dissect your database spend across a variety of different dimensions such as environment, team and tenant_id
- Drill into the execution history of any workload and view query profile diagnostics for each run
- Receive tailored optimization recommendations to improve workload cost and performance

![SELECT custom workloads integration dashboard overview](https://storage.googleapis.com/prod_select_images/changelog/custom_workloads_20230517.gif)

The Custom Workloads feature works by adding metadata to your queries via SQL comments, [Snowflake query tags](https://select.dev/posts/snowflake-query-tags) or [BigQuery job labels](https://docs.cloud.google.com/bigquery/docs/adding-labels#job-label). SELECT will automatically detect this metadata and intuitively surface these workloads in the UI:

![SELECT Snowflake custom workloads example](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/a69a6a8d-28c4-4e96-5f0f-d856d9073700/width=3024,quality=75)

> [!NOTE]
> By default, SELECT categorizes compute spend into [Query Patterns](https://select.dev/docs/reference/using-select/workloads#query-patterns) based on the SQL fingerprint of the query. When users make significant changes to the structure of the SQL (add new fields, filter conditions, or join in new datasets), that SQL will become part of a new Query Pattern. This makes it more difficult to understand the impact of these changes on cost and performance, as you'll have to compare two different query patterns.
>
> With custom workloads, queries are grouped into a workload based on the query tag or comment. No matter what changes are made to the SQL, the queries will consistently be associated with the same workload specified in the query metadata, allowing you to confidently assess the impact of any changes made.

## Integrating with custom workloads

To support custom workload monitoring (i.e. queries run by custom tooling), SELECT utilizes query metadata. The query metadata is a JSON object, and can be set in either the [query's tag](https://select.dev/posts/snowflake-query-tags) or in a comment. The schema is as follows:

```sql
{
    # [REQUIRED]
    # identifiers
    "app": "generated_tables", # first grouping category
    "workload_id": "my_etl_job", # secondary grouping category

    # [STRONGLY RECOMMENDED]
    # Providing a run_id will enable SELECT to show runs of that workload over time or drill to individual runs. Without this, all queries assicated with a workload will be lumped togetehr.
    "run_id": "generate a UUID or timestamp"
    # [OPTIONAL]
    "select_schema_version": "1.0.0",
  
    # If a workload contains multiple queries, include a run_id to group them together
    # If run_id isn't present, each query is considered an individual run
    # This value might be a date but can be anything
    "run_id": "2020-01-01",

    # [OPTIONAL]
    # If specified, these will automatically appear in the UI as options you can filter & slice with
    "environment": "dev",
    "team": "data",

    # [OPTIONAL]
    # Custom tags - you can put whatever you want in here!
    # These fields can be used for slicing and filtering in the SELECT UI
    # after you've specified them in the UI under Settings -> Workloads -> In General workloads -> Custom workloads
    "tenant_id": "customer 1",
    "creator": "Hermione Granger",
    "description": "Processes the data",
    "link": "https://airflow.org.net/dag/my_etl_job",
    "git_sha": "abc123",

    # [OPTIONAL]
    # You can add complex data types, but we currently don't support using these in the UI
    "tags": ["critical", "sensitive"],
    "meta": {"task_key": "A1SF249X", "attempt_number": 1}
}
```

### Example query comments

Both multi-line and single-line comment syntax are supported. If using single-line comments, ensure that the final character in the comment is a closing curly brace and not a semi-colon.

Multi-line comment

```sql
select * from my_table
/*
{
  "app": "ingest",
  "workload_id": "facebook_ads"
}
*/
```

Single-line comment:

```sql
select * from my_table 
-- {"app": "ingest", "workload_id": "facebook_ads"} 
```

> [!WARNING]
> Your SQL statement must not have a semicolon before the query comment, or else the comment will not be sent to Snowflake:
>
> ```sql
> select * from my_table; -- MOVE THIS SEMICOLON AFTER THE COMMENT
> -- {"select_schema_version": "1.0.0", "app": "ingest", "workload_id": "facebook_ads"}
> ```

### Example query tag

Alternatively, in Snowflake, query tags can be used. The downside to using query tags is the need to both [set and unset](https://docs.snowflake.com/en/sql-reference/sql/alter-session) the query tag in the session, adding latency and additional complexity.

```sql
{"app": "ingest", "workload_id": "facebook_ads"} 
```

### Example BigQuery label

Alternatively, in BigQuery, job labels can be used. There is no set and unset concern like in Snowflake.

```bash
bq query --use_legacy_sql=false --quiet --nouse_cache \
    --label app:select_test --label workload_id:labels_test --label run_id:run_labels_1 \
    "SELECT 1 AS select_custom_labels_test"
```

### Example Python implementation using query comments

Here's an example implementation using the [Snowflake Python Connector](https://select.dev/posts/snowflake-python-connector):

```python
import json
import snowflake.connector

# Connect to Snowflake & create a cursor
conn = snowflake.connector.connect(
    user='<your_username>',
    password='<your_password>',
    account='<your_account>',
)
cursor = conn.cursor()

# Generate query comment
select_query_comment = {
  
  "app": "data_transformations",
  "workload_id": "customers",
  # Optional below here
  "select_schema_version": "1.0.0",
  "environment": "prod"
}
select_query_comment_str = f"\n\n/* {json.dumps(select_query_comment)} */"

## Execute query with comment
query = """
  create or replace table db.analytics.customers as (
    select *
    from raw_customers
    where
      not deleted
      and created_at > '2019-01-01'
  )
"""
cursor.execute(query + select_query_comment_str)
```

## How query metadata is reflected in the SELECT UI

### app and workload_id

The most important fields to set in the query comment/tag metadata are the `app` and `workload_id`. By default, they will show up in the UI in the main table as shown below.

![SELECT custom workload main fields](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/6e7c35d4-0102-4eb3-149a-8ebd8d59d300/width=3024,quality=75)

The example query below would be associated with the first row shown in the table above, regardless of which user or warehouse it ran on.

```sql
select * from my_table 
-- {"app": "SELECT_WEB_APP", "workload_id": "DbtQueriesProcessedQuery"} 
```

### When to use run_id

Sometimes a single workload will have multiple steps associated with it. For example, a data transformation workload may run 3 queries everytime it runs:

1. A query to create a temporary table
2. A query to merge data from the temporary table into the main table
3. A final query to delete the temporary table

To help track workloads like this, SELECT's custom workloads feature has first class support for a field called `run_id`. An example workload is shown below which issues 3 queries each time it runs. Notice how **each query has the same unique** **`run_id`** **associated with it**. The `run_id` can be anything, it just needs to be unique to each run. The example below uses Python's uuid module (`str(uuid.uuid4())`).

```sql
-- FIRST QUERY RUN
select
    c_birth_country,
    count(distinct c_customer_id)
from snowflake_sample_data.tpcds_sf10tcl.customer
group by 1
order by 2 desc
/* {"app": "app_name", "workload_id": "a_small_workload", "run_id": "3b80f989-95fb-4256-a918-83d66c5298d7", "environment": "prod", "team": "data-platform", "runner": "dagster", "priority": "high"} */

-- SECOND QUERY RUN
select
    d_year,
    d_moy,
    sum(ws_net_profit)
from
    snowflake_sample_data.tpcds_sf10tcl.web_sales
inner join snowflake_sample_data.tpcds_sf10tcl.date_dim
    on web_sales.ws_sold_date_sk = date_dim.d_date_sk
where
    d_year between 1995 and 2000
group by 1, 2
order by 1 desc, 2 asc;
/* {"app": "app_name", "workload_id": "a_small_workload", "run_id": "3b80f989-95fb-4256-a918-83d66c5298d7", "environment": "prod", "team": "data-platform", "runner": "dagster", "priority": "high"} */

-- THIRD QUERY RUN
select 3
/* {"app": "app_name", "workload_id": "a_small_workload", "run_id": "3b80f989-95fb-4256-a918-83d66c5298d7", "environment": "prod", "team": "data-platform", "runner": "dagster", "priority": "high"} */
```

In the SELECT custom workloads UI, you'll be able to see all queries associated with each workload run, and track run duration and cost over time (as opposed to individual query duration and cost over time).

![SELECT custom workload run_id](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/d4831267-98ec-4a2c-1773-927217602300/width=3024,quality=75)

### Additional Metadata

When coming up with your tagging structure, think about logical ways you may want to analyze your workloads. For example, if your pipeline has a retry mechanism built-in, you may want to add a `is_retry` field to your metadata so you can analyze how much retries are costing you across all workloads.

Other helpful examples of fields you may want to add could include:

- `tenant_id`: i.e. 'customer_a', 'customer_b'
- `environment`: i.e. 'dev', 'staging', 'prod', etc.
- `department`: i.e. 'advanced analytics', 'support', 'core', etc.
- `team`: i.e. 'data_engineering', 'analytics_engineering', 'solutions_architecture', etc.
- `owner_email`: i.e. 'ian@select.dev'
- `domain`: i.e. 'marketing', 'growth'

All of these fields can be exposed as filters in the SELECT UI, allowing you to slice and dice accordingly. Consider a workload which has a query metadata field called `runner`, which tracks the source system that emitted the query. You can see cost and performance data split by the `runner` metadata field directly in the UI:

![SELECT custom workload slice by custom field](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/91a3aa9a-d241-4020-c575-4be74e472000/width=3024,quality=75)

See the section below on "Leveraging custom metadata fields" for more information.

## Leveraging custom metadata fields

[Custom workloads](https://select.dev/docs/reference/integrations/custom-workloads) allow users to tag their queries with comments or [Snowflake query tags](https://select.dev/posts/snowflake-query-tags). Here’s an example query comment SELECT has for one of its custom workloads:

```sql
select ...
from queries
group by all
order by sum_query_cost desc nulls last
limit 1000

/* {
	
	"app": "SELECT_WEB_APP", 
	"workload_id": "DailyWorkloadPoPQuery", 
	# optional:
    "select_schema_version": "1.0.0", 
    "environment": "prod", 

	"tenant_id": "scwxhob-ad38017", 
    "referrer_base_url": "https://select.dev/app/scwxhob-ad38017/workloads",
	"is_admin_user": true,
	"user_id": "google-oauth2|115381118870474481840"
}*/
```

Custom workload fields like `app`, `workload_id` and `environment` are already available for filtering & slicing in the UI out of the box. To enable the same behaviour for other fields you specify in your custom workload query comment/tag, head to Settings → Workloads → in the General workloads section, click the Manage option for Custom workloads, and specify the keys you want enabled.

![SELECT custom workload meta keys settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/customWorkloadsMetaKeys.png/width=2698,quality=75)

Once added, you can filter and slice by this metadata throughout the UI!

[Video](https://storage.googleapis.com/prod_select_images/docs/custom_metadata.mp4)

> [!NOTE]
> Looking to do cost chargebacks or showbacks using the metadata you've added to your query? You can now flexibly allocate costs based on custom workload metadata using SELECT's [Usage Groups](https://select.dev/docs/reference/using-select/usage-groups) feature!

> [!WARNING]
> We currently do not support filtering/slicing on custom meta keys which contain complex data types like arrays or objects. If this is a requirement for you, please let us know and we will add this functionality.
>
> ```sql
> select * from credit_cards
>
> /* { 
>     "app": "generated_tables",
>     "workload_id": "my_etl_job",
>
>     "tags": ["critical", "sensitive"], 
>     "meta": {"task_key": "A1SF249X", "attempt_number": 1} } 
> */
> ```
>
> Note, you can still add this data to your query comment/tag and perform you own analysis on it. You just want be able to interact with that data in the SELECT UI as of today.

---

# Looker

URL: https://select.dev/docs/reference/integrations/looker

> Learn about SELECT's Looker integration

## Feature Overview

The Looker integration attributes query costs to dashboards, looks and explores. Use this feature to identify expensive workloads requiring attention, or identify opportunities to deprecate unused reports and save costs.

![SELECT Looker integration dashboard overview](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/11ecf564-c390-4c29-2ca4-07c650919600/width=3024,quality=75)

Use the query timeline view to understand dashboard performance bottlenecks or costly queries, or leverage the Cost and Performance tabs to see how dashboard usage statistics have been trending. On the Run History tab, you’ll see a complete history of all dashboard runs including which user triggered the run and any error messages they saw.

[Video](https://storage.googleapis.com/prod_select_images/changelog/looker_dashboard_20230508.mp4)

To do this, SELECT runs metadata extracts from Looker's System Activity Explore. To set up the integration, follow the steps below. 

## Step 1: Create a new user in Looker.

The email address can be absolutely anything.

![Create Looker user](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-create-user.png/width=2904,quality=75)

## Step 2: Create API keys for the user

Record the Client ID and Client Secret as these will be entered into SELECT.

![Create Looker API keys](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-create-api-keys.png/width=3044,quality=75)

## Step 3: Create a new Permission Set under Roles

We recommend using the name 'System Activity Viewer'.

![Create Looker permission set](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-create-permission-set.png/width=3052,quality=75)

Check the 'see_system_activity' box under permissions.

![See system activity](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-see-system-activity.png/width=2888,quality=75)

## Step 4: Create a new role

Again, we recommend calling it 'System Activity Viewer'.

![Create Looker role](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-create-role.png/width=3042,quality=75)

Assign the 'System Activity Viewer' permission set to the role, and grant the role to the user created in step 1.

![Assign permission set to role](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-assign-permission-set.png/width=3042,quality=75)

## Step 5: Input into SELECT

Head to SELECT's settings page, and then the Looker tab.

To find the Base URL value, go to the Platform -> API page in Looker's Admin settings, and see 'API Host URL'.

![Finding the Looker Base URL](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/a3c55e38-8b4f-42ab-aec3-8b14fe6fee00/width=1852,quality=75)

Next, enter the Client ID and Client Secret from Step 1, and finally click 'Update'. There will be a short delay while permissions are validated.

![Configure settings in SELECT](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/src/images/setup-guide/looker-select-settings.png/width=2462,quality=75)

## Step 6: All done!

SELECT will now runs metadata extracts from Looker and attribute your query costs to different Looker assets in the SELECT web application. Information will start appearing after the next metadata refresh.

---

# Mode

URL: https://select.dev/docs/reference/integrations/mode

> Learn about using MODE integration in SELECT

## Feature Overview

The Mode integration helps users keep track of dashboard costs and performance, identify unused reports, and flag reports that may be running too frequently.

[Video](https://storage.googleapis.com/prod_select_images/changelog/mode_integration_20230911.mp4)

Use the query timeline view to understand dashboard performance bottlenecks or costly queries, or leverage the Cost and Performance tabs to see how report usage statistics have been trending.

![SELECT Mode integration timeline view](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/mode_timeline_view_20230911.png/width=2974,quality=75)

On the Run History tab, you’ll see a complete history of all report runs including the users in your company who ran the report.

![SELECT Mode integration run history](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/mode_run_history_20230911.png/width=3732,quality=75)

To do this, SELECT queries the Mode API to retrieve report names and view counts. To set up the integration, follow the steps below as a Mode admin user.

## Step 1: Create a workspace API Token

> [!NOTE]
> API tokens can only be created by a Mode user with Admin privileges.

Click your name in the left-hand sidebar, and then Workspace settings.

![SELECT Mode integration setup part 1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/modeAPITokenSetup1.png/width=1856,quality=75)

Next, head to API Keys, and finally the API tab. Click the cog, and then 'Create new API token'.

![SELECT Mode integration setup part 2](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/modeAPITokenSetup2.png/width=2584,quality=75)

Specify a display name and set the token expiration to 365 days. We'll reach out before it expires, but to avoid any potential disruption, make a note to update it before it happens.

![SELECT Mode integration setup part 3](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/modeAPITokenSetup3.png/width=2586,quality=75)

Record the Token ID and Token Secret to be entered into SELECT.

![SELECT Mode integration setup part 4](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/modeAPITokenSetup4.png/width=2558,quality=75)

## Step 2: Find your Workspace identifier

In addition to the Token ID and Secret, you'll need the Mode Workspace identifier. This identifier can be retrieved from the Mode URL. It is usually the bit directly after `app.mode.com/`.

Consider the following Mode URLs:

- [https://app.mode.com/select_development/spaces/060cf76024a7](https://app.mode.com/select_development/spaces/060cf76024a7)
- [https://app.mode.com/organizations/select_development](https://app.mode.com/organizations/select_development)
- [https://app.mode.com/select_development/reports/239239503031](https://app.mode.com/select_development/reports/239239503031)

The Workspace identifier here is `select_development`.

## Step 3: Input into SELECT

In SELECT's settings under the Mode tab, enter the Workspace identifier, Token ID and Secret, and click 'Update'. There will be a short delay while the credentials are validated.

## Step 4: All done!

SELECT will now run metadata extraction from Mode and attribute queries to Mode reports in the SELECT web application. Information will start appearing after the next metadata refresh.

## Upcoming Improvements

In future releases, we'll give SELECT customers the ability to automatically notify business users of upcoming deprecations to unused reports that are still running on a schedule and driving costs. We'll also bring in the associated Mode charges for each report since Mode charges users based on the amount of data returned in the reports.

---

# Hex

URL: https://select.dev/docs/reference/integrations/hex

> Learn about SELECT's HEX integration

## Feature overview

The Hex integration helps users keep track of Hex project costs and performance, and identify unused projects.

![Hex integration](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/51679f4c-cab2-4e12-c723-f6aa271c3700/width=3456,quality=75)

Use the query timeline view to understand project query durations and costs, or leverage the Cost and Performance tabs to see how project usage statistics have been trending. On the Run History tab, you’ll see a complete history of all project runs.

![Hex integration](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/e66625db-9522-4cd2-e3c8-ad9b695b6400/width=3456,quality=75)

SELECT doesn't require any access to your Hex account for this feature, if you're using Hex, you'll get to use the integration's functionality right away.

---

# Periscope

URL: https://select.dev/docs/reference/integrations/periscope

> Learn about using Periscope within SELCT

## Feature overview

The Periscope integration helps users keep track of Periscope dashboard costs and performance.

![Periscope integration](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/periscope_overview_20231012.png/width=2978,quality=75)

Dive into dashboard statistics, cost & performance trends, and understand which charts and users are driving significant usage.

![Periscope integration](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/periscope_dashboard_20231012.png/width=2974,quality=75)

## Setup

SELECT doesn't require any access to your Periscope account for this feature, if you're using Periscope, you'll get to use the integration's functionality right away.

In order to generate hyperlinks to each dashboard in your Periscope environment, we require your Periscope account ID to be entered on the settings page.

![Periscope integration settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/periscope_settings.png/width=2454,quality=75)

## Planned enhancements

In future releases, we'll improve the Periscope integration by showing you a breakdown of users using each dashboard, along with the cost per dashboard chart.

---

# Fivetran

URL: https://select.dev/docs/reference/integrations/fivetran

> Learn about using SELECT's Fivetran integration

## Feature overview

The Fivetran integration attributes query costs to tables loaded by Fivetran. Use this feature to identify your most expensive tables and discover opportunities to reduce sync frequency to [lower costs](https://select.dev/posts/snowflake-cost-optimization).

![SELECT Fivetran integration dashboard overview part 1](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/fivetran_overview_1_20230718.png/width=2992,quality=75)

Dive into a variety of historical cost and performance statistics for each Fivetran workload, and see each historical query run.

![SELECT Fivetran integration dashboard overview part 2](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/changelog/fivetran_overview_2_20230718.png/width=2992,quality=75)

To support this integration, SELECT uses the provided Fivetran user name to query the Snowflake query history and model out the costs and performance of the relevant queries. To set up the integration, follow the steps below.

## Step 1: Input the Fivetran Snowflake Username into SELECT

In SELECT's settings under the Fivetran tab, enter the Snowflake username that corresponds to the Fivetran User. For example, this might be called `FIVETRAN`, `FIVETRAN_USER`, `LOADER`, etc. If you have multiple user names associated with your Fivetran loads, you can select more than one.

Once that has been inputted, select 'Update'. There will be a short delay while permissions are validated.

![SELECT Fivetran integration settings](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/docs/fivetran_settings.png/width=2302,quality=75)

## Step 2: All done!

SELECT will now correctly attribute your query costs to different Fivetran Table assets in the SELECT web application. Information will start appearing after the next metadata refresh (typically the next morning).

---

# Tableau

URL: https://select.dev/docs/reference/integrations/tableau

> Learn how to configure SELECT's Tableau Integration

> [!NOTE]
> **Beta Notice**  
> This integration is currently in **beta**; we appreciate your participation. After you connect SELECT to your Tableau deployment in the Settings page, we’ll enable the new **Tableau Assets** pages in the main UI and follow up with details on the next steps. If you have any questions or feedback, we’d love to hear from you.

SELECT can connect to your Tableau deployment to retrieve metadata about Tableau assets such as workbooks, data sources, and users. These assets are displayed within SELECT and, when possible, linked to other data in your account. For example, Tableau workbooks and data sources can appear in SELECT's lineage view, connected to the upstream tables they query.

> [!NOTE]
> **Note:** Snowflake costs for Tableau assets are not currently displayed but will be supported in a future release.

As part of the integration, SELECT automatically configures views and monitors to help you identify and be alerted about:

- Users who have not logged in recently
- Workbooks that have not been viewed recently
- Data sources that are not linked to a workbook

## Tableau Data Retrieved

SELECT only collects metadata about your Tableau assets, including custom SQL embedded in data sources. We do not retrieve business data from extracts or make any changes to your Tableau deployment.

The following metadata is retrieved and stored to support visualization in SELECT:

- **Workbooks** – Name, description, project, views, owner, tags, created/updated dates. Workbook content is not retrieved.
- **Data Sources** – Name, description, project, owner, tags, created/updated dates.
- **Projects** – Name, description.
- **Users** – Name, email, license type, group memberships, last login.
- **Tableau Cloud only** – View and usage statistics (which users accessed which workbooks).

For lineage purposes, we retrieve and parse the SQL used in custom SQL workbooks and data sources to identify upstream tables. The SQL text itself is not stored in our systems.

## Step 1: Enable Metadata API on Tableau Server

> [!NOTE]
> Step 1 applies **only** to Tableau Server users. Tableau Cloud users can skip it.

The SELECT integration uses the following Tableau APIs:

- **Tableau REST API**
- **Tableau Metadata API**
- **Tableau VizQL Data Service (VDS)**

The REST API and VDS are always enabled on every Tableau deployment.  
The Metadata API is automatically enabled for Tableau Cloud, but must be **manually** enabled for Tableau Server.  
Follow [Tableau’s official instructions](https://help.tableau.com/current/api/metadata_api/en-us/docs/meta_api_start.html#enable) to enable it.

## Step 2: Generate a Personal Access Token (PAT)

A Tableau **Site Administrator** must create a Personal Access Token by following [Tableau’s official instructions](https://help.tableau.com/current/pro/desktop/en-us/useracct.htm#create-a-personal-access-token) or these steps:

- Log in to Tableau.
- Open **My Account Settings** from the top-right menu.

![Get Tableau API Token for SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/3cf747c3b8b2746906a5102cf598813226ea1391-213x224.png)

- In the **Personal Access Tokens** section, create a new token.
   - If you don’t see this section, enable PATs in your site settings using [these instructions](https://help.tableau.com/current/online/en-gb/security_personal_access_tokens.htm#site-settings-for-personal-access-tokens).
- Name the token (e.g., `SELECT Integration`).

**Important:** Save both the **Token Name** and **Token Secret**. The secret is shown **only** at creation and is required for the next step.

## Step 3: Set up the Integration on SELECT

You will need a user with the **SELECT Organization Admin** role to complete this step.

- Go to the **Settings** page SELECT

![SELECT settings page](https://cdn.sanity.io/images/wn0ylp2c/production/29073d5a1d2f5b3a767cad2c083ee7356e4010ac-271x131.png)

- In the left-hand menu, look for **Tableau**.
   - If you don’t see it, confirm that you have the **SELECT Organization Admin** role.
   - During the beta period, this feature is only available to beta customers. If it’s missing, contact us to enable it.

![Tableau integration in SELECT](https://cdn.sanity.io/images/wn0ylp2c/production/ec43fdbea4086582d00ebd3ddb55016dbf92e90e-263x703.png)

- Enter your Tableau connection details:
   - **Tableau Cloud** – Your URL will look like:  
`https://us-east-1.online.tableau.com/#/site/companyname` 
      - **Server URL**: First part of the address (e.g., `https://us-east-1.online.tableau.com/`)
      - **Site Name**: Last part of the address (e.g., `companyname`)
   - **Tableau Server** – Use the URL you normally use to access Tableau (e.g., `https://tableau.mycompany.com`).
      - Most companies have one site named **default**.
- Save the form. SELECT will attempt to validate your credentials by connecting to your Tableau environment.
- If the connection fails, you’ll see an error message. If you believe your details are correct, contact us for assistance.

## Data Sync

The integration fetches metadata from your Tableau connection every 24 hours during the same time SELECT fetches metadata from your Snowflake accounts.  Once data is synced, you’ll see Tableau insights in the navigation bar:

- **Tableau → Workbooks** – Workbooks with cost attribution.
- **Tableau → Data Sources** – Published data sources and usage.
- **Tableau → Users** – User activity and associated costs.

---

# Opsgenie

URL: https://select.dev/docs/reference/integrations/opsgenie

> Learn about using Opsgenie in SELECT

## Feature Overview

The Opsgenie integration lets users forward messages sent by SELECT [Monitors](https://select.dev/docs/monitors/overview) (such as Spend Threshold alerts) directly to Opsgenie, allowing organizations to:

- Benefit from the advanced routing capabilities from Opsgenie to dynamically forward messages to different teams and individuals, including on-call policies;
- Use Opsgenie to acknowledge or escalate alerts based on their criticality;
- Take advantage of Opsgenie reporting and analytics features to gain insights on how often alerts are triggered and how long it takes teams to acknowledge and resolve them.

Our Opsgenie integration works by leveraging our capability to deliver Monitor alerts by email and Opsgenie's native email integration.

## Step 1: Enable the email integration on Opsgenie

The first step is to enable the Opsgenie account to receive notifications by email. You can do that by following [Opsgenie's documentation](https://support.atlassian.com/opsgenie/docs/integrate-your-email-with-opsgenie/). No special settings are needed. The email integration should look similar to the one in the image below.

![Opsgenie email integration form](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/293db2b4-c778-449e-f30b-4de69ad0b700/width=3024,quality=75)

Don't forget to enable new Opsgenie integrations by pressing the blue "Turn on integration" button on the page above.

> [!NOTE]
> By default, Opsgenie will generate a new email address with a unique random ID. We recommend changing it to a more descriptive address, which will help you distinguish different email integrations in the future.

## Step 2: Add Opsgenie integration email address as a Monitor destination

The next step is to take the new email address that Opsgenie has created for the integration and add it as a Monitor destination here in SELECT, as seen below. More detailed instructions can be found in our [Monitors documentation](https://select.dev/docs/monitors).

![SELECT destination created](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/184c2714-e845-4345-6fbb-3825f2cbdd00/width=3024,quality=75)

## Step 3: Set up Monitors using the new destination

Once the destination is set up on SELECT, all that is left is to set up Monitors using the new destination.

![SELECT Monitor created](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/611dead9-4426-4f40-4900-1d4a7dad9100/width=3024,quality=75)

## Step 4: All done!

That's it! Now, Monitor alerts will be sent to Opsgenie and benefit from their forwarding and escalation rules, on-call policies, and schedules.

Opsgenie automatically converts email messages to plain text, making the alerts sent by our Monitors look like the one below.

![Opsgenie Raw Message](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/f4dc4ca3-5caa-4179-a960-fb0b710c2700/width=3024,quality=75)

However, you can always see the original message with our rich formatting and charts by accessing a file that Opsgenie automatically attaches to the incoming alert. To find the attachment, follow the red arrow shown in the image above.

![Opsgenie Raw Message](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/79c76998-72ea-4965-2269-c1061a576a00/width=3024,quality=75)

Don't hesitate to contact us if you need help setting up this or other integrations.

---

# Managing Usage Groups via Terraform

URL: https://select.dev/docs/reference/api-reference/managing-usage-groups-terraform

> Learn how to manage your Usage Groups via Infrastructure as code with Terraform

SELECT offers both an [API](https://api-docs.select.dev/v2/introduction) and a [Terraform provider](https://registry.terraform.io/providers/get-select/select/latest/docs), which can be utilized to manage [usage groups](https://select.dev/docs/reference/using-select/usage-groups) within your organization.

This guide presents a way to migrate from usage groups configured in the SELECT UI to managing those resources in Terraform. We provide a Python script which will generate Terraform resource blocks based on what is already configured in the SELECT UI, simplifying this process greatly.

Here's an overview of the process:

1. Obtain an API key from the SELECT UI
2. Download the helper script and set up its dependencies
3. Run the script with your API key to generate Terraform configuration for all usage groups in your organization
4. (Optional) Move the generated Terraform resources into your existing Terraform repository
5. (IMPORTANT) Move the API key to [secure storage](https://developer.hashicorp.com/terraform/tutorials/configuration-language/sensitive-variables)
6. Run the generated `import.sh` which will pull all your usage groups into your Terraform state.
7. Iteratively execute `terraform plan` and adjust the generated resources until they match what's configured in SELECT

## Step by step guide

### Prerequisites

Before getting started, you'll need:

- **Python 3.6+**
- **Terraform CLI** - [Download here](https://developer.hashicorp.com/terraform/install)

### 1. Obtain an API Key from the SELECT UI

Please see the [API quickstart](https://api-docs.select.dev/v2/introduction) for guidance on how to obtain this from the SELECT UI. The API key you generate will need read/write access to any resources you want to manage in Terraform. For this guide, at a minimum, the usage group and the usage group set read/write access are required. 

### 2. Download and setup the script

```bash
# Clone the repository or download the migration folder
git clone https://github.com/get-select/terraform-provider-select.git
cd terraform-provider-select/scripts/usage_groups_migration

# setup and activate a virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

```

### 3. Generate terraform configuration

Replace `YOUR_API_TOKEN` and `YOUR_ORG_ID` with your actual credentials:

```bash
python generate_usage_group_resources.py --token YOUR_API_TOKEN --org-id YOUR_ORG_ID
```

This creates a `select_usage_groups` directory with:

- `main.tf` - Provider configuration
- `{usage_group_set}.tf` - One file per usage group set
- `import.sh` - Script to import existing resources

### 4. Move the generated Terraform resource files

If you would like to manage SELECT resources alongside your other infrastructure, please move this directory into the corresponding repository now.

If you are setting up a new repository for managing SELECT resources, you can move this directory to the new repository now.

Once you've moved the generated resource files, you can remove the script or the entire terraform repository if you cloned it earlier.

### 5. Secure your API key

⚠️ **IMPORTANT**: The generated `main.tf` contains your API key in plaintext.

If you already manage resources in Terraform it's likely that you have a strategy for securing sensitive variables. Otherwise, a simple way to secure this is to use environment variables.

```bash
cd select_usage_groups

# Move API key to environment variable
export TF_VAR_select_api_key="YOUR_API_TOKEN"

# Edit main.tf to use the variable instead of hardcoded key
# Change: api_key = "YOUR_API_TOKEN"
# To: api_key = var.select_api_key

```

There are alternatives and Hashicorp has extensive [documentation](https://developer.hashicorp.com/terraform/tutorials/configuration-language/sensitive-variables) on this topic.

### 6. Initialize and import

The python script generated a command to import the resources into your Terraform state.

```bash
# Initialize terraform
terraform init

# Import existing resources into terraform state
# You may need to make the script executable first: chmod +x ./import.sh
./import.sh

# Check for configuration drift
terraform plan
```

### 7. Fix any configuration drift

If `terraform plan` shows differences, update your `.tf` files to match your existing resources until the plan shows no changes.

```bash
# After making changes, verify
terraform plan
# Should show: "No changes. Your infrastructure matches the configuration."

```

## Examples

  
The script above will generate a module per snowflake account or organization that API key used has access to, and create a file per Usage Group Set.

To see some examples of what the Terraform resources for Usage Groups looks like, please visit the Terraform provider [docs](https://registry.terraform.io/providers/get-select/select/latest/docs/resources/usage_group#example-usage).

---

# Working with the Usage Groups API

URL: https://select.dev/docs/reference/api-reference/working-with-the-usage-groups-api

> A full guide to programmatically creating and managing Usage Groups in Snowflake using the SELECT Usage Groups API, with examples, best practices, and troubleshooting tips.

While we recommend managing Usage Groups on an ongoing basis via our official Terraform provider, we recognize that not all teams use Terraform or there may be use cases where you want to programmatically create Usage Groups as a one-off.

For example, imagine you want to create a Usage Group per department using a series of object tags you have stored in Snowflake. Alternatively, imagine you have a customer-facing product built on top of Snowflake and want to create a Usage Group for each customer.

The guide below shows how you could accomplish that using our official Usage Groups API.

## **Prerequisites**

### **Step 1: Create an API Key**

To interact with the SELECT API, you'll need to generate an API key and get your organization ID from the SELECT app. Follow the [API quickstart instructions](https://api-docs.select.dev/v2/introduction) to get started.

Your API key must have the `usage_groups:write` scope and access to the appropriate Snowflake account(s) or organization(s).

## **Exploring Filter Options**

**🎯 Before diving into the API, we strongly recommend exploring filter options through the SELECT UI or Terraform documentation:**

### **Option 1: Use the SELECT UI (Recommended)**

1. Navigate to your Usage Groups page in the SELECT app
2. Click "Create Usage Group" or edit an existing one
3. Build your filter expression using the interactive filter builder
4. Click the "Interactive" toggle and switch to "JSON" mode
5. Copy the `filter_expression` field - this is exactly what you'll use in the API!

### **Option 2: Browse Terraform Examples**

Visit our [Terraform provider documentation](https://registry.terraform.io/providers/get-select/select/latest/docs/resources/usage_group) for comprehensive filter examples and available dimensions.

### **Option 3: Check Available Dimensions**

The available filter dimensions depend on whether your usage group set is scoped to an account or organization:

**Account-Scoped Dimensions** (most common):

- `warehouse_name`, `user_name`, `role_name`
- `warehouse_tags`, `user_tags`, `role_tags`, `database_tags`
- `resource_type`, `database_schema_name`
- And many more...

**Organization-Scoped Dimensions**:

- `account_name`, `region`, `service_level`, `usage_type`
- `account_name_and_usage_type`, `account_name_and_warehouse`

## **Complete API Workflow**

### **Step 1: Test Your API Credentials**

```python
import requests
import json

organization_id = "your_organization_id_here"  # Get from SELECT app
api_key = "sl_your_api_key_here"  # Generated from SELECT app

url = f"https://api.select.dev/api/{organization_id}/usage-group-sets"
headers = {"x-api-key": api_key}

# Test credentials by listing existing usage group sets
response = requests.get(url, headers=headers)
if response.status_code == 200:
    print("✅ API credentials working!")
    print("Existing usage group sets:", response.json())
else:
    print(f"❌ API Error: {response.status_code} - {response.text}")
    exit(1)

```

### **Step 2: Create a Usage Group Set (if needed)**

```python
# Create a usage group set scoped to a Snowflake account
usage_group_set_payload = {
    "name": "Department Cost Allocation",
    "order": 1,
    "snowflake_account_uuid": "your_snowflake_account_uuid_here"
    # OR use: "snowflake_organization_name": "your_org_name"
}

response = requests.post(url, headers=headers, json=usage_group_set_payload)
if response.status_code == 200:
    usage_group_set = response.json()
    usage_group_set_id = usage_group_set["id"]
    print(f"✅ Created usage group set: {usage_group_set_id}")
else:
    print(f"❌ Error creating usage group set: {response.text}")
    # You might want to use an existing set instead
    usage_group_set_id = "existing_set_id_here"

```

### **Step 3: Design Your Usage Group Mappings**

Create your mappings based on your business needs:

```python
# Example 1: Department-based allocation using warehouse names
department_mappings = [
    {
        "name": "Finance Department",
        "warehouse_names": ["FINANCE_WH", "FINANCE_ANALYTICS_WH"],
        "budget": 5000,
        "order": 1
    },
    {
        "name": "Marketing Department",
        "warehouse_names": ["MARKETING_WH", "CAMPAIGN_WH"],
        "budget": 3000,
        "order": 2
    }
]

# Example 2: Customer-based allocation using database tags
# (You'd get this from your customer database/CRM)
customer_mappings = [
    {
        "name": "Customer: Acme Corp",
        "database_tag_key": "customer_id",
        "database_tag_value": "acme_corp",
        "budget": 2000,
        "order": 1
    },
    {
        "name": "Customer: Beta Inc",
        "database_tag_key": "customer_id",
        "database_tag_value": "beta_inc",
        "budget": 1500,
        "order": 2
    }
]

```

### **Step 4: Create Filter Expressions**

**💡 Pro Tip**: Build your first filter in the SELECT UI, then copy the JSON structure!

```python
def create_warehouse_filter(warehouse_names):
    """Create a filter for specific warehouse names"""
    return {
        "operator": "or",
        "filters": [
            {
                "field": "warehouse_name",
                "operator": "in",
                "values": warehouse_names
            }
        ]
    }

def create_database_tag_filter(tag_key, tag_value):
    """Create a filter for database tags"""
    return {
        "operator": "or",
        "filters": [
            {
                "field": "database_tags",
                "operator": "array_contains",
                # Note: The format for filter on tags is
                # 'TagKey: TagValue'
                "values": [f"{tag_key}: {tag_value}"]
            }
        ]
    }

def create_user_pattern_filter(user_patterns):
    """Create a filter for user name patterns"""
    return {
        "operator": "or",
        "filters": [
            {
                "field": "user_name",
                "operator": "like",
                "values": user_patterns
            }
        ]
    }

# Example: Complex filter combining multiple criteria
def create_complex_filter():
    """Example of a more complex filter"""
    return {
        "operator": "and",
        "filters": [
            {
                "operator": "or",
                "filters": [
                    {
                        "field": "warehouse_name",
                        "operator": "in",
                        "values": ["ANALYTICS_WH", "REPORTING_WH"]
                    }
                ]
            },
            {
                "operator": "or",
                "filters": [
                    {
                        "field": "user_name",
                        "operator": "like",
                        "values": ["analytics_%", "report_%"]
                    }
                ]
            }
        ]
    }

```

### **Step 5: Create Usage Groups Programmatically**

```python
usage_groups_url = f"{url}/{usage_group_set_id}/usage-groups"

def create_usage_group(name, filter_expression, budget, order):
    """Helper function to create a usage group"""
    payload = {
        "name": name,
        "budget": budget,
        "order": order,
        "usage_group_set_id": usage_group_set_id,
        "filter_expression": filter_expression
    }

    response = requests.post(usage_groups_url, headers=headers, json=payload)

    if response.status_code == 200:
        print(f"✅ Created usage group: {name}")
        return response.json()
    else:
        print(f"❌ Error creating {name}: {response.text}")
        return None

# Create department-based usage groups
for dept in department_mappings:
    filter_expr = create_warehouse_filter(dept["warehouse_names"])
    create_usage_group(
        name=dept["name"],
        filter_expression=filter_expr,
        budget=dept["budget"],
        order=dept["order"]
    )

# Or create customer-based usage groups
for customer in customer_mappings:
    filter_expr = create_database_tag_filter(
        customer["database_tag_key"],
        customer["database_tag_value"]
    )
    create_usage_group(
        name=customer["name"],
        filter_expression=filter_expr,
        budget=customer["budget"],
        order=customer["order"]
    )

```

### **Step 6: Managing Existing Usage Groups**

Check for existing usage groups before creating new ones:

```python
def get_existing_usage_groups():
    """Get all existing usage groups in the set"""
    response = requests.get(usage_groups_url, headers=headers)
    if response.status_code == 200:
        return {group["name"]: group for group in response.json()}
    return {}

def smart_create_usage_groups(mappings, filter_func):
    """Only create usage groups that don't already exist"""
    existing = get_existing_usage_groups()

    for mapping in mappings:
        if mapping["name"] in existing:
            print(f"⏭️  Usage group '{mapping['name']}' already exists, skipping")
            continue

        filter_expr = filter_func(mapping)
        create_usage_group(
            name=mapping["name"],
            filter_expression=filter_expr,
            budget=mapping["budget"],
            order=mapping["order"]
        )

# Usage
smart_create_usage_groups(
    department_mappings,
    lambda dept: create_warehouse_filter(dept["warehouse_names"])
)

```

## **Real-World Examples**

### **Example 1: Multi-Tenant SaaS Application**

```python
# Create usage groups for each customer tenant
def setup_customer_usage_groups(customers_data):
    """
    customers_data: List of dicts with customer info from your database
    Example: [{"name": "Acme Corp", "schema_prefix": "acme_", "budget": 2000}]
    """
    for customer in customers_data:
        filter_expr = {
            "operator": "or",
            "filters": [
                {
                    "field": "database_schema_name",
                    "operator": "like",
                    "values": [f"%{customer['schema_prefix']}%"]
                }
            ]
        }

        create_usage_group(
            name=f"Customer: {customer['name']}",
            filter_expression=filter_expr,
            budget=customer.get('budget'),
            order=customer.get('order', 999)
        )

# Get customer data from your systems
customers = get_customers_from_database()  # Your function
setup_customer_usage_groups(customers)

```

### **Example 2: Environment-Based Cost Tracking**

```python
environments = [
    {"name": "Production", "pattern": "%_prod_%", "budget": 10000, "order": 1},
    {"name": "Staging", "pattern": "%_staging_%", "budget": 3000, "order": 2},
    {"name": "Development", "pattern": "%_dev_%", "budget": 2000, "order": 3}
]

for env in environments:
    filter_expr = {
        "operator": "or",
        "filters": [
            {
                "field": "warehouse_name",
                "operator": "like",
                "values": [env["pattern"]]
            }
        ]
    }

    create_usage_group(
        name=env["name"],
        filter_expression=filter_expr,
        budget=env["budget"],
        order=env["order"]
    )

```

## **Filter Discovery Tips**

### **🔍 Finding the Right Filter Dimensions**

1. **Start with the UI**: The interactive filter builder shows all available dimensions for your specific setup
2. **Use Terraform docs**: Browse real-world examples at [registry.terraform.io/providers/get-select/select](https://registry.terraform.io/providers/get-select/select/latest/docs)
3. **Check your data**: Look at your Snowflake `INFORMATION_SCHEMA` to understand your warehouse names, user patterns, etc.
4. **Test incrementally**: Start with simple filters and add complexity as needed

### **🎛️ Common Filter Patterns**

```python
# Pattern 1: Exact matches
{"field": "warehouse_name", "operator": "in", "values": ["WH1", "WH2"]}

# Pattern 2: Pattern matching
{"field": "user_name", "operator": "like", "values": ["analytics_%", "%_admin"]}

# Pattern 3: Tag-based filtering
{"field": "warehouse_tags", "operator": "array_contains", "values": ["team: analytics"]}

# Pattern 4: Multiple criteria (AND)
{
    "operator": "and",
    "filters": [
        {"field": "warehouse_name", "operator": "in", "values": ["PROD_WH"]},
        {"field": "user_name", "operator": "like", "values": ["prod_%"]}
    ]
}

# Pattern 5: Multiple criteria (OR)
{
    "operator": "or",
    "filters": [
        {"field": "warehouse_name", "operator": "in", "values": ["DEV_WH"]},
        {"field": "warehouse_name", "operator": "in", "values": ["TEST_WH"]}
    ]
}

```

## **Best Practices**

1. **🎯 Start Simple**: Begin with basic warehouse or user-based filters before adding complexity
2. **🧪 Test in UI First**: Always prototype your filters in the SELECT interface before coding
3. **📖 Use Terraform Docs**: Reference our provider docs for proven filter patterns
4. **🔄 Handle Existing Data**: Check for existing usage groups to avoid duplicates
5. **📊 Monitor Results**: Use the SELECT dashboard to verify your usage groups are working as expected
6. **⚡ Order Matters**: Lower order values take precedence when filter expressions overlap

## **Troubleshooting**

**❌ "Filter expression validation failed"**

- Copy a working filter from the UI and modify it incrementally
- Check that field names match available dimensions for your scope (account vs organization)

**❌ "Usage group name already exists"**

- Use the "get existing usage groups" pattern above to check before creating

**❌ "API key doesn't have access"**

- Verify your API key has `usage_groups:write` scope
- Ensure the key has access to the specific Snowflake account/organization
- Organization access grants access to all accounts as well

For more complex scenarios, consider using our [Terraform provider](https://registry.terraform.io/providers/get-select/select/latest/docs) which provides a more robust way to manage usage groups as infrastructure-as-code.

---

# Automated Savings

URL: https://select.dev/docs/reference/faq/automated-savings

> Find out how our Automated Savings feature works

- [How do I assign CAN_MANAGE to all my compute resources in Databricks?](https://select.dev/docs/reference/faq/automated-savings/how-do-i-assign-can_manage-to-all-my-compute-resources-in-databricks.md): An automated way to provision the required access needed for Databricks Automated Savings
- [What Snowflake permissions are required for Automated Savings?](https://select.dev/docs/reference/faq/automated-savings/automated-savings-permissions.md): The single Snowflake grant needed to enable Automated Savings, which is applied automatically by the setup script.
- [How do I enable Automated Savings?](https://select.dev/docs/reference/faq/automated-savings/how-do-i-enable-automated-savings.md): Steps to turn on Automated Savings for one or multiple warehouses from the Warehouses tab.
- [Can Automated Savings negatively impact performance?](https://select.dev/docs/reference/faq/automated-savings/can-automated-savings-impact-performance.md): Explains that Automated Savings has shown no performance drawbacks across SELECT's customer base, since it never resizes warehouses or reduces cluster counts.
- [How does automated savings work?](https://select.dev/docs/reference/faq/automated-savings/how-does-the-automated-savings-feature-work.md): Describes how Automated Savings monitors warehouses in real time and makes short-lived configuration changes to cut cost without affecting query performance.
- [How can I see my Automated Savings in Dollars?](https://select.dev/docs/reference/faq/automated-savings/how-can-i-see-my-automated-savings-in-dollars.md): Shows how to view realized and estimated dollar savings on the warehouse chart, including how to read the total and projected annualized savings figures.

---

# How do I assign CAN_MANAGE to all my compute resources in Databricks?

URL: https://select.dev/docs/reference/faq/automated-savings/how-do-i-assign-can_manage-to-all-my-compute-resources-in-databricks

> An automated way to provision the required access needed for Databricks Automated Savings

This script helps provision the required access in an automated manner.

```python
# Dependencies: pip install databricks-sdk

# ── Config — edit these before running ───────────────────────────────────────

WORKSPACE_URL = "<https://dbc-xxxxx.cloud.databricks.com>"
ACCOUNT_ID = "<databricks-account-id>"
SERVICE_PRINCIPAL_NAME = "<service-principal-name>"

# ─────────────────────────────────────────────────────────────────────────────

from urllib.parse import urlparse

from databricks.sdk import AccountClient
from databricks.sdk.service.compute import ClusterSource
from databricks.sdk.service.iam import (
    AccessControlRequest,
    PermissionLevel,
    WorkspacePermission,
)


def main():
    parsed = urlparse(WORKSPACE_URL)
    workspace_host = f"{parsed.scheme}://{parsed.netloc}"
    account_host = detect_account_host(workspace_host)
    account_client = AccountClient(
        host=account_host, account_id=ACCOUNT_ID, auth_type="external-browser"
    )

    ws_info = find_current_workspace(account_client, workspace_host)
    region = ws_info.location or ws_info.aws_region
    print(f"Workspace : {ws_info.workspace_name}")
    print(f"Region    : {region}")

    sp = get_service_principal(account_client, SERVICE_PRINCIPAL_NAME)

    workspaces_in_region = get_workspaces_in_region(account_client, region)
    print(f"\n{len(workspaces_in_region)} workspace(s) in region '{region}'")

    for ws in workspaces_in_region:
        print(f"\n  Workspace: {ws.workspace_name}")
        ws_client = account_client.get_workspace_client(ws)

        ensure_sp_assigned_to_workspace(account_client, ws_client, ws, sp)

        print(f"  Granting CAN_MANAGE to SP on jobs in {ws.workspace_name}")
        for job in ws_client.jobs.list():
            grant_permission(
                ws_client, "jobs", job.job_id, sp, PermissionLevel.CAN_MANAGE
            )

        print(f"  Granting CAN_MANAGE to SP on clusters in {ws.workspace_name}")
        for cluster in ws_client.clusters.list():
            if cluster.cluster_source not in (ClusterSource.API, ClusterSource.UI):
                continue
            grant_permission(
                ws_client,
                "clusters",
                cluster.cluster_id,
                sp,
                PermissionLevel.CAN_MANAGE,
            )

        print(f"  Granting CAN_MANAGE to SP on warehouses in {ws.workspace_name}")
        for wh in ws_client.warehouses.list():
            grant_permission(
                ws_client, "warehouses", wh.id, sp, PermissionLevel.CAN_MANAGE
            )

    print(
        "\n── Done ─────────────────────────────────────────────────────────────────────"
    )
    print(f"  Account ID   : {ACCOUNT_ID}")
    print(f"  SP Client ID : {sp.application_id}")
    print(
        "─────────────────────────────────────────────────────────────────────────────"
    )


# ── Helpers ───────────────────────────────────────────────────────────────────


def detect_account_host(workspace_host):
    if "azuredatabricks.net" in workspace_host:
        return "https://accounts.azuredatabricks.net"
    elif "gcp.databricks.com" in workspace_host:
        return "https://accounts.gcp.databricks.com"
    else:
        return "https://accounts.cloud.databricks.com"


def find_current_workspace(account_client, workspace_host):
    for ws in account_client.workspaces.list():
        if ws.deployment_name and ws.deployment_name in workspace_host:
            return ws
    raise ValueError(
        f"No workspace found matching '{workspace_host}'. "
        "Ensure WORKSPACE_URL contains a valid Databricks workspace URL."
    )


def get_service_principal(account_client, name):
    for s in account_client.service_principals.list():
        if s.display_name == name:
            print(f"\nFound SP '{s.display_name}' (app_id={s.application_id})")
            return s
    raise ValueError(
        f"No service principal named '{name}' found. "
        "This script does not create one — provision it first."
    )


def get_workspaces_in_region(account_client, region):
    workspaces = []
    for ws in account_client.workspaces.list():
        if (ws.location or ws.aws_region) == region:
            workspaces.append(ws)
    return workspaces


def ensure_sp_assigned_to_workspace(account_client, ws_client, ws, sp):
    for s in ws_client.service_principals.list():
        if s.application_id == sp.application_id:
            return
    print("    Assigning SP to workspace …")
    account_client.workspace_assignment.update(
        workspace_id=ws.workspace_id,
        principal_id=int(sp.id),
        permissions=[WorkspacePermission.USER],
    )


def grant_permission(ws_client, resource_type, resource_id, sp, level):
    sp_app_id = str(sp.application_id)
    perms = ws_client.permissions.get(resource_type, resource_id)
    already_granted = any(
        entry.service_principal_name == sp_app_id
        and any(p.permission_level == level for p in (entry.all_permissions or []))
        for entry in (perms.access_control_list or [])
    )
    if already_granted:
        print(
            f"      {level.value} on {resource_type}/{resource_id} already granted, skipping."
        )
        return
    print(f"      Granting {level.value} on {resource_type}/{resource_id} …")
    ws_client.permissions.update(
        resource_type,
        resource_id,
        access_control_list=[
            AccessControlRequest(
                service_principal_name=sp_app_id, permission_level=level
            )
        ],
    )


if __name__ == "__main__":
    main()
```

---

# What Snowflake permissions are required for Automated Savings?

URL: https://select.dev/docs/reference/faq/automated-savings/automated-savings-permissions

> The single Snowflake grant needed to enable Automated Savings, which is applied automatically by the setup script.

The only permission required is granted during execution of the [setup script](https://select.dev/docs/reference/setup-guide/connect-snowflake-account#2-run-the-setup-script):

```sql
GRANT MANAGE WAREHOUSES ON ACCOUNT TO ROLE <select_role>;
```

---

# How do I enable Automated Savings?

URL: https://select.dev/docs/reference/faq/automated-savings/how-do-i-enable-automated-savings

> Steps to turn on Automated Savings for one or multiple warehouses from the Warehouses tab.

Navigate to the Warehouses tab. Each warehouse in the table will have a toggle that you can turn on. Or use the checkboxes to enable multiple at once.

![Warehouses tab with per-warehouse toggles to enable Automated Savings](https://cdn.sanity.io/images/wn0ylp2c/production/b5f9d4d6f0ed0e9602ba2933e94b6d464b26b1c7-2048x1225.png)

---

# Can Automated Savings negatively impact performance?

URL: https://select.dev/docs/reference/faq/automated-savings/can-automated-savings-impact-performance

> Explains that Automated Savings has shown no performance drawbacks across SELECT's customer base, since it never resizes warehouses or reduces cluster counts.

We have not seen any situation where turning off the [Automated Savings feature](https://select.dev/features/automated-savings) has been beneficial across our customer base of over 200 customers.

Customers can safely enable the feature without worry about performance impact, since the Automated Savings feature does NOT take actions which can impact query performance, like resizing the warehouse or reducing the max cluster count.

---

# How does automated savings work?

URL: https://select.dev/docs/reference/faq/automated-savings/how-does-the-automated-savings-feature-work

> Describes how Automated Savings monitors warehouses in real time and makes short-lived configuration changes to cut cost without affecting query performance.

The Automated Savings feature monitors warehouses in real time and makes configuration changes to improve utilization efficiency and drive cost savings without impacting query performance. The configuration changes are short-lived and shouldn't interfere with e.g. Terraform configurations. More details can be provided upon request.

---

# How can I see my Automated Savings in Dollars?

URL: https://select.dev/docs/reference/faq/automated-savings/how-can-i-see-my-automated-savings-in-dollars

> Shows how to view realized and estimated dollar savings on the warehouse chart, including how to read the total and projected annualized savings figures.

On the warehouses tab, click the 3 dots in the top right of the chart. Click Metric -> Savings -> Dollars Saved to view realized dollars saved. If Automated Savings was not previously enabled, you can choose Metric -> Savings -> Estimated Dollars saved to see what your savings could have been for a prior date range.

![Warehouse chart menu showing the Metric → Savings → Dollars Saved option](https://cdn.sanity.io/images/wn0ylp2c/production/2f418762fb77e4f1aac1e754149e707222773950-1661x450.png)

If you want to see a grand total annualized number, click the 3 dots again, Group By None.

![Warehouse chart grouped by None, showing total and annualized dollars saved](https://cdn.sanity.io/images/wn0ylp2c/production/717afa3a77d752e9ebdb6e62217f54f00a5e6536-1905x619.png)

At the bottom of the chart, you'll see two numbers:

1. **Total**. This is the total dollars saved in the time period. In this screenshot, it is $326 between the period of Sep 15 to Oct 8.
2. **Annualized**. This is the projected annualized savings based on the selected time period. For example, if the savings pattern from the current period continues, the projected annual savings from this example would be $4.96K/year. This annual savings projection is calculated based on the average daily savings in the observed time period. 
   1. Please note, this number can be skewed by days where the feature was not enabled. In the screenshot above, you can see that Sep 15-18 were abnormally low, which could be due to the feature not being enabled on all warehouses. Including this in your analysis could skew the annualized number lower since it lowers the average daily savings used in the projection.

---

# User Management

URL: https://select.dev/docs/reference/faq/user-management

> Learn about managing users in SELECT

- [How do I change my password?](https://select.dev/docs/reference/faq/user-management/how-do-i-change-my-password.md): Explains how non-SSO users reset their password via the "Forgot password?" link.
- [Does SELECT offer SSO?](https://select.dev/docs/reference/faq/user-management/does-select-offer-sso.md): Confirms SSO support via Okta and Azure AD (paid add-on) and free Google sign-in, with links to setup docs.
- [How do I manage users?](https://select.dev/docs/reference/faq/user-management/how-do-i-manage-users.md): Points to SELECT's user management documentation.

---

# How do I change my password?

URL: https://select.dev/docs/reference/faq/user-management/how-do-i-change-my-password

> Explains how non-SSO users reset their password via the "Forgot password?" link.

For non-SSO users, you can change your password by logging out, then click “Forgot password?”.

![Login page with the "Forgot password?" link for non-SSO users](https://cdn.sanity.io/images/wn0ylp2c/production/231d2a80b740c6ea696df92844771d4fce734596-637x816.png)

---

# Does SELECT offer SSO?

URL: https://select.dev/docs/reference/faq/user-management/does-select-offer-sso

> Confirms SSO support via Okta and Azure AD (paid add-on) and free Google sign-in, with links to setup docs.

Yes, you can sign in with SSO with Okta and Azure AD for a small up-charge. We also offer Sign in with Google for free.

[Okta documentation](https://select.dev/docs/reference/using-select/access-management/okta-sso)

[Azure AD documentation](https://select.dev/docs/reference/using-select/access-management/azure-ad-sso)

---

# How do I manage users?

URL: https://select.dev/docs/reference/faq/user-management/how-do-i-manage-users

> Points to SELECT's user management documentation.

Please refer to our docs about [user management](https://select.dev/docs/reference/using-select/access-management).

---

# Data Freshness and Quality

URL: https://select.dev/docs/reference/faq/data-freshness-and-quality

> Learn about data freshness in SELECT

- [The data looks out of date. Is there an issue?](https://select.dev/docs/reference/faq/data-freshness-and-quality/the-data-looks-out-of-date-is-there-an-issue.md): Explains that delayed refreshes or ongoing incidents are surfaced in the app's notifications section.
- [When was my data refreshed?](https://select.dev/docs/reference/faq/data-freshness-and-quality/data-freshness.md): Covers the default daily refresh schedule, how to check the last sync time via the notifications sidebar, and how to request a different schedule.
- [Why does SELECT not offer real time data or high frequency batches?](https://select.dev/docs/reference/faq/data-freshness-and-quality/why-does-select-not-offer-real-time-data-or-high-frequency-batches.md): Breaks down the cost reasons, to you and to SELECT, for daily-only refreshes by default, with Custom SQL Monitors as an alternative for intraday needs.
- [Do you have a Site Status page?](https://select.dev/docs/reference/faq/data-freshness-and-quality/do-you-have-a-site-status-page.md): Points to SELECT's status page and explains how to subscribe to incident updates.

---

# The data looks out of date. Is there an issue?

URL: https://select.dev/docs/reference/faq/data-freshness-and-quality/the-data-looks-out-of-date-is-there-an-issue

> Explains that delayed refreshes or ongoing incidents are surfaced in the app's notifications section.

If your data refresh is delayed for any reason or there is an ongoing incident with our app, you will be notified in the notifications section of the app, as shown here.

![Notifications panel showing a data delay or incident alert](https://cdn.sanity.io/images/wn0ylp2c/production/385d98143f5ffaab096332d653bf93f6eb2495bb-1919x1059.png)

---

# When was my data refreshed?

URL: https://select.dev/docs/reference/faq/data-freshness-and-quality/data-freshness

> Covers the default daily refresh schedule, how to check the last sync time via the notifications sidebar, and how to request a different schedule.

## How often is my data refreshed?

Data in SELECT is refreshed once per day at 3:00 UTC. Since refreshing your data costs us significant money, customers can purchase an add-on to refresh their data more frequently.

## When was my data last refreshed?

You can see information about your data freshness by using the notifications sidebar. Click the notification icon in the bottom left, and check the Metadata Sync Status section

![Notifications sidebar Metadata Sync Status section showing the last data refresh time](https://cdn.sanity.io/images/wn0ylp2c/production/17699c86e057e5464f1bf9988109bcd3828212a5-959x553.png)

## How can I change my refresh schedule?

Just ask us via the in-app chat or Slack / email. We can put you on whatever schedule works for you. As mentioned above, more than once per day refreshes have an extra charge.

---

# Why does SELECT not offer real time data or high frequency batches?

URL: https://select.dev/docs/reference/faq/data-freshness-and-quality/why-does-select-not-offer-real-time-data-or-high-frequency-batches

> Breaks down the cost reasons, to you and to SELECT, for daily-only refreshes by default, with Custom SQL Monitors as an alternative for intraday needs.

## Cost to you

If we were to keep an Extra Small warehouse constantly running to load your data in real time, that would be very costly in terms of Snowflake credits.  Here's an example.

$3 / credit x 24 hours x 365 days = $26,280 per year.

We don't want to cause this much spend in your Snowflake account!

## Cost to us

The data loading and transformation in our Snowflake account is very expensive.  For this reason, we need to charge if a customer requires refreshes more than once per day.

## Alternatives

Keep in mind, Custom SQL Monitors may meet your needs for intraday monitoring.

---

# Do you have a Site Status page?

URL: https://select.dev/docs/reference/faq/data-freshness-and-quality/do-you-have-a-site-status-page

> Points to SELECT's status page and explains how to subscribe to incident updates.

Yes, our site status page is [here](https://status.select.dev/). You can easily subscribe to updates to be notified of all incidents. Or, you can subscribe to individual incidents as they occur.

---

# Using SELECT

URL: https://select.dev/docs/reference/faq/using-select

> FAQs about using SELECT

- [Connect your Snowflake Account](https://select.dev/docs/reference/faq/using-select/connect-your-snowflake-account.md): Links to the setup guide for syncing a Snowflake account with SELECT.
- [Why doesn't the trend chart show all of my data?](https://select.dev/docs/reference/faq/using-select/why-doesn-t-the-trend-chart-show-all-of-my-data.md): Explains that overly granular date ranges get truncated, and how to fix it by shortening the range or widening the time grain.
- [What are 'Retained for Clones' storage costs?](https://select.dev/docs/reference/faq/using-select/what-are-retained-for-clones-storage-costs.md): Explains how storage costs become orphaned as "Retained for Clones" when a cloned object's source is dropped, and how to investigate them in Snowflake.
- [Does SELECT have an OpenAPI spec?](https://select.dev/docs/reference/faq/using-select/does-select-have-an-openapi-spec.md): Links to SELECT's public OpenAPI spec and describes common API use cases like usage groups, teams, and access control.
- [What are all of the storage metrics in SELECT?](https://select.dev/docs/reference/faq/using-select/what-are-all-of-the-storage-metrics-in-select.md): Defines Active, Time Travel, Failsafe, Clone, Total Billable, and Total Bytes storage metrics.
- [How can I monitor long running queries in my Snowflake Account?](https://select.dev/docs/reference/faq/using-select/how-can-i-monitor-long-running-queries-in-my-snowflake-account.md): Shows how to build a Custom SQL Monitor for still-running long queries, including required permissions and an example query and alert template.
- [How long does SELECT retain data](https://select.dev/docs/reference/faq/using-select/how-long-does-select-retain-data.md): Covers the initial historical load, ongoing retention while a customer, and immediate deletion on offboarding.
- [How do I view query profiles in SELECT?](https://select.dev/docs/reference/faq/using-select/how-do-i-view-query-profiles-in-select.md): Steps to reach a query's Profile tab from either the Workloads page or Query History.
- [How do I monitor Cortex Function Spend](https://select.dev/docs/reference/faq/using-select/how-to-i-monitor-cortex-function-spend.md): Steps to build a Saved View in Workloads filtered to non-null Cortex Function Cost.
- [How can I find Unused Tables and Schemas in SELECT?](https://select.dev/docs/reference/faq/using-select/how-can-i-find-unused-tables-and-schemas-in-select.md): Two approaches for finding unused tables and schemas: the Insights page's "Unused Table" filter, or custom date-range filters on the Storage page.
- [How is Utilizable Warehouse Size calculated?](https://select.dev/docs/reference/faq/using-select/how-is-utilizable-warehouse-size-calculated.md): Explains how QUERY_LOAD_PERCENT is converted into a node count and mapped to the smallest warehouse size that could have handled the query.
- [How do I monitor AI and Cortex spend in SELECT?](https://select.dev/docs/reference/faq/using-select/how-do-i-monitor-ai-and-cortex-spend-in-select.md): A complete guide to monitoring Snowflake AI and Cortex spend in SELECT: the AI Services SELECT tracks, the Snowflake AI Usage dashboard, and analyzing costs by resource type, model, user, and workload in Cost Explorer and Workloads, plus setting up spend alerts.
- [Why doesn't the Explorer page show Storage costs for my Usage Groups?](https://select.dev/docs/reference/faq/using-select/db-costs-on-explorer-page.md): Explains that Usage Groups based on object tags aren't reflected on the Explorer page, and to check the Storage page instead.
- [How do I display Snowflake spend by User?](https://select.dev/docs/reference/faq/using-select/how-do-i-display-snowflake-spend-by-user.md): Shows how to group workload charts and tables by user to see per-user Snowflake spend.
- [How do I find queries that underutilize a warehouse?](https://select.dev/docs/reference/faq/using-select/how-do-i-find-queries-that-underutilize-a-warehouse.md): Three methods, Utilizable Warehouse Size, Partitions Scanned, and short queries on large warehouses, for spotting oversized warehouse usage.
- [Can I estimate Snowflake time travel costs in advance using Select?](https://select.dev/docs/reference/faq/using-select/can-i-estimate-snowflake-time-travel-costs-in-advance-using-select.md): Explains that time travel costs can't be predicted in advance and must be measured empirically after enabling the feature.
- [What should I do during my free trial?](https://select.dev/docs/reference/faq/using-select/what-should-i-do-during-my-free-trial.md): Points new customers to the free trial guide and the broader Usage Guides.
- [How do I calculate my Databricks spend from system tables to compare against SELECT?](https://select.dev/docs/reference/faq/using-select/calculate-databricks-spend-from-system-tables.md): How to independently verify SELECT's reported Databricks platform spend using Databricks' own system.billing.usage and system.billing.list_prices tables, including any negotiated discounts.
- [Why is the cost of a query in SELECT different than the cost in Snowflake's Query Attribution History?](https://select.dev/docs/reference/faq/using-select/why-is-the-cost-of-a-query-in-select-different-than-the-cost-in-snowflake-s-query-attribution-history.md): Explains the methodology gaps (idle time, short queries, acceleration, AI, cloud services, data transfer) that make SELECT's cost-per-query figures typically higher than Snowflake's own attribution.
- [Are Adaptive Warehouses supported by SELECT?](https://select.dev/docs/reference/faq/using-select/are-adaptive-warehouses-supported-by-select.md): Confirms cost reporting works normally for adaptive warehouses, but notes Automated Savings isn't compatible with them.
- [I found discrepancies in Iceberg Storage](https://select.dev/docs/reference/faq/using-select/i-found-discrepancies-in-iceberg-storage.md): Clarifies that Iceberg billable-bytes figures are a known Snowflake data-modeling quirk, not an actual charge, and explains the "Bytes" vs "Bytes (Billable)" fields.
- [Why does the Storage total on Home/Explorer differ from the Storage page?](https://select.dev/docs/reference/faq/using-select/why-does-the-storage-total-on-home-explorer-differ-from-the-storage-page.md): Explains that account-level totals include retained clone bytes that dropped-source objects can no longer attribute at the database or table level.
- [How many of my teammates are using SELECT?](https://select.dev/docs/reference/faq/using-select/how-many-of-my-teammates-are-using-select.md): Points to Settings → Users for a list of users, page views, and last-visited dates.
- [What are all of the query time stats in SELECT?](https://select.dev/docs/reference/faq/using-select/what-are-all-of-the-query-time-stats-in-select.md): Defines Execution, Compilation, and Queued time metrics, why Execution Time and Total Duration differ, and which one to optimize.

---

# Connect your Snowflake Account

URL: https://select.dev/docs/reference/faq/using-select/connect-your-snowflake-account

> Links to the setup guide for syncing a Snowflake account with SELECT.

Details on syncing your Snowflake account to SELECT can be found [here](https://select.dev/docs/reference/setup-guide/connect-snowflake-account).

---

# Why doesn't the trend chart show all of my data?

URL: https://select.dev/docs/reference/faq/using-select/why-doesn-t-the-trend-chart-show-all-of-my-data

> Explains that overly granular date ranges get truncated, and how to fix it by shortening the range or widening the time grain.

When you select a date range or time grain that generates too many data points, SELECT automatically truncates the chart to display only the most recent data. A yellow info icon will appear in the top right corner of the chart, indicating that the result set is limited and prompting you to either:

- Select a shorter date range, or
- Choose a larger time grain (e.g., daily instead of hourly)

See the screenshot below for an example.

![Trend chart with an info icon indicating truncated data due to too many data points](https://cdn.sanity.io/images/wn0ylp2c/production/3d9a7aeeaf6c61dad2a4723e8ab515fda88ac072-977x319.png)

---

# What are 'Retained for Clones' storage costs?

URL: https://select.dev/docs/reference/faq/using-select/what-are-retained-for-clones-storage-costs

> Explains how storage costs become orphaned as "Retained for Clones" when a cloned object's source is dropped, and how to investigate them in Snowflake.

The “Retained for Clones” cost shown in some storage views in SELECT reflects the difference between the storage amount Snowflake billed you for and the amount attributed to the databases that currently exist. 

When a database is cloned, the storage cost for the clone is attributed to the upstream object. If the parent object was to be dropped, the storage cost remains unattributed, not reassigned to the clone. We label this unattributed cost as “Retained for clones” in SELECT.

To investigate further, you can query the TABLE_STORAGE_METRICS view in Snowflake by looking up the clone_group_id for the entries which are incurring RETAINED_FOR_CLONE_BYTES. See [here](https://community.snowflake.com/s/article/How-to-manage-RETAINED-FOR-CLONE-BYTES) for more details.

---

# Does SELECT have an OpenAPI spec?

URL: https://select.dev/docs/reference/faq/using-select/does-select-have-an-openapi-spec

> Links to SELECT's public OpenAPI spec and describes common API use cases like usage groups, teams, and access control.

Yes, we do! You can access it at [https://api.select.dev/v2/openapi.json](https://api.select.dev/v2/openapi.json) and use it to assist with code generation.

The SELECT API lets you programmatically manage usage groups, teams, and access controls. Common use cases include creating and updating usage group sets for cost allocation, automating team and member management, configuring role-based access control for users and SSO groups, and managing Snowflake account credentials. For more info about using our API, see the [docs](https://api-docs.select.dev/v2/introduction).

---

# What are all of the storage metrics in SELECT?

URL: https://select.dev/docs/reference/faq/using-select/what-are-all-of-the-storage-metrics-in-select

> Defines Active, Time Travel, Failsafe, Clone, Total Billable, and Total Bytes storage metrics.

This guide explains the different storage byte metrics tracked in [Select.dev](http://Select.dev) for Snowflake tables.

## Active Bytes

Active Bytes represent the storage consumed by data currently in use by your table. This is the primary storage for your active, queryable data. This metric excludes any historical data maintained for Time Travel or Fail-safe purposes.

## Time Travel Bytes

Time Travel Bytes are the storage bytes consumed by historical table data retained for Snowflake's Time Travel feature. Time Travel allows you to query, clone, and restore historical versions of your data within the retention period.

**Time Travel retention periods:**

-  Standard: 0-1 day (available in all editions)
-  Extended: Up to 90 days (requires Snowflake Enterprise Edition or higher)

## Failsafe Bytes

Failsafe Bytes are storage bytes consumed by deleted data that has passed the Time Travel retention period but is still within the Fail-safe period. Fail-safe is a Snowflake disaster recovery feature that provides a non-configurable 7-day period during which Snowflake may be able to recover historical data.

Fail-safe begins immediately after the Time Travel retention period ends. Data in Fail-safe cannot be accessed by users directly. Only Snowflake Support can initiate recovery in exceptional circumstances.

## Clone Bytes (Retained for Clone)

Clone Bytes (also called Retained for Clone Bytes) represent storage consumed by deleted data that is no longer in Time Travel or Fail-safe, but is still retained because one or more clones of the table reference these bytes.

When you clone a table, Snowflake uses zero-copy cloning technology. No data is physically duplicated initially. As the clone and original table diverge over time, storage may be retained for the clone even after data is deleted from the original table.

## Total Billable Bytes

Total Billable Bytes is the sum of all billable storage categories:

Total Billable Bytes = Active Bytes + Time Travel Bytes + Failsafe Bytes + Clone Bytes

```sql
Total Billable Bytes = Active Bytes + Time Travel Bytes + Failsafe Bytes + Clone Bytes
```

This represents the total amount of storage you're being charged for by Snowflake. Use this metric to understand your complete billable storage footprint.

## Total Bytes

Total Bytes represents the total physical storage consumed by the table, including both billable and non-billable storage types.

This metric includes:

- All billable storage (Active, Time Travel, Failsafe, Clone)
- Non-billable storage types (e.g., Iceberg tables, External tables)

Iceberg tables and External tables store data outside Snowflake's managed storage and therefore don't incur Snowflake storage charges, but their size may still be tracked in Total Bytes for visibility.

---

# How can I monitor long running queries in my Snowflake Account?

URL: https://select.dev/docs/reference/faq/using-select/how-can-i-monitor-long-running-queries-in-my-snowflake-account

> Shows how to build a Custom SQL Monitor for still-running long queries, including required permissions and an example query and alert template.

Most monitors in SELECT are based on the data we extract from your Snowflake account once per day. Anything you'd like to monitor on an intra-day basis can be done with **Custom SQL Monitors**. These fire inside of **your** Snowflake account when triggered.

## Why is this different from other Custom SQL monitors?

Monitoring queries that are still running has a special complexity: the Account Usage views in Snowflake will not show queries that haven't completed yet. To find "long running queries still running", we need to use Snowflake's `query_history()` table function instead.

## Required Permissions

By default, the `query_history()` table function only returns queries run by the calling user or queries that the user's role has access to. Since the SELECT service user runs these monitors, you'll only see queries that SELECT itself has run, not your team's queries.

To monitor queries across all users, grant the SELECT role **one of the following**:

### Option 1: Account-level privilege (recommended)

This grants visibility into all running queries across the account:

```sql
GRANT MONITOR EXECUTION ON ACCOUNT TO ROLE SELECT_ROLE;
```

It's worth noting that SELECT already has access to all historical queries in your account through the `account_usage.query_history` view so this permission isn't giving SELECT access to any net new information. The permission just allows the data to also be retrieved via `information_schema.query_history` for more real time reporting.

### Option 2: Warehouse-level privileges

If you prefer more granular control, grant MONITOR on each warehouse you want to observe:

```sql
GRANT MONITOR ON WAREHOUSE my_warehouse TO ROLE SELECT_ROLE;
GRANT MONITOR ON WAREHOUSE another_warehouse TO ROLE SELECT_ROLE;
-- Repeat for each warehouse
```

**Note:** With Option 2, the monitor will only capture long-running queries on warehouses where MONITOR has been granted.

## Example Query

Here's a SQL query you can use in a Custom SQL Monitor to capture currently running long queries, along with estimated credit usage and cost:

```sql
WITH credit_rate AS (
    SELECT column1 AS warehouse_size, column2 AS credits_per_hour
    FROM VALUES
        ('X-Small', 1),
        ('Small', 2),
        ('Medium', 4),
        ('Large', 8),
        ('X-Large', 16),
        ('2X-Large', 32),
        ('3X-Large', 64),
        ('4X-Large', 128),
        ('5X-Large', 256),
        ('6X-Large', 512)
)
SELECT
    qh.user_name,
    qh.query_type,
    qh.query_id,
    SUBSTRING(qh.query_text, 1, 100) AS trimmed_query_text,
    qh.warehouse_name,
    qh.warehouse_size,
    cr.credits_per_hour,
    DATEDIFF('second', qh.start_time, CURRENT_TIMESTAMP()) AS total_elapsed_time_s,
    3 AS cost_per_credit, -- Adjust this to match your cost per credit
    cr.credits_per_hour / 60 / 60 * total_elapsed_time_s * cost_per_credit AS dollars_spent,
    qh.start_time,
    qh.end_time
FROM TABLE(SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY(RESULT_LIMIT => 100)) qh
LEFT JOIN credit_rate cr ON qh.warehouse_size = cr.warehouse_size
WHERE qh.execution_status = 'RUNNING'
  AND DATEDIFF('second', qh.start_time, CURRENT_TIMESTAMP()) > 3600; -- 1 hour threshold

```

**Customize for your environment:**

- Adjust `cost_per_credit` to match your contract rate
- Change the `3600` second threshold (1 hour) to your desired alert duration
- Increase `RESULT_LIMIT` if you have high query concurrency

## Important: Understanding the Table Function Behavior

The `query_history()` table function has some quirks you need to understand to use it effectively.

### Result Limit

The function has a maximum of **10,000 rows** it can return (default is 100). If you have high query concurrency, you may not get all running queries.

### Filtering happens AFTER the result limit

This is the gotcha that trips people up: **the WHERE clause filters results AFTER the table function returns its rows**.

The function first returns up to `RESULT_LIMIT` rows (sorted by most recent end time), then your WHERE clause filters that set. This means if you filter by a specific user:

```sql
SELECT * 
FROM TABLE(SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY(RESULT_LIMIT => 100)) 
WHERE user_name = 'SOME_USER'
```

...and that user's queries aren't in the first 100 rows returned, you'll get zero results even though they have running queries.

### Pre-filtering with function parameters

To filter more reliably, use **parameters inside the function call** rather than the WHERE clause. These parameters are applied _before_ the result limit, so you get the rows you actually want. But these pre-filtering options are limited.

The base `QUERY_HISTORY()` function supports these parameters:

```sql
QUERY_HISTORY(
  END_TIME_RANGE_START => <timestamp>,  -- Only queries ending after this time
  END_TIME_RANGE_END => <timestamp>,    -- Only queries ending before this time
  RESULT_LIMIT => <number>              -- Max rows (default 100, max 10000)
)
```

### Alternative functions for filtering by user, warehouse, or session

If you need to pre-filter by something other than time, Snowflake provides variant functions:

- `QUERY_HISTORY_BY_USER()` — Pre-filters by user + time range
- `QUERY_HISTORY_BY_WAREHOUSE()` — Pre-filters by warehouse + time range
- `QUERY_HISTORY_BY_SESSION()` — Pre-filters by session + time range

## Suggested Jinja template

You can use this Jinja template in the Monitor but replace the Snowflake account and org identifiers with your own. Here's an example from our account: [https://app.snowflake.com/scwxhob/ad38017/#/compute/history/queries/01ba8ed5-0000-be68-0000-3de928e6403a/detail](https://app.snowflake.com/scwxhob/ad38017/#/compute/history/queries/01ba8ed5-0000-be68-0000-3de928e6403a/detail)

The structure is:

`https://app.snowflake.com/<organization_name>/<account_id>/#/compute/history/queries/<query_id>/detail`

```markup
A total of {{count}} rows were returned:

{% for row in rows[:5] %}

* **{{ row.query_type }}** by `{{ row.user_name }}` running for `{{ "%0.1f" | format(row.total_elapsed_time_s | float) }}`s on `{{ row.warehouse_name }}` ({{ row.warehouse_size }}) — **${{ "%.2f" | format(row.dollars_spent | float) }}** spent so far

Query Text (trimmed)
  > `{{ row.trimmed_query_text }}...`

  [→ View Query](https://app.snowflake.com/scwxhob/ad38017/#/compute/history/queries/{{row.query_id}}/detail)

{% endfor %}
```

When the Monitor arrives in Slack it will look like this:

![Slack alert from a Custom SQL Monitor showing long-running query details with a link to Snowflake](https://cdn.sanity.io/images/wn0ylp2c/production/40ce5411758d3f4f2c90cf89151a3c2084a069e2-779x621.png)

The links will bring you right to the query in Snowflake!

Here's what it looks like in the SELECT UI:

![SELECT UI showing the same long-running query monitor alert](https://cdn.sanity.io/images/wn0ylp2c/production/55bcbf1e1e2269094e4a230653ecee2cb58898b2-1885x1020.png)

---

# How long does SELECT retain data

URL: https://select.dev/docs/reference/faq/using-select/how-long-does-select-retain-data

> Covers the initial historical load, ongoing retention while a customer, and immediate deletion on offboarding.

When we first sync your account, we load **6 months of query history** and **1 year of cost data**.

After that, SELECT retains your data **for as long as you’re a customer**, allowing you to perform flexible cost analysis and query history searches over time.

When you offboard from SELECT, **all data associated with your account is deleted immediately.**

---

# How do I view query profiles in SELECT?

URL: https://select.dev/docs/reference/faq/using-select/how-do-i-view-query-profiles-in-select

> Steps to reach a query's Profile tab from either the Workloads page or Query History.

## From Workloads page

From the Workloads page, click any workload name to drill into that workload. (bottom table, left column). 

![Workloads page with a workload name link to drill into query details](https://cdn.sanity.io/images/wn0ylp2c/production/01c36f422167d61ae3068940e021ac11d412b5ab-1876x992.png)

On the next page, click "view latest run" in the right sidebar.

![Workload detail page with the "view latest run" link in the right sidebar](https://cdn.sanity.io/images/wn0ylp2c/production/06a91e9d15550374529bc7599b206970a66da26f-1896x931.png)

On the this page, click one of the queries you want to study:

![List of queries for a workload run, with one query selected to inspect](https://cdn.sanity.io/images/wn0ylp2c/production/ddcfeaf7cfc656cc49ebb833de43b9774ef94b6e-1201x373.png)

On the next page that loads up, click the Profile tab at the top:

![Query detail page with the Profile tab selected](https://cdn.sanity.io/images/wn0ylp2c/production/e5fdfd260248bb45397486939aa1c7bc56574081-1675x1003.png)

## From the Query History page

Navigate to query history in the left sidebar. Click any Query Text to drill in, then click the same Profile tab shown in the screenshot above.

---

# How do I monitor Cortex Function Spend

URL: https://select.dev/docs/reference/faq/using-select/how-to-i-monitor-cortex-function-spend

> Steps to build a Saved View in Workloads filtered to non-null Cortex Function Cost.

In SELECT, you can create a Saved View that is filtered to Cortex Functions spend.

1. Go to Workloads -> All Workloads
2. In the bottom table, Use the blue + to add a column to the table for "Cortex Function Cost".  Sort descending on this column.
3. In the filter bar at the top, apply a filter for "Cortex Function Cost" is not null.

![All Workloads table filtered to rows with a non-null Cortex Function Cost](https://cdn.sanity.io/images/wn0ylp2c/production/8cf16e9b101ffe444af9fee03d932b0d855a9bc7-968x88.png)

---

# How can I find Unused Tables and Schemas in SELECT?

URL: https://select.dev/docs/reference/faq/using-select/how-can-i-find-unused-tables-and-schemas-in-select

> Two approaches for finding unused tables and schemas: the Insights page's "Unused Table" filter, or custom date-range filters on the Storage page.

## Method 1: Using Insights

Go to the Insights page, Explorer tab (at the top), then add a filter for "Insight is Unused Table"

![Insights Explorer tab filtered to the "Insight is Unused Table" condition](https://cdn.sanity.io/images/wn0ylp2c/production/3f9179d23efdee8cb89c6385a1ea2759540ddfa3-1905x1045.png)

## Method 2: Using the Storage page

You can create a flexible, customizable view of unused tables or schemas on the Storage page by applying filters. Navigate to the **Tables** or **Schemas** tab.

The example below shows advanced filters on the **Schemas** tab to find schemas that haven’t been queried or altered in the last 30 days. The same approach works on the Tables tab. You can set any date range you prefer, or remove the “last altered” filter to identify tables or schemas that no one is reading but have active ETL.

![Schemas tab filtered to schemas not queried or altered in the last 30 days](https://cdn.sanity.io/images/wn0ylp2c/production/0a03bb2fc08f8e1a792368c9708bbb315334e04a-1900x1031.png)

---

# How is Utilizable Warehouse Size calculated?

URL: https://select.dev/docs/reference/faq/using-select/how-is-utilizable-warehouse-size-calculated

> Explains how QUERY_LOAD_PERCENT is converted into a node count and mapped to the smallest warehouse size that could have handled the query.

## Field Meaning

This field tells you the smallest warehouse that could have fully handled the query’s workload. It’s always capped at the size of the warehouse the query actually ran on.

## Field Calculation

The calculation uses the `QUERY_LOAD_PERCENT` field from the `query_history` view, which shows how much of a warehouse’s compute a query actually used. That value is translated into the number of nodes consumed, then mapped back to the equivalent warehouse size.

**Example**  
A query on a Large warehouse (8 nodes) with `QUERY_LOAD_PERCENT = 45` is equivalent to using about 4 nodes. Four nodes align with a Medium warehouse, so the utilizable warehouse size is Medium.

---

# How do I monitor AI and Cortex spend in SELECT?

URL: https://select.dev/docs/reference/faq/using-select/how-do-i-monitor-ai-and-cortex-spend-in-select

> A complete guide to monitoring Snowflake AI and Cortex spend in SELECT: the AI Services SELECT tracks, the Snowflake AI Usage dashboard, and analyzing costs by resource type, model, user, and workload in Cost Explorer and Workloads, plus setting up spend alerts.

SELECT gives you full visibility into Snowflake AI and Cortex costs — from a single "how much are we spending on AI" figure down to the individual model, user, and query driving it. This guide covers what SELECT tracks and the tools for analyzing it.

## What SELECT tracks

SELECT groups every Snowflake AI feature under a single **AI Services** category, broken down by resource type:

\- **Cortex Functions** — LLM inference and embedding calls (e.g. Complete, Embed)

\- **Cortex Search** — text embedding and search indexing

\- **Cortex Search Serving** — serving search queries from Cortex Search indexes

\- **Cortex Analyst** — AI-powered SQL generation, attributed by user

\- **Cortex Agent** — agentic workflows

\- **Cortex Code** — AI code assistance (Snowsight and CLI), attributed by user

\- **Cortex Fine Tuning** — custom model training

\- **Cortex Provisioned Throughput** — reserved model capacity

\- **Document AI** — document processing

\- **Snowflake Intelligence** — native Snowflake intelligence features

For the LLM-based services — **Cortex Functions, Agent, Code, and Snowflake Intelligence** — SELECT also attributes spend and tokens to the **specific model** used, even when a single request invokes more than one model.

## Start with the Snowflake AI Usage dashboard

The fastest way to get oriented is to create a dashboard from the built-in **Snowflake AI Usage** template. It comes preconfigured with:

\- **Total Spend** and **Total Spend (Annualized)**

\- **Daily Spend by Resource Type** — which Cortex service the spend comes from

\- **Daily Spend by AI Model** — which models cost the most

\- **Daily Spend by User**

\- **Top AI Workloads**

This gives you the full picture in one place before you drill into any single dimension.

## Explore AI spend in Cost Explorer

Navigate to **Cost Explorer** and group by **Service** — AI-related costs appear as a distinct **AI Services** category, so you can see what share of your Snowflake bill is AI. Filter to **AI Services** to isolate it, then break it down further by grouping by:

\- **Resource Type** — costs per Cortex service (Cortex Functions, Search, Agent, Document AI, etc.)

\- **AI Model** — costs per model, so you can compare, say, Claude vs. Llama vs. Mistral spend across your account

**Tracking a specific service — e.g. Cortex Code:** apply the filters **Service = 'AI Services'** and **Resource Type = 'Cortex Code'** ([open in Cost Explorer](https://select.dev/app/v2/~/cost-explorer?filters=v2[{%22field%22:%22usage_type%22,%22operator%22:%22in%22,%22values%22:[%22AI+Services%22]},{%22field%22:%22resource_type%22,%22operator%22:%22in%22,%22values%22:[%22Cortex+Code%22]}]&measure_filters=v2[]&columns=Platform,Organization,Instance,Resource,Spend,Percentage+of+Total+Spend,Annual+Cost,%25+Change)). Each **Resource** row is a user in your account incurring Cortex Code costs. Note that Cortex Code was free to use until April 1st 2026.

## Analyze at the workload and query level

On **Workloads → All Workloads** and other query-based pages, use **+ Add Column** to bring in AI-specific columns:

\- **Cortex Function Spend** — total spend from Cortex Function calls

\- **Cortex Tokens** — total tokens consumed

\- **Cortex Function Credits** — credits consumed by Cortex Functions

\- **Document AI Spend** — spend from Document AI processing

\- **Document AI Credits** — credits consumed by Document AI

\- **AI Model** — the model(s) each workload or query used

From here you can:

\- **Sort** by Cortex Function Spend (descending) to find your highest-spending workloads, and filter **Cortex Function Spend is not null** to show only workloads with AI spend.

\- **Group by User** (three-dot menu on the chart, or add User as a column) to see which people drive the most AI spend.

\- **Group by AI Model** to compare model usage across workloads.

\- Save any of these as a **Saved View** for ongoing monitoring.

## Set up spend alerts

From **Cost Explorer**, filter to the **AI Services** service type, then click **Create Monitor** to set up a **Spend Monitor**. Configure your threshold and notification preferences, and SELECT will alert you proactively if AI spend spikes unexpectedly.

---

# Why doesn't the Explorer page show Storage costs for my Usage Groups?

URL: https://select.dev/docs/reference/faq/using-select/db-costs-on-explorer-page

> Explains that Usage Groups based on object tags aren't reflected on the Explorer page, and to check the Storage page instead.

When you assign storage to a Usage Group using the **All Object Tags** filter, those storage costs won't appear in the Explorer tab.

For example, if you tag a table in Snowflake with `OWNER:ian` and then create a Usage Group with `All Object Tags = Owner:ian`, the storage costs for that table won't be attributed in Cost Explorer.

This is because the Explorer page only supports showing database-level storage costs over time, not costs allocated via object tags at the schema or table level.

### To view your allocated storage costs:

Go to the Storage page and filter by your Usage Group. You'll see all the relevant tables and their associated costs there.

---

# How do I display Snowflake spend by User?

URL: https://select.dev/docs/reference/faq/using-select/how-do-i-display-snowflake-spend-by-user

> Shows how to group workload charts and tables by user to see per-user Snowflake spend.

On the All Workloads page, you can use the 3 dots in the chart and group by user. (Use the search and type ‘user’).

![All Workloads chart three-dot menu with "Group By" set to user](https://cdn.sanity.io/images/wn0ylp2c/production/519968652cc65e31e7b9e74c5b0332c493f82f7c-1908x655.png)

Also, in the table, you can remove columns by clicking a column header and click Hide. Then add ‘user’ to the _Group By_ using the blue **+** Add Column on the right.

Here’s an example at the Workload + User grain.

![Workloads table grouped by Workload and User, showing spend at that grain](https://cdn.sanity.io/images/wn0ylp2c/production/abd120aae0b0bc4f9be700f3969323023ee1b9c9-1682x459.png)

Here’s an example at the User grain.

![Workloads table grouped by User only, showing spend per user](https://cdn.sanity.io/images/wn0ylp2c/production/76f3783d25176b82bd687b4dbdb364baa03d6390-1675x452.png)

---

# How do I find queries that underutilize a warehouse?

URL: https://select.dev/docs/reference/faq/using-select/how-do-i-find-queries-that-underutilize-a-warehouse

> Three methods, Utilizable Warehouse Size, Partitions Scanned, and short queries on large warehouses, for spotting oversized warehouse usage.

There are a few approaches to this:

## Method 1: Use the "Utilizable Warehouse Size" column

The Query History page has a field called "Utilizable Warehouse Size". You can use this field to filter to queries that could have run on a smaller warehouse compared to what it actually ran on. Here is an example of filtering to all queries that ran on a Large that could have run on Medium, Small, or X-Small:

![Query History filtered by Utilizable Warehouse Size to find oversized warehouse usage](https://cdn.sanity.io/images/wn0ylp2c/production/814b044aa9c28a1469e3b42b22c2c3fe9ef11710-1889x760.png)

## Method 2: Use the Partitions Scanned column

You can add the Partitions Scanned column to the query history, and use that as a heuristic to determine warehouse size. Here's some more info:

[https://select.dev/posts/snowflake-warehouse-sizing#using-partitions-scanned-as-a-heuristic](https://select.dev/posts/snowflake-warehouse-sizing#using-partitions-scanned-as-a-heuristic)

## Method 3: Find short running queries on larger warehouses

BI users need all queries to run fast. But for ETL jobs where the user is not waiting for the query to finish, it is optimal for each query that runs on a warehouse larger than an X-Small to run for 60 seconds. If you have an ETL query running on a medium that takes 10 seconds, we can assume it would run for ~40 seconds on an X-Small.

Therefore, one thing you can do is filter the query history to larger warehouses and shorter run times. Sort by query cost to find the most impactful queries. Here's an example:

![Query History filtered to larger warehouses with short run times, sorted by cost](https://cdn.sanity.io/images/wn0ylp2c/production/7e19ae76ac134a34c980f6e624b7b5d635da9546-1894x1033.png)

---

# Can I estimate Snowflake time travel costs in advance using Select?

URL: https://select.dev/docs/reference/faq/using-select/can-i-estimate-snowflake-time-travel-costs-in-advance-using-select

> Explains that time travel costs can't be predicted in advance and must be measured empirically after enabling the feature.

## Context

Customers often want to estimate the approximate cost of enabling time travel for specific tables in Snowflake before actually implementing it. This helps with budgeting and planning storage costs for their data warehouse.

## Answer

Unfortunately, there is no mechanism to estimate time travel costs in advance using Select or any other method. The costs cannot be predicted accurately before enabling the feature.

Time travel costs depend heavily on how frequently your tables are updated:

- For tables with infrequent updates, time travel costs may be only a small fraction of your active storage cost
- For frequently updated tables, time travel costs can be up to 10 times the active storage cost, as older micro partitions are constantly being rewritten

The only way to determine actual time travel costs is to:

1. Enable time travel for your specific tables
2. Monitor the storage costs over time
3. Analyze the cost impact based on your actual usage patterns

This monitoring approach will give you the exact numbers needed for accurate cost planning and budgeting decisions.

---

# What should I do during my free trial?

URL: https://select.dev/docs/reference/faq/using-select/what-should-i-do-during-my-free-trial

> Points new customers to the free trial guide and the broader Usage Guides.

Great question. Glad you asked! We have a [free trial guide](https://select.dev/docs/reference/usage-guide/select-free-trial-guide) to walk you through how to make the most of your trial experience.

Also, it is a great idea to take this time to skim all of our [Usage Guides](https://select.dev/docs/reference/usage-guide)

---

# How do I calculate my Databricks spend from system tables to compare against SELECT?

URL: https://select.dev/docs/reference/faq/using-select/calculate-databricks-spend-from-system-tables

> How to independently verify SELECT's reported Databricks platform spend using Databricks' own system.billing.usage and system.billing.list_prices tables, including any negotiated discounts.

SELECT reports Databricks platform spend, priced using Databricks' own `system.billing.usage` and `system.billing.list_prices` system tables. You can run the query below directly against your own Databricks workspace to independently verify SELECT's numbers.

This only covers Databricks platform spend. It does not include underlying cloud infrastructure spend (e.g. Azure, AWS, or GCP compute/storage billed directly by the cloud provider), which is a separate cost outside of what SELECT currently reports.

Fill in your `workspace_id` and `account_id` in the `params` CTE, and any negotiated discounts in the `discounts` CTE, then run the query in a Databricks SQL editor connected to your workspace. The total should line up with SELECT's Cost Explorer for the same date range.

```sql
with
-- === fill in below ===
params as (
    select
        '2026-06-01'::date as start_date,
        '2026-06-30'::date as end_date,
        '<your-workspace-id>' as workspace_id,
        '<your-account-id>' as account_id
),
-- add negotiated discounts here; more specific rows win (sku over product over global)
discounts (sku_name, billing_origin_product, discount_pct) as (
    values
        ('<sku-name>', null, 0.00)
    -- (null, '<billing-origin-product>', 0.00)  -- product-level discount
    -- (null, null, 0.00)  -- global discount
),
-- === end fill in ===
global_discount as (
    select discount_pct
    from discounts
    where sku_name is null
        and billing_origin_product is null
),
usage as (
    select
        u.record_id,
        u.account_id,
        u.workspace_id,
        u.sku_name,
        u.cloud,
        u.usage_unit,
        u.usage_date,
        u.usage_quantity,
        u.billing_origin_product
    from
        system.billing.usage as u
        cross join params as p
    where
        u.usage_date >= p.start_date
        and u.usage_date <= p.end_date
        and u.workspace_id = p.workspace_id
        and u.account_id = p.account_id
),
list_prices as (
    select
        account_id,
        sku_name,
        cloud,
        usage_unit,
        pricing ['effective_list'] ['default']::decimal(38, 10) as list_price,
        date_trunc(
            'day',
            timestampadd(second, 10, price_start_time)
        )::date as price_start_date,
        price_end_time::date as price_end_date
    from
        system.billing.list_prices
),
usage_priced as (
    select
        u.account_id,
        u.workspace_id,
        u.sku_name,
        u.billing_origin_product,
        u.usage_date,
        u.usage_quantity,
        lp.list_price,
        coalesce(sku_d.discount_pct, product_d.discount_pct, gd.discount_pct, 0) as discount_pct,
        lp.list_price * (
            1 - coalesce(sku_d.discount_pct, product_d.discount_pct, gd.discount_pct, 0)
        ) as discounted_price
    from
        usage as u
        inner join list_prices as lp on u.account_id = lp.account_id
        and u.sku_name = lp.sku_name
        and u.cloud = lp.cloud
        and u.usage_unit = lp.usage_unit
        and u.usage_date >= lp.price_start_date
        and (
            lp.price_end_date is null
            or u.usage_date < lp.price_end_date
        )
        left join discounts as sku_d on u.sku_name = sku_d.sku_name
        left join discounts as product_d on u.billing_origin_product = product_d.billing_origin_product
        and product_d.sku_name is null
        left join global_discount as gd on true
    -- price windows can overlap; keep only the latest one per usage record
    qualify row_number() over (
        partition by u.record_id
        order by
            lp.price_start_date desc
    ) = 1
)
select
    account_id,
    workspace_id,
    billing_origin_product,
    sku_name,
    sum(usage_quantity * list_price) as total_spend_list_price,
    sum(usage_quantity * discounted_price) as total_spend_discounted
from
    usage_priced
group by
    all
order by
    total_spend_discounted desc;

```

---

# Why is the cost of a query in SELECT different than the cost in Snowflake's Query Attribution History?

URL: https://select.dev/docs/reference/faq/using-select/why-is-the-cost-of-a-query-in-select-different-than-the-cost-in-snowflake-s-query-attribution-history

> Explains the methodology gaps (idle time, short queries, acceleration, AI, cloud services, data transfer) that make SELECT's cost-per-query figures typically higher than Snowflake's own attribution.

SELECT and Snowflake use different methodologies to calculate the cost of a query, which can cause the numbers to differ — sometimes significantly.  
Snowflake's Query Attribution History table has several limitations in what it accounts for:

- **Idle time** — Snowflake does not factor in warehouse idle time when attributing cost to queries
- **Short-running queries** — Snowflake excludes queries below a certain execution time threshold
- **Query Acceleration costs** — Snowflake does not include query acceleration service charges
- **Data transfer costs** — Snowflake does not include data transfer costs driven by a query
- **AI costs** — Snowflake does not include AI-related costs (e.g., Cortex) attributed to a query
- **Cloud Services costs** — Snowflake does not include cloud services layer costs

SELECT's cost per query calculation accounts for all of the above. We look at all queries running concurrently in a warehouse alongside granular billing data, then allocate cost to each query proportional to its execution time — including idle time, query acceleration, AI, cloud services, and data transfer costs.  


As a result, SELECT will often report a higher cost per query than what you see in Snowflake's Query Attribution History.  


For a deeper dive into our methodology, see [Calculating cost per query in Snowflake](https://select.dev/posts/cost-per-query).

---

# Are Adaptive Warehouses supported by SELECT?

URL: https://select.dev/docs/reference/faq/using-select/are-adaptive-warehouses-supported-by-select

> Confirms cost reporting works normally for adaptive warehouses, but notes Automated Savings isn't compatible with them.

SELECT reports the costs associated with adaptive warehouses like any other warehouse type, no special set up needed.

Automated Savings are not compatible with adaptive warehouses, as they're a different operating model than standard warehouses and don't have any configuration available such as auto suspend or size.

---

# I found discrepancies in Iceberg Storage

URL: https://select.dev/docs/reference/faq/using-select/i-found-discrepancies-in-iceberg-storage

> Clarifies that Iceberg billable-bytes figures are a known Snowflake data-modeling quirk, not an actual charge, and explains the "Bytes" vs "Bytes (Billable)" fields.

## The SELECT app is showing Billable Bytes for Iceberg Tables

Rest assured you are not billed for Iceberg tables. This is a known issue due to how Snowflake has modeled the data. The field called "Bytes (Billable)" in SELECT (charts and tables) comes from the `SNOWFLAKE.ACCOUNT_USAGE.TABLE_STORAGE_METRICS` view, `ACTIVE_BYTES` field. We may choose to zero out this measure for Iceberg in the near future.

## The "Bytes" field in SELECT does not match "Bytes (Billable)" for Iceberg Tables

The "Bytes" field in the SELECT UI comes from the `SNOWFLAKE.ACCOUNT_USAGE.TABLE_STORAGE_METRICS` view, `ACTIVE_BYTES` field. Snowflake has confusingly modeled to only count the metadata storage for Iceberg. Therefore this number is always smaller for Iceberg tables.  


The "Bytes (Billable)" field in the SELECT UI comes from `SNOWFLAKE.ACCOUNT_USAGE.TABLES` view, `BYTES` field. Snowflake has modeled this data to show the total of the physical data files  + Iceberg metadata.

---

# Why does the Storage total on Home/Explorer differ from the Storage page?

URL: https://select.dev/docs/reference/faq/using-select/why-does-the-storage-total-on-home-explorer-differ-from-the-storage-page

> Explains that account-level totals include retained clone bytes that dropped-source objects can no longer attribute at the database or table level.

In SELECT, the Storage totals on the Home and Account Explorer pages come from `SNOWFLAKE.ACCOUNT_USAGE.STORAGE_USAGE` (account level). This view includes `retained_for_clone_bytes` in the grand total and matches Snowflake billing. It is aggregated at `date + account`, so there is no per-database or per-table detail in this view.

The Storage page uses more detailed sources, including `SNOWFLAKE.ACCOUNT_USAGE.DATABASE_STORAGE_USAGE_HISTORY` (per-database history) and `SNOWFLAKE.ACCOUNT_USAGE.TABLE_STORAGE_METRICS` (per-table metrics). These do not attribute bytes from dropped original objects that still back zero-copy clones. Once the original object is dropped, those retained bytes continue to appear only in the account-level total in `STORAGE_USAGE`, not under any active database or table. As a result, the detailed sums can be lower than the grand total.

**Example**  
You clone `fct_sales` to `fct_sales_backup`. Billing stays with the original. If `fct_sales` is later dropped, Snowflake still charges for the shared data and reports it in `STORAGE_USAGE` as `retained_for_clone_bytes`. The original no longer appears in `DATABASE_STORAGE_USAGE_HISTORY` or `TABLE_STORAGE_METRICS`, so the detailed view will not include those retained bytes.

This is a modeling limitation of the current `ACCOUNT_USAGE` views.

---

# How many of my teammates are using SELECT?

URL: https://select.dev/docs/reference/faq/using-select/how-many-of-my-teammates-are-using-select

> Points to Settings → Users for a list of users, page views, and last-visited dates.

In SELECT, the best way to see active users on your team is to go to Settings -> Users.  On this page you will find a list of users, Page Views, and their Last Visited date.  At this time we do not have trend charts or "active monthly users" metrics available.

![Settings → Users page listing team members, page views, and last-visited dates](https://cdn.sanity.io/images/wn0ylp2c/production/6fa4e1b037852b3964e459359aff57438a48b86a-1567x706.png)

---

# What are all of the query time stats in SELECT?

URL: https://select.dev/docs/reference/faq/using-select/what-are-all-of-the-query-time-stats-in-select

> Defines Execution, Compilation, and Queued time metrics, why Execution Time and Total Duration differ, and which one to optimize.

SELECT includes several time-based metrics across different views.

## Metric Definitions

### Query History Page

- **Execution Time** – Actual time spent processing or running the query
- **Compilation Time** – Time spent parsing and optimizing the query before execution
- **Queued Provisioning Time** – Time waiting for the warehouse to start up or scale
- **Queued Repair Time** – Time waiting while warehouse infrastructure is being repaired
- **Queued Overload Time** – Time waiting because the warehouse is overloaded with queries
- **Transaction Blocked Time** – Time waiting for transaction locks to release

### Workloads Page

The workloads page offers similar metrics which average the time across historical runs of the workload.

For example, **Average Execution Time,** averages the execution duration across all historical runs of the workload.

## Why doesn’t Execution Time equal Total Duration?

Execution Time only measures active query processing.  
Total Duration includes all waiting periods (queuing and compilation) in addition to execution.  
For example, a query might execute in 5 seconds but have a total duration of 30 seconds if it spent 25 seconds waiting for the warehouse to start.

## Which time metric should I optimize?

We recommend focusing on Query Execution time as the best mechanism to reduce Snowflake spend.

---

# Security

URL: https://select.dev/docs/reference/faq/security

> Frequently asked questions about Security.

- [Security Architecture](https://select.dev/docs/reference/faq/security/security-architecture.md): Points customers to SELECT's public Security FAQ and Security page for architecture details.

---

# Security Architecture

URL: https://select.dev/docs/reference/faq/security/security-architecture

> Points customers to SELECT's public Security FAQ and Security page for architecture details.

A good place to start learning about our security architecture can be found in our security FAQ, [here](https://select.dev/security/faq). 

We recommend reading all 4 articles on our Security page, [here](https://select.dev/security).

---

# Resources

URL: https://select.dev/docs/resources/snowflake-developer-guide/resources

> Cost Management & Optimization Resources from SELECT

## Cost Management & Optimization

The SELECT team regularly writes about the best practices for managing costs and [improving query performance](https://select.dev/posts/snowflake-query-optimization) in Snowflake. Whether you're a new or experienced Snowflake user, there will be something for you in our [blog](https://select.dev/posts). If you're unsure where to begin, the posts below serve as a great starting point for your [cost optimization](https://select.dev/posts/snowflake-cost-optimization) journey!

- [Snowflake Architecture Overview](https://select.dev/posts/snowflake-architecture): Learn how the internals of Snowflake work
- [Introduction to Snowflake's Micro-Partitions](https://select.dev/posts/snowflake-micro-partitions): Learn how Snowflake stores data, and how it can greatly accelerate query performance
- [Effective Clustering in Snowflake](https://select.dev/posts/snowflake-clustering): Understand the most important query optimization strategy in Snowflake.
- [Choosing the right warehouse size in Snowflake](https://select.dev/posts/snowflake-warehouse-sizing): A step by step guide to one of the most important settings in Snowflake.
- [Snowflake Query Optimization Strategies](https://select.dev/posts/snowflake-query-optimization): Our top tips for accelerating your query workloads in Snowflake.
- [Snowflake Cost Reduction Strategies](https://select.dev/posts/snowflake-cost-optimization): Actionable recommendations and strategies for lowering your Snowflake bill.

Here are some other resources:

- [Snowflake Pricing](https://select.dev/posts/snowflake-pricing)
- [Storage Costs](https://select.dev/posts/snowflake-storage)
- [Query Timeouts](https://select.dev/posts/snowflake-query-timeouts)
- [Resource Monitors](https://select.dev/posts/snowflake-resource-monitors)
- [Query Tags](https://select.dev/posts/snowflake-query-tags)
- [How to use the Snowflake Query Profile](https://select.dev/posts/snowflake-query-profile)
- [How to calculate cost per query](https://select.dev/posts/cost-per-query)

## Query Optimization

- [60x faster database clones](https://select.dev/posts/snowflake-clones)
- [Effective Clustering in Snowflake](https://select.dev/posts/snowflake-clustering)
- [Query Optimization Strategies](https://select.dev/posts/snowflake-query-optimization)
- [Should you use CTEs in Snowflake?](https://select.dev/posts/snowflake-ctes)
- [Optimizing Range Joins](https://select.dev/posts/snowflake-range-join-optimization)
- [Defining multiple cluster keys in Snowflake with materialized views](https://select.dev/posts/clustering-with-materialized-views)

## Administration

- [Stored Procedures](https://select.dev/posts/snowflake-stored-procedures)
- [Tasks](https://select.dev/posts/snowflake-tasks)
- [Terraform for Snowflake](https://select.dev/posts/snowflake-terraform)
- [Alerts & Notifications](https://select.dev/posts/snowflake-alerts)
- [Error Notifications for Tasks](https://select.dev/posts/error-notifications-for-snowflake-tasks)
- [Snowflake Query Queuing: Analyzing Queuing Overload](https://select.dev/docs/resources/snowflake-developer-guide/snowflake-queuing)
- [Enabling and Disabling Snowflake MFA](https://select.dev/docs/resources/snowflake-developer-guide/snowflake-mfa)
- [Using the Snowflake Query History: 9 Practical Examples](https://select.dev/posts/snowflake-query-history)
- [Snowflake key pair authentication](https://select.dev/docs/resources/snowflake-developer-guide/snowflake-key-pair)
- [Snowflake User Type property](https://select.dev/posts/snowflake-user-type)

## Security

- [Roles & Access Control](https://select.dev/posts/snowflake-roles)

## Loading Data

- [Data Loading Options](https://select.dev/posts/snowflake-data-loading-overview)
- [Batch Data Loading Best Practices](https://select.dev/posts/snowflake-batch-loading)
- [Snowpipe](https://select.dev/posts/snowflake-snowpipe)
- [Merge Command](https://select.dev/posts/snowflake-merges)

## Data Transformations

- [Stored Procedures](https://select.dev/posts/snowflake-stored-procedures)
- [Tasks](https://select.dev/posts/snowflake-tasks)
- [Working with Semi-Structured Data](https://select.dev/posts/snowflake-semi-structured-data)
- [JSON Data Parsing](https://select.dev/posts/snowflake-json-data-parsing)
- [Exclude and rename](https://select.dev/posts/exclude-rename)
- [DATEDIFF](https://select.dev/docs/snowflake-developer-guide/snowflake-datediff)

## dbt & Snowflake

- [3 ways to configure Snowflake warehouse sizes in dbt](https://select.dev/posts/configuring-snowflake-warehouse-sizes-in-dbt)
- [Monitoring dbt model spend and performance with metadata](https://select.dev/posts/dbt-query-metadata)
- [Identifying Unused dbt Models](https://select.dev/posts/dbt-unused-models)
- [dbt Optimization Strategies](https://select.dev/posts/dbt-optimization)
- [dbt threads](https://select.dev/posts/snowflake-dbt-threads)

---

# Snowflake Query Queuing: Analyzing Queued Overload

URL: https://select.dev/docs/resources/snowflake-developer-guide/snowflake-queuing

> In this post, Ian explains what queuing means in Snowflake and how to check for it.

Queuing is an important topic to understand, whether you're [optimizing costs](https://select.dev/posts/snowflake-cost-optimization) or trying to make your [queries run faster](https://select.dev/posts/snowflake-query-optimization). Let's dive in!

![Snowflake warehouse load history](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/eee330f7-70fd-4d19-bb89-02dcd85b9700/width=2966,quality=75)

## What is Queuing in Snowflake?

When a query is submitted to run on a virtual warehouse, Snowflake will first check if there are enough computing resources available to run the query. If the warehouse is busy processing other queries and does not have enough capacity to run the new query, Snowflake will automatically put that query in a queue. As soon as the required capacity becomes available, Snowflake will immediately run that query.

All of this is managed behind the scenes for Snowflake users, and you do not have to do anything to enable this behavior.

### What is Queued Overload?

When talking about query queuing, you will often hear **queued overload**. This is simply the name of the metric reported by Snowflake to measure queuing. **queued overload time** is the amount of time in milliseconds that a query sits in the queue. You can identify this time programmatically using the Snowflake query history view (more on this below).

## Is Queuing Bad?

Whether queuing should be avoided is highly dependent on the application running on your Snowflake virtual warehouse. For example, if the warehouse is being used to serve interactive queries issued by humans, then you may want to avoid queuing because it means the users will have to wait longer than normal for their queries to run.

On the flip side, if your virtual warehouse is primarily used for automated processes like batch data transformations or for [alerting](https://select.dev/posts/snowflake-alerts), then queuing can be tolerable as long as the queries finish in a time that is acceptable to the business's needs. When a warehouse begins to queue queries, it means that the compute capacity is being fully utilized. From a [cost optimization](https://select.dev/posts/snowflake-cost-optimization) standpoint, this is a fantastic thing! Many Snowflake warehouses sit underutilized and result in wasted spend. Queuing is one way to know this is not happening!

## How to avoid Queuing?

If you **need** to eliminate queuing, there are a few different ways you can avoid it:

1. Increase the [warehouse size](https://select.dev/posts/snowflake-warehouse-sizing)
2. Increase the maximum cluster count (if you are on an Enterprise or higher [Snowflake Edition](https://select.dev/posts/snowflake-pricing#snowflake-editions)) so that Snowflake can spin up additional computing clusters when queuing starts to occur
3. Ensure the scaling policy is set to Standard. The Economy policy will wait for ~6 minutes of queuing before spinning up an additional cluster.

## How to check for queuing?

There are three different ways to check if queries are being queued in your Snowflake virtual warehouse.

### Method 1: Querying the query history

Snowflake provides a view called `query_history` for all its customers, which contains 1 row per query. You can query this view to see how long each query was queued for due to the warehouse being overloaded / fully saturated using the `queued_overload_time` column.

```sql
select start_time, query_text, user_name, queued_overload_time, execution_time from snowflake.account_usage.query_history where start_time > current_date - 7 and warehouse_name='YOUR_WAREHOUSE_NAME'
```

### Method 2: Using the Queries tab in Snowflake

While your warehouse is actively being used, you can pull open the Query History tab in the Snowsight UI and look whether any queries are currently queued:

![Snowflake query history status](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/2fad8257-1b09-44df-a8b2-f9583ec12c00/width=2976,quality=75)

For historical queries, you can hover over the "Duration" column and you'll see the amount of time spent in **queued overload**.

### Method 3: Using the warehouse load history graph

On the warehouses page in the Snowsight UI, you can look at the Warehouse Activity graph, which gives you a quick overview of how often there is queueing in the warehouse. This screenshot is from our the warehosue we run our dbt project on. There is consistent queueing in the warehouse, which is exactly what we want since it gives us confidence that the warehouse is being fully utilized. Even with this queuing, our dbt project runs finish well within the SLA so this queuing is not a concern for us.

![Snowflake warehouse load history](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/eee330f7-70fd-4d19-bb89-02dcd85b9700/width=2966,quality=75)

---

# How to setup Snowflake key pair authentication

URL: https://select.dev/docs/resources/snowflake-developer-guide/snowflake-key-pair

> In this post Ian covers how to secure your Snowflake machine users with key pair authentication

Key-pair authentication in Snowflake represents a significant enhancement in security by replacing the traditional username-password method with a more robust and secure approach. It is particularly important for machine users where [enabling MFA](https://select.dev/docs/snowflake-developer-guide/snowflake-mfa) is not practical.

This authentication method leverages public and private cryptographic keys, ensuring that access to the Snowflake environment is restricted to authorized users who possess the corresponding private key. By eliminating the reliance on passwords, which can be vulnerable to attacks such as phishing, brute force, and credential stuffing, key-pair authentication mitigates the risk of unauthorized access and enhances the overall security posture of your data operations in Snowflake.

In this post, I'll cover how to set up key-pair authentication in <10 minutes, step by step with screenshots and code snippets you can use directly.

Let's dive in.

## Step 1: Generate a private key

Open your terminal and run the following command to create **an encrypted private key file**. If you’re using a mac, `openssl` should be installed by default, no additional setup required.

`openssl genrsa 2048 | openssl pkcs8 -topk8 -v2 des3 -inform PEM -out rsa_key.p8`

You’ll be prompted to enter an encryption password twice:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-6-626d39bb442a7950d3bda2035419f93c69aa0b3177e76da3620f5aa2c6ff0523/width=1484,quality=75)

## Step 2: Generate a public key

Next, you’ll generate the public key file by reference the private key created in the previous step. Run the following command in your terminal:

`openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub`

Again, you’ll be prompted to enter the passphrase (encryption password) from before:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-2-8b803a11f68a7dab26d6c6b917258e9ee026ee363c2907296dca9ac463e950b4/width=1494,quality=75)

## Step 3: Store your private key, public key, and encryption password

This commands above will create the private key (`rsa_key.p8` ) and public key (`rsa_key.pub`) files in whatever directory you ran the command above from. You can run `open .` in your terminal to open the current folder you’re in. You should see something like the screenshot below:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-5-ab97db75650a9064cc2ac73d7c35b92d52947bb48ad6d1b3630cd0643e482567/width=1920,quality=75)

You should securely store both files along with the encryption password for future reference.

If you’re creating the key-pair for use in a SaaS tools like [SELECT](https://select.dev/), you’ll be prompted to enter the private key file contents and encryption password:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-1-215f40c4280d4352b41f55dae0fac5a2b29a7503e88ee66d73eab5ffea1fd2f4/width=2328,quality=75)

## Step 4: Assign the public key to a Snowflake user

Open the `rsa_key.pub` public key file in a text editor, or run `cat rsa_key.pub` from your terminal. Copy the text in between the `----BEGIN ...----` and `----END...----`:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-4-dd5d8c48250d43a54643fc8854baeade7123d003f204181d17ce63b59e9bc6d3/width=1480,quality=75)

Next, assign the public key to a Snowflake user. In this example, the user name is `machine_user`.

```sql
alter user machine_user set rsa_public_key='MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3nD038XL7myESbMhTxzN af1dUU2jkfeJby5/l5rQ6gy5uivtdgysdACKuMWmahhmdT1x77lbHa6LxlIebeVN EJhOCg8wH0c2v+TvAcv9RQgvRy1Er2dz188ETExPhUfq7CJ3AqJ6RYiREPiE1iTE ikkTXt0Vq8m3tP/DbO45g5v43ZWvPyQ+MhvCN3mECRTwzE6uueA+QiK4K5jOn1R1 CWsKDzzLUd3IvGnBoqoigOq5zQJfQN7ZaIOTYakab48oXi4iFZKBrhhGDw0Cnl+S olfm4uu+DshVbiR9gY4Xmh/KOX9fpLjGH6wCVfOgtOa3/gkFDJzrfwb+v8R27HJx rQIDAQAB';
```

If the Snowflake user previously had a password set, you should disable that login method by running the following command:

```sql
alter user machine_user unset password
```

## Step 5: Validate public key was set correctly (OPTIONAL)

If you would like to validate everything was setup correctly, run the command `desc user machine_user` and make note of the output under `RSA_PUBLIC_KEY_FP`:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-0-297973814bd19736c2f1c771c3b73294c3088feea0712f1b6766f2ddc2e38246/width=3004,quality=75)

Then head back to the terminal and run the following command:

`openssl rsa -pubin -in rsa_key.pub -outform DER | openssl dgst -sha256 -binary | openssl enc -base64`

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-key-pair-3-33e343db6504b711bf7bc7cffd1842425b6a895ae9dfa5f48702a4cfe700dee5/width=1750,quality=75)

If the value in the output (`ANpiIIzQDezLa6SvoU+zOAWX8tY9R8eLCC2KYJjjJLw=`) matches the output of `RSA_PUBLIC_KEY_FP` shown in the screenshot, then everything was setup correctly.

## Connecting a Key-Pair User via Python

If the Snowflake machine user is for use in a Python application, you can connect the user using the private key file and passphrase using the [Snowflake Connector for Python](https://select.dev/posts/snowflake-python-connector). The scripts below show how this can be done.

First, read and decrypt the private key file:

```python
import snowflake.connector
import os
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization

with open("<path>/rsa_key.p8", "rb") as key:
    private_key = serialization.load_pem_private_key(
        key.read(),
        password=os.environ['ENCRYPTION_PASSWORD'].encode(),
        backend=default_backend()
    )

private_key_bytes = private_key.private_bytes(
    encoding=serialization.Encoding.DER,
    format=serialization.PrivateFormat.PKCS8,
    encryption_algorithm=serialization.NoEncryption()
)
```

Next, connect to Snowflake using the same `connect` method as is used with username/password authentication, but instead of a password supply the private key:

```python
conn = snowflake.connector.connect(
    user='<user>',
    account='<account_identifier>',
    private_key=private_key_bytes,
    )

cs = conn.cursor()
```

You can find your [account identifier](https://select.dev/docs/connect-snowflake-account#finding-your-account-identifier) by following the instructions [here](https://select.dev/docs/connect-snowflake-account#finding-your-account-identifier).

---

# Snowflake MFA Setup: Enabling & Disabling

URL: https://select.dev/docs/resources/snowflake-developer-guide/snowflake-mfa

> In this post, Ian explains how to enable MFA in your Snowflake account, and how account admins can disable it for a given user.

Multi-factor authentication (MFA) can be used for free by all Snowflake customers to provide increased security when connecting to Snowflake. Let's talk about how you can get setup with MFA in Snowflake.

## How to enable MFA for a Snowflake User?

To enable MFA for your Snowflake user, click on your profile in the Snowsight UI:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-mfa-0-eb999c632e20d9f9a894836b7c1b15315ba5b6237b16ec8b8c992f83d850a759/width=2352,quality=75)

At the bottom of your profile card, you'll see a section for **Multi-factor authentication**. Click the **Enroll** button:

![](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-mfa-1-086d99204c290ae4c454d67a42b259939f56e43508217259d3b1f763c2583a2e/width=2334,quality=75)

Snowflake's MFA feature is powered by [Cisco Duo](https://duo.com/). You do not need to separately sign up for Duo, but you will need to download the [Duo Mobile Application](https://guide.duo.com/) on your phone.

Once enrolled, you'll see a [screen like this](https://docs.snowflake.com/en/user-guide/security-mfa#using-mfa-with-sf-web-interface) the next time you log in.

## How to disable MFA in Snowflake?

Disabling MFA can only be done programmatically with SQL by a user with the [ACCOUNTADMIN role](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles). Here's the command to disable MFA for a user:

```sql
use role accountadmin; alter user ian set disable_mfa=true;
```

## Is MFA enabled by default?

MFA is not enabled by default in Snowflake. It must be enabled on a per-user basis and users are not automatically enrolled in MFA. Administrators cannot automatically enforce that all users require MFA. They can instead build automations using the queries below to identify users without MFA and take the appropriate action.

## Identify all users without MFA enabled

To identify users without MFA enabled, you can query the `snowflake.account_usage.users` view (details [here](https://docs.snowflake.com/en/sql-reference/account-usage/users)).

```sql
select * from snowflake.account_usage.users where ext_authn_duo = false and deleted_on is null order by last_success_login desc ;
```

Note, this view can be up to 2 hours stale. For an up-to-date copy of the data, run the command `show users`.

The output of this query will likely contain many machine users, for which it doesn't make sense to enable MFA. Instead, it is recommended to setup [key-pair authentication](https://select.dev/docs/snowflake-developer-guide/snowflake-key-pair) for these users, instead of the traditional username-password authentication method.

## Identify all PRIVILEGED users without MFA

It's especially important to identify any users without MFA who have privileged access (one of the [default](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles) [`SYSADMIN`,](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles) [`SECURITYADMIN`, or](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles) [`ACCOUNTADMIN`](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles) [roles](https://select.dev/posts/snowflake-roles#default-snowflake-account-roles)), as a compromise of one of these users would be much more significant.

The following query identifies all users with access to one of these roles who do not have MFA enabled:

```sql
with 
privileged_roles (role_name) as (
    select grantee_name
    from snowflake.account_usage.grants_to_roles
    where
        privilege = 'MANAGE GRANTS'
        or grantee_name in ('SYSADMIN', 'SECURITYADMIN', 'ACCOUNTADMIN')
)
select
    users.name,
    users.has_password,
    timediff(days, users.last_success_login, current_timestamp())|| ' days ago' as last_login,
    timediff(days, users.password_last_set_time, current_timestamp(6)) || ' days ago' as password_age
from snowflake.account_usage.grants_to_users
inner join privileged_roles
    on grants_to_users.role = privileged_roles.role_name
    and grants_to_users.deleted_on is null
inner join snowflake.account_usage.users
    on grants_to_users.grantee_name = users.name
    and users.ext_authn_duo = false
    and users.deleted_on is null
order by users.last_success_login desc
;
```

## What other security measures should you implement?

In addition to enabling MFA for all human users in your Snowflake Account, consider the following additional security measures recommended by Snowflake:

- Use [key pair authentication](https://select.dev/docs/snowflake-developer-guide/snowflake-key-pair) for all users without MFA
- Rotate passwords frequently
- Utilize account-level Network Policies to only allow logins from known IP addresses for all users
- Utilize user-level Network to ensure service accounts can be only used from a more specific list of IP's
- Use Authentication Policies to ensure service accounts cannot use the UI
- Create a custom role for each service account and ensure least-privilege
- Consider using Session Policies and Password Policies
- Utilize a username naming convention to monitor all SQL closely and have a low threshold for suspicious activities

---

# Snowflake DATEDIFF: 4 Quick Reference Examples

URL: https://select.dev/docs/resources/snowflake-developer-guide/snowflake-datediff

> Learn how to leverage Snowflake's most popular date function: DATEDIFF

As a powerful engine for large-scale analytics, Snowflake is often used to process large datasets containing time-series data. Oftentimes, Snowflake users will need to calculate the difference in time between two dates or timestamps. Luckily, Snowflake makes this super easy with its `DATEDIFF` function.

![Snowflake DATEDIFF function syntax](https://select.dev/cdn-cgi/imagedelivery/1zmOcgV1p520E4lLTrYjjg/snowflake-datediff-4-quick-reference-examples-0-0d7aa181a30473d8f597fc1f7f4656e063c6a05e07107e87c06674962b631c4e/width=1908,quality=75)

Let’s dive in.

## DATEDIFF Function Syntax

As shown in the image above, there are three inputs to the `DATEDIFF` function:

`DATEDIFF( <unit>, <date/timestamp #1>, <date/timestamp #2>)`

1. `**unit**`**:** the unit of time to calculate the difference in (eg. `'day'`, `'month'`, `'hour'`, etc.)
2. `**date/timestamp #1**`**:** the first date/timestamp
3. `**date/timestamp #2**`**:** the second date/timestamp

### Example Usage

When you call `datediff('day', date'2024-04-01', date'2024-05-11')`, Snowflake will evaluate that as **the number of days between 2024-05-11 and 2024-04-01** (40).

```sql
 select datediff('day', date'2024-04-01', date'2024-05-11') -- evaluated as: 2024-05-11 - 2024-04-01
```

### **Supported Date Parts**

When comparing two dates, the following values are accepted for the `unit` parameter:

- `year`
- `quarter`
- `month`
- `week`
- `date`

### **Supported Time Parts**

When comparing two timestamps, the following values are accepted for the `unit` parameter:

- `hour`
- `minute`
- `second`
- `millisecond`
- `microsecond`
- `nanosecond`

Now that we’ve covered the function syntax and required parameters, let’s look at some examples.

## Number of days between two dates

To calculate the difference between two dates, you’ll supply `day` as the unit followed by the two dates `datediff('day', date'2024-04-01', date'2024-05-11')`.

```sql
select datediff('day', date'2024-04-01', date'2024-05-11') -- Output: 40 -- Quotes around the unit are not required! 'day' and day are both accepted select datediff(day, date'2024-04-01', date'2024-05-11') -- Output: 40
```

This effectively gets evaluated as `2024-05-11` - `2024-04-01`.

Note, you can also just use the `-` sign directly in the SQL rather than passing the dates into the `DATEDIFF` function:

```sql
select date'2024-05-11' - date'2024-04-01' -- Output: 40
```

### Impact of date ordering

`DATEDIFF` always calculates the difference as `date 2 - date 1`. If we swap the example above, we’ll get a negative:

```sql
select datediff('day', date'2024-05-11', date'2024-04-01') -- Output: -40
```

## Number of months between two dates

To calculate the number of months between two dates, we just need to supply `month` as the unit:

```sql
select datediff('month', date'2024-04-01', date'2024-05-11') -- Output: 1
```

Note how Snowflake still outputs 1 month between the two dates, even though they are only 11 days apart:

```sql
select datediff('month', date'2024-04-30', date'2024-05-11') -- Output: 1
```

If two dates fall in the same month, the output will be 0:

```sql
select datediff('month', date'2024-05-01', date'2024-05-11') -- Output: 0
```

## Number of hours between two timestamps

`DATEDIFF` can also compare two timestamps instead of dates. Here’s an example showing the number of hours between two timestamps:

```sql
select datediff('hour', timestamp'2024-04-11 12:00:00', timestamp'2024-04-11 14:00:00') --Output: 2
```

## Number of seconds between two timestamps

When working with timestamps, the difference calculation can be done more granularly. As per the accepted time part options listed above, you can pass in `second` to calculate the number of seconds between two timestamps:

```sql
select datediff('second', timestamp'2024-04-11 12:00:00', timestamp'2024-04-11 14:00:00') -- Output: 7200
```

## DATEDIFF Rounding

DATEDIFF does not do any rounding.

If we calculate the number of days between April 1 and May 30, it is 59:

```sql
select datediff(day, date'2024-04-01', date'2024-05-30') -- Output: 59
```

When we calculate the number of months between these same dates, the output is 1, even though they are closer to two months apart:

```sql
select datediff(month, date'2024-04-01', date'2024-05-30') -- Output: 1
```

---

# About

URL: https://select.dev/docs/resources/health-check-app/health-check-app

> SELECT Health Check Native App

## Introduction

SELECT's Health Check native app provides high level account usage insights, along with an estimate of the automated savings that the [full SELECT app](https://select.dev) can achieve.

## Installation

Install the app from the [native app marketplace listing](https://app.snowflake.com/marketplace/listing/GZTSZ1GQQPI/select-health-check). The app is available for free to all Snowflake accounts running in AWS. Native apps are not yet in public availability in other cloud providers.

Once installed, the app requires some permissions to run metadata queries in your account. From an `accountadmin` role, the permissions can be granted by running the below commands.

```sql
grant manage warehouses on account to application health_check;
grant imported privileges on database snowflake to application health_check;
```

Please note that if you rename the share from `health_check` to something else, you will need to replace the `health_check` text in the SQL above with the name you used.

## Available Procedures

Before running the procedures, follow the setup instructions above.

### health_check.core.healthcheck()

This stored procedure provides a high-level account health report. The stored procedure highlights:

1. Unused warehouses
2. The top 5 warehouses by credit usage from the last 24 hours
3. The top 5 query patterns from the last 24 hours

A query pattern is a grouping of queries by similar structure, ignoring literals and other typically changeable elements.

To run this procedure:

```sql
call health_check.core.healthcheck();
```

Example response:

```markdown
SELECT Healthcheck Results
There are 20 warehouses in the account.
The following warehouses have not been used in 60 days and can likely be dropped:
DBT_CLOUD
LOOKER
BI
TRANSFORM_XSMALL
The top 5 most expensive warehouses in the last 24 hours were:
- APP_BACKEND_SMALL with 36.005 credits used (77.00% of total).
- APP_BACKEND_DBT_MEDIUM with 6.664 credits used (14.00% of total).
- APP_BACKEND with 2.653 credits used (6.00% of total).
- APP_BACKEND_LARGE with 0.842 credits used (2.00% of total).
- APP_BACKEND_SMALL with 0.625 credits used (1.00% of total).
The top 5 query patterns in the last 24 hours were:
1. Example query ID: 01af1c27-0000-33b3-0000-3de90338e232
- 28 executions.
- 2m average execution time.
- 7h 13m 16s of weighted execution time.
- Users APP, APP_BACKEND.
- Warehouses APP, APP_BACKEND_SMALL.
...
Where weighted execution time is calculated as execution time * warehouse size as integer (e.g. 1 for X-Small, 2 for Small, etc.)
```

### health_check.core.savings_estimate(date)

This stored procedure provides an annualized estimate for the automated savings that the [full SELECT app](https://select.dev) would have provided on a given date.

To run this procedure:

```sql
-- this may take more than a few minutes to complete
call health_check.core.savings_estimate('2024-11-25'); -- update the date as required for an up-to-date estimate
```

Example response:

| WAREHOUSE_NAME | ANNUALIZED_USAGE_DOLLARS | ANNUALIZED_USAGE_DOLLARS_WITH_SELECT | ANNUALIZED_SELECT_SAVINGS_DOLLARS | SELECT_SAVINGS_PERCENT |
| --- | --- | --- | --- | --- |
| APP_BACKEND | 117895 | 108405 | 9490 | 8.05 |
| APP_SMALL | 76650 | 68255 | 8395 | 10.95 |
| APP_LARGE | 51465 | 43800 | 7665 | 14.89 |
| ADMIN | 64240 | 59130 | 5110 | 7.95 |
| APP | 12410 | 9855 | 2555 | 20.59 |
| APP_DBT_MEDIUM | 9855 | 8760 | 1095 | 11.11 |

For an accurate savings estimate, we recommend running the `savings_estimate` procedure across several dates and average the results. Please reach out to the SELECT team by clicking the chat bubble at the bottom right if you run into any issues.

---

# Frequently Asked Questions

URL: https://select.dev/docs/resources/comparisons/faq

> Frequently asked questions about how SELECT compares to other tools

## How does SELECT differ from other products in the market?

SELECT is the leading cost management & optimization platform for Snowflake, adopted by over 200 companies. Our primary differentiation from other products in the market like [Slingshot](https://select.dev/docs/select-vs-slingshot) or [Keebo](https://select.dev/docs/select-vs-keebo) comes from four main factors:

### 1. We are an end to end platform

In addition to having [fully automated warehouse optimization capabilities](https://select.dev/features/automated-savings) which reduce compute costs by up to 20% like Keebo, we also provide [comprehensive cost visiblity](https://select.dev/features/cost-visibility), [surface additional optimizations](https://select.dev/features/insights) teams can implement, flexible [cost allocation capabilities](https://select.dev/features/usage-groups), and a variety of [out of the box alerts](https://select.dev/features/monitors) to help teams control costs long term.

### 2. Depth

Many Snowflake customers will have some [home grown cost dashboards](https://select.dev/select-vs-homegrown) and alerts in place, which they eventually learn fall short of their needs due to a lack of depth. If you have a single BI tool warehouse costing hundreds of thousands of dollars per year, how can you identify which dashboards and business users are responsible for that consumption? What if you want to filter costs by object tag or identify which specific queries are driving the highest AI services spend? What is the cost of each Dynamic Table, SPROC, and Task.

Due to having to support over 200 customers, we get asked questions like this every day and have grown our product accordingly.

### 3. Integrations

A large majority of Snowflake consumption comes from 3rd party tools like dbt, Fivetran, Coalesce, Sigma, Looker, Tableau, and more. SELECT actively builds and maintains first party integrations for these tools so you can see the Snowflake costs they drive, along with their [lineage](https://select.dev/docs/lineage).

### 4. World Class Product Support

We create a shared Slack/Teams channel with every SELECT customer, allowing us to give you a world class support experience. We respond to questions and feature requests within hours. We regularly hop on a calls with customers to help them implement a new Snowflake feature, refactor one of their queries, or help them implement advanced object tagging strategies to improve their cost allocation strategy.

Interactions like these go a long way and help you get additional value from working with us.

> [!NOTE]
> As the co-founder & CEO of SELECT, I, [Ian](https://www.linkedin.com/in/ian-whitestone/), am obviously very biased writing this doc.
>
> At SELECT, we genuinely believe we are the best option in the market, and many of our customers who've evaluated multiple offerings have told us so, but you should form your own educated opinion.
>
> We encourage you to try out multiple products and come to that conclusion yourself.

## Why doesn't SELECT charge a % of savings?

Many of our customers ask us why we don't charge a % of savings as a cost management product.

First off, it becomes very complicated. Let's say we identify a compute process that costs $50/day loading a table that no one queries. Or maybe we identify a workload that has been failing for the past month, wasting $40/day.

After you action that insight and deprecate the workload, how much did we save you? $50/day for 6 months ($9K)? $50/day for 12 months ($18K)? This quickly becomes a game of what-if. How long would it have taken for you to notice this compute process and deprecate it yourself?

What about when we alert of you a job which has suddenly increased in costs then the developer goes and fixes the query the next day in response?

It gets messy.

There's an important quote from the world of FinOps which we strongly believe in:

> The greatest cloud savings come from costs never born.

One of the biggest value adds of SELECT is we help you catch things before they become an issue. In addition, through ongoing usage of our product Snowflake users around a company become more educated on best practices and how different actions impact costs.

Our goal is to help you prevent spend issues or misconfigured warehouses from existing in the first place! We'd much prefer that then letting things run innefficiently for 6 months just so we can claim a % of the savings.

## Why doesn't SELECT automatically resize warehouses?

Many customers ask us if we automatically change the size of a virtual warehouse throughout the day. While this sounds great in theory, it doesn't turn out that way in practice.

With many of our customers who have tried solutions which offer this, they've experienced both:

1. Performance issues as a workload suddenly loses over half of its available computing power / memory.
2. User confusion, as users who are expecting to run a query against a Large warehouse suddenly have a Small warehouse to work with.

Changing the warehouse size will always impact performance, unless your warehouse is grossly oversized and none of the queries running on it are properly utilizing the warehouse's available compute.

If that's the case, why not downsize the warehouse yourself versus pay a provider 25% of the cost savings from having their solution downsize it for you?

When it comes to warehouse sizing, we believe in providing customers with the tools & insights that tell them which specific workloads should be run on a different warehouse size, or if the warehouse as a whole should be downsized or upsized based on the workloads running on it.

---

# SELECT vs. Homegrown Cost Monitoring

URL: https://select.dev/docs/resources/comparisons/select-vs-homegrown

> Learn how SELECT compares to homegrown Snowflake cost monitoring tools.

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customers’ compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## Homegrown tooling

Many customers we speak with build and maintain their own spend monitoring tooling, typically in their BI tool of choice (i.e. PowerBI).

## Detailed Comparison

The comparison below is based on what we see customers typically build in house.

### Cost Visibility & Allocation

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ✅ |
| See cost per query | ✅ | ✅ |
| Analyze remaining contract usage and forecast | ✅ | ❌ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ✅ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ❌ |
| Analyze data spillage warehouse | ✅ | ❌ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ❌ |
| View and audit generated savings | ✅ | ❌ |
| Identify misconfigured warehouses | ✅ | ❌ |
| Query optimization recommendations | ✅ | ❌ |
| Under-utilized warehouses | ✅ | ❌ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ✅ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ✅ |
| Send alerts to Email | ✅ | ✅ |
| Send alerts to Slack | ✅ | ❌ |
| Send alerts to Teams | ✅ | ❌ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ❌ |
| Customize the alert message & contents | ✅ | ❌ |
| Send alerts to multiple destinations | ✅ | ❌ |

### Cost Allocations

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Allocate costs by database | ✅ | ✅ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ✅ |
| Allocate costs using Snowflake object tags | ✅ | ❌ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ❌ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ❌ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ❌ |

### Security

| Capability | SELECT | Homegrown Dashboard |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ✅ |
| Key-Pair Authentication Support | ✅ | ✅ |
| Flexible RBAC & roles assignment | ✅ | ✅ |
| Restrict access at the resource level | ✅ | ❌ |
| Automatically rotate keys via API | ✅ | ❌ |
| Integration with Okta SSO | ✅ | ✅ |
| Integration with Entra ID (Azure AD) SSO | ✅ | ✅ |
| Integration with OneLogin SSO | ✅ | ✅ |
| Integration with PingFed SSO | ✅ | ✅ |
| Automatically assign roles based on SSO groups | ✅ | ❌ |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

A homegrown dashboard can easily cost hundreds of thousands of dollars per year depending on the level of effort and involvement by the team. Some example costs, assuming a fully loading employee cost of $250,000/year:

- 1 month/year --> $20,000/year
- 3 months/year --> $60,000/year
- 1 full time employee --> $250,000/year
- 2 full time employees --> $500,000/year

## Frequently Asked Questions

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save 10-20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### What makes SELECT a better alternative to homegrown dashboard?

SELECT offers the following key advantages over a typical homegrown dashboard:

1. **Fully automated savings:** SELECT helps customers automatically lower their spend by 10-20% through fully automated and real time warehouse optimizations.
2. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and innefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
3. **Deeper cost visbility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
4. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
5. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
6. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department.
7. **Zero maintenance and effort required:** SELECT has a team of over 10 engineers who build and maintain SELECT's cost monitoring capabilities, add cost monitoring support for new Snowflake functionality, and handle breaking changes to the Snowflake metadata database.
8. **Battle tested:** SELECT product is used and battle tested by hundreds of companies every day.

---

# SELECT vs. Snowflake's Native Cost Monitoring Features

URL: https://select.dev/docs/resources/comparisons/select-vs-snowflake

> SELECT vs Snowflake's Cost Monitoring Features

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customer’s compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## Snowflake's Native Cost Monitoring Features

Many customers we speak start with Snowflake's out of the box cost monitoring features. Customers quickly outgrow these capabilities and choose to either:

1. Build a [homegrown monitoring dashboard](https://select.dev/docs/select-vs-homegrown) in-house
2. Decide to purchase a 3rd party tool like [SELECT](https://select.dev/) or [Capital One Slingshot](https://select.dev/docs/select-vs-slingshot)

## Detailed Comparison

### Cost Visibility & Allocation

| Capability | SELECT | Snowflake Cost Management |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ❌ |
| See cost per query | ✅ | ❌ |
| Analyze remaining contract usage and forecast | ✅ | ❌ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ✅ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | Snowflake Cost Management |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ❌ |
| Analyze data spillage warehouse | ✅ | ❌ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | Snowflake Cost Management |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ❌ |
| View and audit generated savings | ✅ | ❌ |
| Identify misconfigured warehouses | ✅ | ❌ |
| Query optimization recommendations | ✅ | ❌ |
| Under-utilized warehouses | ✅ | ❌ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ✅ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | Snowflake Native Cost Management |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | Snowflake Native Cost Management |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ✅ |
| Send alerts to Email | ✅ | ✅ |
| Send alerts to Slack | ✅ | ❌ |
| Send alerts to Teams | ✅ | ❌ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ✅ |
| Customize the alert message & contents | ✅ | ✅ |
| Send alerts to multiple destinations | ✅ | ✅ |

### Cost Allocations

| Capability | SELECT | Snowflake Native Cost Management |
| --- | --- | --- |
| Allocate costs by database | ✅ | ✅ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ✅ |
| Allocate costs using Snowflake object tags | ✅ | ✅ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | Snowflake Native Cost Management |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ✅ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ✅ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ✅ |

### Security

| Capability | SELECT | Snowflake Native Cost Mangement |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ✅ |
| Key-Pair Authentication Support | ✅ | ✅ |
| Flexible RBAC & roles assignment | ✅ | ✅ |
| Restrict access at the resource level | ✅ | ❌ |
| Automatically rotate keys via API | ✅ | ✅ |
| Integration with Okta SSO | ✅ | ✅ |
| Integration with Entra ID (Azure AD) SSO | ✅ | ✅ |
| Integration with OneLogin SSO | ✅ | ✅ |
| Integration with PingFed SSO | ✅ | ✅ |
| Automatically assign roles based on SSO groups | ✅ | ✅ |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

Snowflake's native cost management features are available at no cost.

## Frequently Asked Questions

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save 10-20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### What makes SELECT a better alternative to Snowflake's native cost management features?

SELECT offers the following key advantages over Snowflake's native cost management features:

1. **Fully automated savings:** SELECT helps customers automatically lower their spend by 10-20% through fully automated and real time warehouse optimizations.
2. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and inefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
3. **Deeper cost visibility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
4. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
5. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
6. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department .

---

# SELECT vs. Keebo

URL: https://select.dev/docs/resources/comparisons/select-vs-keebo

> SELECT vs Keebo

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customer’s compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## About Keebo

Keebo was founded in 2019. They started by building a BI query acceleration product which re-wrote Looker queries on the fly, which involves all user’s queries and data being routed through Keebo’s servers. They have since expanded to focus on Snowflake cost optimization, by automatically resizing customer’s warehouses throughout the day based on machine learning algorithms.

## Detailed Comparison

### Cost Visibility & Allocation

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ✅ |
| See cost per query | ✅ | ❌ |
| Analyze remaining contract usage and forecast | ✅ | ❌ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ❌ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ❌ |
| Analyze data spillage warehouse | ✅ | ❌ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ✅ |
| View and audit generated savings | ✅ | ✅ |
| Identify misconfigured warehouses | ✅ | ❌ |
| Query optimization recommendations | ✅ | ❌ |
| Under-utilized warehouses | ✅ | ❌ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ❌ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | Keebo |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ❌ |
| Send alerts to Email | ✅ | ❌ |
| Send alerts to Slack | ✅ | ❌ |
| Send alerts to Teams | ✅ | ❌ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ❌ |
| Customize the alert message & contents | ✅ | ❌ |
| Send alerts to multiple destinations | ✅ | ❌ |

### Cost Allocations

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Allocate costs by database | ✅ | ❌ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ❌ |
| Allocate costs using Snowflake object tags | ✅ | ❌ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ❌ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ❌ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ❌ |

### Security

| Capability | SELECT | Keebo |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ? |
| Key-Pair Authentication Support | ✅ | ? |
| Flexible RBAC & roles assignment | ✅ | ? |
| Restrict access at the resource level | ✅ | ? |
| Automatically rotate keys via API | ✅ | ? |
| Integration with Okta SSO | ✅ | ? |
| Integration with Entra ID (Azure AD) SSO | ✅ | ? |
| Integration with OneLogin SSO | ✅ | ? |
| Integration with PingFed SSO | ✅ | ? |
| Automatically assign roles based on SSO groups | ✅ | ? |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

Keebo charges ~40% of the savings they generate through their warehouse optimization feature.

## Frequently Asked Questions

### How does SELECT's pricing compare to Keebo's?

SELECT charges a flat rate instead of a % of savings. Learn why [here](https://select.dev/docs/comparisons/faq#why-doesnt-select-charge-a-of-savings).

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save up to 20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### How is SELECT different than Keebo?

SELECT and Keebo both offer a fully automated savings feature which yields cost savings with no effort.

In addition to fully automated savings, SELECT:

- Provides deep cost visibility into Snowflake workloads and spend drivers to help developers self diagnose issues and opportunities
- Surfaces detailed optimization opportunities that teams can action themselves. These are things that can't / shouldn't be automated, like dropping unused tables or adding a specific filter to a dashboard's query.
- Offers out-of-the-box flexible alerts to notify teams and users of spend anomalies
- Integrates with platforms that sit on top of Snowflake and drive compute costs

### What makes SELECT a better alternative to Keebo?

Keebo solely focuses on automated warehouse optimizations, which SELECT does a well with our [Automated Savings feature](https://select.dev/features/automated-savings).

In addition to Automated Savings, here is a summary of other key differentiators:

1. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and inefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
2. **Deeper cost visibility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
3. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
4. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
5. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department.
6. **Battle tested:** SELECT product is used and battle tested by hundreds of companies every day.

## Commitment to Transparency

This comparison guide was put together based on publicly available documentation and customer references. If you spot an innaccuracy, please contact [hello@select.dev](mailto:hello@select.dev) and provide verifiable evidence to demonstrate your claim, and we will update the page accordingly.

---

# SELECT vs. Capital One Slingshot

URL: https://select.dev/docs/resources/comparisons/select-vs-capital-one-slingshot

> SELECT vs. Capital One Slingshot

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customer’s compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## About Slingshot

Slingshot was founded in 2022 based on Capital One’s need to manage spend across 50 separate business units & snowflake accounts. The tool focuses on governance features required by large banks to streamline access requests as well as cost monitoring. Slingshot can help some customers lower Snowflake spend by providing them with scheduled downsizing of their oversized warehouses on nights and weekends. They also provide email alerts when usage increases.

## Detailed Comparison

### Cost Visibility & Allocation

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ✅ |
| See cost per query | ✅ | ✅ |
| Analyze remaining contract usage and forecast | ✅ | ✅ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ❌ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ❌ |
| Analyze data spillage warehouse | ✅ | ❌ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ✅ |
| View and audit generated savings | ✅ | ✅ |
| Identify misconfigured warehouses | ✅ | ✅ |
| Query optimization recommendations | ✅ | ✅ |
| Under-utilized warehouses | ✅ | ✅ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ❌ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ✅ |
| Send alerts to Email | ✅ | ✅ |
| Send alerts to Slack | ✅ | ❌ |
| Send alerts to Teams | ✅ | ❌ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ❌ |
| Customize the alert message & contents | ✅ | ❌ |
| Send alerts to multiple destinations | ✅ | ❌ |

### Cost Allocations

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Allocate costs by database | ✅ | ✅ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ✅ |
| Allocate costs using Snowflake object tags | ✅ | ❌ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ❌ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ❌ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ❌ |

### Security

| Capability | SELECT | Slingshot |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ✅ |
| Key-Pair Authentication Support | ✅ | ✅ |
| Flexible RBAC & roles assignment | ✅ | ✅ |
| Restrict access at the resource level | ✅ | ❌ |
| Automatically rotate keys via API | ✅ | ❌ |
| Integration with Okta SSO | ✅ | ❌ |
| Integration with Entra ID (Azure AD) SSO | ✅ | ❌ |
| Integration with OneLogin SSO | ✅ | ❌ |
| Integration with PingFed SSO | ✅ | ❌ |
| Automatically assign roles based on SSO groups | ✅ | ❌ |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

Slingshot charges $30,000/year if under 25M queries per month and $72,000/year if over 25M queries per month.

## Frequently Asked Questions

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save 10-20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### How is SELECT different than Slingshot?

SELECT and Slingshot provide similar capabilities at a high level. They both offering cost visibility, insights, and fully automated savings. The core differences come from the depth and number of features offered in each core capability. Refer to the detailed comparison above for examples.

### How does SELECT's pricing compare to Capital One Slingshot's?

SELECT's ensures all of its customers receive a positive ROI through the savings generated from the product. With the exception of high spend customers (over $3M/year) SELECT will generally be less expensive than Slingshot.

### What makes SELECT a better alternative to Slingshot?

Simply put: we go much deeper and provide more extensive capabilities as shown in the detailed comparison above. We encourage you to try us out and see for yourself. Of all customers who have tried both Slingshot and SELECT, all have chosen SELECT. Here is a summary of our key differentiators:

1. **Fully automated savings:** SELECT helps customers automatically lower their spend by 10-20% through fully automated and real time warehouse optimizations.
2. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and inefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
3. **Deeper cost visibility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
4. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
5. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
6. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department .
7. **Battle tested:** SELECT product is used and battle tested by hundreds of companies every day.

## Commitment to Transparency

This comparison guide was put together based on publicly available documentation and customer references. If you spot an innaccuracy, please contact [hello@select.dev](mailto:hello@select.dev) and provide verifiable evidence to demonstrate your claim, and we will update the page accordingly.

---

# SELECT vs. Chaos Genius

URL: https://select.dev/docs/resources/comparisons/select-vs-chaos-genius

> SELECT vs. Chaos Genius

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customer’s compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## About Chaos Genius

Chaos Genius, based out of India, started out as a data observability tool in 2021, but has since pivoted to Snowflake cost monitoring. Their tool provides basic cost reporting and alerting in Slack.

## Detailed Comparison

### Cost Visibility & Allocation

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ✅ |
| See cost per query | ✅ | ✅ |
| Analyze remaining contract usage and forecast | ✅ | ❌ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ❌ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ✅ |
| Analyze data spillage warehouse | ✅ | ✅ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ❌ |
| View and audit generated savings | ✅ | ❌ |
| Identify misconfigured warehouses | ✅ | ✅ |
| Query optimization recommendations | ✅ | ✅ |
| Under-utilized warehouses | ✅ | ❌ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ❌ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ✅ |
| Send alerts to Email | ✅ | ✅ |
| Send alerts to Slack | ✅ | ✅ |
| Send alerts to Teams | ✅ | ✅ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ❌ |
| Customize the alert message & contents | ✅ | ❌ |
| Send alerts to multiple destinations | ✅ | ✅ |

### Cost Allocations

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Allocate costs by database | ✅ | ❌ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ✅ |
| Allocate costs using Snowflake object tags | ✅ | ❌ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ❌ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ❌ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ❌ |

### Security

| Capability | SELECT | Chaos Genius |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ✅ |
| Key-Pair Authentication Support | ✅ | ✅ |
| Flexible RBAC & roles assignment | ✅ | ❌ |
| Restrict access at the resource level | ✅ | ❌ |
| Automatically rotate keys via API | ✅ | ❌ |
| Integration with Okta SSO | ✅ | ❌ |
| Integration with Entra ID (Azure AD) SSO | ✅ | ❌ |
| Integration with OneLogin SSO | ✅ | ❌ |
| Integration with PingFed SSO | ✅ | ❌ |
| Automatically assign roles based on SSO groups | ✅ | ❌ |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

Chaos Genius's pricing is not disclosed for customers spending more than $200,000/year - though they say they charge a small % of total spend. Below that level, Chaos Genius costs $4,200/year.

## Frequently Asked Questions

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save 10-20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### How is SELECT different than Chaos Genius?

SELECT and Chaos Genius both provide spend visibility and optimization insights. A key differentiator for SELECT is that it helps customers automatically lower their spend by 10-20% through fully automated and real time warehouse optimizations. Refer to the detailed comparison above for more examples.

### How does SELECT's pricing compare to Chaos Genius?

SELECT's ensures all of its customers receive a positive ROI through the savings generated from the product.

Chaos Genius' pricing is not disclosed for customers spending more than $200K/year, though they say they charge a small % of total spend.

### What makes SELECT a better alternative to Chaos Genius?

Here is a summary of our key differentiators:

1. **Fully automated savings:** SELECT helps customers automatically lower their spend by 10-20% through fully automated and real time warehouse optimizations.
2. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and inefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
3. **Deeper cost visibility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
4. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
5. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
6. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department .
7. **Battle tested:** SELECT product is used and battle tested by hundreds of companies every day.

## Commitment to Transparency

This comparison guide was put together based on publicly available documentation and customer references. If you spot an innaccuracy, please contact [hello@select.dev](mailto:hello@select.dev) and provide verifiable evidence to demonstrate your claim, and we will update the page accordingly.

---

# SELECT vs. espresso.ai

URL: https://select.dev/docs/resources/comparisons/select-vs-espresso-ai

> SELECT vs. espresso.ai

## About SELECT

[SELECT](https://select.dev/) was founded in 2022 and is led [by experienced practitioners](https://select.dev/about) with deep expertise in optimizing Snowflake spend. Their SaaS web application helps users monitor, control and automatically lower spend. They integrate tightly with popular data tools such as dbt, Looker, Sigma (and more...) that sit on top of Snowflake by providing cost attribution to the workloads originating from these systems. In addition, SELECT automatically lowers Snowflake customer’s compute spend by 10-20% with no engineering effort required by the customer by automatically adjusting warehouse configurations in real time to improve efficiency.

## About espresso.ai

espresso.ai was founded in 2024 by former engineers from Google & Apple. Similar to [Keebo](https://select.dev/docs/select-vs-keebo), they are solely focused on applying automated cost optimizations (resizing warehouses, routing queries to different warehouses) versus an end to end platform for managing costs.

## Detailed Comparison

### Cost Visibility & Allocation

| Capability | SELECT | espressio.ai |
| --- | --- | --- |
| Out of the box spend dashboards | ✅ | ✅ |
| See costs in dollars, not credits | ✅ | ✅ |
| See cost per query | ✅ | ❌ |
| Analyze remaining contract usage and forecast | ✅ | ❌ |
| Connect multiple Snowflake accounts | ✅ | ✅ |
| Drill into cloud services cost drivers | ✅ | ❌ |
| Warehouse Utilization | ✅ | ❌ |
| Intelligent query grouping based on query similarity | ✅ | ❌ |
| Analyze Snowflake (Cortex) AI costs | ✅ | ❌ |
| See top users and queries per table | ✅ | ❌ |
| See costs by query tag | ✅ | ❌ |
| See costs by table accessed | ✅ | ❌ |
| See cost per Snowpipe | ✅ | ❌ |
| See cost by table | ✅ | ❌ |
| See query read/write costs by table | ✅ | ❌ |
| See cost per Task | ✅ | ❌ |
| See cost per Stored Procedure | ✅ | ❌ |
| See cost per Dynamic Table | ✅ | ❌ |
| See cost per dashboard | ✅ | ❌ |
| Advanced filtering capabilities | ✅ | ❌ |
| Understand total asset cost with lineage | ✅ | ❌ |
| Save frequently applied filters as views | ✅ | ❌ |
| Analyze costs by Snowflake object tags | ✅ | ❌ |

### Performance Monitoring

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| Analyze query performance by warehouse | ✅ | ✅ |
| Analyze query queuing by warehouse | ✅ | ❌ |
| Analyze data spillage warehouse | ✅ | ❌ |
| See historical query performance | ✅ | ❌ |
| See historical query stats (data scanned, pruning efficiency, spillage, etc.) | ✅ | ❌ |

### Savings & Recommendations

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| Automatically save costs through warehouse optimization | ✅ | ✅ |
| View and audit generated savings | ✅ | ✅ |
| Identify misconfigured warehouses | ✅ | ❌ |
| Query optimization recommendations | ✅ | ❌ |
| Under-utilized warehouses | ✅ | ❌ |
| Table clustering key analysis | ✅ | ❌ |
| Workloads with high spillage | ✅ | ❌ |
| Inefficient file sizes for data loading | ✅ | ❌ |
| Query warehouse sizing recommendations | ✅ | ❌ |
| Identify unused tables, schemas and databases | ✅ | ❌ |
| Exploding join detection | ✅ | ❌ |
| Identify tables with unnecessary data retention costssed | ✅ | ❌ |
| Filter recommendations by team, department, warehouse, or tags | ✅ | ❌ |

### Integrations

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| dbt core | ✅ | ❌ |
| dbt Cloud | ✅ | ❌ |
| Looker | ✅ | ❌ |
| Sigma | ✅ | ❌ |
| Mode | ✅ | ❌ |
| Hex | ✅ | ❌ |
| Periscope | ✅ | ❌ |
| Coalesce.io | ✅ | ❌ |
| Tableau | Coming soon... | ❌ |
| PowerBI | Coming soon... | ❌ |

### Alerting

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| Get notified of account level cost spikes | ✅ | ❌ |
| Send alerts to Email | ✅ | ❌ |
| Send alerts to Slack | ✅ | ❌ |
| Send alerts to Teams | ✅ | ❌ |
| Send alerts to OpsGenie | ✅ | ❌ |
| Send alerts to PagerDuty | ✅ | ❌ |
| Long running query alerts | ✅ | ❌ |
| Configure alerts for warehouses, users or queries | ✅ | ❌ |
| Define alerts with custom SQL queries | ✅ | ❌ |
| Customize the alert message & contents | ✅ | ❌ |
| Send alerts to multiple destinations | ✅ | ❌ |

### Cost Allocations

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| Allocate costs by database | ✅ | ❌ |
| Allocate costs by warehouse | ✅ | ✅ |
| Allocate costs by Snowflake user | ✅ | ❌ |
| Allocate costs using Snowflake object tags | ✅ | ❌ |
| Allocate AI costs by user, warehouse or LLM model | ✅ | ❌ |
| Allocate AI costs from query tags / query metadata | ✅ | ❌ |
| Allocate by dbt tags or resource names | ✅ | ❌ |
| Allocate costs by table/schema | ✅ | ❌ |
| Allocate costs by BI tool user | ✅ | ❌ |
| See version history of cost allocations | ✅ | ❌ |
| Manage cost allocations via JSON/YAML | ✅ | ❌ |

### Budgets

| Capability | SELECT | espresso.ai |
| --- | --- | --- |
| Set monthly budgets | ✅ | ❌ |
| Set yearly budgets | ✅ | ❌ |
| Set a budget for spend from multiple Snowflake accounts in your organization | ✅ | ❌ |
| Set a budget for individual warehouses, or groups of warehouses | ✅ | ❌ |
| Set a budget for individual Snowflake users, or groups of users | ✅ | ❌ |
| Get alerted if budget is exceeded | Coming soon... | ❌ |

### Security

| Capability | SELECT | espressio.ai |
| --- | --- | --- |
| Whitelisted IP address | ✅ | ? |
| Key-Pair Authentication Support | ✅ | ? |
| Flexible RBAC & roles assignment | ✅ | ? |
| Restrict access at the resource level | ✅ | ? |
| Automatically rotate keys via API | ✅ | ? |
| Integration with Okta SSO | ✅ | ? |
| Integration with Entra ID (Azure AD) SSO | ✅ | ? |
| Integration with OneLogin SSO | ✅ | ? |
| Integration with PingFed SSO | ✅ | ? |
| Automatically assign roles based on SSO groups | ✅ | ? |

### Pricing

SELECT offers flat rate pricing tailored to your Snowflake usage, at 4% of your annual Snowflake spend. You can pay with your existing Snowflake credits, and we always ensure we save you more than we charge, guaranteed.

espresso.ai charges ~40% of the savings they generate through their warehouse optimization feature.

## Frequently Asked Questions

### How does SELECT's pricing compare to Espresso's?

SELECT charges a flat rate instead of a % of savings. Learn why [here](https://select.dev/docs/comparisons/faq#why-doesnt-select-charge-a-of-savings).

### What are the benefits of using SELECT?

SELECT is an end-to-end Snowflake cost management and optimization platform. SELECT powers over 200 customers, helping them:

- Automatically save up to 20% through fully automated and real time warehouse optimizations
- Gain unparalleled cost visibility into all Snowflake workloads and spend drivers
- Optimize Snowflake resources through tailored insights & recommendations
- Monitor and control spend through flexible alerting
- Democratize spend management by providing users and teams throughout the organization with fine-grained access into their Snowflake usage

### How is SELECT different than espresso.ai?

SELECT and espresso.ai both offer a fully automated savings feature which yields cost savings with no effort.

In addition to fully automated savings, SELECT:

- Provides deep cost visibility into Snowflake workloads and spend drivers to help developers self diagnose issues and opportunities
- Surfaces detailed optimization opportunities that teams can action themselves. These are things that can't / shouldn't be automated, like dropping unused tables or adding a specific filter to a dashboard's query.
- Offers out-of-the-box flexible alerts to notify teams and users of spend anomalies
- Integrates with platforms that sit on top of Snowflake and drive compute costs

### What makes SELECT a better alternative to espresso.ai?

espresso.ai solely focuses on automated warehouse optimizations, which SELECT does a well with our [Automated Savings feature](https://select.dev/features/automated-savings).

In addition to Automated Savings, here is a summary of other key differentiators:

1. **Detailed insights & recommendations:** SELECT scans all Snowflake resources and queries for common anti-patterns and innefficiencies, and surfaces up detailed optimization recommendations to its end users, typically providing additional savings opportunities of 20-30%.
2. **Deeper cost visbility:** SELECT provides much deeper visibility into Snowflake usage, highlighting the annualized costs of all Snowflake resources (tables, tasks, LLM models) and making them readily accessible within a few clicks.
3. **Integrations:** SELECT integrates with common tools used on top of Snowflake that drive significant consumption (dbt, Looker, Sigma, etc.) and surfaces their costs along with optimization opportunities.
4. **Flexible & automated cost anomaly alerts:** SELECT allows customers to setup flexible anomaly alerts with a few clicks and allows users to send those alerts to Email, Slack, MS Teams, or other alerting destinations like Pager Duty. SELECT's alerts can be configured at a very granular level: by team, user, query, tags, and more.
5. **Cost visibility access controls:** Many Snowflake customers do not want to expose their total Snowflake bill to all users in the company. SELECT allows its customers to expose a subset of Snowflake costs from selected resources to each team/department .
6. **Battle tested:** SELECT product is used and battle tested by hundreds of companies every day.

## Commitment to Transparency

This comparison guide was put together based on publicly available documentation and customer references. If you spot an innaccuracy, please contact [hello@select.dev](mailto:hello@select.dev) and provide verifiable evidence to demonstrate your claim, and we will update the page accordingly.

---

# About

URL: https://select.dev/docs/resources/dbt-snowflake-monitoring/dbt-snowflake-monitoring

> Learn about the open-source dbt package written by SELECT

> [!NOTE]
> Looking to get started? Jump ahead to the [quickstart](https://select.dev/docs/dbt-snowflake-monitoring/quickstart) to get the dbt package installed in minutes, or checkout the [example queries](https://select.dev/docs/dbt-snowflake-monitoring/example-usage) to see what you can do with it!

## About

dbt-snowflake-monitoring is a **totally free**, [open-source](https://github.com/get-select/dbt-snowflake-monitoring) dbt package built by the SELECT team to help you monitor both Snowflake and dbt performance and costs. Use the package to:

- Understand Snowflake costs over time across a variety of useful dimensions (service, warehouse, database, etc.)
- Attribute costs within a warehouse to different users or query workloads using our [cost-per-query](https://select.dev/posts/cost-per-query) algorithm
- Identify unused tables and their associated storage costs
- Tag all dbt issued queries with [rich metadata](https://select.dev/docs/dbt#example-dbt-query-metadata), enabling you to understand dbt resource costs and performance over time

## Package Maintenance

Prior to releasing this package, [snowflake-spend](https://gitlab.com/gitlab-data/snowflake_spend) by the Gitlab data team was the only package available for monitoring Snowflake spend. According to their README, the package is currently maintained by the Gitlab data team, but there does not appear to be any active development in it (as of January 2023).

The `dbt-snowflake-monitoring` package is actively developed & maintained by the [SELECT](https://select.dev/) team. The package goes beyond modeling warehouse spend - it calculates [cost per query](https://select.dev/posts/cost-per-query) using the methodology described [here](https://select.dev/posts/cost-per-query) and all billable Snowflake components.

---

# Quickstart

URL: https://select.dev/docs/resources/dbt-snowflake-monitoring/quickstart

> A quickstart guide from SELECT

## Step 1: Grant access

Grant dbt's role access to the `snowflake` database:

```sql
grant imported privileges on database snowflake to role your_dbt_role_name;
```

## Step 2: Update packages.yml

Add the package to your `packages.yml` file:

```yaml
packages:
  - package: get-select/dbt_snowflake_monitoring
    version: [">=5.0.0", "<6.0.0"] # We'll never make a breaking change without creating a new major version.
```

## Step 3: Enable query tags

1. To attribute costs to individual models via the `dbt_metadata` column in the `query_history_enriched` model, query comments and tags are added to all dbt-issued queries. Both query comments and tags are needed to collect the required metadata for the `dbt_queries` model. First, make sure to remove any existing `+query_tag: dbt` from your `dbt_project.yml`, then add the query tags with one of these two options:

Option 1: If running dbt >= 1.2, simply configure the dispatch search order in `dbt_project.yml`.

```yaml
dispatch:
  - macro_namespace: dbt
    search_order:
      - <YOUR_PROJECT_NAME>
      - dbt_snowflake_monitoring
      - dbt
```

Option 2: If running dbt < 1.2, create a folder named `macros` in your dbt project's top level directory (if it doesn't exist). Inside, make a new file called `query_tags.sql` with the following content:

```sql
{% macro set_query_tag() -%}
{% do return(dbt_snowflake_monitoring.set_query_tag()) %}
{% endmacro %}

{% macro unset_query_tag(original_query_tag) -%}
{% do return(dbt_snowflake_monitoring.unset_query_tag(original_query_tag)) %}
{% endmacro %}
```

## Step 4: Enable query comments

1. To configure the query comments, add the following config to `dbt_project.yml`.

```yaml
query-comment:
  comment: '{{ dbt_snowflake_monitoring.get_query_comment(node) }}'
  append: true # Snowflake removes prefixed comments.
```

That's it! All dbt-issued queries will now be tagged and start appearing in the `dbt_queries.sql` model.

## Building the models

By default, the models in this package will automatically get built during any full run of your dbt project. If you would like to manually create the models, or run them on a separate schedule, you can do so using [dbt's node selection syntax](https://docs.getdbt.com/reference/node-selection/syntax):

`dbt run -s dbt_snowflake_monitoring`

## dbt Cloud URLs

If you're using dbt Cloud, columns `dbt_cloud_job_url` and `dbt_cloud_run_url` can be configured in the `dbt_queries` model. To do so, set the variable `dbt_cloud_account_id`. This id can be retrieved from in between `/deploy/` and `/projects/` in any dbt Cloud URL.

By default, the URL prefix of `https://cloud.getdbt.com/deploy/` is used. If you're using a different region of dbt Cloud, this prefix can be overridden by specifying the `dbt_cloud_url` variable.

## Troubleshooting

### No data populated in the tables

If you aren't seeing data populated in the tables (example [here](https://github.com/get-select/dbt-snowflake-monitoring/issues/145)), it is possible that your account is missing `ORGADMIN` access. Please see [here](https://select.dev/docs/snowflake-metadata-access#step-5-ensure-your-account-has-orgadmin-access) for instructions on enabling `ORGADMIN` access in your Snowflake account.

---

# Example Usage

URL: https://select.dev/docs/resources/dbt-snowflake-monitoring/example-usage

> See some examples queries you can run on top of the models.

## Introduction

The following sample queries are useful starting points for performing analysis on top of the models provided by this package. Before running the queries, make sure that the default database and schema have been set:

```sql
use database [dbt-snowflake-monitoring model database]
use schema [dbt-snowflake-monitoring model schema]
```

## Reproducing the Monthly Snowflake Invoice

The `daily_spend` model reproduces the monthly Snowflake invoice precisely, broken down by the same service categories. The `daily_spend` model contains additional columns for `warehouse_name`, `database_name` and `storage_type` to further break down spend (see more examples below).

```sql
select
    date_trunc(month, date)::date as month,
    service,
    sum(spend)
from daily_spend
group by 1, 2
```

## Warehouse Costs

### Monthly Spend per Warehouse

This query uses the `daily_spend` model to explore spend by warehouse name. Some queries don't require a running warehouse, which is why you'll see a row for `warehouse_name` = 'Cloud Services Only'.

```sql
select
    date_trunc(month, date)::date as month,
    warehouse_name,
    sum(spend_net_cloud_services) as spend
from daily_spend
where service in ('Compute', 'Cloud Services')
group by 1, 2
```

## Storage Costs

### Monthly Spend per Database

This query uses the `daily_spend` model to explore table spend by database. Note that this also returns storage costs for data held in stages (user and table stages or internal named stages), see the `storage_type` column.

```sql
select
    date_trunc(month, date)::date as month,
    service,
    storage_type,
    database_name,
    sum(spend)
from daily_spend
where service in ('Storage')
group by 1, 2, 3, 4
```

### Unused Tables

This query uses the `query_base_table_access` model (available as of 4.0.0) to identify tables which have not been queried in the last 30 days. The total storage costs of each table are also shown, which uses the account's current storage rate (usually between $20-$40 per TB per month). See [this post](https://select.dev/posts/snowflake-unused-tables#direct-versus-base-objects-accessed) for more details on the difference between base versus direct tables accessed.

## Query Cost Attribution

Snowflake bills for the number of seconds that a warehouse is running, not by query. Query cost attribution helps understand how queries are contributing to warehouse active time. Removing a query will not reduce the bill by the exact amount attributed to the query if other queries are running at the same time and causing the warehouse to stay active.

Cloud services credits are only billed if they exceed 10% of the compute credit consumption on a given day. The cost per query model accounts for this, but current day values will change up until the end of the day.

### Top 10 costliest queries in the last 30 days

```sql
with
max_date as (
    select max(date(end_time)) as date
    from query_history_enriched
)
select
    md5(query_history_enriched.query_text_no_comments) as query_signature,
    any_value(query_history_enriched.query_text) as query_text,
    sum(query_history_enriched.query_cost) as total_cost_last_30d,
    total_cost_last_30d*12 as estimated_annual_cost,
    max_by(warehouse_name, start_time) as latest_warehouse_name,
    max_by(warehouse_size, start_time) as latest_warehouse_size,
    max_by(query_id, start_time) as latest_query_id,
    avg(execution_time_s) as avg_execution_time_s,
    count(*) as num_executions
from query_history_enriched
cross join max_date
where
    query_history_enriched.start_time >= dateadd('day', -30, max_date.date)
    and query_history_enriched.start_time < max_date.date -- don't include partial day of data
group by 1
order by total_cost_last_30d desc
limit 10
```

## dbt Cost Attribution

### Top 10 costliest dbt models in the last 30 days

```sql
with
max_date as (
    select max(date(end_time)) as date
    from dbt_queries
)
select
    dbt_queries.dbt_node_id,
    sum(dbt_queries.query_cost) as total_cost_last_30d,
    total_cost_last_30d*12 as estimated_annual_cost
from dbt_queries
cross join max_date
where
    dbt_queries.start_time >= dateadd('day', -30, max_date.date)
    and dbt_queries.start_time < max_date.date -- don't include partial day of data
group by 1
order by total_cost_last_30d desc
limit 10
```

### Daily dbt model running costs

```sql
select
    date(start_time) as date,
    sum(query_cost) as cost
from dbt_queries
where dbt_node_id = '<dbt model node id>'
group by 1
order by 1 desc
```

---

# Contributing

URL: https://select.dev/docs/resources/dbt-snowflake-monitoring/contributing

> Contributing set-up guide

## Initial setup

1. Install pipx

```bash
pip install pipx
pipx ensurepath
```

2\. Install tox

```bash
pipx install tox
```

3\. Install pre-commit

```bash
pipx install pre-commit
pre-commit install
```

4\. Configure your profile (follow the prompts)

```bash
dbt init
```

## Adding a CHANGELOG Entry

We use changie to generate CHANGELOG entries. Note: Do not edit the CHANGELOG.md directly. Your modifications will be lost.

Follow the steps to [install changie](https://changie.dev/guide/installation/) for your system.

Once changie is installed and your PR is created, simply run `changie new` and changie will walk you through the process of creating a changelog entry. Commit the file that's created and your changelog entry is complete!

## SQLFluff

We use SQLFluff to keep SQL style consistent. By installing `pre-commit` per the initial setup guide above, SQLFluff will run automatically when you make a commit locally. A GitHub action automatically tests pull requests and adds annotations where there are failures.

Lint all models in the /models directory:

```bash
tox -e lint_all
```

Fix all models in the /models directory:

```bash
tox -e fix_all
```

Lint (or substitute lint to fix) a specific model:

```bash
tox -e lint -- models/path/to/model.sql
```

Lint (or substitute lint to fix) a specific directory:

```bash
tox -e lint -- models/path/to/directory
```

### Rules

Enforced rules are defined within `tox.ini`. To view the full list of available rules and their configuration, see the [SQLFluff documentation](https://docs.sqlfluff.com/en/stable/rules.html).