We’re excited to announce that SELECT users can now leverage additional metadata associated with query workloads to better understand and attribute their Snowflake consumption. If you’re using our dbt or custom workloads integrations, this will apply to you!

Leveraging dbt meta & tags

Considering the example dbt_project.yml below, which implements a combination of dbt meta fields and tags.

# dbt_project.yml
name: 'select_dbt'
version: '1.0.0'
config-version: 2

models:
	+meta:
	   team: 'data' # all models get tagged with team = 'data' by default
  engineering:
    +meta:
      team: 'engineering'
	marketing:
    +meta:
      team: 'marketing'
  forecasts:
    +meta:
      team: 'finance'
		+tags:
			- "sensitive"
			- "critical"

The file structure for this dbt project would look something like what is shown below. All resources under the engineering directory would be get a team meta field added with the value engineering. Similarly, the one model in the forecasts directory would get a team field of finance added. It would also have following tags: ["sensitive", "critical"].

.
├── dbt_project.yml
├── packages.yml
└── models/
    ├── engineering/
    |   ├── syncs.sql
    |   └── ai.sql
    ├── marketing/
    |   ├── attribition.sql
    |   └── page_clicks.sql
    └── forecasts/
        └── forecast.sql

To expose dbt meta fields in SELECT, you can go into the settings and specify which fields you want enabled. No such specification is required for dbt tags.

Once added, you can filter and slice by this metadata throught the UI!

Leveraging custom metadata fields

Custom workloads allow users to tag their queries with comments or Snowflake query tags. Here’s an example query comment SELECT has for one of its custom workloads:

select ...
from queries
group by all
order by sum_query_cost desc nulls last
limit 1000

/* {
	"select_schema_version": "1.0.0", 
	"app": "SELECT_WEB_APP", 
	"workload_id": "DailyWorkloadPoPQuery", 
	"environment": "prod", 

	"tenant_id": "scwxhob-ad38017", 
  "referrer_base_url": "https://select.dev/app/scwxhob-ad38017/workloads",
	"is_admin_user": true,
	"user_id": "google-oauth2|115381118870474481840"
}*/

Custom workload fields like app, workload_id and environment are already available for filtering & slicing in the UI out of the box. To enable the same behaviour for other fields you specify in your custom workload query comment/tag, head to Settings → Custom and specify the keys you want enabled.

Once added, you can filter and slice by this metadata throught the UI!

select * from credit_cards

/* { 
    "select_schema_version": "1.0.0",
    "app": "generated_tables",
    "workload_id": "my_etl_job",

    "tags": ["critical", "sensitive"], 
    "meta": {"task_key": "A1SF249X", "attempt_number": 1} } 
*/