Skip to content

Realtime monitoring of running, queued and blocked queries in Snowflake

License

Notifications You must be signed in to change notification settings

littleK0i/SnowKill

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SnowKill

PyPI

SnowKill is a near real-time query monitoring tool for Snowflake Data Cloud.

SnowKill helps to detect potential problems with queries which are currently running. It analyzes query stats and plans, detects bad patterns, generates notifications and optionally "kills" some queries automatically.

The core logic of SnowKill relies on internal REST API calls instead of SQL queries. It does not require an active warehouse to run, which makes it possible to maintain the constant monitoring almost free of charge.

SnowKill has programmatic access to query plan from "Query Profile" page in SnowSight. SnowKill also has access to information about locks and tries to report the exact reason for transaction collisions.

SnowKill operates on present data, which normally allows it to react much faster relative to conventional monitoring tools operating on past data from QUERY_HISTORY and GET_QUERY_OPERATOR_STATS.

How does it work?

  1. Load list of queries which are currently RUNNING, QUEUED or BLOCKED.
  2. Load additional information about query plans and active locks, if necessary.
  3. Check queries against list of fully customizable conditions.
  4. Optionally terminate matched queries exceeding specific thresholds.
  5. Detect and skip previously reported queries, avoid duplicates.
  6. Send notifications about newly matched queries (via Slack, Email, etc.).

Notification example

Notification example 1

Quick links

Built-in conditions:

Built-in formatters:

Built-in storages:

Future plans

  • More conditions, formatters, storages.
  • Automated testing on push.
  • Maybe some conditions for groups of queries, e.g. "more than 4 queries are queued on warehouse".

Issues? Questions? Feedback?

Please use GitHub "Issues" to report bugs and technical problems.

Please use GitHub "Discussions" to ask questions and provide feedback.

Created by

Vitaly Markov, 2023

Enjoy!