Skip to content

GuinsooLab/darkseal

badge
logo

Darkseal

Unlock the value of data assets with an end-to-end metadata platform that includes data discovery, governance, data quality, observability, and people collaboration.

Quickstart

wget https://raw.githubusercontent.com/GuinsooLab/darkseal/master/docker-compose.yml
docker-compose up -d

For more information, please referer to here.

Main Features

  • Data Trust
  • Data Documentation
  • Data Governance
  • Discovery & Collaboration
  • Teams & Users with Roles and Policies
  • Data Glossaries
  • Data Insights
  • Data Lineage
  • Security & Compliance
  • Alerts & Notifications

Core Components

Darkseal includes the following:

  • Metadata schemas - Defines core abstractions and vocabulary for metadata with schemas for Types, Entities, Relationships between entities. This is the foundation of the Open Metadata Standard.
  • Metadata store - Stores metadata graph that connects data assets, user, and tool generated metadata.
  • Metadata APIs - For producing and consuming metadata built on schemas for User Interfaces and Integration of tools, systems, and services.
  • Ingestion framework - A pluggable framework for integrating tools and ingesting metadata to the metadata store. Ingestion framework already supports well know data warehouses - Google BigQuery, Snowflake, Amazon Redshift, and Apache Hive, and databases - MySQL, Postgres, Oracle, MSSQL, and Guinsoo.
  • Metadata User Interface - One single place for users to discover, and collaborate on all data.

Snapshot & Gifs

Database Service

database_service

Workflow DAG

workflow_dag

Data Tables

data_tables

Table Columns

table_columns

Stream Lineage

stream_lineage

Documentation & Supports

Contributors

We ❤️ all contributions, big and small! Check out our CONTRIBUTING guide to get started and let us know how we can help.

Don't want to miss anything? Give the project a ⭐ 🚀

License

Darkseal is released under Apache License, Version 2.0

license