Dynamic data testing engine based on pySpark
-
Updated
Aug 10, 2019 - Jupyter Notebook
Dynamic data testing engine based on pySpark
National Grid ( Python, SQL Server, SSIS, SSRS, Tableau, Power BI, SQL Server Import Export Wizard, Data Validations, Data Integrations, Data Conversions )
A sample repository showcasing, implementation of testing for ETL pipeline developed with Apache Spark
This project creates machine learning models capable of classifying candidate exoplanets from the raw dataset from NASA Kepler Space Telescope
This library is inspired by the Great Expectations library. The library has made the various expectations found in Great Expectations available when using the inbuilt python unittest assertions.
Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes
Various files useful for manual testing and test automation etc.
I'm learning how to use dbt with BigQuery so I can apply that knowledge wherever we end up working. It seems like a good DWH interface tool to know for data transformation and testing, and allows me to solidify concepts of testing in data ops.
Software Testing in Open Source and Data Science: A talk delivered at the Data Umbrella speaker series
Develop a data science project using historical sales data to build a regression model that accurately predicts future sales. Preprocess the dataset, conduct exploratory analysis, select relevant features, and employ regression algorithms for model development. Evaluate model performance, optimize hyperparameters, and provide actionable insights.
Develop a data science project using historical sales data to build a regression model that accurately predicts future sales. Preprocess the dataset, conduct exploratory analysis, select relevant features, and employ regression algorithms for model development. Evaluate model performance, optimize hyperparameters, and provide actionable insights.
⚡ Prevent downstream data quality issues by integrating the Soda Library into your CI/CD pipeline.
Documentation for Data Caterer
Data generation and validation tool for any data source
Example API implementation for Data Caterer
Code review for data in dbt
data and pipeline testing with and for SQL
re_data - fix data issues before your users & CEO would discover them 😊
Great Expectations Airflow operator
re_data - fix data issues before your users & CEO would discover them 😊
Add a description, image, and links to the data-testing topic page so that developers can more easily learn about it.
To associate your repository with the data-testing topic, visit your repo's landing page and select "manage topics."