Annual Revenue Vs. Executive Pay for Recipients of U.S. Federal Funds; uses Scala Spark in Zeppelin notebook.

A collection of Spark/Scala example programs

This example demonstrates creating a Lambda function in Python language and deploy it using Terraform. The example also demonstrates the use of Pandas, S3, Parquet & Redshift.

Export data from MySQL to parquet file.

Daily scraps the data from rpi-imager-stats

Организация Data Lake

A simple toolkit to transform datasource generate by img2dataset from parquet file to Huggingface dataset.

Example usage of python with parquet for IoT data storage and analytics

Dresser-le-procureur.com

Package to convert parquet files into hyper files used in Tableau Server

a tiny sample about how to build a parquet file from whatever you need.

Writing Parquet files in Scala using AWS Firehose and Glue

Convert Parquet files to CSV stored in S3 Using AWS Glue

Substreams Sink Parquet

Desktop Application that makes the conversion of mass flat files to parquet easy

Parquet lib for clojure, powered by Rust.

Project involved the development of a data pipeline using airflow and python. The data pipeline ingested trending movies' and distributors' data from imdb and box office, cleansed, formatted, combined and indexed the data on elastic search. Also, a dashboard was created from the data using kibana analytics. The tools and libraries used in this p…

Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .

DSS plugin for fast loading between Snowflake and HDFS