Codes/Notebooks for AI Projects

Learn the basic commands to use Pandas in Jupyter-Notebook to accomplish the most important Data Enginnering tasks. Read the underlying article on Medium:

Learning Data Science with Hacktiv8 using Python, Jupiter Notebook, and else

This project showcases the implementation of a data engineering solution using Python within a Jupyter Notebook environment.

Este projeto se trata de um simples etl com um dataset com as variações dos preços diários do bitcoin no período de 2020-2022. Os códigos do notebook foram desenvolvidos tanto em pyspark quanto em sql, numa simulação de solucão referentes a perguntas de négocio.

This repository contains my hands-on journey through the Big Data training by NTI. Includes labs, notebooks, and notes on tools like HDFS, Spark, Kafka, Flink, Hive, HBase and more.

This project demonstrates a data engineering pipeline using FastAPI, MySQL, and Jupyter Notebook. It processes raw transactional data, cleans and transforms it using Pandas, and loads it into a structured MySQL database. A FastAPI-based REST API allows querying customer summaries and product sales data efficiently.

The pipeline integrates multiple data sources web-scraped community areas, taxi trip data from the Chicago Open Data API, and weather data from Open-Meteo. It cleans, normalizes, and merges them with Python, loads the processed results to AWS S3 via Lambda, and provides interactive analysis in Jupyter notebooks.