# AirFlow-ML-Data-Integration

An example Apache Airflow project that orchestrates a daily ETL pipeline fetching weather data from a free API, cleaning and transforming it, and storing it in PostgreSQL, ready for downstream machine learning tasks.

This project demonstrates orchestrating a daily ETL pipeline with Apache Airflow: fetching weather data from the free OpenWeather API, cleaning and transforming it, and storing the result in a PostgreSQL database, as a reproducible, extensible example for training machine learning models on weather data later.

A quickstart clones the repository, creates a Python virtual environment, installs dependencies, and runs a test suite that verifies the Airflow DAG parses correctly without needing an API key or a database. The full stack, including Airflow and PostgreSQL, can also be run with Docker Compose after adding a free OpenWeather API key to a local .env file, which starts Airflow with the weather_data_pipeline DAG and both connections already registered.

The README also documents manual setup instructions for Airflow, the database, the project folder layout, an IDE and the Airflow-to-database and Airflow-to-API connections, for readers who prefer not to use Docker.

Website: https://github.com/gbazad93/AirFlow-ML-Data-Integration