# Big Data for beginners

A set of self-contained Jupyter notebook tutorials introducing Hadoop, Spark and MapReduce, most runnable directly on Google Colab with no separate setup.

These notebooks walk a beginner through setting up a single-node Hadoop cluster on Google Colab and running basic HDFS and MapReduce commands, then through running Apache Spark in standalone mode, including a Bigtop-distribution variant and a version that also demonstrates Spark on SLURM. Separate MapReduce notebooks progress from a plain bash implementation of the classic word-count example to using Python's mrjob library for the same task, building an understanding of the model before relying on a library that hides the mechanics.

Most notebooks are self-contained with their own setup instructions and can run on Google Colab or in a virtual Ubuntu machine or container, making the underlying cluster software approachable without a pre-existing Hadoop or Spark installation.

It is written as Jupyter notebooks and licensed under MIT.

Website: https://github.com/groda/big_data