Back to directory

ppgrid

An open-source tool that turns tens of millions of point-data rows into a smooth, adaptive-resolution raster in seconds, using a push-pull mipmap approximation of inverse distance weighting that stays fast as the dataset grows.

ppgrid

ppgrid is an open-source tool, described in a blog post by its author, for turning very large point datasets, tens of millions of rows of coordinates paired with a numeric value, into a smooth, adaptive-resolution raster suitable for mapping, in seconds rather than the days that tools such as ArcGIS, QGIS or GRASS can take on the same data. It approximates inverse distance weighting through a push-pull mipmap technique, an approach borrowed from image processing, so that its running time stays roughly flat as the input grows.

The author built it after repeatedly needing to visualize large point datasets, such as tens of millions of rows of modelled risk data, as part of day-to-day geographic-information-system work, and found existing tools too slow or otherwise unsuited to the task. The post frames the tool as part of a broader effort to make large spatial datasets easier to communicate visually and to keep the geographic-information-system tooling ecosystem more open.

More in Data engineering

See all 98
  • Scope Lineage

    github.com

    An offline static analyzer that turns Spark and Hive SQL into structured, versioned column-lineage and diagnostics files, so AI agents and knowledge bases can reason from addressable, checkable facts instead of guessing from raw SQL.

  • A pair of Colab notebooks demonstrating a retrieval-augmented generation workflow for extracting structured datasets from scientific paper abstracts, applied here to hydrogen-storage properties of metal alloys and hydrides.

  • Gnosis

    github.com

    A multimodal semantic search pipeline that ingests documents and images through Apache Airflow DAGs, extracts text and generates image descriptions, embeds the content, and serves cosine-similarity search over a Qdrant vector index with PostgreSQL for metadata.