GovData Germany Scraper - CKAN Open Data Portal

resource-row

One file or service — its URL, normalised format, MIME type, byte size, checksum, licence and dates — carrying the identity, publisher, themes and keywords of the dataset it belongs to.

contact-row

One deduplicated publishing body or contact person: name, email, phone, postal address, its organization and source portal, how many of the scanned datasets it is responsible for and which themes they cover.

organization-row

One of the publishing organizations on the portal, with its total dataset count, how many match the filters on this run, its creation date and its page on GovData.

category-row

One EU data theme with its German and English titles, its data-theme URI, and dataset counts both catalogue-wide and within the current query.

tag-row

One keyword from the catalogue vocabulary with the number of datasets carrying it, ranked, and honouring whatever filters the run set.

format-row

One distribution format with its dataset count, whether it is machine-readable or a service, and every raw spelling the catalogue uses for it with each variant's own count.

license-row

One licence from the registered vocabulary with its title, URL, Open Definition and Open Source conformance, and how many datasets declare it at package level.

link-check

Optional. One distribution URL fetched to see whether it is actually alive: HTTP status, content type, byte size, redirect target and latency. GovData harvests metadata and never re-verifies links — 5.3% of a spread sample were dead 404s.

resource-preview

Optional. The first 64 KB of a CSV, TSV, JSON, GeoJSON or XML distribution read with a range request, returning its real column names, delimiter, character encoding and a sample row. Not charged for formats that are skipped without a request.

organization-profile

Optional. One extra query per organization returning what it actually publishes: its EU themes, its top distribution formats and its top keywords, each with counts.