ArchitectureComponentsDXP Frontend

DXP Frontend

Purpose

The public face of NESO. Where end users discover and search datasets, preview resources, and start downloads.

  • Render the catalogue: home, search, dataset, resource preview pages.
  • Drive a great search experience (filters, facets, sorting).
  • Render previews (CSV table, GeoJSON map, JSON viewer).
  • Surface persistent dataset URLs for citation.
  • Handle public + authenticated end-user flows (sign in, profile, API tokens).

Tech stack

LayerTech
CMS / FrameworkDrupal
Metadata ingestionDrupal Feeds module (pulls from CKAN API)
Data previewData Explorer library (calls CKAN datastore_search directly from the browser)
Entity cacheDrupal database + Memcache
Page cacheVarnish, then Cloudflare
RevalidationEvery 5 minutes, a scheduled job pulls metadata from CKAN API

CKAN APIs consumed

The Feeds-driven scheduled job and the in-browser data explorer call the following CKAN endpoints:

  • current_package_list_with_resources
  • datapackage_show
  • datastore_search
  • datastore_search_sql
  • organization_list
  • resource_show
  • resource_view_list
  • show_popular_datasets
  • tag_list

Data flow & caching

  • Metadata sync. A scheduled job runs every 5 minutes, pulling metadata from CKAN via the Feeds module and persisting it as Drupal entities.
  • Entity cache. Drupal entities are cached in the database and Memcache, so page renders do not hit CKAN.
  • HTML cache. Rendered pages are cached in Varnish, then Cloudflare in front of that.
  • Live data preview. When a user opens a data file page, the Data Explorer library calls CKAN datastore_search directly from the browser — this path is not cached by Drupal/Varnish.
  • Invalidation. Each 5-minute sync updates the Drupal entities, which triggers cache invalidation downstream. Effective freshness for catalogue metadata is therefore ~5 minutes; data previews are live.

Repo location

Managed by the NESO DXP team; Code lives within their repo.

See also


Last reviewed: 2026-05-04

Built with LogoFlowershow