DXP Frontend
The public-facing data experience portal where end users search, browse, and download.
DXP Frontend
Purpose
The public face of NESO. Where end users discover and search datasets, preview resources, and start downloads.
- Render the catalogue: home, search, dataset, resource preview pages.
- Drive a great search experience (filters, facets, sorting).
- Render previews (CSV table, GeoJSON map, JSON viewer).
- Surface persistent dataset URLs for citation.
- Handle public + authenticated end-user flows (sign in, profile, API tokens).
Tech stack
| Layer | Tech |
|---|---|
| CMS / Framework | Drupal |
| Metadata ingestion | Drupal Feeds module (pulls from CKAN API) |
| Data preview | Data Explorer library (calls CKAN datastore_search directly from the browser) |
| Entity cache | Drupal database + Memcache |
| Page cache | Varnish, then Cloudflare |
| Revalidation | Every 5 minutes, a scheduled job pulls metadata from CKAN API |
CKAN APIs consumed
The Feeds-driven scheduled job and the in-browser data explorer call the following CKAN endpoints:
current_package_list_with_resourcesdatapackage_showdatastore_searchdatastore_search_sqlorganization_listresource_showresource_view_listshow_popular_datasetstag_list
Data flow & caching
- Metadata sync. A scheduled job runs every 5 minutes, pulling metadata from CKAN via the Feeds module and persisting it as Drupal entities.
- Entity cache. Drupal entities are cached in the database and Memcache, so page renders do not hit CKAN.
- HTML cache. Rendered pages are cached in Varnish, then Cloudflare in front of that.
- Live data preview. When a user opens a data file page, the Data Explorer library calls CKAN
datastore_searchdirectly from the browser — this path is not cached by Drupal/Varnish. - Invalidation. Each 5-minute sync updates the Drupal entities, which triggers cache invalidation downstream. Effective freshness for catalogue metadata is therefore ~5 minutes; data previews are live.
Repo location
Managed by the NESO DXP team; Code lives within their repo.
See also
Last reviewed: 2026-05-04