Skip to content

A package for crawling THREDDS catalogs

License

Notifications You must be signed in to change notification settings

TerriaJS/thredds-catalog-crawler

Repository files navigation

thredds-catalog-crawler

A package for crawling THREDDS catalogs, including nested catalogs.

Install

npm install thredds-catalog-crawler --save

Basic usage

Using async/await & modules

import threddsCrawler from 'thredds-catalog-crawler'

async function crawlThredds() {
    const catalog = await threddsCrawler('http://something/thredds/catalog/my/catalog.xml')
    await catalog.loadAllNestedCatalogs()
    const datasets = catalog.getAllChildDatasets()
}
crawlThredds()

Using traditional promises & modules

const threddsCrawler = require('thredds-catalog-crawler')

threddsCrawler('http://something/thredds/catalog/my/catalog.xml')
.then(function (catalog) {
    catalog.loadAllNestedCatalogs()
})

Dependencies

This package uses the Fetch API. If you're using this library in NodeJS you'll need a fetch polyfill like isomorphic-fetch or if you want to support older browsers we'd suggest unfetch. You may also need a URL API polyfill such as url-polyfill for older browsers.

import 'isomorphic-fetch'
import 'url-polyfill'
import threddsCrawler from 'thredds-catalog-crawler'

API

Catalog

A Catalog represents the root object in a THREDDS heirarchy. Catalogs can contain datasets, as well as references to other catalogs.

Properties

Property Type Description
url String Url of the catalog
name String Name of the catalog
isLoaded Boolean Indicates whether the endpoint has been loaded. If true then datasets and services will be populated, and limited catalog information. If false then most information will be unavailable.
datasets Array An array of datasets found in the catalog
catalogs Array An array of nested catalogs found in the catalog
services Object An object containing the available services for a catalog, organised by service type
hasDatasets Boolean Indicates whether the catalog contains datasets directly
hasNestedCatalogs Boolean Indicates whether the catalog contains nested catalogs
wmsBase String The base url for a WMS service. If no WMS support is available then null
supportsWms Boolean Does a catalog have a WMS service for datasets

Methods

Method Returns Description
loadAllNestedCatalogs N/A Loads all nested catalogs recurrsively
loadNestedCatalogById(id) Catalog Crawls a single nested catalog based on a id string
getAllChildDatasets Array[Dataset1, Dataset2...] Returns a flattened array of datasets from all nested catalogs which have been loaded.

Dataset

A Dataset can contain datasets, as well as references to other catalogs.

Properties

Property Type Description
id String An id of the dataset
name String Name of the dataset
parent Object A catalog or another dataset
urlPath String Partial url path of the dataset
isParentDataset Boolean Indicates whether this dataset contains other datasets
datasets Array[Dataset] An array of nested datasets
wmsUrl String A url for the GetCapabilities endpoint for the dataset. Constructed from the dataset and its parent catalog item
parentCatalog Catalog A reference to the datasets parent catalog
supportsWms Boolean Does this dataset have a WMS available

Methods

Method Returns Description
loadAllNestedCatalogs N/A Loads all nested catalogs recurrsively

About

A package for crawling THREDDS catalogs

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published