Fetchers

In DBnomics, data acquisition is done by fetchers, small programs that download data from the provider infrastructure, and convert it to a common data model and format. Each fetcher covers a particular data provider.

Components

Diagram of the components of a fetcher

Pipeline

Fetchers are scheduled on a regular basis in order to keep DBnomics data up to date.

As explained on the architecture page, the DBnomics project relies heavily on GitLab and its continuous integration system as a development platform.

Diagram of the pipeline of a fetcher

Contribute

If you want to contribute to DBnomics by adding support for a new data provider, or enhance an existing one, please read the dedicated dbnomics-fetcher-toolbox documentation.

DBnomics core team is open to external contributions and we make increasing efforts to ease the contribution path. Thank you for your interest in DBnomics!