Fetchers

In DBnomics, data acquisition is done by fetchers, small programs that download data from the provider infrastructure, and convert it to a common data model and format.

The source code of the fetchers written by DBnomics core team is here on DBnomics GitLab instance. You can also look at the dashboard.

In general each fetcher covers a provider.

Components

Pipeline

Fetchers are scheduled on a regular basis in order to keep DBnomics data up to date.

As explained on the architecture page, the DBnomics project relies heavily on GitLab and its continuous integration system as a development platform.