FHIR pipeline: a smart ETL to standardize health data
Fhir-Pipe is an ETL tool backed by IA solutions, which makes standardisation of health data easy. It extracts and converts data from SQL databases into the standardized health format FHIR, using mapping rules build from the pyrog interface.
Mapping rules
The mapping rules are provided through the pyrog graphql API. To each FHIR Resource (e.g. Patient) corresponds a set of rules, and each attribute of the ressource (like patient.name.firstname) has a mapping instruction which details which DATABASE/TABLE/COLUMN to select and which processing scripts to apply, as the data might need to be cleaned. The list of generic and health care specific scripts is available at arkhn/scripts.
Goal of fhir-pipe
Fhir-Pipe is an ETL which is able to parse complex mapping rules which may describe intricated relations between tables. It is agnostic of the type of SQL databases used as input for its extraction, and uses Spark to perform highly scalable processing of data. The FHIR objects created are stored in a distributed file system to meet strict performance requirements.
Get started
Read our guide to start standardizing health data!
Contribute
We have reported several issues with the label Good first issue which can be a good way to start! You can also join our Slack to contact us if you have trouble or questions :)
If you're enthusiastic about our project,