Configuration Reference

This page contains the list of all available Airflow configurations for the apache-airflow-providers-openlineage provider that can be set in the airflow.cfg file or using environment variables.

Note

The configuration embedded in provider packages started to be used as of Airflow 2.7.0. Previously the configuration was described and configured in the Airflow core package - so if you are using Airflow below 2.7.0, look at Airflow documentation for the list of available configuration options that were available in Airflow core.

Note

For more information see Setting Configuration Options.

Sections:

[openlineage]

This section applies settings for OpenLineage integration. For backwards compatibility with openlineage-python one can still use openlineage.yml file or OPENLINEAGE_ environment variables. However, below configuration takes precedence over those. More in documentation - https://openlineage.io/docs/client/python#configuration.

config_path

Path to YAML config. This provides backwards compatibility to pass config as openlineage.yml file.

Type

string

Default

''

Environment Variable

AIRFLOW__OPENLINEAGE__CONFIG_PATH

disable_source_code

If disabled, OpenLineage events do not contain source code of particular operators, like PythonOperator.

Type

boolean

Default

None

Environment Variable

AIRFLOW__OPENLINEAGE__DISABLE_SOURCE_CODE

disabled

Set this to true if you don’t want OpenLineage to emit events.

Type

boolean

Default

False

Environment Variable

AIRFLOW__OPENLINEAGE__DISABLED

disabled_for_operators

New in version 1.1.0.

Semicolon separated string of Airflow Operator names to disable

Type

string

Default

''

Environment Variable

AIRFLOW__OPENLINEAGE__DISABLED_FOR_OPERATORS

Example

airflow.operators.bash.BashOperator;airflow.operators.python.PythonOperator

extractors

Semicolon separated paths to custom OpenLineage extractors.

Type

string

Default

''

Environment Variable

AIRFLOW__OPENLINEAGE__EXTRACTORS

Example

full.path.to.ExtractorClass;full.path.to.AnotherExtractorClass

namespace

OpenLineage namespace

Type

string

Default

None

Environment Variable

AIRFLOW__OPENLINEAGE__NAMESPACE

Example

food_delivery

transport

OpenLineage Client transport configuration. It should contain type and additional options per each type.

Currently supported types are:

  • HTTP

  • Kafka

  • Console

Type

string

Default

''

Environment Variable

AIRFLOW__OPENLINEAGE__TRANSPORT

Example

{"type": "http", "url": "http://localhost:5000"}

Was this entry helpful?