source: https://marquezproject.github.io/marquez/

Importance on Metadata

Why we need metadata

What is Marquez

Where does it sits?

Core concepts

Versioning

Benefits of Marquez architecture

Data model

Metadata Collection

Workflow

Register a Job

Register Job Run

Start

Complete

Register job Run Outputs

Demo Time

./docker/up.sh

Bigdata Engineer — Love for BigData, Analytics, Cloud and Infrastructure. Want to talk more? Ping me in Linked In: https://www.linkedin.com/in/ajshetty28/