Apache Atlas is open-source data governance and metadata management tool.

Apache Atlas can be easily integrated with popular big data tools like Hadoop, spark Kafka, hive, etc.

[{"selector":"#anim-140ba628-076a-435c-a1cc-18589e42f59b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-212953c5-2f63-4fec-87f3-b658a98d6398","keyframes":{"transform":["translate3d(0px, -251.22054%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

It allows data engineers to ingest, classify, discover, and govern data assets from various data sources.

[{"selector":"#anim-678d1703-dbb4-4836-8393-046a33d38f5a","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6348dda5-6f08-49b8-ba0a-fac7dff1c187","keyframes":{"transform":["translate3d(0px, -251.59932%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

Atlas supports Lineage and has a simple UI to view the lineage of data as it moves through various processes

[{"selector":"#anim-e3d301fd-333f-40f3-a285-d238f8dc70de","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b95869b1-e273-4eae-8bd6-87f78a5e8119","keyframes":{"transform":["translate3d(0px, -250.46295%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

Some of the Capabilities of Apache Atlas are:- 1. Data Classification 2.Search & Lineage 3. Centralized Metadata 4. Security and Policy Engine

[{"selector":"#anim-8c4009f6-4448-4112-91d0-b26605acde80","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-fe7e835e-889f-4c0a-be37-04944424ab97","keyframes":{"transform":["translate3d(0px, -172.94685%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

It has many pre-defined types, and users can add new types based on their requirements.

[{"selector":"#anim-74fd2a41-0e41-415f-a164-3ef8b7eee350","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-753adcd2-c742-4fd7-a8e0-98aa02b26ec8","keyframes":{"transform":["translate3d(0px, -275.66615%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

It supports a SQL-like query engine to search entities.

[{"selector":"#anim-634eed37-16e2-400b-9df4-d0bd3c3fdd49","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-44bee628-efd8-4d78-a648-e2873794c001","keyframes":{"transform":["translate3d(0px, -394.23868%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

Apache atlas also provides many REST APIs to access and update lineage

[{"selector":"#anim-8a04e2ac-88c6-468d-924b-f4fe42146f71","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3422e698-7d76-407f-b378-73766c67c84d","keyframes":{"transform":["translate3d(0px, -323.15854%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]