Data Pipelines

Apache Oozie

Oozie is a Java-based tool used for building workflow jobs running on Hadoop Map/Reduce and Pig. It is a flexible, affordable, and reliable system to manage, schedule, and perform difficult Hadoop tasks. It’s a Java application that runs in a Java servlet framework. 

 Project Background
  • Platform: Oozie Java Web-Application
  • Author: Apache Foundation Software
  • Released: December 2019
  • Type: Server-based Workflow
  • License: Apache License 2.0
  • Support: NA
  • Language: hPDL
  • GitHub :apache/oozie
  • Runs on: NA

Applications

  • Workflow scheduling
  • Directed Acyclic Graphs (DAGs) of actions
  • Combines multiple jobs sequentially into one logical unit of work
  • Manage and schedule Apache Hadoop jobs

Summary

  • It consists of action nodes and flows nodes
  • Three types of jobs are common in Oozie – workflow, coordinator, and bundle jobs
  • Oozie workflow action starts in a remote environment. On completion, it calls back for notification.
  • Oozie offers support for different types of actions: Hadoop map-reduce, Hadoop file system, Pig, SSH, HTTP, email, and Oozie sub-workflow.
Scroll to Top