Luigi is a Python module used to build complex pipelines for batch jobs. Developed by Spotify, it was open-sourced in 2012. Popular companies like  Stripe, Foursquare, and Asana use Luigi as a workflow engine. It manages dependency resolutions, workflow management, visualization, and more. Also, there’s built-in Hadoop support. 

Project Background

  • Framework: Luigi 
  • Author: Erik Bernhardsson, Elias Freider
  • Released: 2012
  • Type: Open source workflow management
  • License: Apache License 2.0
  • Support: NA
  • Language: Python, JavaScript, HTML
  • GitHub: spotify/luigi
  • Runs on: Application setup without a framework, Angular, SAPUI5/OpenUI5, VUE.JS, React


  •  Custom calendar Scheduling
  • Dependency resolution
  • Workflow management
  • Visualization
  • Handling failures
  • Command-line integration 


  • Provides a framework to manage and develop data processing pipelines.
  • It automatically works on various related tasks for final output.
  • Luigi API provides PySparkTask for writing custom spark jobs.
  • Visualizer page in the web interface for viewing all your ongoing tasks.
Scroll to Top