ML Tools

Argo Workflow

Argo is a popular, open-source container-native workflow engine for orchestrating jobs in Kubernetes. Although it is used for general purpose workflows, it’s starting to play a more important role in machine learning pipelines. If data science engineers wish to use a workflow engine developed specifically for machine learning, there is Kubeflow which was built on top of Argo.ย  ย 

Argo was developed in August 2017 by Applatix, which was later acquired by Intuit. The team working at Applatix saw the need for an “integratedย workflowย engine” that is able to orchestrate jobs and deploy microservice-based applications in the Kubernetes environment. One of the main features of Argo is it “enables the implementation of each step in a workflow as a container.”

Project Background

  • Tool:ย Argo Workflow
  • Authors: Applatix (now Intuit)
  • Initial Release: October 2017
  • Type: Kubernetes native workflow engine for orchestrating jobs
  • License: Apache 2.0
  • Language: Go and others
  • GitHub:ย argo-workflows
  • Twitter:ย argoproj
  • Uses: run compute-intensive jobs for data processing and machine learning

Main Features

  • Fully featured UI
  • Native artifact support: S3, HTTP, Git, GCS, raw, OSS, Alibaba Coud, Artifactory)
  • Templating and composability
  • Workflow Archive
  • Cron Workflow
  • REST API
  • Source: Argo Workflows in 5 minutes

Project Timeline

  • Aug 2017:ย  Argo incubated at Applatix
  • Jan 2018:ย  Applatix acquired by Intuit
  • Mar 2018:ย  Argo CD incubated at Intuit
  • May 2018: BlackRock contributes Argo Events
  • June 2018: Kubeflow Pipelines powered by Argo
  • July 2018:ย  Argo CD in production at Intuit
  • Nov 2018:ย  Argo Rollouts incubated at Intuit
  • May 2019:ย  Intuit selected as CNCF Top End-User
  • April 2020: Argo gets accepted as incubating CNCF project
  • Aug 2020:ย  Argo Bootstrap Committee formed
  • Source: Argo presentationย 

Community Benchmarks

  • 15,000+ Stars
  • 3,000 forks
  • 900+ code contributors
  • 5700 Slack members
  • Members: Users, individuals, and vendors
  • 350 releases, 50% in last year
  • Source: Argo presentationย 

Intuit Argo Kubernetes Infrastructure

  • 11,000+ applications
  • 11,500 managed namespaces
  • 350+ clusters
  • 15,500+ nodes
  • 80+ Kubernetes platform developers
  • Source: Argo presentationย 

Scroll to Top