We all have worked around the Kubernetes clusters, right? Managing multiple Helm charts and Kubernetes manifests across those clusters is such a hassle but not anymore with the help of ARMADA. Let's understand why I'm saying this a bit more in detail...
So exactly What is Armada?
I think you can already guess what it is about just from the title of this blog and yes Armada is a multi-Kubernetes-cluster batch job meta-scheduler. But all those who weren't able to guess this must be wondering what are all these batch jobs and meta-scheduler things.
Ok ok let me explain this word by word:
Multi-Kubernetes-Cluster - Armada is designed to work with multiple Kubernetes clusters simultaneously. It enables organizations to utilize resources from multiple clusters to execute their batch jobs, providing enhanced scalability and resource utilization.
Batch Job Scheduling - Armada acts as a meta-scheduler, meaning it provides an additional layer of scheduling and coordination on top of the underlying Kubernetes cluster schedulers.
It allows users to define and submit batch jobs to Armada, which then intelligently schedules and distributes these jobs across the connected clusters based on predefined policies, resource availability, and optimization strategies.
I think now you got an idea of what Armada is and what it does but now the question arrives, how did you get to know about this CNCF project? Let's talk about this now...
How did I get to know about Armada?
So, I first got to know about Armada while searching for organizations to contribute to under GSoC'23(Google Summer Of Code). While searching I stumbled across Armada which is a sandbox project under CNCF(Cloud Native Computing Foundation) ecosystem.
The Armada project came under my skillset and I applied for the same and I'm glad to share that I also got selected woohoo!!!!
This summer I'll be focusing on creating a Krew plugin for armadactl (a command-line utility tool for Armada).
Armada's Architecture
Armada consists of two main components
The Armada server is responsible for accepting jobs from users and deciding in what order, and on which Kubernetes cluster, jobs should run. Users submit jobs to the Armada server through the
armadactlcommand-line utility or via a gRPC or REST API.The Armada executor, of which there is one instance running in each Kubernetes cluster that Armada is connected to. Each Armada executor instance regularly notifies the server of how much spare capacity it has available and requests jobs to run. Users of Armada never interact with the executor directly.
Now that we know how I got to know about Armada and its architecture, let's talk about Why Armada?
Why Armada?
- FAIR-USE SCHEDULING
Armada maintains fair resource share over time (inspired by HTCondor priority)
- HIGH THROUGHPUT
Armada can handle large amounts of queued jobs (million+).
- UNINTERRUPTED
Armada allows adding and removing clusters from the system without disruption
- MASSIVE SCALE
By utilizing multiple Kubernetes clusters, the system can scale beyond the limits of a single Kubernetes cluster
Wanna know more, click here.
Well, I have been talking about jobs, queues, job sets and all that but what exactly are these? Let's take a look...
Key Concepts of Armada
Queues:
Queues in Armada represent logical entities that act as containers for batch jobs. They help organize and categorize jobs based on their characteristics or priorities.
Jobs are submitted to queues, and Armada ensures the appropriate scheduling and execution of jobs within each queue, considering factors like resource availability and priority.
Jobs:
Jobs in Armada represent individual units of work or tasks that need to be executed on a Kubernetes cluster.
Users define jobs with specific attributes like resource requirements, input data, command or script to be executed, and any dependencies.
Job status, execution progress, and results can be monitored and tracked through Armada.
Job Set:
Job sets in Armada provide a way to group related jobs together. They allow users to define dependencies and relationships between jobs.
A job set can consist of multiple jobs that need to be executed in a specific order or have interdependencies.
Armada ensures that job sets are executed in a coordinated manner, respecting the defined dependencies and relationships.
Excited yet to learn more about Armada? Then what better way to contribute towards it and learn from professionals.
Contribute to Armada
Whether you have an idea to share, a feature to add, an issue to report, or a pull request of the finest, most scintillating code ever, I want you to participate and contribute to Armada!
You can check out Armada's main project fork here.
Do make sure to take a look at Contributing Guide before contributing to the project.
Wanna connect with everyone in Armada? Then follow the steps below:
Join the CNCF Slack, if you're not already in it you can get your invitation from here.
Once inside search for #armada channel and voila!
Looking forward to building Armada with you!
Conclusion
So I think now you have a pretty good idea of how powerful of a tool Armada is. Really makes managing all those helm charts and manifests look easy. But Armada is still in the sandbox stage and we have a lot of work to do so feel free to contribute towards Armada.
Wanna connect with me, you can find me on Twitter and LinkedIn.