INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS AND DATA SCIENCE

From YAML to Queue: DevOps Automation for Reproducible Benchmarks

1.The University of Texas at Dallas

Published in issue

October 2026

Volume / Issue

Vol. 1 No. 2

Access

Open access

View PDF

The article details presented on this page reflect the published version of record. Readers may access the full-text PDF, review the citation information, and export the reference in supported citation formats.

Abstract

This paper reframes scientific benchmarking as "experiments-as-code," presenting a DevOps automation approach that codifies multi‑step runs, hyperparameter sweeps, and cross‑site portability in declarable runbooks executed against heterogeneous compute tiers. A pair of complementary Python‑based orchestration stacks demonstrates how containerized tasks, scheduler adapters, and templated workflows enable continuous, repeatable experiment execution from laptops to leadership‑class clusters and public cloud. The approach emphasizes Git‑driven configuration, immutable artifacts, and environment capture to strengthen provenance and FAIR‑aligned reproducibility, while policy‑aware submission integrates with batch systems and secure remote access patterns. Iteration‑first specifications (loops, arrays, and conditional stages) simplify large ensembles beyond DAG‑only models, and cost‑aware planning supports pragmatic cloud bursts without sacrificing portability. The result is a practical blueprint for applying CI/CD‑style discipline to versioned configurations, automated provisioning, template reuse, and consistent reporting to scientific benchmarking at scale. By aligning workflow definition, execution, and evidence collection with DevOps practices, the framework reduces operational toil, shortens feedback cycles for model and system tuning, and promotes shareable templates that accelerate onboarding and collaboration across research teams.
Keywords:High-performance computing(HPC)BenchmarkingWorkflow OrchestrationCloud ComputingDevOps

Cite this Article

Use the following citation when referencing this publication in academic or professional work.

J. S. Kavitha, "From YAML to Queue: DevOps Automation for Reproducible Benchmarks", Int. J. Intell. Syst. Data Sci., vol. 1, no. 2, Oct. 2026, doi: https://doi.org/10.67231/k829mj42.

DOWNLOAD CITATION

From YAML to Queue: DevOps Automation for Reproducible Benchmarks | International Journal of Intelligent Systems and Data Science