How Do Users Design Scientific Workflows? The Case of Snakemake and Nextflow
Published in Proceedings of the 36th International Conference on Scientific and Statistical Database Management, 2024
Scientific workflows automate the analysis of large-scale scientific data, fostering the reuse of data processing operators as well as the reproducibility and traceability of analysis results. In exploratory research, however, workflows are continuously adapted, utilizing a wide range of tools and software libraries, to test scientific hypotheses. Script-based workflow engines cater to the required flexibility through direct integration of programming primitives, but lack abstractions for interactive exploration of the workflow design by a user during workflow execution. In this paper, we study the requirements for the design of interactive workflows through the lens of existing workflow collections. Specifically, we focus on two widely used script-based workflow engines: Snakemake and Nextflow. For Snakemake, we collected workflows from 1602 GitHub repositories that are listed in the Snakemake workflow catalog. For Nextflow …
Recommended citation: Sebastian Pohl, Nourhan Elfaramawy, Artur Miling, Kedi Cao, Birte Kehr, Matthias Weidlich. (2024). "How Do Users Design Scientific Workflows? The Case of Snakemake and Nextflow." Proceedings of the 36th International Conference on Scientific and Statistical Database Management.
Download Paper