Job scheduler

Shreyas Ragavan

Created: 2018-06-28 Thu 23:28

Quick note

This presentation was created using reveal.js, leveraging the org-reveal package in Emacs.

Type '?' to see a list of the keyboard shortcuts.

  • Swipe or use the arrow keys / n / space to navigate.
  • Navigate/Swipe horizontally for main slides
  • Navigate/Swipe downwards for sub-slides (if any)
  • Menu key (bottom right corner) to jump to slides.
  • Hit 'o' for an overview of the slides.

Conception

  • Available distributed simulation solver licenses were drastically under-utilised due to the lack of a cohesive management system to run distributed simulations.
  • Team workflow and productivity was severely impacted due manual firing of simulations.

Conception …

  • It was extremely cumbersome to submit test simulations to facilitate learning!
  • Potentially: A huge change could be achieved with a simple system to fire simulations 24/7, utilizing the entire capacity through a logical and optimized basis.

Project Goals

  • Reliable, automated and continuous software license utilisation in a First-In-First-Out (FIFO) basis, without manual intervention.
  • Streamlined project management and resource utilisation, keeping track of multiple, distributed ANSYS CFX simulations of various types and priorities.

Project Goals …

  • Customised workflow and simplified interface to submit simulations for the entire team.

Pseudo Algorithm

scheduler-algo.png

Algorithm description

Hierarchy based logic: loop through a set order of folders and run the latest simulation files. The folders are meant to serve as multiple priority levels, and at times simulation size filters.

  • For example, a Small (short) run's priority would be higher than a non-priority, larger job at any time
  • And then a Priority folder whose runs would run first no matter what.

Results Achieved

  • Increased software license utilisation by 50+%.
  • Saved 40min (75%) in a critical design step.
  • Eliminated repetitive manual script setups to submit multiple simulations.
  • Significantly improved project management and output.

Results Achieved …

  • Enabled the use of consistent solver and memory utilisation parameters and settings, allowing efficient deployment and reducing inefficiencies due to errors.
  • Allowed optimal or perfect utilisation of available licensing scheme, resulting in a significant increase in team output and productivity.

Relevant Links / Information

Tools used

  • Written with Python 2.7, using portable python, Spyder, Notepad ++ and Sublime Text 3.
  • System Scheduler
  • Documentation and presentation re-prepared using Emacs and org-mode and org-reveal at a later time.

Code On Github

Detailed notes

Wiki