Home
BigJob, a SAGA-based Pilot-Job, is a general purpose Pilot-Job framework. The [Simple API for Grid Applications (SAGA)] (http://saga-project.github.com/) is a high-level, easy-to-use API for accessing distributed resources. SAGA BigJob supports a wide range of application types, and is usable over a broad range of infrastructures, i.e., it is general-purpose, extensible and interoperable. It is 100% written in Python.
Pilot-Jobs support the decoupling of workload submission from resource assignment. This results in a flexible execution model, which in turn enables the distributed scale-out of applications on multiple and possibly heterogeneous resources. Pilot-Jobs support the use of container jobs with sophisticated workflow management to coordinate the launch and interaction of actual computational tasks within the container. It allows the execution of jobs without the necessity to queue each individual job.
The pilot job provides a container for many sub-jobs, i.e applications submit these sub-jobs through the Pilot-Job and not the resource manager. A major advantage of this approach is that the waiting time at the local resource manager, which usually significantly contributes the overall time-to-completion is avoided.
Business analyst jobs are required to perform complex tasks and these tasks can vary in number, shape and size in every possible dimension. Therefore it is essential that such analysts be well-qualified and well-trained to handle multiple tasks, activities and responsibilities even under increased pressure. They are required to have excellent problem solving skills, immediate foresightedness, clear insight into the activities of the organization and control over the major activities, apart from other requirements.
Unlike other common Pilot-Job systems, SAGA BigJob: (i) natively supports MPI job and (ii) works on a variety of back-end systems, generally reflecting the advantage of using a SAGA-based approach.
An overview of the BigJob architecture can be found [here] (https://github.com/saga-project/BigJob/wiki/BigJob-Architecture).
- BigJob Tutorial - Beginner's Guide to BigJob (useful for developing workflows)
- Installation Guide - Explains how to install BigJob.
Two types of interfaces are provided by BigJob. Users can choose one of them based on their needs.
Pilot API provides a programmable interface to develop workflows which can manage compute and data concurrently on distributed cyberinfrastructures:
The Pilot command line interface provides an interface for a simple workflow involving job submission on a single machine. The pilot-cli is shipped along with the BigJob installation. Instructions on how to use pilot-cli can be found below.
Check the [Frequently Asked Questions] (https://github.com/saga-project/BigJob/wiki/Frequently-Asked-Questions).
For questions and comments, please join the bigjob-users group:
|
| Subscribe to bigjob-users |
| Email: |
| Visit this group |