Cloud platform for scheduling algorithm research
Simulation has long been a common and important way for conducting performance eval- uation in the research area of parallel job scheduling due to its flexibility for testing different workload configurations and scalability for evaluating large platforms that the researchers do not even have access to. However, in current practice, most researchers have to develop their own simulation programs. This makes it tedious, time-consuming, or sometimes even difficult to conduct performance evaluation of a new parallel job scheduling algorithm and compare it with previous scheduling methods in the literature. Moreover, most of the simple simulation pro- grams developed by the parallel job scheduling researchers simply manage to conduct simulation experiments and output summary performance information, e.g. average turnaround time of all jobs. When a simulation experiment reveals a strange or unexpected performance outcome for a specific scheduling algorithm, researchers usually have to figure it out by themselves and the simulation program can provide very little help during the scheduling performance debugging process. To improve current practice in parallel job scheduling performance simulation, we have developed a Parallel Extensible Workload Scheduling Simulator (Pewss), focusing on aiding research work in parallel job scheduling. The current version of Pewss has now been used in helping our research work on workflow scheduling [1–3], which is an important sub-area in parallel job scheduling for dealing with task-parallel jobs, in contrast to another well-known sub-area for scheduling data-parallel job [27]. Workflow scheduling in itself is a broad research area and can be classified into sub-fields for tackling the scheduling issues of different kinds of workflows. For example, according to the characteristics of workflow structure, there are different types of workflows and each has its own scheduling issues to deal with, e.g. task-parallel workflow [1], rigid mixed-parallel workflows [28], moldable mixed-parallel workflows [29]. In terms of the workload characteristics, there also could be several sub-fields, such as single static workflow [1], pipelined workflows [30], and online multiple workflows [31]. Pewss has been developed aiming for providing a common simulation infrastructure supporting research work on different types of workflow scheduling problems.
Pewss has been developed with the following goals in mind.
- Ease the performance simulation process when conducting parallel job scheduling research with the help of friendly GUI.
- Minimize the efforts needed to implement a simulation experiment for a specific workflow scheduling algorithm, based on Pewss’s extensible common parallel job scheduling simu- lation infrastructure for both scheduling algorithms and schedule visualization functions.
- Achieve the goal of parallel job scheduling performance simulation as a service based on web-based client/server architecture, allowing users to conduct research works from anywhere.
- Provide functionality for comparative analysis of different scheduling algorithms based on graphic representation of resultant parallel job execution schedules.
- Help identifying characteristics or performance bottlenecks of a specific scheduling al- gorithm based on the schedule visualization function, facilitating improvement of the scheduling algorithms. Facilitate collaborative research work by providing a multi-user simulation environment, enabling sharing scheduling algorithm implementation and sim- ulation results among researchers
Pewss was developed based on a client/server architecture as shown in Figure 4.1. The server side consists of a Pewss server and a scheduling simulator. The client side includes various components presenting the web-based user interface for configuring, conducting, and visual- izing simulation experiments. The client side communicates with the server side by AJAX (Asynchronous JavaScript and XML) [32] and Restful API [33]. The following describes the major components of Pewss in more details.