Many scientific achievements today are derived from analyzing large amounts of data. However, the computations themselves might be very complex and consume significant resources. These resources are often drawn from a heterogeneous pool of geographically distributed computing and data storage resources. In such environment, running large-scale and collaborative applications arises many challenges: systematic management of resource pool, guarantee of Quality of Service (QoS) according to Service Level Agreement (SLA) with users, as well as successful and efficient execution of applications. In fact, Grid Computing [1][2][3] can bring heterogeneous resources together and allocate them efficiently to applications. However, some of the previous challenges such as the Qos guarantee still remain with Grid Computing technology.
One of existing resource management schemes is Policy Quorum based Resource Management (PQRM) system [4][5]. It is a QoS-constrained Grid resource management middleware that can guarantee a specific QoS according to SLA and provide available resource sets determined by decision function based on Markov Decision Process (MDP). However, it overlooks some information which is important to Grid workflow management, because it focuses on mechanism that introduces abstraction of resource status and adaptive resource allocation policy respectively.
Meanwhile, the execution of large scale and collaborative application requires analyzing large amounts of data, and the dependency among different applications is very complex. Workflow management can make the process concise and easy to handle for unskilled users. However, a large-scale workflow can be a burdensome job because it may consume significant resources in distributed environment or manage the data between resources. This kind of workflows can be managed by Grid workflow management system. Existing Grid workflow management systems can be classified into Workflow sequence management system and Workflow pre-scheduler system. However, these Grid workflow management systems do not provide service level guarantee to users.
Therefore, we propose a Grid workflow integrated resource management system, which concerns resource management, user service level, implementation facility and policy based management respectively. To devise workflow integrated resource management, we study a new architecture that presents policy-based workflow management. To evaluate our proposed mechanism, we derived cost-adaptive policy negotiator to handle workflow management polices and resource management policies. Based on cost adaptive decision scheme, policy selection is according to service level agreement of users and dynamic resource status.
Our work brings workflow management and resource management together for the first time. We evaluate this model through a virtual heart simulator application, executing Message Passing Interface (MPI) jobs in different types of SLA and comparing the performance to that of the former system. The result shows that proposed management scheme guarantees QoS for workflow applications.
