Efficient resource usage is a key to achieve better performance in cluster systems. Until now, most research has focused on balancing the load of each node to use resources of the entire system more effectively. However, we can achieve further performance improvement when the initial placement system considers the resource requirement by the task being assigned. The resource based initial placement needs the resource usage of a task in order to fit it to the most suitable node. Since the initial placement requires that tasks be scheduled before execution, any resource requirements must be provided in terms of the estimates based on historical behavior. This approach can severely affect the execution time on wrong estimation.
We propose a novel load metric, called the effective number of tasks in order to resolve the problem due to the wrong estimation, so the resource based initial placement can work without priori knowledge about the resource requirement of processes. Simulation results show that the dynamic load balancing system with the effective number of tasks incurs shorter execution time than the conventional approach with the historical behavior based estimates.
