The purpose of this dissertation is to introduce a hierarchical approach toward MIS evaluation.
MIS evaluation is difficult due to its multi-dimensionality, qualitative and quantitative aspects, and evaluators' various viewpoints. The most critical problem of MIS evaluation is the lack of the systematic framework to cover various viewpoints and factors. To solve this problem, this study takes the multi-level and contingent approach to performance evaluation, and proposes a new hierarchical approach, composed of three hierarchies: evaluating the contribution of MIS to an organization [strategy hierarchy]; evaluating the activities of MIS department or MIS function as an organizational sub-function through the overall MIS lifecycle [function hierarchy]; and evaluating the quality or productivity of the application systems as MIS outputs [system hierarchy]. In each evaluation hierarchy, the following criteria should be utilized, such as:
[strategy hierarchy]
- MIS impact on the firm's competitive position
- integration of MIS planning with organizational strategy
- MIS contribution to organization's financial and technical performance
- strategic use of information technology
[function hierarchy]
- operational and managerial efficiency
- adequacy of MIS planning, system development and control processes
- user/management attitudes and perceptions about MIS functions
- MIS resources competence
[system hierarchy]
- user satisfaction on system use
- technical quality of application systems
- quality of information outputs or service quality
- cost efficiency
Ideal MIS evaluation should include all three levels of the hierarchy with equal importance. However, practical MIS evaluation can be divided into three types, such as strategy-oriented, function-oriented and system-oriented evaluation, depending on the focus and emphasis of evaluation. The usage pattern of each evaluation type is analyzed according to uncontrollable contingent variables of MIS evaluation such as MIS maturity, information intensity and firm size, and controllable variables such as evaluator type and top management's intent.
The survey of 130 Korean firms, conducted in 1991 and 1994 successively, show that majority of firms use system-oriented or function-oriented evaluation type, although the usage rate of strategy-oriented type is slightly higher in 1994 than in 1991 and most MIS manager consider the strategy-oriented evaluation type as the ideal one. It is also found that the firms of higher MIS maturity and information intensity use the strategy-oriented evaluation type, and the firms with strategy-oriented evaluation type show a higher MIS performance. Further, the firms of more evaluators' type and higher top management' intent are also shown to use the strategy-oriented evaluation type and to have higher MIS performance.
Some managerial implications can be drawn based on the results of the study. First, strategy-oriented evaluation of MIS is more important as many firms more often use information technology as a strategic weapon. Second, MIS performance varies with evaluation type. Therefore, the design of MIS evaluation framework should be done carefully in the strategic and managerial contexts. Third, firms are recommend to use a different evaluation type according to organizational characteristics such as MIS maturity and information intensity.
Finally, limitations of this study and future research directions are suggested.