We introduce a new concept to evaluate Web documents and products through human computer interaction. In the initial stage, evaluation data by general users for Web documents and products are difficult to collect. The citation frequency can be a good measure for document evaluation. Many automatic ranking systems have used this citation system to measure the relative importance of consumer products or documents. However, the automatic citation analysis has a limitation; it does not truly reflect the importance of the varying viewpoints of human evaluation. Therefore, human expert evaluations of Web documents and products are very helpful in finding relevant information in a specific domain, especially in the initial stage.
Our contributions are developing the four metrics for finding the closeness between the expert’s evaluation and user’s evaluation. When there is an inadequate data for the general user’s opinion we propose a recommendation system. By using all 4 metrics, the evaluation effectiveness measure for ranking processes is measured. This shows that the system is improving and getting closer to a system based on the general user’s opinion.
The current method of ranked retrieval and the resultant presentation methods are inadequate for fulfilling the significant number of queries where users wish to learn about a new topic or retrieve information to facilitate analysis or decision making. In order to meet these user demands, search engines must provide services that serve various kinds of information beyond the traditional lookup. Nowadays, weblogs have become very popular and advances in search technology have paved the way for more novel methods of retrieving information, inferring queries and guidance towards the exact results. The evaluation of weblogs based on Human-Computer interaction is a new method, which has become a popular in the internet survey. Therefore, based on the Human-Computer interaction, human expert evaluations of web documents are very helpful in determining relevant information in a specific domain, especially in the initial stage.
We also suggest that a dynamic group of experts for a certain Web document and products be automatically created among users to evaluate domain specific Web documents and products. The experts have dynamic authority weights depending on their performance in the ranking evaluation. In addition, we develop evaluation effectiveness metric for ranking processes. Furthermore, the dynamic change of authority weight increases the credibility of the evaluation effectiveness of experts.
Further, we also describe the decision tree model and the 3-Dimensional representation of the information retrieved from various weblogs in relation to the argumentative logics. The weblogs are considered as datasets that show significant correlations among the queries applied to them. Those relations as a set of linear classifiers can be obtained from neural network modeling based on the back-propagation. We have extracted a compact set of rules to support the dataset with the queries and employed effective evaluation metrics to evaluate the weighted average of the weblogs categorized into different types. We used the ID3, a simple decision tree learning algorithm, to filter the opinions from various weblogs. The minimization of the depth of the decision trees was carried out with the optimal parameters. The efficiency of the ID3 with the target values that has discrete output values such as a Boolean decision is helpful in obtaining the desired results in our experiment. The opinions from the weblogs are retrieved and represented as an object oriented 3-dimensional system. The goal of our approach is to generate rules following the shape and characteristics of response surfaces and to represent them in a 3-dimensional interactive system, Blog Cosmos.
We have also used a method of evaluations of weblogs using the 3-D Blog Cosmos result from a task analysis carried out in the context of Human-Computer Interaction (HCI). Specifically, we carried out the experiments based on the cognitive techniques to bridge the gap between cognition and computer interfaces. The engineering model of human performance that was used was based on the method suggested by Card et al who developed GOMS (Goals, Operators, Methods and Selection rules to perform a task), a representation of “how to do it” knowledge that is required by the system in order to accomplish the intended tasks. Our experiments showed two aspects. First, after a certain period, our dynamic expert recommendation system became stable. i.e. the result of our system converged and got close enough to the general users’ evaluation for both absolute and relative feedback method. Therefore, our system can indicate general users’ evaluation when the general user evaluation data is inadequate. Second, for the specific domain (movie, or music), users preferred our system over other systems for the domain specific queries. We also showed that our system can be the indication for the general users opinion on the Blog Space.
웹에서의 기계적인 순위결정 시스템은 사용자의 의견을 반영하지 못하는 치명적인 결점을 지니고 있다. 따라서 일반 사용자의 의견이 반영되어야 하는데 일반 사용자의 의견이 충분한 수에 이르지 않으면 반영이 어렵다. 하지만 이러한 자동평가에는 분명한 한계가 있다. 그것은 사용자 평가의 다양성이 반영되지 않았다는 것이다.
따라서 전문가에 의한 평가가 초기 단계에 매우 중요하고 이것은 일반 사용자의 의견과 비교하여 다이나믹한 전문가 추천시스템을 만드는 것이 본 연구의 초점이다. 이를 위해 다양한 메트릭을 만들어 본 연구시스템을 검증하였고 Rank order function, Rank order Window, Fisherman’s Correlation, $F_\beta$ measure with partition으로 분석하였다. 우리 연구의 공헌은 4개의 메트릭을 이용하여 전문가와 일반 사용자의 평가의 근접성을 측정한 것이다. 일반 사용자의 데이터가 없을 때 우리의 시스템이 특히 짙은 성능을 나타낸다. 모든 4개의 메트릭을 통해서 랭킹 절차에서의 유효성이 측정되었다. 분석결과 전문가에 의한 평가는 사용자의 의견과 근접하고 질의와 답변이 많아질수록 결과가 점점 나아진다는 것을 실험 결과로 보였다.
또한 웹블로그 상에서의 문서에 대한 댓글들을 분석하고 이를 통계화하여 Blog Cosmos라는 시스템을 구현하였다. 이를 통하여 적은 시간과 노력으로 사용자들의 의견에 대한 분석이 가능하고 특정한 제품이나 특정한 정책에 대한 시민의 의견을 수렴하여 더 나은 대책을 세울 수 있게 된다. 또한 3-D 모델의 blog 평가 시스템을 개발하였고 ID3를 바탕으로 한 결정 트리로 설계되어있다. 또, GOMS Analysis를 이용 우리 시스템의 사용자인터페이스의 우수성을 측정하였다.
결론적으로 일정한 시간이 흐른 후 우리의 전문가 시스템 평가는 사용자의 평가와 근접하였고 블로그 시스템도 특정한 분야에서 사용자의 데이터가 적을 때 우수한 기능을 나타낸다.
