The explosive growth of the Internet brings about the diversity and heterogeneity of digital contents, terminals and network connections. Content adaptation currently appears to be the key solution to support the quality of service (QoS) for multimedia services over heterogeneous environments. In this trend, rich multimedia contents are adapted to meet various constraints of terminals, network connections, and user preferences. In this thesis, we study modality conversion as an important adaptation method. We extend the traditional formulation of content adaptation to effectively support modality conversion. Two main challenging issues of this extended formulation are: 1) the quantification of the content value when contents are drastically scaled and/or converted to other modalities, and 2) the method to accurately decide the modality and the content value for each object given that quantification.
For the first issue, by extending the concept of utility function in content scaling, we propose the overlapped content value (OCV) model to represent the dependence of content value on resource and modalities. To build realistic OCV model, we evaluate cross-modality quality (or content value). In this context, we point out two important aspects of content value: the perceptual quality and the semantic quality. We present a subjective method to evaluate the content value of a content which may be drastically scaled or converted to different modalities. For the specific case of video and image modalities, we discuss some computational methods to replace the time-consuming subjective evaluation. Further, we investigate multi-modality quality from semantic perspective. Based on the theory of conceptual graphs, we propose an analytical framework to evaluate the semantic quality for multimedia adaptation and communication. The general formulation is then applied to the popular audiovisual streaming service.
As quality is very subjective, we propose a modality conversion preference to customize the quality for different users. Without this user preference tool, user has to accept any adaptation solution provided by the provider. To quantitatively integrate user preference into overall quality, we proposed systematic methods to modify overlapped content value model based on user preference parameters.
For the second issue, we propose two algorithms based on dynamic programming method to find optimal and sub-optimal allocation solutions. These algorithms are effective because they can provide accurate solution even content value curve is strongly convex. Our experiments demonstrate that modality conversion helps increase the overall content value of adapted multimedia documents under limited resource availability. The combination of the overlapped content value model, the modality conversion preference, and the dynamic programming method, is a crucial point in making efficient decisions on modality conversion as well as content scaling, so as to meet various conditions of the resource constraint and the user preference.
