Visual communication is one of the most demanding applications in various are as. Generally, visual information coding systems are designed to minimize overall distortions of the input image without regard to its semantic importance. However, there are certain cases in which one region of the input image is more meaningful than other; for example, a person who is speaking at a videoconference, a player in a sports game, or an enemy''s vehicle in tactical scenes. Therefore, because information loss in this region, the foreground region, is more serious than in the background region, a visual communication system should be capable of handling the difference between regional characteristics. In order to meet this requirement, it is necessary to differentiate among the allocated bits for each region. According to classical rate-distortion theory, higher reconstructed quality for the foreground region can be obtained by allocating more bits to it than to the background region. Even some techniques have been proposed to handle the bit quantity of the foreground region, these methods were only concerned with presenting methodologies of system implementation. A unifying mathematical model or a definitive principle has not been presented, so there is no basis on which to compare one method with another, and no rule about how much the foreground quality can be improved.
To resolve these ambiguities, first, we present a concrete concept about the coding for the variable of foreground quality - we define the coding as `quality scalable coding (QSC)‘. Previous works only concerned about presenting implementing method of QSC idea. However, in this paper, we present a mathematical model of the QSC using a ''quality scalable parameter'' which is a normalized value between 0 and 1. This model is a more extended coding concept than conventional video coding and unifies the coding concepts than previous things.
Secondly, we present QSC scheme for the still image coding. For the still picture QSC, we present a fast quantizer assigning algorithm based on block activity for still picture coding to reduce the computational load with maintaining close performance to optimal results. Generally, block activity has be en considered a characteristic value of the human visual system. However, we regard this activity as the degree of the image complexity and a characteristic feature of rate-distortion relation and we use this activity for bit allocation subject to the bit-budget constraint, and aim the approximation of optimal bit allocated results with less computational load.
Third, as an example of the QSC implementation for the moving picture coding, a H.263 compatible coding scheme is presented. For temporal and spatial quality scaling, some techniques (EP insertion, QP change, and residual error suppression) are presented. The simulation results show that the proposed system can efficiently control the foreground quality by changing the spatial and temporal quality scale parameters.
Finally, we propose a novel optimal bit allocation method. The bit allocation problem is one of the fundamental issues in lossy picture coding. Mostly, it is represented as minimizing overall distortions within the given constraint that satisfies the bit budget. Existing bit allocation methods can be classified into two groups. Some are designed for real-time application with comparatively low computational complexity, and others aim to obtain the optimal bit allocation using optimization techniques. The methods in the first group are used in many practical applications because of their computational efficiency. However, the proprieties of the methods are not fully supported because they rely on some assumptions with empirical results. On the other hand, the methods in the second group regard the bit allocation as a constrained optimization problem assuming that the whole input image sequence is known (i.e., noncausal). Although these optimization-based approaches require a high computational burden, they are worth tackling because we can acquire useful information about a coding system. The optimal result enables us to understand the performance of the system. It can also serve as a benchmark for assessing the performance of the bit allocation algorithm. Moreover, it can be a basis for the development of a fast approximation method. The most famous optimal bit allocation method is the Lagrangian method that uses a dynamic programming search algorithm (e.g., Viterbi algorithm) after the problem is converted into an unconstrained optimization problem by the Lagrange multiplier. However, for common block-based interframe coding systems the Lagrangian method becomes formidably complicated as the number of input frames increases because of the inter-dependency among macroblocks caused by the motion compensation. To resolve the computational burden while maintaining the results close to the optimum, we propose a novel approach to the bit allocation problem based on a partitioned form of the bit allocation problem. A conventional representation of the bit allocation problem is decomposed into two-level: a frame-level problem and one-frame macroblock-level problems. To solve the frame-level problem, we use a ‘two-phase optimization technique’ using an interframe dependency model and an exponential rate-distortion model. We s how simulation results that the proposed algorithm can find the macroblock-level bit allocation (i.e., quantizer assigned result) that is closer to the optimal one.'
