Presently, the problem of noise robustness is one of the most important issues in speech recognition. In this dissertation work, we devote to solve noise robustness problems in speech recognition based on speech feature vector transform(data transform) and model parameter compensation (distribution transform).
First, we presented novel data transformation algorithm which estimates clean speech feature vector from corrupted one. Nonlinear contamination procedure of speech signal in noisy environment was approximated to linear function based on Taylor series expansion. Additive noise was modeled as a Gaussian distribution and spectral tilt was assumed fixed unknown vector. In this case, additive noise mean and variance, and spectral tilt are called by environmental variables those are estimated iteratively in maximum likelihood sense. Different from previous method, we incorporated variance of additive noise into re-estimation procedure with which we had more rigorous solution for environmental variables. We called this algorithm model-based linear approximation(MLA) method. Although the MLA methods was originally devised to compensate speech feature vector without a priori knowledge about noisy environment, we could easily combine the MLA methods with a priori knowledge by Bayesian estimation method. Also, the MLA method was extended to multiple noise condition. Each noise source was assumed to have an independent Gaussian distribution, and mean and variance of each noise source were considered as environmental variables. Experimental results showed that performance of MLA is comparable to that of stereo-data-based data transform algorithm. It is worthy of note that stereo-data-based data transform algorithm resulted in poor performance when there is insufficient adaptation data, while MLA does not need any adaptation data. Comparison with other on-line algorithm was also conducted and it was observed that MLA outperformed other methods especially at low SNR conditions.
Second, we developed HMM parameter transform method based on the MLA approach. Generally, distribution transform is known to be superior to data transform method when sufficient adaptation data are available. However, when there is no adaptation data available or the amount of adaptation data is insufficient comparing to the number of parameters to be transformed, distribution transform results in drastically poor performance. Current speech recognition system have complex form to exactly model various phenomena like co-articulation effect, and it results in dramatical increase of the number of model parameters. Therefore, it is considered to be very important to transform many model parameters with limited small amount of adaptation data. Using proposed method, we could adapt HMM parameters in noisy environments without any adaptation data. Since our interest is consistently concentrated on transform of parameters without any adaptation data and a priori knowledge about environment, we adopted joint-optimization concept. Similar to data transform algorithm, an extension to multiple noise condition was easily achieved. Also, we proposed practical delta parameter compensation method based on MLA framework that has been known to be difficult when there is no adaptation data available. Performance evaluation was conducted using isolated word and continuous speech recognition experiments. Experimental results showed that HMM parameter transform based on proposed algorithm is superior to data transform method, and outperform well-known HMM compensation method, PMC, in all experimental conditions. Differential parameter was also successfully adapted, and by which speech recognition system could be operated reliably in a heavy noisy condition.
Finally, we analyzed Lombard effect on Korean digit and tried to compensate Lombard effect and additive noise simultaneously. It was observed that Lombard effect increase signal power and duration of utterance by about 10. Phonetic-acoustic analysis was also performed in cepstral domain with which we could conclude that Lombard reflex affect each phoneme differently. In this work, Lombard effect was modeled as a kind of cepstral bias at various levels including whole word and phonetic unit. Though preliminary analysis result showed that Lombard effect is different for each phoneme, it is difficult to reliably obtain cepstral bias for all phoneme from short utterance like digit. So, we presented general framework for bias tying, by which we could control the number of environmental variables and choose appropriate trade-off between the number of variables to be estimated and the amount of observation data. Using the MLA, we successfully compensated effects of Lombard reflex and additive noise simultaneously. To our best knowledge, our work is the first trial of on-line joint compensation of Lombard effect and additive noise in the strict sense, especially for Korean.
음성인식 시스템이 실제로 이용되기 위해서는 잡음환경에서 신뢰도있게 동작하여야 한다. 기존의 음성인식시스템들은 잡음의 영향이 상대적으로 작은 환경에서는 높은 성능을 나타내는 반면 배경잡음이나 약간의 채널 잡음에도 민감하게 반응하며 인식률이 저하된다. 본 논문에서는 이러한 문제를 해결하기 위해 기존의 음성인식을 위한 잡음처리 기술들의 장단점을 비교하고, 선형 근사화 방식에 근거한 새로운 방식을 제안하였다. 제안된 방식들의 내용은 다음과 같다.
첫째 잡음환경에서 구한 음성의 특징벡터를 변환하여 인식성능을 높히는 방식을 제안하였다. 기존의 방식들이 음성 특징벡터 변환을 위해 동시에 녹음된 스테레오 데이터를 이용하거나, 잡음환경에 대해 미리 자세한 정보를 알고 있어야 함에 비해 제안된 방식은 인식대상이 되는 음성이외에 잡음환경에 대한 어떠한 사전 정보도 사용하지 않으면서 신뢰도 있는 성능을 보였다. 이 방식은 부가잡음과 채널의 영향을 환경변수로 모델링하여 인식대상인 입력 음성으로부터 환경변수를 반복적으로 재추정하고 이를 이용하여 음성의 특징벡터를 추정하는 것으로, 기존의 관련연구와 비교하여 잡음의 통계치를 가정하고 잡음의 분산항을 평균치와 동시에 재추정할 수 있다는 특징이 있다.
두번째로는 HMM에 근거한 음성인식기의 모델변수들을 환경의 변화에 따라 변화시키는 방식을 제안하였다. 일반적으로 음성의 특징벡터를 변환하는 방식에 비해 음성인식의 모델변수들을 바꾸어 줄 경우 분산항의변화도 정확히 반영할 수 있어서 인식률이 더욱 향상되는 것으로 알려져 있으나, 복잡도가 증가하여 계산량의 부담이 늘어나는 단점이 있다. 이 방식에서는 음성 특징벡터 변화에서와 같이 잡음환경에 대한 별도의 정보가 필요없이 빠르게 변화하는 잡음환경에서도 사용할 수 있도록 음성인식기의 모델변수들을 변환시키며, 이 경우 뚜렷한 해결책이 알려져 있지 않은 동적 변수의 변환문제를 적절한 근사화 방식을 이용하여 해결하였으며 그 성능을 실험결과를 통해 확인하였다.
세번째는 잡음환경에서 배경잡음의 영향에 의해 화자의 발성특징이 변화하는 롬바드 효과의 보상에 제안된 선형 근사화 방식을 적용하였다. 이를 위해 한국어 숫자음으로 구성된 롬바드 음성 데이터베이스를 구축하였으며, 롬바드 효과가 음성인식에 어떤 영향을 미치는 지를 분석하였다. 제안된 방식은 다양한 실험환경에서 종래의 방식에 비해 일관되게 우수한 성능을 보였으며 동적 변수 변환과 결합됨으로써 심한 잡음환경에서도 신뢰도 있는 인식률을 나타내었다.
