The recognition system for real world must have robustness to noise and rejection capability for false inputs. In this thesis, the isolated word recognition system is designed with ZCPA model based on human auditory system and RBF networks which shows the high recognition rate in the small isolated recognition system and rejection performance. The filter banks of ZCPA model are constructed as FIR filter with powers-of-two coefficients. These need more clocks than FIR filter using multiplier, but less area. In addition, the feature instead of signal buffering reduce the area. ZCPA feature vectors are normalized as 16 $\times$ 64 matrix by time normalization with trace segment algorithm and energy normalization. RBF networks need 1024 input nodes, 50 hidden nodes, and 50 output nodes. Naturally, the design topics are reduction of memory operations and fully use of memory bandwidth. It is designed as Finite Stage Machine with pipelined structure. Designed chip works in real time, and shows good performance.