This study is conducted for the purpose of applying a contemporary non-parametric approach - kernel density estimation - to Value at Risk (VaR). The basic idea of this method is introduced by Stutzer who constructs a risk-neutral probability distribution using information from the historical record of underlying security. This approach allows all the higher moments as well as stochastic volatility to vary, so it is called stochastic Historical Higher Moments VaR(HHM-VaR) model. HHM-VaR, which was proposed by Cakici and Foster, uses a normal kernel with adjustable bandwidth, and it permits a tremendously flexible, completely non-parametric dependence of the distribution of asset price changes upon the level of interest rates. The evaluating test of HHM-VaR is performed using two alternative specifications: an EWMA model and a traditional Historical Simulation model. Furthermore, the data used in this study are three interest rates of two kinds of bonds: a 3-year treasury SPOT and yield to maturity (YTM), and 1-year monetary stabilizing bond rate. This study shows that conditional distributions of $Δr_{t+1}$ given $r_t$ are different from each different level of interest rates reflecting properties of their own, so VaR bounds vary dynamically depending on the level of interest rates. Under back-testing, forecasted VaR of 4 in-sample periods are compared with realized returns in out-of-sample periods. The results are as follows. HHM-VaR estimates show much smaller exceptions than the other two models. In addition, other tests, including accuracy, required capital, and loss excess VaR, show that HHM-VaR presents a variety of results, some less or better, or some the same. Finally, the HHM-VaR approach has better results without coupon or long term bonds than with coupon or short term ones. It is likely that the kernel density methodology, with a greater amount of historical data, can improve the estimation and prediction of the interest rate percentile for risk analysis.