To reduce the number of unnecessary handover in hierarchical macro/femto-cell networks, it is necessary to avoid macro $\rightarrow$ femto cell handovers of \emph{temporary femtocell visitors} who stay in the femtocell for a relatively short time. In this paper, we propose a smart handover decision algorithm exploiting future mobility pattern prediction scheme to prevent macro $\rightarrow$ femto cell handovers of such \emph{temporary femtocell visitors}. Our simulation result shows that the proposed algorithm effectively reduces the number of unnecessary handovers.