The conventional organic electroluminescent (EL) devices with Al/$Alq_3$/TPD/ITO structure are thermally unstable due to low glass transition temperature of TPD, which is used as a hole-transport layer (HTL). In this paper, inorganic material (Tantalum Oxide, $TaO_x$) was proposed as a HTL to improve the thermal stability. The electron/hole injection properties of $TaO_x$/ITO was analyzed and it was showed that $TaO_x$ can be used as a HTL and electron-blocking layer. The new organic EL device with Al/$Alq_3$/$TaO_x$/ITO structure was fabricated and EL characteristics was investigated. The EL spectrum was similar to the organic EL device using TPD layer. The maximum EL intensity was 138 cd/㎡ at 20V and the external maximum luminous efficiency was 0.02 lm/W. We showed the possibility of organic EL device using inorganic material as a HTL.