A method for stiffness matrix correction of FE model to match with modal testing data is presented. The correction is performed under the condition of preserving not only the connectivity of the stiffness matrix but also the consistency between the stiffness matrix and the corresponding physical configuration of the structure. The amount of correction of the stiffness matrix is done as adjusting the scaling factors properly. A formulation to identify the scaling factors is found by utilizing the results of sensitivity analysis and minimizing the objective function of eigen-properties. The proposed method is demonstrated by using T-type beam structure and a 1/2 scale simple car body structure. The analysis of these examples reveals that the flexibility of joints is one of the most important factors in constructing an accurate FE model.