This paper proposes an evolutionary design of neural network architecture, with a new encoding scheme, named as EONN. In this scheme, all the neurons to be used in the neural network are arranged in a one-dimensional array, and their order informations in the array play important roles in genetic operations. Using the proposed scheme, genetic operators can be easily implemented to obtain the optimal neural network architecture for a given problem. To avoid the permutation problem, evolutionary programming (EP) is chosen rather than genetic algorithm (GA). In other words, only mutation operators are applied to generate offspring. The effectiveness of the proposed scheme is tested with XOR and 3 parity problems for optimal neural network architectures. It is also applied to a 2-link robot manipulator to control the position of the end effector. Satisfactory simulation results with the obtained simpler neural network architecture compared to the conventional one demonstrate the effectiveness and applicability of the proposed scheme.