For the case of a strongly magnetized plasam where the beam may be treated as one dimensional, the most unstable transverse magnetic (TM) mode is dominantly interacting with the relativistic electron beam and grows until it saturates in the final steady state. The nonlinear phenomena in the dielectric-lined Cherenkov masers are investigated via particle simulation and nonlinear analysis. Both studies are proceeded independently; the one is kinetic approach and the other is fluid approach. Both results supplement each other and well explain the experimental result. In the particle simulation finite difference method (FDM), particle-in-cell (PIC) method and leap-frog methods are used. Poisson equation is solved using Fourier analysis cyclic reduction (FACR) method. The phase diagrams of the beam electrons indicate that the saturation of the electromagnetic (EM) wave comes from the trapping of the electrons in the EM wave potential. From the result of the particle simulation we can know the efficiency, the saturation length and time, and the output power. However, it is a very hard and time consuming work to follow fully the trajectries of the electrons from the linear stage to the final saturation state. Hence, we find out the coupled nonlinear equation sets, which express the saturation state, using the cold fluid-Maxwell equations. From this nonlear analysis the efficiency of the most unstable wave as well as the real frequency shift is investigated. The effects of the beam current, the beam energy, the beam position, and the dielectric materials on the efficiency are discussed. The fraction of the electrons, which are deeply trapped in the electromagnetic wave and contribute to its amplification, is expressed as $(\gamma/\gamma_{60})^q$ with q = 1.8. The saturation frequency is up-shifted from the frequency with the largest linear growth rate. The present study suggests that the frequency-upshifts are from the nonlinear parametric coupling of the transverse magnetic waveguide mode and the space-charge wave. This may explain the recent experimental results where the observed upshifts were attributed to the dielectric liner charging [F. Garate, R. Cook, P. Heim, R. Layman, and J. Walsh J. Appl. Phys. 58, 627 (1985)].
