In the process of the engineering surveying by FOG (fiber optic gyroscope), there are a lot of measurement error caused by many kinds of factors, vibration error is one of them. Analyzing the output signal of FOG on the effect of vibration, Kalman filter can be used to inhibit the drift of output signal, and it is experimentally validated based on the filtering algorithm. The results show that, the vibration error of FOG was reduced, and the validity of the method has been proved.