A safe and reliable autonomous vehicle system depends largely on accurate location of obstacles in the environment, and precise calibration between light detection and ranging (LiDAR) and camera is a prerequisite for multisensor fusion. However, calibration with target is laborious and time-consuming. In addition, the calibrated sensors may drift due to some factors such as the turbulence of the vehicle while the vehicle is in motion, which will cause the accumulation of errors and thus affect the driving. To solve these problems, a novel method is proposed to solve the extrinsic parameters of LiDAR and camera in road scenes. Thus, a feature extractor is constructed to extract lane and vehicle semantics from a pair of point clouds and image first, and then an objective function is maximized by an optimizer based on the initial extrinsic parameters. Because lane lines and vehicles on the road are complex, a lot of experiments on the KITTI dataset are conducted. The experimental results quantitatively and qualitatively demonstrate the accuracy and robustness of this method.