The visibility of images captured in foggy conditions is impaired severely by a decrease in the contrasts of objects and veiling with a characteristic gray hue, which may limit the performance of visual applications out of doors. Contrast enhancement together with color restoration is a challenging mission for conventional fog-removal methods, as the degrading effect of fog is largely dependent on scene depth information. Nowadays, people change their minds by establishing a variational framework for contrast enhancement based on a physically based analytical model, unexpectedly resulting in color distortion, dark-patch distortion, or fuzzy features of local regions. Unlike previous work, our method treats an atmospheric veil as a scattering disturbance and formulates a foggy image as an energy functional minimization to estimate direct attenuation, originating from the work of image denoising. In addition to a global contrast measurement based on a total variation norm, an additional local measurement is designed in that optimal problem for the purpose of digging out more local details as well as suppressing dark-patch distortion. Moreover, we estimate the airlight precisely by maximization with a geometric constraint and a natural image prior in order to protect the faithfulness of the scene color. With the estimated direct attenuation and airlight, the fog-free image can be restored. Finally, our method is tested on several benchmark and realistic images evaluated by two assessment approaches. The experimental results imply that our proposed method works well compared with the state-of-the-art defogging methods.