I think you suspect wrong direction, Fresnel is correct but you have Glossy node that not physically perfect, especially in case of conductor, at extreme angles. I remember there was attempt to bring better more general glossy bsdf with respect to "imaginary K" as in Maxwell or Luxrender as example, but nothing in trunk yet.