Potential roughness has been reported to severely impair experiments in magnetic microtraps. We show that these obstacles can be overcome as we measure disorder potentials that are reduced by two orders of magnitude near lithographically patterned high-quality gold layers on semiconductor atom chip substrates. The spectrum of the remaining field variations exhibits a favorable scaling. A detailed analysis of the magnetic field roughness of a 100-mu m-wide wire shows that these potentials stem from minute variations of the current flow caused by local properties of the wire rather than merely from rough edges. A technique for further reduction of potential roughness by several orders of magnitude based on time-orbiting magnetic fields is outlined.