We show that policies that eliminate corruption can depart from socially desirable policies and this inefficiency can be large enough to allow corruption to live on. Political competition between an honest (welfare maximiser) and corrupt politicians is studied. In our model the corrupt politician is at a distinct disadvantage: there is no asymmetric information, no voter bias and voters are fully rational. Yet, corruption cannot be eliminated when voters have heterogeneous preferences. Moreover, the corrupt politician can win the majority, as the honest politician tries to trade off the cost of eliminating corruption with its benefits.