Institutional investors’ ownership in public firms has become increasingly concentrated in the last decades. We study the heterogeneous effects of large versus more dispersed institutional owners on firms’ innovation strategies and their innovation output. We find that large institutional investors induce managers to increase spending in internal R&D by reducing short-term pressure. However, to avoid empire building and dilution, large institutional investors prevent acquisitions, which reduces firms’ investment in external innovation. The overall effect on firms’ future patents and citations is negative. By acquiring less innovation from external sources, firms reduce the returns of their investment in internal R&D, jeopardizing their total innovation output. We use the mergers of financial institutions as exogenous shocks on firms’ institutional ownership concentration. Our findings complement the previously found positive effects of institutional ownership on firm innovation and indicate that the effects become negative when institutional investors become large owners.