Ecological and genetic factors have influenced the composition of the human microbiome during our evolutionary history. We analysed the oral microbiota of the Agta, a hunter–gatherer population where some members have adopted an agricultural diet. We show that age is the strongest factor modulating the microbiome, probably through immunosenescence since we identified an increase in the number of species classified as pathogens with age. We also characterised biological and cultural processes generating sexual dimorphism in the oral microbiome. A small subset of oral bacteria is influenced by the host genome, linking host collagen genes to bacterial biofilm formation. Our data also suggest that shifting from a fish/meat diet to a rice-rich diet transforms their microbiome, mirroring the Neolithic transition. All of these factors have implications in the epidemiology of oral diseases. Thus, the human oral microbiome is multifactorial and shaped by various ecological and social factors that modify the oral environment.