The town of San Basilio de Palenque, in Colombia, was proclaimed “Intangible Cultural Heritage of Humanity” (ICHH) by UNESCO in 2005. Since the declaration, a small group of community members has disproportionately benefited from the nomination and gained privileged access to financial resources and political power. In this paper, I reveal how local intellectual elites have used the language of heritage-based rights to justify their current status, resulting in entrenched social exclusion among Palenqueros. Heritage benefits are not distributed solely on the grounds of race, ethnicity, or descent, but are chiefly based on those who worked toward the ICHH nomination. Perceiving themselves as self-made heritage entrepreneurs, elites feel exclusively entitled to benefit from Palenque’s heritage declaration. Although Colombian state bureaucrats consider heritage nominations as vehicles for integration of Afro-descendants as a whole, I show how the language of rightful entitlements to heritage limits the scope of their social inclusion. Ultimately, I argue that the entrenchment of elitism and rising inequity in Palenque are not altogether unexpected consequences of the heritage declaration. Rather, they demonstrate the difficulty of employing heritage to foster social inclusiveness and economic equality. This paper examines the contested connection between heritage and rights discourse. The ethnographic example I provide and the analysis of the harmful effects of the heritage-rights discourse on a local scale thus contribute to the goal of critically examining the extent to which engagement with rights (in legal or discursive sense) actually provides a means to address issues of social justice and inequality; it also helps to understand the limits and potentials of linking heritage and rights, and asks: when rights are coupled with heritage, what does heritage change?