Health disparities are increasingly studied in and across a growing array of societies. While novel contexts and comparisons are a promising development, this commentary highlights four challenges to finding appropriate and adequate health measures when making comparisons across groups within a society or across distinctive societies. These challenges affect the accuracy with which we characterize the degree of inequality, limiting possibilities for effectively targeting resources to improve health and reduce disparities. First, comparisons may be challenged by different distributions of disease and second, by variation in the availability and quality of vital events and census data often used to measure health. Third, the comparability of self-reported information about specific health conditions may vary across social groups or societies because of diagnosis bias or diagnosis avoidance. Fourth, self-reported overall health measures or measures of specific symptoms may not be comparable across groups if they use different reference groups or interpret questions or concepts differently. We explain specific issues that make up each type of challenge and show how they may lead to underestimates or inflation of estimated health disparities. We also discuss approaches that have been used to address them in prior research, note where further innovation is needed to solve lingering problems, and make recommendations for improving future research. Many of our examples are drawn from South Africa or the United States, societies characterized by substantial socioeconomic inequality across ethnic groups and wide disparities in many health outcomes, but the issues explored throughout apply to a wide variety of contexts and inquiries.