Whose Standpoint do LLMs Reflect? Surfacing and Mitigating Epistemic Blindspots

Abstract

When a user prompts a LLM to produce an output that affects an external group, the model must implicitly act on behalf of people whose preferences the user may not fully know. Thus, it is important to assess where the models fail to surface such materially relevant but unstated preferences: a second-order pluralistic alignment problem. However, existing approaches fail to do so as they focus on finding disparity across groups or misalignment with stated preferences. To fill these gaps, we leverage standpoint theory to posit that LLMs reflect a standpoint that determines which needs are visible by default. We introduce BlindSpot, a dataset of 1,830 underspecified scenarios each with four prompt conditions that gradually increase the salience of the latent need. Across 6 models, we find failures concentrated in religious, cultural, and socioeconomic needs. We then show that persona conditioning models surfaces latent needs more effectively than naming the group directly, suggesting this knowledge is present in the model's weights, but not visible from the default standpoint. We also show that retrieval and fine-tuning methods can help mitigate operationalization failures. We hope this work opens a broader research agenda around surfacing and broadening models' underlying standpoint.