Actionable NLP Beyond Prediction: Domain Adaptation, Counterfactual Generation, and Concept-Based Explainability

Abstract: Natural language processing (NLP) models increasingly mediate consequential decisions and scientific inquiry. Yet predictive accuracy alone rarely provides the actionable knowledge stakeholders need: whether a model will remain reliable in a new setting, which meaningful factors shape its behavior, or how its outputs would change if one of those factors were different. This thesis asks how NLP systems can help stakeholders investigate phenomena expressed through language and understand the behavior of the systems themselves. It develops this agenda through three intertwined pillars: domain adaptation, counterfactual generation, and concept-based explainability. The three pillars share a central object: the high-level concept. Concepts abstract raw text into human-interpretable variables that stakeholders can name, measure, and reason about. The thesis further shows how large language models (LLMs) make concept-based analysis more scalable and flexible by generating counterfactuals, discovering and assigning concepts, and serving as validated alternatives to human annotators. Across applications involving suicidal ideation, cognitive decline, and preference mechanisms in LLM post-training alignment, the thesis demonstrates how NLP can move beyond prediction to support stakeholder inquiry, understanding, and decision-making.