BibTeX download is not available in the pre-proceeding
As large language models (LLMs) are increasingly deployed as decision-making agents in strategic environments, their performance depends critically on how they model human counterparts. Drawing on level-$k$ thinking, we design normal-form games and develop a structural estimator that infers, from observed choices, the strategic reasoning type most consistent with an LLM's behavior. Across models, we find a systematic mismatch: LLMs overwhelmingly behave as if their counterparts are fully rational Nash players and respond with Nash-oriented actions, whereas human participants in our online experiments exhibit substantial heterogeneity, ranging from Random to Nash. This mismatch can help or hurt. An LLM may outsmart a boundedly rational human, but overestimating human sophistication can also create a Nash equilibrium trap, in which Nash play is not the payoff-maximizing response to observed human behavior. We design two prompting interventions to address this problem. The behaviorally informed intervention can substantially reduce Nash-oriented behavior, but it often induces untargeted deviations, including departures from Nash when equilibrium play remains appropriate. The welfare-aware intervention reduces Nash play less aggressively but yields higher empirical best-response rates and smaller welfare losses. Effective strategic alignment therefore requires not simply moving models away from Nash, but calibrating behavior to the relevant counterpart population and evaluating deviations against an explicit payoff or welfare objective.