LLMs on the Political Compass: Prompt-Conditioned Positioning and Directional Drift

Abstract

Ever since Large Language Models (LLMs) became easily accessible to the general public, one of the main concerns has been their potential for influencing users and shaping their worldview through frequent interactions. Some critics state that LLMs are inherently biased, while others strongly believe that they cater to users and affirm their preexisting beliefs. In this paper, we investigate how LLMs express political positions under a standardised political questionnaire and how those expressed positions change under prompt conditioning and framing. We do so by focusing on two core research questions: (1) How do large language models express political positions when evaluated using a standardised political questionnaire under stance-inducing prompts? (2) How susceptible are these expressed positions to controlled prompt- and framing-based manipulations? To answer these questions, we propose a dual‑LLM architecture that separates content generation from stance evaluation. The LLMs subject to test respond to the 62 items of the Political Compass Test (PCT) instrument, while an independent Judge LLM, specifically trained as a model-as-a-judge, evaluates the orientation and strength of each response. We conduct three experimental conditions: (T1) baseline testing with the original PCT items; (T2) context‑injection via prompt manipulation applied before presenting the same items; and (T3) question‑level framing in which the items themselves are prefaced with left‑ or right‑leaning cues. Our results show that, under our experimental protocol, most models exhibit a predominantly left-of-centre baseline positioning. However, this apparent ideological tendency is accompanied by pronounced directional drift under prompting, particularly in response to right-leaning contextual priming, which consistently produces larger shifts than left-leaning priming and can even relocate models into right-leaning quadrants. Some safety‑aligned models (notably Phi-4) display high neutrality and refusals, whereas most others still take clear positions with high commitment rates. The raw results, along with a user-friendly dashboard allowing visualisation and further testing, are provided as supplementary materials. This work contributes to the growing literature on AI alignment, risk assessment, and the measurement of prompt-conditioned political output behaviour under a standardised evaluation protocol.