Preference Alignment in Personalized Shopping Agents

Preference alignment in personalized shopping agents is an inference problem. For shoppers to trust agents with a search, agents must infer what shoppers mean, tell lasting preferences from current needs, and keep to the constraints throughout a recommendation. Agents must do that without demanding more effort than the shopper would spend browsing.

Why shopping agents require preference inference rather than preference retrieval

Dong et al. 2026 model preference specification as articulation under effort rather than retrieval of a complete, pre-existing answer. Consumers have finite attention budgets. Translating latent intent into search criteria consumes that attention, even when an agent takes over product discovery. Asking more clarifying questions does not remove this burden if the shopper cannot yet express what would make a product satisfactory.

The paper’s home-office chair example makes the distinction concrete. Price, dimensions, and color are relatively easy to specify before searching. Whether a chair will feel supportive during long work sessions, fit the room visually, or look professional on video calls is harder to encode in advance, although those qualities may be easy to evaluate upon inspection. The paper calls these hard-to-articulate, satisfaction-relevant dimensions “preference complexity.” This makes alignment a technical problem because the agent must work with an incomplete representation of satisfaction.

How agents infer latent intent from underspecified queries

Agent Personalized Benchmark (APeB) examines the work between a raw query and a useful product search. A query such as “campus wear?” leaves material, season, and price range unresolved. The agent must interpret it against browsing patterns involving running shoes, oversized jeans, and casual shirts, together with search keywords such as “effortless chic.” Those signals support an inference of mid-range casual clothing with an all-season bias.

The next task is to distinguish lasting preferences from the purpose of this purchase. A brand affinity or budget band may apply across searches, while the current request narrows which attributes are relevant now. As the conversation goes on, the agent must revise its view of which preferences last and which apply now, and tell closely matched choices apart.

APeB finds that models handle explicit queries well but struggle with early-stage requests where they must work out the shopper’s intent and preferences. Its rubric analysis identifies ineffective history use as the main gap. The authors’ VQRA history-aware query-refinement pipeline brings consistent gains, which suggests that modules built to use history can help. The improvement concerns how evidence becomes a query, rather than simply how many products the agent can search.

What interaction history does and does not reliably encode

Browsing can reflect curiosity rather than purchase intent, and an earlier purchase can reflect circumstances that no longer apply. What a shopper does across unrelated domains may reveal little about preferences that last. The agent nevertheless has to assign weight to each signal without knowing which interpretation is correct. More history is not automatically better evidence.

Amazon’s “Alexa for Shopping” announcement, cited by Dong et al., describes a cross-channel approach to accumulating preference signals. What users share on Echo devices informs shopping, while conversations, browsing, and purchases on Amazon make Alexa more helpful across experiences. This feedback loop broadens the evidence available over time. It does not, by itself, settle whether a particular action reflects a durable preference or a temporary need.

Dong et al. find that consumers with larger attention budgets can make up for lower AI fidelity by putting more effort into preference refinement. Users who engage less do less of that work to correct the agent. Inferences from history therefore have more at stake for people who give fewer explicit updates, even though what their past actions mean remains just as unclear.

How agents handle explicit constraints and where they violate them

PACESHOP, developed by researchers affiliated with Amazon, Northeastern University, and Expedia in 2026, shows that even clearly stated constraints can fail. Its illustrative example pairs a request for “adidas shoes for men” with a persona that explicitly avoids Nike. The response nevertheless organizes product categories as “Nike Cross-Trainers / Classic Icons / Running Shoes / Casual Sneakers.” A standard judge rates the response as GOOD. This is not an ambiguous preference inference.

The benchmark uses structured personas, including budget, household, and brand fields, together with auditable evidence pools to make such defects identifiable. It also distinguishes persona violations from contradictions within a response. In another example, related queries mention “adidas cleats with studs” while the overview describes “daily wear.” The defect lies in the relationship between components, rather than a conflict with a shopper’s stated brand constraint.

PACEJUDGE, the paper’s judging protocol, outperforms generic judges through structured per-axis diagnosis that specifies defect family and location. It judges the response in light of the shopper’s context, checks how its parts fit together, and checks whether evidence supports it. The design implication is that evaluation must be persona-conditioned and component-aware. Fluency cannot establish whether a recommendation respects either the shopper or its own supporting claims.

Why preference complexity determines when agentic search outperforms manual browsing — and when it doesn't

Dong et al. compare manual search, which accurately evaluates a limited product set, with agentic search, which screens a broad catalog using noisy representations. In their model, increasing preference complexity eventually makes manual inspection cease at a finite threshold. Mismatch then returns to its no-search level, and platform revenue falls to zero. Agentic search avoids that collapse because complexity raises the marginal value of preference refinement.

The chair example explains why the advantage remains conditional. An agent can search against price and dimensions, but a shopper may still need to refine what visual fit or professional appearance means. Catalog breadth is only one advantage. Searches across many alternatives with clear constraints suit delegation better than purchases whose experiential or aesthetic requirements remain difficult to articulate. For the latter, the bottleneck shifts from inspection to communication.

The same model also finds an “inverted fidelity allocation.” Platforms may give lower AI fidelity to consumers with larger attention budgets because those consumers can make up for it through more refinement. Greater capacity to articulate preferences can therefore lead to worse agent support by design. That allocation is rational for the platform, but not for the consumer, exposing a difference between reducing platform costs and reducing shopper effort.

How agents learn preferences across conversations rather than within a single session

For personalization to last across conversations, an agent must decide what to keep after the current conversation ends. Brand affinities and ethical constraints tend to be durable, while price sensitivity and aesthetic preferences can shift with a gift purchase, season, or life stage. Memory alone cannot make that distinction reliably. A stored preference must keep enough of its context for the agent to tell a temporary choice from a lasting trait.

APeB notes that existing multi-turn memory benchmarks largely use homogeneous conversations and focus on information extraction, leaving a gap in testing intent inference under ambiguity. Doing well on those tests therefore tells little about how well an agent keeps track of lasting preferences in the real world. The cross-channel architecture described earlier offers one way to accumulate evidence, but low-engagement users still provide less active correction. Persistence can extend the life of a mistaken inference as well as a useful one.

Yuan et al. 2026 propose a broader adaptation target in the PAMR framework, which they call mediation-policy learning. A user-side agent should learn which sources to consult, what context to disclose, and how to weight disagreement across sources. For preference storage itself, the design implication is an explicit long-term layer for stable traits and a session layer for current context, rather than one flat representation updated uniformly.

How agents can be steered away from user preferences by sellers or by their own training

Accurate preference inference does not guarantee action in the shopper’s interest. An agent’s training can imprint responses to commercial signals that differ from human responses. Agents can penalize explicitly labeled “Sponsored” items while rewarding platform-level endorsements, giving sellers an incentive to optimize structured, machine-readable signals that appear organic. The signal rewarded can diverge from the shopper’s welfare even when the agent understands the requested product.

Multi-agent negotiation introduces a separate confidentiality problem. A buyer agent instructed to keep a budget confidential may nevertheless signal willingness-to-pay to seller-side agents. No external manipulation is necessary for that failure. The buyer-side system can undermine its own instruction through the information it communicates, making disclosure control part of preference alignment.

PAMR identifies a related evidence-boundary problem when the platform controls the candidate set, evidence sources, and disclosure choices behind a conversational interface. An agent can sound as though it serves the user while working within choices that serve the platform. This allocation of control may be invisible to the shopper. Technical capability must therefore be distinguished from authority over the evidence and information exchanges that shape a recommendation.

Where the inference pipeline currently breaks down, measured across benchmarks

History noise, early-stage intent, and dialogue tracking introduce uncertainty before product selection even begins. Doing well with explicit inputs offers no proof that an agent can clear up that uncertainty. Preference complexity helps explain why results deteriorate as satisfaction depends on attributes the user has not yet articulated.

CoShop’s interactive benchmark reports that five frontier models achieved at most 56% accuracy, with failures attributed to insufficient expansion of user knowledge. APeB’s history-use diagnosis locates a related weakness in the evidence used to support that construction. Together, these findings point to limitations before the final recommendation is written.

The ACL 2026 framework finds that, when preferences are unknown, performance across domains falls to a narrow range of low accuracy. Its result concerns the difficulty of implicitly inferring and tracking preferences through dialogue. PACESHOP exposes a separate measurement problem because generic judges can overlook constraint violations and cross-component inconsistencies in fluent responses. These benchmarks test different tasks and identify connected weaknesses in inference, persistence, and evaluation.

What better-aligned agents would need to do differently

Better elicitation would follow the construction model rather than assume that every preference is waiting to be retrieved. A mixed-initiative agent would use examples, framings, and questions to help shoppers articulate requirements they did not fully hold before the interaction. That work must be selective. VQRA’s gains suggest that history-aware refinement can help reduce how much the user has to explain directly.

Preference architecture would address the different lifetimes of the signals being learned. Stable traits, including brand avoidances, budget bands, and ethical constraints, need a different update process from session-specific intent. Explicit tiers with different update rates and confidence thresholds would give that distinction a place in the system. Evidence mediation would address a separate failure by giving the user side control over sources consulted, context disclosed, and disagreement surfaced, as PAMR proposes.

Evaluation would then test whether those choices produce coherent, supported recommendations for the particular shopper. Persona-conditioned, component-aware judging would identify defect family and location rather than compress quality into a single scalar score. The remaining constraint is user effort, because an agent that demands excessive refinement can be worse than the storefront it replaces. The engineering target is maximum alignment gain per unit of user effort.