SAFE-SEARCH
Safety, Accountability, Faithfulness, and Evaluation in Conversational Search
Half-day, in-person workshop
About
Modern conversational search extends traditional IR into multi-turn interaction, where systems maintain context, infer evolving intent, reformulate queries, and increasingly invoke tools or take agentic actions. These capabilities create new guardrail needs across the search process. Intent may become adversarial over multiple turns. Retrieved content can steer downstream generation or propagate earlier errors. Systems must also decide when to answer, clarify, correct, abstain, or refuse.
SAFE-SEARCH is organized around four dimensions. Safety covers harmful and adversarial interactions, including evolving malicious intent, jailbreaks, and unsafe agent behavior. Accountability addresses privacy, bias, transparency, and responsible treatment of users and their data over time. Faithfulness concerns reliable retrieved information and grounded responses that do not carry errors across turns. Evaluation and effective intervention focuses on how guardrails should be assessed and how systems should respond to uncertainty or risk.
The workshop brings together researchers and practitioners from IR, conversational search, NLP, recommender systems, and security to develop a shared taxonomy of multi-turn failure modes, identify evaluation and benchmark gaps, and outline priorities for future guardrail research and shared tasks.
How should a conversational search system behave safely and reliably as the user's information need, retrieved evidence, conversational context, and system actions evolve over multiple turns?
Topics of Interest
SAFE-SEARCH welcomes research on how conversational search systems can remain safe, accountable, faithful, and useful throughout an evolving interaction. We are particularly interested in work that looks beyond individual prompts and responses to study how risks develop across dialogue, retrieval, personalization, memory, generation, and tool use.
Topics include, but are not limited to:
Safety and Robustness
- Multi-turn attacks and evolving harmful intent: attacks that unfold gradually through dialogue, including jailbreaks, intent obfuscation, adversarial prompting, and strategies that exploit information accumulated across turns.
- Security of retrieval and RAG pipelines: prompt injection, retrieval or corpus poisoning, malicious or manipulated documents, adversarial sources, and defenses against unsafe information entering the conversational pipeline.
- Safety of tools and search agents: unsafe planning or execution, inappropriate tool invocation, authorization and permission failures, cascading errors, and mechanisms for constraining agent actions.
- Conversational context and memory: risks created by accumulated dialogue state, persistent or cross-session memory, context contamination, information leakage, and unsafe reuse of earlier interactions.
Accountability and Responsible Interaction
- Privacy and responsible personalization: appropriate collection, retention, updating, and use of conversational histories, preferences, profiles, and memories.
- Privacy-aware training: methods such as noising or anonymizing personal data to prevent models from memorizing or absorbing sensitive user information during training.
- Sycophancy, over-reliance, and selective exposure: systems that repeatedly reinforce users’ beliefs, assumptions, or preferences, and their effects on user autonomy and information access.
- Fairness and transparency: identifying differential harms, hidden assumptions, and opaque decision-making that can emerge or intensify over the course of an interaction.
Faithfulness and Reliability
- Grounding across conversations: maintaining support for claims, summaries, corrections, and citations as retrieved evidence and conversational context change.
- Uncertain, conflicting, or changing evidence: reasoning over disagreement between sources, outdated information, misinformation, manipulated evidence, and conflicts between retrieved and model-internal knowledge.
- Error propagation and recovery: understanding how early retrieval, interpretation, or generation failures affect later turns and developing mechanisms for recognizing, correcting, and recovering from them.
Evaluation and Effective Intervention
- Limitations of current evaluation practices: what existing datasets, benchmarks, metrics, and experimental setups fail to capture about conversational safety and reliability.
- What should we measure next: overlooked behaviors, interaction-level risks, cumulative effects, recovery capabilities, and other properties that cannot be adequately assessed using single-turn evaluation.
- Biases, confounders, and controls: dataset and benchmark biases, judge and annotator effects, model contamination, unrealistic attack assumptions, missing baselines, and experimental controls needed for credible conclusions.
- Evaluation methodology: reliability and validity of LLM-based judges, human evaluation, adversarial testing, longitudinal evaluation, simulation, and methods for measuring safety alongside relevance, usefulness, latency, and user autonomy.
- Intervention and recovery strategies: deciding when a system should clarify, retrieve additional evidence, verify information, correct itself, warn the user, abstain, refuse, or limit an agent action.
- Red teaming and deployment evidence: realistic multi-turn stress tests, incident analyses, production failures, negative results, and lessons from deployed conversational systems.
Call for Papers
Conversational search is becoming more capable, more personalized, and more action-oriented. Systems can maintain context across long interactions, retrieve information from external sources, remember user preferences, and increasingly invoke tools or perform actions on a user’s behalf. These capabilities also introduce new forms of risk.
A failure early in a conversation may affect later retrieval and generation. A seemingly harmless sequence of requests may develop into harmful intent. Untrusted retrieved content may influence subsequent reasoning or tool use. Personalization and memory may improve usefulness while also introducing privacy, fairness, or over-reliance concerns. At the same time, safeguards must make difficult decisions about when to answer, seek more information, verify evidence, correct an earlier response, restrict an action, or decline a request.
SAFE-SEARCH: Safety, Accountability, Faithfulness, and Evaluation in Conversational Search aims to bring together researchers and practitioners studying these challenges across information retrieval, conversational systems, RAG, recommender systems, NLP, AI safety, security, and agentic systems.
We invite submissions that advance our understanding of safety and reliability in conversational information access, including new methods, empirical studies, evaluations, datasets, benchmarks, systems, deployment experiences, and forward-looking research perspectives.
We welcome the following contribution types:
| Type | Length | Description |
|---|---|---|
| Original research papers | Short (5–9 pages) or full (10–15 pages) | Completed research presenting new methods, findings, evaluations, or empirical analyses relevant to safe and reliable conversational search. |
| Systems and applied papers | Short (5–9 pages) or full (10–15 pages) | Implemented systems, methods, tools, datasets, benchmarks, or other applied contributions for safer and more trustworthy conversational search. We especially welcome systems deployed in real-world settings or being prepared for deployment, with emphasis on technical design, implementation, and evaluation. |
| Experience reports | Short (5–9 pages) or full (10–15 pages) | Lessons from building, deploying, or evaluating conversational search systems. These may include incident analyses, production failures, red-teaming findings, operational trade-offs, and other practical insights, without requiring a novel system or method. |
| Position and vision papers | Short (5–9 pages) or full (10–15 pages) | New research questions, conceptual frameworks, taxonomies, critiques of existing practices, emerging problem formulations, or proposals for future research directions. |
- Page limits exclude references. All papers must use the CEURART single-column style.
- Each submission will be reviewed by at least two members of the Program Committee.
- At least one author of every accepted paper is expected to attend and present the work in person.
- We plan to pursue archival publication through CEUR Workshop Proceedings, subject to eligibility.
- Formatting requirements and submission instructions: see the submission page.
We especially encourage work that challenges current assumptions about how conversational safety should be defined, measured, and evaluated, as well as research that reveals important failure modes that existing benchmarks do not yet capture.
Important Dates
- Submissions open
- 10 December 2026
- Paper submission deadline
- 15 January 2027
- Notification of acceptance
- 23 February 2027
- Camera-ready deadline
- 9 March 2027
- Conference
- 21–25 March 2027
- Workshop day
- 25 March 2027
All deadlines are 23:59 UTC-12:00 (AoE, Anywhere on Earth).
Program
Tentative half-day program. Times may shift depending on the ECIR schedule.
| Time | Session |
|---|---|
| 09:00 – 09:10 | Welcome and workshop goals |
| 09:10 – 09:40 | Keynote I (Academic) Emine Yilmaz, University College London |
| 09:40 – 10:10 | Spotlight talks from accepted contributions |
| 10:10 – 10:30 | Coffee break |
| 10:30 – 11:15 | Poster session |
| 11:15 – 11:45 | Keynote II (Industry) TBA |
| 11:45 – 12:30 | Invited panel and community discussion |
Keynote Speakers
Emine Yilmaz
Professor of Computer Science at UCL, where she leads the Web Intelligence Group. Her work spans information retrieval, conversational search, and evaluation.
To be announced
Team
Organizers
Contact
Questions about the workshop? Email us at safe-search-workshop@googlegroups.com.