Publications

Abstract

Eye-tracking data has been shown to correlate with a user’s knowledge level and query formulation behaviour. While previous work has focused primarily on eye gaze fixations for attention analysis, often requiring additional contextual information, our study investigates the memory-related cognitive dimension by relying solely on pupil dilation and gaze velocity to infer users’ topic familiarity and query specificity without needing any contextual information. Using data collected via a lab user study (𝑁 = 18), we achieved a Macro F1 score of 71.25% for predicting topic familiarity with a Gradient Boosting classifier, and a Macro F1 score of 60.54% with a k-nearest neighbours (KNN) classifier for query specificity. Furthermore, we developed a novel annotation guideline – specifically tailored for question answering – to manually classify queries as Specific or Non-specific. This study demonstrates the feasibility of eye-tracking to better understand topic familiarity and query specificity in search.

Cite

@inproceedings{he2025characterising,
 author = {He, Jiaman and Leng, Zikang and McKay, Dana and Trippas, Johanne R and Spina, Damiano},
 booktitle = {Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval},
 series = {SIGIR ‘25},
 title = {Characterising Topic Familiarity and Query Specificity Using Eye-Tracking Data},
 year = {2025},
 doi = {10.1145/3726302.3730174},
 address = {New York, NY, USA},
 location = {Padua, Italy},
 publisher = {ACM},
 pages = {5}
}

Abstract
Social media platforms must filter sexist content in compliance with governmental regulations. Current machine learning approaches can reliably detect sexism based on standardized definitions, but often neglect the subjective nature of sexist language and fail to consider individual users’ perspectives. To address this gap, we adopt a perspectivist approach, retaining diverse annotations rather than enforcing gold-standard labels or their aggregations, allowing models to account for personal or group-specific views of sexism. Using demographic data from Twitter, we employ large language models (LLMs) to personalize the identification of sexism.
Our empirical results show that OpenAI’s LLMs (GPT-3.5, GPT-4, and GPT-4o) and two open-source LLMs (Mistral and Qwen) exhibit higher Krippendorff’s alpha label agreement with female annotators than with male annotators. As well, each LLM presents higher Krippendorff’s alpha agreement with a specific annotator age group. We then sought to counter these trends by providing “persona” instructions as part of the LLM prompt, with somewhat surprising outcomes, highlighting the potential of user-centered perspectivist methods to improve content moderation systems.

Cite

@inproceedings{magnosao2025effects,
 author = {Magnosão de Paula, Angel Felipe and Culpepper, J. Shane and Moffat, Alistair and Cherumanal, Sachin Pathiyan and Scholer, Falk and Trippas, Johanne},
 booktitle = {Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval},
 series = {SIGIR ‘25},
 title = {The Effects of Demographic Instructions on LLM Personas},
 year = {2025},
 doi = {10.1145/XXXXXX.XXXXXX},
 address = {New York, NY, USA},
 location = {Padua, Italy},
 publisher = {ACM}
}

Abstract

The effectiveness of search systems is evaluated using relevance labels that indicate the usefulness of documents for specific queries and users. While obtaining these relevance labels from real users is ideal, scaling such data collection is challenging. Consequently, third-party annotators are employed, but their inconsistent accuracy demands costly auditing, training, and monitoring. We propose an LLM-based modular classification pipeline that divides the relevance assessment task into multiple stages, each utilising different prompts and models of varying sizes and capabilities. Applied to TREC Deep Learning (TREC-DL), one of our approaches showed an 18.4% Krippendorff’s 𝛼 accuracy increase over OpenAI’s GPT-4o mini while maintaining a cost of about 0.2 USD per million input tokens, offering a more efficient and scalable solution for relevance assessment. This approach beats the baseline performance of GPT-4o (5 USD). With a pipeline approach, even the accuracy of the
GPT-4o flagship model, measured in 𝛼, could be improved by 9.7%.

Cite

@inproceedings{schnabel2025multistage,
  title={Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment},
  author={Schnabel, Julian~A and Trippas, Johanne and Scholer, Falk and Hettiachchi, Danula},
  booktitle={Proceedings of the ACM Web Conference},
  year={2025},
 address = {New York, NY, USA},
 location = {Sydney, Australia},
 publisher = {ACM}
}

Abstract

This half-day interactive tutorial provides researchers with practical skills to use large language models (LLMs) for interactive information retrieval research. Through hands-on exercises and real-world research examples, participants will learn to set up LLMs locally, integrate them via APIs, and evaluate their outputs. The tutorial will explain which models suit specific research needs, offering participants a robust toolkit to enhance their work. Attendees will also gain insights into the latest developments in the field, ensuring they stay at the forefront of innovation. Ideal for researchers eager to explore new methodologies in information retrieval, this tutorial offers foundational knowledge and cutting-edge strategies to use LLMs in interactive information retrieval research.

Cite

@inproceedings{trippas2025applying,
  title={Applying Large Language Models to Interactive Information Retrieval: A Practical Exploration},
  author={Trippas, Johanne and Zendel, Oleg and Roegiest, Adam},
  booktitle={Proceedings of the 2025 Conference on Human Information Interaction and Retrieval},
  year={2025},
 address = {New York, NY, USA},
 location = {Melbourne, Australia},
 publisher = {ACM}
}

Abstract

The Information Retrieval (IR) field is undergoing rapid transformation, charged by advancements in Artificial Intelligence (AI) and evolving research practices. As IR redefines its role within the broader computing and information sciences, this is a crucial moment to reflect as ECIR community on its future direction. Critical discussions will focus on the challenges and opportunities ahead, and the emerging research questions reshaping how we understand and approach search technologies. This second Search Futures workshop will serve as a forum for the IR community to exchange ideas, voice concerns, and propose bold strategies to strengthen and move the field forward.

Cite

@inproceedings{clarke2025second,
  title={{The Second Search Futures Workshop}},
  author={Charles L. A. Clarke and Paul Kantor and Adam Roegiest and Ian Soboroff and Johanne~R. Trippas and Zhaochun Ren},
  booktitle={Proceedings of the European Conference on Information Retrieval (ECIR'25)},
  pages = {1--4},
  year={2025},
  series = {ECIR '25}
}

Abstract
Advancements in large language models (LLMs) have changed information retrieval, offering users a more personalised and natural search experience with technologies like OpenAI ChatGPT, Google Bard (Gemini), or Microsoft Copilot. Despite these advancements, research into user tasks and information needs remains scarce. This preliminary work analyses a Google Bard prompt log with 15,023 interactions called the Bard Intelligence and Dialogue Dataset (BIDD), providing an understanding akin to query log analyses. We show that Google Bard prompts are often verbose and structured, encapsulating a broader range of information needs and imperative (e.g., directive) tasks distinct from traditional search queries. We show that LLMs can support users in tasks beyond the three main types based on user intent: informational, navigational, and transactional. Our findings emphasise the versatile application of LLMs across content creation, LLM writing style preferences, and information extraction. We document diverse user interaction styles, showcasing the adaptability of users to LLM capabilities.

Cite

@inproceedings{trippas2024what,
 author = {Trippas, Johanne R and Al Lawati, Sara Fahad Dawood and Mackenzie, Joel and Gallagher, Luke},
 booktitle = {Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval},
 series = {SIGIR '24},
 title = {What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild},
 year = {2024},
 doi = {10.1145/3626772.3657914},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

Conversational search is rising in popularity. Recent advantages in machine learning and natural language processing increasingly enable the development of conversational systems such as chatbots or speech-enabled personal assistants. The general population is adopting conversational systems to give commands and search, or interact with information. This chapter provides an introduction and overview to searching through conversational interactions. This part of the book includes neighbouring research fields of information retrieval, such as spoken dialogue systems and question answering, illustrating the complexity of these conversational systems while arguing about approaching this problem in a multi-disciplinary manner. This chapter aims to provide a starting point to conversational search while identifying research challenges and opportunities through contrasting aspects of traditional web-search and conversational systems. We illustrate that conversational search is a different mode of interaction and needs renewed attention, expanding beyond the query-response paradigm. We highlight the inherently interactive procedure of searching for information. We also show that conversational search is a different mode of interaction and needs a revised focus to define and understand the interaction components.

Cite

@inbook{10.1145/3674127.3674135,
author = {Trippas, Johanne R.},
title = {Conversational Search},
year = {2024},
isbn = {9798400710506},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
edition = {1},
url = {https://doi.org/10.1145/3674127.3674135},
booktitle = {Information Retrieval: Advanced Topics and Techniques},
pages = {285–319},
numpages = {35}
}

Abstract

Conversational assistants are becoming prevalent among the wider population due to their simplicity and increasing utility. However, the shortcomings of these tools are as renowned as their benefits. In this work, we present a “first look” at an extensive collection of conversational queries, aiming to identify limitations and improvement opportunities specifically related to information access (i.e., search interactions). We explore over 600,000 Google Assistant interactions from 173 unique users, examining usage trends and the resulting deficiencies and strengths of these assistants. We aim to provide a balanced assessment, highlighting the assistant’s shortcomings in supporting users and delivering relevant information to user needs and areas where it demonstrates a reasonable response to user inputs. Our analysis shows that, although most users conduct information-seeking tasks, there is little evidence of complex information-seeking behaviour, with most interactions consisting of simple, imperative instructions. Finally, we find that conversational devices allow users to benefit from increased naturalistic interactions and the ability to apply acquired information in situ, a novel observation for conversational information seeking.

Cite

@inproceedings{trippas2024reevaluating,
 author = {Trippas, Johanne R and Gallagher, Luke and Mackenzie, Joel},
 booktitle = {Proceedings of the 33rd ACM International Conference on Information and Knowledge Management},
 series = {CIKM '24},
 title = {Re-evaluating the Command-and-Control Paradigm in Conversational Search Interactions},
 year = {2024},
 doi = {doi.org/10.1145/3627673.3679588},
 address = {New York, NY, USA},
 location = {Boise, ID, USA},
 publisher = {ACM}
}

Abstract
The increasing reliance on digital information necessitates advancements in conversational search systems, particularly in terms of information transparency. While prior research in conversational information-seeking has concentrated on improving retrieval techniques, the challenge remains in generating responses useful from a user perspective. This study explores different methods of explaining the responses, hypothesizing that transparency about the source of the information, system confidence, and limitations can enhance users’ ability to objectively assess the response. By exploring transparency across explanation type, quality, and presentation mode, this research aims to bridge the gap between system-generated responses and responses verifiable by the user. We design a user study to answer questions concerning the impact of (1) the quality of explanations enhancing the response on its usefulness and (2) ways of presenting explanations to users. The analysis of the collected data reveals lower user ratings for noisy explanations, although these scores seem insensitive to the quality of the response. Inconclusive results on the explanations presentation format suggest that it may not be a critical factor in this setting.

Cite

@inproceedings{lajewska2024explainability,
 author = {Łajewska, Weronika and Spina, Damiano and Trippas, Johanne R. and Balog, Krisztian},
 booktitle = {Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval},
 series = {SIGIR '24},
 title = {Explainability for Transparent Conversational Information-Seeking},
 year = {2024},
 doi = {10.1145/3626772.3657768},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

Searching through conversational interactions has been emphasized as the next frontier. Nowadays, conversational agents can generate natural language responses, transforming how we search for information. A key challenge in conversational information-seeking is how these agents present information: should they only reflect facts, cater to human cognitive preferences, or strike a balance between them? These challenges raise questions about aligning conversational agents with human cognitive processes. Our position paper emphasizes the role of narrative in addressing these questions. We explore how narratives influence human comprehension and propose a framework for optimal conversational narratives. These narratives aim to enhance interaction between humans and conversational agents in explanatory information-seeking scenarios.

Cite

@inproceedings{sadiri2024unveiling,
 title ={Unveiling Information Through Narrative In Conversational Information Seeking},
 Author = {Sadiri Javadi, Vahid and Trippas, Johanne R. and Lucie Flek},
 Booktitle = {Proceedings of the ACM International Conference on Conversational User Interfaces (CUI 2024)},
 Year = {2024},
 numpages = {6},
 pages = {1--6},
 doi={10.1145/3640794.3665884},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

The effectiveness of clarification question models in engaging users within search systems is currently constrained, casting doubt on their overall usefulness. To improve the performance of these models, it is crucial to employ assessment approaches that encompass both real-time feedback from users (online evaluation) and the characteristics of clarification questions evaluated through human assessment (offline evaluation). However, the relationship between online and offline evaluations has been debated in information retrieval. This study aims to investigate how this discordance holds in search clarification. We use user engagement as ground truth and employ several offline labels to investigate to what extent the offline ranked lists of clarification resemble the ideal ranked lists based on online user engagement. Contrary to the current understanding that offline evaluations fall short of supporting online evaluations, we indicate that when identifying the most engaging clarification questions from the user’s perspective, online and offline evaluations correspond with each other. We show that the query length does not influence the relationship between online and offline evaluations, and reducing uncertainty in online evaluation strengthens this relationship. We illustrate that an engaging clarification needs to excel from multiple perspectives, and SERP quality and characteristics of the clarification are equally important. We also investigate if human labels can enhance the performance of Large Language Models (LLMs) and Learning-to-Rank (LTR) models in identifying the most engaging clarification questions from the user’s perspective by incorporating offline evaluations as input features. Our results indicate that Learning-to-Rank models do not perform better than individual offline labels. However, GPT, an LLM, emerges as the standout performer, surpassing all Learning-to-Rank models and offline labels.

Cite

@inproceedings{tavakoli2024online,
 title ={Online and Offline Evaluation in Search Clarification},
 author = {Tavakoli, Leila and Trippas, Johanne R. and Zamani, Hamed and Scholer, Falk and Sanderson, Mark},
 journal = {ACM Transactions on Information Systems (TOIS)},
 year = {2024},
 numpages = {28},
 doi = {10.1145/3681786},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

Generative Information Retrieval (GenIR) signifies an advancement in Information Retrieval (IR). GenIR encourages more sophisticated, conversational responses to search queries by integrating generative models and chat-like interfaces. However, this approach retains core principles of traditional IR and conversational information seeking, illustrating its capacity to augment current IR frameworks. In this chapter, we propose that introducing GenIR enhances traditional information retrieval tasks and expands their scope. This allows systems to manage more complex queries, including generative, critiquing, and extractive tasks. These advancements surpass traditional systems, handling queries with greater depth and flexibility. This sometimes speculative chapter suggests Generative Information Access (GenIA), a term that more accurately encapsulates the widened scope and enhanced functionalities of GenIR, particularly in how this relates to tasks. By investigating the impact of GenIR, this discussion aims to reiterate that generative research should not abandon traditional interactive information retrieval research but rather incorporate it into future research and development efforts.

Cite

@incollection{trippas2024adapting,
 address = {Cham, Switzerland},
 author = {Trippas, Johanne R. and Spina, Damiano and Scholer, Falk},
 booktitle = {Information Access in the Era of Generative AI},
 editor = {White, Ryen W. and Shah, Chirag},
 publisher = {Springer Nature Switzerland AG},
 title = {Adapting Generative Information Retrieval Systems to Users, Tasks, and Scenarios},
 year = {2024}
}

Abstract
Creating and deploying customized applications is crucial for operational success and enriching user experiences in the rapidly evolving modern business world. A prominent facet of modern user experiences is the integration of chatbots or voice assistants. The rapid evolution of Large Language Models (LLMs) has provided a powerful tool to build conversational applications. We present Walert, a customized LLM-based conversational agent able to answer frequently asked questions about computer science degrees and programs at RMIT University. Our demo aims to showcase how conversational information-seeking researchers can effectively communicate the benefits of using best practices to stakeholders interested in developing and deploying LLM-based chatbots. These practices are well-known in our community but often overlooked by practitioners who may not have access to this knowledge. The methodology and resources used in this demo serve as a bridge to facilitate knowledge transfer from experts, address industry professionals’ practical needs, and foster a collaborative environment. The data and code of the demo are available at https://github.com/rmit-ir/walert.

Cite

@inproceedings{pathiyan2024walert,
 author = {Sachin Pathiyan Cherumanal and Lin Tian and Futoon M.~Abushaqra and Angel F.~ Magnossao de Paula and Kaixin Ji and Halil Ali and Danula Hettiachchi and Johanne~R. Trippas and Falk Scholer and Damiano Spina},
 booktitle = {Proceedings of the ACM Conference on Information Interaction and Retrieval},
 series = {CHIIR '24},
 title = {{Walert: Putting Conversational Search Knowledge into Action by Building and Evaluating a Large Language Model-Powered	Chatbot}},
 year = {2024},
 doi = {10.1145/3627508.3638309},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

With the rise and growing prevalence of generative models, particularly multi-modal ones, it is an opportune time to explore beyond existing interactive information retrieval research trends. Indeed, it is essential to determine new avenues to explore how users interact with these models as well as revisit existing avenues that can be embellished with new technology. In this session, we aim to create a venue to workshop ideas that explore the future of search experiences and user interactions with information in a collaborative, low-pressure environment. This UnExplored FrontCHIIRs workshop enables participants to form a sub-community within CHIIR to facilitate further development of the proposed ideas and allow deeper collaborative problem-solving than just presenting late-breaking work.

Cite

@inproceedings{roegiest2024unexplored,
  title={UnExplored FrontCHIIRs: A Workshop Exploring Future Directions for Information Access},
  author={Roegiest, Adam and Trippas, Johanne},
  booktitle={Proceedings of the 2024 Conference on Human Information Interaction and Retrieval},
  pages={436--437},
  year={2024},
  doi = {10.1145/3627508.3638302},
 address = {New York, NY, USA},
 location = {Washington, DC, USA},
 publisher = {ACM}
}

Abstract

The First Search Futures Workshop, in conjunction with the Fourty-sixth European Conference on Information Retrieval (ECIR) 2024, looked into the future of search to ask questions such as:
-How can we harness the power of generative AI to enhance, improve and re-imagine Information Retrieval (IR)?
-What are the principles and fundamental rights that the field of Information Retrieval should strive to uphold?
-How can we build trustworthy IR systems in light of Large Language Models and their ability to generate content at super human speeds?
-What new applications and affordances does generative AI offer and enable, and can we go back to the future, and do what we only dreamed of previously?

The workshop started with seventeen lightning talks from a diverse set speakers. Instead of conventional paper presentations, the lightning talks provided a rapid and concise overview of ideas, allowing speakers to share critical points or novel concepts quickly. This format was designed to encourage discussion and introduce a wide range of topics within a short period, thereby maximising the exchange of ideas and ensuring that participants could gain insights into various future search areas without the deep dive typically required in longer presentations. This report, co-authored by the workshop’s organisers and its participants, summarises the talks and discussions. This report aims to provide the broader IR community with the insights and ideas discussed and debated during the workshop – and to provide a platform for future discussion.

Cite

@article{azzopardi2024report,
 title={Report on The Search Futures Workshop at ECIR 2024},
 author={Leif Azzopardi and Charles L. A. Clarke and Paul Kantor and Bhaskar Mitra and Johanne R. Trippas and Zhaochun Ren},
 year = {2024},
 issue_date = {June 2024},
 publisher = {Association for Computing Machinery},
 address = {New York, NY, USA},
 volume = {58},
 number = {1},
 journal = {SIGIR Forum},
 numpages = {41}
}

Abstract

Spoken Conversational Search (SCS) poses unique challenges in understanding user-system interactions due to the absence of visual cues, and the complexity of less structured dialogue. Tackling the impacts of cognitive bias in today’s information-rich online environment, especially when SCS becomes more prevalent, this paper integrates insights from information science, psychology, cognitive science, and wearable sensor technology to explore potential opportunities and challenges in studying cognitive biases in SCS. It then outlines a framework for experimental designs with various experiment setups to multimodal instruments. It also analyzes data from an existing dataset as a preliminary example to demonstrate the potential of this framework and discuss its implications for future research. In the end, it discusses the challenges and ethical considerations associated with implementing this approach. This work aims to provoke new directions and discussion in the community and enhance understanding of cognitive biases in Spoken Conversational Search.

Cite

@inproceedings{ji2024towards,
author = {Ji, Kaixin and Pathiyan Cherumanal, Sachin and Johanne R. Trippas and Hettiachchi, Danula
and Salim, Flora D. and Scholer, Falk and Spina, Damiano},
booktitle = {Adjunct Publication of the 26th International Conference on Human-Computer Interaction with Mobile Devices and Services},
doi = {10.1145/3640471.3680245},
series = {MobileHCI '24},
title = {Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search},
year = {2024}
}

Abstract

Voice-based systems like Amazon Alexa, Google Assistant, and Apple Siri, along with the growing popularity of OpenAI’s Chat-GPT and Microsoft’s Copilot, serve diverse populations, including visually impaired and low-literacy communities. This reflects a shift in user expectations from traditional search to more interactive question-answering models. However, presenting information effectively in voice-only channels remains challenging due to their linear nature. This limitation can impact the presentation of complex queries involving controversial topics with multiple perspectives. Failing to present diverse viewpoints may perpetuate or introduce biases and affect user attitudes. Balancing information load and addressing biases is crucial in designing a fair and effective voice-based system. To address this, we (i) review how biases and user attitude changes have been studied in screen-based web search, (ii) address challenges in studying these changes in voice-based settings like SCS, (iii) outline research questions, and (iv) propose an experimental setup with variables, data, and instruments to explore biases in a voice-based setting like Spoken Conversational Search.

Cite

@inproceedings{pathiyan2024towards,
author = {Pathiyan Cherumanal, Sachin and Scholer, Falk and Johanne R. Trippas and Spina, Damiano},
booktitle = {ACM International Conference on Multimodal Interaction},
doi = {10.1145/3686215.3690156},
series = {ICMI '24},
title = {{Towards Investigating Biases in Spoken Conversational Search}},
numpages = {6},
year = {2024}
}

Abstract
Information-seeking dialogues span a wide range of questions, from simple factoid to complex queries that require exploring multiple facets and viewpoints. When performing exploratory searches in unfamiliar domains, users may lack background knowledge and struggle to verify the system-provided information, making them vulnerable to misinformation. We investigate the limitations of response generation in conversational information-seeking systems, highlighting potential inaccuracies, pitfalls, and biases in the responses. The study addresses the problem of query answerability and the challenge of response incompleteness. Our user studies explore how these issues impact user experience, focusing on users’ ability to identify biased, incorrect, or incomplete responses. We design two crowdsourcing tasks to assess user experience with different system response variants, highlighting critical issues to be addressed in future conversational information-seeking research. Our analysis reveals that it is easier for users to detect response incompleteness than query answerability and user satisfaction is mostly associated with response diversity, not factual correctness.

Cite

@inproceedings{lajewska2024explainability,
 author = {\L{}ajewska, Weronika and Balog, Krisztian and Spina, Damiano and Trippas, Johanne R.},
 booktitle = {Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region},
 series = {SIGIR-AP '24},
 title = {{Can Users Detect Biases or Factual Errors in Generated Responses in Conversational Information-Seeking?}},
 year = {2024},
 url = {https://doi.org/10.1145/3673791.3698409},
 doi = {10.1145/3673791.3698409},
 address = {New York, NY, USA},
 location = {Tokyo, Japan},
 publisher = {ACM}
}

Abstract

Cite

@inproceedings{javadi2024can,
title={Can Stories Help {LLM}s Reason? Curating Information Space Through Narrative},
author={Vahid Sadiri Javadi and Johanne Trippas and Lucie Flek},
booktitle={The First Workshop on System-2 Reasoning at Scale, NeurIPS'24},
year={2024}
}

Abstract

This study is the first attempt to explore the impact of clarification question modality on user preference in search engines. We introduce the multi-modal search clarification dataset, MIMICS-MM, containing clarification questions with associated expert-collected and model-generated images. We analyse user preferences over different clarification modes of text, image, and combination of both through crowdsourcing by taking into account image and text quality, clarity, and relevance. Our findings demonstrate that users generally prefer multi-modal clarification over uni-modal approaches. We explore the use of automated image generation techniques and compare the quality, relevance, and user preference of model-generated images with human-collected ones. The study reveals that text-to-image generation models, such as Stable Diffusion, can effectively generate multi-modal clarification questions. By investigating multi-modal clarification, this research establishes a foundation for future advancements in search systems.

Cite

@inproceedings{tavakoli2024understanding,
author = {Leila Tavakoli and Giovanni Castiglia and Federica Cal{\`o} and Yashar Deldjoo and Hamed Zamani and Johanne R. Trippas},
booktitle = {1st Workshop on Multimodal Search and Recommendations},
series = {CIKM MMSR '24},
title = {{Understanding Modality Preferences in Search Clarification}},
numpages = {9},
year = {2024}
}

Abstract

The field and community of Information Retrieval (IR) are changing and evolving in response to the latest developments and advances in Artificial Intelligence (AI) and research culture. As the field and community re-oriented and re-consider its positioning within computing and information sciences more generally – it is timely to gather and discuss more seriously our field’s vision for the future – the challenges and threats that the community and field faces – along with the bold new research questions and problems that are arising and emerging as we re-imagine search. This workshop aims to provide a forum for the IR community to voice and discuss their concerns and pitch proposals for building and strengthening the field and community.

Cite

@inproceedings{azzopardi2024search,
  title={{The Search Futures Workshop}},
  author={Leif Azzopardi and Charles L. A. Clarke and Paul Kantor and Bhaskar Mitra and \textbf{J.~R. Trippas} and Zhaochun Ren},
  booktitle={Proceedings of the European Conference on Information Retrieval (ECIR'24)},
  pages = {1--4},
  year={2024},
  series = {ECIR '24}
}

Abstract
Topic models such as Latent Dirichlet Allocation (LDA) have been extensively used for characterizing text collections according to the topics discussed in documents. Organizing documents according to topic can be applied to different information access tasks such as document clustering, content-based recommendation or summarization. Spoken documents such as podcasts typically involve more than one speaker (e.g., meetings, interviews, chat shows or news with reporters). This paper presents a work-in-progress based on a variation of LDA that includes in the model the different speakers participating in conversational audio transcripts. Intuitively, each speaker has her own background knowledge which generates different topic and word distributions. We believe that informing a topic model with speaker segmentation (e.g., using existing speaker diarization techniques) may enhance discovery of topics in multi-speaker audio content.

Cite

@inproceedings{spina2015speakerlda,
  title={SpeakerLDA: Discovering topics in transcribed multi-speaker audio contents},
  author={Spina, Damiano and Trippas, Johanne R and Cavedon, Lawrence and Sanderson, Mark},
  booktitle={Proceedings of the Third Edition Workshop on Speech, Language \& Audio in Multimedia},
  pages={7--10},
  year={2015}
}

Abstract
We propose research to investigate a new paradigm for Interactive Information Retrieval (IIR) where all input and output is mediated via speech. Our aim is to develop a new framework for effective and efficient IIR over a speech-only channel: a Spoken Conversational Search System (SCSS). This SCSS will provide an interactive conversational approach to determine user information needs, presenting results and enabling search reformulations. We have thus far investigated the format of results summaries for both audio and text, features such as summary length and summaries documents (noisy document or clean document) generated from (noisy) speech-recognition output from spoken document. In this paper we discuss future directions regarding a novel spoken interface targeted at search result presentation, query intent detection, and interaction patterns for audio search.

Cite

@inproceedings{trippas2015results,
author = {Trippas, Johanne R. and Spina, Damiano and Sanderson, Mark and Cavedon, Lawrence},
title = {Results Presentation Methods for a Spoken Conversational Search System},
year = {2015},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/2810355.2810356},
booktitle = {Proceedings of the First International Workshop on Novel Web Search Interfaces and Systems},
pages = {13--15},
numpages = {3},
location = {Melbourne, Australia}
}