MASARYK U N I V E R S I T Y FACULTY OF INFORMATICS Personalized Similarity Search for Vector Databases Advanced Master's Thesis MGR. ET MGR. MATÚŠ ŠIKYŇA Brno, Fall 2025 MA SA R Y K U N I V E R S I T Y FACULTY OF INFORMATICS Personalized Similarity Search for Vector Databases Advanced Master's Thesis MGR. ET MGR. MATÚŠ ŠIKYŇA Advisor: prof. Ing. Pavel Zezula, CSc. Department of Machine Learning and Data Processing Brno, Fall 2025 Signature of Thesis Advisor Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. A l l sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. During the preparation of this thesis, I used ChatGPT and Grammarly to improve my writing style. I declare that I used the tool in accordance with the principles of academic integrity. I checked the content and take full responsibility for it. Mgr. et Mgr. Matúš Šikyňa Advisor: prof. Ing. Pavel Zezula, CSc. iii Abstract Vector databases store large collections of high-dimensional embeddings and make them searchable by building an index around a fixed similarity function, most commonly Euclidean distance. This design enables efficient retrieval at scale, but it also hard-codes a single notion of similarity for all users, even though human similarity judgements are subjective and context-dependent. Changing the similarity model typically implies rebuilding the index, which is impractical when many users share the same collection but require different views of relevance. We address this gap by introducing a similarity search engine that maintains a shared Euclidean index while allowing each user to search through a personalized similarity view, expressed as a learned Mahalanobis metric parameterized by a small matrix. We validate the proposed theory experimentally and show that semantic relevance feedback from user interactions can effectively learn and update user matrices. To translate the relevance-efficiency trade-off into practical deployment guidance, we define operational scenarios that balance personalization quality against computational cost, ranging from configurations that prioritize efficiency and stable rankings to those that maximize retrieval effectiveness. Across multiple feedback-processing strategies and a broad suite of metric learning models, we achieve high personalization gains, increasing the mean average precision by up to 0.211 while keeping scaling factors low, demonstrating that effective and efficient personalized similarity search is possible without modifying the shared index. Keywords metric learning, similarity search, vector embeddings, learning user profiles, relevance feedback, vector databases, personalized search iv Contents 1 Introduction 1 2 State of the Art 5 2.1 Traditional Personalized Search Approaches 5 2.1.1 Interactive Relevance Feedback and Similarity Learning 5 2.1.2 Implicit Feedback Profiling 6 2.1.3 ODP-based Approaches 7 2.1.4 Group and Collaborative Approaches 8 2.1.5 Context, Engagement, and Temporal Dynamics . . . 9 2.1.6 Other Approaches 10 2.2 Vector Embeddings for Personalized Search 11 2.2.1 Query Expansion and Query Reformulation 11 2.2.2 Approaches Based on User Context 12 2.2.3 Product or Marketplace Search and Recommendation 13 2.2.4 Social-augmented and Knowledge-augmented Embeddings 15 2.2.5 Multimodal Personalized Retrieval 15 2.3 Our Approach 16 3 Achieved Results 18 3.1 Towards Personalized Similarity Search for Vector Databases 18 3.2 Integrating Relevance Feedback for Effective Personalisation in Vector Search 21 3.3 Author's Publications 24 Bibliography 25 A Appendix 33 B Appendix 48 v List of Tables 3.1 Scaling factors SM,E(A) and average candidate set sizes under Euclidean retrieval: #£ for range and #opt£ for range roptfC^)-The dataset contains 226,778 vectors. . . 21 3.2 Calculated evaluation measures for the best model given the scenario and its variant. The Model column is in the form: model and hyperparameters. The Feedback column is in the form: feedback strategy and parameters - 1: (Number of pairs, Replacement), 2: (Batch), 3:(Number of queries) 23 vi List of Figures 3.1 Overview of the personalised similarity search system . . 18 3.2 Inclusions between the answer sets 19 3.3 VE, TM> a n d ?OptE{~cl) across test queries for six matrices. . 20 3.4 Workflow of the semantic-feedback pipeline. The query is shown in yellow, positive results in green, and negative results in red 22 vii 1 Introduction Recent advancements in artificial intelligence, particularly in representation learning, have produced models that map text, images, and other modalities into high-dimensional vector embeddings. These embeddings encode rich semantic information, enabling efficient similaritybased retrieval, and are employed across various applications, such as recommender systems [1], clustering [2], classification [3], and information retrieval [4]. To operate with such representations at scale, vector databases are used to store and manage embeddings, enabling efficient similarity search over billions of high-dimensional vectors. Such databases are now commonly used as the retrieval layer in semantic search, recommendation, and retrieval-augmented generation pipelines, where relevant items or documents are retrieved based on similarity before further processing. To achieve low latency at scale, they commonly employ approximate nearest-neighbour (ANN) indexing techniques that trade a small amount of recall for substantial speedups. This efficiency comes from precomputing index structures over the vectors, so the A N N index acts as a candidate generator, narrowing the search space before the final similarity evaluation. Vector databases, therefore, expose similarity search as their core query operation. The quality of the returned results depends not only on the embedding model, but also on how the index generates and orders candidates under the chosen similarity function. In practice, vector databases commonly rely on a fixed distance function (e.g., Euclidean distance) to organize the index and to select the most relevant vectors with respect to a given query or reference vector. However, once the content is embedded and a single distance function is utilized, the candidate selection behaviour is identical across all users, while changing this notion of similarity generally requires rebuilding or substantially reconfiguring the index. This is in sharp contrast with human perception of similarity, which is subjective and context-dependent [5]. This observation connects the indexing issue with a broader problem. In principle, there is a gap between the global relevance suggested by general-purpose embeddings and the personal relevance experienced by an individual user. What is relevant is not only a property of the 1 i . INTRODUCTION content itself, but also of the user, the current intent, and the situation in which the request is made. A practical answer to this gap is contextaware computing, where the same query can legitimately produce different results for different users, even when operating over the same underlying database. Such motivation has been visible for a long time in large-scale systems. Web search moved beyond purely universal ranking early on, as users typing identical queries often meant different things and valued different sources. Therefore, industrial systems introduced personalization signals derived from user interactions to adjust rankings accordingly. In a different but closely related setting, services such as Netflix built their user experience around personalization from the start: the catalogue is the same, but what is shown, in what order, and in what context is tailored to the individual. Similar motivations apply in advertising and e-commerce, where even minor improvements in personal relevancy can translate into substantial gains. Because such techniques have commercial potential, the precise system designs and evaluation procedures behind deployed solutions are typically proprietary. As a result, the research community has introduced a broad spectrum of personalized search methods that exploit behavioural and contextual evidence, including query history, clicks, dwell time, location, and social or topical profiles, to rerank results for each user. The proposed techniques range from probabilistic models and collaborative filtering to deep neural architectures designed to model both shortterm and long-term interests together with situational context [6, 7,8, 9,10,11,12,13,14,15,16,17,18,19,20]. Later, representation-based approaches moved this idea into vector spaces by learning neural embeddings for users, queries, items, and entities, so that similarity in the learned space directly encodes user-specific preferences for search or recommendation [21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33]. More recently, personalization has been extended with knowledgegraph signals, self-supervised objectives, and LLM-based components, including editable or entity-centric memories, motivation-aware embeddings, and multimodal user representations that provide finergrained and more context-sensitive signals [34, 35, 36, 37]. Despite this progress, embedding-based retrieval systems that rely on a sin- 2 i . INTRODUCTION gle shared index still face the practical constraint that the candidate generation stage is tied to a fixed metric. In this work, we focus on vector databases and bridge the gap between the shared notion of similarity encoded by general-purpose embeddings stored using the fixed index and the personalized notion of similarity that individual users apply during similarity search. Specifically, we elaborate on an architecture in which all vectors are stored in a single shared index built for a standard metric (Euclidean), while each user queries the same database using a personalized similarity model expressed as a Mahalanobis metric parameterized by a learned matrix. We learn this matrix from user relevance feedback, framing personalization as a metric learning problem [38] in which the geometry of the embedding space is adapted to reflect an individual's notion of similarity. More generally, metric learning uses supervision, such as pairwise or triplet preferences, to learn a distance function that brings relevant items closer together while pushing irrelevant items further apart. Furthermore, our design exploits the lower bounding relationship [39] between the Euclidean and Mahalanobis distances. This relationship provides a formal guarantee that Euclidean-based candidate generation returns a superset of the true Mahalanobis neighbours, enabling personalization in a refinement stage without rebuilding the shared index. To implement a practicalfilter and refine search, the Euclidean index efficiently retrieves a guaranteed superset of candidates, which is then refined using the user-specific Mahalanobis metric. The approach is investigated through two studies. Our first study formulates relevance feedback as supervision for metric learning, proposes a search engine architecture, and validates the proposed approach using synthetic feedback signals. Our second study focuses on semantic relevance feedback collected from real user interactions with the returned results, and uses it to learn and update the personalized similarity models in practice. We propose and compare several feedback processing strategies for constructing feedback judgements (in the form of triplets), evaluate them across a range of established metric learning models, and suggest practical system setup strategies under different operational priorities (e.g., trading relevance gains against computational cost, or prioritizing efficiency vs. precision). 3 i . INTRODUCTION The work is organized as follows: Chapter 2 reviews personalized search approaches, ranging from traditional feature-based methods that leverage behavioural and contextual signals to representationbased techniques using embeddings and large language models in vector spaces. Chapter 3 summarizes the achieved results from the published papers. Appendix A and Appendix B contain the full versions of our publications. 4 2 State of the Art Personalized search utilizes signals from a user's prior interactions to infer their interests and tailor rankings, ensuring results better fit their current information needs. This chapter surveys two broad families of methods. First, we review traditional personalization techniques that rely on engineered behavioural and contextual signals (including queries, clicks, dwell time, time, and location) and incorporate them through relevance feedback, user profiling, topical taxonomies, collaborative signals, and learning-to-rank or neural models. Second, we cover embedding-based approaches that represent users, queries, and items in shared vector spaces, enabling personalization through representation learning and similarity-based retrieval, as well as modern transformer and LLM-based encoders with explicit user memory. Finally, we position our approach relative to these lines of work. 2.1 Traditional Personalized Search Approaches Traditional personalized search methods typically break down the problem into: extracting behavioural and contextual evidence, constructing a user model (e.g., profile, topic distribution, or similarity function), and incorporating that model into ranking through re-ranking, query expansion, or feature augmentation. The literature encompasses (i) interactive relevance feedback loops that update user profiles online, (ii) implicit feedback profiling from queries and clicks, and (iii) ontology-based approaches (e.g. using ODP) that extend personalization to the topic level. To address sparse or noisy individual signals, (iv) group and collaborative methods smooth user models using related users. Furthermore, (v) context-aware and time-aware models capture situational intent and temporal drift across sessions. In the following subsections, we describe these lines of work and highlight representative methods within each category. 2.1.1 Interactive Relevance Feedback and Similarity Learning Early personalization systems treat personalization as an interactive loop. The system observes feedback (explicit or implicit), updates 5 2. STATE OF THE A R T a user model online, and decides what to show next. This family includes relevance-feedback-style Bayesian updating and online similarity/metric learning. PicHunter [6], a content-based image retrieval system was introduced, using a Bayesian relevance feedback loop: after each display, it updates a posterior probability over all images from user actions (e.g., selecting relevant images) using a user model derived from similarity judgment experiments, and selects the next display via an entropy-minimizing strategy, leveraging hidden semantic annotation and pictorial feature vectors (e.g., color). The limitation of the system is that it scales linearly with database size (feature storage and repeated user model evaluations). Therefore, it is not suited for largescale collections (e.g., hundreds of millions to billions of images). Chechik et al. [40] introduced OASIS, an online method that learns a bilinear similarity over sparse image features from relative similarity constraints (e.g., images sharing a label or retrieved for the same text query), using a large margin criterion and an efficient hinge loss cost, scaling linearly to millions of images. The learned model is classindependent. Therefore, it is not aware of which queries or categories were shared by two similar images. 2.1.2 Implicit Feedback Profiling A large body of work personalizes search by building profiles from implicit behavioural signals (e.g., queries, clicks, visited pages, and query reformulations) and uses the resulting profile for re-ranking or for generating query suggestions. Sugiyama et al. [7] built per-user profiles from user browsing histories (in one day). A limitation of the approach is its dependence on a sufficient and reliable history, as well as sensitivity to the window size (in days) parameters. Shen et al. [8] proposed a client-side agent, UCAIR, that performs eager implicit feedback, using the immediately preceding query and clickthrough information for query expansion to improve the accuracy of ad-hoc retrieval. Teevan et al. [9] constructed user profiles implicitly based on past queries, visited pages, and even local documents and emails, treating them as a form of relevance feedback for reranking the results. A 6 2. STATE OF THE A R T limitation of the approach is that personalization gains are highly parameter- and query-dependent, so it likely requires tuning per query or user and can yield volatile results as profiles and contexts change. Another work [41] implicitly learned past and current topical preferences from clickthrough logs (queries and visited pages) using a topical ontology. The approach assigns topics to previously visited pages, then updates the user profile by relating the current query to the user's past topic preferences. Search results are then re-ranked using a scoring function that favours pages whose inferred topics best match the topics in the user's profile. A limitation is that the effectiveness of the approach depends on the accuracy of topic classification. Furthermore, V u et al. [42] personalized query suggestions for each search session by building two temporal user profiles: one from clicked documents and one from the user's query modification history. A learning-to-rank model (LambdaRank [43]) is used to re-rank the query suggestion list. Bennett etal. [13] studied how short-term (session) and long-term (historic) behaviour interact in personalization using large-scale query and click logs, and show that historic signals help most at the start of a session, while session signals contribute most of the gains later in longer sessions. They further find that combining historic and session evidence outperforms either alone. 2.1.3 ODP-based Approaches ODP-based approaches personalize search by mapping user interests onto a fixed topical directory (the Open Directory Project). Personalization then becomes topic-level matching by inferring which O D P categories a user prefers and re-rank results using topic-biased scores. Qiu and Cho [11] inferred a user's preference from past click history and personalized ranking by selecting Topic-Sensitive PageRank scores over O D P topics for the user and query. Extending the approach to richer signals raises challenges in integrating heterogeneous sources. Another work [10] built implicit profiles from search-engine side activity (via a Google wrapper), using queries that led to clicks and the clicked results' snippets (titles and summaries), mapping this peruser information to ODP concepts, and re-ranking results accordingly. 7 2. STATE OF THE ART However, the approach is constrained to what the site can observe (queries and snippets for clicked items) and depends on the tuning and dynamicality of the concept hierarchy. Zhongming et al. [44] proposed a client-side personalized categorization system, PCAT, that maps known user interests (e.g., workplace skills) to ODP categories, trains ODP-based text classifiers, and then categorizes search results by those interests on top of a standard engine (e.g., Google). A limitation of the approach is that it assumes reliable, pre-known interest data and accurate mappings into the ODP taxonomy. In practice, users may have unknown or shifting interests, or issue out-of-profile queries (e.g., unrelated to work), so many tasks may not align with any known interest, and personalization cannot help beyond what the system already knows. Sieg et al. [45] built ontological user profiles by assigning and updating interest scores on concepts in a domain ontology (ODP), using a spreading activation algorithm driven by the user's ongoing implicit behaviour, and then re-rank results using these interest scores. A limitation authors highlight is the need to understand the stability and convergence properties of the profiles to represent user interests reliably. 2.1.4 Group and Collaborative Approaches When individual histories are sparse or noisy, personalization can be improved by leveraging signals from other individuals, such as similar users, participation in groups, social connections, or crowds performing similar tasks. These methods cluster users, smooth individual profiles with group behaviour, and improve cold-start robustness. Some works constructed user profiles based on collaborative filtering. Xue et al. [46] modelled personalization with a user language model that combines individual profiles, based on queries and visited page content, with group and global behaviour. Users are clustered into groups, and group models are built. Relevance is computed via a two-step smoothing process: global smoothing for unseen terms, followed by group-based interpolation. However, the effectiveness of the approach depends on the quality of the clustering and the tuned parameters. 8 2. STATE OF THE A R T Carmel et al. [47] personalized search by re-ranking results using signals from a user's social network, including familiarity-based and similarity-based relations inferred from social activity (e.g., tagging, commenting) and boosting documents by their relationship strength to the user's related people. Teevan et al. [48] studied groupization, combining an individual's data with that of related users, analyzing similarity in query selections, desktop content, and explicit relevance judgments across people grouped in different ways. They found that the approach is most effective for explicit groups (e.g., task, occupation, or interest-based) on queries related to the group's focus, while implicitly inferred groups and off-theme queries are less cohesive and yield fewer reliable gains. White et al. [49] moved beyond query matching by mining search logs to build task models, finding other users performing similar tasks, and promoting URLs based on on-task click behaviour. The authors note that the approach requires accurate modelling of search tasks, and for further gains, likely needs richer task features beyond queries or clicks (e.g., success signals or post-click behaviour), and potentially deeper re-ranking results when relevant URLs are not already near the top. Vu et al. [15] proposed query-dependent dynamic grouping: given an input query, they identify a group of users with similar interests for that query, enrich the user profile (based on L D A [50]) using those users' signals, and re-rank results accordingly. Furthermore, PerSaDoR [16] constructs a personalized social document representation for each user-document pair from social-tagging (folksonomy) annotations using matrix factorization, and combines this personalized representation with the document's textual ranking score to rerank results. The authors report significant performance gains with query-time costs that scale linearly with the number of matched documents. 2.1.5 Context, Engagement, and Temporal Dynamics Beyond what the user likes, personalization also depends on situational and temporal factors: where the user is, what device they use, how engaged they are with items, and how intent drifts over time. These works incorporate context features (e.g., location, time, or device), 9 2. STATE OF THE A R T engagement measures (e.g., dwell time), and temporal modelling assumptions. Lu et al. [12] focused on implicit local-intent queries, using users' physical location to personalize results. Furthermore, Y i et al. [14] used item-level dwell time to measure the time of user engagement on a specific item. The primary practical challenges of this approach lie in measuring and normalizing dwell time robustly and in designing engagement objectives that more effectively capture long-term user satisfaction. Zamani et al. [17] shifted context modelling from history-based signals to situational context (e.g., time, location, device) that is independent of query content and user history, proposing neural contextaware ranking models that learn dense representations from sparse contextual features. They evaluated the approach on a large personal search engine, where their model significantly improved the personal- ization. Ge et al. [18] exploited sequential structure in user logs using a hierarchical Recurrent Neural Network (RNN). A lower-level R N N models current session (short-term behaviour), a higher-level R N N models dependencies across sessions (long-term behaviour), and a query-aware attention mechanism dynamically weights past sessions to form a query-specific user profile before re-ranking. They observed significant gains over traditional personalization approaches and showed that attention is able to highlight the important parts from previous queries and sessions. Ma et al. [20] argued that sequential personalization models overlook fine-grained time information associated with user actions, and introduced a new model, PSTIE, to incorporate temporal information. The time-aware LSTM model captures the short-term evolution of query intent and document interest, while long-term re-finding behaviour is modelled via a query-specific Gaussian mixture that drifts over time. They report improvements in the performance of personalized ranking. 2.1.6 Other Approaches Lu et al. [19] propose KEPS, a knowledge graph enhanced personalized search model that performs personalized entity linking on the 10 2. STATE OF THE A R T user's queries to form a better intent representation, builds a knowledge enhanced user profile by storing linked entities and predicted intents from search history in a memory network, and uses click feedback to post-adjust entity linking for past queries to correct earlier disambiguation errors. Experiments on the AOL log show that these three components jointly improve ranking accuracy. Another approach [51] proposes query-specific concept-based user profiles learned from clickthrough logs, explicitly modelling both positive and negative preferences. The approach utilizes Ranking S V M [52] to learn weighted concept vectors that improve personalized query clustering. The study finds that incorporating negative preferences not only improves cluster quality but also increases the separation between similar and dissimilar queries, pushing them into more distant clusters. 2.2 Vector Embeddings for Personalized Search In contrast to traditional feature-based methods, a growing line of work frames personalization as representation learning in shared embedding spaces. This section organizes embedding-based personalization into four themes: (i) query expansion and reformulation, (ii) user-context injection methods that personalize matching and scoring directly, including self-supervised contrastive pretraining under sparse logs, (iii) product and marketplace search models that jointly embed users, queries, and items and often use sequential transformers over purchase or click histories, and (iv) embeddings that incorporate social graphs, groups, or knowledge graphs to disambiguate intent and improve cold-start robustness. We also briefly note on multimodal personalized retrieval. 2.2.1 Query Expansion and Query Reformulation These methods personalize at the query level. They use embedding proximity to add terms, substitute terms, or rewrite queries, aiming to resolve vocabulary mismatch and capture user-specific semantics. Amer et al. [21] used Word2Vec [53] embeddings for query expansion by training the embedding model on the user profile text. 11 2. STATE OF THE ART They discovered that sparse or low-quality profile text can yield weak embeddings. Therefore, the personalized variant does not consistently improve effectiveness and may require richer profile evidence or auxiliary signals (e.g., neighbours) to be useful. Kuzi et al. [22] proposed query expansion where Word2Vec is trained once on the entire target corpus and expansion terms are selected by semantic proximity in the learned vector space, either to the query as a whole or to individual query terms. The selected terms are used either directly to expand the query or are integrated with a pseudo-relevance-feedback relevance model. The authors report significant improvements in performance over using the raw query alone. The method is not personalized, but it can serve as a starting point for personalization by introducing user-specific embedding spaces or combining them with the global space. Zhang [32] personalized query reformulation with embeddings integrated into both candidate generation and scoring. Candidates are generated via term expansions or substitutions augmented with a user embedding, and then re-ranked using a graphical model enriched by term, user, and topic embeddings to capture semantic consistency and topical dependencies. Limitations include reliance on sufficient query history to form topic profiles and the constrained expressivity of single-term edits, which may miss more complex reformulations. Bassani et al. [33] proposed a method, PQEWC, for personalized query expansion with contextual word embeddings, addressing redundancy and scalability caused by multiple similar embeddings. Their approach clusters user-term embeddings offline and selects representative expansion terms per cluster, coupled with an approximation scheme that enables sub-millisecond expansion while improving effectiveness and term diversity. Limitations include reliance on cosine similarity (rather than a learned scoring function) and the use of a fixed number of expansion terms, rather than predicting the optimal expansion amount. 2.2.2 Approaches Based on User Context These approaches personalize ranking by injecting user context, learned from a person's past queries, clicks, or stored memories, directly into the matching and scoring process. 12 2. STATE OF THE A R T Vu et al. [23] proposed a user profiling method where each profile is represented by a user embedding together with two projection matrices that capture interest-dependent aspects of submitted queries and clicked (relevant) documents, while the user embedding captures the relationship between the queries and documents in this user interest-dependent subspace. The resulting profile is then used directly to re-rank a commercial search engine's results, yielding stable and significant improvements in ranking. Yao et al. [26] proposed PEPS, which personalizes ranking by learning personal word embeddings so that ambiguous terms acquire user-specific semantics from each person's search history. Queries and documents are encoded using multi-head self-attention to form personalized contextual representations, and a neural matching model ( K N R M [54]) is used to score relevance. Experiments on large-scale query logs showed significant performance improvements. Mysore et al. [35] proposed CtrlCE, a controllable personalized search cross-encoder augmented with an editable user memory built from historical user documents (item-based or concept-based). A calibrated mixing model determines when personalization should be applied. PSSL [29] proposed a two-stage, self-supervised framework for personalized search that pre-trains both a sentence encoder and a sequence encoder before fine-tuning on ranking. It uses contrastive sampling to mine paired signals from query logs at two levels: within a user (self-contrastive) and across users (user-contrastive). From these, it constructs four kinds of contrastive pairs: query pairs, document pairs, sequence augmentation pairs, and user pairs. Then, it optimizes contrastive losses so that representations of similar queries/documents/behaviour sequences are pulled closer together in the embedding space, yielding more robust representations for personalization under sparse logs. 2.2.3 Product or Marketplace Search and Recommendation These works focus on commerce retrieval, where short queries, rich item content (e.g., titles or reviews), and strong behavioural signals (e.g., click or purchase) make joint embedding spaces particularly effective. Models typically embed users, queries, and items or products 13 2. STATE OF THE ART together, often with sequence models over history to capture evolving intent. In a field of personalized product search, A i et al. [24] proposed a hierarchical embedding model that jointly learns latent representations of users, queries, and products from associated language data (e.g., reviews). Experiments on Amazon benchmarks showed consistent performance improvements, while the authors note that gains and the importance of personalization vary substantially across datasets (e.g., with review or query sparsity). Bi et al. [27] proposed T E M , a transformer-based embedding model that encodes the sequence of the current query, along with the purchase history, to dynamically control how much personalization should influence retrieval. Multi-layer T E M can also model interactions among historical purchases, yielding more informative attention weights and better dynamic representations. Experiments on the Amazon product search dataset show significant performance improvements. In an applied marketplace setting, Grbovic and Cheng [25] learned Airbnb-specific listing and user embeddings for real-time personalization in search ranking and similar-listing recommendations, explicitly modelling each guest's short-term and long-term interests. Wang et al. [31] proposed an embedding model that maps users and items into a shared vector space for e-commerce recommendations at eBay. To reduce cold-start, item embeddings rely on content features, while user embeddings are learned from multi-modal onsite behaviour (e.g., item clicking and query searching). Candidates are retrieved via K N N in the embedding space. A limitation noted by authors is the single-vector user representation, which may collapse multiple concurrent interests. Furthermore, Qin et al. [36] proposed MAPS for personalized product search by incorporating pre-search consultation text to capture user motivations that are not expressed in short queries. Queries and consultations are embedded into a unified semantic space with Large Language Model (LLM). A Mixture of Attention Experts focuses on critical semantics and dual alignment (contrastive and bidirectional attention), bridges consultation text, product features, and user history. While M A P S improves personalized search via query-consultation 14 2. STATE OF THE ART semantic alignment, it does not fully address efficiency and real-time scalability and lacks explicit modelling of evolving user behaviour. 2.2.4 Social-augmented and Knowledge-augmented Embeddings When individual history is sparse or ambiguous, embedding personalization can be improved by injecting structure beyond the user's own logs, such as friend networks, group behaviour, or knowledge graphs. FNPS [28] is a group-based personalized search model that constructs dynamic group profiles from users' friend networks and historical search behaviour using graph attention and cross-attention mechanisms, and combines these with individual profiles to generate personalized results. ReBKC [30] treats the knowledge graph as a heterogeneous network and learns unified embeddings that combine users' behaviour signals with knowledge relations. It mines short-term and long-term preferences from historical interactions using a self-attention mechanism, and then leverages graph attention over the knowledge graph neighbourhood to weight different knowledge entities by their userspecific importance. The authors note that their knowledge graph embeddings are primarily learned to reconstruct triples rather than being optimized for recommendation. Baek et al. [34] recently proposed a lightweight LLM-based personalization framework that constructs an entity-centric knowledge store for each user from their search and browsing histories, aligns these aggregated entities with public knowledge graphs, and then leverages the resulting user-specific context to augment L L M prompts for tasks like contextual query suggestion, yielding more relevant and useful personalized generations. 2.2.5 Multimodal Personalized Retrieval Finally, in the multimodal domain, Ryan et al. [37] propose P O L A R for personalized vision-language retrieval by adapting a CLIP model's language encoder via regularized low-rank adaptation (LoRA) on the final layer, learning new personal concepts from a few images while aiming to preserve general knowledge. 15 2. STATE OF THE ART 2.3 Our Approach Most existing personalization models intervene at the representation or ranking stages. They mine user-specific or group-specific signals from a person's own history and context (queries, clicks, dwell time, time, location), and often also from other users (similar users, groups, social graphs), then apply personalization through query expansion/reformulation, user/profile embeddings, or learned ranking functions that re-rank candidates produced by a global retriever. This can deliver substantial gains, but it often couples personalization with data and the ranking model, increasing the amount of user-specific state to maintain (e.g., embeddings, memories, models, features or profiles) and complicating deployment. More broadly, even when personalization is powerful, many pipelines still rely on a shared firststage retrieval mechanism to generate candidates, then personalize after retrieval. In vector databases, however, the candidate set is determined by an index built around a fixed similarity function (e.g., Euclidean), so a user's true nearest neighbours under their personalised notion of similarity may never be considered unless the system rebuilds indices per user. In contrast, our approach (see Chapter 3 for more details) personalizes search in vector databases at the level of the similarityfunction itself. Specifically, we model personalization as learning a per-user matrix that parameterizes a user-specific similarity function (Mahalanobis distance) over a common embedding space, indexed by a fixed index similarity function (Euclidean distance). Each user is represented by only this small matrix, which encodes how they weigh and correlate the embedding dimensions. The per-user matrix is learned from relevance feedback. Users' judgments about retrieved items, in the form of triplets {query, positive, negative), are used to update their matrix, utilizing metric learning [38] to learn a similarity function that brings relevant items closer together while pushing irrelevant items further apart. At retrieval time, the system firstfilters the shared index using the index metric (Euclidean) to retrieve an enlarged candidate set, then refines that set using the user's personalized Mahalanobis distance. The enlargement step is justified by a theoretical lower bounding relationship between Euclidean and Mahalanobis distances [39]. This relationship provides a way to expand the index metric search scope 16 2. STATE OF THE ART so that the refinement stage guarantees retrieval of the precise nearest neighbours under the user's personalized Mahalanobis view, therefore providing theoretical bounds on the approach. From a practical point of view, we treat the vector index as a black box built for a fixed index metric (Euclidean). Our method requires only that the index support similarity search (range or k N N queries) under this metric. It does not depend on a particular indexing scheme (metric trees, HNSW, LSH, quantization, etc.). Moreover, personalization requires storing only the per-user matrix learned from feedback. Because embeddings and the index remain shared, deployment is straightforward, and the amount of user-specific state is small compared to approaches that maintain per-user profiles, memories, or retrained models. 17 3 Achieved Results This chapter summarizes the research results achieved so far. 3.1 Towards Personalized Similarity Search for Vector Databases In Towards Personalized Similarity Searchfor Vector Databases [55] (full paper in Appendix A ) , we proposed a personalised similarity search system for vector databases where the index is built with an Index metric (e.g., Euclidean distance) over learned embeddings, while users query with a View metric given by a Mahalanobis distance. Figure 3.1 shows the architecture and the roles of the metrics. Figure 3.1: Overview of the personalised similarity search system Personalisation follows the metric learning paradigm [38]. Each user is associated with a matrix A (initially the identity matrix, so the View metric equals Euclidean). During the search, users provide relevance feedback on the returned results, which is used to update A, therefore adapting the metric to the user's notion of similarity. Since the index remains Euclidean, the system must retrieve larger candidate sets by increasing the search range or the number of nearest neighbours to ensure that the Mahalanobis nearest neighbours are included. This guarantee is grounded in the lower-bounding relationship between Euclidean and Mahalanobis distances [56]: 18 3. ACHIEVED RESULTS • Having all eigenvalues Aj of A , and given two vectors, their Euclidean distance d.£ is always smaller or equal to the scaled Mahalanobis distance d^{X): ^ M ( A ) ^Jminj(Aj) • This introduces the scaling factor: S M , E ( A ) = \/yJmirij{\j). In the work, we focused on the range queries. Using the scaling factor, we defined the Euclidean range r^ = r^ • SM,E(A), which is used for retrieving the candidate set RdE,rE using range query on the Euclidean index. The range r^ can be calculated instantly and is^query independent. We further defined the optimum range ropt£( q ) as the minimum Euclidean range, such that the answer set Rd£r0ptE(~^) retrieved using this range contains the retrieved answer set RdM,rM using the Mahalanobis range and the Mahalanobis distance parameterised by A . However, this range cannot be calculated exactly without first having the wanted answer set R^MiTu, and it is dependent on the query q. Figure 3.2 shows the inclusions between the sets. Figure 3.2: Inclusions between the answer sets We experimentally verified that RdM,rM C RdE,r0piE(-f) C RdE,rEIn practice, we suggested the filter and refine principle: the engine first 19 3. ACHIEVED RESULTS uses Euclidean distance to efficiently retrieve a candidate superset RdE,rE' which is refined using the more expensive Mahalanobis distance parameterised by A to obtain the answer set RdM,rMFor the validation of the approach, we used 226,778 CLIP [57] embeddings from the Profiset [58] dataset. We trained 30 matrices using metric learning models ITML [59] and SDML [60], and different numbers of pairs of vectors synthetically chosen as similar or dissimilar. We abstracted from the semantics of the vectors and learned the matrices purely on a geometric level. We conducted all experiments using 1000 test query vectors randomly selected from the dataset. Figure 3.3 plots the ranges (blue), (green), and ?"Opt£(^f) (red) for 6 different matrices A , while the corresponding scaling factor S M , E ( A ) is shown atop each plot. The y-axis reports range values and the x-axis indexes query vectors q, ordered so that TM is non-decreasing. rE (0.t>Bl =t O.ICJO) rM (U.U4S_0.:U1) — rotxEiq) (0.045 ± 0.101) \Corr(r optE = +0.99997] 1.00 f0.75 0.50 (1.25 0.00 ITML 100 barest [..,,, , A) - 1 030) I | E (D.m4 ± 0.107) rM (0.070± 0.103) — joptE(g) (0.653 Jfc 0.102) 200 400 600 S00 1000 Query index 200 100 000 800 1000 Query index 0 200 100 GOO 800 1000 Query index Figure 3.3: r^, VM, and ?"opt£(^f) across test queries for six matrices. As expected, /"OptfC^) never exceeds r^, and moreover ropte( q ) is highly correlated with (Pearson Corr(roptE(~cf), r^) between +0.992 and +0.99998), suggesting that estimating roptcC^) from is a promising direction for future work. We also examined the sizes of the Euclidean candidate sets retrieved with ranges and ?"OptE(~ 0.1), and M P prioritises maximising A M A P (strongest personalisation). Each scenario is considered in three variants: a Baseline (BAS) variant, a Low Learning Time (LLT) variant prioritising training efficiency, and a Rank Stability (RS) variant favouring stable rankings. To address the second limitation, We used or re-implemented a broad suite of widely used metric learning methods: OASIS [40], ITML [59], P O L A [62], A R O M A [63], O M D M L [64], M L O M L [65], RobustODML [66], SORS [67], AdaSORS [67], and OPML [68]. We ran an extensive hyperparameter grid search to identify the best-performing combination of (i) metric learning model and hyperparameters and (ii) feedback strategy for each operational scenario and its variants. Table 3.2 summarises the best configurations and their outcomes in terms of the scaling factor (FSF), the personalisation gain ( A M A P ) , average learning time (ALT), and rank-stability measures (Average Spearman@1000 and Average Kendall's Tau@1000). Table 3.2: Calculated evaluation measures for the best model given the scenario and its variant. The Model column is in the form: model and hyperparameters. The Feedback column is in the form: feedback strategy and parameters - 1:(Number of pairs, Replacement), 2: (Batch), 3:(Number of queries). Scenario-Variant Model Feedback FSF AMAP ALT Aseiooo AKSIOOO BPSF-BAS OMDML p. le-3, C: 4e-3, 7: 7e-3 2 True 1.095 0.203 1.373 0.708 0.532 BPSF-LLT OASIS C: 7e-2, enjbrcejpsd: False 1 1, False 1.147 0.103 0.001 0.621 0.456 BPSF-RS OMDML p. 7e-2, C: le-3, 7: le-4 2 False 1.039 0.134 1.416 0.899 0.738 MSAP-BAS MLOML nl: 2, 7: le-4, act: tanh, A: 7e-6 3 5 1.017 0.102 18.098 0.863 0.693 MSAP-LLT OASIS C: le-2, enjbrcejpsd: False 1 8, False 1.062 0.106 0.008 0.813 0.636 MSAP-RS OMDML p. 1.0, C: 7e-4, 7: 4e-4 1 64, True 1.028 0.101 1.768 0.939 0.798 MP-BAS OMDML fi: 7e-l, C: le-2, 7: le-2 2 False 1.148 0.211 0.712 0.566 0.411 MP-LLT OASIS C: 7e-2, enjbrcejpsd: False 1 2, True 1.271 0.130 0.003 0.308 0.211 MP-RS OMDML (3: 4e-2, C: le-2, 7: le-4 2 True 1.077 0.193 1.104 0.788 0.609 23 3­ ACHIEVED RESULTS Two clear patterns emerged: OMDML dominates when effectiveness or rank stability is prioritised (BPSF­BAS/RS, MP­BAS/RS, MSAPRS), whereas OASIS is consistently selected for fast updates (all LLT variants), with MLOML appearing only in MSAP­BAS. Overall, the selected configurations maintain low scaling factors (scaling factor < 1.15 in most cases) while achieving gains up to AMAP = 0.211, showing that semantic feedback can improve relevance without substantial candidate set inflation, making the whole proposed approach effective and efficient. 3.3 Author's Publications Below are the publications relevant to this work. The full texts of these publications are available in Appendix A and Appendix B. 1. Marek Mahrík, Matúš Sikyňa, Vladimir Mic and Pavel Zezula. "Towards Personalized Similarity Search for Vector Databases". In: 17th International Conference on Similarity Search and Applications. SISAP 2024, Providence, RI, USA, pp. 126­139. • CORE conference ranking B • M y contribution: 30% • Evaluation of experiments, data visualisation, related work review, drafting and editing the paper, and submission management as corresponding author. 2. Matúš Sikyňa and Pavel Zezula. "Integrating Relevance F eedback for Effective Personalisation in Vector Search". In: 18th International Conference on Similarity Search and Applications. SISAP 2025, Reykjavik, Iceland, pp. 154­162. • CORE conference ranking B • M y contribution: 90% • Conceptualisation and schema design, implementation, experiment evaluation and interpretation, data visualisation, related work review, drafting and revising the paper, and managing the submission as corresponding author. 24 Bibliography 1. Z H A O , Xiangyu; W A N G , Maolin; Z H A O , Xinjian; LI, Jiansheng; Z H O U , Shucheng; YIN, Dawei; LI, Qing; T A N G , Jiliang; G U O , Ruocheng. Embedding in Recommender Systems: A Survey. arXiv preprint arXiv:2310.18608. 2023. 2. Z H O U , Sheng; X U , Hongjia; Z H E N G , Zhuonan; C H E N , Jiawei; LI, Zhao; BU, Jiajun; WU, Jia; WANG, Xin; Z H U , Wenwu; ESTER, Martin. A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions. ACM Computing Surveys. 2025, vol. 57, no. 3,69:1-69:38. 3. COSTA, Liliane Soares da; OLIVEIRA, Italo Lopes; FILETO, Renato. Text classification using embeddings: a survey. Knowledge and Information Systems. 2023, vol. 65, no. 7, pp. 2761-2803. 4. WANG, Jiajia; H U A N G , Jimmy Xiangji; TU, Xinhui; WANG, Junmei; H U A N G , Angela Jennifer; LASKAR, M d . Tahmid Rahman; BHUIYAN, Amran. Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges. ACM Computing Surveys. 2024, vol. 56, no. 7,185:1-185:33. 5. TVERSKY, Amos. Features of Similarity. Psychological Review. 1977, vol. 84, no. 4, pp. 327-352. 6. COX, Ingemar J.; MILLER, Matthew L.; MINKA, Thomas P.; PAPATHOMAS, Thomas V.; YIANILOS, Peter N . The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Transactions on Image Processing. 2000, vol. 9, no. 1, pp. 20-37. 7. SUGIYAMA, Kazunari; HATANO, Kenji; YOSHIKAWA, Masatoshi. Adaptive web search based on user profile constructed without any effort from users. In: Proceedings of the 13th international conference on World Wide Web. 2004, pp. 675-684. 8. SHEN, Xuehua; TAN, Bin; ZHAI, ChengXiang. Implicit user modeling for personalized search. In: Proceedings of the 14th ACM international conference on Information and Knowledge Management. 2005, pp. 824-831. 9. TEEVAN, Jaime; DUMAIS, Susan T; HORVITZ, Eric. Personalizing search via automated analysis of interests and activities. In: 25 BIBLIOGRAPHY Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. 2005, pp. 449-456. 10. SPERETTA, Mirco; G A U C H , Susan. Personalized Search Based on User Search Histories. In: The 2005 IEEE/WIC/ACM International Conference on Web Intelligence. 2005, pp. 622-628. 11. QIU, Feng; CHO, Junghoo. Automatic identification of user interest for personalized search. In: Proceedings of the 15th international conference on World Wide Web. 2006, pp. 727-736. 12. L U , Yumao; PENG, Fuchun; WEI, Xing; D U M O U L I N , Benoit. Personalize web search results with user's location. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2010, pp. 763-764. 13. BENNETT, Paul N ; WHITE, Ryen W.; C H U , Wei; DUMAIS, Susan T; BAILEY, Peter; BORISYUK, Fedor; CUI, Xiaoyuan. Modeling the impact of short- and long-term behavior on search personalization. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2012, pp. 185-194. 14. YI, Xing; H O N G , Liangjie; Z H O N G , Erheng; LIU, Nathan Nan; R A J A N , Suju. Beyond clicks: dwell time for personalization. In: Proceedings of the 8th ACM Conference on Recommender Systems. 2014, pp. 113-120. 15. V U , Thanh Tien; SONG, Dawei; WILLIS, Alistair; T R A N , Son Ngoc; LI, Jingfei. Improving search personalisation with dynamic group formation. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2014, pp. 951-954. 16. BOUADJENEK, Mohamed Reda; HACID, Hakim; BOUZEGHOUB, Mokrane; VAKALI, Athena. PerSaDoR: Personalized social document representation for improving web search. Information Sciences. [N.d.], vol. 369, pp. 614-633. 17. Z A M A N I , Hamed; BENDERSKY, Michael; Z H A N G , Mingyang; W A N G , Xuanhui. Situational Context for Ranking in Personal Search. In: Proceedings of the 26th International Conference on World Wide Web. 2017, pp. 1531-1540. 18. GE, Songwei; DOU, Zhicheng; JIANG, Zhengbao; NIE, Jian-Yun; W E N , Ji-Rong. Personalizing Search Results Using Hierarchical R N N with Query-aware Attention. In: Proceedings of the 27th ACM 26 BIBLIOGRAPHY International Conference on Information and Knowledge Management. 2018, pp. 347-356. 19. L U , Shuqi; D O U , Zhicheng; XIONG, Chenyan; W A N G , Xiaojie; WEN, Ji-Rong. Knowledge Enhanced Personalized Search. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, pp. 709-718. 20. M A , Zhengyi; D O U , Zhicheng; BIAN, Guanyue; W E N , Ji-Rong. PSTIE: Time Information Enhanced Personalized Search. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 2020, pp. 1075-1084. 21. A M E R , Nawal Ould; M U L H E M , Philippe; GERY, Mathias. Toward Word Embedding for Personalized Information Retrieval. 2016. 22. KUZI, Saar; SHTOK, Anna; K U R L A N D , Oren. Query Expansion Using Word Embeddings. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 2016, pp. 1929-1932. 23. V U , Thanh; N G U Y E N , Dat Quoc; JOHNSON, Mark; SONG, Dawei; WILLIS, Alistair. Search Personalization with Embeddings. In: 39th European Conference on Information Retrieval. 2017, vol. 10193, pp. 598-604. 24. AI, Qingyao; Z H A N G , Yongfeng; BI, Keping; C H E N , Xu; CROFT, W. Bruce. Learning a Hierarchical Embedding Model for Personalized Product Search. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, pp. 645-654. 25. GRBOVIC, Mihajlo; C H E N G , Haibin. Real-time Personalization using Embeddings for Search Ranking at Airbnb. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2018, pp. 311-320. 26. YAO, Jing; D O U , Zhicheng; W E N , Ji-Rong. Employing Personal Word Embeddings for Personalized Search. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, pp. 1359-1368. 27. BI, Keping; AI, Qingyao; CROFT, W. Bruce. A Transformer-based Embedding Model for Personalized Product Search. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, pp. 1521-1524. 27 BIBLIOGRAPHY 28. Z H O U , Yujia; D O U , Zhicheng; WEI, Bingzheng; XIE, Ruobing; W E N , Ji-Rong. Group based Personalized Search by Integrating Search Behaviour and Friend Network. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, pp. 92-101. 29. Z H O U , Yujia; DOU, Zhicheng; Z H U , Yutao; W E N , Ji-Rong. PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling. In: Proceedings of the 30th ACM International Conference on Information and Knowledge Management. 2021, pp. 2749-2758. 30. H U I , Bei; Z H A N G , Lizong; Z H O U , Xue; W E N , Xiao; N I A N , Yuhui. Personalized recommendation system based on knowledge embedding and historical behavior. Applied Intelligence. 2022, vol. 52, no. 1, pp. 954-966. 31. W A N G , Tian; B R O V M A N , Yuri M.; M A D H V A N A T H , Sriganesh. Personalized Embedding-based e-Commerce Recommendations at eBay. CoRR. 2021, vol. abs/2102.06156. 32. Z H A N G , Xiaojuan. Improving personalised query reformulation with embeddings. Journal of Information Science. 2022, vol. 48, no. 4, pp. 503-523. 33. BASSANI, Elias; TONELLOTTO, Nicola; PASI, Gabriella. Personalized Query Expansion with Contextual Word Embeddings. ACM Transactions on Information Systems. 2023, vol. 42, no. 2, pp. 1- 35. 34. BAEK, Jinheon; C H A N D R A S E K A R A N , Nirupama; C U C E R Z A N , Silviu; HERRING, Allen; JAUHAR, Sujay Kumar. KnowledgeAugmented Large Language Models for Personalized Contextual Query Suggestion. In: Proceedings of the ACM Web Conference 2024. 2024, pp. 3355-3366. 35. MYSORE, Sheshera; D H A N A N I A , Garima; PATIL, Kishor; M C C A L L U M , Andrew; K A L L U M A D I , Surya; Z A M A N I , Hamed. Bridging Personalization and Control in Scientific Personalized Search. In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2025, pp. 2309-2319. 36. Q I N , Weicong; X U , Yi; Y U , Weijie; S H E N , Chenglei; H E , Ming; FAN, Jianping; Z H A N G , Xiao; XU, Jun. MAPS: Motivation-Aware Personalized Search via LLM-Driven Consultation Alignment. 28 BIBLIOGRAPHY In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics. 2025, pp. 3039-3051. 37. RYAN, Fiona; SIVIC, Josef; HEILBRON, Fabian Caba; H O F F M A N , Judy; REHG, James M.; RUSSELL, Bryan C. Improving Personalized Search with Regularized Low-Rank Parameter Updates. In: Proceedings of the Computer Vision and Pattern Recognition Conference 2025. 2025, pp. 19748-19757. 38. BELLET, Aurelien; HABRARD, Amaury; SEBBAN, Marc. Metric Learning. 2015. Synthesis Lectures on Artificial Intelligence and Machine Learning, ISBN 978-3-031-00444-5. 39. CIACCIA, Paolo; PATELLA, Marco. Searching in metric spaces with user-defined and approximate distances. ACM Transactions on Database Systems. 2002, vol. 27, no. 4, pp. 398-437. 40. CHECHIK, Gal; SHALIT, Uri; SHARMA, Varun; BENGIO, Samy. A n Online Algorithm for Large Scale Image Similarity Learning. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems. 2009, pp. 306-314. 41. STAMOU, Sofia; NTOULAS, Alexandros. Search personalization through query and page topical analysis. User Modeling and UserAdapted Interaction. 2009, vol. 19, no. 1-2, pp. 5-33. 42. V U , Thanh; WILLIS, Alistair; KRUSCHWITZ, Udo; SONG, Dawei. Personalised Query Suggestion for Intranet Search with Temporal User Profiling. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. 2017, pp. 265-268. 43. BURGES, Christopher J. C ; RAGNO, Robert; LE, Quoc Viet. Learning to Rank with Nonsmooth Cost Functions. In: Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems. 2006, pp. 193-200. 44. M A , Zhongming; PANT, Gautam; SHENG, Olivia R. Liu. Interestbased personalized search. ACM Transactions on Information Systems. 2007, vol. 25, no. 1, p. 5. 45. SIEG, Ahu; MOBASHER, Bamshad; BURKE, Robin. Ontological user profiles for personalized web search. In: Proceedings of the 5th Workshop on Intelligent Techniques for Web Personalization. 2007, pp. 84-91. 46. X U E , Gui-Rong; H A N , Jie; Y U , Yong; Y A N G , Qiang. User language model for collaborative personalized search. ACM Transactions on Information Systems. 2009, vol. 27, no. 2, pp. 1-28. 29 BIBLIOGRAPHY 47. CARMEL, David; ZWERDLING, Naama; GUY, Ido; OF EK­KOIF MAN, Shila; H A R ' E L , Nadav; R O N E N , Inbal; UZIEL, Erel; YOGEV, Sivan; CHERNOV, Sergey. Personalized social search based on the user's social network. In: Proceedings of the 18th ACM conference on Information and Knowledge Management. 2009, pp. 1227­1236. 48. TEEVAN, Jaime; MORRIS, Meredith Ringel; BUSH, Steve. Discovering and using groups to improve personalized search. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining. 2009, pp. 15­24. 49. WHITE, Ryen W.; C H U , Wei; A W A D A L L A H , Ahmed Hassan; HE, Xiaodong; SONG, Yang; W A N G , Hongning. Enhancing personalized search by mining and modeling task behavior. In: Proceedings of the 22nd international conference on World Wide Web. 2013, pp. 1411­1420. 50. BLEI, David M . ; N G , Andrew Y ; JORDAN, Michael I. Latent Dirichlet Allocation. Journal of Machine Learning Research. 2003, vol. 3, pp. 993­1022. 51. L E U N G , Kenneth Wai­Ting; LEE, Dik Lun. Deriving ConceptBased User Profiles from Search Engine Logs. IEEE Transactions on Knowledge and Data Engineering. 2010, vol. 22, no. 7, pp. 969­ 982. 52. JOACHIMS, Thorsten. Optimizing search engines using clickthrough data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2002, pp. 133­142. 53. MIKOLOV, Tomas; C H E N , Kai; CORRADO, Greg; D E A N , Jeffrey. Efficient Estimation of Word Representations in Vector Space. In: 1st International Conference on Learning Representations. 2013. 54. XIONG, Chenyan; DAI, Zhuyun; C A L L A N , Jamie; LIU, Zhiyuan; POWER, Russell. End­to­End Neural Ad­hoc Ranking with Kernel Pooling. In: Proceedings of the 40th International Conference on Research and Development in Information Retrieval. 2017, pp. 55­64. 55. MAHRÍK, Marek; ŠIKYŇA, Matúš; MIC, Vladimir; Z E Z U L A , Pavel. Towards Personalized Similarity Search for Vector Databases. In: 17th International Conference on Similarity Search and Applications. 2024, pp. 126­139. 30 BIBLIOGRAPHY 56. CIACCIA, Paolo; PATELLA, Marco. Searching in metric spaces with user-defined and approximate distances. ACM Transactions on Database Systems. 2002, vol. 27, no. 4, pp. 398-437. 57. RADFORD, Alec; KIM, Jong Wook; HALLACY, Chris; RAMESH, Aditya; G O H , Gabriel; A G A R W A L , Sandhini; SASTRY, Girish; ASKELL, Amanda; MISHKIN, Pamela; CLARK, Jack; KRUEGER, Gretchen; SUTSKEVER, Ilya. Learning Transferable Visual Models From Natural Language Supervision. In: International Conference on Machine Learning, ICML 2021. 2021, vol. 139, pp. 8748- 8763. 58. BUDÍKOVÁ, Petra; BAŤKO, Michal; Z E Z U L A , Pavel. Evaluation Platform for Content­Based Image Retrieval Systems. In: International Conference on Theory and Practice of Digital Libraries, TPDL 2011. 2011, vol. 6966, pp. 130­142. 59. DAVIS, Jason V ; KULIS, Brian; JAIN, Prateek; SRA, Suvrit; DHILLON, Inderjit S. Information­theoretic metric learning. In: International Conference on Machine Learning, ICML 2007. 2007, vol. 227, pp. 209­ 216. 60. QI, Guo­Jun; TANG, Jinhui; Z H A , Zheng­Jun; C H U A , Tat­Seng; Z H A N G , Hong­Jiang. A n efficient sparse metric learning in highdimensional space via ^­penalized log­determinant regularization. In: International Conference on Machine Learning, ICML 2009. 2009, vol. 382, pp. 841­848. 61. ŠIKYŇA, Matúš; ZEZULA, Pavel. Integrating Relevance Feedback for Effective Personalisation in Vector Search. In: 18th International Conference on Similarity Search and Applications. 2025, pp. 154­162. 62. SHALEV­SHWARTZ, Shai; SINGER, Yoram; N G , Andrew Y. Online and batch learning of pseudo­metrics. In: ICML 2004. 2004, vol. 69. 63. C R A M M E R , Koby; CHECHIK, Gal. Adaptive Regularization for Weight Matrices. arX iv preprint arXiv:1206.4639. 2012. 64. W U , Pengcheng; HOI, Steven C. H ; ZHAO, Peilin; MIAO, Chunyan; LIU, Zhiyong. Online Multi­Modal Distance Metric Learning with Application to Image Retrieval. IEEE Transactions on Knowledge and Data Engineering. 2016, vol. 28, no. 2, pp. 454­467. 65. LI, Wenbin; LIU, Yanfang; HUO, Jing; SHI, Yinghuan; GAO, Yang; W A N G , Lei; L U O , Jiebo. A Multilayer F ramework for Online 31 BIBLIOGRAPHY Metric Learning. IEEE Transactions on Neural Networks and Learning Systems. 2023, vol. 34, no. 10, pp. 6701-6713. 66. ZABIHZADEH, Davood; T U A M A , Amar; KARAMI-MOLLAEE, Ali; MOUSAVIRAD, Seyed Jalaleddin. Low-rank robust online distance/similarity learning based on the rescaled hinge loss. Applied Intelligence. 2023, vol. 53, no. 1, pp. 634-657. 67. YAO, Dezhong; ZHAO, Peilin; YU, Chen; JIN, Hai; LI, Bin. Sparse Online Relative Similarity Learning. In: 2015 IEEE International Conference on Data Mining. 2015, pp. 529-538. 68. LI, Wenbin; GAO, Yang; WANG, Lei; Z H O U , Luping; HUO, Jing; SHI, Yinghuan. OPML: A one-pass closed-form solution for online metric learning. Pattern Recognition. 2018, vol. 75, pp. 302- 314. 32 A Appendix Marek Mahrík, Matúš Šikyňa, Vladimir Mic and Pavel Zezula. "Towards Personalized Similarity Search for Vector Databases". In: 17th International Conference on Similarity Search and Applications. SISAP 2024, Providence, RI, USA, pp. 126­139. 33 ®Check for updates T o w a r d s P e r s o n a l i z e d S i m i l a r i t y S e a r c h f o r V e c t o r D a t a b a s e s Marek Mahrík1 , Matúš Šikyňa1 ^ ­* , Vladimir Mic2 ©, and Pavel Zezula1 © 1 F aculty of Informatics, Masaryk University, Brno, Czech Republic 492727@mail.muni.cz, {xsikyna,zezula}@fi.muni.cz 2 Department of Computer Science, Aarhus University, Aarhus, Denmark v.mic@cs.au.dk Abstract. The importance of similarity search has become prominent in the fast­evolving vector databases, which apply content embedding techniques on complex data to produce and manage large collections of high­dimensional vectors. Processing of such data is only possible by using a similarity function for storage, structure, and retrieval. However, if multiple users access the collection, their views on similarity can differ as similarity, in general, is subjective and context­dependent. In this article, we elaborate on the problem of a similarity search engine implementation, where users use a common index but search with personalised views of similarity, implemented by a possibly different similarity model. Specifically, we define a foundational theoretical framework and conduct experiments on real­life data to confirm the viability of such an approach. The experiments also indicate future research directions needed to propose and implement an effective and efficient personalised similarity search engine. Keywords: Similarity search • Personalized similarity • Vector databases 1 Introduction Current A I models produce vector embeddings that carry semantic information capable of performing context­based reasoning. The concept of similarity is central to this process, given its critical role in both learning and teaching. Such vector embedding models produce high­dimensional vectors, the collections of which are structured into partitions of a vector database index to facilitate efficient similarity querying. Typically, vector databases utilise Euclidean distance to structure the index and identify the most similar vectors in response to a given query vector. This work was supported by the research grants (VIL50110) from VILLUM F OND EN, and by the Open Calls for Security Research 2023­2029 (OPSEC) program granted by the Ministry of the Interior of the Czech Republic under No. VK01010147 ­ Automated digital data forensics lab for complex crime detection. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 E. Chavez et al. (Eds.): SISAP 2024, LNCS 15268, pp. 126­139, 2024. https://doi.org/10.1007/978­3­031­75823­2_ll Towards Personalized Similarity Search for Vector Databases 127 Table 1. Notation used throughout this paper D;X CD domain of the searched vectors; searched dataset query vector i f and the set of tested query vectors 0 from D vectors from the domain of the searched vectors 6 the dimensionality of vectors in the domain D A positive semi-definite matrix used in the Mahalanobis distance function dE ; ,)he range query with distance function and search range R,(~ct) the (precise) answer of the range query Q(lf, < f >, < r >) VE the precision of answer RdE,rE{~c[) with respect to RdM,rM(~rE{~c[) for a given i f #Opt_E the number of vectors in -RdB,r-0ptB(~3K )C dsC^,~$)Having the domain D of <5-dimensional vectors, we define the goal of the personalised similarity search in a dataset X C D as follows. Having a query vector ~qf G D and matrix A , we want to evaluate a range query Q(~gt, dM, TM), i.e., to retrieve the Mahalanobis answer set RdM,rM(~$): RdM,rM(t) = I dM(!?,t,A) < rM} C X (2) Due to the property given by Eq. 1, range TE defined as: TE = rM • S M , B ( A ) (3) defines the range query Q(~^, ds, TE) that provides the Euclidean answer set R dE,rE(~t)RdE,rEOt) = I d E & , t ) n " where is either SDML or ITML, n is the number of vector pairs used in A learning, and denotes the way how the vector pairs were sampled, i.e., either Random or Nearest. Matrices SDML 100 Nearest and ITML 100 Nearest have almost the same scaling factor S M , B ( A ) , and we demonstrate sizes of the corresponding Euclidean answer sets created as the super sets of the Mahalanobis answer sets with these A . In total, we present results for 6 matrices where 3 matrices were learnt with 100 vector pairs (i.e., SDML 100 Random, SDML 100 Nearest, and ITML 100 Nearest), and the other 3 were learnt in the same way just with more vector pairs {SDML 500 Random, SDML 1000 Nearest, and ITML 1000 Nearest). This selection of matrices helps us to demonstrate the consequences of an increasing number of learning vector pairs. We conduct all experiments with 1000 query vectors selected randomly from the dataset as the query set 0. These query vectors were not used to learn any A . Lower Bounding Relationship Verification For the following experiments, we need to choose the Mahalanobis search ranges T M - We consider each " j / e 0 independently, and compute dM(~f, it, A ) for all it £ X. For each if, we define rM = dM(lf, ^IOO> A ) which is the Mahalanobis distance of if to its 100th nearest neighbour rcioo from X. These ranges T M define the Mahalanobis answer sets RdM,rM(~f) which each contains 100 vectors1 . The following experiments verify the key result of the theoretical Sect. 4, i.e., the relation of the answer sets defined by Eqs. 2, 4, and 5: R dM,rM{~t) Q R dE,r0ptE{-t)^) - R dE,rE{~t) (8) We start with the Mahalanobis ranges rjy, and compute the optimum range ^Opt E (if) for each if as the maximum Euclidean distance ds(lt',~ and < selection >, the precisions VE and VoptE decrease with increasing number of vector pairs n used for the A learning, which also increases the scaling factor S M , B ( A ) . - Given two different matrices A with nearly the same scaling factor S M , B ( A ) , the precision VE can be distinct even for the same query vector if with the search range rE derived from T M according to Eq. 3. - Precision VoPtE is usually much higher than precision VE because roptE(~^) is usually much lower than range even when the scaling factor S M , _ B ( A ) is constant. - Scaling factor S M , E ( A ) much depends on the way how the matrix A for the Mahalanobis distance function is learnt. To develop a personalised similarity search engine, the following challenges need to be properly explored: - The search range rE guaranteed by Eq. 3 to cover the Mahalanobis answer set for given matrix A is usually too large. Given an extreme Pearson correlation Towards Personalized Similarity Search for Vector Databases 137 between Mahalanobis ranges TM and optimum Euclidean ranges roPtE(~(?), a heuristic to estimate ropts(^) from TM should be proposed. The heuristic could be approximate, i.e., without guarantees on its correctness. - We need theories and paradigms which would lead to a proper definition of the personalised matrix A . This metric learning should be fast, simple, but effective from the user's point of view. At the same time, it should produce matrices A with a small scaling factor S M , B ( A ) as they still can lead to small Euclidean answer sets RdE,rE(~dE(x,y), (1) where dE{x,y) = \\x — y\\ represents the Euclidean distance. The Mahalanobis distance dM(x,y, A ) is expressed as: dM(x,y, A ) = \JJx — y)T A(x — y), and the eigenvalues Xj are derived from the matrix A . This inequality introduces a scaling factor S M , B ( A ) = 1 /-^mnij{Xj}. In this paper, we use the term Mahalanobis distance to denote a general quadratic form distance parameterised by a learned PSD matrix, as commonly done in metric learning literature [5-7,16]. Given a specified Mahalanobis radius TM, the corresponding Mahalanobis answer set for any query vector q and all z from the dataset is defined as: RdM,rM(q) = {z \ dM(z,q,A)