Join Discord
Created at 7pm, Jan 19
benjaminArtificial Intelligence
0
User Embedding Model for Personalized Language Prompting
Contract ID
0pt1V_TSiZLIBmkYFTXzWUsIXixfEyz04ZlzH7bxDBo
File Type
PDF
Entry Count
42
Embed. Model
jina_embeddings_v2_base_en
Index Type
hnsw

Abstract of the paper: Modeling long user histories plays a pivotal role in enhancing recommendation systems, al- lowing to capture users’ evolving preferences, resulting in more precise and personalized rec- ommendations. In this study, we tackle the challenges of modeling long user histories for preference understanding in natural language. Specifically, we introduce a new User Embed- ding Module (UEM) that efficiently processes user history in free-form text by compressing and representing them as embeddings, to use them as soft prompts to a LM. Our experiments demonstrate the superior capability of this ap- proach in handling significantly longer histories compared to conventional text-based methods, yielding substantial improvements in predic- tive performance. Models trained using our ap- proach exhibit substantial enhancements, with up to 0.21 and 0.25 F1 points improvementover the text-based prompting baselines. The main contribution of this research is to demon- strate the ability to bias language models via user signals.https://doi.org/10.48550/arXiv.2401.04858

Limitations
id: 69b1b67de5ef98bea519f7c33bf6a1ac - page: 5
While we argue and demonstrate in this work that using a UEM is an efficient way to encode long user histories with easier extensions to multimodal inputs, we acknowledge that text prompting can be further optimized, by using text-to-text prompt compression models. These trade-offs could be further studied. The simplicity of the UEM architecture leaves a lot of headroom as demonstrated by LongT5 baselines in Table 1. Our presentations for U are using generic semantic embeddings with the use of SentenceT5 (Ni et al., 2022), these can be further improved with the use of domain specific embeddings. Our experiments are using LMs that are <1B parameters, which are usually considered smaller family of LLMs. It would be a good future direction to consider larger models with parameter efficient tuning techniques. Furthermore, our research has primarily focused on preference understanding, and hasnt been tested on tasks extending to areas such as rating prediction or item recommendati
id: 40b110918d7e290eaa11cce23951688b - page: 5
We expect our conclusions here are likely apply to these tasks. We plan to address these limitations and pursue these avenues in our future research efforts.
id: 15f13c4956d239c1aedcd798f3046000 - page: 5
Ethics Statement The datasets and models utilized in this study are based on publicly available and open-source resources. While we acknowledge the inherent ethical considerations associated with language models, we do not anticipate any additional ethical concerns arising from the datasets and models developed in the course of this research.
id: 6e23d5111b7ec15d363efd528194515a - page: 5
How to Retrieve?
# Search

curl -X POST "https://search.dria.co/hnsw/search" \
-H "x-api-key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"rerank": true, "top_n": 10, "contract_id": "0pt1V_TSiZLIBmkYFTXzWUsIXixfEyz04ZlzH7bxDBo", "query": "What is alexanDRIA library?"}'
        
# Query

curl -X POST "https://search.dria.co/hnsw/query" \
-H "x-api-key: <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"vector": [0.123, 0.5236], "top_n": 10, "contract_id": "0pt1V_TSiZLIBmkYFTXzWUsIXixfEyz04ZlzH7bxDBo", "level": 2}'