Aditya Pal | Applied Science, Chantat Eksombatchai | Applied Science, Yitong Zhou | User Understanding, Bo Zhao | User Understanding, Charles Rosenberg | Applied Science, Jure Leskovec | Applied Science

Aditya Pal | 應用科學，Chantat Eksombatchai | 應用科學，周一彤| 用戶理解，趙波| 用戶理解，Charles Rosenberg | 應用科學，Jure Leskovec | 應用科學

As we build a visual discovery engine that powers 2B+ Pins, it’s crucial to understand user interests and preferences in order to serve relevant content. One standard approach to encode user preferences is via an embedding-based representation in a high dimensional space. Most prior methods tried at Pinterest infer a single high-dimensional embedding for each user in compatibility with the content embedding. This is a good starting point but falls short in delivering a full understanding of the user.

當我們構建為2B +引腳供電的視覺發現引擎時，了解用戶的興趣和偏好以提供相關內容至關重要。 一種對用戶偏好進行編碼的標準方法是通過在高維空間中進行基于嵌入的表示。 在Pinterest上嘗試的大多數現有方法都會為每個用戶推斷與內容嵌入兼容的單個高維嵌入。 這是一個很好的起點，但不足以全面了解用戶。

In this work, we postulate that a single embedding is not sufficient for encoding multiple facets of a user’s interests that might have no obvious linkage between them. They can evolve, with some interests persisting long term while others span a short time period. Recommended items are also represented in the same embedding space. A good embedding must encode a user’s multiple tastes, interests, styles, etc., whereas a recommended item (a video, an image, a news article, a house listing, a pin, etc.) typically only has a single focus. Hence it becomes important to represent a user with multiple embeddings, with each embedding capturing a specific aspect of their interest.

在這項工作中，我們假設單個嵌入不足以對用戶興趣的多個方面進行編碼，而這些方面之間可能沒有明顯的聯系。 它們可以發展，某些利益可以長期持續，而其他利益則可以在很短的時間內出現。 推薦項目也顯示在相同的嵌入空間中。 良好的嵌入必須對用戶的多種口味，興趣，風格等進行編碼，而推薦項(視頻，圖像，新聞，房屋列表，圖釘等)通常只有一個重點。 因此，重要的是用多個嵌入來表示用戶，每個嵌入都捕獲他們感興趣的特定方面。

PinnerSage模型 (PinnerSage Model)

In order to better understand our users’ preferences, we developed PinnerSage, a highly scalable, flexible and extensible recommender system that internally represents each user with multiple embeddings. Figure 1 provides an end-to-end overview of the PinnerSage recommendation model. The starting point for our model is to organize the repins and clicks of a user into multiple interest clusters by running the Ward clustering model and then generating a summary of each of those clusters using a medoid, an embedding, and a cluster importance score. Next, a subset of these clusters are picked by the online cluster selection, and it employs a nearest-neighbor index to generate recommendations to the user. Users’ actions are processed in real-time to update the interest clusters. In order for PinnerSage to provide relevant recommendations to our 400M+ monthly active users and adapt in real-time, we made several model design choices that we describe next.

為了更好地了解我們的用戶的偏好，我們開發了PinnerSage，這是一種高度可擴展，靈活且可擴展的推薦系統，可以在內部代表每個用戶多個嵌入對象。 圖1提供了PinnerSage推薦模型的端到端概述。 我們模型的出發點是，通過運行Ward聚類模型 ，然后使用medoid，嵌入和聚類重要性評分，將用戶的撥動和點擊組織到多個興趣聚類中，以生成每個聚類的摘要。 接下來，通過在線集群選擇來選擇這些集群的子集，并且它使用最近鄰居索引來生成對用戶的推薦。 實時處理用戶的操作以更新興趣群。 為了讓PinnerSage為我們的400M +月活躍用戶提供相關建議并實時進行調整，我們做出了以下幾個模型設計選擇。

設計選擇1：固定引腳嵌入 (Design Choice 1: Pin Embeddings are Fixed)

The interest clusters in Figure 1 are generated by clustering the embeddings of repins and clicks of a user. The embeddings of repins and clicks are trained via the PinSage model that optimizes for contextual and visual similarity between Pins via a Graph convolutional model. Since our goal is to project users in the same space as the Pin embedding space, we consider the Pin embeddings to be fixed. This design choice simplifies our models considerably and allows us to run inference pipelines in parallel for each user.

圖1中的興趣聚類是通過聚類用戶的圖釘和點擊的嵌入而生成的。 通過PinSage模型訓練圖釘和點擊的嵌入，該模型通過Graph卷積模型優化圖釘之間的上下文和視覺相似性。 由于我們的目標是在與Pin嵌入空間相同的空間中投影用戶，因此我們認為Pin嵌入是固定的。 這種設計選擇大大簡化了我們的模型，并允許我們為每個用戶并行運行推理管道。

Joint embedding inference models, where both user and Pin embeddings are inferred together, can be too complex and hard to scale. Moreover, we posit that in practice they compromise recommendation relevance, as some spurious connections between pins can be established via the users. To see this point, consider the example in Figure 2.

聯合嵌入推理模型(將用戶嵌入和Pin嵌入一起推斷)可能太復雜且難以擴展。 此外，我們認為在實踐中它們會損害推薦的相關性，因為可以通過用戶在引腳之間建立一些虛假的連接。 要了解這一點，請考慮圖2中的示例。

Figure 2: Three interests of a given user.圖2：給定用戶的三個興趣。

In the above example figure, a user is interested in painting, shoes, and sci-fi. Jointly learned users and Pin embeddings would bring pin embeddings on these disparate topics closer, which can compromise the relevance of the nearest neighbor-based recommender. Pin embeddings should only operate on the underlying principle of bringing similar pins closer while keeping the rest of the pins as far as possible. For this reason, we use PinSage, which precisely achieves this objective without any dilution.

在上面的示例圖中，用戶對繪畫，鞋子和科幻小說感興趣。 共同學習的用戶和Pin嵌入將使這些不同主題上的pin嵌入更加緊密，這可能會損害最近的基于鄰居的推薦者的相關性。 引腳嵌入僅應遵循使相似的引腳靠近的基本原則，同時將其余的引腳保持盡可能遠。 因此，我們使用PinSage，無需任何稀釋即可精確實現此目標。

設計選擇2：無限的用戶嵌入 (Design Choice 2: Unlimited User Embeddings)

Prior work either fixes the number of embeddings to a small number or puts an upper bound on them. At best, such restrictions hinder developing a full understanding of the users and, at worst, merge different concepts together, leading to bad recommendations. For example, merging embeddings could yield an embedding that lies in a very different region. Figure 2 shows that a merger of three disparate pin embeddings results in an embedding that is best represented by the concept energy boosting breakfast. Needless to say, recommendations based on such a merger can be problematic.

先前的工作要么將嵌入數量固定為少量，要么將嵌入數量設置為上限。 最好的情況是，這樣的限制阻礙了對用戶的全面了解，最壞的情況是將不同的概念融合在一起，從而導致不好的建議。 例如，合并嵌入可能會產生位于非常不同區域中的嵌入。 圖2顯示，三個不同的針狀嵌入物的合并產生的嵌入物最好用能量增強早餐的概念來表示。 不用說，基于這樣的合并的建議可能會有問題。

PinnerSage generates as many interest clusters as the underlying data supports. This is achieved by clustering users’ actions into conceptually coherent clusters via a hierarchical agglomerative clustering algorithm (Ward). A light user might get represented by 3–5 clusters, whereas a heavy user might get represented by 75–100 clusters.

PinnerSage會生成基礎數據支持的盡可能多的興趣集群。 這是通過層次化的聚集聚類算法(Ward)將用戶的行為聚類為概念上一致的聚類來實現的。 輕量級用戶可能由3–5個群集代表，而重度用戶可能由75–100個群集代表。

設計選擇3：基于Medoid的集群表示 (Design Choice 3: Medoid-based Cluster Representation)

Typically, clusters are represented by centroid, which requires storing an embedding. Additionally, centroid can be sensitive to outliers in the cluster. To compactly represent a cluster, we pick a cluster member pin, called medoid. Medoid, by definition, is a member of the user’s originally interacted pin set. Hence it avoids the pit-fall of topic drift and is robust to outliers. From a systems perspective, medoid is a concise way of representing a cluster, as it only requires storage of medoid’s pin id, and leads to cross-user and even cross-application cache sharing. It also allows our system to be compatible with other non-embedding-based recommendation systems such as Pixie.

通常，群集以質心表示，需要存儲嵌入。 此外，質心可能對聚類中的離群值敏感。 為了緊湊地表示群集，我們選擇一個群集成員銷釘，稱為medoid。 根據定義，Medoid是用戶最初交互的密碼集的成員。 因此，它避免了主題漂移的陷阱，并且對異常值具有魯棒性。 從系統角度來看，medoid是表示集群的一種簡潔方法，因為它僅需要存儲medoid的pin ID，并導致跨用戶甚至跨應用程序的緩存共享。 它還使我們的系統與其他基于非嵌入的推薦系統(例如Pixie)兼容。

設計選擇4：候選檢索的Medoid采樣 (Design Choice 4: Medoid Sampling for Candidate Retrieval)

PinnerSage provides a rich representation of a user via cluster medoids. However, in practice we cannot use all the medoids simultaneously for candidate retrieval due to cost concerns. Additionally, the user would be bombarded with too many different items. To address these concerns, we sample 3 medoids proportional to their importance scores and recommend their nearest neighboring pins. The importance scores of medoids are updated daily, and they can adapt with the user’s changing tastes.

PinnerSage通過聚類medoids提供了豐富的用戶表示。 但是，實際上，由于成本方面的考慮，我們無法同時使用所有類固醇進行候選檢索。 另外，用戶會被太多不同的物品轟炸。 為了解決這些問題，我們對3種類固醇按其重要性得分成比例進行采樣，并推薦與它們最接近的相鄰引腳。 類固醇的重要性評分每天都會更新，它們可以適應用戶不斷變化的口味。

設計選擇5：處理實時更新的兩管齊下的方法 (Design Choice 5: Two-Pronged Approach for Handling Real-Time Updates)

It is important for a recommender system to adapt to the current needs of its users. At the same time, an accurate representation of users requires looking at their past 60–90 days of activities. Sheer size of the data and the speed at which it grows makes it hard to consider both aspects together. We address this issue by combining two methods: (a) a daily batch inference job that infers multiple medoids per user based on their long-term interaction history, and (b) an online version of the same model that infers medoids based on the users’ interactions on the current day. As new activity comes in, only the online version is updated. At the end of the day, the batch version consumes the current day’s activities and resolves any inconsistencies. This approach ensures that our system adapts quickly to the users’ current needs and at the same time does not compromise their long-term interests.

推薦系統必須適應其用戶的當前需求，這一點很重要。 同時，要準確地表示用戶，需要查看他們過去60-90天的活動。 數據的龐大規模及其增長速度使得很難同時考慮這兩個方面。 我們通過以下兩種方法來解決此問題：(a)每日批處理推斷工作，根據其長期交互歷史來推斷每個用戶的多個類固醇；以及(b)相同模型的在線版本，根據用戶來推斷類固醇當天的互動。 隨著新活動的到來，僅在線版本被更新。 在一天結束時，批處理版本將消耗當日的活動并解決所有不一致問題。 這種方法可確保我們的系統快速適應用戶的當前需求，同時又不損害其長期利益。

A / B測試 (A/B Tests)

PinnerSage is currently deployed in production and used by many products within Pinterest, ranging from Homefeed, Related Pins, Ads, Shopping, and Creators, in both their retrieval and ranking ML models. Our wins on the initial A/B test on two surfaces are highlighted in Table 1.

PinnerSage目前已在生產中部署，并且在Pinterest內的許多產品(包括Homefeed，相關的Pins，廣告，購物和Creators)的檢索和排名ML模型中都使用。 表1突出顯示了我們在兩個表面上進行初始A / B測試的勝利。

Table 1 shows that PinnerSage provides significant engagement gains on increasing overall engagement volume (repins and clicks) as well as increasing engagement propensity (repins and clicks per user). Any gain can be directly attributed to increased quality and diversity of PinnerSage recommendations.

表1顯示，PinnerSage通過增加總體參與量(回復和點擊)以及提高參與傾向(每位用戶的回復和點擊)可以顯著提高參與度。 任何收益都可以直接歸因于PinnerSage建議的質量提高和多樣性。

Table 1: A/B test of PinnerSage vs current production, which includes a single embedding model.表1：PinnerSage與當前產品的A / B測試，其中包括一個嵌入模型。

結論 (Conclusion)

We proposed an end-to-end system, called PinnerSage, that powers personalized recommendation at Pinterest. In contrast to prior production systems that are based on a single embedding-based user representation, PinnerSage proposes a multi-embedding-based user representation scheme. Our proposed clustering scheme ensures that we get full insight into the needs of a user and understand them better. To make this happen, we adopt several design choices that allow our system to run efficiently and effectively. Our large A/B tests show that PinnerSage provides significant gains in user engagement. Much of the improvements delivered by our model can be attributed to its better understanding of user interests and its quick response to their needs.

我們提出了一個名為PinnerSage的端到端系統，該系統可為Pinterest提供個性化推薦。 與基于單個基于嵌入的用戶表示的現有生產系統相比，PinnerSage提出了一種基于多嵌入的用戶表示方案。 我們提出的集群方案可確保我們全面了解用戶需求并更好地理解它們。 為了實現這一目標，我們采用了幾種設計選擇，使我們的系統能夠高效運行。 我們的大型A / B測試表明，PinnerSage可顯著提高用戶參與度。 我們的模型提供的許多改進都可以歸因于其對用戶興趣的更好理解以及對用戶需求的快速響應。

附錄 (Appendix)

PinnerSage paper is to appear in KDD 2020. Read more details about the paper here: https://arxiv.org/abs/2007.03634

PinnerSage論文將出現在KDD 2020中。有關此論文的更多詳細信息，請訪問： https ：//arxiv.org/abs/2007.03634

致謝 (Acknowledgements)

We would like to extend our appreciation to Homefeed and Shopping teams for helping in setting up online A/B experiments. Our special thanks to the embedding infrastructure team for powering embedding nearest neighbor search.

我們要感謝Homefeed和Shopping團隊幫助建立在線A / B實驗。 我們特別感謝嵌入基礎架構團隊為嵌入最近鄰居搜索提供支持。

翻譯自: https://medium.com/pinterest-engineering/pinnersage-multi-modal-user-embedding-framework-for-recommendations-at-pinterest-bfd116b49475

總結

以上是生活随笔為你收集整理的PinnerSage模型的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。