Information
WSDM 2016
Background
Collaborative Filtering (CF): 사용자가 아이템에 대해 평가하거나 상호작용한 데이터를 사용한다. 사용자 간 또는 아이템 간의 유사성을 기반으로 추천한다. 즉, 비슷한 취향을 가진 사용자나 아이템을 찾고, 이를 바탕으로 추천한다. (희소성 문제) 많은 사용자와 아이템이 있을 경우, 사용자가 평가한 아이템이 매우 적어서 추천할 데이터가 부족할 수 있다. (콜드 스타트 문제) 새로운 사용자나 아이템에 대한 정보가 없을 때, 추천을 제공하기 어렵다.
Denosing Auto-Encoders(DAE): 일반적인 AE는 입력 데이터를 그대로 복원하지만, DAE는 노이즈를 추가한 데이터를 입력받아 노이즈를 제거하고 원본을 복원하는 방법을 학습한다. 노이즈는 모델이 데이터를 더 잘 일반화하도록 유도하는 역할을 하며, DAE는 실제 노이즈가 있는 데이터에 robust한 특성을 가진다.
Abstract & 1 Introduction
recommend system: Given a set of users, items, and observed user-item interactions, these systems can recommend other items that the users might like.
we present a new model-based collaborative filtering (CF) method for top-N recommendation called Collaborative Denoising Auto-Encoder (CDAE). The model learns latent representations of corrupted user-item preferences that can best reconstruct the user’s full preference set. at prediction time, the model recommends new items to the user given the existing preference set as input.
Our contributions can be summarized as follows:
- We propose a new model CDAE, which formulates the topN recommendation problem using DAE. CDAE is novel in both model definition and objective function.
- SOTA이고, 다양한 구성 요소로 실험한 결과 공유
2 Problem definition
a set of users $\tilde{U} = \{u = 1, ..., U\}$
a set of items $\tilde{I} = \{i or j = 1, ..., I\}$
a log of the user's past preferences of items $\tilde{O} = \left (u, i, y_{ui}\right )$, all the $y_{ui}$ are 1 as implicit feedback
a set of unobserved (negative sample) $\bar{O}$
an augmented user-itme pairs dataset that includes some data sampled from $\bar{O}$ $O'$
items in $\bar{O_{u}}$ are the candidates to be recommended to user $u$.
The goal of the recommender is to pick for each user $u$ a subset of items from the candidate set, the predicted values of which are most likely to be 1.
2.1 Overview of model-based recommenders
Recommender models
In general, recommender models are defined as
$$\hat{y}_{ui} = F_{\theta}\left( u, i \right)$$
$\hat{y}_{ui} $ is the predicted preference of user $u$ on item $i$
$\theta$ is trainable parameter
$F_{\theta}$ is diverse assumption about how the output depends on the input.
Objective functions for recommenders
notation: loss function $\ell\left( \cdot \right)$, regularization term $\Omega\left( \theta \right )$
1) point-wise objectives only depend on the accuracy of the prediction of individual preferences
$$\sum_{\left(u,i\right)\in O' } \ell_{point} \left( y_{ui}, \hat{y}_{ui}\right) + \lambda$$
2) pair-wise objectives approximates ranking loss by considering the relative order of the predictions for pairs of items
$$\sum_{\left(u,i,j\right)\in P } \ell_{pair} \left( y_{uij}, \hat{y}_{uij}\right) + \lambda$$
$y_{uij} = y_{ui} - y_{uj}$, $\hat{y}_{uij} = \hat{y}_{ui} - \hat{y}_{uj}$
$P$ is set of triplets sampled from $O'$
usually $i$ is a positive item and $j$ is a negative item.
2.2 Denoising Auto-Encoders
$$z = h \left(x\right) = \sigma\left( W^{T}x+b \right)$$
$W$ 와 $b$를 매개변수로 가지는 neural network에 input $x \in \mathbb{R}^{D}$를 입력해 hidden representation $z\in\mathbb{R}^{K}$를 생성한다.
$$\hat{x} = \sigma\left(W'z+b'\right)$$
그리고서 다시 $W'$와 $b'$를 매개변수로 가지는 neural network에 $z$를 입력해 input $x$를 복원한 $\hat{x}\in\mathbb{R}^{D}$을 생성한다.
$$\underset{W, W', b, b'} {\mathrm{arg min}} \frac{1}{n} \sum^{n}_{i=1}\ell\left(x_{i}, \hat{x}_{i}\right)$$
마지막으로 $x_{i}와 \hat{x}_{i}$사이의 error가 최소화되도록 매개변수를 학습한다.
DAE에서는 $x$의 corrupted version $\tilde{x}$를 입력으로 사용한다. 일반적으로 Gaussian nosie를 추가하거나, Multiplicative mask-out/drop-out noise로 원본 데이터의 일부 값을 0으로 만들어 생성한다.
3 Proposed methodology
Collaborative Denoising Auto-Encoder (CDAE) learns correlations between the user’s item preference by training on a corrupted version of the known preference set.
3.1 Collaborative Denoising Auto-Encoder
1) training set
training set $\tilde{O} = \{y_{1}, y_{2}, ..., y_{U}\} \in |\tilde{U}| \times |I|$
feedback vector of user $u$ on all the item $y_{u} = \{y_{u1}, y_{u2}, ..., y_{uI}\} \in |I|$
user feedback on a item $y_{ui} \in \{0, 1\}$
2) input layer
$I$+1 nodes. the first $I$ nodes as item input nodes. the last node as user input node .
input is consisted of corrupted feedback vector $\tilde{y}_{u}$ which is generated by dropping out elements of $y_{u}$ randomly.
3) hidden layer
CDAE first maps the input to a latent representations $z_{u}$
$$z_{u} = h(W^{T}\tilde{y}_{u} + V_{u} + b)$$
$W \in \mathbb{R}^{I}\times{K}$ for item input nodes, $V_{u} \in \mathbb{R}^{K}$ for user input node, $b \in \mathbb{R}^{K}$ for offset
4) output layer
the latent representation is then mapped back to the original input space to reconstruct the input vector.
$$\hat{y}_{ui}=f(W'^{T}_{i}z_{u}+b'_{i})$$
$W \in \mathbb{R}^{I}\times{K}$ for item input nodes, $V_{u} \in \mathbb{R}^{K}$ for user input node, $b \in \mathbb{R}^{K}$ for offset
5) objective function
$$\underset{W, W', b, b'} {\mathrm{arg min}} \frac{1}{U} \sum^{U}_{u=1}\ell\left(\tilde{y}_{u}, \hat{y}_{u}\right)+R(W, W', V, b, b')$$
$\ell$ is loss function, $R$ is regularization term