Dialog Context Language Modeling With Recurrent Neural Networks

Dialog, NLG Comments

Paper Link

Overview

Background

However all these methods focused on applying context by encoding preceding text without considering interactions in dialogs.

Architecture

Context Dependent RNNLM

Let D = be a dialog with K turns and involve 2 speakers. In this case, turn is just the utterance of a single speaker and could involve multiple messages. The kth turn is represented as a sequence of T_k words.

Dialog RNN

Context Representations

However, the above 2 methods treat dialog history as a sequence of inputs, without modeling dialog interactions. In order to deal with this, the paper proposes 2 different architectures -

Interactive Dialog Context LM

In the above model, we define the context and initial hidden state as follows - and

External State Interactive Dialog Context LM

In the above model, we have an external RNN to encode the respresentation from 1 turn to another.

Results

Model K=1 K=2 K=3 K=5
RNNLM 60.4 - - -
DRNNLM - 60.1 58.6 59.1
CCDCLM - 63.9 61.4 62.2
IDCLM - - 58.8 58.6
ESIDCLM - - 58.4 58.5

Kaushik Rangadurai

Code. Learn. Explore

Share this post

Comments