DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
The dataset is filtered for the following conditions -
- There is a URL in source or target.
- Where the target contains word repetitions of at least three words
- Where the response does not contain at least one of the top-50 most frequent English words (e.g., “the”, “of”, “a”), since this probably indicates it might not be an English sentence.
- Where the response contains special markers such as “[” or “]”, as this could be markup language.
- Where source and target sequences together are longer than 200 words.
- Where the target contains offensive language.
- GPT2 architecture.
Maximum Mutual Information
Train P(Source Target).
- Use the top-k sample to
The model does near human performance on DSTC.
Code. Learn. Explore