ConvSumm

Official

ConvSumm/CODS at master · salesforce/ConvSumm

Chien-Sheng Wu*, Linqing Liu*, Wenhao Liu, Pontus Stenetorp, Caiming Xiong Please cite our work if you use the code or model in this repository @article{wu2021cods, title={Controllable Abstractive Dialogue Summarization with Sketch Supervision}, author={Wu, Chien-Sheng and Liu, Linqing and Liu, Wenhao and Stenetorp, Pontus and Xiong, Caiming}, journal={arXiv preprint arXiv:2105.14064}, year={2021} } In this paper, we aim to improve abstractive dialogue summarization quality and, at the sametime, enable granularity control.

https://github.com/salesforce/ConvSumm/tree/master/CODS

Summary

We aim to improve abstractive dialogue summarization quality and, at the same time, enable granularity control. Our model has two primary components and stages:

• a two-stage generation strategy that generates a preliminary summary sketch serving as the basis for the final summary. This summary sketch provides a weakly supervised signal in the form of pseudo-labeled interrogative pronoun categories and key phrases extracted using a constituency parser.

• a simple strategy to control the granularity of the final summary, in that our model can automatically determine or control the number of generated summary sentences for a given dialogue by predicting and highlighting different text spans from the source text.

To solve these challenges, we propose CODS, a COntrollable abstractive Dialogue Summarization model equipped with sketch generation. We first automatically create a summary sketch that contains user intent information and essential key phrases that may appear in summary. It identifies the interaction between speakers and salient information in each turn. This summary sketch is prefixed to the human-annotated summary while fine-tuning a generator, which provides weak supervision as the final summary is conditioned on the generated summary sketch. In addition, we propose a length-controllable generation method specifically for dialogue summarization. Desired lengths of summaries strongly depend on the amount of information contained in the source dialogue and granularity of information the user wants to understand. We first segment the dialogue into different segments by matching each summary sentence linearly to its corresponding dialogue context. Then we train our model to generate only one sentence for each dialogue segment. This strategy makes use of the distributed information of the dialogue and make the generated summaries more trackable.

Architecture

Experiments

pyrouge

You can evaluate your plain text summaries like this: from pyrouge import Rouge155 r = Rouge155() r.system_dir = 'path/to/system_summaries' r.model_dir = 'path/to/model_summaries' r.system_filename_pattern = 'some_name.(\d+).txt' r.model_filename_pattern = 'some_name.[A-Z].#ID#.txt' output = r.convert_and_evaluate() print(output) output_dict = r.output_to_dict(output) In order to evaluate summaries, ROUGE needs to know where your summaries and the gold standard summaries are, and how to match them.

https://pypi.org/project/pyrouge/

Performance

Further Readings

Controllable Summarization with Constrained Markov Decision Process

Abstract. We study controllable text summarization, which allows users to gain control on a particular attribute (e.g., length limit) of the generated summaries. In this work, we propose a novel training framework based on Constrained Markov Decision Process (CMDP), which conveniently includes a reward function along with a set of constraints, to facilitate better summarization control.

https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00423/108199/Controllable-Summarization-with-Constrained-Markov

Connected Papers | Find and explore academic papers

Connected Papers is a visual tool to help researchers and applied scientists find academic papers relevant to their field of work.

https://www.connectedpapers.com/main/95085501ff72c296b2df3f12969e0f8a57c224fd/Controllable-Abstractive-Dialogue-Summarization-with-Sketch-Supervision/graph

Conversational AI | Salesforce Research

Communication platforms such as Slack have changed the ways we work and share. Salesforce AI Research focuses on building cutting edge AI solutions to understand these conversations and facilitate both human-human and human-machine interactions.

https://www.salesforceairesearch.com/research/conversational-ai

GitHub - microsoft/DialoGPT: Large-scale pretraining for dialogue

This repository contains the source code and trained model for a large-scale pretrained dialogue response generation model. The human evaluation results indicate that the response generated from DialoGPT is comparable to human response quality under a single-turn conversation Turing test.

https://github.com/microsoft/DialoGPT

GitHub - microsoft/unilm: Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

We are hiring at all levels (including FTE researchers and interns)! If you are interested in working with us on Foundation Models (aka large-scale pre-trained models) and AGI, NLP, MT, Speech, Document AI and Multimodal AI, please send your resume to fuwei@microsoft.com.

https://github.com/microsoft/unilm

BART

DISCLAIMER: If you see something strange, file a Github Issue and assign @patrickvonplaten The Bart model was proposed in BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer on 29 Oct, 2019.

https://huggingface.co/docs/transformers/model_doc/bart