![]() Moreover, the model is better equipped for long-form generation, since it does not have to (autoregressively) decode the blueprint and its summary in one go, avoiding the risk of exceeding the maximum decoder length. We do not generate a global blueprint, rather, our planning process is incremental and informed by generation, which we argue affords greater control over the output and its fluency. Aside from generating blueprints and their corresponding text in one go, we propose a new architecture that iteratively plans and generates a sentence at a time, conditioning on the input and the output sentences generated so far. We develop three models that vary in how they integrate blueprints in the generation process and their ability to handle long outputs. We then convert input-output pairs into input-blueprint-output tuples and propose to learn encoder-decoder models from these augmented annotations. We enhance existing datasets (e.g., for summarization) with similar blueprints which we obtain automatically. Table 1 illustrates a plan for generating a Wikipedia abstract from the AQuaMuSe dataset (Kulkarni et al., 2020). We propose to make QUDs explicit by exploiting state-of-the-art question generation technology (Alberti et al., 2019 Lu and Lu, 2021) and use them as an intermediate representation layer for conditional generation, i.e., a question-answering (QA) blueprint operating as a proxy for both content selection (i.e., what to say) and planning (i.e., in what order). These questions and answers can be understood in terms of their use in moving a discourse forward to achieve communicative goals. Theoretical models of QUD assume that discourse contains implicit questions for each of the assertions made, which are thereby turned into answers. Specifically, we draw inspiration from the “Questions under Discussion” (QUD) theory of discourse structure, which posits that one way of articulating the structure of a text is to identify the questions and sub-questions that are raised and answered by subsequent spans of text (Carlson, 1983 Ginzburg, 1994 Van Kuppevelt, 1995 Larson, 2002 Roberts, 2012 Riester, 2019). Our work proposes a new conceptualization of text plans as a sequence of question-answer pairs. The lack of modularity further affects controllability as these systems cannot be easily tailored to individual needs. An additional challenge concerns the blackbox nature of deep learning systems, which hides the inherent complexity of modeling multiple interconnected linguistic phenomena in text generation, and makes it difficult to examine model decisions and attribute errors to specific components. These phenomena are amplified when generating long-form text, i.e., documents with multiple paragraphs (Wiseman et al., 2017), when dealing with non-linguistic data (e.g., database tables), or very long input-which is common when summarizing multiple documents (Liu and Lapata, 2019 Perez-Beltrachini et al., 2019), books (Kryściński et al., 2021), or dialogue (Chen et al., 2022 Zhong et al., 2021). Neural generation models are often prone to hallucination (Song et al., 2018 Maynez et al., 2020 Kryscinski et al., 2020 Gabriel et al., 2021), repetition and redundancy (Li et al., 2018 Suzuki and Nagata, 2017), and struggle to identify which content units are salient (Tan et al., 2017a). Evaluation across metrics and datasets demonstrates that blueprint models are more factual than alternatives which do not resort to planning and allow tighter control of the generation output. We develop Transformer-based models, each varying in how they incorporate the blueprint in the generated output (e.g., as a global plan or iteratively). We obtain blueprints automatically by exploiting state-of-the-art question generation technology and convert input-output pairs into input-blueprint-output tuples. ![]() We propose a new conceptualization of text plans as a sequence of question-answer (QA) pairs and enhance existing datasets (e.g., for summarization) with a QA blueprint operating as a proxy for content selection (i.e., what to say) and planning (i.e., in what order). In this work, we advocate planning as a useful intermediate representation for rendering conditional generation less opaque and more grounded. Do not duplicate in any form without permission of the Dallas Cowboys.The ability to convey relevant and faithful information is critical for many tasks in conditional generation and yet remains elusive for neural seq-to-seq models whose outputs often reveal hallucinations and fail to correctly cover important details.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |