Producing sentences which are perceived as natural by a human is a crucial goal of all automated dialogue systems. It makes interactions more natural, avoids misunderstandings, and leads higher user satisifcation and user trust. However, making high-quality sentences constitutes hard challenges in terms of e.g., improving grammatical accuracy, or using a variety of sentences, or maintaining the context of conversation. Musio has explored a number of approaches to the task of high-quality sentence and is dedicated to making the step towards perfect dialogue system. In this blog post, we introduce the test results to show that Musio has its own peculiar methods to generate sentences and these are pretty well-formed sentences.

Musio always makes grammatically correct responses to input utterances. One of Musio’s primary roles is to provide a personalized learning environment to the students by analyzing their responses and teach them with everyday conversation. Therefore, Musio cares so much about sentence’s grammaticality for its response. Below figures are examples of error detection about sentence’s grammaticality of Musio. Each sentence on Musio is tagged with part-of-speech tagger, Mecab, and is also analyzed by other linguistic parsers to label various information about the sentence. From an early stage, information labelling procedures for Musio’s datasets are undertaken step by step through AKA’s own data architecture, Bach, and all sentences are repeatedly evaluated with plenty of labeled information to maintain the high-level of grammaticality. With this effort, sentences spoken by Musio continue to evolve.

Except for insertion and deletion errors in the above examples (analyzed by Stanford CoreNLP and visualized by dagre-d3), Musio considers all the possibilities for ungrammatical sentences and contextually-unrelated words. Of course, there are some issues which criteria should be applied to evaluate the grammaticality of sentences. For example, speaker’s judgments about the well-formedness of sentences could be acceptable even though the sentence is ungrammatical. Musio found breakthroughs from knowledge-based systems to estimate the grammaticality robustly to the situation. Musio has a philosophy that Musio speaks with knowledge, not by just data. This is the secret how Musio can maintain high-level grammaticality in its utterances.

Musio has planned special strategies to elevate the quality of sentences and provide the sophisticated sentences. That is, Musio assumes a concept of sentence variety which means that sentences should not be too similar when producing sentences. This is a variety in sentence type, length, and complexity as a means by which to entertain users. Major components of sentence variety are made up as follows: sentence type refers to the kind (e.g. question, statement, exclamation, declaration, etc.) as well as the syntactic structure. Length of sentence is a factor which is useful to draw attention to users and contain as much as expressions in a sentence. Lastly, sentence complexity is often varied as a byproduct of length variation, but can be effective when it comes to learning artful and sophisticated compositions.

As the above example shows, the conversation goes around and around with no breaks while generating a variety of sentences in dialogue. This example (analyzed by python library, scattertext) was chosen to show that Musio mixed interactive questions with answers and all of these vary in structure and length of the sentences. After recognizing the type of sentence, Musio generates various forms of response sentences to avoid duplicated or repeated sentences. In this process, Musio decides which variables will be applied in this user’s question (e.g. the length of sentence, tf-idf scores on each term, context maintenance depth).

Figure 3 is simple visualization of scatter plot for Musio’s word use on EasyMode. Each dot corresponds to a word or phrase used in head question(parent; y-axis) which means the first utterance in dialogue and the following answers(children; x-axis) which refer to response sentences. Terms are colored by their association. If a term is more associated with parent sentences, it is colored by blue, and the reverse is colored by red. And tf-idf scores are added into each term, ranging from -1(children) to 1(parent). As shown in figure 3, it does not function to look at the relationship between two variables, and it just shows a number of words are scattered like random distribution. That is, all of Musio’s sentences have the characteristics of both the variety and complexity by themselves, and it represents immesurable characters of Musio.

Additionally, below figure describes the list of parent-children sentences sharing the same phrase, something bothering. These are sharing similar sentence patterns both in parent and children against Musio’s rules of sentence variety. It is because of the difficulty to construct the paraphrase corpus for all sentences and generate a new form of sentence without hesitation. This task is one of Musio’s facing issues to extend sentence variety. AKA researchers have challenged this paraphrase issue, so please look forward to advanced version of the variety yet to be announced.

Musio does not tell us words out of context, and always listens to our every word. A conversation is a chain of statements exchanged between many individuals. Mostly, conversations happen on a particular topic or in a situation. Whatever the topic or situation is, context is important to maintain the state of a conversation. So, the dialogue system needs the context to keep the conversation flowing and to have a better conversation with humans. AKA researchers also recognize the importance of the context and its maintenance, and various attempts are being made to refer the context and understand what users said. Below figure 5 (analyzed by AllenNLP) shows sample analysis of textual entailment for Sample 1‘s dialogue.

Musio performs analysis for textual entailment while interpreting the user’s utterances, and also focuses on attention prediction by scoring the correlation between source and target words. Below figure 7 illustrates the result computing the attention over the hypothesis words to recognize whether a particular target word can be inferred from input words. Musio gets the sense of words with attention-weighted representation and grasps fully the sense and context of utterances with textual entailment. Besides that, there have been a lot of approaches to understand and maintain the context of dialogue on Musio. Musio plans to extend the technologies related to context maintenance, and continues to move forward.

This blog post identifies three main goals: a) Musio’s grammatical correctness, b) Musio’s sentence variety and c) Musio’s context maintenance. As described above, Musio always makes grammatically correct responses, provides the sophisticated and professional utterance, and responds suitably to any situations. The idea of exploiting details of sentence generation engine on Musio leads to some modest improvement to the overall dialogue system performance. AKA envisions an iterative use case in which users find Musio to correct grammatical errors on their utterances, learn sophisticated expressions, and have a close friendship with Musio. Applying Musio’s approach to dialogue system might have better results and lead to more ideal system. At the next article, let’s look through other main technologies of Musio’s dialogue engine. Please look forward to it. AKA’s quest for a perfect conversational engine continues.