Today, we have the sneak preview of MUSE API (Beta). MUSE API is what Musio actually talks to in the cloud to manage the dialogue and generate things to say, recognize faces, etc. In order to show you better some of the major things going on behind the Musio, we built a temporary front-end for the API. We have a video explaining features of API as well as accompanying material. Our API is on its way, so stay tuned!
Table of Contents 1. Conditional Neural Network Architectures 1.1. goal 1.2. motivation 1.3. ingredients 1.4. steps 1.5. outlook 1.6. resources Conditional Neural Network Architectures goal Today we are going to have a look at conditional neural network architectures and present some of the findings in the recent papers “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” and “PathNet: Evolution Channels Gradient Descent in Super Neural Networks”. motivation The interest in conditional models is mainly based on their capability to incorporate a huge number of parameters compared to standard architectures without increasing the need for more computationally powerful hardware. Furthermore such models seem to be able to reduce the training time and are interesting for multi-, online-task learning and transfer learning. […]
Ever since we’ve introduced Musio to the whole world, we’ve received countless questions from our partners, investors, customers as well as curious enthusiasts. So we thought it would be a good idea to answer the most frequently asked questions to make Musio more engaging and easier to understand. We hope that the following Q&A sessions enlighten our readers on who we are and what we dream with Musio. The first round of questions is about Musio as an AI robot. So let’s start! Q. I’ve heard lots of AIs in the market, starting from likes of Apple’s Siri to robots like Pepper. What kind of AI is Musio? Musio is a robot run on AKA’s special AI system (software engine) that […]
Table of Contents 1. Adversarial techniques for dialogue generation 1.1. goal 1.2. motivation 1.3. ingredients 1.4. steps 1.5. outlook 1.6. resources Adversarial techniques for dialogue generation goal This week we are going to have a look at the latest developments of generative adversarial networks (GANs) in the field of dialogue generation by summarizing the paper “Adversarial Learning for Neural Dialogue Generation”. motivation General encoder decoder models for response generation are usually not able to produce meaningful utterances and instead come up with short, generic, repetitive and non-informative sequences of words. The idea here is to apply adversarial methods so far only successful in computer vision to NLP problems, in particular dialogues. Adversarial training with respect to modeling conversations can be […]
Table of Contents 1. Compression and distillation of models 1.1. goal 1.2. motivation 1.3. ingredients 1.4. steps 1.5. outlook 1.6. resources Compression of neural networks goal In this blogpost we will have a look at methods for compressing and reducing deep neural network models in size. motivation The simple fact that bigger and deeper is better for training, leads to models that take up quite some space in memory. However, most of the time one is limited when it comes to memory. Either it is the budget on hardware or more recently developing models for mobile devices is becoming more and more popular. Another important point for deploying neural networks in applications is inference time. In general, larger models also […]
Table of Contents 1. Dilated causal convolutions for audio and text generation causal dilation convolution 1.1. goal 1.2. motivation 1.3. ingredients 1.4. steps 1.5. outlook 1.6. resources Dilated causal convolutions for audio and text generation goal In today’s summary we dive into the architecture of WaveNet and its successor ByteNet which are autoregressive generative models for generating audio and respectively sentences on character-level. motivation The architectures behind both models are based on dilated causal convolutional layers which recently got much attention also in image generation tasks. Especially modeling sequential data with long term dependencies like audio or text seem to benefit from convolutions with dilations to increase the receptive field. ingredients dilation, causal convolution, residual blocks, skip connection, gated activation function, steps Without […]
Table of Contents 1. Covering rare words 1.1. goal 1.2. motivation 1.3. ingredients 1.4. steps 1.5. outlook 1.6. resources Covering rare words goal This week’s blogpost treats a new network architecture, named pointer models, for taking care of rare words. We will dive into some details of the implementation and give a short analysis of the benefits of these kind of models. motivation Motivation for the introduction of new architectures comes directly from short-comings of RNN language models, as well as encoder decoder frameworks. Rare words, especially named entities do not experience good word embeddings and hence do not lead to appropriate sentence embeddings which might be used to initialize a decoder component for predicting an output sequence. Furthermore, the […]