AKA Story

Tag: Memory

Attention/Memory in Deep Learning

Musio TeamA.I.

Goal Attention mechanisms in neural networks are a quite new phenomena and we are going to provide some background on them here. Motivation Generally speaking attention mechanisms allow the network to focus only on a certain subset of the data provided for a given task. Being able to distinguish between the necessary information at a specific step of a task further reduces the amount of information that has to be processed. Ingredients recurrent neural networks, convolutional neural networks, encoder, decoder, embedding, weights, memory, reinforcement learning Steps The idea behind attention mechanisms is certainly motivated by observing the visual attention of humans. Despite processing the visual input all at the same time, humans rather pay attention to small regions one after […]

Answering Tasks for AI

Musio TeamA.I.

Goal The aim of this short summary is to provide an overview of the tasks an AI system is, should be and will be capable to execute in the near future. In particular we focus on classifying tasks with respect to NLP. Motivation When testing existing algorithms it is of great importance to have a clear interpretation of the results. This can be achieved by setting up specific learning environments in such a way that the complexity of the task is always under control. More generally, the state-of-the-art problems determine next development steps for Musio. Ingredients common sense, coreference, compound, conjunction, deduction, induction, argument relation, counting, negation, indefinite knowledge, time reasoning, positional and size reasoning, path finding, motivational reasoning. Steps […]