Meet Musio

Beam Search

Beam search Goal In this short summary we will have a look at the beam search algorithm, which is applied in NLP for optimizing the generation of sequences of words or characters. Motivation Most recurrent neural networks are optimized on predicting the next most probable output based on the history of some input sequence. However, in general this does not lead to the most probable sequence. Ingredients RNN, decoder, greedy search, conditioning Steps Decoder architectures in the form of recurrent neural networks, LSTMs or GRUs as they are used for generating sequences of words or characters are optimized on predicting the next word. Hence during training, the networks sees a certain input sequence and should learn to predict the next […]