Word2Vec Model

distributed representations of words

word2vec is a class of neural network models, trained on unlabelled trainign corpus, that produce a vector for each word in the corpus that encode its valuable syntactic and semantic informaton .

Skip-grams (SG) - predict context words given target (position independent)

Continuous Bag of Words (CBOW) - Predict target word from bag-of-words context

Stochastic Gradient Descent

reference

• Linguistic Regularities in Continuous Space Word Representations,
• [The amazing power of word vectors](https://blog.acolyer.org/2016/04/21/the-amazing-power-of-word-vectors/)
• [Word2vec tutorial](http://mccormickml.com/assets/word2vec/Alex_Minnaar_Word2Vec_Tutorial_Part_I_The_Skip-Gram_Model.pdf)
• [Deep Learning Basics: Neural Networks, Backpropagation and Stochastic Gradient Descent](http://alexminnaar.com/deep-learning-basics-neural-networks-backpropagation-and-stochastic-gradient-descent.html)