Using theano to implement maximum likelihood learning in neural probability language model Python -


i'm trying implement maximum likelihood learning neural probability language model in python code of log-bilinear model: https://github.com/wenjieguan/log-bilinear-language-models/blob/master/lbl.py

i used grad function in theano compute gradient , try using function train update parameters of model, got errors. here code:

def train(self, sentences, alpha = 0.001, batches = 1000):     print('start training...')     self.alpha = alpha     count = 0      rare = self.vocab['<>']     #print rare     q = np.zeros(self.dim, np.float32)     #print q     delta_context = [np.zeros((self.dim, self.dim), np.float32) in range(self.context) ]     #print delta_context     delta_feature = np.zeros((len(self.vocab), self.dim), np.float32)     #print delta_feature     sentence in sentences:         sentence = self.start_sen + sentence + self.end_sen         pos in range(self.context, len(sentence) ):             count += 1             q.fill(0)             featurew = []             contextmatrix = []             indices = []             i, r in enumerate(sentence[pos - self.context : pos]):                 if r == '<_>':                     continue                 index = self.vocab.get(r, rare)                 print index                 indices.append(index)                 ri = self.featurevectors[index]                 #print ri                 ci = self.contextmatrix[i]                 #print ci                 featurew.append(ri)                 contextmatrix.append(ci)                 #caculating predicted representation target word                 q += np.dot(ci, ri)             #computing energy function             energy = np.exp(np.dot(self.featurevectors, q) + self.biases)             #print energy             #computing conditional distribution             probs = energy / np.sum(energy)             #print probs             w_index = self.vocab.get(sentence[pos], rare)               #computing gradient             logprobs = t.log(probs[w_index])             print 'gradient start...'             delta_context, delta_feature = t.grad(logprobs, [self.contextmatrix, self.featurevectors])             print 'gradient completed!'             train = theano.function(                                     inputs = [self.vocab],                                     outputs = [logprobs],                                     updates=((self.featurevectors,self.featurevectors - self.alpha * delta_feature),                                               (self.contextmatrix,self.contextmatrix - self.alpha * delta_context)),                                     name="train"                                     )        print('training finished!') 

i have learnt python , neural probability language model, quite difficult me. me, please! thank you!


Comments

Popular posts from this blog

matlab - "Contour not rendered for non-finite ZData" -

javascript - Any ideas when Firefox is likely to implement lengthAdjust and textLength? -

delphi - Indy UDP Read Contents of Adata -