This is another straightforward example of deeplearning in Keras. Let’s summarize what is going on here:
- IMDB reviews are bits or text consisting of words (duh!) but all these words are converted to numbers. In addition, the number of a word corresponds to its importance in terms of occurrence. So, a small numbers means it occurs more often than a large number.
- reviews have different lengths, so one truncates everything to the same length (here 5000)
- part of the reviews are taken for testing (here 33% of the whole)
- only the 500 most important words are considered
- the input is hence a vector of size 5000 which is a bit large and sparse so an embedding is applied which maps these vectors to a vector of size 32. Word embeddings is a topic of its own and you should have a look at the Glove site for instance.
- the one-dimensional convolution is well explained in this article
- the max-pooling is picking up the maximum out of a neighborhood and is a way to reduce the dimension
- flattening a matrix is simply putting the rows of the matrix one after another
- the dense layers are the usual deeplearning layers where the weights are trained. Note that we want a yes/no classification and the last dense layer is dimension one.
- the whole networks has a low accuracy but is trained in a minute, which is altogether not too bad
The beauty of Keras resides really in the ease to add/change things. It runs on top of Theano and TensorFlow so you get really a delicious framework with all you can wish for.
import numpy as np import os os.environ['THEANO_FLAGS'] = "device=gpu" from keras.datasets import imdb from keras.models import Sequential from keras.layers import Dense from keras.layers import Flatten from keras.layers.convolutional import Convolution1D from keras.layers.convolutional import MaxPooling1D from keras.layers.embeddings import Embedding from keras.preprocessing import sequence as prep import time # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) # load the dataset but only keep the top n words, zero the rest top_words = 5000 test_split = 0.33 (X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=top_words, test_split=test_split) # pad dataset to a maxumum review length in words max_words = 500 X_train = prep.pad_sequences(X_train, maxlen=max_words) X_test = prep.pad_sequences(X_test, maxlen=max_words) # create the model model = Sequential() model.add(Embedding(top_words, 32, input_length=max_words)) model.add(Convolution1D(nb_filter=32, filter_length=3, border_mode='same', activation='relu')) model.add(MaxPooling1D(pool_length=max_words)) model.add(Flatten()) model.add(Dense(250, activation='relu')) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) print(model.summary()) start = time.time() model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=5, batch_size=128, verbose=2) print("> Training is done in %.2f seconds." % (time.time() - start)) scores = model.evaluate(X_test, y_test, verbose=2) print("Accuracy: %.2f%%" % (scores * 100)) # Accuracy: 86.87% # gpu: 62s # cpu: 58s