Text Classification with Naive Bayes

In this assignment, you will implement the Naive Bayes classification method and use it for sentiment classification of customer reviews. Write a report containing your answers, including the visualizations. Submit your report and your Python code/notebook.

Preliminaries
Read up the Naive Bayes classifier: how to compute apply the Naive Bayes rule, and how to estimate the probabilities you need.

If you wish, you may also have a look at the following classic paper:

· Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan : Thumbs up? Sentiment Classification using Machine Learning Techniques . In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002).

The dataset we are using was originally created for the experiments described in the following paper. The research described here addresses the problem of domain adaptation, such as adapting a classifier of book reviews to work with camera reviews.

· John Blitzer, Mark Dredze, and Fernando Pereira: Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL 2007).

Preparatory remarks
Frequency-counting in Python. The built-in data structure called Counter is a special type of Python dictionary that is adapted for computing frequencies. In the following example, we compute the frequencies of words in a collection of two short documents. We use Counter in three different ways, but the end results are the same (freqs1, freqs2, and freqs3 are identical at the end). The Counter will not give a KeyError if you look for a word that you didn’t see before.

from collections import Counter

example_documents = [‘the first document’.split(), ‘the second document’.split()]

freqs1 = Counter()

for doc in example_documents:

for w in doc:

freqs1[w] += 1

freqs2 = Counter()

for doc in example_documents:

freqs2.update(doc)

freqs3 = Counter(w for doc in example_documents for w in doc)

print(freqs1)

print(freqs1[‘the’])

print(freqs1[‘neverseen’])

Logarithmic probabilities. If you multiply many small probabilities you may run into problems with numeric precision: the probability becomes zero. To handle this problem, I recommend that you compute the logarithms of the probabilities instead of the probabilities. To compute the logarithm in Python, use the function log in the numpy library.

The logarithms have the mathematical property that np.log(P1 * P2) = np.log(P1) + np.log(P2). So if you use log probabilities, all multiplications (for instance, in the Naive Bayes probability formula) will be replaced by sums.

If you’d like to come back from log probabilities to normal probabilities, you can apply the exponential function, which is the inverse of the logarithm: prob = np.exp(logprob). (However, if the log probability is too small, exp will just return zero.)