One of the questions that often comes up is what’s the difference between fastText and Word2Vec? Aren’t they both the same? Yes and no. They are conceptually the same, but there is a minor difference—fastText operates at a character level but Word2Vec operates at a word level. Why this difference? Before we dive into fastText , let’s quickly …
Previously, we learned how to use CountVectorizer for text processing. In place of CountVectorizer, you also have the option of using HashingVectorizer. In this tutorial, we will learn how HashingVectorizer differs from CountVectorizer and when to use which. CountVectorizer vs. HashingVectorizer HashingVectorizer and CountVectorizer are meant to do the same thing. Which is to convert …
Scikit-learn’s CountVectorizer is used to transform a corpora of text to a vector of term / token counts. It also provides the capability to preprocess your text data prior to generating the vector representation making it a highly flexible feature representation module for text. In this article, we are going to go in-depth into the …
Text classification is the automatic process of predicting one or more categories given a piece of text. For example, predicting if an email is legit or spammy. Thanks to Gmail’s spam classifier, I don’t see or hear from spammy emails! Other than spam detection, text classifiers can be used to determine sentiment in social media …
Build Your First Text Classifier in Python with Logistic Regression Read More »
Training a Word2Vec model with phrases is very similar to training a Word2Vec model with single words. The difference: you would need to add a layer of intelligence in processing your text data to pre-discover phrases. In this tutorial, you will learn how to create embeddings with phrases without explicitly specifying the number of words …
How to incorporate phrases into Word2Vec – a text mining approach Read More »
In evaluating multi-class classification problems, we often think that the only way to evaluate performance is by computing the accuracy which is the proportion or percentage of correctly predicted labels over all predictions. However, we can always compute precision and recall for each class label and analyze the individual performance on class labels or average the …
How to compute precision and recall for a multi-class classification problem Read More »