Tips and tricks based on first hand experience
Most text classification examples that you see on the Web or in books focus on demonstrating techniques. This will help you build a pseudo usable prototype. If you want to take your classifier to the next level and use it within a product or service workflow, then there are things you need to do from …
Text Classification: Best Practices for Real World Applications Read More »
Learn what text preprocessing is, the different techniques for text preprocessing and a way to estimate how much preprocessing you may need. For those interested, I’ve also made some text preprocessing code snippets in python for you to try. Now, let’s get started!
Having spent a big part of my career as a graduate student researcher and now a Data Scientist in the industry, I have come to realize that a vast majority of solutions proposed both in academic research papers and in the work place are just not meant to ship — they just don’t scale! And …
Stop words are a set of commonly used words in any language. For example, in English, “the”, “is” and “and”, would easily qualify as stop words. In NLP and text mining applications, stop words are used to eliminate unimportant words, allowing applications to focus on the important words instead. While it is fairly easy to …
In evaluating multi-class classification problems, we often think that the only way to evaluate performance is by computing the accuracy which is the proportion or percentage of correctly predicted labels over all predictions. However, we can always compute precision and recall for each class label and analyze the individual performance on class labels or average the …
How to compute precision and recall for a multi-class classification problem Read More »