These are tutorials with code samples that teach you how to accomplish specific goals with text data. For example, extracting topics from text, using vectorizers correctly and etc.
A Hands-On Word2Vec Tutorial Using the Gensim Package The idea behind Word2Vec is pretty simple. We’re making an assumption that the meaning of a word can be inferred by the company it keeps. This is analogous to the saying, “show me your friends, and I’ll tell who you are”. If you have two words that …
One of the really nice things about spark is the ability to read input files of different formats right out of the box. Though this is a nice to have feature, reading files in spark is not always consistent and seems to keep changing with different spark releases. This article will show you how to …
How to read CSV & JSON files in Spark – word count example Read More »
These are some handy notes for MEAD. What is MEAD? MEAD is a publicly available framework for summarization. It is not really an ‘algorithm’. By default (I guess when it was first implemented) it was developed based on a centroid based approach…
This article talks about how to work with ROUGE for evaluation of summarization tasks. The original ROUGE toolkit uses a perl implementation which is really hard to understand, so I decided to piece together some information that may be helpful to others…