Tagged: Stats

Interpreting A/B Test using Python

Written by Taro Sato on . Tagged: Python stats visualization

Suppose we ran an A/B test with two different versions of a web page, $a$ and $b$, for which we count the number of visitors and whether they convert or not. We can summarize this in a contingency table showing the frequency distribution of the events: ... Continue reading.

Brand Positioning by Correspondence Analysis

Written by Taro Sato on . Tagged: Python stats visualization

I was reading an article about visualization techniques using multidimensional scaling (MDS), the correspondence analysis in particular. The example used R, but as usual I want to find ways to do it on Python, so here goes. The correspondence analysis is useful when you have a two-way contingency table for which relative values of ratio-scaled data are of interest. ... Continue reading.

PCA and Biplot using Python

Written by Taro Sato on . Tagged: Python stats visualization

There are several ways to run principal component analysis (PCA) using various packages (scikit-learn, statsmodels, etc.) or even just rolling out your own through singular-value decomposition and such. Visualizing the PCA result can be done through biplot. I was looking at an example of using prcomp and biplot in R, but it does not seem like there is a comparable plug-and-play way of generating a biplot on Python. ... Continue reading.

Near-Duplicate Detection using MinHash: Background

Written by Taro Sato on . Tagged: math Python stats

There are numerous pieces of duplicate information served by multiple sources on the web. Many news stories that we receive from the media tend to originate from the same source, such as the Associated Press. When such contents are scraped off the web for archiving, a need may arise to categorize documents by their similarity (not in the sense of meaning of the text but the character-level or lexical matching). ... Continue reading.

Searching for Nearest-Neighbors between Two Coordinate Catalogs

Written by Taro Sato on . Tagged: astro stats

Say I have two catalogs of points, each in two-dimensional space. For each object in a catalog, I want to find the nearest object(s) in the other catalog. I can do this by computing the distances between every single unique pairs of objects and find the ones within a search radius and possibly doing an additional sort. ... Continue reading.