arXiv++ Combinatorics

Browse math.CO papers from arXiv

Using Ramsey theory to measure unavoidable spurious correlations in Big Data

Published: 2017-12-27 | Updated: 2018-03-04
Comments: 21 pages

Abstract

Given a dataset we quantify how many patterns must always exist in the dataset. Formally this is done through the lens of Ramsey theory of graphs, and a quantitative bound known as Goodman's theorem. Combining statistical tools with Ramsey theory of graphs gives a nuanced understanding of how far away a dataset is from random, and what qualifies as a meaningful pattern. This method is applied to a dataset of repeated voters in the 1984 US congress, to quantify how homogeneous a subset of congressional voters is. We also measure how transitive a subset of voters is. Statistical Ramsey theory is also used with global economic trading data to provide evidence that global markets are quite transitive.

BibTeX

Loading...