arXiv++ Combinatorics

Let $G$ be a graph and $U\subset V(G)$ be a set of vertices. For each $v\in U$, let $h_v\colon U\to \{0, 1\}$ be the function defined by \[h_v(u)=\begin{cases} &1 ~\mbox{if}~u\sim v, u\in U\\&0 ~\mbox{if}~u\not\sim v, u\in U\end{cases},\] and set $\mathcal{H}(U):=\{h_v\colon v\in U\}$. The first purpose of this paper is to study the following question: What families of graphs $G$ and what conditions on $U$ do we need so that the VC-dimension of $\mathcal{H}(U)$ can be determined? We show that if $G$ is a pseudo-random graph, then under some mild conditions, the VC dimension of $\mathcal{H}(U)$ can be bounded from below. Specific cases of this theorem recover and improve previous results on VC-dimension of functions defined by the well-studied distance and dot-product graphs over a finite field.

We study the Vapnik-Chervonenkis (VC) dimension of the set of quadratic residues (i.e. squares) in finite fields, $\mathbb F_q$, when considered as a subset of the additive group. We conjecture that as $q \to \infty$, the squares have the maximum possible VC-dimension, viz. $(1+o(1))\log_2 q$. We prove, using the Weil bound for multiplicative character sums, that the VC-dimension is $\geq (\frac{1}{2} + o(1))\log_2 q$. We also provide numerical evidence for our conjectures. The results generalize to multiplicative subgroups $Γ\subseteq \mathbb F_q^\times$ of bounded index.

Given a domain $X$ and a collection $\mathcal{H}$ of functions $h:X\to \{0,1\}$, the Vapnik-Chervonenkis (VC) dimension of $\mathcal{H}$ measures its complexity in an appropriate sense. In particular, the fundamental theorem of statistical learning says that a hypothesis class with finite VC-dimension is PAC learnable. Recent work by Fitzpatrick, Wyman, the fourth and seventh named authors studied the VC-dimension of a natural family of functions $\mathcal{H}_t^{'2}(E): \mathbb{F}_q^2\to \{0,1\}$, corresponding to indicator functions of circles centered at points in a subset $E\subseteq \mathbb{F}_q^2$. They showed that when $|E|$ is large enough, the VC-dimension of $\mathcal{H}_t^{'2}(E)$ is the same as in the case that $E = \mathbb F_q^2$. We study a related hypothesis class, $\mathcal{H}_t^d(E)$, corresponding to intersections of spheres in $\mathbb{F}_q^d$, and ask how large $E\subseteq \mathbb{F}_q^d$ needs to be to ensure the maximum possible VC-dimension. We resolve this problem in all dimensions, proving that whenever $|E|\geq C_dq^{d-1/(d-1)}$ for $d\geq 3$, the VC-dimension of $\mathcal{H}_t^d(E)$ is as large as possible. We get a slightly stronger result if $d=3$: this result holds as long as $|E|\geq C_3 q^{7/3}$. Furthermore, when $d=2$ the result holds when $|E|\geq C_2 q^{7/4}$.

We introduce tree dimension and its leveled variant in order to measure the complexity of leaf sets in binary trees. We then provide a tight upper bound on the size of such sets using leveled tree dimension. This, in turn, implies both the famous Sauer-Shelah Lemma for VC dimension and Bhaskar's version for Littlestone dimension, giving clearer insight into why these results place the exact same upper bound on their respective shatter functions. We also classify the isomorphism types of maximal leaf sets by tree dimension. Finally, we generalize this analysis to higher-arity trees.

A long-standing sample compression conjecture asks to linearly bound the size of the optimal sample compression schemes by the Vapnik-Chervonenkis (VC) dimension of an arbitrary class. In this paper, we explore the rich metric and combinatorial structure of oriented matroids (OMs) to construct proper unlabeled sample compression schemes for the classes of topes of OMs bounded by their VC-dimension. The result extends to the topes of affine OMs, as well as to the topes of the complexes of OMs that possess a corner peeling. The main tool that we use are the solutions of certain oriented matroid programs.

Support Vector Machines (SVMs) are one of the most popular supervised learning models to classify using a hyperplane in an Euclidean space. Similar to SVMs, tropical SVMs classify data points using a tropical hyperplane under the tropical metric with the max-plus algebra. In this paper, first we show generalization error bounds of tropical SVMs over the tropical projective torus. While the generalization error bounds attained via Vapnik-Chervonenkis (VC) dimensions in a distribution-free manner still depend on the dimension, we also show numerically and theoretically by extreme value statistics that the tropical SVMs for classifying data points from two Gaussian distributions as well as empirical data sets of different neuron types are fairly robust against the curse of dimensionality. Extreme value statistics also underlie the anomalous scaling behaviors of the tropical distance between random vectors with additional noise dimensions. Finally, we define tropical SVMs over a function space with the tropical metric.

Any binary string can be associated with a unary predicate $P$ on $\mathbb{N}$. In this paper we investigate subsets named by a predicate $P$ such that the relation $P(x+y)$ has finite VC dimension. This provides a measure of complexity for binary strings with different properties than the standard string complexity function (based on diversity of substrings). We prove that strings of bounded VC dimension are meagre in the topology of the reals, provide simple rules for bounding the VC dimension of a string, and show that the bi-infinite strings of VC dimension $d$ are a non-sofic shift space. Additionally we characterize the irreducible strings of low VC dimension (0,1 and 2), and provide connections to mathematical logic.

An upper bound on the trace function of a hypergraph $H$ is derived and its applications are demonstrated. For instance, a new upper bound for the VC dimension of $H$, or $vc(H)$, follows as a consequence and can be used to compute $vc(H)$ in polynomial time provided that $H$ has bounded degeneracy. This was not previously known. Particularly, when $H$ is a hypergraph arising from closed neighborhoods of a graph, this approach asymptotically improves the time complexity of the previous result for computing $vc(H)$. Another consequence is a general lower bound on the {\it distinguishing transversal number } of $H$ that gives rise to applications in domination theory of graphs. To effectively apply the methods developed here, one needs to have good estimations of degeneracy, and its variation or reduced degeneracy which is introduced here.

The induced odd cycle packing number $iocp(G)$ of a graph $G$ is the maximum integer $k$ such that $G$ contains an induced subgraph consisting of $k$ pairwise vertex-disjoint odd cycles. Motivated by applications to geometric graphs, Bonamy et al.~\cite{indoc} proved that graphs of bounded induced odd cycle packing number, bounded VC dimension, and linear independence number admit a randomized EPTAS for the independence number. We show that the assumption of bounded VC dimension is not necessary, exhibiting a randomized algorithm that for any integers $k\ge 0$ and $t\ge 1$ and any $n$-vertex graph $G$ of induced odd cycle packing number at most $k$ returns in time $O_{k,t}(n^{k+4})$ an independent set of $G$ whose size is at least $α(G)-n/t$ with high probability. In addition, we present $χ$-boundedness results for graphs with bounded odd cycle packing number, and use them to design a QPTAS for the independence number only assuming bounded induced odd cycle packing number.

Function classes are collections of Boolean functions on a finite set, which are fundamental objects of study in theoretical computer science. We study algebraic properties of ideals associated to function classes previously defined by the third author. We consider the broad family of intersection-closed function classes, and describe cellular free resolutions of their ideals by order complexes of the associated posets. For function classes arising from matroids, polyhedral cell complexes, and more generally interval Cohen-Macaulay posets, we show that the multigraded Betti numbers are pure, and are given combinatorially by the Möbius functions. We then apply our methods to derive bounds on the VC dimension of some important families of function classes in learning theory.

We examine connections between combinatorial notions that arise in machine learning and topological notions in cubical/simplicial geometry. These connections enable to export results from geometry to machine learning. Our first main result is based on a geometric construction by Tracy Hall (2004) of a partial shelling of the cross-polytope which can not be extended. We use it to derive a maximum class of VC dimension 3 that has no corners. This refutes several previous works in machine learning from the past 11 years. In particular, it implies that all previous constructions of optimal unlabeled sample compression schemes for maximum classes are erroneous. On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes. We leave as open whether our unlabeled sample compression scheme extends to ample (a.k.a. lopsided or extremal) classes, which represent a natural and far-reaching generalization of maximum classes. Towards resolving this question, we provide a geometric characterization in terms of unique sink orientations of the 1-skeletons of associated cubical complexes.

Fix positive integers $k$ and $d$. We show that, as $n\to\infty$, any set system $\mathcal{A} \subset 2^{[n]}$ for which the VC dimension of $\{ \triangle_{i=1}^k S_i \mid S_i \in \mathcal{A}\}$ is at most $d$ has size at most $(2^{d\bmod{k}}+o(1))\binom{n}{\lfloor d/k\rfloor}$. Here $\triangle$ denotes the symmetric difference operator. This is a $k$-fold generalisation of a result of Dvir and Moran, and it settles one of their questions. A key insight is that, by a compression method, the problem is equivalent to an extremal set theoretic problem on $k$-wise intersection or union that was originally due to Erdős and Frankl. We also give an example of a family $\mathcal{A} \subset 2^{[n]}$ such that the VC dimension of $\mathcal{A}\cap \mathcal{A}$ and of $\mathcal{A}\cup \mathcal{A}$ are both at most $d$, while $\lvert \mathcal{A} \rvert = Ω(n^d)$. This provides a negative answer to another question of Dvir and Moran.

We show that any family of subsets $A\subseteq 2^{[n]}$ satisfies $\lvert A\rvert \leq O\bigl(n^{\lceil{d}/{2}\rceil}\bigr)$, where $d$ is the VC dimension of $\{S\triangle T \,\vert\, S,T\in A\}$, and $\triangle$ is the symmetric difference operator. We also observe that replacing $\triangle$ by either $\cup$ or $\cap$ fails to satisfy an analogous statement. Our proof is based on the polynomial method; specifically, on an argument due to [Croot, Lev, Pach '17].

Let $G$ be an abelian group of bounded exponent and $A \subseteq G$. We show that if the collection of translates of $A$ has VC dimension at most $d$, then for every $ε>0$ there is a subgroup $H$ of $G$ of index at most $ε^{-d-o(1)}$ such that one can add or delete at most $ε|G|$ elements to/from $A$ to make it a union of $H$-cosets. We also establish a removal lemma with polynomial bounds, with applications to property testing, for induced bipartite patterns in a finite abelian group with bounded exponent.

First we consider families in the hypercube $Q_n$ with bounded VC dimension. Frankl raised the problem of estimating the number $m(n,k)$ of maximal families of VC dimension $k$. Alon, Moran and Yehudayoff showed that $$n^{(1+o(1))\frac{1}{k+1}\binom{n}{k}}\leq m(n,k)\leq n^{(1+o(1))\binom{n}{k}}.$$ We close the gap by showing that $\log \left(m(n,k)\right)= {(1+o(1))\binom{n}{k}}\log n$ and show how a tight asymptotic for the logarithm of the number of induced matchings between two adjacent small layers of $Q_n$ follows as a corollary. Next, we consider the integrity $I(Q_n)$ of the hypercube, defined as $$I(Q_n) = \min\{ |S| +m(Q_n \setminus S) : S \subseteq V (Q_n) \},$$ where $m(H)$ denotes the number of vertices in the largest connected component of $H$. Beineke, Goddard, Hamburger, Kleitman, Lipman and Pippert showed that $c\frac{2^n}{\sqrt{n}} \leq I(Q_n)\leq C\frac{2^n}{\sqrt{n}}\log n$ and suspected that their upper bound is the right value. We prove that the truth lies below the upper bound by showing that $I(Q_n)\leq C \frac{2^n}{\sqrt{n}}\sqrt{\log n}$.

We construct near optimal linear decision trees for a variety of decision problems in combinatorics and discrete geometry. For example, for any constant $k$, we construct linear decision trees that solve the $k$-SUM problem on $n$ elements using $O(n \log^2 n)$ linear queries. Moreover, the queries we use are comparison queries, which compare the sums of two $k$-subsets; when viewed as linear queries, comparison queries are $2k$-sparse and have only $\{-1,0,1\}$ coefficients. We give similar constructions for sorting sumsets $A+B$ and for solving the SUBSET-SUM problem, both with optimal number of queries, up to poly-logarithmic terms. Our constructions are based on the notion of "inference dimension", recently introduced by the authors in the context of active classification with comparison queries. This can be viewed as another contribution to the fruitful link between machine learning and discrete geometry, which goes back to the discovery of the VC dimension.

In computational complexity, a complexity class is given by a set of problems or functions, and a basic challenge is to show separations of complexity classes $A \not= B$ especially when $A$ is known to be a subset of $B$. In this paper we introduce a homological theory of functions that can be used to establish complexity separations, while also providing other interesting consequences. We propose to associate a topological space $S_A$ to each class of functions $A$, such that, to separate complexity classes $A \subseteq B'$, it suffices to observe a change in "the number of holes", i.e. homology, in $S_A$ as a subclass $B$ of $B'$ is added to $A$. In other words, if the homologies of $S_A$ and $S_{A \cup B}$ are different, then $A \not= B'$. We develop the underlying theory of functions based on combinatorial and homological commutative algebra and Stanley-Reisner theory, and recover Minsky and Papert's 1969 result that parity cannot be computed by nonmaximal degree polynomial threshold functions. In the process, we derive a "maximal principle" for polynomial threshold functions that is used to extend this result further to arbitrary symmetric functions. A surprising coincidence is demonstrated, where the maximal dimension of "holes" in $S_A$ upper bounds the VC dimension of $A$, with equality for common computational cases such as the class of polynomial threshold functions or the class of linear functionals in $\mathbb F_2$, or common algebraic cases such as when the Stanley-Reisner ring of $S_A$ is Cohen-Macaulay. As another interesting application of our theory, we prove a result that a priori has nothing to do with complexity separation: it characterizes when a vector subspace intersects the positive cone, in terms of homological conditions. By analogy to Farkas' result doing the same with *linear conditions*, we call our theorem the Homological Farkas Lemma.

The metric dimension of a graph is the minimum size of a set of vertices such that each vertex is uniquely determined by the distances to the vertices of that set. Our aim is to upper-bound the order $n$ of a graph in terms of its diameter $d$ and metric dimension $k$. In general, the bound $n\leq d^k+k$ is known to hold. We prove a bound of the form $n=\mathcal{O}(kd^2)$ for trees and outerplanar graphs (for trees we determine the best possible bound and the corresponding extremal examples). More generally, for graphs having a tree decomposition of width $w$ and length $\ell$, we obtain a bound of the form $n=\mathcal{O}(kd^2(2\ell+1)^{3w+1})$. This implies in particular that $n=\mathcal{O}(kd^{\mathcal{O}(1)})$ for graphs of constant treewidth and $n=\mathcal{O}(f(k)d^2)$ for chordal graphs, where $f$ is a doubly-exponential function. Using the notion of distance-VC dimension (introduced in 2014 by Bousquet and Thomassé) as a tool, we prove the bounds $n\leq (dk+1)^{t-1}+1$ for $K_t$-minor-free graphs, and $n\leq (dk+1)^{d(3\cdot 2^{r}+2)}+1$ for graphs of rankwidth at most $r$.

A critical set in an $n \times n$ Latin square is a minimal set of entries that uniquely identifies it among all Latin squares of the same size. It is conjectured by Nelder in 1979, and later independently by Mahmoodian, and Bate and van Rees that the size of the smallest critical set is $\lfloor n^2/4\rfloor$. We prove a lower-bound of $n^2/10^4$ for sufficiently large $n$, and thus confirm the quadratic order predicted by the conjecture. We prove a lower-bound of $n^2/10^4$ for sufficiently large $n$, and thus confirm the quadratic order predicted by the conjecture. This improves a recent lower-bound of $Ω(n^{3/2})$ due to Cavenagh and Ramadurai. From the point of view of computational learning theory, the size of the smallest critical set corresponds to the minimum teaching dimension of the set of Latin squares. We study two related notions of dimension from learning theory. We prove a lower-bound of $n^2-(e+o(1))n^{5/3}$ for both of the VC-dimension and the recursive teaching dimension.

It is a long-standing open problem whether there always exists a compression scheme whose size is of the order of the Vapnik-Chervonienkis (VC) dimension $d$. Recently compression schemes of size exponential in $d$ have been found for any concept class of VC dimension $d$. Previously, compression schemes of size $d$ have been given for maximum classes, which are special concept classes whose size equals an upper bound due to Sauer-Shelah. We consider a generalization of maximum classes called extremal classes. Their definition is based on a powerful generalization of the Sauer-Shelah bound called the Sandwich Theorem, which has been studied in several areas of combinatorics and computer science. The key result of the paper is a construction of a sample compression scheme for extremal classes of size equal to their VC dimension. We also give a number of open problems concerning the combinatorial structure of extremal classes and the existence of unlabeled compression schemes for them.