arXiv++ Combinatorics

Any Littlestone class, or stable graph, has finite sets which function as ``virtual elements'': these can be seen from the learning side as representing hypotheses which are expressible as weighted majority opinions of hypotheses in the class, and from the model-theoretic side as an approximate finitary version of realizing types. We introduce and study the epsilon-saturation of a Littlestone class, or stable graph, which is essentially the closure of the class under inductively adding all such virtual elements. We characterize this closure and prove that under reasonable choices of parameters, it remains Littlestone (or stable), though not always of the same Littlestone dimension. This highlights some surprising phenomena having to do with regimes of epsilon and the relation between Littlestone/stability and VC dimension.

This paper is about the surprising interaction of a foundational result from model theory, about stability of theories, with algorithmic stability in learning. First, in response to gaps in existing learning models, we introduce a new statistical learning model, called ``Probably Eventually Correct'' or PEC. We characterize Littlestone (stable) classes in terms of this model. As a corollary, Littlestone classes have frequent short definitions in a natural statistical sense. In order to obtain a characterization of Littlestone classes in terms of frequent definitions, we build an equivalence theorem highlighting what is common to many existing approximation algorithms, and to the new PEC. This is guided by an analogy to definability of types in model theory, but has its own character. Drawing on these theorems and on other recent work, we present a complete algorithmic analogue of Shelah's celebrated Unstable Formula Theorem, with algorithmic properties taking the place of the infinite.

Call a hereditary family $\mathcal{F}$ of graphs strongly persistent if there exists a graphon $W$ such that in all subgraphons $W'$ of $W$, $\mathcal{F}$ is precisely the class of finite graphs that have positive density in $W'$. Our first result is a complete characterization of the hereditary families of graphs that are strongly persistent as precisely those that are closed under substitutions. We call graphons with the self-similarity property above weakly random. A hereditary family $\mathcal{F}$ is said to have the weakly random Erdős--Hajnal property (WR) if every graphon that is a limit of graphs in $\mathcal{F}$ has a weakly random subgraphon. Among families of graphs that are closed under substitutions, we completely characterize the families that belong to WR as those with "few" prime graphs. We also extend some of the results above to structures in finite relational languages by using the theory of theons.

The celebrated Erdős-Hajnal Conjecture says that in any proper hereditary class of finite graphs we are guaranteed to have a clique or anti-clique of size $n^c$, which is a much better bound than the logarithmic size that is provided by Ramsey's Theorem in general. On the other hand, in uncountable cardinalities, the model-theoretic property of stability guarantees a uniform set much larger than the bound provided by the Erdős-Rado Theorem in general. Even though the consequences of stability in the finite have been much studied in the literature, the countable setting seems a priori quite different, namely, in the countably infinite the notion of largeness based on cardinality alone does not reveal any structure as Ramsey's Theorem already provides a countably infinite uniform set in general. In this paper, we show that the natural notion of largeness given by upper density reveals that these phenomena meet in the countable: a countable graph has an almost clique or anti-clique of positive upper density if and only if it has a positive upper density almost stable set. Moreover, this result also extends naturally to countable models of a universal theory in a finite relational language. Our methods explore a connection with the notion of convergence in the theory of limits of dense combinatorial objects, introducing and studying a natural approximate version of the Erdős-Hajnal property that allows for a negligible error in the edges (in general, predicates) but requires linear-sized uniform sets in convergent sequences of models (this is much stronger than what stable regularity can provide as the error is required to go to zero). Finally, surprisingly, we completely characterize all hereditary classes of finite graphs that have this approximate Erdős-Hajnal property. The proof highlights both differences and similarities with the original conjecture.

We use algorithmic methods from online learning to explore some important objects at the intersection of model theory and combinatorics, and find natural ways that algorithmic methods can detect and explain (and improve our understanding of) stable structure in the sense of model theory. The main theorem deals with existence of $ε$-excellent sets (which are key to the Stable Regularity Lemma, a theorem characterizing the appearance of irregular pairs in Szemerédi's celebrated Regularity Lemma). We prove that $ε$-excellent sets exist for any $ε< \frac{1}{2}$ in $k$-edge stable graphs in the sense of model theory (equivalently, Littlestone classes); earlier proofs had given this only for $ε< 1/{2^{2^k}}$ or so. We give two proofs: the first uses regret bounds from online learning, the second uses Boolean closure properties of Littlestone classes and sampling. We also give a version of the dynamic Sauer-Shelah-Perles lemma appropriate to this setting, related to definability of types. We conclude by characterizing stable/Littlestone classes as those supporting a certain abstract notion of majority: the proof shows that the two distinct, natural notions of majority, arising from measure and from dimension, densely often coincide.

We prove a regularity lemma with respect to arbitrary Keisler measures mu on V, nu on W where the bipartite graph (V,W,R) is definable in a saturated structure M and the formula R(x,y) is stable. The proof is rather quick and uses local stability theory. The special case where (V,W,R) is pseudofinite, mu, nu are the counting measures and M is suitably chosen (for example a nonstandard model of set theory), yields the stable regularity theorem of Malliaris-Shelah (Transactions AMS, 366, 2014, 1551-1585), though without explicit bounds or equitability.