arXiv++ Combinatorics

Why does the low dimensionality of representations, typically $d\approx 1000$, not prevent modern embedding-based retrieval models from scaling to billions, or even trillions, of data points? To answer this question, we study maximal-margin embeddings in the following retrieval model, classically studied in communication complexity [PS86] and more recently in embedding-based retrieval [WBNL26]. Let $A\in \{0,1\}^{N\times n}$ be a matrix indicating whether each of $N$ queries is relevant to each of $n$ documents. We are interested in the largest margin $m>0,$ denoted by $\mathsf{m}^{\mathsf{rd}}(d, A),$ for which there exist unit norm embeddings of the queries and documents $\{U_j\}_{j = 1}^N, \{V_i\}_{i = 1}^n$ with the following property. $\langle U_j, V_i\rangle \ge m$ whenever $A_{ji} = 1$ and $\langle U_j, V_i\rangle \le -m$ otherwise. A large margin is a key proxy for representation quality: it controls both robustness to perturbations and compositional generalization across queries. Our main theorem establishes that the best possible margin without a restriction on the dimension, $\mathsf{m}^{\mathsf{rd}}(+\infty, A),$ can be nearly achieved in dimension $d = O(\mathsf{m}^{\mathsf{rd}}(+\infty, A)^{-2}\log n)$ which improves a theorem of [BDES02]. Together with a matching lower bound in Theorem 1.5, we conclude that when $A\in \{0,1\}^{\binom{n}{k}\times n}$ is the matrix containing all possible $k$-sparse rows once, dimension $d = O(k\log (n/k))$ is necessary and sufficient for the maximal possible margin $\mathsf{m}^{\mathsf{rd}}(+\infty, A) = Θ(k^{-1/2})$ in this setting. This fully resolves the setup of [WBNL26]. We also give several constructions for large margins when $d = o(k\log (n/k)).$ Finally, we empirically test the InfoNCE and sigmoid losses for producing large margin embeddings and demonstrate a clear advantage of the sigmoid loss.

The celebrated theorem of Chung, Graham, and Wilson on quasirandom graphs implies that if the 4-cycle and edge counts in a graph $G$ are both close to their typical number in $\mathbb{G}(n,1/2),$ then this also holds for the counts of subgraphs isomorphic to $H$ for any $H$ of constant size. We aim to prove a similar statement where the notion of close is whether the given (signed) subgraph count can be used as a test between $\mathbb{G}(n,1/2)$ and a stochastic block model $\mathbb{SBM}.$ Quantitatively, this is related to approximately maximizing $H \longrightarrow |Φ(H)|^{\frac{1}{|\mathsf{V}(H)|}},$ where $Φ(H)$ is the Fourier coefficient of $\mathbb{SBM}$, indexed by subgraph $H.$ This formulation turns out to be equivalent to approximately maximizing the partition function of a spin model over alphabet equal to the community labels in $\mathbb{SBM}.$ We resolve the approximate maximization when $\mathbb{SBM}$ satisfies one of four conditions: 1) the probability of an edge between any two vertices in different communities is exactly $1/2$; 2) the probability of an edge between two vertices from any two communities is at least $1/2$ (this case is also covered in a recent work of Yu, Zadik, and Zhang); 3) the probability of belonging to any given community is at least $c$ for some universal constant $c>0$; 4) $\mathbb{SBM}$ has two communities. In each of these cases, we show that there is an approximate maximizer of $|Φ(H)|^{\frac{1}{|\mathsf{V}(H)|}}$ in the set $\mathsf{A} = \{\text{stars, 4-cycle}\}.$ This implies that if there exists a constant-degree polynomial test distinguishing $\mathbb{G}(n,1/2)$ and $\mathbb{SBM},$ then the two distributions can also be distinguished via the signed count of some graph in $\mathsf{A}.$ We conjecture that the same holds true for distinguishing $\mathbb{G}(n,1/2)$ and any graphon if we also add triangles to $\mathsf{A}.$

The distribution $\mathsf{RGG}(n,\mathbb{S}^{d-1},p)$ is formed by sampling independent vectors $\{V_i\}_{i = 1}^n$ uniformly on $\mathbb{S}^{d-1}$ and placing an edge between pairs of vertices $i$ and $j$ for which $\langle V_i,V_j\rangle \ge τ^p_d,$ where $τ^p_d$ is such that the expected density is $p.$ Our main result is a poly-time implementable coupling between Erdős-Rényi and $\mathsf{RGG}$ such that $\mathsf{G}(n,p(1 - \tilde{O}(\sqrt{np/d})))\subseteq \mathsf{RGG}(n,\mathbb{S}^{d-1},p)\subseteq \mathsf{G}(n,p(1 + \tilde{O}(\sqrt{np/d})))$ edgewise with high probability when $d\gg np.$ We apply the result to: 1) Sharp Thresholds: We show that for any monotone property having a sharp threshold with respect to the Erdős-Rényi distribution and critical probability $p^c_n,$ random geometric graphs also exhibit a sharp threshold when $d\gg np^c_n,$ thus partially answering a question of Perkins. 2) Robust Testing: The coupling shows that testing between $\mathsf{G}(n,p)$ and $\mathsf{RGG}(n,\mathbb{S}^{d-1},p)$ with $εn^2p$ adversarially corrupted edges for any constant $ε>0$ is information-theoretically impossible when $d\gg np.$ We match this lower bound with an efficient (constant degree SoS) spectral refutation algorithm when $d\ll np.$ 3) Enumeration: We show that the number of geometric graphs in dimension $d$ is at least $\exp(dn\log^{-7}n)$, recovering (up to the log factors) the sharp result of Sauermann.

A random algebraic graph is defined by a group $G$ with a uniform distribution over it and a connection $σ:G\longrightarrow[0,1]$ with expectation $p,$ satisfying $σ(g)=σ(g^{-1}).$ The random graph $\mathsf{RAG}(n,G,p,σ)$ with vertex set $[n]$ is formed as follows. First, $n$ independent vectors $x_1,\ldots,x_n$ are sampled uniformly from $G.$ Then, vertices $i,j$ are connected with probability $σ(x_ix_j^{-1}).$ This model captures random geometric graphs over the sphere and the hypercube, certain regimes of the stochastic block model, and random subgraphs of Cayley graphs. The main question of interest to the current paper is: when is a random algebraic graph statistically and/or computationally distinguishable from $\mathsf{G}(n,p)$? Our results fall into two categories. 1) Geometric. We focus on the case $G =\{\pm1\}^d$ and use Fourier-analytic tools. For hard threshold connections, we match [LMSY22b] for $p = ω(1/n)$ and for $1/(r\sqrt{d})$-Lipschitz connections we extend the results of [LR21b] when $d = Ω(n\log n)$ to the non-monotone setting. We study other connections such as indicators of interval unions and low-degree polynomials. 2) Algebraic. We provide evidence for an exponential statistical-computational gap. Consider any finite group $G$ and let $A\subseteq G$ be a set of elements formed by including each set of the form $\{g, g^{-1}\}$ independently with probability $1/2.$ Let $Γ_n(G,A)$ be the distribution of random graphs formed by taking a uniformly random induced subgraph of size $n$ of the Cayley graph $Γ(G,A).$ Then, $Γ_n(G,A)$ and $\mathsf{G}(n,1/2)$ are statistically indistinguishable with high probability over $A$ if and only if $\log|G|\gtrsim n.$ However, low-degree polynomial tests fail to distinguish $Γ_n(G,A)$ and $\mathsf{G}(n,1/2)$ with high probability over $A$ when $\log |G|=\log^{Ω(1)}n.$

For two graphs $X$ and $Y$ with vertex sets $V(X)$ and $V(Y)$ of the same cardinality $n,$ the friends-and-strangers graph $\mathsf{FS}(X,Y)$ was recently defined by Defant and Kravitz. The vertices of $\mathsf{FS}(X,Y)$ are the bijections from $V(X)$ to $V(Y),$ and two bijections $σ$ and $τ$ are adjacent if they agree everywhere except at two vertices $a,b\in V(X)$ such that $a$ and $b$ are adjacent in $X$ and $σ(a)$ and $σ(b)$ are adjacent in $Y.$ We study generalized versions of two problems by Alon, Defant, and Kravitz. First, we show that if $X$ and $Y$ have minimum degrees $δ(X)$ and $δ(Y)$ that satisfy $δ(X)> n/2, δ(Y)>n/2,$ and $2\min(δ(X), δ(Y))+3\max(δ(X), δ(Y))\ge 3n,$ then $\mathsf{FS}(X,Y)$ is connected. As a corollary, we settle a recent conjecture by Alon, Defant, and Kravitz stating that there exists a number $d_n = 3n/5 + O(1)$ such that if both $X$ and $Y$ have minimum degrees at least $d_n,$ the graph $\mathsf{FS}(X,Y)$ is connected. When $X$ and $Y$ are bipartite, a parity obstruction prevents $\mathsf{FS}(X,Y)$ from being connected. We show that if $X$ and $Y$ are edge-subgraphs of $K_{r,r}$ that satisfy $δ(X)+δ(Y)\ge 3r/2+1,$ then the graph $\mathsf{FS}(X,Y)$ has exactly two connected components. As a corollary, we provide an almost complete answer to another recent question of Alon, Defant, and Kravitz asking for the minimum number $d^*_{r,r}$ such that for any edge-subgraph $X$ of $K_{r,r}$ satisfying $δ(X)\ge d^*_{r,r},$ the graph $\mathsf{FS}(X,K_{r,r})$ has exactly two connected components. We show that $d^*_{r,r} = r/2+1$ when $r$ is even and $d^*_{r,r}\in \{\lceil r/2\rceil, \lceil r/2\rceil+1\}$ when $r$ is odd.

For the word $ω= \underbrace{11\ldots 1}_{x_1}\underbrace{22\ldots2}_{x_2}\ldots\underbrace{nn\ldots n}_{x_n},$ denote by $\mathsf{A}(x_1, x_2, \ldots, x_n)$ the number of its anagrams without fixed letters. While the function $\mathsf{A}()$ bears significant importance to economic theory \cite{MCKELVEY1997411}, it is not known whether it can be computed in polynomial time. The desire to answer efficiently certain queries related to this function motivates our study of its combinatorial properties. Our first main result shows that $\mathsf{A}(x_1, x_2, \ldots, x_n)\pmod{p}$ can be efficiently computed for any prime $p = O((\log n)^{1/3}).$ Our second main result establishes that the function $\mathsf{A}()$ is Schur-concave, which means that certain ordinal queries about $\mathsf{A}()$ can be answered in linearithmic time. Our second direction of study is structural. We introduce the anagraph, which generalizes derangement graphs. For $(x_1, x_2, \ldots, x_n)\in \mathbb{Z}_{\ge 0}^n,$ $\mathcal{AG}(x_1, x_2, \ldots, x_n)$ is a graph on vertex set all words over the alphabet $[n]$ which have exactly $x_i$ letters $i.$ Two vertices are adjacent if they are anagrams without fixed letters of each other. Our main result fully determines the $n$-tuples $(x_1, \ldots, x_n)$ for which the anagraph is connected and leads to a linear algorithm for this task. We end with a conjecture, which fits into the ongoing debate about the connection between hamiltonicity and vertex-transitivity \cite{transitive}. One contribution of the current paper is a systematic development of techniques for analyzing anagrams without fixed letters. We illustrate the power of these techniques with further arithmetic, ordinal, and structural results.