Papers by Persi Diaconis
42 paper(s) by this author
· All BibTeX
Schur--Weyl duality for diagonalizing a Markov chain on the hypercube
We show how the tools of modern algebraic combinatorics -- representation theory, Murphy elements, and particularly Schur--Weyl duality -- can be used to give an explicit orthonormal basis of eigenfunctions for a "curiously slowly mixing Markov chain" on the space of binary $n$-tuples. The basis is used to give sharp rates of convergence to stationarity.
A curiously slowly mixing Markov chain
We study a Markov chain with very different mixing rates depending on how mixing is measured. The chain is the "Burnside process on the hypercube $C_2^n$." Started at the all-zeros state, it mixes in a bounded number of steps, no matter how large $n$ is, in $\ell^1$ and in $\ell^2$. And started at general $x$, it mixes in at most $\log n$ steps in $\ell^1$. But, in $\ell^2$, it takes $\frac{n}{\log n}$ steps for most starting $x$. The $\ell^2$ mixing results follow from an explicit diagonalization of the Markov chain into binomial-coefficient-valued eigenvectors.
Markov chains on Weyl groups from the geometry of the flag variety
This paper studies a basic Markov chain, the Burnside process, on the space of flags $G/B$ with $G = GL_n(\mathbb{F}_q)$ and $B$ its upper triangular matrices. This gives rise to a shuffling: a Markov chain on the symmetric group realized via the Bruhat decomposition. Actually running and describing this Markov chain requires understanding Springer fibers and the Steinberg variety. The main results give a practical algorithm for all n and q and determine the limiting behavior of the chain when q is large. In describing this behavior, we find interesting connections to the combinatorics of the Robinson-Schensted correspondence and to the geometry of orbital varieties. The construction and description is then carried over to finite Chevalley groups of arbitrary type, describing a new class of Markov chains on Weyl groups.
Permuton and local limits for the Luce model
We investigate the asymptotic properties of permutations drawn from the Luce model, a natural probabilistic framework in which permutations are generated sequentially by sampling without replacement, with selection probabilities proportional to prescribed positive weights. These permutations arise in applications such as ranking models, the Tsetlin library, and related Markov processes. Under minimal assumptions on the weights, we establish a permuton limit theorem describing the global behavior of Luce-distributed permutations and derive an explicit density of the limiting permuton. We further compute limiting pattern densities and analyze the differences between exact Luce permutations and their permuton approximations. We also study the local convergence of these permutations, proving a quenched Benjamini--Schramm limit and a central limit theorem for consecutive pattern occurrences. Finally, we prove a central limit theorem for the number of inversions.
Random sampling of contingency tables and partitions: Two practical examples of the Burnside process
Published in Stat Comput 35, 181 (2025)
• View Publication
• BIB
This paper gives new, efficient algorithms for approximate uniform sampling of contingency tables and integer partitions. The algorithms use the Burnside process, a general algorithm for sampling a uniform orbit of a finite group acting on a finite set. We show that a technique called `lumping' can be used to derive efficient implementations of the Burnside process. For both contingency tables and partitions, the lumped processes have far lower per step complexity than the original Markov chains. We also define a second Markov chain for partitions called the reflected Burnside process. The reflected Burnside process maintains the computational advantages of the lumped process but empirically converges to the uniform distribution much more rapidly. By using the reflected Burnside process we can easily sample uniform partitions of size $10^{10}$.
Counting the number of group orbits by marrying the Burnside process with importance sampling
This paper introduces a novel and general algorithm for approximately counting the number of orbits under group actions. The method is based on combining the Burnside process and importance sampling. Specializing to unitriangular groups yields an efficient algorithm for estimating the number of conjugacy classes of such groups.
An algorithm for uniform generation of unlabeled trees (Pólya trees), with an extension of Cayley's formula
Pólya trees are rooted, unlabeled trees on $n$ vertices. This paper gives an efficient, new way to generate Pólya trees. This allows comparing typical unlabeled and labeled tree statistics and comparing asymptotic theorems with `reality'.
Along the way, we give a product formula for the number of rooted labeled trees preserved by a given automorphism; this refines Cayley's formula.
Poisson approximation for large permutation groups
Published in Advances in Applied Mathematics Volume 167, Article 102883 (2025)
• View Publication
• BIB
Let $G_{k,n}$ be a group of permutations of $kn$ objects which permutes things independently in disjoint blocks of size $k$ and then permutes the blocks. We investigate the probabilistic and/or enumerative aspects of random elements of $G_{k,n}$. This includes novel limit theorems for fixed points, cycles of various lengths, number of cycles and inversions. The limits are compound Poisson distributions with interesting dependence structure.
A Vershik-Kerov theorem for wreath products
Let $G_{n,k}$ be the group of permutations of $\{1,2,\ldots, kn\}$ that permutes the first $k$ symbols arbitrarily, then the next $k$ symbols and so on through the last $k$ symbols. Finally the $n$ blocks of size $k$ are permuted in an arbitrary way. For $σ$ chosen uniformly in $G_{n,k}$, let $L_{n,k}$ be the length of the longest increasing subsequence in $σ$. For $k,n$ growing, we determine that the limiting mean of $L_{n,k}$ is asymptotic to $4\sqrt{nk}$. This is different from parallel variations of the Vershik-Kerov theorem for colored permutations.
A random walk on the Rado graph
Published
• View Publication
• BIB
The Rado graph, also known as the random graph $G(\infty, p)$, is a classical limit object for finite graphs. We study natural ball walks as a way of understanding the geometry of this graph. For the walk started at $i$, we show that order $\log_2^*i$ steps are sufficient, and for infinitely many $i$, necessary for convergence to stationarity. The proof involves an application of Hardy's inequality for trees.
Isomorphisms between random graphs
Published
• View Publication
• BIB
Consider two independent Erdős-Rényi $G(N,1/2)$ graphs. We show that with probability tending to $1$ as $N\to\infty$, the largest induced isomorphic subgraph has size either $\lfloor x_N-\varepsilon_N\rfloor$ or $\lfloor x_N+\varepsilon_N \rfloor$, where $x_N=4\log_2 N -2 \log_2 \log_2 N - 2\log_2(4/e)+1$ and $\varepsilon_N = (4\log_2 N)^{-1/2}$. Using similar techniques, we also show that if $Γ_1$ and $Γ_2$ are independent $G(n,1/2)$ and $G(N,1/2)$ random graphs, then $Γ_2$ contains an isomorphic copy of $Γ_1$ as an induced subgraph with high probability if $n\le \lfloor y_N - \varepsilon_N \rfloor$ and does not contain an isomorphic copy of $Γ_1$ as an induced subgraph with high probability if $n>\lfloor y_N+\varepsilon_N \rfloor$, where $y_N=2\log_2 N+1$ and $\varepsilon_N$ is as above.
Statistical Enumeration of Groups by Double Cosets
Published
• View Publication
• BIB
Let $H$ and $K$ be subgroups of a finite group $G$. Pick $g \in G$ uniformly at random. We study the distribution induced on double cosets. Three examples are treated in detail: 1) $H = K = $ the Borel subgroup in $GL_n(\mathbb{F}_q)$. This leads to new theorems for Mallows measure on permutations and new insights into the LU matrix factorization. 2) The double cosets of the hyperoctahedral group inside $S_{2n}$, which leads to new applications of the Ewens's sampling formula of mathematical genetics. 3) Finally, if $H$ and $K$ are parabolic subgroups of $S_n$, the double cosets are `contingency tables', studied by statisticians for the past 100 years.
Guessing about Guessing: Practical Strategies for Card Guessing with Feedback
Published
• View Publication
• BIB
In simple card games, cards are dealt one at a time and the player guesses each card sequentially. We study problems where feedback (e.g. correct/incorrect) is given after each guess. For decks with repeated values (as in blackjack where suits do not matter) the optimal strategy differs from the "greedy strategy" (of guessing a most likely card each round). Further, both optimal and greedy strategies are far too complicated for real time use by human players. Our main results show that simple heuristics perform close to optimal.
Card Guessing with Partial Feedback
Published in Combinator. Probab. Comp. 31 (2022) 1-20
• View Publication
• BIB
Consider the following experiment: a deck with $m$ copies of $n$ different card types is randomly shuffled, and a guesser attempts to guess the cards sequentially as they are drawn. Each time a guess is made, some amount of "feedback" is given. For example, one could tell the guesser the true identity of the card they just guessed (the complete feedback model) or they could be told nothing at all (the no feedback model).
In this paper we explore a partial feedback model, where upon guessing a card, the guesser is only told whether or not their guess was correct. We show in this setting that, uniformly in $n$, at most $m+O(m^{3/4}\log m)$ cards can be guessed correctly in expectation. This resolves a question of Diaconis and Graham from 1981, where even the $m=2$ case was open.
Central Limit Theorems for some Set Partition Statistics
Published
• View Publication
• BIB
We prove the conjectured limiting normality for the number of crossings of a uniformly chosen set partition of [n] = {1,2,...,n}. The arguments use a novel stochastic representation and are also used to prove central limit theorems for the dimension index and the number of levels.
The mathematics of the flip and horseshoe shuffles
Published
• View Publication
• BIB
We consider new types of perfect shuffles wherein a deck is split in half, one half of the deck is "reversed", and then the cards are interlaced. Flip shuffles are when the reversal comes from flipping the half over so that we also need to account for face-up/face-down configurations while horseshoe shuffles are when the order of the cards are reversed but all cards still face the same direction. We show that these shuffles are closely related to faro shuffling and determine the order of the associated shuffling groups.
Universal Limit Theorems in Graph Coloring Problems With Connections to Extremal Combinatorics
Published in Annals of Applied Probability, Vol. 27 (1), 337-394, 2017
• View Publication
• BIB
This paper proves limit theorems for the number of monochromatic edges in uniform random colorings of general random graphs. These can be seen as generalizations of the birthday problem (what is the chance that there are two friends with the same birthday?). It is shown that if the number of colors grows to infinity, the asymptotic distribution is either a Poisson mixture or a Normal depending solely on the limiting behavior of the ratio of the number of edges in the graph and the number of colors. This result holds for any graph sequence, deterministic or random. On the other hand, when the number of colors is fixed, a necessary and sufficient condition for asymptotic normality is determined. Finally, using some results from the emerging theory of dense graph limits, the asymptotic (non-normal) distribution is characterized for any converging sequence of dense graphs. The proofs are based on moment calculations which relate to the results of Erd\H os and Alon on extremal subgraph counts. As a consequence, a simpler proof of a result of Alon, estimating the number of isomorphic copies of a cycle of given length in graphs with a fixed number of edges, is presented.
Combinatorics of balanced carries
Published
• View Publication
• BIB
We study the combinatorics of addition using balanced digits, deriving an analog of Holte's "amazing matrix" for carries in usual addition. The eigenvalues of this matrix for base b balanced addition of n numbers are found to be 1,1/b,...,1/b^n, and formulas are given for its left and right eigenvectors. It is shown that the left eigenvectors can be identified with hyperoctahedral Foulkes characters, and that the right eigenvectors can be identified with hyperoctahedral Eulerian idempotents. We also examine the carries that occur when a column of balanced digits is added, showing this process to be determinantal. The transfer matrix method and a serendipitous diagonalization are used to study this determinantal process.
Carries, group theory, and additive combinatorics
Published
• View Publication
• BIB
Given a group G and a normal subgroup H we study the problem of choosing coset representatives with few carries.
Unseparated pairs and fixed points in random permutations
Published
• View Publication
• BIB
In a uniform random permutation Πof [n] := {1,2,...,n}, the set of elements k in [n-1] such that Π(k+1) = Π(k) + 1 has the same distribution as the set of fixed points of Πthat lie in [n-1]. We give three different proofs of this fact using, respectively, an enumeration relying on the inclusion-exclusion principle, the introduction of two different Markov chains to generate uniform random permutations, and the construction of a combinatorial bijection. We also obtain the distribution of the analogous set for circular permutations that consists of those k in [n] such that Π(k+1 mod n) = Π(k) + 1 mod n. This latter random set is just the set of fixed points of the commutator [ρ, Π], where ρis the n-cycle (1,2,...,n). We show for a general permutation ηthat, under weak conditions on the number of fixed points and 2-cycles of η, the total variation distance between the distribution of the number of fixed points of [η,Π] and a Poisson distribution with expected value 1 is small when n is large.