We study non-holomorphic modular forms built from iterated integrals of holomorphic modular forms for SL(2,\mathbb Z) known as equivariant iterated Eisenstein integrals. A special subclass of them furnishes an equivalent description of the modular graph forms appearing in the low-energy expansion of string amplitudes at genus one. Notably the Fourier expansion of modular graph forms contains single-valued multiple zeta values. We deduce the appearance of products and higher-depth instances of multiple zeta values in equivariant iterated Eisenstein integrals, and ultimately modular graph forms, from the appearance of simpler odd Riemann zeta values. This analysis relies on so-called zeta generators which act on certain non-commutative variables in the generating series of the iterated integrals. From an extension of these non-commutative variables we incorporate iterated integrals involving holomorphic cusp forms into our setup and use them to construct the modular completion of triple Eisenstein integrals. Our work represents a fully explicit realisation of the modular graph forms within Brown's framework of equivariant iterated Eisenstein integrals and reveals structural analogies between single-valued period functions appearing in genus zero and one string amplitudes.
