For the Gauss circle problem $$ R(x):=\sum_{0 \leq n \leq x} r_2(n)=\pi x+P(x), P(x)=O(x^{\frac{1}{4}+\epsilon}) $$ Gauss may not know the integral formula of the error term. $$ \int_0^X|P(x)| d x=O\left(X^{1+1 / 4}\right) $$ Assuming he really doesn't know, how does he make the guess of the exponent $\frac{1}{4}$?
Is there any chance to recover the path Gauss gets the reasonable exponent $\frac{1}{4}$?
EDIT: According to https://cmsr.rutgers.edu/images/people/lebowitz_joel/publications/jll.pub_347.pdf, the conjecture that the exponent is $\frac{1}{4} + \epsilon$ is due to Hardy, and not Gauss. The paper of Hardy that it cites as containing the conjecture (Hardy, G.H.: The average order of the arithmetical functions $P(x)$ and $\Delta(x)$. Proc. London Math. Soc. 15, 192-213 (1916)) proves $\int_1^X |P(t)| dt = O(X^{1 + 1/4 + \epsilon})$. In the proof, Hardy uses his identity expressing $P$ as a sum of Bessel functions, where each individual term of the sum is $O(X^{1/4})$ (e.g. see https://mathoverflow.net/a/314011/481175). This was probably the true inspiration for his conjecture.
My original answer, describing a probabilistic heuristic for the error term, is contained below.
I’m not sure how Gauss thought about this, but here is a heuristic way to see why this should be true.
To leading order, one would expect $4\pi \sqrt{x}$ lattice points within a distance of 1 to the circle, by taking the difference in the areas of circles of radius $\sqrt{x}\pm 1$.
If one pretends that each of these points falls inside/outside of the circle with iid probability $\frac{1}{2}$, the total number that fall inside is $2\pi\sqrt{x} + O_p(x^{1/4})$ by the central limit theorem.
$2\pi\sqrt{x}$ of these points are already included in the crude estimate $\pi x$ (as the area between the circles of radius $\sqrt{x}$ and $\sqrt{x}-1$).
Thus, if one buys the probabilistic heuristic, the error term is of order $x^{1/4}$ for a fixed $x$. One may expect additional logarithmic factors when one considers the worst deviation over all $x$, as in the extension from the central limit theorem to the law of the iterated logarithm. To be safe, one can conjecture the error term is $O(x^{1/4 + \epsilon})$.
