As a physicist "in nature" perhaps I can give a few examples that illustrate how non-analytic functions can appear in physics and counter the idea that physicists do not worry about the justification of these procedures.

Example 1 involves one of the most precise comparisons between experiment and theory known to physics, namely the g factor of the electron. The quantity g is a proportionality factor between the spin of the electron and its magnetic moment. Perturbation theory in QED gives a formula $$g-2= c_1 \alpha + c_2 \alpha^2 + c_3 \alpha^3 + \cdots $$ where the coefficients $c_i$ can be computed from i-loop Feynman diagrams and $\alpha=e^2/\hbar c \simeq 1/137$ is the fine structure constant. Including up to four loop diagrams gives an expression for $g$ which agrees to one part in $10^{8}$ with experiment. Yet it is known that that this perturbative series has zero radius of convergence. This is true quite generally in quantum field theory. Physicists do not ignore this, rather they regard it as evidence that QFT's are not defined by their perturbation series but must also include non-perturbative effects, generally of the form $e^{-c/g^2}$ with $g$ a dimensionless coupling constant. Much effort has gone into understanding these non-perturbative effects in a variety of quantum field theories. Instanton effects in non-Abelian gauge theory are an important example of non-perturbative phenomena.

Example 2 involves the Hydrogen atom in an electric field of magnitude $E$, aka the Stark effect. One can compute the shift in the energy eigenvalues of the Hydrogen atom Hamiltonian due to the applied electric field as a power series in $E$ using perturbation theory and again one finds excellent agreement with experiment. One can also prove that this series has zero radius of convergence. In fact, the Hamiltonian is not bounded from below and does not have any normalizable energy eigenstates. The physics of this situation explains what is going on. The electron can tunnel through the potential barrier and escape from being bound to the nucleus of the Hydrogen atom, but for reasonable size electric fields the lifetime of these states exceeds the age of the universe. The perturbation theory does not converge because there are no energy eigenstates to converge to, but it still provides an excellent approximation to the energy eigenstates measured by experimentalists which are done over a time scale which is very short compared to the lifetime of the metastable state.

So I would say that at least in these examples there is a very nice interplay between the physics and the mathematics. The lack of analyticity has a clear physical interpretation and this is something that is understood by physicists. Of course I'm sure there are other example where such approximations are made without a clear physical justification, but this just means that one should understand the physics better.

As a physicist "in nature" perhaps I can give a few examples that illustrate how non-analytic functions can appear in physics and counter the idea that physicists do not worry about the justification of these procedures.

Example 1 involves one of the most precise comparisons between experiment and theory known to physics, namely the g factor of the electron. The quantity g is a proportionality factor between the spin of the electron and its magnetic moment. Perturbation theory in QED gives a formula $$g-2= c_1 \alpha + c_2 \alpha^2 + c_3 \alpha^3 + \cdots $$ where the coefficients $c_i$ can be computed from i-loop Feynman diagrams and $\alpha=e^2/\hbar c \simeq 1/137$ is the fine structure constant. Including up to four loop diagrams gives an expression for $g$ which agrees to one part in $10^{8}$ with experiment. Yet it is known that that this perturbative series has zero radius of convergence. This is true quite generally in quantum field theory. Physicists do not ignore this, rather they regard it as evidence that QFT's are not defined by their perturbation series but must also include non-perturbative effects, generally of the form $e^{-c/g^2}$ with $g$ a dimensionless coupling constant. Much effort has gone into understanding these non-perturbative effects in a variety of quantum field theories. Instanton effects in non-Abelian gauge theory are an important example of non-perturbative phenomena.

Example 2 involves the Hydrogen atom in an electric field of magnitude $E$, aka the Stark effect. One can compute the shift in the energy eigenvalues of the Hydrogen atom Hamiltonian due to the applied electric field as a power series in $E$ using perturbation theory and again one finds excellent agreement with experiment. One can also prove that this series has zero radius of convergence. In fact, the Hamiltonian is not bounded from below and does not have any normalizable energy eigenstates. The physics of this situation explains what is going on. The electron can tunnel through the potential barrier and escape from being bound to the nucleus of the Hydrogen atom, but for reasonable size electric fields the lifetime of these states exceeds the age of the universe. The perturbation theory does not converge because there are no energy eigenstates to converge to, but it still provides an excellent approximation to the energy eigenstates measured experimentally because the experiments are done on a time scale which is very short compared to the lifetime of the metastable state.

So I would say that at least in these examples there is a very nice interplay between the physics and the mathematics. The lack of analyticity has a clear physical interpretation and this is something that is understood by physicists. Of course I'm sure there are other example where such approximations are made without a clear physical justification, but this just means that one should understand the physics better.