These are pseudosquares. The first $73$ are tabulated at https://oeis.org/A002189 and many references to the literature are given.
Thanks to @Oscar Lanzi for pointing out that OP only wants prime pseudosquares. The links at the oeis site still give some useful information on best methods for finding pseudosquares, and estimates for how fast they grow.