This is an old question, but I think it is worth upgrading my comment to an answer.
Implicit in the question seems to be the following presupposition, which I think is rather widespread, especially among students.
The purpose of metamathematical investigations is to justify mathematics, by showing that mathematical reasoning is reliable and free from contradiction. Therefore, in metamathematical discussions, we must refrain from using mathematical reasoning. It would be circular to justify the correctness of arithmetic by invoking arithmetic theorems, or to justify set theory by using set-theoretic reasoning.
The simplest response is to deny that the purpose of metamathematics is to justify mathematics. If we are not trying to justify mathematics, then there is no reason why we should tie one hand behind our backs when doing metamathematics. Indeed, in Gödel's paper, On the completeness of the calculus of logic, in which he proved his completeness theorem, he wrote:
In conclusion, let me make a remark about the means of proof used in what follows. Concerning them, no restriction whatsoever has been made.
He then went on to defend, in particular, his use of the law of the excluded middle. The details of that defense need not concern us here; the point is that Gödel certainly understood that the means of proof used in metamathematical investigations need to be understood if one wants to draw philosophical conclusions, but that does not mean that those means of proof always need to be restricted.
Still, you might object, didn't Hilbert's second problem, and more generally Hilbert's program, seek to justify mathematical reasoning? If Gödel's incompleteness theorems are supposed to be relevant to Hilbert's program, then don't they have to follow the rules of the game that Hilbert laid out? In particular, doesn't he have to "start from scratch" when proving his metamathematical results?
It is true that Hilbert was concerned with justifying mathematics, and so if we want our metamathematical investigations to be relevant to Hilbert's program, then we do need to pay some attention to the means of proof that we allow. However, even Hilbert understood that justifying mathematics "from scratch" makes no sense. If you're not allowed to assume anything, then you can't get off the ground. (See this MO answer for a particularly radical flavor of skepticism that you could adopt.) Hilbert proposed that we take finitary mathematical reasoning for granted. His original hope was that by using finitary reasoning, we could prove that set theory is consistent. The word finitary is not completely precise, but is often taken to mean "provable in primitive recursive arithmetic (PRA). So, returning to your original question about the fundamental theorem of arithmetic, the answer is that FTA is a theorem of PRA, so it's okay to invoke it.
There is one other remark worth making. Gödel's incompleteness theorem implies that
$(*)$ If PA is consistent, then PA does not prove that PA is consistent.
Sometimes people get confused about the significance of $(*)$. The thought process goes something like, "Well, who would ever have thought that the negation of $(*)$ would be of any interest anyway? Suppose that, per impossibile, PA did prove its own consistency; if we were doubtful about PA's reliability in the first place, then those doubts would certainly not be assuaged by a PA-proof of its own consistency! That would be circular. So why is Gödel's result interesting, if all it does is refute something that wouldn't have been interesting anyway?"
The point is that $(*)$ implies the following:
$(**)$ If PA is consistent, then PA does not prove that ZF is consistent.
And $(**)$ directly refutes (at least one form of) Hilbert's program to prove the consistency of set theory by finitary means. Even though $(**)$ is logically weaker than $(*)$, it is easier to see why it is interesting. The scenario is not that we don't trust PA; the scenario is that we do trust PA, and are wondering if we can "bootstrap" our way up to the consistency of ZF. And the answer is no; in fact, on the basis of PA, we can't even arrive at the consistency of PA, let alone the consistency of a stronger system.