Symmetric matrices in orthogonal groups of ternary quadratic forms

Question

This is related to my earlier question: Intersection of orthogonal groups

(See also Matrix expression for elements of $SO(3)$)

I am interested in extracting symmetric elements of the orthogonal groups $O_f$ for $f$ a (non-singular) real ternary quadratic form. However, there seems to be a discrepancy that I find perplexing.

If we take $f(x,y,z) = x^2 + y^2 + z^2$, so that $O_f$ is simply the standard orthogonal group $O_3$, then the rotations in $O_3$ can be parametrized as

$$\displaystyle \begin{bmatrix} a^2+b^2-c^2-d^2&2bc-2ad &2bd+2ac \\ 2bc+2ad &a^2-b^2+c^2-d^2&2cd-2ab \\ 2bd-2ac &2cd+2ab &a^2-b^2-c^2+d^2\\ \end{bmatrix}$$

with $a^2 + b^2 + c^2 + d^2 = 1$ (see David Speyer's answer to the second linked question above). Requiring this to be symmetric can be done simply by imposing the condition $a = 0$, which gives a generic family of the shape

$$\displaystyle \begin{bmatrix} b^2 - c^2 - d^2 & 2bc & 2bd \\ 2bc & -b^2 + c^2 - d^2 & 2cd \\ 2bd & 2cd & -b^2 - c^2 + d^2 \end{bmatrix}, \quad b^2 + c^2 + d^2 = 1.$$

However, if we consider the quadratic form $f(x,y,z) = y^2 -xz$, then the according to the paper Binary quartic forms having bounded invariants, and the boundedness of the average rank of elliptic curves, the generic element of $O_f$ takes the form

$$\displaystyle \frac{1}{ad-bc}\begin{bmatrix} d^2 & cd & c^2 \\ 2bd & ad + bc & 2ac \\ b^2 & ab & a^2 \end{bmatrix}, \quad ad - bc = 1.$$

Requiring a matrix of this shape to be symmetric is far more stringent. Indeed, one would require $b = c = 0$ and the resulting matrix takes the form

$$\displaystyle \frac{1}{ad} \begin{bmatrix} d^2 & 0 & 0 \\ 0 & ad & 0 \\ 0 & 0 & a^2 \end{bmatrix}, \quad ad = 1.$$

Is there a reason why the two cases differ? How does one characterize quadratic forms for which one scenario happens or the other?

Answer 1

This is perhaps not a satisfying explaination of what's going on for the specific $f$, but it shows what happens "generically".

Let $V = K^n$ ($\operatorname{char}K \neq 2$) and $\Lambda \colon V\to V^*$ an isomorphism with $\Lambda^\intercal = \Lambda$. $$ \operatorname{O}_\Lambda = \{B \in \operatorname{End}(V) \mid B^\intercal \Lambda B = \Lambda\}, \qquad \operatorname{Sym}_\Lambda = \{B \in \operatorname{End}(V) \mid B^\intercal \Lambda = \Lambda B\} $$ For $\Lambda = I_n$ you obtain the usual orthogonal group and symmetric matrices. If we pick the same $\Lambda$ twice, then we can nicely describe $$ \operatorname{O}_\Lambda \cap \operatorname{Sym}_\Lambda = \{B \in \operatorname{End}(V) \mid \Lambda^{-1}B^\intercal \Lambda = B = B^{-1} \} = \{B \in \operatorname{O}_\Lambda\mid B = B^{-1}\}. $$ If $\Lambda \cong I_n$ (for example $K = \overline K$, or $K=\mathbb R$ and $\Lambda$ positive definite), then this is isomorphic to $\operatorname{O}(n) \cap \operatorname{Sym}(n)$. This is a reducible algebraic variety with smooth irreducible components $X_0,\dots,X_n$ $$X_k = \{B \in \operatorname{O}(n) \cap \operatorname{Sym}(n) \mid \text{eigenvalues }(+1)^k(-1)^{n-k}\}, \qquad \dim X_k = k(n-k). $$ In your example ($n=3$), there are four components, $X_3=\{I_3\},X_0=\{-I_3\}$, the component $X_1$ you characterize and $X_2 = -X_1$.

On the other hand, the intersection $\operatorname{O}_\Lambda \cap \operatorname{Sym}(n)$ for $\Lambda \neq I_n$ can be much smaller. In the extreme case consider $\Lambda = \operatorname{diag}(\lambda_1,\dots,\lambda_n)$ with $\Lambda^2$ having $n$ distinct eigenvalues and $C := \Lambda B$ $$ \operatorname{O}_\Lambda \cap \operatorname{Sym}(n) = \{B \in \operatorname{Sym}(n) \mid (\Lambda B)^2 = \Lambda^2\} \subseteq \{\Lambda^{-1}C \mid C = \operatorname{diag}(\pm\lambda_1,\dots,\pm\lambda_n)\} $$ and the latter is a finite set (matrices with distinct eigenvalues have $2^n<\infty$ square roots). Thus for generic $\Lambda$, the intersection $\operatorname{O}_\Lambda \cap \operatorname{Sym}(n)$ is finite (and non-empty, containing at least $B=I_n$).

In your situation, $\Lambda = \left[\begin{smallmatrix}0&0&\frac12\\0&1&0\\\frac12&0&0\end{smallmatrix}\right]$ is sort of in between these extremes, since $\Lambda^2 = \operatorname{diag}(\frac14,1,\frac14)$ has one repeated eigenvalue. A Macaulay2 computation ($\operatorname{char}K=0$) gives six components, four points and two curves (again, additive inverses of each other) $$ \operatorname{O}_\Lambda \cap \operatorname{Sym}(n) = \left\{\left.\begin{bmatrix}0&0&\varepsilon_1\\0&\varepsilon_2&0\\\varepsilon_1&0&0\end{bmatrix} \, \right\vert\, \varepsilon_1,\varepsilon_2 = \pm1\right\} \cup \left\{\left.\begin{bmatrix}a&0&0\\0&\varepsilon&0\\0&0&a^{-1}\end{bmatrix} \, \right\vert\, a \in K^\times,\ \varepsilon = \pm 1\right\}. $$ I expect that the "similarity" of $\Lambda$ to $I_n$ determines how large the intersection is, the more similar, the bigger (but this is just based on the extreme cases and some computations).