Let $A$ be a large matrix (say, $1000 \times 1000$), and let $\mathcal I = \{2,3,5\}$ be a set of row/column indices. Let $(A^{-1})_{\cal I \times I}$ denote the submatrix of $A^{-1}$ that consists of the $\{2,3,5\}$ rows and columns of $A^{-1}$.
Is there an efficient way of computing the following $3 \times 3$ matrix inverse $((A^{-1})_{\cal I \times \cal I})^{-1}$ without inverting the large matrix $A$?
One way to go about this is as follows:
For $i,j \in \mathcal{I}$ Compute $e_i^TA^{-1}e_j$ by using the approach based on Gaussian quadrature; see for instance, a precise algorithm and analysis in our paper "Gauss quadrature for matrix inverse forms with applications."
Now that you have $[A^{-1}]_{\mathcal{I},\mathcal{I}}$, getting its inverse is an easy matter since $|\mathcal{I}|$ is small.
Let me denote $B=A^{-1}$. The question is how to efficiently compute the inverse of a submatrix of $B$ given the fact that the inverse of the full matrix $B$ is known (since $B^{-1}=A$). An efficient algorithm for this task is given in Relationship between the Inverses of a Matrix and a Submatrix.
