Unfortunately, the following solutions are not independent of $N,$ (they can't be, see comment to your question) but may still help reduce the computations:

1. By using Schur complement, you can compute the upper $n\times n$ matrix $(\boldsymbol{D}_n)$ by using an inverse of an $(N-n)\times (N-n)$ matrix and an $n\times n$ matrix.

See: https://en.wikipedia.org/wiki/Schur_complement

2. You may use the method (in the link below) based on matrix inversion using Cholesky decomposition but stop during the second step (equation solving) at the required number of elements.

See: https://de.mathworks.com/matlabcentral/fileexchange/41957-matrix-inversion-using-cholesky-decomposition

Unfortunately, the following solutions are not independent of $N,$ (they can't be, see comment to your question) but may still help reduce the computations:

1. By using Schur complement, you can compute the upper $n\times n$ matrix $(\boldsymbol{D}_n)$ by using an inverse of an $(N-n)\times (N-n)$ matrix and an $n\times n$ matrix. (You can also try a few interesting approximations based on your matrix structure which may turn out to be better than direct inversion of the $n\times n$ submatrix.)

See: https://en.wikipedia.org/wiki/Schur_complement

2. You may use the method (in the link below) based on matrix inversion using Cholesky decomposition but stop during the second step (equation solving) at the required number of elements.

See: https://de.mathworks.com/matlabcentral/fileexchange/41957-matrix-inversion-using-cholesky-decomposition