Skip to content

Commit a0c3118

Browse files
Santos SafraoSantos Safrao
Santos Safrao
authored and
Santos Safrao
committed
first commit
0 parents commit a0c3118

File tree

8 files changed

+271
-0
lines changed

8 files changed

+271
-0
lines changed

‎README.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Mutual Subspace Methods Repository
2+
3+
This repository hosts a collection of mutual subspace methods and their implementations.
4+
5+
## Methods
6+
- [x] Mutual Subspace Method (MSM)
7+
- [ ] Constraint Mutual Subspace method (CMSM)
8+
- [ ] Orthogonal Mutual subspace method (OMSM)
9+
- [x] Kernel Mutual Subspace Method (KMSM)
10+
- [ ] Kernel Constraint Mutual Subspace method (KCMSM)
11+
- [ ] Kernel Orthogonal Mutual subspace method (KOMSM)
12+
- [ ] Random Fourier Features MSM (RFFMSM)
13+
- [ ] K-means KOMSM
14+
- [ ] RFF + K-means KOMSM
15+
16+
## Sample Implementation
17+
Below is a sample implementation for the Mutual Subspace Method (MSM):
18+
19+
## Datasets Used
20+
1. CVLabFace
21+
2. TsukubaHand24x24
22+
Each example implementation is available in files named as `example_(name_of_the_method).m`.
23+
24+
## Further Reading
25+
If you are interested in learning more about these methods, consider reviewing the following papers:
26+
```matlab
27+
reference_subspaces = cvlBasisVector(training_data, num_dim_reference_subspaces);
28+
input_subspaces = cvlBasisVector(testing_data, num_dim_input_subspace);
29+
similarities = cvlCanonicalAngles(reference_subspaces,input_subspaces);
30+
accuracy = cvlComputeAccuracy(similarities, num_sets, num_classes);
31+
32+
fprintf('Accuracy MSM: %.2f%%\n', accuracy * 100);
33+
```
34+
35+
36+
### Basics of Subspace Methods
37+
- [Subspace Methods](http://www.cvlab.cs.tsukuba.ac.jp/~kfukui/english/epapers/subspace_method.pdf)
38+
39+
### CMSM, OMSM, KCMSM, KOMSM
40+
- [Comparison between Constrained Mutual Subspace Method and Orthogonal Mutual Subspace Method](https://www.cs.tsukuba.ac.jp/internal/techreport/data/CS-TR-06-7.pdf)
41+
- [Face Recognition with the Multiple Constrained Mutual Subspace Method](http://www.cvlab.cs.tsukuba.ac.jp/~kfukui/english/epapers/AVBPA05.pdf)
42+
43+
### Kmeans KOMSM
44+
- [Hand Shape Recognition based on Kernel Orthogonal Mutual Subspace Method ](http://www.cvlab.cs.tsukuba.ac.jp/~kfukui/english/epapers/MVA2009.pdf)
45+
46+
---
47+
48+
Feel free to explore and contribute to the repository!

‎computeBasisVectors.m‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
function [basis_vectors] = computeBasisVectors(X, num_sub_dim)
2+
size_of_X = size(X);
3+
num_sets = prod(size_of_X)/prod(size_of_X(1:2));
4+
X = reshape(X, size_of_X(1), size_of_X(2), num_sets);
5+
basis_vectors = zeros(size_of_X(1), num_sub_dim, num_sets);
6+
for i = 1:num_sets
7+
[~, basis_vectors(:,:,i), ~, ~, ~] = computePCA(X(:,:,i), num_sub_dim)
8+
end

‎computePCA.m‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
function [eig_vectors, eig_values, eig_ratio] = computePCA(X, num_sub_dim, varargin)
2+
% computePCA: compute PCA of X
3+
% Input:
4+
% X: num_dim x num_samples data matrix
5+
% num_sub_dim: number of sub-dimensions to keep (Principal Components). if num_sub_dim < 1
6+
% then it is the ratio of variance to keep
7+
% varargin: if not specified PCA is computed on covariance matrix,
8+
% if set to 'R' then PCA is computed on correlation matrix
9+
% Output:
10+
% Z: projected data to the principal components
11+
% eig_vectors: eigenvectors of the covariance/correlation matrix
12+
% eig_values: eigenvalues of the covariance/correlation matrix
13+
% eig_ratio: ratio of variance explained by each principal component
14+
% Last Update: 2023/10 by Santos Enoque
15+
% Computer vison laboratory, University of Tsukuba
16+
% http://www.cvlab.cs.tsukuba.ac.jp/
17+
X = X(:, :);
18+
[num_dim, num_samples] = size(X);
19+
20+
use_covariance_matrix = true;
21+
if nargin > 2
22+
if strcmp(varargin{1}, 'R')
23+
use_covariance_matrix = false;
24+
end
25+
end
26+
27+
apply_matrix_normalization = use_covariance_matrix;
28+
29+
if num_sub_dim < 1
30+
c_ratio = num_sub_dim;
31+
if num_dim < num_samples
32+
if use_covariance_matrix
33+
C = cov(X', 1);
34+
else
35+
C = X*X'/(num_samples-1);
36+
end
37+
[eig_vectors, temp_diagonal] = eig(C);
38+
[eig_values, ind] = sort(diag(temp_diagonal), 'descend');
39+
eig_vectors = eig_vectors(:, ind);
40+
num_sub_dim = find(cumsum(eig_values)/sum(eig_values) >= c_ratio, 1);
41+
eig_vectors = eig_vectors(:, 1:num_sub_dim);
42+
eig_values = eig_values(1:num_sub_dim);
43+
eig_ratio = sum(eig_values)/trace(C);
44+
else
45+
if apply_matrix_normalization
46+
K = X'*X;
47+
IN = ones(num_samples, num_samples)/num_samples;
48+
K = K - IN*K - K*IN + IN*K*IN;
49+
else
50+
K = X'*X;
51+
end
52+
[A, B] = eig(K);
53+
[B, ind] = sort(diag(B), 'descend');
54+
A = A(:, ind);
55+
eig_values = B/num_samples;
56+
num_sub_dim = find(cumsum(eig_values)/sum(eig_values) >= c_ratio, 1);
57+
A = A(:, 1:num_sub_dim);
58+
B = B(1:num_sub_dim);
59+
A = A/sqrt(diag(B));
60+
eig_vectors = X*A;
61+
eig_values = eig_values(1:num_sub_dim);
62+
eig_ratio = sum(eig_values);
63+
end
64+
elseif num_sub_dim >= 1
65+
num_sub_dim = floor(num_sub_dim);
66+
if num_dim < num_samples
67+
if use_covariance_matrix
68+
C = cov(X', 1);
69+
else
70+
C = X*X'/(num_samples-1);
71+
end
72+
73+
OPTS.disp = 0;
74+
if num_sub_dim < num_dim
75+
[eig_vectors, temp_diagonal] = eigs(C, num_sub_dim, 'LM', OPTS);
76+
else
77+
[eig_vectors, temp_diagonal] = eig(C);
78+
end
79+
[eig_values, ind] = sort(diag(temp_diagonal), 'descend');
80+
eig_vectors = eig_vectors(:, ind);
81+
eig_ratio = sum(eig_values)/trace(C);
82+
else
83+
if apply_matrix_normalization
84+
K = X'*X;
85+
IN = ones(num_samples, num_samples)/num_samples;
86+
K = K - IN*K - K*IN + IN*K*IN;
87+
else
88+
K = X'*X;
89+
end
90+
OPTS.disp = 0;
91+
[A, B] = eigs(K, num_sub_dim, 'LM', OPTS);
92+
[B, ind] = sort(diag(B), 'descend');
93+
A = A(:, ind);
94+
eig_values = B/num_samples;
95+
A = A/sqrt(diag(B));
96+
eig_vectors = X*A;
97+
eig_ratio = sum(eig_values);
98+
end
99+
end
100+

‎simplePCA.m‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
function [coeff, score, latent] = simplePCA(data)
2+
% Center the data
3+
data_centered = data - mean(data);
4+
5+
% Calculate covariance matrix
6+
covarianceMatrix = cov(data_centered);
7+
8+
% Perform eigenvalue decomposition
9+
[coeff, latent] = eig(covarianceMatrix);
10+
11+
% Calculate the scores
12+
score = data_centered * coeff;
13+
14+
% Sort the components by explained variance (in descending order)
15+
[latent, idx] = sort(diag(latent), 'descend');
16+
coeff = coeff(:, idx);
17+
score = score(:, idx);
18+
end

‎testComputePCA.m‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
function tests = testComputePCA
2+
tests = functiontests(localfunctions);
3+
end
4+
5+
6+
function testPCADimensions(testCase)
7+
data = rand(4, 4);
8+
num_sub_dim = 2;
9+
[eig_vecs, eig_vals, eig_rat] = computePCA(data, num_sub_dim);
10+
verifySize(testCase, eig_vecs, [size(data, 1), num_sub_dim]);
11+
verifySize(testCase, eig_vals, [num_sub_dim, 1]);
12+
verifySize(testCase, eig_rat, [1,1]);
13+
end
14+
15+
function testPCACovMatrix(testCase)
16+
data = [1, 2, 3, 4; 5, 6, 7, 8];
17+
num_sub_dim = 2;
18+
[eig_vecs, eig_vals, eig_rat] = computePCA(data, num_sub_dim);
19+
20+
% expected values
21+
exp_eig_vecs = [0.7071, -0.7071; 0.7071, 0.7071];
22+
exp_eig_vals = [2.5000; 0];
23+
exp_eig_rat = 1;
24+
25+
% Custom comparison function to handle sign ambiguity in PCA
26+
verifyEqual(testCase, abs(eig_vecs), abs(exp_eig_vecs), 'AbsTol', 0.0001);
27+
verifyEqual(testCase, abs(eig_vals), abs(exp_eig_vals), 'AbsTol', 0.0001);
28+
verifyEqual(testCase, abs(eig_rat), abs(exp_eig_rat), 'AbsTol', 0.0001);
29+
end
30+
31+
function testPCACorrelationMatrix(testCase)
32+
data = [1, 2, 3, 4; 5, 6, 7, 8];
33+
num_sub_dim = 2;
34+
[eig_vecs, eig_vals, eig_rat] = computePCA(data, num_sub_dim, 'R');
35+
36+
% expected values
37+
exp_eig_vecs = [ 0.3762, -0.9266; 0.9266, 0.3762];
38+
exp_eig_vals = [67.4730; 0.5270];
39+
exp_eig_rat = 1;
40+
41+
% Custom comparison function to handle sign ambiguity in PCA
42+
verifyEqual(testCase, abs(eig_vecs), abs(exp_eig_vecs), 'AbsTol', 0.0001);
43+
verifyEqual(testCase, abs(eig_vals), abs(exp_eig_vals), 'AbsTol', 0.0001);
44+
verifyEqual(testCase, abs(eig_rat), abs(exp_eig_rat), 'AbsTol', 0.0001);
45+
end
46+
47+
function testPCAOnCumultativeRatio(testCase)
48+
data = [1, 2, 3, 4; 5, 6, 7, 8];
49+
c_rate = 0.6;
50+
[eig_vecs, eig_vals, eig_rat] = computePCA(data, c_rate, 'R');
51+
52+
% expected values
53+
exp_eig_vecs = [ 0.3762; 0.9266];
54+
exp_eig_vals = 67.4730;
55+
exp_eig_rat = 0.9923;
56+
57+
% Custom comparison function to handle sign ambiguity in PCA
58+
verifyEqual(testCase, abs(eig_vecs), abs(exp_eig_vecs), 'AbsTol', 0.0001);
59+
verifyEqual(testCase, abs(eig_vals), abs(exp_eig_vals), 'AbsTol', 0.0001);
60+
verifyEqual(testCase, abs(eig_rat), abs(exp_eig_rat), 'AbsTol', 0.0001);
61+
end

‎testSimplePCA.m‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
function tests = testSimplePCA
2+
tests = functiontests(localfunctions);
3+
end
4+
5+
function testPCADimensions(testCase)
6+
data = rand(10, 3);
7+
[coeff, score, latent] = simplePCA(data);
8+
verifySize(testCase, coeff, [3, 3]);
9+
verifySize(testCase, score, [10, 3]);
10+
verifySize(testCase, latent, [3, 1]);
11+
end
12+
13+
function testPCAValues(testCase)
14+
data = [1, 2; 3, 4; 5, 6];
15+
16+
% Centering the data manually
17+
data = data - mean(data);
18+
19+
[coeff, score, ~] = simplePCA(data);
20+
21+
% Define expected values
22+
expCoeff = [0.7071, 0.7071; -0.7071, 0.7071];
23+
expScore = [-2.8284, 0; 0, 0; 2.8284, 0];
24+
25+
% Custom comparison function to handle sign ambiguity in PCA
26+
verifyEqual(testCase, abs(coeff), abs(expCoeff), 'AbsTol', 0.0001);
27+
verifyEqual(testCase, abs(score), abs(expScore), 'AbsTol', 0.0001);
28+
end

‎unitTests.m‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
clc
2+
results = runtests('testComputePCA');
3+
disp(results)

‎untitled.m‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
data = [1, 2, 3, 4; 5, 6, 7, 8;];
2+
[vec, vals, ratio] = computePCA(data, 0.6, 'R');
3+
vec
4+
vals
5+
ratio

0 commit comments

Comments
 (0)