1
$\begingroup$

I have the following optimization problem (like the LASSO problem but with maximization instead of minimization):

$\mathbf{maximize}_{\boldsymbol{\alpha}} \|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2$ s.t. $\|\boldsymbol{\alpha}\|_0 \leq k$

with $\alpha$ a sparse vector with no more than $k$ non-zero elements.

We can notice I have a maximization problem instead of a minimization one. To solve this problem, I have first converted into a minimization problem:

$\mathbf{minimize}_{\boldsymbol{\alpha}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2$ s.t. $\|\boldsymbol{\alpha}\|_0 \leq k$

What I want first to know is if my above minimization problem is correct or not?

I also would like to know if it is possible to solve directly the maximization problem without converting it into a minimization one? If it is possible, so please can you provide me the detailed calculation?

Then, I can proceed as follows:

$\mathbf{minimize}_{\boldsymbol{\alpha}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2 + \lambda \|\boldsymbol{\alpha}\|_1$.

This problem can be simply solved via alternating direction method of multipliers. For example by introducing an auxiliary vector $\boldsymbol{\beta}$ such that $\boldsymbol{\alpha} = \boldsymbol{\beta}$:

$\mathbf{minimize}_{\boldsymbol{\alpha}, \boldsymbol{\beta}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2 + \lambda \|\boldsymbol{\beta}\|_1$ s.t. $\boldsymbol{\alpha} = \boldsymbol{\beta}$.

Is that correct? Do I have some special trick to solve the above maximization problem?

$\endgroup$
2
  • $\begingroup$ Notice that the correct LaTeX code for a double bar is not || but \|. $\endgroup$ Commented Sep 24, 2018 at 15:58
  • $\begingroup$ @AlexM. Thank you for your time in editing my question. $\endgroup$ Commented Sep 24, 2018 at 18:48

1 Answer 1

2
$\begingroup$

That problem does not have a maximum. Unless $A$ or $k$ are zero, you can take $\alpha = Me_j$, where $e_j$ is a vector of the canonical basis, and then the objective function diverges when $M \to \infty$.

$\endgroup$
3
  • $\begingroup$ Thank you for the answer. Can you please write in your answer how my new optimization problem will become in more details by applying your proposition? $\endgroup$ Commented Sep 24, 2018 at 18:46
  • 2
    $\begingroup$ Your optimization problem doesn't become anything. It is ill-posed. You need to go back to the modelling phase, double-check that all your signs are correct and make sense, and think about adding another penalty term if that's really what you want to solve. $\endgroup$ Commented Sep 24, 2018 at 18:53
  • $\begingroup$ Yes I see. I will try to edit my whole question very soon. I will keep you informed. Thank you a lot dear Federico! $\endgroup$ Commented Sep 24, 2018 at 19:00

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.