Questions tagged [hidden-markov-models]
The hidden-markov-models tag has no summary.
18 questions
0 votes
1 answer
324 views
Hidden Markov model with two hidden states?
I am currently studying what Markov models are, and have a question. If we have a hidden Markov model with 2 hidden states or observations, then how do we find the probability of just the main state ...
1 vote
0 answers
93 views
Estimates on number of observations and iterations depending on number of states, when applying the Baum-Welch algorithm to HMMs
Are there papers estimating how many observations $\mathcal{O}$ and how many random initializations $\ell$ one needs, to get an arbitrary good agreement between the model $\lambda = (\pi,A,B)$ that ...
3 votes
0 answers
179 views
Is the Kalman Filter computationally optimal for Kalman filtering?
Kalman filtering is known to be a recursive process that minimizes mean square error in linear problems. My question is: has anybody shown that this algorithm is computationally optimal, i.e. that you ...
1 vote
0 answers
78 views
What's the autocovariance of 2 concurrent hidden states of a stochastic gaussian walk, using the Kalman filter?
A latent variable evolves as $x_t = x_{t-1} + e_t$ where $e_t$ has a gaussian distribution with 0 mean and a variance which defines the volatility of the overall process. Let $o_t$ be the noisy ...
1 vote
0 answers
74 views
Reconciling notation for forwards algorithm
I am studying the forwards algorithm for hidden Markov models via Rabiner's tutorial paper and Kevin Murphy's "Machine Learning: A Probabilistic Perspective". In Rabiner's paper, he ...
2 votes
1 answer
178 views
The reference on Markov chains uncovering the power of the subject in a better way for a working macro-economist
This is by no means a research question. But asking here I hope for the most expert opinion. A friend of mine, who is a working economist, asked me for advice about a book which uncovers wealth and ...
1 vote
0 answers
66 views
Estimation of probability matrix from samples at different time intervals
I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate. Let $X_t$ be the ...
0 votes
0 answers
167 views
Formal definition of episodic Markov Decision process?
David Silver, in his lecture 4 from his Youtube lectures, speaks about episodic Markov Decision Processes (MDPs) and Monte-Carlo Policy Evaluation. I could not find a formal definition of episodic ...
1 vote
1 answer
231 views
Comprehensive reference for lumped or projected markov chains
Consider a Markov chain $X_n$ taking values in finite countable set $\mathcal{X}$ with transition matrix $P$. Consider a function $f:\mathcal{X}\to\mathcal{Y}$ inducing a partition $\mathcal{Y}=\{\...
1 vote
1 answer
567 views
Markov processes: Construction of the state variables
I have asked this question on stats.se.com but I did not receive an answer. Given is the description of a probabilistic finite state machine and I want to 'translate this' into a Markov process 'on it'...
1 vote
0 answers
377 views
Existence of solution for Poisson equation in Markov chain
Consider $X_n\in \mathcal{X}$ a controlled Markov chain taking value in a compact set $\mathcal{X}$ with action $a\in \mathcal{A}$, where the action set $|\mathcal{A}|$ is finite. (In particular, we ...
-1 votes
1 answer
1k views
Efficiency of the Baum-Welch Algorithm [closed]
One of our famous mathematicians, James Simons, used an extension of the Baum-Welch algorithm to 'crack' the wall street when he started trading on the stock market. Now, as Google, all informations ...
9 votes
4 answers
3k views
How can the Kalman filter be adapted to handle binary observations?
Imagine a coin with a time-varying probability of coming up heads. (For example, perhaps the probability follows a random walk that is constrained to live in $[0, 1]$. And say we have some ...
1 vote
0 answers
64 views
Bounding Hidden Markov model Bayesian filter error with inexact models
In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
2 votes
1 answer
1k views
forward algorithm Hidden Markov Model
I am studying the the forward-backward algorithm used in Hidden Markov Models. I understand that you are trying to propagate through a sequence (and the available states) to find the most probable ...
1 vote
1 answer
283 views
Hidden Markov: representing joint probability for set of observations as a product of two subset probabilities
Good day to everyone! My question concerns Hidden Markov Models and is pretty basic. In one of the books ("Introduction to Machine Learning" by Ethem Alpaydin, 2nd Edition, p.373), I get the ...
2 votes
1 answer
2k views
How to find next to optimal path in hidden Markov model or what should be LIST-Viterbi algorithm?
The Viterbi algorithm is an algorithm for finding the most likely sequence of hidden states – called the Viterbi path. Question If I am interested in list of several paths - optimal, sub-optimal, ...
12 votes
5 answers
2k views
What is hidden in Hidden Markov Models? [closed]
Why the word "hidden" present in hidden markov model? What exactly is hidden. Whatever is hidden in HMM isn't it hidden in normal Markov Models?