Skip to main content

Questions tagged [hidden-markov-models]

0 votes
1 answer
324 views

Hidden Markov model with two hidden states?

I am currently studying what Markov models are, and have a question. If we have a hidden Markov model with 2 hidden states or observations, then how do we find the probability of just the main state ...
MarleyMania's user avatar
1 vote
0 answers
93 views

Estimates on number of observations and iterations depending on number of states, when applying the Baum-Welch algorithm to HMMs

Are there papers estimating how many observations $\mathcal{O}$ and how many random initializations $\ell$ one needs, to get an arbitrary good agreement between the model $\lambda = (\pi,A,B)$ that ...
Ben123's user avatar
  • 285
3 votes
0 answers
179 views

Is the Kalman Filter computationally optimal for Kalman filtering?

Kalman filtering is known to be a recursive process that minimizes mean square error in linear problems. My question is: has anybody shown that this algorithm is computationally optimal, i.e. that you ...
Diego Méndez's user avatar
1 vote
0 answers
78 views

What's the autocovariance of 2 concurrent hidden states of a stochastic gaussian walk, using the Kalman filter?

A latent variable evolves as $x_t = x_{t-1} + e_t$ where $e_t$ has a gaussian distribution with 0 mean and a variance which defines the volatility of the overall process. Let $o_t$ be the noisy ...
Nick Gregory's user avatar
1 vote
0 answers
74 views

Reconciling notation for forwards algorithm

I am studying the forwards algorithm for hidden Markov models via Rabiner's tutorial paper and Kevin Murphy's "Machine Learning: A Probabilistic Perspective". In Rabiner's paper, he ...
Alex's user avatar
  • 111
2 votes
1 answer
178 views

The reference on Markov chains uncovering the power of the subject in a better way for a working macro-economist

This is by no means a research question. But asking here I hope for the most expert opinion. A friend of mine, who is a working economist, asked me for advice about a book which uncovers wealth and ...
Evgeny Kuznetsov's user avatar
1 vote
0 answers
66 views

Estimation of probability matrix from samples at different time intervals

I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate. Let $X_t$ be the ...
N. Gast's user avatar
  • 562
0 votes
0 answers
167 views

Formal definition of episodic Markov Decision process?

David Silver, in his lecture 4 from his Youtube lectures, speaks about episodic Markov Decision Processes (MDPs) and Monte-Carlo Policy Evaluation. I could not find a formal definition of episodic ...
Vasileios Anagnostopoulos's user avatar
1 vote
1 answer
231 views

Comprehensive reference for lumped or projected markov chains

Consider a Markov chain $X_n$ taking values in finite countable set $\mathcal{X}$ with transition matrix $P$. Consider a function $f:\mathcal{X}\to\mathcal{Y}$ inducing a partition $\mathcal{Y}=\{\...
Three Diag's user avatar
1 vote
1 answer
567 views

Markov processes: Construction of the state variables

I have asked this question on stats.se.com but I did not receive an answer. Given is the description of a probabilistic finite state machine and I want to 'translate this' into a Markov process 'on it'...
Fabian Werner's user avatar
1 vote
0 answers
377 views

Existence of solution for Poisson equation in Markov chain

Consider $X_n\in \mathcal{X}$ a controlled Markov chain taking value in a compact set $\mathcal{X}$ with action $a\in \mathcal{A}$, where the action set $|\mathcal{A}|$ is finite. (In particular, we ...
Sung-En Chiu's user avatar
-1 votes
1 answer
1k views

Efficiency of the Baum-Welch Algorithm [closed]

One of our famous mathematicians, James Simons, used an extension of the Baum-Welch algorithm to 'crack' the wall street when he started trading on the stock market. Now, as Google, all informations ...
J.Doe's user avatar
  • 1
9 votes
4 answers
3k views

How can the Kalman filter be adapted to handle binary observations?

Imagine a coin with a time-varying probability of coming up heads. (For example, perhaps the probability follows a random walk that is constrained to live in $[0, 1]$. And say we have some ...
8one6's user avatar
  • 251
1 vote
0 answers
64 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
mikkola's user avatar
  • 171
2 votes
1 answer
1k views

forward algorithm Hidden Markov Model

I am studying the the forward-backward algorithm used in Hidden Markov Models. I understand that you are trying to propagate through a sequence (and the available states) to find the most probable ...
brucezepplin's user avatar
1 vote
1 answer
283 views

Hidden Markov: representing joint probability for set of observations as a product of two subset probabilities

Good day to everyone! My question concerns Hidden Markov Models and is pretty basic. In one of the books ("Introduction to Machine Learning" by Ethem Alpaydin, 2nd Edition, p.373), I get the ...
AGro's user avatar
  • 13
2 votes
1 answer
2k views

How to find next to optimal path in hidden Markov model or what should be LIST-Viterbi algorithm?

The Viterbi algorithm is an algorithm for finding the most likely sequence of hidden states – called the Viterbi path. Question If I am interested in list of several paths - optimal, sub-optimal, ...
Alexander Chervov's user avatar
12 votes
5 answers
2k views

What is hidden in Hidden Markov Models? [closed]

Why the word "hidden" present in hidden markov model? What exactly is hidden. Whatever is hidden in HMM isn't it hidden in normal Markov Models?
user1692's user avatar
  • 139