6
$\begingroup$

As far as I know, there are three definitions of Markov processes (or of Markov chains).

DEFINITION 1 (WEAKER). A process $(X_t)_{t\in[0,\infty)}$ on $(\Omega,\mathcal{F},\mathbb{P})$ with values in an arbitrary measurable space $(E,\mathcal{E})$ is a Markov process iff, for all $s,t\geq 0$, $X_{t+s}|\mathcal{F}^X_t\sim X_{t+s}|X_t$, where $\mathcal{F}^X_t:=\sigma\{X_s\,|\, s\in[0,t]\}$.

DEFINITION 2 (STRONGER). A process $(X_t)_{t\in[0,\infty)}$ on $(\Omega,\mathcal{F},\mathbb{P})$ with values in an arbitrary measurable space $(E,\mathcal{E})$ is a Markov process iff it satisfies Definition 1 and there exists a family $\{P_{t,t+s}|s,t\geq 0\}$ of probability transition kernels $P_{t,t+s}:E\times \mathcal{E}\to [0,1]$ such that, for all $s,t\geq 0$, $P_{t,t+s}$ is a regular version of $X_{t+s}|X_t$ (meaning that for all $A\in \mathcal{E}$ it holds that $\mathbb{E}(1_A(X_{t+s})|X_t)=P_{t,t+s}(A,X_t)$ $\mathbb{P}$-a.e.).

DEFINITION 3 (EVEN STRONGER). Same as Definition 2 but adding the requirement that $P_{t,t}=I$ for all $t\geq 0$ (where $I(x,dy)=\delta_x(dy)$) and that the $P_{t,t+s}$ satisfy the Chapman-Kolmogorov equation, that is $P_{t+s,t+s+r}\circ P_{t,t+s}=P_{t,t+s+r}$ for all $t,s,r\geq 0$ (where, if $P$ and $K$ are kernels on $E\times \mathcal{E}$, $P\circ K$ is defined for $(x,A)\in E\times\mathcal{E}$ as $(P\circ K)(x,A)=\int_E P(y,A)K(x,dy)$).

I have 3 questions:

  1. Can anyone provide an example of a process that satisfies Definition 1 but not Definition 2? I am aware that such a process should be valued in some non-Borel-standard space $E$, so I imagine it might be difficult to construct. SOLVED: see Cinlar's Probability and Stochastic, page 446, Remark 11.1 for a counterexample of a process which is Markov but doesn't have a transition kernel.
  2. Similarly, can someone provide an example of a process that satisfies Definition 2 but does not satisfy Definition 3 for any transition kernels (possibly different from the ones that satisfy Definition 2)? I suspect such a counterexample to exist but I am not sure.
  3. (IMO THE MOST INTERESTING ONE) If a process satisfies Definition 1 and the space $(E,\mathcal{E})$ is Borel-standard, is Definition 3 automatically satisfied? (We know that Definition 2 is, by the existence of regular versions of conditional expectations). SOME PROGRESS: For a Markov Chain the answer is positive since one can just start from $P_{0,1}$,$P_{1,2}$,... and define $P_{i,j}:=P_{i,i+1}\circ...\circ P_{j-1,j}$.

Some very closely related (unanswered) questions on math.stackexchange are this one and this other one.

$\endgroup$
5
  • $\begingroup$ What do you mean by $X_{t+s}|\mathcal{F}^X_t\sim X_{t+s}|X_t$ in a general state space $(E,\mathcal{E})$? $\endgroup$ Commented Jul 28, 2024 at 17:39
  • 1
    $\begingroup$ @Akira That for all $A\in \mathcal{E}$ it holds that $\mathbb{E} (1_A(X_{t+s})|\mathcal{F}_t^X)= \mathbb{E}(1_A(X_{t+s})|X_t)$ $\,\mathbb{P}$-a.e. $\endgroup$ Commented Jul 28, 2024 at 17:42
  • $\begingroup$ I guess that if $(E,\mathcal{E})$ is nice enough (e.g., Polish space) then Definition 1 is equivalent to Definition 2. You can have a look at Theorem 2.29 in Onno van Gaans' notes Gradient flows in measure spaces. $\endgroup$ Commented Jul 28, 2024 at 17:53
  • 1
    $\begingroup$ @Akira Thank you! However I was aware of that, and that is why in Question 1 I mentioned that a counterexample should consist of a process with values in a non-Borel-standard (and thus in particular non-Polish) space. $\endgroup$ Commented Jul 28, 2024 at 17:57
  • $\begingroup$ I am curious to see a counter-example too. Non-Polish state spaces are "pathological" for probabilists. $\endgroup$ Commented Jul 28, 2024 at 17:59

1 Answer 1

3
$\begingroup$

(To reduce clutter, let $u = t + s \geq t$ and $v = t + s + r \geq u$.)

For Question 3: does Definition 2 imply the Chapman-Kolmogorov equation just by the tower rule? Definition 2 says that for all $t, u$, almost surely $$ P_{t, u}(X_t, A) = \mathbb{E}[1_A(X_u) \mid X_t]. $$ But this should imply that for any bounded $f$, almost surely $$ \mathbb{E}[f(X_u) \mid X_t] = \int_E f(x) \, P_{t, u}(X_t, \mathrm{d}x). $$ (This is the step I'm least confident about—is there a subtlety I'm missing?) From this, we compute $$\begin{aligned} P_{t, v}(A, X_t) &= \mathbb{E}[1_A(X_v) \mid X_t] \\ &= \mathbb{E}[\mathbb{E}[1_A(X_v) \mid X_u] \mid X_t] \\ &= \mathbb{E}[P_{u, v}(X_u, A) \mid X_t] \\ &= \int_E P_{u, v}(x, A) \, P_{t, u}(X_t, \mathrm{d}x) \\ &= (P_{u, v} \circ P_{t, u})(X_t, A). \end{aligned}$$

I think this tells you at least that the kernels $P_{t, u}$ form some semigroupoid with identities $P_{t, t}$ (plug in $t = u < v$ or $t < u = v$). But that semigroupoid might be something like "kernels modulo how they act on parts of $E$ that the process can't reach". So you would need something like "the initial distribution $X_0$ covers all of $E$" to have a hope of saying that Chapman-Kolmogorov is satisfied everywhere.

$\endgroup$
4
  • $\begingroup$ Thank you, but I am aware of the fact that for all $A\in Bor(E)$ and all $t\leq u\leq v$ one has $(P_{u,v}\circ P_{t,u})(x,A)=P_{t,v}(x,A)$ for $\mathbb{P}\circ X_t^{-1}$-almost every $x\in E$. What I'm really wondering is if the kernels can be chosen so that the composition holds everywhere, or at least, if for all $t$ it holds on a set $\Omega_t$ of full measure which is the same for all $A$ and $u,v$. Since $A\in Bor(E)$, which is countably generated if $E$ is Polish, the fact that one can take $\Omega_t$ independent of $A$ should follow. The difficulties lie in the independence of $u,v$. $\endgroup$ Commented Jul 30, 2024 at 9:31
  • $\begingroup$ Ahh, sorry, I missed the everywhere (not merely almost everywhere) subtlety. But I realize now there is an easy counterexample via the "what if you can't reach this part of the state space" idea. Take $E = \mathbb{R}$, set $X_0 = 0$, then have $X_t$ always moving at rate $\pm 1$, switching directions at the increments of a Poisson process. Then the effective state space for time $t$ is $[-t, +t]$, so $P_{t, u}(x, \cdot)$ is unconstrained for $|x| > t$. But maybe there's a way to exclude this concern? $\endgroup$ Commented Jul 30, 2024 at 12:13
  • 1
    $\begingroup$ Yes my concern is if it is always possible to find some kernels such that the composition holds everywhere. In other words, I'm asking if it is always possible to define the kernels in the unconstrained parts in a coherent way. In your specific example it surely is, but I would guess that in general it is not because one would have "uncountably many coherent choices" to make. $\endgroup$ Commented Jul 30, 2024 at 12:38
  • $\begingroup$ Oh of course, my example admits both bad and good kernels, which is fine—never mind :). $\endgroup$ Commented Jul 30, 2024 at 13:30

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.