| 1. |
|
|
A Markov system can be in several states at one time. |
| 2. |
|
|
The (1,3) entry in the transition matrix is the probability of going from state 1 to state 3 in two steps. |
| 3. |
|
|
The (6,5) entry in the transition matrix is the probability of going from state 6 to state 5 in one step. |
| 4. |
|
|
The entries in each row of the transition matrix add to zero. |
| 5. |
|
|
To find the probability of going from state 3 to state 5 in six steps, we take the sixth power of the (3,5) entry in P. |
| 6. |
|
|
To find the probability of going from state 3 to state 5 in six steps, we take the (3,5) entry in P6. |
| 7. |
|
|
Given the initial distribution vector [10, 23, 77, 1] and a 4 4 transition matrix P, the distribution 4 steps later is given by [10, 23, 77, 1]P4. |
| 8. |
|
|
Given the initial distribution vector [10, 23, 77, 1] and a 44 transition matrix P, the distribution 4 steps later is given by [10, 23, 77, 1]P.P.P.P. |
| 9. |
|
|
The entries in a distribution vector add to the same number before and after multiplication on the right by P. |
| 10. |
|
|
The entries in a steady state distribution vector of a regular Markov system give the fractions of time the system spends in each of the states. |
| 11. |
|
|
The rows in the steady state transition matrix Pinfinity give the fractions of time the system spends in each of the states. |
| 12. |
|
|
Every system with one or more absorbing states is an absorbing system. |
| 13. |
|
|
No regular system can be absorbing. |
| 14. |
|
|
No absorbing system can be regular. |
| 15. |
|
|
In an absorbing system, if the (2,3) entry of the fundamental matrix is 4, this means that, starting in state 2, you can expect the system to be in state 3 four times prior to absorption. |
| 16. |
|
|
In an absorbing system, if the (3,3) entry of the fundamental matrix is 5, this means that, starting in state 3, you can expect the system to be in state 3 five more times prior to absorption. |
| 17. |
|
|
In an absorbing system, if the (3,3) entry of the fundamental matrix is 5, this means that, starting in state 3, you can expect the system to be in state 3 four more times prior to absorption. |
| 18. |
|
|
The sum of the entries in row number 4 of the fundamental matrix is the total number of time steps you expect the system to be in state 4 prior to absorption. |
| 19. |
|
|
The sum of the entries in column number 4 of the fundamental matrix is the total number of time steps you expect the system to be in state 4 prior to absorption. |
| 20. |
|
|
The sum of the entries in row number 4 of the fundamental matrix is the total number of steps you expect the system to take until absorption if it starts in state 4. |
