Cover; Introduction to Stochastic Models; Title Page; Copyright Page; Table of Contents; Preface; Chapter 1. Introduction to Stochastic Processes; 1.1. Sequences of random variables; 1.2. The notion of stochastic process; 1.3. Martingales; 1.3.1. Stopping time; 1.3.2. Discrete-time martingales; 1.3.3. Martingale convergence; 1.3.4. Square integrable martingales; 1.4. Markov chains; 1.4.1. Markov property; 1.4.2. Transition function; 1.4.3. Strong Markov property; 1.5. State classification; 1.5.1. Stationary probability; 1.6. Continuous-time Markov processes; 1.6.1. Transition function
1.6.2. Kolmogorov equations1.7. Semi-Markov processes; 1.7.1. Markov renewal processes; 1.7.2. Semi-Markov processes; Chapter 2. Simple Stochastic Models; 2.1. Urn models; 2.2. Random walks; 2.3. Brownian motion; 2.3.1. Introduction; 2.3.2. Basic properties; 2.4. Poisson processes; 2.5. Birth and death processes; Chapter 3. Elements of Markov Modeling; 3.1. Markov models: ideas, history, applications; 3.2. The discrete-time Ehrenfest model; 3.2.1. The microscopic chain; 3.2.2. The macroscopic chain; 3.2.3. Some characteristics of the Ehrenfest model
3.2.4. The discrete-time Ehrenfest model: history, generalizations, similar models3.3. Markov models in genetics; 3.3.1. Laws of heredity and mathematics; 3.3.2. Haploid models; 3.3.3. Models with two genotypes and without mutations; 3.3.4. Models with several genotypes and without mutations; 3.3.5. Models with two genotypes and mutations; 3.3.6. Models with several genotypes and mutations; 3.3.7. Models with partitioned population; 3.3.8. Genealogy models for large size populations; 3.4. Markov storage models; 3.4.1. Discrete-time models; 3.4.2. Continuous-time models
3.4.3. A generalized storage model3.5. Reliability of Markov models; 3.5.1. Introduction to reliability; 3.5.2. Some classes of survival distributions; 3.5.3. Discrete-time models; 3.5.4. Continuous-time models; Chapter 4. Renewal Models; 4.1. Fundamental concepts and examples; 4.2. Waiting times; 4.3. Modified renewal processes; 4.4. Replacement models; 4.5. Renewal reward processes; 4.6. The risk problem of an insurance company; 4.7. Counter models; 4.7.1. Type I counters; 4.7.2. Type II counters; 4.8. Alternating renewal processes; 4.9. Superposition of renewal processes
4.10. Regenerative processesChapter 5. Semi-Markov Models; 5.1. Introduction; 5.2. Markov renewal processes; 5.2.1. Definitions; 5.2.2. Markov renewal theory; 5.3. First-passage times and state classification; 5.3.1. Stationary distribution and asymptotic results; 5.4. Reliability; 5.5. Reservoir models; 5.5.1. Model I; 5.5.2. Model II; 5.6. Queues; 5.6.1. The G/M/1 queue; 5.6.2. The M/G/1 queue; 5.7. Digital communication channels; Chapter 6. Branching Models; 6.1. The Bienaymé-Galton-Watson model; 6.1.1. Historical considerations; 6.1.2. Some elementary results; 6.1.3. A fundamental example