Modelling, Analysis and Design of Stochastic Systems

A system is characterized as “stochastic” when its evolution in time is arbitrary. To deal with such systems the first step is to understand how it operates and the purpose of studying it, in order to be able to build a model that is simple yet sufficiently true to the real. The second steps consists in carefully analyzing the model and compute the desired measures. To facilitate this special classes of stochastic processes are used, like discrete-time Markov chains, Poisson processes and continuous-time Markov chains. For each of these processes, the transient distributions, limiting distributions and cost evaluations are studied. The main part of the course deals with a particular class of stochastic system called queuing systems which is commonly used to model systems’ behavior in production and service providing systems. Typically, a queuing system consists of a stream of customers that arrive at a service facility, get served according to a given service discipline and then depart. We are interested in designing a queuing system that will help us answering questions like “How many customers are there in the queue on average?”, “How long does a typical customer spend in the queue?”, “How many customers are rejected or lost due to capacity limitations?”. Finally, an introduction to Markov Decision process is provided.

Module Contents (Syllabus):

Week 1. Probability Models Review, Introduction to Stochastic Process and Queuing Systems

Week 2. Introduction to stochastic modeling, Discrete Time Markov Chains (DTMC),Continuous Time Markov Chains (CTMC)

Week 3. Characteristics of queuing systems, PASTA, Little’s Law

Week 4. The Μ/Μ/1/1 model and the Μ/Μ/1 model

Week 5. The Μ/Μ/1 model- Exercises

Week 6. Other  Μ/Μ/1 models

Week 7. The Μ/Μ/k model - Exercises

Week 8. The Μ/Μ/1/k model - Exercises

Week 9.The Μ/Μ/k/k - Exercises

Week 10. The Μ/Μ/1/k/k and M/M/s/k/k models - Exercises

Week 11. The  M/M/inf model-Retrial model - Exercises

Week 12. Expected Cost

Week 13. Expected Cost- Exercises

Module Objective:

The aim of the course is to provide the students the capability of modeling, analysis and design of systems the evolution of which is arbitrary. To this direction the course provides the appropriate background for understanding the behavior of a real world system and modeling its evolution using stochastic processes such as Markov processes. The course mainly focuses on queuing systems and their application is production and service systems