Downloaded 16 times
![Reference for today’s lecture
Chapter 1 [Raj Jain]
Find more lecture resources at the course‟s companion site:
http://sites.google.com/site/netperfeval](https://image.slidesharecdn.com/lecture1courseintroduction-120806233539-phpapp02/75/Lecture-1-course-introduction-29-2048.jpg)
![[ end of lecture ]](https://image.slidesharecdn.com/lecture1courseintroduction-120806233539-phpapp02/75/Lecture-1-course-introduction-31-2048.jpg)
![Reference for today’s lecture
Chapter 1 [Raj Jain]
Find more lecture resources at the course‟s companion site:
http://sites.google.com/site/netperfeval](https://crownmelresort.com/image.slidesharecdn.com/lecture1courseintroduction-120806233539-phpapp02/75/Lecture-1-course-introduction-29-2048.jpg)
![[ end of lecture ]](https://crownmelresort.com/image.slidesharecdn.com/lecture1courseintroduction-120806233539-phpapp02/75/Lecture-1-course-introduction-31-2048.jpg)
Uploaded byJunaid Qadir
Lecture 1, course introduction
The document outlines the structure and content of a course on Network Performance Evaluation (NPE) led by Junaid Qadir, Ph.D. It covers essential topics such as the importance of NPE, methodologies for empirical/experimental and simulation-based evaluations, and provides a detailed course outline including lecture modules. The document emphasizes the role of statistical analysis and modeling in improving network performance.
More Related Content
![9789513932527[1]](https://cdn.slidesharecdn.com/ss_thumbnails/4b39227e-4227-46a8-9f39-4011b7e685c5-150407145007-conversion-gate01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Lecture 1, course introduction
Lecture 1, course introduction
- 1. Network Performance &Evaluation (NPE) Junaid Qadir, Ph.D.
- 2. Today’s agenda: 1 What is Network Performance Evaluation (NPE)? 2 About this course: contents and resources
- 3. 1 What is NPE? ˉ Importance of Computer Networks ˉ Importance of NPE ˉ What does performance mean?
- 4. NPE (design, analysisand evaluation) 1 For a given packet arrival pattern and desired performance (e.g., low packet drops), what should be the size of router buffers? (design question) 2 What is the quantitative improvement in performance of the router after the latest software (or hardware) upgrade? (analysis/ evaluation question)
- 5. Goals of NPE 1 Comparison of design alternatives 2 System dimensioning 3 Relationship (causal/ correlational) modeling 4 Performance description in appropriate metrics.
- 6. 2 About this course
- 7. Course Outline 45 lectures
- 8. Today’s lecture Introduction tothis course (Lecture 1)
- 9. Module 1/3 The Scienceand Art of NPE Part A 12 lectures Lecture 2-13
- 10. The Science andArt of NPE Part A 12 lectures 12 lectures Lecture 2-13 Statistical The Science of background The Art of NPE NPE (grammar of science)
- 11. The Science andArt of NPE Part A 12 lectures The Science of NPE "The fundamental principle of science, the definition almost, is this: the sole test of the validity of any idea is experiment." - Richard Feynman. 'In questions of science the authority of a thousand is not worth the humble reasoning of a single individual.' Galileo Galilei.
- 12. The Science andArt of NPE Part A 12 lectures Statistical background It is better to be approximately right than be exactly wrong – John Tukey
- 13. The Science andArt of NPE Part A 12 lectures The Art of NPE “All models are wrong; some are useful." – George Box. "Excellence in statistical graphics consists of complex ideas communicated with clarity, precision, and efficiency." – Edward Tufte.
- 14. Module 2/3 Empirical/ Experimental NPE Part B 13 lectures Lecture 2-13 Lecture 14-26
- 15. Empirical/ Experimental NPE Empirical/ Part B experimental NPE 13 lectures Lecture 14-26 Workload Design of Internet Modeling Experiments Measurement
- 16. Empirical/ Experimental NPE PartB 13 lectures Workload Modeling "The essence of modeling, as opposed to just observing and recording, is one of abstraction. This means two things: generalization and simplification." - Dror Feitelson.
- 17. Empirical/ Experimental NPE Part B 13 lectures Design of Experiments Does the „magic tablet‟ work as advertised or is it just a gimmick? Hypothesis testing is at the very core of the scientific method.
- 18. Empirical/ Experimental NPE PartB 13 lectures Internet Measurement "Measure what is measurable, and make measurable what is not so." - Galileo Galilei
- 19. Module 3/3 Simulation/ Modeling based NPE Part C 18 lectures Lecture 2-13 Lecture 14-26 Lecture 27-44
- 20. Simulation/ Modeling basedNPE Part C 18 lectures Lecture 27-44 Probability and Stochastic Analytical Simulation Processes Modeling Modeling Preliminaries
- 21. Simulation/ Modeling basedNPE Part C 18 lectures Stochastic Background PROBABILITY RANDOM VARIABLE’S VALUE
- 22. Simulation/ Modeling basedNPE Part C 18 lectures Analytical Modeling SERVER PACKET QUEUE ROUTER "The essence of modeling is one of abstraction. This means two things: generalization and simplification."
- 23. Simulation/ Modeling basedNPE Part C 18 lectures Simulation Modeling Computer based simulation modeling can be performed when experimenting with real system is not practical or efficient and when simpler analytical methods do not exist
- 24. Course conclusion Summary of this course (Lecture 45) Lecture 2-13 Lecture 14-26 Lecture 27-44
- 25.
- 26.
- 27. Course companion site http://vulms.vu.edu.pk
- 28.
- 29. Reference for today’slecture Chapter 1 [Raj Jain] Find more lecture resources at the course‟s companion site: http://sites.google.com/site/netperfeval
- 30. Credits/ Acknowledgments Books Raj Jain,“The Art of Computer Systems Performance Analysis” Figures and Images: Car Crash: http://www.pclaunches.com/other_stuff/audi_uses_supercomputer_for_crash_simulations.php Galileo‟s Portrait: http://en.wikipedia.org/wiki/File:Justus_Sustermans_-_Portrait_of_Galileo_Galilei,_1636.jpg George Box‟s Picture: http://gallery.socionix.com/d/49618-2/1990_George_Box.jpg Edward Tufte‟s Picture: http://www.freebase.com/view/en/envisioning_information VU LMS: http://vulms.vu.edu.pk Clipart supplied with Microsoft Office These resources have been used in these lecture slides for educational purpose under the fair use doctrine. The ownership of these resources, if copyrighted, is retained by their respective copyright owners.
- 31. [ end oflecture ]
Editor's Notes
- #2 Welcome to the first lecture of this new course offered by Virtual University titled: 'Network Performance and Evaluation’ being offered by Virtual University. I’m the instructor for this course and my name is JunaidQadir.Firstly, a few things about myself.I’mJunaidQadir, currently an Assistant Professor at SEECS, NUST. I received my PhD in Computer Engineering in May 2008 from the University of New South Wales (UNSW), Sydney, Australia where I worked on the research topic of "Improving broadcast performance in multi-radio multi-channel multi-rate wireless mesh networks"'.Currently, I am the director of the Cognet lab which operates under the PTCL, Cisco, NUST IP technology center of excellence situated at SEECS, NUST.Before my PhD studies, I have worked in the networking industry for around 4 years where I gained hands-on experience working on the largest data networks in the country. Amongst other places, I have worked at PTCL’s Pakistan Internet Exchange (PIE) and at the national data networks of National Telecommunication Corporation (NTC). I completed my bachelor’s degree in electrical engineering from University of Engineering and Technology (UET), Lahore, Pakistan in 2000.
- #3 Today, I want to talk about two things. Firstly, I want to introduce you to the notion of NPE and show you how fundamental it is to the efficient operation of computer and communication networks.Secondly, I want to talk about this particular course: about myself, my pedagogical approach to teaching this course, the outline of this course, and the relevant resources such as the course textbook, the course guidebook, and the course websites.
- #4 So we begin with what is NPE:Importance of NPE and its basic tasks:With computer networks playing such a central role in modern life, it is imperative that: 1) we design high-performance reliable networks and 2) we have the ability to analyze and evaluation the performance of computer networks.NPE, like probability, can often contradict intuition and therefore it would be foolhardy to only rely on common-sense and gut feeling for evaluating network performance. As an example of NPE results contradicting common-sense consider the Braess' Paradox1 which states that adding extra capacity to a network when the moving entities selfishly choose their route, can in some cases reduce overall performance. This contradicts the conventional wisdom which implies that the solution to all network problems is to increase capacity but this paradox is indicating that it may not always be productive to do so. Since we cannot rely on intuition and gut feeling only for NPE, this motivates us to develop formal tools to facilitate analysis and evaluation. NPE is also workload and metric specific, changing which can lead to dramatically different results. 1 http://en.wikipedia.org/wiki/Braess%27s_paradox
- #5 There are two broad categories of NPE questions: design questions, and analysis or evaluation questions.
- #6 NPE is about quantifying the service delivered by a computer or communication system. The goal of a performance evaluation may either be:1) a comparison of design alternatives, i.e. quantify the improvement brought by a design optionComparison of designs requires a well-defined load model: e.g., we might be interested in: comparing the power consumption of several server farm configurations; however, the exact value of its intensity does not have to be identified. 2) system dimensioning, i.e. determine the size of all system components for a given planned utilization. With a performance goal in mind (e.g., a service level agreement, SLA, of 99.999% uptime), what infrastructure is needed? System dimensioning requires a detailed estimation of the load intensity. Like any prediction exercise, this is very hazardous and liable to going very wrong.3) determining causal relationship model between design parameter and performance metric of choice4) describing the performance of a system in terms on performance metrics.
- #10 In the first module of this course, we are going to learn about the philosophy of NPE and discover that it is both a work of art and a work of science.
- #12 Scientific method is a very important method for gaining knowledge in which we focus on building knowledge through repeatable and controlled (empirical) observations. It is a useful method for NPE since it minimizes the effect of biases and forms conclusion only on the basis of experimental evidence.
- #13 Statistics is an integral tool that we use to deal with uncertainty. It helps us to quantify uncertainty and to make informed decisions in the face of uncertain circumstances. It is the grammar of science and is used by all the physical and social sciences since all measurements are beset by errors and it’s necessary for any meaningful conclusions from such data that we are able to quantify the amount of error (not in the sense of a blunder but in the sense of natural unavoidable uncertainty) To read more about the example of the man with two watches, refer to ‘Statistical Modeling: A Fresh Approach’ by Daniel Kaplan.
- #14 The model is the most basic element of the scientific method. Everything done in science is done with models. A model is any simplification, substitute or stand-in for what you are actually studying or trying to predict. Models are used because they are convenient substitutes that facilitates experimentation, analysis or observation. We've all heard about hypotheses and theories, especially in physics and chemistry. Theories usually comprise some idea that scientists have about how nature works, but that they aren't totally sure. Hypotheses and theories are merely particular kinds of models that we will refer to below as abstract models. The scientific method is a procedure for the construction and verification of models. We note here that science progresses by continuously trading current models for better models whenever they may become available and there's no right answer at which we would stop our search for better models.It must be remembered always that all models are only approximations of reality. They are used since they are useful for learning about the system and how it will behave if the system parameters are changed. The role of a NPE model can be compared to a city map. A city map has two key properties. It is an abstract representation of the real situation in that the distances between any two points on the map are not in geographical proportion, and it is simple because it is uncluttered by unimportant real-world physical details. The natural urge is to create a NPE "map” adorned with an abundance of physical detail because that would seem to constitute a more faithful representation of the computer system being analyzed. In spite of this urge, you should strive instead to make your NPE models as simple and abstract as a city map. Adding complexity does not guarantee accuracy. (Adapted from Neil Gunther’s book: ‘Analyzing Computer System Performance with Perl::PDQ’.)Unfortunately, there is no simple recipe for constructing a city map (or a NPE model). Einstein reputedly said that things should be as simple as possible, but no simpler. That should certainly be the goal for applying NPE, but like drawing any good map there are aspects that remain more in the realm of art than science.
- #15 In the second module of this course, we are going to learn about the use of empirical methods (experiment based) to perform NPE.
- #17 To perform NPE of any system, it is important that we know what input load would be applied to such a system. All performance evaluation is contingent on what workload is applied to the system, and it is possible (and is usually the case) that performance conclusions be reversed if we make different set of assumptions about what the workload would be like. We will see in the subpart 1 of the module on empirical methods (Part B of this course) that workload modeling is essentially an abstraction (or simplification) process in which we generalize the actual workload by its parameterized model so that we can perform evaluation on the system by tweaking the workload model and observing how the performance varies in response. "The essence of modeling, as opposed to just observing and recording, is one of abstraction. This means two things: generalization and simplification." - DrorFeitelson.The following is an excerpt from DrorFeitelson’s book1 on Workload Modeling:An important property of good models is simplicity. A good model doesn’t just define new useful quantities—it also leaves out many useless ones. The act of modeling distills the cumulative experience gained from performing experimental measurements, and sets then in a format that can be used as the basis for further progress [14]. In fact, this is also the basis for natural science, where experimental observations are summarized in simple laws that are actually a model of hownature operates.It should be stressed that finding new models is not easy. The largest obstacle is actually noticing that a new model is needed. It is very tempting to interpret experimental results in the light of prevailing preconceptions. This runs the risk of fitting the measurements to the theory, rather than the theory to the measurements.NPE analysts use models for various tasks: describing a variable (lets say the input to the system under consideration which is not known apriori exactly but known statistically) or to describe a process or system under study (a model may be more amenable to analysis or experimental manipulation). Both approximation facilitates (and in some cases makes possible) efficient analysis and evaluation. 1http://www.cs.huji.ac.il/~feit/wlmod/
- #18 Design of experiments uses hypothesis testing to ascertain the validity of a claim (or a hypothesis) which explains some observable phenomena. We seek to differentiate real effects from coincidental or chance effects which may also produce the same result (due to chance only).
- #19 An important part of the empirical method is measuring and learning how to measure.
- #20 In the thirdmodule of this course, we are going to learn about the use of modeling and simulation to perform NPE.
- #22 We will focus on the necessary stochastic and probability background that we shall be using for the remainder of this course.
- #23 "One of life's more disagreeable activities, namely, waiting in line, is the delightful subject of this book. One might reasonably ask, "What does it profit study such unpleasant phenomena?" from preface to Queueing Systems, Vol. 1, Leonard Kleinrock.In analytical NPE, we model the system under consideration using mathematical symbols that are mathematically related e.g, the system may be represented as a set of equations or inequalities. a sequence on sourcing statistics. Analytical models are typically mathematical models for which closed-form solutions exist and are one of the quickest methods of NPE. However, they are applicable to only a subclass of NPE systems. In this course, we will learn the utility of mathematical modeling. Mathematical models govern our economy and help forecast our weather. They predict who will win the election and decide whether your bank loan application should be granted. However, the man on the street knows little about what mathematical models are and how they work. In this course, we will use NPE to explain how mathematical models are designed, built, and validated.The most important framework that we shall study in this context would be queueing theory. It is amazing how much we can model using the plain concept of a queue of customers waiting for service from a server. We can model a router or a switch as a queue in which packets line up in a queue to obtain switching service: using queueing theory, we can obtain average case statistics like how much time does a packet wait on average in the system, at any given time, how many packets are queued in the router buffers waiting for service?
- #24 In many cases, real life systems are too complex and may not be even approximated closely by analytical models. In such cases, we usually recourse to simulation methods in which we build computational models (or computer program models) of the real system that we would like to model. Thereafter, we can analyze the system by executing the computation model (or program) and make conclusions on the basis of such executions. Simulation modeling is a very general technique and is used much in NPE studies especially for evaluation of algorithms, and systems of computer networks.
- #29 In this course, our aim is to present performance evaluation in an accessible way which can be understood and implemented by all computer engineers and scientists. The foundations of performance evaluation are in statistics and queuing theory, therefore, this course traditionally involves a lot of mathematics. However, I will aim to present ideas intuitively and in the context of concrete applications. I will also post links to resources that you can use to further develop your intuition. We will see in this course that we can often use computer simulations to develop intuition about intractable problems that can be solved by analytical models. Similarly, I would encourage you to explore active demo material including applets which will be linked to in the companion site of this course. You would learn that you can gain a lot of intuition by practicing through these demos and applets.