Question

In: Computer Science

7. Amdahl’s law provides a very useful metric to measure speedup when we make applications parallel...

7. Amdahl’s law provides a very useful metric to measure speedup when we make applications parallel using more resources. However, Amdahl’s law’s basic form makes an assumption about the application that may not hold in real-world scenarios. State the assumption made by Amdahl’s law and describe conditions that may break the assumption.

Solutions

Expert Solution

Let's suppose you have two people search through a telephone book for a particular telephone number, one starting at the front and the second in the middle. Simple reasoning will guess that this is a perfectly parallelizable problem, so should give a speedup of two. The average speedup will be two, given enough trials, but on any one search, the speedup may range from one to almost any number. If the number in question is in the first half of the phone book, then having the second person look in the second half of the book gives no benefit. If the number is the first number in the second half, then the second person will find it immediately, giving an almost infinite speedup for that one search. Amdahl’s Law fails to predict this because it assumes that adding processors won’t reduce the total amount of work that needs to be done, which is reasonable in most cases, but not for search.

The second case is where cache memories come into play. The Law computes speedup as affected by adding more compute resources, that is, more processors. However, processors typically come with other resources that can affect performance, specifically cache memory. In particular, in a strong scaling regime, adding more processors means each processor does less total work. Less total work often means less data to process, and less data means a smaller working set size at each step. A smaller working set will be more likely to fit in the processor cache memory. Fitting a working set into the cache memory is a step function: it fits, or it doesn’t. If it doesn’t, then the data will be fetched from main memory at memory speeds. When you add enough processors that the working set fits into cache, then data will be fetched from cache at cache speeds, which can be ten times faster than main memory.

The Law assumes that the performance improvement from adding processors will be a linear function of the number of processors. However, building larger caches may improve performance for some problems more than by adding more processors, allowing you to get a speedup greater than allowed by The Law. A better performance limit to address this is to change the resource unit you measure. Instead of computing performance or speedup per processor, measure the performance or speedup per transistor or gate.

Conclusion:

The Law has achieved truly prominent importance, capturing in one inequality a strict limit on the achievable speedup from parallelism, when its assumptions are valid. Because of this, it is often quoted and misquoted, used and abused to justify or attack various design decisions. Its original motivation was to highlight that speedup is not necessarily directly related to processor count, so working on faster uniprocessors is still important.

But the assumptions in the Law are no longer valid. Some of the assumptions are that the problem size is fixed, there are no discontinuities in the processor performance curve, the only resource that is varied is the number of processors, parallel overhead is minimal, and the processors are homogeneous. It simply does not capture the performance effects of systems that break these assumptions.


Related Solutions

If we measure computer performance by the “Rmax” metric on the top 500 list, when was...
If we measure computer performance by the “Rmax” metric on the top 500 list, when was the first time that the top 10 computers all performed at 1 petaFLOP/s or higher?
Composites: Properties of a material that make it useful for engineering uses Typical applications for the...
Composites: Properties of a material that make it useful for engineering uses Typical applications for the material A brief discussion of how the material is produced Advantages of the material in its typical uses Disadvantages of the material in its typical uses & how design engineers overcome them
We are developing an online to give to job applications to measure their suitability for a...
We are developing an online to give to job applications to measure their suitability for a sales position and one of the key attributes for a good salesperson is emotional intelligence. We plan to use Trait Emotional Intelligence Questionnaire (TEIQue) to measure emotional intelligence. Explain how and why you would use convergent validity to validity the TEIQue before using it?
describe what Gauss’s Law is about and when is it useful?
describe what Gauss’s Law is about and when is it useful?
3M has developed a composite measure, a 3M™ Value Index Score, which provides a standard metric...
3M has developed a composite measure, a 3M™ Value Index Score, which provides a standard metric to assess accountable value in health care. This index score is a composite measure based on six critical primary care domains derived from 16 measures of key processes and outcomes that effect value in health care. According to 3M, this Value Index Score can increase the understanding of provider and system performance thereby accelerating and prioritizing the areas where improvement is needed. This measure...
We have been discussing the parallel between Newton’s Universal Law of Gravity and Coulomb’s force law:...
We have been discussing the parallel between Newton’s Universal Law of Gravity and Coulomb’s force law: • Both laws define force. Coulomb's law describes the force between electric charges whereas Newton’s law describes the force between masses. • Both are inverse square laws. The forces are proportional to the inverse square of the distance between masses for Newton’s law and the inverse square of the distance between charges for Coulomb’s law. • The forces defined by both laws are central...
The statement of cash flows is very useful because it provides valuable information to investors, creditors,...
The statement of cash flows is very useful because it provides valuable information to investors, creditors, and other users. Distinguish among the three types of activities that are reported in the statement of cash flows. Please discuss in detail. (Please no handwritten responses - they can be difficult to decipher.)
7. What is one problem with the U3 - unemployment rate? Which other metric could we...
7. What is one problem with the U3 - unemployment rate? Which other metric could we look at instead? 8. What would happen to the price of tuition at universities if the demand for college degrees increased and the accreditation process became more costly?
4- Labor Law provides that when the majority of workers want to be represented by a...
4- Labor Law provides that when the majority of workers want to be represented by a labor union, __________________________________________________ replaces “individual bargaining”. 5- If the administration of the labor contract has a dispute, it can be submitted to _______________________________ , which uses a neutral party to settle the dispute. 6- Normally, the court system is the primary institution for enforcing civil law. In contrast, the primary institution for enforcing labor law is the ______________________________________________________________________ . 7- The Labor Act explicitly...
In a Gauss's Law problem where we use a cylinder for the Gaussian surface, when we...
In a Gauss's Law problem where we use a cylinder for the Gaussian surface, when we compute the surface we ignore the end caps of the cylinder and just use 2*pi*r*L as an answer. Explain why we drop out the geometry for the end caps.
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT