Community Discussions:

Communication is one of the most important features of the communities, and the community discussion forums lets you interact with your peers around the world. Ask a question, share insights and experiences, and develop best practices.

Want to take advantage of this great feature? Login or join PMI to gain full access to this community.

Already a Member?

Login
Not a Member Yet?

Join Now
If you have any questions, please contact us at vcpsupport@pmi.org

Active Forums

Minimize
use of Monte Carlo Simulation
Last Post 09 Jan 2011 10:38 AM by David Hillson - The Risk Doctor. 17 Replies.
Printer Friendly
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
David Donaldson New Member New Member Posts:1

--
12 Jul 2010 11:36 AM
    wondering how many people use Monte Carlo simulation on their projects? If not, why not? If you do, how effective do you find it?
    Kirk McKay New Member New Member Posts:1

    --
    22 Jul 2010 03:21 AM
    I am aware of it but would likely only use it where simplier assessment methods would not provide the accuracy of risk assessment required in a high-cost failure situation.
    Karim Kiani New Member New Member Posts:24

    --
    24 Jul 2010 08:06 AM
    We use a tool (for Projects an Excel) which is very known (but I don't wont to name it here) to calculate the probability of milestones, including of course the end date, by using the Monte Carlo simulation. The milestones we commit to, are given in an probability range, so we need to calculate based on the risk we enter into this tool, these probabilities and typical we commit to a 80% probability to the customer, while we internally take another percentage to set as our goal. Our Risk Register and this tools, works hand in hand and therefor it's not a big overhead.
    Agustin Zafra New Member New Member Posts:2

    --
    26 Jul 2010 02:53 PM
    We also use a risk analysis tool and we found that very useful in order to explain the customer what could be the impact if they don’t have there task on time, is very useful also for determined the impact of efficiency diminish with the subcontractor or a variation in there crew, and to determined the risk penalties that is necessary to include in the bugged
    Claude LeComte New Member New Member Posts:2

    --
    26 Jul 2010 03:14 PM
    We use Monte-Carlo simulation on all of our projects during the bid/development phase for all of our clients, to get a feel for the likelihood of milestone achievement. The simulation is done in MS Project using a sophisticated VB program developed by one of our development managers. It's a good sanity check, provided you can come up with realistic estimate ranges for the durations of the activities.
    Anas M.Saleh New Member New Member Posts:1

    --
    26 Jul 2010 11:43 PM
    I am using Monte-Carlo simulation on all of our projects during planning and control phases, very familiar with Primavera Risk Analysis recent version.
    Any one wants any help I am ready.
    John Aucoin New Member New Member Posts:3

    --
    05 Aug 2010 05:28 AM
    We use Primavera Risk Analysis on most of our projects that have high profile milestones. The one drawback was the price tag! Other than that, I think that the software was very well engineered and is a robust tool to use for project risk analysis.
    Lynne Ralston New Member New Member Posts:3

    --
    16 Aug 2010 05:39 AM
    I have used excel with @risk and Microsoft project with risk+ tools to do cost and schedule risk profiling in large multimillion dollar IT projects. In the planning phase this works well not as a control tool, but in getting sponsors to really understand the risk profile they are signing up for. High risk projects asked 90% risked budget approval to proceed vs. lower risk projects at 50-70% - says sponsor is willing to proceed given the chance of potential +1mill on potential risky work because the business case supports still proceeding. This is a great way to effectively get your sponsor onboard and paying attention early in projects that are important to overall business value.
    San Francisco and Wind Country New Member New Member Posts:1

    --
    25 Aug 2010 06:04 AM
    Could someone help me understand how the" excel with @risk and Microsoft project with risk+ tools to do cost and schedule risk profiling " works.

    I have never used any Monte Carlo simulation tools before but I would like to.
    Daniel Amaral New Member New Member Posts:1

    --
    28 Aug 2010 05:23 PM
    Folks,

    I've used both @Risk and Oracle Cristal Ball. Cristal Ball it's really fantastic and you will find plenty of documentation available in the Oracle site with exemples, templates and guides.

    But notice that the quantitative analysis should not add much value if the qualitative analysis were not performed previously. I usually focuse on identification and qualitative analysis first and I won't proceed to the Monte Carlo simulation before evaluating the cost of doing it and whether it worth or not.
    Lynne Ralston New Member New Member Posts:3

    --
    29 Aug 2010 07:01 AM
    For learning more on how @risk and risk+ tools work suggest that you check out their product sites which are very informative for those new to simulations.

    http://www.palisade.com/riskproject/
    http://www.deltek.com/products/riskplus/

    Key as mentioned by others is to spend the time up front in understanding how the project plan work packages need to be organized with good raw input on worst/most likely/best estimates for effort and/or durations to use as the bounded value data for the underlying probability distributions for the simulations.
    Carlos Eduardo Braga New Member New Member Posts:2

    --
    10 Sep 2010 05:56 PM
    I use several tools for quantitative risk assessmnet. It depends of the source. Sometimes only a spreadsheet with the budget. Sometimes a schedule and rarely a schedule with the resources allocated... But anyway, the previous step is to do the qualitative risk assessment. Then create risk estimates considering identified risks.

    All the best,

    Braga
    343636 New Member New Member Posts:1

    --
    30 Sep 2010 04:42 AM

    I am a follower of the Success Driven Project Management methodology, created in Russia and based in a three-scenario approach.

    You may apply SDPM with the assistance of Monte Carlo or even PERT, but the creator of the methodology has also created Spider Project software which replaces the risk analysis of Monte Carlo for its own algorithm with a twist in how to do things: It is better precision to accuracy!

    The SDPM approach starts in a very similar way to Monte Carlo as it will collect three scenarios. However, it is easier to set up as from the original schedule we have (the one we agreed with everyone, so it should be the most probable), we create alternative views (optimistic, with more people, reduced times, less problems, etc and the pessimistic, with more restrictions, lack of money, longer time to finish activities, new events because of problems, etc) and the tool will evaluate the three scenarios creating a fourth view, what we call the critical schedule. The critical schedule is based in the three scenarios with the calculation of probability of success to achieve certain percentage. The closest our probability of success is chosen to be high, the closer the critical schedule is from the pessimistic scenario.

    What is absolutly great about SDPM using the three-scenario approach instead of Monte Carlo is that we solve the problem of SISO (Shit In, Shit Out). This means that even if the information at start is poor, what really matters is not the accuracy of the probabilty curve (which is the general result of Monte Carlo), but the precision of the calculation.

    Imagine that by using the three scenarios, Monte Carlo calculates that I have 90% of probability of finishing on 12/12/12.
    Imagine that by using the triangular calculation of Spider it calculates that I have 85% of probability of finishing on 12/12/12.

    Imagine that we know the future and in fact 90% is the correct answer. This means that Monte Carlo is ACCURATE, Spider is NOT.

    HOWEVER, if you generate MC (Monte Carlo) again, you may have 89% or 91% as a result for the same date because of random seed for the calculation.
    Spider will always reply it is 85%.

    This means that for the first estimate of probability, Monte Carlo is better than Spider... 89% or 91% is more accurate than 85%.

    Then, the project starts and every week you calculate AGAIN the probability of keeping the date of 12/12/12.

    This means that resources have changed, durations may have been better estimated, risks have happened, etc.
    The scenario from day 1 to END is now different from day 7 to END.

    You use MC again and you get 89% of result.
    You use SPIDER and you get 84% of result.

    With MC, you CANNOT be sure if 89% is because you have a tendency of negative result to your project or it is a result of the random difference.

    With Spider, you CAN BE SURE that 84% is a negative trend of achieving the goal because the calculation is made in the exact same form, but with new data.

    So, you operate by TRENDS... If your success probability increases (85, 86, 90, etc), you are sure to deliver your work, when using Spider/Liberzon approach
    (Spider is the tool that generates the calculation formulated by Liberzon, the math genious from Russia that created the Method)

    You may find information of SDPM at the website of one of the founders of PMI, www.russarchibald.com (I have written a paper about SPDM with both Mr. Russell Archibald and Vladimir Liberzon, for the PMI Global of Australia, Mexico and Brazil).

    Other information about Mr. Liberzon can be found at www.spiderproject.ru and www.sdpmworld.com

    Regards,

    Peter Mello, SpS, PMI-SP, PMP
    Member of The Spider Team
    David Hulett New Member New Member Posts:3

    --
    12 Nov 2010 03:15 PM
    Monte Carlo simulation is the best practice recognized internationally to estimate project cost and schedule risk analysis. It is a quantitative approach to estimating how the possibility of individual risks and uncertainties (risks with 100% chance of occurring, such as estimating error) combine to determine the risk to the total project cost and schedule. There are several well-known simulation tools identified earlier in this discussion.
    There have been developments in Monte Carlo simulation methods that are proving to be valuable. The use of the prioritized risks in the Risk Register to drive the cost and schedule risk is such an important development. The key risks (even as few as 25 - 40 strategic risks) can be used to drive the overal risk of very large projects. The method:
    -- Starts with the project schedule for schedule risk, and can start with a resource-loaded scheudle or a spreadsheet for cost risk.
    -- Specifies the risks by both their probability and impact (a range of multiplicative factors, say .90, 1.05, 1.20) if they do occur, agreeing with the notion that risk events are probabilistic events
    -- Assigns the risks to the activities or element costs that they affect. A risk could affect several or many activities (schedule risk) and an activity duration or element cost can be affected by several risks. This way the risk is the fundamental driving force so we can prioritize risks, not activities or paths.
    -- In each iteration the risk either does or does not happen depending on its probability. If it happens, a multiplicative factor is chosen from the distribution of multiplicative facotrs and is applied to the duration of the activigties to which it is assigned. This is the way risks can be used to affect the durations of activities in the schedule.
    -- Normal results such as histograms and cumulative distributions are shown.
    -- Not done yet, because we want to identify the high-priority risks to schedule (and to cost, see next). Do that by taking the risks out (probabiltiy = 0.0) thus cancelling its effect on all activities to which it is assigned. The most important risk is the one that, when it is taken out, shows the earliest date at teh P-80.
    -- Risks prioritized in this way can lead to focused risk mitigation exercises.

    This is called the Risk Driver method. At present I am using Primavera Risk Analysis to implement this on schedules.
    David Hulett New Member New Member Posts:3

    --
    12 Nov 2010 03:28 PM
    Another major development is the integration of cost and schedule risk analysis. The platform is the resource loaded and costed project schedule. The method is Monte Carlo simulation.
    -- The resources need to be designated as time-dependent such as labor, rented equipment and LOE, or time-independent such as procured equipment and raw materials.
    -- The resources can be summary in nature. I have done this with as few as 8 - 10 resources even on large projects. These resources are not detailed enough to do resource planning and leveling, but they serve to put the project budget on the right activiteis and to designate them as time dependent or time independent.
    -- Of course more detailed resources can be used if available. Their costs should be free of contingency padding since the cost contingency reserve is the result of the Monte Carlo simulation.
    -- If the resources' costs are time dependent, then their costs will go up or down with the uncertainty in the duration of the activities they support.
    -- In addition, for time dependent resources there is uncertainty in the burn rate, so their cost may be uncertain even if the schedule is static.
    -- There is also uncertainty in the time-independent resources, although their cost is not affected by scheudler risk.
    -- Again, using Risk Drivers the risks can be assigned to multiple resource-loaded activities. Doing this generates correlation between activity durations. Correlation is a very common fact of project risk, and generating the correlation by modeling the way risks cause correlation ensures an internally-consistent correlation matrix.
    -- Results include the standard histograms for both cost and schedule. Other results include scatter diagrams of time and cost so the correlation of time and cost can be observed. NASA uses these scatter diagrams for their Joint time-cost Confidence Level (JCL) calculations.
    -- In addition, a probabilistic cash flow, say at the P-80 for every month, is generated and can be compared to the monthly funding availabiltiy.

    More accurate and understandable results for cost risk occur when the full impact of schedule risk is modeled. We often find that some of the main risks to cost are risks to scheudle since some of the cost contingency reserve should be held against uncertain project duration.

    The main method, the best practice standard, is Monte Carlo simulation.
    Ronald PC Waller New Member New Member Posts:1

    --
    18 Nov 2010 07:42 AM
    Others have covered much of the benefits of MCS, but the most important one is as follows:

    Those charged with evaluating financial elements (CEO, CFO, Controller etc.) prefer a single number for outcome estimates. Deciding what to use in such an estimate is problematic. By using MCS to estimate costs to be incurred, change orders to be received etc. you will find that yoou can put in ranges and probabilities, and sum thes to a single number that approximates the project outcome. I like to send an estimate in that includes a cummulative probability chart of the summation with s suggestion to pick what probability is desired for the outcome with the attendant project outcome shown (I.E. pick the probability you want. follow the chart to the corresponding outcome. Using this mehtod allows the PM to convey the estimate along with a risk factor.
    Edmund Conrow New Member New Member Posts:22

    --
    20 Nov 2010 12:55 PM
    My background in Monte Carlo (MC) simulations is that I’ve been developing and running them for more than 30 years (first MC publication in 1978 based upon results from a multi-thousand SLOC simulation I wrote) across numerous industries and projects. Per the PMBOK perspective, MC simulations are used for estimating probabilistic cost and schedule, which is certainly valid. However, in the real world, almost all MC simulations developed and run are used for modeling the (technical) performance dimension-everything from evaluating nuclear weapons characteristics and effects (one of the first, if not the first MC simulations performed), complex integrated circuits (ICs), air flow and interaction over an aircraft wing, weather forecasting, chemical interactions, financial modeling (which is different than cost estimating), etc. If the measure is based on computer CPU hours, then performance-based MC simulations are probably “run” a million to a billion times more than cost and schedule simulations combined. For example, on one complex IC project I helped manage we had to simulate timing interactions within the IC before committing the part to fabrication. We spent more than a million equivalent desktop PC CPU hours modeling the timing interactions using many high-powered dual processor workstations running MC simulations with optimized code, etc. Although the PMBOK doesn’t recognize the (technical) performance dimension (scope is not an adequate proxy for technical performance), the simulations run by high-end multi-processor workstations and supercomputers are largely performance related, rather than associated with cost and schedule. (Also note that joint cost and schedule simulations often do not properly account for the performance dimension, and the interaction of the three dimensions is typically complex, nonlinear, and varies with time.)

    An overlooked area is that MC simulations are sometimes applied to models that have inherent (underlying) structure errors. This is very common with schedule risk analyses and can render the results meaningless. I haven’t worked with a single project schedule for many years whose underlying inherent structure (e.g., logic, constraints) was error free. (For example, in one case representing the sixth generation of an item the delivery milestones were disconnected from the underlying schedule logic.) In most cases, such as the one just mentioned these errors would have led to inaccurate MC simulation results. Sometimes the errors have been as simple as the project manager directing that a “finish no later than” constraint be placed on a critical delivery date. The net effect of this action is that no matter how late the development or production of an item will be the deterministic and stochastic (simulated) schedule will show that the item’s delivery date will be met (and typically on-time). In other cases the errors are more subtle and/or complex and can be difficult to diagnose. I strongly recommend that you invest appropriate resources to debug and verify each and every model (whether cost, performance, or schedule) before you develop probability distributions and run the simulation. Otherwise, your results can be both difficult to verify, and in the worst case meaningless.

    Another overlooked areas in developing the simulation is the selection of probability distributions and their critical values (e.g., mean and standard deviation for a normal distribution). The following two widely referenced books should help you in this matter:

    Law, A. M., “Simulation Modeling and Analysis,” Fourth Edition, McGraw–Hill, New York, 2007. (Contains unique and helpful insights into statistical distribution fitting, MC simulations, etc.)
    Evans, M. Hastings, N. and Peacock, B. Statistical Distributions, Third Edition, Wiley-Interscience, New York, 2000. (Great book—I’ve used it since the First Edition in 1974.)

    Regards,
    Dr. Edmund (Ed) H. Conrow, CMC, CRM, PMP
    INCOSE Fellow (Risk Management, and Cost, Performance, Schedule Trade Methods)
    www.risk-services.com
    David Hillson - The Risk Doctor Basic Member Basic Member Posts:143

    --
    09 Jan 2011 10:38 AM
    David Hulett is too modest to mention his own recent book on the topic, so I will. I recommend:
    Hulett D. T. 2009. “Practical schedule risk analysis”. Farnham, UK: Gower. ISBN 978-0-566-08790-5
    David is also writing a second book on integrated cost/schedule risk analysis.

    David Vose has also written a major book on hth topic:
    Vose D. 2008. “Risk Analysis – A Quantitative Guide” (third edition). Chichester, UK: J Wiley. ISBN 978-0-470-51284-5

    I'm also not sure anyone has answered the original question "how many people use Monte Carlo simulation on their projects?" In my experience working in over 40 countries in most industry sectors, it seems to me that about 10% of all projects worldwide use this technique. Of course it is more common in some sectors (defence, energy, aerospace) or for some clients.
    You are not authorized to post a reply.