How Traditional Risk Reporting Has Let Us Down

Posted by Megan Cacioppo on November 4, 2016

How Traditional Risk Reporting Has Let Us Down Header Image

As project management software tools have evolved, project risk analysis has gained a reputation for being overly complicated and disconnected from the real-world of successful project execution.

A typical risk workshop goes something like this… “Do we really have to sit through this in order to determine we are not going to finish on time?” or, “We’ll participate, but this is just a theoretical model—the real world does not work against P50 dates.” The truth is, risk analysis can deliver true value and insight. It all comes down to an accurate interpretation of the results, and an overall shift in your project team’s perception.

What is Project Risk Analysis Anyways?

Project risk analysis is based on the simple premise of accounting for the uncertainty and discrete risk events that have not yet been taken into account in the base project plan. Rates, quantities, scope, and manpower are all examples of variables that can impact the realism of even the best-laid plan. Add to the mix the potential presence of risk events (threats or opportunities) and you start to wonder why so much emphasis is given in generating a CPM schedule that inevitably turns out to be wrong! By capturing these uncertainties and risk events through simulation (such as Monte Carlo) organizations can better predict ranges of project costs and durations as well as gain confidence in the likelihood of achieving the established plan.

The Trouble with Traditional Risk Reporting

Today there are numerous types of risk reports: risk histograms, risk tornadoes, scatter charts, sensitivity diagrams, fish-bone diagrams, and many more. The trouble is when used individually or inaccurately, these reports fail to give project teams a holistic picture.

Take for example, a risk histogram. This type of report shows a distribution of results from a Monte Carlo simulation, and can be overlaid with the cumulative set of results (in an ordered list) to report P-dates and P-costs. The histogram can also be used to report confidence level, or the probability of achieving a given date. But the truth is, from this single metric, there is no way to know if your schedule is risky. Confidence level reporting is a widely misused and skewed metric for reporting risk.

The risk tornado chart, on the other hand, is used to report key risk drivers. Historically focus has been given to metrics such as criticality, which reports how many times an activity falls on the critical path. The idea is that the more times the activity falls on the critical path, the more often it is going to be a risk driver. The drawback of looking at just criticality, however, is that while it reports frequency of being on the critical path, it does not give any indication of the size or degree of impact on the critical path or finish date of the project. An activity can have a high criticality but only have a couple of days’ impact on the project. This is not as concerning as an activity that only falls on the critical path 30% of the time, but when it does has a six-month impact on the project finish date.

New Methods of Risk Reporting

Today, the Risk Range Factor™ is a much more meaningful way to measure risk exposure than the traditional confidence level from a risk histogram. The risk range factor is a metric that shows the degree of uncertainty relative to the amount of remaining duration. Take for example, a project where the total risk range (P100 - P0) is 69 days. Compare that to the remaining duration (655 days) and the Risk Range Factor™ is 11% - meaning the remaining variability in completion date represents only 11% of the total remaining duration of the project.

Combine that with a Schedule Contribution Factor™ and you start to get a holistic, more meaningful picture. The Schedule Contribution Factor reports risk drivers in true cost and schedule terms and differentiates between uncertainty and risk events in terms of contribution. It also takes the biggest risk drivers in a schedule and reports contribution to risk in terms of duration. Further, it separates contribution from uncertainty, and contribution from risk events in order to clarify whether it is the activity scope/certainty or indeed a risk event impacting the activity that causes it to become a key risk driver.

The bottom line is that project risk analysis is indisputably valuable as long as the information generated helps the project team understand the risk exposure and helps them to pinpoint and reduce the key drivers.

Using the likes of risk histograms for reporting context-based metrics such as Risk Range Factor™ helps give a project team a true sense of exposure. Combine this with metrics such as the Schedule Contribution Factor™ and for the first time, you can truly understand not only what activities and risks are causing the risk exposure, but also how much of an impact they are having.


Expert Q&A: What Matters in a Risk Workshop

Learn helpful tips for Risk Workshops you can put to work with your project teams, as explained by Dr. Dan Patterson.

Download Now »