Three Solutions for Establishing Useful Probability Distributions

Establishing useful probability distributions can be interactive and entertaining.

Monte Carlo simulations are the best approach for assessing uncertainty for future infrastructure and facilities funding requirements. However, developing (or choosing) a probability distribution is a major factor in most technical professionals’ decisions not to use this approach. Here are three solutions for establishing a useful probability distribution.

Probability Distributions: Definitions That Confuse

A probability density function provides a relative likelihood that a given value would equal a given point in the sample space. The probability density function (PDF) is applied to continuous random variables, whereas the probability distribution function is defined for discrete random variables. For discussion purposes, these are often interchangeably called probability distributions and PDFs.

The probability density function (PDF) is used to specify the probability of the random variable equaling a value in the sample space, whereas the cumulative distribution function (CDF) is the probability that the variable takes a value less than or equal to a value in the sample space. The PDF and the CDF are non-negative everywhere and the integral over the entire space equals 1.

There are many resources to help you understand probability distributions.

The finite differences are important but are also one reason why even savvy professionals shy away from Monte Carlo simulations. To keep it simple, we will refer to everything as a probability distribution in this discussion.

Three Probability Approaches with Pros and Cons

1. Good Lifetime Data with Classic Statistics

This is the standard statistical approach. On the one hand, it is nirvana (an ideal or idyllic place); on the other hand, it is frequently like the lonely wanderer in search of El Dorado (a mythical place of fabulous wealth and opportunity).

Pros

• Most definitive

• This is what we are all hoping to have and use

Cons

• Laboratory testing does not reflect actual operating conditions

• Field testing is expensive and often not practical

• Field data inherently inconsistent due to updated equipment or changed contexts

• Running controlled tests to failure takes time and is expensive

• Data capture from work orders is erratic and of questionable quality

• In practice, we seldom run important things to failure

• Few organizations do formal Root Cause Failure Analysis (RCFA)

2. Distribution Fitting & Bootstrapping

This approach applies to partial lifetime data. Efron and Tibshirani (1993) are credited with the computational approach of the bootstrapping method.

Pros

• Proven and accepted approach

• Distribution fitting is a standard tool in most Monte Carlo simulation software

Cons

• Frequently develop distribution curves beyond the range of available data

• Requires some knowledge of the typical distribution for the class/type of data

3. Build Your Own Probability Distributions (Cumulative Density Function)

This approach is applicable when there is limited data or no data. It is an Interview-based approach with roots in Howard (1960s) and specific methods like probability wheels developed by Spetzler and Holstein (1974).

Pros

• Proven and accepted approach

• A common approach in practice. Either stand-alone or complimenting bootstrapping.

Cons

• Requires some knowledge of the typical distribution for the class/types of data

• Reliance on subject matter experts

• Highly dependent on qualitative survey skills (facilitation, administration, and analysis)

• Proprietary tools claiming to produce better results than others

Get In the Game

The world is richly skewed and highly uncertain. Use Monte Carlo simulations in your funding forecasts for future infrastructure and facility needs. It helps to have a trusted partner helping you build your forecast model, but you do not need to be able to recall all of your college statistics or be an expert at data collection to do it.

There are many resources to help you. The first step is to get in the game.

---

JD Solomon Inc provides program development, asset management, and facilitation services at the nexus of facilities, infrastructure, and the natural environment. Monte Carlo simulations are a standard tool in our approach to addressing risk and uncertainty. Contact us for more information on developing lifecycle forecasts, assessing the total cost of ownership, or providing third-party reviews of capital improvement programs or operating budgets.

Add Me to Your Contact List for Future Information

Why Are You Wasting Infrastructure Money By Not Using Monte Carlo Analysis?

If you are not using Monte Carlo simulations to inform your asset management decision making, then it is nearly certain that you are leaving value on the table.

Understanding the total cost of ownership is a fundamental principle of asset management of facilities and infrastructure. However, all of our knowledge is about the past. All of our decisions are about the future.

Monte Carlo analysis is our best tool for bridging our knowledge of the past with future uncertainties.

Monte Carlo analysis calculates the probability of outcomes by running multiple simulations using random variables. Analysis using Monte Carlo simulations strongly aligns with funding forecasts for facilities and infrastructure, where thousands of assets vary in cost and decay rates by orders of magnitude.

The power of most first-generation models using Monte Carlo simulations lies more in the insights than the absolute values of the forecasts.

Facility and Infrastructure Applications

All infrastructure assets deteriorate with time and use. To maintain the effectiveness and value of an asset, renewal work should be performed periodically. When the asset has reached the end of its reliable life, it should be replaced.

At the heart of an asset management program is the effort to preserve the existing system's performance and reliability by anticipating future renewal and replacement (R&R) needs and ensuring that adequate and timely funding is planned into the capital improvement program (CIP).

Asset Management Systems Need a 20-year Forecast

According to the Institute of Asset Management, asset management involves balancing costs, opportunities, and risks against the desired performance of assets to achieve an organization's objective. Asset management is the art and science of making the right decisions and optimizing value delivery. A common objective is to minimize the whole-life cost of assets. Still, there may be other critical factors such as risk or business continuity to be considered objectively in this decision making.

More simply, you are not really doing asset management if you do not know your lifecycle costs and sensitivities. Monte Carlo analysis is the preferred method to address the many associated uncertainties.

Insights Gained

1. Data Quality (result)

Asset lists, asset conditions, and replacement values are all required to perform the forecast. In one recent example, the data attributes were fully populated but the pipe material indicated was not available when associated with many of the pipe install dates (PVC pipe was not manufactured in the 1930s and 1940s). In two other recent examples, the unit price had been installed rather than the total replacement costs, thereby making the particular asset classes look inconsequential; however, when corrected, both asset classes were at the top of the priority list.

The bottom line is that you may think you have quality data but will not know for certain until you use it in an application. The best way to test your data quality is the 20-year R&R forecast using Monte Carlo simulations.

Establishing life cycle ranges (and distributions) and existing rebuild frequencies also produces an understanding of maintenance approaches.

2. Long-term capital forecasts (or Cone Diagrams)

• Critical two-dimensional graph of Funding versus Time, including probabilities

• Seek to minimize peaks and troughs

• Identify pent-up or deferred requirements and/or need for better data

3. Assets needed for upcoming capital improvement program (lists)

• List used to verify whether capital improvement program includes the neediest assets

4. Baseline renewal and replacement frequencies and strategies (table)

• Should we modify these? Should we revisit our standard practice?

5. Tradeoffs analysis (tables)

• Tradeoffs of debt funding vs. yearly O&M expenditure

• Projected capital and O&M funding requirements

6. Sensitivity analysis (Tornado Diagrams)

• Financially, what is most important and what is least important

7. Looking at the combined facilities under a single lens (tables)

• Provides a well-defined justification and transparent plan in economically challenging, uncertain times.

8. Common understanding by staff (result)

• Staff turnover in all industries is high.

• More junior-level staff are advancing quickly into middle management roles.

What it Means

An R&R forecast using Monte Carlos simulations is a highly effective approach for structuring facility and infrastructure problems and gaining insights about key inputs. When done properly, an R&R forecast using Monte Carlo simulations improves a decision maker's understanding of risks, business value drivers, and the sensitivities of key decisions. The forecast also provides an understanding of key variables' relevant importance and interdependencies and, in turn, the value of both acquiring additional information and the potential areas for business process improvements.

An R&R forecast using Monte Carlos simulations is essential to support reliability and make risk-informed decisions. You may be making good decisions without this type of analysis, and that may be up for debate; however, if you are not using Monte Carlo simulations to inform your asset management decision making, then it is nearly certain that you are leaving value on the table.

---

JD Solomon Inc provides program development, asset management, and facilitations services at the nexus of facilities, infrastructure, and the environment. Contact us for more information on developing renewal and replacement forecasts, criticality analysis, and third-party reviews of previous forecasts using Monte Carlo simulations.

Send Me More Information about JD Solomon, Inc.

Why I Will Never See an Ensemble Model on an Asset Management Project

Most models aim to be the best, but each has a different way of getting to that result. Ensemble (consensus) models provide some of the best tropical cyclones forecasts but will not be seen on an asset management project.

Hurricane tracking is very good these days, especially in forecasting storm intensity and location two to seven days in advance.

Forecasting lifecycle costs (or the total cost of ownership) in asset management programs can learn a lot from forecasting tropical cyclones.

The most powerful model type is the ensemble, but there is little chance we will see it in asset management practice. Here are a few reasons why.

Spaghetti Diagrams

Spaghetti weather diagrams are a simplistic way of quickly conveying a lot of tropical information. Spaghetti diagrams derive their name from the appearance of tens of thin lines, each depicting a separate forecast.

The first lesson for asset managers is that forecast for tropical cyclones includes many lines, whereas the asset management lifecycle forecast includes a single line. There is not much spaghetti in asset management forecasting.

Another lesson is that many different forecasts (and different types of forecasts) are shown on the same diagram. The recognition that there are different types of forecasts underscores that one model is not significant or relevant for all predictions of an uncertain future.

Types of Forecast Models

A model is a system of postulates, data, and inferences presented as a mathematical description of an entity or state of affairs. In math, a model is a bunch of equations.

The term "forecast model" refers to any objective tool used to generate a prediction of a future event, such as the state of the atmosphere. The National Hurricane Center (NHC) uses many models as guidance in the preparation of official track and intensity forecasts.

There are a handful of model types:

1. Statistical models, in contrast, do not explicitly consider the physics of the atmosphere but instead are based on historical relationships between storm behavior and storm-specific details such as location and date.

2. Dynamical models, also known as numerical models, are the most complex and use high-speed computers to solve the physical equations of motion governing the atmosphere.

3. Statistical-dynamical models blend both dynamical and statistical techniques by making a forecast based on established historical relationships between storm behavior and atmospheric variables provided by dynamical models.

4. Trajectory models move a tropical cyclone along based on the prevailing flow obtained from a separate dynamical model.

5. Ensemble (or consensus) models are created by combining the forecasts from a collection of other models.

Accuracy

There were 394 official forecasts issued during the 2021 Atlantic hurricane season, which is above the long-term average number of forecasts and a similar level of activity as the 2016-2018 seasons. The mean National Hurricane Center (NHC) official track forecast errors in the Atlantic basin were close to or below their previous 5-year means. Records for track accuracy were set from 48-72 h in 2021.

The official track forecasts were slightly outperformed by the ensemble (consensus) models at some time periods.

Mean official intensity errors for the Atlantic basin in 2021 were lower than the previous 5-year means at all forecast times. The official forecasts were quite skillful and beat all of the models at 12-36 h and 72 h. Records for intensity accuracy were set from 12-60 h in 2021. Although there is a considerable amount of year-to-year variability, the intensity forecast errors have gradually decreased over the past decade.

While the official intensity forecasts beat the model forecasts in 2021, the top two forecast models were ensemble (consensus) models.

The Power of the Ensemble

It is impossible to definitively predict the future state of uncertain future events, including the atmosphere, because of their chaotic nature. Whether it is observation networks in weather or limited monitoring points in asset management, there is limited resolution in both space and time. Lack of observations introduces uncertainty into the true initial state of the atmosphere.

There are four primary considerations related to ensemble (consensus) forecasts.

1. If the ensembles are close together, the confidence in a forecast is higher. However, the forecast still should be checked against other ensembles (if available) because each ensemble member is part of the analyst’s ideas or preferences.

2. If an ensemble forecast is tightly packed but still diverges from other ensemble (consensus) models, the forecast could be either very arrogant or likely to be correct. This is where forecaster judgment is important.

3. If the ensembles are spread apart, the model has less confidence or the overall forecast is highly uncertain.

4. Ensemble forecasts are appropriate for planning purposes (medium and long-range forecasts) but not applicable for real-time. This is largely a function of the time it takes to collect, process, forecast, and assemble the model.

Asset Management Limitations

Forecasts with ensemble models provide more insights and are usually more accurate than single models. Ensemble forecasts will continue to have limited application in asset management for the following reasons.

It isn't easy to obtain resources for one lifecycle forecast model, much less more than one.

• We seldom run things to failure, so judging the forecast results is not practicable.

• We seldom perform forecast post-mortems even if the data is available.

• We tend to focus more on collecting physical equipment data and less on asset valuation data, which is needed to develop a lifecycle model.

• We are usually more focused on the near-term rather than the long term.

What This Means

Forecasting lifecycle costs (or the total cost of ownership) in asset management programs can learn a lot from forecasting tropical cyclones. For several reasons, the ultimate power of forecasting with ensemble models will never be fully utilized in asset management.

But fret not. Ensemble models are used in the black boxes of machine learning to develop business rules!

References

Portions of the article were taken from the NHC Forecast Variation Report for the 2021 Hurricane Season.

---

JD Solomon Inc provides forecasting services related to asset management and new infrastructure programs. Contact us for more information on our quantitative forecasting services, including those using Monte Carlo simulations.

Receive Updates from JD Solomon, Inc.