5 Best Practices for Building an AI Predictive Analytics Model

“Prediction is possible only in a world in which events are preordained… The one we inhabit is quite different.” - Paul Saffo, Harvard Business Review

With a literal reading, this statement still rings true. There are infinite variables that influence outcomes, including the unpredictable nature of human beings. But, when Paul Saffo wrote this nearly 20 years ago, did he consider the rise of artificial intelligence (AI)?

AI can help drive efficiency and elevate employee and customer experiences, from natural-language processing (NLP) behind chatbots to large language models (LLM) for creative ideation. So, it’s no wonder that today’s organizations are rolling out AI-enabled workflows and tools across their business.

For the project or portfolio management office (PMO), AI predictive analytics can transform forecasting accuracy for their organizations. Predictive analytics can refer to both technology and techniques that identify patterns in historical data. They use these patterns to predict the most likely outcome based on a set of parameters (drivers).

To realize these benefits, PMOs will need to build their own AI predictive analysis model (or invest in predictive analytics software). But, there are no ‘one-size-fits-all’ AI models—it depends on your objectives for predictive analysis, your organization’s data capabilities, and the data you want to use. That’s what we’ll be exploring below—the best practices that yield better success rates from your predictive analytics software, from defining your objectives to future-proofing your solution.

1. Define Your Objectives

Before you start ideating the parameters or algorithms of your AI model, you need a clear idea of the ‘problems’ you’re trying to solve with predictive analytics. For example, your problem might be:

• Identifying which requests for proposal (RFP) are worth responding to
• Determining which projects would add the most strategic value to a portfolio
• Detecting risks and their potential impact in the early phases of a project

Understanding your problems sets you up to define your objectives for predictive analysis, which will help you evaluate the success rate and ROI of your model. Questions you should prepare to answer include:

• What are we trying to predict?
• What are the challenges involved in the process today?
• What data does this process rely on?
• Where does that data come from?
• What does the output need to look like?

Your answers will indicate the infrastructure you need to build and implement an effective predictive analysis model, including your tech stack and internal skillsets. They will also indicate the type of model you need, which you’ll need to know to select your algorithms (more on that later). These are two of the most popular models for predictive analysis:

• Classification, which predicts outcomes based on category fit (such as ‘Win’ or ‘Lose’ for an RFP).
• Regression, which predicts outcomes based on trends and is expressed as a continuous value (such as the ROI of a project idea).

2. Ensure Data Quality

PMOs will be no strangers to the importance of data quality for project and portfolio management (PPM). In AI predictive analytics, high-quality data enables more efficient and reliable decision-making. That’s because it removes the need for guesswork, which is more prone to the influence of human cognitive bias.

Most PPM tools already use parametric estimations—a calculation of effort or investment required based on historical data. AI-enabled predictive analysis takes this a step further by computing the most likely outcome based on the performance of similar projects or portfolios. But, to predict outcomes with a high degree of confidence, AI needs high-quality data in sufficient volume from credible sources.

Strong data governance is one of the first lines of defense against poor data quality. If there are clear policies, best practices, and data quality champions for teams to follow, you’re less likely to experience human input delays and errors. It makes data quality everyone’s responsibility and ensures that the data AI uses is reliable enough to predict from. You can help reinforce data governance with certain PPM tools, which automatically alert users if they detect a data anomaly or lack of data input where required.

3. Consider Ethics and Compliance

Ethics in AI project management are a whole discussion in their own right. But, the key takeaway is this: rigorously considering ethics and compliance before implementing your AI predictive analytics model will protect your long-term ROI from unexpected costs. These costs could originate from:

• Employee or customer discrimination: Human behavior influences the data that AI uses to learn. If a subconscious bias means project stakeholders overlook specific customer groups in product development or certain employees for a project, historical project data will reflect this bias. At best, this will limit your company’s relevance and revenue opportunities. At worst, it could result in brand damage or employment lawsuits.
• Regulatory non-compliance: Complying with regulations such as the EU’s General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA) will include AI usage. If they apply to your business, you must update your data privacy policies and allow customers and employees to opt out of AI using their data. If you don’t, you could face significant fines from regulatory bodies.

Factoring ethics and compliance into building and implementing your AI model (or the predictive analytics software you buy) is an essential risk-management exercise. With those elements considered, you can more confidently embrace the benefits of AI predictive analysis.

4. Select the Right Algorithms

As we’ve already mentioned, there are no ‘one-size-fits-all’ models for AI predictive analysis. It will depend on the type of problem you’re trying to solve, how complex it is, and the data the model uses to solve it. Training and optimizing your AI model to predict the outcome of a specific project or portfolio requires the right algorithms—sets of rules for problem-solving.

Algorithms commonly used for predictive analysis include linear regression, decision trees, and random forest. But, you’ll often need to combine a few to solve complex problems. Planisware, for example, uses an algorithmic approach called gradient-boosted regression trees for its predictive analysis ****capabilities. ****It’s flexible, robust, and can handle numeric and non-numeric input.

You’ll have to train your AI model using your selected algorithms and one sample of data. Then, you’ll need to test its efficacy with a different sample of data that’s still relevant to your problem. You may need to go through multiple iterations to find the best fit. Alternatively, you can take advantage of predictive analysis features in PPM tools. Providers often optimize their AI models for multiple use cases, from new product development (NPD) to digital transformation.

5. Future-Proof Your Solution

The first time you train and test your AI predictive analysis model should never be the last. As your business evolves, you’ll need to evaluate your model’s performance by providing it with new data samples and validating its predictions. This will ensure that your model stays relevant to your use case and continues to deliver value.

The most effective way to stay on top of this learning loop is to build a workflow and assign responsibilities within your team for reviewing and validating your model’s decisions. They might also be responsible for identifying and following up on data anomalies that might be impacting its logic. But they shouldn’t work in a silo—user feedback is also invaluable for flagging performance issues.

Finally, you should document the process of building, training, and testing your model and its applications. This documentation will be crucial for compliance-related audits, as well as helping to train your team on the model’s use and ensuring future stakeholders can continue to develop it.

Building your AI predictive analysis model is a resource-intensive but worthwhile project that requires a program of maintenance and innovation work. As an alternative approach, look for AI-enabled PPM tools with these capabilities:

• Predictive analytics that allow you to train a model on any data in the tool and use it to estimate any output.
• Platform-agnostic technology that can integrate with your critical enterprise applications, including built-in-house solutions.
• Anomaly detection features that automatically flag potential data issues.
• Centralized data storage that creates a single source of truth for portfolio and project data, for AI models and human users.