D2M Blog

Building an AI Center of Excellence: Defining Success

Defining Success
D2M has advised several organizations on the creation of Centers of Excellence (CoE) focused on Machine Learning (ML) and automation. A critical first step for any organization undertaking a new project is defining the measures of success. Creating a CoE is no different.

Determining the business objectives for the formation of a CoE and developing quantifiable key performance indicators (KPIs) to measure the achievement of those objectives will support the case for creating a CoE and guide stakeholders through the process.

At D2M, we define goals using a method similarly to the Agile Methodology technique that creates user stories. These stories define the purpose of a CoE and determine the acceptance criteria for success

We have provided several examples below. While all examples may not be relevant to your organization, they provide a starting point for defining and measuring what a successful CoE will look like.

As a data consumer, I want better data consistency so that I can more accurately compare today’s results with past ones.

Acceptance criteria:

  • CoE supports and automates data pipelines to ensure consistent data treatment
  • CoE provides documentation of data pipelines guaranteeing consistent data feeds

As a business unit owner, I want to find cognitive automation opportunities, so that I can react to customer needs, faster.

  • CoE supports use case discovery for business lines
  • CoE provides a ranked list of automation opportunities with preliminary ROI estimates
  • CoE maintains a list of common use cases with known ROI

As a business unit owner, I want to ensure that my unit is always innovating

  • CoE provides support for review of new and existing processes
  • CoE provides suggestions for cognitive automation
  • CoE produces frequent communications about the advances of ML applicable to business lines, both within the organization and outside

As a technical manager, I want to reduce reliance on external skills

  • CoE provides training for engineers who want to expand into ML
  • CoE provides in-house resources for in-demand jobs, such as data scientist, and data engineer

As a business unit owner, I want to ensure that my unit is meeting all data and regulatory guidance for machine learning

  • CoE guides explainable AI in a legal setting
  • CoE provides traceability guidance and documentation for pipeline and model deployment

As a project owner, I want to ensure that my data project meets organizational standards for maintainability

  • CoE provides organizational standards for
  • Scalability
  • Deployment
  • Model acceptance criteria

Overall, a CoE requires measurable goals as a prerequisite to its establishment.

How do you measure success in your CoE? We’d love to hear from you.