BRP AI Audit Stakeholder Survey

BRP Stakeholder Consultation Survey

Step 1 of 8

12%

Thank you for participating in the BRP AI Audit Survey. This short survey helps us understand how AI is used and governed in your organization and will help us explore the current landscape of AI development at BRP.

Please note that the questions you see are dependent on your department and role. As a result, you may notice that certain question numbers are skipped as you move through survey sections.

The survey should take approximately 15-20 minutes to complete.

Background — About you and your organization

Respondent Name(Required)

Corporate Level(Required)

Department / Team(Required)

How often do you currently interact with AI systems in your work?

Section 1 — AI Governance and Accountability

G-01: Which of the following best describes AI governance policy in your organization?(Required)

A formal, published AI governance policy exists, and I can locate it

I've been told a policy exists, but I haven't seen it or couldn't find it

There are informal guidelines or principles but no formal published policy

None of the above

G-02: If an AI system in your area produced a harmful or incorrect output tomorrow, who would be accountable for investigating and resolving it?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

I know exactly who is accountable — there are named owners with defined responsibilities

I have a general idea (e.g., "IT would handle it") but specific individuals aren't designated

It would depend on the situation — no pre-defined accountability exists

I am not sure

G-03: Which of the following governance mechanisms are currently operational (not just planned)?(Required)

AI ethics board or governance committee that meets regularly

Formal risk assessment process required before AI deployment

AI system registry or inventory maintained centrally

Defined escalation path for AI-related concerns from any level

Regular review cycle for AI policies (at least annually)

Role-based access controls (RBAC) enforced for AI models and training data in production

None of the above are currently operational

I am not sure

G-04: Think about the last time you or a colleague raised a concern about an AI system's output, fairness, or appropriateness. What happened?(Required)

The concern was acknowledged, investigated, and the outcome was communicated back

The concern was heard but I'm not sure what came of it

I or others have had concerns but didn't raise them — it wasn't clear how or whether it would be taken seriously

This has not happened

G-05: Does a published Safe Use Policy exist for AI, and do you personally know what you are and aren't permitted to do with AI tools in your role?(Required)

Yes — a published policy exists and I can clearly describe the boundaries for my role

A policy exists but I'm unsure about the specifics as they apply to my work

There are informal norms but no published policy

I am not sure if a policy exists

G-06: Is the Safe Use Policy reviewed and updated as AI tools, capabilities, and regulations change — or is it a static document from initial rollout?(Required)

Reviewed and updated at defined intervals (e.g., quarterly) or when significant changes occur

Updated occasionally but not on a defined schedule

It hasn't been updated since it was first created

There isn't a policy to update

I am not sure

G-07: As AI-specific regulations continue to evolve across multiple jurisdictions, does your organization have a designated function responsible for monitoring regulatory changes — including international frameworks such as the EU AI Act and China's AI regulations — assessing their impact on your operations, and updating AI policies accordingly?(Required)

Yes — a named function owns AI regulatory monitoring with a defined process for impact assessment and compliance planning

Legal or compliance teams are aware of emerging regulations but there is no structured monitoring or response process

Regulatory awareness depends on individuals rather than a formal function — no systematic tracking exists

We do not currently have a process dedicated to AI regulatory compliance monitoring

I am not sure

G-08: What is the single biggest gap in how your organization governs AI today?

Section 2 — AI Risk Identification and Context

M-01: Does your organization maintain a current inventory of all AI systems in use — including those adopted by individual teams or departments without central approval?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — a centrally maintained inventory exists that includes both sanctioned and shadow AI

Partially — centrally approved tools are tracked but team-level adoption is not

Informally — some people could list the tools but there's no maintained registry

No — there is no systematic tracking of which AI systems are in use

I am not sure

M-02: For the AI systems you interact with, has a documented risk assessment been completed that identifies potential harms, failure modes, and affected stakeholders?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — I've seen or contributed to the risk assessment for the systems I use

I believe assessments exist but I haven't seen them

Risk considerations were discussed informally but not documented

No formal risk assessment has been completed for the AI systems I'm aware of

I am not sure

M-03: Are AI systems classified by risk level (e.g., high / medium / low) with different governance requirements specific to AI applied accordingly?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — a tiered classification system is in use and drives governance intensity

Classification exists conceptually but doesn't yet determine governance requirements

All AI systems are treated the same regardless of risk level

We haven't established risk categories for AI systems

I am not sure

M-04: Does your organization have a review process for evaluating AI-specific data security risks of new tools, models, and external applications to understand data exfiltration pathways?(Required)

Yes — a formal review process exists that assesses data exfiltration pathways for all new AI tools, models, and external APIs before adoption

Reviews are conducted for some tools or higher-risk integrations but there is no consistent process applied to all new AI adoptions

Data security risks are considered informally during procurement but no structured review process or exfiltration pathway analysis exists

We do not currently have a review process for evaluating AI-specific data security risks or exfiltration pathways

I am not sure

M-05: Are there AI-related risks in your area that you believe are not currently being tracked or assessed? Describe them.

Section 3 — AI Risk Measurement and Monitoring

ME-01: Before an AI system goes into production (or after significant changes), which types of testing are actually performed?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Accuracy / performance testing against defined benchmarks

Bias / fairness testing across protected groups

Security / adversarial robustness testing

User acceptance or domain expert validation

No formal testing is routinely performed

I am not sure

ME-02: Once AI systems are in production, how is their performance and risk profile monitored over time?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Automated monitoring with defined KPIs and alerting for drift or degradation

Periodic manual reviews (e.g., quarterly accuracy checks, sample audits)

Monitoring is reactive — we investigate when users report problems

No systematic post-deployment monitoring exists

I am not sure

ME-03: When underlying models are updated (e.g., a vendor releases a new version, or a model is retrained), is the updated version re-evaluated before being put into production?(Required)

Yes — model updates go through a defined re-evaluation and approval process

Sometimes — it depends on the significance of the update

Vendor-managed models update automatically without our evaluation

We have not established a process for evaluating model updates

I am not sure

ME-04: If an AI system in your area started producing biased, inaccurate, or harmful outputs, how would the problem most likely be discovered?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Automated monitoring would flag the anomaly before users are affected

A periodic review or audit would catch it within a reasonable timeframe

An end user or customer would notice and report it

It might go undetected for a long time — we don't have good detection in place

I am not sure

ME-05: Beyond initial pre-deployment testing, are AI systems in production actively protected and monitored against adversarial threats — such as prompt injection attacks, data poisoning, or attempts to manipulate model outputs through crafted inputs?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — adversarial threat protections are in place, monitored continuously, and updated as new attack patterns emerge

Some protections exist for higher-risk systems but adversarial monitoring is not standard across all AI deployments

Adversarial risks are tested at deployment but there is no ongoing monitoring or updating of protections

Adversarial threats to AI systems have not been formally addressed in our testing or monitoring processes

I am not sure

ME-06: What aspect of AI system performance or risk is most challenging for your organization to measure today?

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Section 4 — AI Risk Response & Mitigation

MG-01: Think about the most recent time an AI system produced an unexpected, incorrect, or harmful output. What happened next?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

A defined incident response process was triggered — containment, investigation, and remediation followed

It was escalated to someone who handled it, but the process was ad hoc

The output was corrected manually but no formal investigation occurred

I am not sure

MG-02: If you discovered a serious AI-related issue right now, which of the following best describes what you would do?(Required)

I know the exact escalation path and who to contact — and I've used it before

I know generally who to tell but haven't had to use the process

I'd tell my manager and hope they know the process

I am not sure

MG-03: For AI systems in production, can the system be quickly disabled, rolled back, or overridden by a human if needed?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — kill switches, rollback, and human override are built in and tested

Some systems have these controls; others don't

We could disable systems but it would be manual and disruptive

We haven't built or tested these controls

I am not sure

MG-04: When AI-related issues have been identified and resolved, are the lessons systematically fed back into governance, risk assessments, and training?(Required)

Yes — post-incident reviews are standard and drive updates to policy, controls, and training

Sometimes — it depends on the severity; only major issues trigger a review

Lessons are discussed informally but not captured or acted on systematically

We haven't established a feedback loop from incidents to governance

I am not sure

MG-05: Is there a formal intake or approval process that teams must follow before adopting a new AI tool or service?(Required)

Yes — a formal process exists and is consistently enforced

A process exists on paper but is inconsistently followed

No — teams can adopt AI tools without formal approval

I am not sure

MG-06: For AI systems that can autonomously call external tools, APIs, or services, is a defined permission structure in place that restricts which tools can be invoked, under what conditions, and with what level of data access — following least-privilege principles?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — tool permissions are explicitly scoped using least-privilege principles, with allow-lists and audit logging in place

Some restrictions exist but AI tool access is broader than strictly necessary for the defined use case

AI systems can invoke available tools or services without explicit permission scoping or restrictions

We have not defined or reviewed permission structures for autonomous AI tool calling

I am not sure

MG-07: For AI systems that influence significant operational decisions, are human oversight and explainability requirements formally defined — ensuring that outputs are interpretable, that decision rationale can be communicated, and that a human can review or override the system before high-stakes actions are taken?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — human-in-the-loop requirements and explainability standards are formally defined for all decision-relevant AI systems

Human review is required for some high-stakes decisions but explainability requirements are not formally documented

AI outputs are generally acted upon without a required human review step or interpretability standard

We have not established policies for human oversight or explainability requirements for our AI systems

I am not sure

MG-08: If you could change one thing about how your organization responds to and manages AI risks, what would it be?

Section 5 — AI Financial Risk and Cost Governance

F-01: How are AI-related costs (API usage, cloud compute, licensed models) currently monitored and controlled?(Required)

Real-time dashboards with automated alerts and enforced budget thresholds

Dashboards or reports exist but thresholds are not enforced or alerts are not automated

Costs are reviewed periodically (e.g., monthly invoices) but not monitored in real-time

AI spend is not tracked separately from general IT/cloud costs

I am not sure

F-02: When AI costs exceed expectations (e.g., a model call spike, an unexpected invoice), what happens?(Required)

A defined escalation process kicks in with named decision-makers and spending authority levels

It gets flagged informally — someone notices and raises it, but there's no defined process

It typically isn't noticed until the next budget review cycle

I am not sure

F-03: If AI costs doubled unexpectedly next month, how quickly would your organization detect and respond to the increase?(Required)

Within days — automated alerting would flag it immediately

Within a week or two — manual monitoring would catch it fairly quickly

At the next monthly or quarterly review — it would take weeks

We might not notice until a significant budget overrun is reported

I am not sure

F-04: When selecting or renewing AI models and services, is cost-to-performance analysis a formal part of the decision?(Required)

Yes — comparative cost-performance evaluation is documented and reviewed before decisions

Cost is considered but not systematically compared against performance metrics

Decisions are primarily capability-driven — cost is secondary

We don't have a defined process for evaluating model cost vs. performance

I am not sure

F-05: What role does Finance play in tracking whether AI investments are delivering the benefits claimed in their business cases?(Required)

Finance actively tracks benefit realization against business case commitments post-deployment

Finance tracks costs but benefit tracking is left to the business owner

Finance reviews AI spend but doesn't track benefits — no one systematically does

Finance is not involved in AI investment tracking beyond initial budget approval

I am not sure

F-06: For AI systems that have been in production for 6+ months, has the business value they were expected to deliver been formally evaluated?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — post-deployment benefit reviews are standard practice

For some high-profile initiatives but not consistently across all AI systems

Benefits were stated in the business case but no one has gone back to verify them

I am not sure

F-07: Where do you see the greatest financial risk in your organization's current approach to AI — unexpected costs, unrealized value, or uncontrolled tool growth? Describe the specific gap.

Section 6 — AI Change Management & Adoption

CM-01: Are role-specific AI competency requirements defined for different functions — so that a finance analyst's expected AI skills differ from an IT engineer's or a legal reviewer's?(Required)

Yes — role-specific AI competency profiles exist and inform training

General AI awareness training exists but isn't differentiated by role

Individual teams handle training themselves with no central framework

No AI-specific training or competency framework exists

I am not sure

CM-02: Think about the most recent AI-related training you received. How well did it address the specific AI tasks and decisions you actually face in your role?(Required)

Directly relevant — it addressed the specific AI tools and decisions in my role

Somewhat relevant — general AI concepts were covered but not my specific use cases

Not relevant — the training didn't map to what I actually do with AI

I haven't received any AI-related training

CM-03: Is there a process for identifying AI skill gaps across the organization and closing them — not just at onboarding, but on an ongoing basis as tools and responsibilities evolve?(Required)

Yes — skill gaps are assessed periodically and drive targeted upskilling

Initial training is provided but there's no ongoing gap assessment

Skill gaps are addressed reactively — when a problem occurs, not proactively

There's no process for identifying or closing AI skill gaps

I am not sure

CM-04: When AI systems are being designed, built, or rolled out, are end users from the affected business areas involved in the process — not just consulted at the end?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — end users are embedded in AI squads throughout the lifecycle

Users are consulted at key milestones but not embedded in the team

User feedback is gathered after deployment, not during development

AI development happens without structured user involvement

I am not sure

CM-05: When end users are involved in AI development or rollout, does their input actually influence decisions — and is there a safe channel to raise concerns or suggest improvements?(Required)

Yes — user input has visibly shaped decisions, and there's a clear feedback channel

Input is collected and appreciated, but it's hard to see how it changed outcomes

There's a feedback channel but it's rarely used or feels performative

No structured channel exists — concerns are raised informally if at all

I am not sure

CM-06: When a new AI system is introduced, are all affected parties — including staff, customers, and downstream decision-makers — identified and informed in advance with enough context to understand what is changing, why, and how it affects them?(Required)

"AI systems" include applications, tools or programs that use artificial intelligence as a core component of their functionality.

Yes — affected parties are systematically identified and proactively informed with clear context before deployment

Staff are informed in advance but broader stakeholder groups such as customers or downstream decision-makers are not consistently identified or notified

Some communication happens but it is inconsistent — typically incomplete, last-minute, or reactive rather than planned

There is no structured process for identifying or informing those affected by AI systems

I am not sure

CM-07: How clearly do you understand your organization's overall AI strategy — and whether and how it will affect your role, workload, or team structure?(Required)

Very clear — I understand the strategy and what it means for my work specifically

I understand the general direction but not how it will affect my role

I've heard vague statements about AI but nothing concrete enough to act on

I am not sure

CM-08: If you could make one change to how your organization manages AI-related change (training, communication, involvement, or safe use), what would have the biggest impact?

Business Consulting

Data and Technology Consulting

Business Consulting

Data and Technology Consulting

BRP Stakeholder Consultation Survey

Background — About you and your organization

Section 1 — AI Governance and Accountability

Section 2 — AI Risk Identification and Context

Section 3 — AI Risk Measurement and Monitoring

Section 4 — AI Risk Response & Mitigation

Section 5 — AI Financial Risk and Cost Governance

Section 6 — AI Change Management & Adoption

Privacy Statement

Ottawa

Toronto

Vancouver

Contact