Five questions to answer before deploying private AI

1. What business problem will the system solve?

“We want to use AI” is not an implementable objective. The need should be narrowed to a task such as retrieving internal policy, organising project material, drafting content, supporting service teams or coordinating approvals, with clear definitions of acceptable and unacceptable outcomes.

The NIST AI RMF recommends mapping context, tasks, risks, affected parties and expected benefits before deployment. Model capability, infrastructure and integration can only be compared meaningfully once this context is clear.

2. Which knowledge may the system use?

The difficult part of an enterprise knowledge base is rarely uploading files. It is deciding which sources are authoritative, current, owned, confidential and maintained. Unapproved drafts, conflicting policies and documents without access labels directly reduce reliability.

• Which materials are authoritative, and who approves and updates them?
• Which sources contain personal information, trade secrets or third-party rights?
• Which teams, roles and projects may access each information class?
• Can an answer be traced to a source, and how is outdated material withdrawn?

3. How will access and data boundaries be enforced?

Private deployment can improve control over infrastructure and data flows, but identity, least privilege, network segmentation, encryption, logging, supply chain, backups and incident response still need to be designed. External models, plugins and data services require a separate assessment of what they receive and retain.

Security boundaries belong in architecture, operating procedures and responsibility matrices—not only in a claim that data stays on premises.

4. How will outputs be evaluated and human responsibility retained?

Testing should reflect business risk through representative cases, error categories, acceptance thresholds and human review points. Content organisation and high-impact legal, financial, medical or employment decisions cannot share the same tolerance for error.

Generative AI can be inaccurate, incomplete, biased or poorly sourced. Material decisions require suitable human oversight, and accountability cannot be transferred to the model.

5. Who will operate the system after launch?

Knowledge updates, model changes, user feedback, permissions, cost and incident handling need long-term ownership. A pilot will quickly lose trust if no business, data and technology owners are responsible after the initial build.

A sound starting pattern is to select a bounded, valuable task; establish a baseline; validate a small prototype; record failure modes; and expand only when evidence supports the decision.