The AI Accountability Gap: Why Human Judgement and Secure, Trusted Data Will Be Critical in 2027
By Bel Lloyd, Customer Success Lead at Amperity.
AI accountability will become increasingly important in 2027 as brands expand their use of artificial intelligence. AI systems can now access sensitive customer information, trigger workflows, and take action across multiple platforms. These capabilities also raise concerns about data integrity, consent and responsibility when something goes wrong.
For data, security and technology leaders, the challenge extends beyond securing AI systems. They must also establish clear controls over the decisions those systems make. Human oversight, trusted customer data and effective governance will play an important role.
Looking ahead to 2027, Bel Lloyd, Customer Success Lead at Amperity, shares her predictions for responsible AI adoption. She highlights the importance of human judgement, AI literacy and trusted data in managing enterprise risk.
AI Accountability, Governance and Human Oversight
AI literacy and governance will become practical leadership capabilities. Leaders must establish boundaries for how AI supports customer decisions. They also need to determine who approves significant actions and who remains accountable for the outcomes.
For example, using AI to analyse an audience differs from approving a campaign that changes customer treatment. Leaders must consider customer impact, data reliability, and whether they can review or reverse an action. These factors should guide how much authority they delegate to AI.
Over the next year, I expect more organisations to establish approval points and escalation paths within customer workflows. Teams should understand when to act on an AI recommendation, when to check it, and when human approval is necessary.
Effective leaders will understand AI literacy and recognise where human judgement matters most. People must help define positive customer outcomes and assess whether AI performs as intended.
AI Literacy and the Emerging Skills Gap
The first wave of AI training focused on using the tools. However, the next phase must teach employees how to evaluate AI recommendations. Teams should check convincing answers against customer information, business objectives, and underlying assumptions.
For example, an AI system might recommend targeting a supposedly high-value audience. Marketers should question how the system defined that value and whether the records represent distinct customers. They should also identify missing information and determine how to measure the campaign’s results.
Over the next 12 months, I expect more organisations to include these questions in training and campaign reviews. AI literacy will involve knowing when to trust a recommendation, challenge its findings, or seek additional evidence.
Leaders establish the rules for AI, while teams need the skills to apply them. Employees must exercise judgement in their daily work and understand what makes an AI recommendation reliable. They must also retain responsibility for the decisions they make.
Customer Data Governance, Consent and Trust
Richer customer profiles alone will not create a competitive advantage. Brands also need reliable customer context, secure boundaries and a clear value exchange. First-party data represents a relationship asset because customers share information when they trust a brand and receive value.
Across Asia Pacific, we are already seeing customers focus projects on consent and improving how they manage customer preferences. As AI expands what organisations can infer and act upon, those preferences must guide data use. They should not remain separate from marketing decisions.
Over the next 12 months, I expect more brands to integrate customer preferences into their data and activation workflows. They will also need to explain how sharing information benefits customers. Permission to receive one type of communication should not automatically extend to every form of engagement.
The opportunity lies in earning and maintaining permission to deliver relevant customer experiences. Brands that combine customer understanding with secure, appropriate and ethical data use will be better positioned to build lasting relationships. This becomes increasingly important as AI accelerates customer engagement.
About Bel Lloyd, Customer Success Lead, Amperity
Bel Lloyd is a contributor in the Australian Martech landscape and the Customer Success Lead at Amperity, where she operates at the intersection of people, technology, and data. Recognised as the 2023 Digital Marketer of the Year (Women in Digital), Bel drives impactful growth for enterprise brands by unifying complex customer data to deliver personalised experiences and measurable business outcomes, including leading the charge in closed-loop attribution, having facilitated a first-of-its-kind industry partnership with MixIn by Endeavour Group and Criteo to bridge the gap between digital advertising and verified in-store sales.
Beyond her technical leadership, Bel is a dedicated advocate for Women in Tech. She serves as a Lead Mentor in the She Codes community, empowering women to transition into technical roles, and is a frequent industry speaker on the role of diversity in fueling innovation. Through her advocacy and technical expertise, Bel continues to champion inclusion and share forward-thinking perspectives on the future of the digital economy.
About Amperity
Amperity helps brands turn trusted customer context into better decisions and action. Its AI-powered Customer Data Platform resolves identity and unifies fragmented data with real-time signals, creating a governed customer context foundation for marketing, service, analytics and AI. More than 400 leading brands worldwide, including Accent Group, Alaska Airlines, DICK’S Sporting Goods, BECU and Wyndham Hotels & Resorts, rely on Amperity to improve customer experiences and drive measurable growth. Founded in 2016, Amperity operates globally with offices in Seattle, New York City, London, and Melbourne. Learn more at amperity.com.