How we keep AI-assisted work trustworthy

AI made first drafts fast. Turning a draft into work a client can trust is still the hard part.

Trust comes down to three decisions: 

1. What information went into the tool

2. How the draft was reviewed 

3. Who owns the recommendation

People make those calls. That's what "human in the loop" means in practice.

At Alloy, we use AI across research, analysis, creative production and development. Our Innovation Center of Excellence (CoE) is a cross-discipline group from strategy, media, creative and development. It writes the policies, builds the shared workflows and runs the support channels that govern how we use AI on client work.

As a member of Anthropic's Claude Partner Network, our team trains and certifies on Claude directly with Anthropic and can go to their technical team when a workflow needs it.

When assessing your own AI adoption (or an agency partner's), consider these practices:

Define "better" before picking a tool

Our CoE starts with how work gets done today: 

• What are the repetitive tasks that can be automated?

• Where in the process do we lose project context? 

• What activities slow our path to a recommendation? 

The cross-discipline group defines the problem, tests an approach and decides whether it becomes standard practice. When it does, the CoE packages it as a shared skill with a written workflow that every team runs the same way. Our library of skills continues to grow, covering recurring work like brand visibility audits, paid media audits, SEO handoffs and monthly performance reporting.

Measure speed and quality together. A faster first draft is worth little if it takes just as long to get right. Decide what you're trying to improve first. 

• Do you want to spend less time pulling research? 

• Is your aim to minimize reporting errors? 

• Or, are you after a clearer basis for a decision? 

Then judge the tool against it.

Support people while they're doing the work

Training covers the basics, but the harder problems show up once people are doing real work. We run weekly AI Office Hours and a #claude-support Slack channel where people bring a prompt that isn't working, a task they're unsure how to approach or a request for a second opinion. Sharing what they tried and what happened gives others enough context to help. When a fix keeps coming up, the CoE folds it into the relevant skill so the next person starts from the working version.

Most of these conversations are about judgment. When an output misses a detail, the fix might be a better prompt, a more focused task or a person stepping in, and choosing the right one is part of using AI well. For clients, that means more consistent work across the team.

Set data and access rules before anyone uploads a file

Our policies separate public information, internal material, confidential client information and regulated data. What's permitted depends on the data, the tool and the client's agreement. Teams use only the data a task requires and escalate anything sensitive or unclear.

Anything built for shared use gets reviewed before deployment. That covers reusable instructions, plugins, scheduled tasks and connections to business systems. The review looks at purpose, what it can access and what permissions it needs. Drafting a paragraph and writing into a shared system carry different risks, so our connectors always default to read-only.

When you evaluate an AI workflow, ask what the AI can access and what it can change. Approving a tool doesn't confirm that its permissions fit the task, so check both.

Give human review a defined job

Every AI-assisted output gets human review before it reaches an Alloy client. The reviewer is accountable for what ships.

That review goes past the presentation. For analysis, the reviewer validates the data and checks that it supports the conclusion. For creative, they check brand fit and the claims being made. Whoever presents a recommendation should be able to explain why it makes sense for that client. As we've written about AI and reporting, faster assembly leaves the team responsible for deciding what matters and what to do next.

Ask who reviews the work and what they check. "Human oversight" means something only when a person has clear criteria and owns the final decision.

Questions clients should ask their agency

• Which parts of the work use AI? What is it helping the team do?

• What can those tools access? How are our restrictions applied?

• Who reviews the output? What do they check?

• Where do concerns go when something doesn't look right?

At Alloy, AI use and client restrictions are covered in the governing agreement, and we explain on request how AI shaped a specific recommendation or output.

These practices are part of why Alloy joined Anthropic’s Claude Partner Network. Read the announcement to see what it means for our clients.