How I Built AI That Could Work Across My Company

In one of my companies, some AI-produced work went straight to clients with very little checking. People were under pressure to deliver on time, then came the errors and complaints, and management had to deal with problems we hadn't caught before the work went out.

We had people using different tools, each figuring out how to use AI for themselves. I built a shared agent system so they could work with common instructions and the right client history, within rules about what each person and their agent could see or change.

I worked through it in two layers, the building blocks an agent needs to do useful work, and the business layer that organizes it around projects, people and permissions. Most of my time and effort went into those permissions.

I couldn't find a suitable system I could buy and afford. Building one was complicated, but understanding the pieces helped me see what we needed, and you can use that understanding whether you buy or build AI agents for your business.

Everyone was figuring it out separately

The differences were easy to see in the files. Some reports were short and to the point, others were full of AI slop, and presentations came back in different voices, templates and colors.

And there was no easy way to pass along what worked. When someone found a better way to use an agent, the rest of the team didn't automatically benefit.

We were missing company-wide rules about what could and could not be done, feedback about what was working, and visibility into how people were using AI. From my side, that made it impossible to understand the impact on the business and where we should put more time and effort.

Individual AI use can be useful, of course, and people can do a lot with their own tools. But we had left them to solve too much separately, then expected their work to come together as the work of one company.

The pressure to deliver was still there, and we couldn't rely on someone catching every problem before the client saw it. Leaving the team to figure out AI use on their own was our decision, so the common rules and the way work got checked were our responsibility too.

I think of that as single-player AI, each person working with their own agent and their own context. The work we needed to do was multiplayer, several people contributing to the same project and needing to understand what the others had done.

I split the problem into two layers

The first layer is the agent itself, what I call its primitives, basically the building blocks it needs to work. The business layer determines who can use those building blocks and for which work.

Take a client report. Someone asks the agent to analyze the data and explain what it means for that client. It needs the source files and somewhere to save the report, so another person can open the work and check it.

The files won't explain everything. The agent also needs memory, useful information kept from earlier work, so it can account for what the client asked for and what has happened since, instead of making the person explain the project again.

And it needs a skill, the instructions for how we do and check that kind of analysis, including how to prepare the report.

To carry out that work, I also have to choose the model and how much reasoning effort it should spend. I want the report done properly without paying for more than it needs.

To produce the file, the agent needs tools that let it act, and integrations, connections to the systems holding the material. For us, those include Google services and WhatsApp, because that is where parts of the client conversation and work happen.

Now take that same report and ask which client's files the agent should be able to read, whose instructions apply and who can approve a change.

That is the business layer. I put the work into projects and workspaces, with users and permissions controlling access. The project carries the particular job, and a workspace lets a team or unit share the work that belongs together.

The report might use a company-wide template and information from one client's project. The agent needs access to both, without gaining access to another client's records just because it can read the same kind of file.

The next person needs the project history too

Imagine a client project with two junior consultants and a senior overseeing the work. One junior uses an agent to do research and prepare a report, while the other analyzes data and builds a spreadsheet.

Both need to know what the client wants, what was agreed, what has happened so far and what changed in the latest conversation. The senior needs to inspect the work and understand what they used to produce it.

If that history stays inside one person's chats or local files, the next person has to reconstruct it. Even when the files are shared, they still need the explanation of why the work is being done and what the client expects now.

That is why we built the system around shared project history, from the first client interaction with sales to the latest WhatsApp message. Meeting transcripts, notes and conversations belong with the work, where the agents helping that team can reach them within the project's access rules.

Our system also processes interactions in the background to keep useful information for later. There is an overnight process that goes through the day's interactions and builds memory at the personal, project and company levels, rather than treating everything it learns as something everyone should see.

That gives the next interaction more to work with, but finding the history doesn't settle what every record means. If a client asks for a different deadline, the agent still needs to distinguish that request from an approved change.

A person remains responsible for that decision, and the shared history needs to help them make it.

Sharing the work means deciding who gets access

Data analysis made the permission problem very clear. In our franchise structure, an agent working for one unit cannot expose another unit's financial data, sales or campaign performance.

The same separation applies to instructions. Some franchisor rules aren't for franchisees, and rules intended for our internal teams must not get mixed into work for clients. At the same time, a franchisee's agent needs the brand guidance and its own unit's rules to be useful.

So we couldn't put everything in a shared space and tell the agent to use what seemed relevant. A company-wide presentation template and a unit's financial records need different access rules, even when the agent uses both in one presentation.

That was where most of my time and effort went, building controls to keep people from seeing or changing things they weren't allowed to, even if they tried to trick the system into giving them access.

We had to build those rules into the software around the user and where the work was happening. A sentence asking the agent to keep information confidential wouldn't settle which files it could open or which changes the system would allow.

And restricting everything wasn't an answer either. Too little access leaves you with an agent that can't help with the actual work, while too much creates ways to leak information or damage something. We had to decide who could use each part and what they were allowed to do with it.

And I would separate permission to read from permission to act. Letting an agent read a client conversation to help prepare a response is a different decision from letting it send that response on the company's behalf.

As a manager, I also need to inspect what people and agents are doing, understand which uses are helpful and check whether the rules are being followed. That needs to be part of the system we ask for.

Counting conversations alone wouldn't tell me whether the company was doing better work. I'd still want to follow what came out of those conversations and what it took to get something we could use.

We stopped teaching the company rules in every chat

The clearest change came from a set of company-wide skills that taught the agent how we work, our voice and tone, brand colors, presentation templates, data analysis and reports.

Before that, we had 100 presentations in 100 different styles and voices, with variable quality and low accuracy. Afterward, we had a consistent voice, style and branding, and much more reliable quality.

People also worked faster and typed fewer instructions, because the agent had the company guidance and client history available. They could ask it to work on the project without explaining everything from the sales conversation onward.

That is the improvement I saw in our work. It doesn't mean a report no longer needs checking, and consistent branding doesn't tell you whether the analysis is right.

And when we update a shared skill, the people it is shared with get the new version, whether that is one team or the wider company.

So when the way we prepare reports changes, we can update the instructions the team uses instead of leaving each person to maintain a separate version. That gives us somewhere to put what we learn from the work, where the next person can benefit from it too.

Use the two layers to decide what to buy or build

I built because I couldn't find a system I could purchase and afford that combined the shared files, memory and permissions we needed for people to work together.

That led me to build a system that checked a lot of our boxes. It doesn't mean you need to copy our franchise structure or build all the same things, but you do need to be able to describe what working across your company requires.

I would take a recurring piece of work that passes between people, such as a client report, and sit down with the people who prepare and approve it.

Write down what the agent needs to do the job, the files, the relevant history, the instructions, the actions it needs to take and the systems it needs to reach. Include how someone will check the result.

Then work through the business layer for that same report. Which project does it belong to, who works on it, which company instructions apply, and who can read, change or approve the work?

Use that description to ask for a demonstration. Have another authorized teammate continue the job with the relevant history and instructions, and check that someone outside the permitted group cannot get that same access. That is a starting check, not a complete security review.

If a tool you can buy does the job, buying it is a valid answer. If nothing fits, you have a much clearer description of what needs building, instead of a request for an agent that somehow understands the whole company.

Either way, someone needs to own the instructions and access rules as the work changes. Whoever buys or builds the system needs people in the company who can make those decisions.

I'm also working toward an open-source release because I want this kind of AI to be more accessible to small businesses.

The agent still doesn't manage the projects by itself

The part we are still figuring out is management, having the agent proactively coordinate the next actions, tasks, communication and much of the administrative work around a project.

Helping someone prepare a report with shared instructions and history is useful today. Reliably keeping the project moving, deciding the next action and coordinating the people involved is work we are still exploring, designing and testing.