5 Lessons Learned Building MCPs

MCP sounds like a brilliant idea that can be implemented in an afternoon. But once you start building it, especially in an enterprise app setting, rather than a consumer-level application, all kinds of difficulties crop up.

These are some lessons learned from those difficulties.

1 APIs Need to be Simplified

Enterprise-level software usually has too much going on a single screen. And each screen calls countless granular APIs to drive a cohesive outcome. For example, when creating a marketing campaign, there would be separate APIs for the campaign, its variants, the frequency-capping settings, and even the 3^rd^-party service provider to use for that particular campaign.

Thus, it’s not as simple as saying, “create a campaign.”

For that reason, there needs to be a simple wrapper in front of the core APIs, one that takes some of these default decisions. Otherwise, the LLM would have to orchestrate several API calls just to get a single thing done. And also discover the right orchestration in the first place amongst dozens of granular APIs.

This step usually needs both the Engineering and the Product teams to come together.

2 Writes Are Not Straightforward

There’s usually no problem reading and writing via APIs, even complicated ones. Because the orchestration is coded once and used deterministically many times. However, granting write access to LLMs is not as straightforward.

Any kind of hallucination or wrong decision can lead to changes you never intended to make. For example, if you want to modify Campaign A, then your deterministic code most likely will modify it, or fail to modify it. But an LLM can modify Campaign B, and also delete Campaign C for whatever unknown reasons (like the stories of it deleting files on a computer).

Safely having LLMs modify things means one needs to be able to stage these changes, or at least make them reversible. Even the seemingly “simple” implementation of soft-deletes has its own array of problems.

The interim solution is to ensure that the API adapter grants only restricted write/modify access and is stricter about what can be modified. Maybe certain API keys cannot even make certain categories of modifications.

3 Authentication & Authorization Needs to be Clearly Thought Of

API keys, especially in an enterprise setting, might have to be per-user and per-intention, rather than just per-organization. MCP APIs, then, don’t have the same privileges as Developer APIs.

First, you need separate API keys for audit, because the APIs would be used by potentially every end user, rather than for server-to-server communication. Second, you don’t want people making accidental changes in areas they never intended, just because the LLM thought so. That’s where having API keys that compartmentalize actions comes in handy. This compartmentalization would likely differ from role-based or object-based access control, which is generally targeted at end users rather than LLMs.

It might also become necessary to implement a system that detects rogue patterns in MCP calls and short-circuits the LLM with a helpful message rather than letting it do whatever it wants.

4 It’s Very Difficult to Get Data Out

One big assumption is that one can just point Claude to an API and it will “analyze everything.” But what if there are millions of rows out there? Would Claude spend time paging through all of that? Or maybe demand a huge export that it starts analyzing, a different way each time?

APIs usually expose two kinds of things:

1. Raw data.
2. Or some fully aggregated data for end-user consumption.

However, what LLMs usually need sits right between the two. Not very raw, but it does not have to be fully aggregated. It can be in some form in which further analysis can be performed as needed, but the output should not be so big that ad-hoc analyses become infeasible.

This is not as simple as applying the same “API pattern” to all use cases. It needs to be considered from a use-case perspective. That’s what makes “getting data out” to an LLM difficult.

5 Your Tools Might Need to be Agents Themselves

One should not focus too much on MCP alone, treating the API as a dumb, deterministic tool used only for data access, while the client LLM does “all the magic.” Another approach is to have an agent endpoint too, and have the client LLM talk to it.

Many of the problems we discussed earlier can be tackled using this. Can’t run ad-hoc analytics on big data? Well, the server-side agent might be able to handle it better and pass on sophisticated results to the client without having to move all data to the client.

The client LLM can also issue commands in natural language (instead of overly structured queries), which the server agent can translate into appropriate outcomes. This way, not every aspect of the responsibility falls on the client-side LLM, and the client-side LLM also does not have to be overly powerful to do complex coordination. Maybe the on-device language model itself can use complex APIs if those APIs expose an agentic API rather than just data-access APIs.

That’s where I think the future might be going. But we’ll see how this all pans out.

Conclusion

That’s all for this week! See you in Issue 11.