AI Data Architecture for Commodities Firms
I speak to my AI agents constantly, so quite a bit of my personal information gets passed to OpenAI servers. Personally I don’t really care if OpenAI knows I got bloated from coconut shrimp soup, so that’s fine.
Commodities firms, however, could be feeding AI information that is far more valuable and sensitive: live positions, PnL, inventories, contracts, credit limits, etc. Some of that data are arguable so critical that they should never leave the company.
A tension naturally arises for a firm where:
- You need to give your model enough data for it to do a task well.
- But the more data you feed into a model, the greater the risk of that data being exposed, mishandled, or leaked.
i.e. You want to run AI, but you also want to control where your data goes.
This is where the concept of AI sovereignty comes to mind. In terms of data architecture, there is a spectrum of possible AI deployment options to choose from, depending on how tight you want the data boundary to be:
If you have no time to digest the whole table, the crux to understand for firms using AI is basically:
- Enterprise AI SaaS = outsource the application and inference.
- Managed AI service = control the application but outsource inference.
- Self-managed Model = control the application and inference.
Essentially, whenever you need to use a closed-weight model (i.e. ChatGPT/ Claude/ Gemini) you will need to send your data to the provider for them to run inference.
When I think about commodities firms especially, be it trading houses or hedge funds, I feel that a lot of the internal data are sensitive enough that sending it to someone else’s servers feels uncomfortable, even if enterprise AI SaaS providers all promise strict limits on how that data is handled (which on a side note, makes legal negotiations around provisions on data training/ retention/ human access in enterprise AI contracts extremely critical in a closed source world)
There is an alternative though, that does not require firms to trust external providers altogether.
A future of self-managed fine-tuned open-weight models
In my imagined ideal world, the end state would be for firms to run their own models on their own infrastructure. That would allow all data to remain within the firm’s environment, making it much easier to give models access to all the data they need to perform well.
This can be accomplished most easily by using open sourced models as a starting point, and then fine-tuning these models on your firm data. The adjusted weight models then become your firm’s proprietary edge.
Scroll horizontally to compare every field →
| Deployment | Data flow | Data boundary | How much must your firm rely on an outside company to protect the data and operate the system correctly? | Models | Who runs and maintains the AI service and the infrastructure used to perform inference? | Can the AI provider use your prompts, files or outputs to improve its models? This is separate from your firm deliberately fine-tuning a model. | Effort | Cost |
|---|---|---|---|---|---|---|---|---|
| 01Public AIConsumer tools and public APIse.g. Consumer ChatGPT, Claude and Gemini apps | ⇄⇄ | Data leaves the firm | Highest | Latest proprietary + some open models | Vendor | Depends on product, plan and settings | Very low | Subscription or usage |
| 02Enterprise AI SaaSVendor-hosted workspace with enterprise controlse.g. ChatGPT Enterprise, Claude Enterprise, Harvey or hosted Mistral | ⇄⇄ | Data enters a vendor-managed environment | High | Vendor models or vendor-selected underlying models | Vendor | No, under enterprise terms | Low | Enterprise contract + usage |
| 03aPrivate cloud: managedPrivate network path to a managed model servicee.g. Azure OpenAI or AWS Bedrock | ⇄⇄⇄ | No public-internet route; provider still runs inference | Medium — provider code and control plane | Provider catalog: proprietary and open models | Cloud / model provider | No, under enterprise terms | Medium | Usage + managed-service fees |
| 03bPrivate cloud: self-managedYour serving stack on GPUs in your cloud accounte.g. vLLM or similar on cloud GPU instances | ⇄⇄ | Data stays in the firm-controlled cloud workload | Lower — cloud infrastructure remains external | Open-weight models; your firm can fine-tune the weights | Your firm | No provider receives the data | Medium–high | GPU capacity + operations |
| 04On-prem / isolatedA firm-run model on local infrastructuree.g. Open-weight model on firm-controlled GPUs | ⇄⇄ | No external transfer by design | Lowest | Open-weight models; your firm can fine-tune the weights | Your firm | No provider receives the data | High | Hardware + full operations |
Private endpoint ≠ self-hosted. It provides a private route to a provider-managed service; it does not move the provider’s inference stack into your VPC.
Hosting ≠ training. A local or open-weight model keeps its existing weights unless you deliberately run fine-tuning or another training process.