AI Infrastructure
Run your own AI. Private, fast, and cheaper than you think.
I deploy open-source AI models on infrastructure you control, on AWS or on European infrastructure with UpCloud. You get a private, cost-controlled endpoint that your applications talk to exactly like they talk to OpenAI (for example). Same API format. No per-token fees.
The problem
Per-token pricing stops making sense at scale
OpenAI, Anthropic and Google charge per token. At low volumes that’s fine. But as your usage grows, costs scale linearly, and every prompt you send passes through a third-party server. For companies handling sensitive data, that’s a compliance risk. For companies at scale, it’s an unnecessary expense.
There is an alternative
| OpenAI API (GPT-4o) | Self-hosted (Llama 3.1 8B) | |
|---|---|---|
| Output tokens | $15 / 1M tokens | ~$0 (infrastructure only) |
| Data privacy | Third-party servers | Your own cloud account |
| Control | None | Full |
| Setup cost | None | One-time deployment fee |
| Monthly cost at scale | Hundreds to thousands | $50–150/month on AWS with auto-shutdown |
Self-hosting makes sense once you’re past the early experimentation stage and have a predictable, recurring AI workload.
* On UpCloud, for companies that can’t use US infrastructure, the same setup costs about €450 a month if the server runs around the clock (no spot pricing on the L4 plan).
Where it runs
Your choice of provider, with real numbers behind it
I’ve deployed and benchmarked the same model on both. AWS is the lower cost option. UpCloud is for companies that can’t use US infrastructure, and European regulated hosting costs more.
On AWS
34
tokens/s generation
1,093
tokens/s prompts
$0.53
per hour
- 15 GB VRAM, 4 vCPU, 16 GB RAM
- Spot instances can cut the price by 60 to 70%
- About $390 a month if it runs around the clock, much less with auto-shutdown
- Region and data residency are configurable
- Setup takes about 90 minutes
Best for: teams that can use US infrastructure, especially if they shut the server down when idle or run on spot instances.
On UpCloud, European infrastructure
50.5
tokens/s generation
3,039
tokens/s prompts
€0.616
per hour
- 24 GB VRAM, 8 vCPU, 64 GB RAM: room for larger models such as Llama 3 70B (Q4_K_M)
- EU-native: Finnish provider, ISO 27001, GDPR by default
- Predictable pricing, no spot instances: about €450 a month if it runs around the clock
- Setup takes about 7 minutes
Best for: companies that can’t use US infrastructure, such as EU data residency requirements. European regulated hosting costs more.
Measured with Llama 3.1 8B (Q4_K_M) on llama.cpp with full CUDA acceleration. The figures come from the two hands-on guides. UpCloud prices are in euros and AWS prices in dollars.
What I offer
From a running model to a managed endpoint
LLM deployment on GPU instances
A running Llama, Mistral or other open-source model on an AWS or UpCloud GPU instance, served through an OpenAI-compatible REST endpoint. Point your existing code at a new URL and you’re done.
Inference cost optimization
The right instance type, quantization level and pricing strategy for your workload. A typical 8B model runs for around $1 an hour or less and generates 34 to 50 tokens per second, enough for most production use cases.
RAG pipelines
Connect your model to your own documents, knowledge base or database with retrieval-augmented generation. It answers from your data, not just its training.
Ongoing management
Auto-shutdown when idle, monitoring and snapshots for fast restarts. Set it up once, pay only when you use it.
How it works
Four steps, and you own all of it
- Free 30-minute call. You describe your use case, I assess whether self-hosting makes sense for you.
- Proposal. A fixed-fee quote for deployment, or an hourly estimate for more complex setups.
- Deployment. I set up the infrastructure on your AWS or UpCloud account. You keep full ownership and access.
- Handover. I document everything and hand over a running system that you or your team can manage.
Who this is for
- SaaS companies with growing AI API bills
- Companies handling sensitive or regulated data that cannot leave their environment, including EU data residency requirements
- Development teams that need a private LLM for internal tooling
- Startups that want AI capabilities without long-term API vendor lock-in
Does self-hosted AI make sense for you?
Describe your situation and I’ll reply within 24 hours with an honest assessment, and what it would cost.