Own the box or rent the cloud?
A trained model has to run somewhere. That single choice decides two things:
- what it costs you, month after month
- where your data lives, and who can reach it
The same trade-off holds whether the model reads medical scans, inspects parts on a production line, or scores transactions in a bank.
Two bills
Rent in the cloud and you pay by the hour, forever. A comparable single-GPU VM, one NVIDIA T4, costs about $0.53 an hour on AWS, Azure, and GCP alike. Run it a working day, 8 hours across 22 days, and the bill comes to about $93 a month. Two years: roughly $2,200.
Buy the box and you pay once. A Jetson Orin Nano Super costs $249, draws 16 watts, and runs on under a dollar of power a month. Two years, all in: about $269.
| Up front | Every month | After 2 years | |
|---|---|---|---|
| Cloud GPU (rent) | $0 | ~$93 | ~$2,200 |
| $249 box (own) | $249 | ~$0.85 | ~$269 |
The rental overtakes the box in about three months. After that the gap only widens: owning runs roughly 8 times cheaper over two years. And cheap does not mean weak: that same box clears over 500 inferences a second.
Data that never leaves
Cost is only half of it. A cloud model ships every input out of the building, onto hardware you don’t control. That is what GDPR, HIPAA, and finance regulators police, and every vendor in the chain needs a data-processing agreement. On premise, the data is captured, processed, and answered in the same room. Nothing leaves, so there is nothing to vet. Our first post made this case from the security side, here it is from the cost side.
A dependency you don’t control
Renting also ties you to a company whose prices, terms, and regions can change without you. Instances get retired, rates climb, and access can vanish for reasons that have nothing to do with you: a policy shift, a licensing fight, the growing friction between US and EU tech. A service you rent today may not serve your region tomorrow. The box on your shelf runs through all of it.
Cheaper, private, and yours to keep.
The boundary
The dollars are illustrative: they assume a steady daily workload and today’s on-demand rates. Renting still wins for spiky or short-lived load, where the cloud is elastic and someone else runs the hardware. Owning wins for the steady, on-premise, privacy-bound case this series is about.
That is what we build: your model on hardware you own, in the building where your data already lives. Cheaper to run, with nothing leaving the room.
In fairness, the box’s number is hardware and power only. Owning also means setup, updates, and the day it fails, which the cloud bill quietly covers. Cheap if you already run a fleet, worth counting if no one is there to run it.