Feedback

Building a Steam Caching Server for Faster Game Downloads

Author:
Manufacturer: PC Perspective

I Have a Need, a Need for Download Speed

Thanks to Wendell from Level1Techs for his all of his help on this project and pointing us in the right direction!

A few years ago, we were fortunate enough to get a fiber internet connection installed at the PC Perspective office. Capable of 1Gbps download speeds and about 250Mbps upload, we were excited at the possibilities that laid ahead.

However, when you have access to a very fast internet connection, you begin to notice that the bottleneck has shifted from your connection to the servers on the other side of the content delivery networks (CDNs) that power the internet. While these CDNs have very fast links to the internet, they generally limit bandwidth so that there is more speed to go around to multiple people at the same time.

A look back at what once was

One of the services that we found would max out our connection was Steam. Since we download a lot of PC games at the office, it was a nice benefit to have an internet connection as fast as our NICs could handle, and that the Steam CDNs would serve us at our maximum potential. In fact, the bottleneck shifted over to storage performance, as the random writing nature of Steam thrashed our SSDs at the time.

View Full Size

By no stretch of the imagination is 60MB/s slow.. but what happened to our 100MB/s!

Unfortunately, this has ceased to remain the case. At some point, Steam downloads started getting slower on our same internet connection. Not only did storage utilization during a Steam download start to increase, but also CPU usage, pointing to a potential change in how Steam distributed their data. While downloads on our high-end systems fell to around 50-60MB/s, systems with less CPU horsepower started to see speeds fall to 20-30MB/s. All hope was lost for fast game downloads.. or was it?

Recently, Wendell from Level1Techs mentioned on Twitter that they were running a local Steam caching server on their network with great success. After some guidance from Wendell, we decided to tackle this project and see if it would help our specific scenario.

Continue reading our experiences setting up a Steam cache!

The Setup

The theory behind caching Steam downloads is simple. While you might not get the appearance of it from the Steam client, all that's happening is a bunch of HTTP requests to Valve's CDN. Simply speaking, this means, that once you know the addresses of the servers Steam uses for downloading, you can monitor the traffic coming from them, save the data to a cache file, and then when it is needed again, simply deliver the local copy instead of going out to the internet to fetch it.

In order to be able to hijack this traffic, you need to intercept the DNS requests for Valve's CDN and point them to your local HTTP server configured for delivering the data.

View Full Size

The steamcache project on Github does just this, through the use of preconfigured Docker containers. 

To accomplish this, you need to run two Docker containers. The first is the steamcache "generic" container, which provides a preconfigured nginx-powered HTTP proxy to intercept and store the relevant data.

Second, you need to be running the steamcache-dns container to redirect the DNS requests to your local server. This requires your computer sending all of it's DNS requests through to Docker container. This can be configured either by changing the local DNS server configuration on your computers or by changing the DNS server configuration in your router.

Running a local DNS server will provide additional speed benefits to activities such as web browsing since your computer no longer needs to make a request to an external DNS server for every possible HTTP request. This means a lower access time for all HTTP requests.

We aren't going to get into the minutia of configuring Docker for people who have never used it, as there are much better tutorials for that on the Internet by people who understand the process way better than we do. The notes on the steamcache website are a good place to start if you are thinking of doing this yourself. 

It's worth noting that the steamcache generic container provides support for a multitude of other download services, like Uplay, Origin, and even Windows Updates. For now, we are sticking with strictly caching Steam downloads. We don't download a lot of games from Uplay or Origin, and when we do, they still manage to fully utilize our 1Gbps internet connection.

As time goes on, and the Steam caching proves to be stable, we might look into adding more cached services.

The Hardware

View Full Size

In order to run the docker containers, we turned to our new Ryzen 7 1700-powered QNAP TS-877 NAS (full review on this to come soon!). While it might seem like a bit of an oddity to run this sort of thing on a NAS, the desktop CPU, 10 Gigabit NIC, and available 2.5-in drive bays for SSDs made it a great pick.

For the cache storage, we are using two 1TB Samsung 840 EVO SATA SSDs in a RAID 0 configuration. While we wouldn't normally recommend a RAID 0 configuration in production, all this will be doing is holding a download cache of files we can easily get back if needed, so we aren't necessarily worried about the potentials of data loss.

Our Linux environment is provided by an Ubuntu 16.04 VM running in the KVM-based QNAP Virtualization Station. While the QNAP has native support for Docker containers running outside of a VM, we wanted to be able to move the VM image to other hardware if needed, and the QNAP Container Station software didn't seem to be very well documented at this point.

Results

With our Steam caching server in place, we first started by downloading several games to a local machine. Since this is the initial download, and there is no data already cached, it will not be any faster than downloading games without the cache enabled. However, in our experiences, this seeding of data to the cache also isn't any slower than the normal process of downloading the game.

To evaluate the performance of the cache, we then downloaded the same games on another PC. In order to eliminate any potential performance impacts on our client, we used an overkill system with an 8-core/16-thread Core i7-5960X CPU, 10Gbit NIC, and an Intel Optane 900P SSD.

View Full Size

With a hard limit of about 250 MB/s (!!!), now, the bottleneck seems to be the on the server side. When downloading at around 250MB/s, the Ryzen 7 1700 CPU in our QNAP NAS seems to be maxing out a single thread of the processor, within our Ubuntu VM. 

While we could move this setup to yet another dedicated server with faster single-threaded performance, for now, we are happy with our results. Considering most of our machines aren't equipped with 10Gbit NICs in the office yet, being able to max out our Gigabit connection again is good enough for now.

It does seem like a bit of overkill for an office of less than 5 people to have a Steam caching server, but given what we do it make sense for us. For organizers who are running a LAN party, this would be a great solution to prevent game downloads/updates from saturating your link to the Internet.


May 11, 2018 | 08:44 PM - Posted by Matt (not verified)

I'm a little unclear why a Steam-specific cache was chosen instead of just a regular http caching proxy that would take care of caching anything http without having to configure specific services.

May 13, 2018 | 07:46 PM - Posted by John Blanton (not verified)

Are you trying to download the entire internet, or just play some games?

May 14, 2018 | 11:17 AM - Posted by Anonymous2 (not verified)

Point is, you can more easily set up a less-specific web cache server that will not only maintain cache of all Steam games, but also Origin, Uplay, YouTube, Netflix, OS updates, and more. You can control what gets cached, not cached, or even blocked.

May 14, 2018 | 03:38 PM - Posted by Matt (not verified)

Would you like faster Steam, or faster Internet which includes faster Steam? A traditional cache would reduce your download for the Internet as a whole. Why limit the speedup/download-reduction to only Steam when it is actually EASIER to reduce download and speed up everything?

The one catch is that caching everything would mean more disk space, but if you have enough disk space to cache a lot of Steam content, you probably also already have enough disk space to cache years of non-Steam and non-video content.

However, I will say that the general trend of the past couple years is "https everywhere", which stops proxy caches from being effective for that content. Before reading this article, I would have guessed that Steam is surely https as well.

May 11, 2018 | 09:00 PM - Posted by sircod

There are also some Docker containers for unRAID now, so it is pretty easy to set up on an existing server if you have one already. Guide here: https://squishedmooo.com/how-to-install-steamcache-on-unraid/

For some reason mine is not working for steam, but it is working for battle.net and Origin. Not sure what the problem is.

May 12, 2018 | 01:50 PM - Posted by sean (not verified)

Why didn't you just backup steam games to a local file server and load them over 10gb lan?

May 13, 2018 | 01:27 PM - Posted by anon123 (not verified)

Yes, exactly what Sean said. It's a far more convoluted process to build a server to intercept steam requests than it is to just use the backup / restore feature that's built into the client. You don't need any fancy software and you can save the files anywhere you'd like.

May 14, 2018 | 05:10 AM - Posted by malurt

One reason to do this is for instance if you're hosting a LAN party and you don't know what games you're going to be downloading before it's underway. The first download of any game will be "slow", but the following downloads will not stress the internet connection and it causes no disruption of the user experience for downloading a game from steam. Which is arguably just as important as most people are not tech savvy enough to be able to just "connect to this network location and use the steam restore feature on it".

We've been doing it for years in our group, and it works really well for the purpose. And we don't want to have all that data backed up.

May 24, 2018 | 10:26 PM - Posted by jcastro (not verified)

So instead of fixing the problem in one place with 2 commands you'd rather go to each individual client and run a manual backup/restore? That's way more convoluted.

May 12, 2018 | 09:06 PM - Posted by Cyclops

PCPer personal are the most relatable bunch of people in the world.

"Do you also have a gigabit connection? Are you also suffering from bellow gigabit download speeds? We have the solution for you!"

Jokes aside, I also suffer from having a gigabit connection. I'm glad this article addressed us 1% ers.

May 14, 2018 | 05:17 AM - Posted by malurt

This article actually addresses way more users.

Consider my case: We host a LAN party about 3 times a year. Something like 20 people show up, we try to have people agree on what games to play beforehand, but there's always someone who hasn't downloaded their game, there's always some other game that noone thought of, or a free weekend of something that we want to try out.

So now we're on a 50mbit connection and 20 people that need to download a 20gb game... well, one person downloads the game, the other 19 people wait till the download is done and then use the LAN cache and get around gigabit speeds from the cache. And they don't need to do anything special, like access a network location and use the restore feature in steam. No, just click "install" on their game in steam and everything just works like normal.

You don't even need 20 people or a fast connection for it to be a cool idea. The fewer you are arguably it becomes easier to do something else, like direct network share, but there's a point where LAN-caching makes perfect sense.

May 13, 2018 | 12:51 PM - Posted by RadioActiveLobster

I'd kill to have a gigabit connection.

It sucks being 100+ miles from the nearest ISP that offers that service.

September 18, 2018 | 08:06 PM - Posted by Benjamin L Sheppard (not verified)

I think its worse to be 1 mile from it and be unable to get it. Like me

May 15, 2018 | 02:01 AM - Posted by odizzido2 (not verified)

I just copy my games from drive to drive /shrug

May 21, 2018 | 05:00 PM - Posted by razor512

Do the caching servers also cache and keep track of updates? if not then it will not work for many games that are regularly updated, e.g., a few months old copy of ark survival evolved, will mean a 50+GB download due to how they update their game.

May 24, 2018 | 10:24 PM - Posted by jcastro (not verified)

Yes, it keeps track of everything basically, so if you only need one patch you're fine but if your friend shows up and hasn't update their laptop in 6 months it will also have those in the cache.

May 25, 2018 | 10:08 PM - Posted by Rob (not verified)

You know what would be a nice addition for those of us that are really lazy... a link to the time code in the weeks podcast in each story you talk about.. somewhat reverse of what's in the podcast linking back to each story.

September 1, 2018 | 03:15 PM - Posted by Gamer lounge (not verified)

This solution is great!
But unfortunately, it will not work for “epic games” like fortnite but only for games hosted on steam.
Is there any alternative, like creating an update patcher? It would be great of a solution is posted for this issue.
Thanks in advance.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.