Fixing YouTube Throttling at the Router Level

:(

A categorical welcome to everyone. Literally on the first of August, right in the middle of the night, YouTube started lagging terribly for me. Naturally, I didn't like this at all. Well then, let's figure out why and how to fix it within my personal network.

What happened?

What happened is well described here, on Habr. In short, in my own words - during the SSL connection setup, the domain we are connecting to (the so-called SNI) is transmitted in plain text. And if it's googlevideo.com, then "interesting things" start to happen. You can check this locally with the commands from the article.

$ curl https://speedtest.selectel.ru/100MB -o/dev/null

In my laptop's terminal

As you can see, the download speed is quite decent. Now let's try to connect to the same server, but passing a different domain during the handshake.

$ curl --connect-to ::speedtest.selectel.ru https://manifest.googlevideo.com/100MB -k -o/dev/null

In my laptop's terminal

And that's it, we're done for...

What to do?
Obviously, we are being throttled by domain using DPI. There are already quite well-known projects, like Goodbye DPI. But it's for Windows, which I don't have at home at all, and it requires installation on a single computer and doesn't allow me to fix my entire home network. Thanks to the author, right in the readme he has collected a list of projects that do similar tricks for other operating systems. The whole point of these projects is to somehow distort the domain string transmitted in plain text. To explain it in very simple terms, in the case of an open HTTP request, you change the Host header, for example, like this:

Host: googlevideo.com
vs
hoSt:googlevideo.com  # поменян регистр и убран пробел

And those strange contraptions doing weird things won't be able to read it correctly, analyze the traffic, and do something. In the case of a TLS connection, as I understand it, the TLS packet is simply taken and split into several TCP packets, so that parts of the domain string are in different TCP packets. And at the same time, they change the order of the TCP packets, sending the end first, then the beginning. The TCP protocol itself allows this, and everything will work correctly on the destination server. But the wonderful pieces of hardware are too stupid, and their authors clearly haven't read the spec. Such a cute competition.

As a reminder, I wanted to fix my entire home network, and to do that, it needs to be done right on the router. My router is a Zyxel Keenetic Giga III, which is probably about ten years old... Keenetic has already become a separate brand without Zyxel, and my router has been working like a clock all this time. I started thinking about upgrading and buying a router on which I could install OpenWrt. The manual that follows is valid for any Linux-based router that has iptables and init.d.

After a little reading, what was my surprise to find that on a rather closed Zyxel, you can install a set of Linux binaries and the OPKG package manager from OpenWrt. The router was quite expensive, I didn't pay for it for nothing, after so many years it was able to surprise me. The next couple of paragraphs can be skipped if you don't have a Keenetic. Now we will run OPKG on my router. There is actually a whole manual from the manufacturer on how to do this. I will describe it very briefly.

First, you need to go to the web interface and install all the OPKG components.

Router UI where we install the native firmware components

Internal memory, as is usually the case in such devices, is next to none, barely enough for the firmware. No problem. We take a USB flash drive, format it to EXT4. In the root, we create an 'install' directory and put the set of binaries from the official site (more precisely, from the link to the entware project in the official documentation). In my case, for the mipsel platform - https://bin.entware.net/mipselsf-k3.4/installer/EN_mipsel-installer.tar.gz After browsing https://bin.entware.net/ you can find a similar installer for ARM or big-endian mips, if you have some tricky keenetic. Next, we plug the flash drive into the router, and if all is well, the router will unpack and delete the archive itself. And we will be able to connect via ssh to port 222.

By the way, if you choose the wrong platform, it's better to just go to the router's ssh on port 22 and try in the console exec sh .

If it's the wrong platform, the error will be clearly visible. I didn't immediately understand what processor I had, so I had to try them all.

If the file exists and is for the wrong platform, you get an 'exec format error'. If the file doesn't exist, it clearly says it's not there.

The password for ssh login on port 22 is the same as in the web interface. On port 222, the login/password is root/keenetic. After the first login, don't forget to do passwd and change the password. If everything was successful, then either via an ssh connection to port 222, or via exec sh we finally get an almost full-fledged Linux on the Zyxel. Hooray! And a huge thank you to the manufacturer for this feature.

And here we are on the router, in Linux

And now for the part that is common for routers with OpenWrt or a similar Linux. All we have left to do is install a piece of software that will twist and turn and want to trick TCP packets and wrap all traffic to this software. There's plenty of such software. First, let's install what we might need, in my case:

# opkg update
# opkg install ipset curl gzip grep git-http

If you are low on space, you don't have to install anything. You can just scp the binary from your computer, and that's it. Something like this scp -P 222 ./tpws root@192.168.0.1:/opt/root/ .

Now I started choosing a project that would run. I quite liked this one. Simply because it compiled from source right on the router with no effort, started up, and worked. But, unfortunately, it only provides a SOCKS proxy, and I have no desire to configure every device on my home network. Moving on, I found the project zapret. It looks very cool, but its automatic installation scripts started throwing too many errors, more than I'm willing to deal with to get the project installed properly. And the script code is also quite complex. Oh well, a quick read of the readme told me that there is a binary that does everything needed, it's even available pre-compiled right in the repo for different platforms. That's the one I need. As for the scripts, well, I'll do everything myself without them. So, here's what's needed:

# opkg update
# opkg install iptables
# mkdir git
# cd git
# git clone https://github.com/bol-van/zapret.git
# cd zapret
# ./install_bin.sh
# cd /opt/etc/init.d/
# vi S51tpws

And then insert the service code that I wrote.

#!/bin/sh

SCRIPT=/opt/root/git/zapret/tpws/tpws
PIDFILE=/var/run/tpws.pid
ARGS="--daemon --bind-addr 192.168.0.1 --port 999 --disorder --tlsrec=sni --split-pos=2 --pidfile $PIDFILE"


start() {
  if [ -f $PIDFILE ] && kill -0 $(cat $PIDFILE); then
    echo 'Service TPWS is already running' >&2
    return 1
  fi
  $SCRIPT $ARGS
  iptables -t nat -A PREROUTING -i br0 -p tcp --dport 80 -j REDIRECT --to-port 999
  iptables -t nat -A PREROUTING -i br0 -p tcp --dport 443 -j REDIRECT --to-port 999
  echo 'Started TPWS service'
}

stop() {
  if [ ! -f "$PIDFILE" ] || ! kill -0 $(cat "$PIDFILE"); then
    echo 'Service TPWS is not running' >&2
    return 1
  fi
  echo 'Stopping TPWS service...'
  kill -15 $(cat "$PIDFILE") && rm -f "$PIDFILE"
  iptables -t nat -D PREROUTING -i br0 -p tcp --dport 80 -j REDIRECT --to-port 999
  iptables -t nat -D PREROUTING -i br0 -p tcp --dport 443 -j REDIRECT --to-port 999
}

status() {
  if [ -f $PIDFILE ] && kill -0 $(cat $PIDFILE); then
    echo 'Service TPWS is running'
  else
    echo 'Service TPWS is stopped'
  fi
}

case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  status)
    status
    ;;
  restart)
    stop
    start
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|status}"
esac

Next

# chmod +x /opt/etc/init.d/S51tpws
# /opt/etc/init.d/S51tpws start

Let's measure

On my laptop, after starting the service on the router

We try YouTube manually and see with our own eyes that it has started working veeery fast.

A few explanations. The code above uses only one binary from the zapret project - tpws . It has a transparent proxy interface that mangles packets to trick DPI. This binary has a bunch of flags for starting. In the init.d service code, I specified the launch arguments that work for me. You can remove --daemon and just run it in the terminal, playing with the parameters. Don't forget to add routes to your service in iptables before running it manually. In my case, the routes are added and removed right in the service code.

Specifics. For your case, you might need to make changes to the service code in these places. For some reason, init.d on the Zyxel is located in /opt/etc. On a bare OpenWrt, it's most likely in the root /etc. The S51 prefix of the service file S51tpws is taken from somewhere in the guts of entware, without this prefix the service won't start automatically on boot. Again, on a bare OpenWrt, this is probably not needed. br0 - is the name of the local network interface, on which my router's IP is 192.168.0.1. So that the tpws service listens only on this interface (and not on the external one as well), I specified it among the service launch arguments.

So, that's basically it. Now all traffic going to external ports 80 and 443 from my network is processed and mangled by the tpws service. I think it's no secret if I say that YouTube will most likely be completely banned in the foreseeable future. Based on this, this solution is temporary. Or, perhaps, in the future there will be another binary that will send all traffic via ICMP to my private virtual machine somewhere in the Netherlands... The main thing is to get started, and I really like this setup on my home router. Thanks to everyone who read to the end.

p.s. I give full consent to reprint this article in any source. I declare it public domain. Do what you want, but it's better to improve and develop it :)