21

I´m having issues with slow internet. I have used speedtest.net at different times of the day and week and the result tells me that I get 10-20% of the bandwidth that I pay for. My ISP tells me that the promised bandwidth is an average, so a single test is not proof of a poor connection.

So now I want to conduct several tests (like every x minute) for several days while I´m away, thus giving me an average.

How do I best collect this data?

kodeaben
  • 323
  • 2
  • 5
  • 5
    Honestly, you should contact your ISP again and get to talk to someone that is more experienced. What you do is run a speedtest to get the bad speed, then call them and tell them you get it now, and let them measure the line. You can even run the speed test multiple times while you talk to them and ask them how it can be an average if you get it every time while you speak to them. – LPChip Apr 01 '22 at 06:47
  • Unless you find an API to consume of a speed test provider that your ISP trusts, you may as well do it a few times a day and write them down manually. – MyDaftQuestions Apr 01 '22 at 06:53
  • 3
    Speedtest released a command line tool a while back - download it from them and either configure a task to run it and output to a text file with `>>` or configure a script to run it and output to a text file with the same. Throughput issues are often an issue on the consumer side, as ISP bandwidth is pooled so when there is an issue ISP side, they'll generally know from complaints within a few days or so. To rule out consumer side, backup your router config, reset it to default, test, reflash it's firmware, test _(test with wired and wireless clients)_ – JW0914 Apr 01 '22 at 09:45
  • _(Cont'd...)_ Other issues client side could be too much noise on incoming line _(excl. Fiber)_, too low of a signal/bad splitter if cable broadband _(splitters aren't a one size fits all, as each has a dB rating and do different things beyond splitting the signal, so don't replace splitters unless they match exactly)_, bad cable between splitter and modem, issue with the modem _(unplug for 5m - 10m to clear it's memory)_, bad ethernet cable between modem and router, etc. – JW0914 Apr 01 '22 at 09:54
  • 1
    Who is your provider and what kind of provider are they (cable, dsl, fiber, satellite, wireless, etc). – DrZoo Apr 01 '22 at 18:37
  • 1
    One more thing to consider. Your ISP can prioritize traffic to the major speedtest sites so you might not be getting realistic results from your tests. An ISP may provide their own speedtest site but you'll have to trust them on these results. – doneal24 Apr 01 '22 at 19:20
  • 1
    What kind of connection do you have? Cable? Fiber? xDSL? Wireless? For some technologies (like dedicated fibre) the bandwidth you get is just a matter of the peering/upstream capacity. For other technologies (like cable or cellular), the bandwidth is shared among users in an area, so the advertised speed is just a maximum, but you're unlikely to ever get it. For yet other technologies (like xDSL), the bandwidth you get depends on the quality of the copper link back to the CO, with the main parameter being distance from the CO, so again, the advertised speed is the maximum. – jcaron Apr 02 '22 at 12:16
  • 3
    Key questions before complaining to ISP: 1 - What speed to they say you *should* get? 2 - Are you testing via a wired computer or a wireless (WiFi) connection? Because if you are supposed to get Gigabit speeds but you are using a typical WiFi laptop, you will not actually get anywhere close to Gigabit speeds. On the other hand, if they say you should get 100 Megabit and you have a decent *wired* computer, you should get very close to 100 Megabit in a speed test. – manassehkatz-Moving 2 Codidact Apr 03 '22 at 20:44
  • 1
    In addition to all the other good ideas, pay attention to the weather if you're on DSL. If your connection gets worse when it's wet or raining this is a tell-tale that there's a bad connection outside somewhere that's getting wet. If there is, your ISP should fix this. – J... Apr 04 '22 at 15:30

4 Answers4

22

That is a known problem, I had to talk to my ISP multiple times over 3 weeks in order to convince them that something is off with the connection. My solution was to automate speedtests so that they run every 30 minutes using a docker container made by Henry Whitacker. Set is up on a raspberry pi, an old laptop or a NAS, and let it run for a week. After that you have a very good base of measurement data to show your ISP. It even is presented in a way any normal person can understand so you don’t need to find someone more competent than the last time.

To set it up, follow the instructions on the GitHub page, I recommend using docker-compose as it is much easier than anything else. If you want to have a look at it first, there is a running demo version on Henry Whitacker‘s personal website here.

DarkDiamond
  • 1,875
  • 11
  • 12
  • 19
  • based on one of the issues on the repo (https://github.com/henrywhitaker3/Speedtest-Tracker/issues/932) it seems as though this docker container may be unmaintained. – DeveloperACE Apr 03 '22 at 15:58
  • 1
    It may be unmaintained, but probably still works. The last time I used it was in january – DarkDiamond Apr 03 '22 at 16:13
8

I'm going to answer this as an X-Y problem instead of an answer to your question, given there is already a really good answer here that adresses your question.

Theory

When an ISP tells you a certain speed is what you should get according to your subscription, this is indeed a speed that you should be allowed to get on average.

But for consumers, internet works as follows: many users share bandwidth along many households, and that is called overbooking. The idea is that most households don't use up the entire bandwidth at the same time so the speeds can be delivered when necessary, but it also means, that it may happen that lots of people do try to up- or download at the same time causing congestion which can reduce the speeds a little bit.

Because of the last sentence in the previous paragraph, ISP's will tell you that the speeds you pay for are an average and that in most cases you should get that speed.

Unfortunately, when unexperienced employees get this explained, they don't understand the technology, and all they hear is: it's an average, so a lower speed can be expected sometimes.

You have experienced a significant drop to 10-20% of what your connection should be capable of, which is far more than what normally overbooking would cause, which suggests that there's an actual problem on your line.

One thing to note is the following though: If either the up or download is fully saturated, the other will suffer as well. So if you have a 1 MBit upload and 1000 MBit download, but you fully utilize your 1 MBit upload, then the download will slow down to a crawl too. The reason for this is because every sequence of up or download will send a packet back to confirm it was received by the other party, and if this packet cannot be sent back but is queued up, the other side will wait sending you the rest of the data, and thus the connection slows down significantly.

This also means that if for some reason your upload is way lower than you should get, more towards 0.1mbit, you can bet your download will fluctuate a lot and be slow too because the confirmation packets are sent very slow too.

Actions for making troubleshooting easier

The ISP will always want to rule out any local network issues, so it is recommended to use a LAN cable directly from the modem to a laptop or PC (if its a modem/router combi) or to the router behind it and perform the speedtest then. Make sure all other devices are unplugged. If this is slow, call your ISP, because this is the ideal state for you to be in, and it's not working either.

Also, another thing the ISP will want to rule out, is services hanging inside the modem, so always turn off the modem, wait 10 seconds and turn it on again, and see if that solves your issues. When you call them, you can tell them that you already performed these actions. It will tell them that you know what you're doing and they will take you more seriously.

You can ask the ISP to measure the line from their end, and tell them you perform a speedtest with only one device connected to the modem using a LAN cable and that this should give optimal results.

If the ISP says they cannot measure the line, ask to be transferred to a person with more experience, or if they refuse to their supervisor.

If the ISP measures the line, but they claim it is working correctly, they will likely have to send a mechanic over.

LPChip
  • 59,229
  • 10
  • 98
  • 140
  • 2
    Directly connecting a PC to a modem is a bad idea all around, as they're not designed or configured to be connected directly to WAN. There are more secure methods to test and why this is still being recommended two decades later is bonkers because it's high-risk and can compromise a PC, especially when most PC have UPnP auto-enabled _(because they're not intended to be directly connected to WAN)_ - any who are aware of how much inbound WAN traffic is blocked by their router daily [thousands of malicious requests, more if the IP used to belong to a business] wouldn't consider this as an option – JW0914 Apr 01 '22 at 10:06
  • 2
    **A modem is _not_ a router** - they don't even use the same communication protocols as routers, as they're fundamentally two completely different pieces of equipment with two entirely different purposes _(they're about as apple and orange as you can get)_, and even modem-router combos keep the two separate AFAIK. Modem's have no firewall, stateful or otherwise, and only some have a very limited link-local DHCP server, with many also having a basic webserver for a status/configuration page, and the _only_ device that should ever be directly connected to a modem and exposed to WAN is a router. – JW0914 Apr 01 '22 at 10:20
  • 2
    @JW0914 I've edited the answer to clarify. – LPChip Apr 01 '22 at 16:02
  • 1
    *> If either the up or download is fully saturated, the other will suffer as well.* => FWIW, various forms of QoS at the main gateway (router) can help with this by prioritising the ACKs and preventing them from getting stuck at the end of a long queue. A lot of modern routers now support this as part of an anti-bufferbloat QoS using fq_codel or similar. – Bob Apr 04 '22 at 01:23
  • @Bob while true, it was more my point that it could be the problem. Just a checklist pointing out why a line can be slow. QoS is a way to give an alternative solution to a problem, but of course it helps to find what the route cause is and fix that instead. – LPChip Apr 04 '22 at 06:31
  • 1
    @JW0914 The Information Security SE questions [connecting a secure computer directly to the internet without a router](https://security.stackexchange.com/q/64073/244958), [Securing a modem - computer connection](https://security.stackexchange.com/q/102995/244958) and [What measures can I take to secure a MacBook Pro connected directly to a modem with no router?](https://security.stackexchange.com/q/135168/244958) do not seem to support your dire security assessment here. – Brian Drake Apr 05 '22 at 05:54
  • When I searched for “UPnP”, the first result was an (admittedly biased) blog post that claimed [**routers are part of the problem**](https://www.upguard.com/blog/what-is-upnp). – Brian Drake Apr 05 '22 at 05:54
  • @BrianDrake Routers aren't the issue, folks not taking the time to configure port forwarding are, as UPnP should never be enabled on a router, instead individual port forward rules for the specific device should be configured _(most folks will only have a handful of devices that need a handful of port forward rules, mainly for Xbox, Playstation, or PC gaming)_. This also isn't only applicable to consumers, as security researchers have been warning for over a decade that some critical infrastructure still has UPnP enabled on their systems - it's bonkers _(60 Minutes has covered this twice)_ – JW0914 Apr 05 '22 at 11:46
  • @LPChip Keyword there is a _secure_ setup _(I echoed this in a third comment I deleted after your comment it was in response to was deleted)_. OS firewalls are configured by default to allow most, if not all, network traffic as PCs and servers are intended to be behind a router's stateful firewall. Anyone can secure their firewall and OS services so it is secure to connect the PC directly to WAN, however this is impractical and inefficient for the subject matter of this answer, as a recommendation to connect a PC to a modem to rule out a router issue is bad advice from an InfoSec perspective. – JW0914 Apr 05 '22 at 11:59
  • @JW0914 oh, I did not intend this as a permanent issue, but for a simple test to see if the problem is in the network or not, ISP's often refuse to help if the pc is in a more complex network situation and blame it on: That is not our hardware. So if you connect the pc to the internet without anything else and perform a speedtest with similar results, that is something you can tell the ISP. That was my thing. – LPChip Apr 05 '22 at 12:29
  • @LPChip I understand that, and while ISPs have stated to do this for >20yrs, it's playing Russian Roulette with the security of the data/privacy on the device _(I'm not being hyperbolic - the threat can be visually seen by creating a firewall rule to log all blocked inbound traffic from WAN)_. ISPs should accompany such a request with how to secure the firewall and OS services of the OS before doing so, however even this is inefficient when simpler, more secure methods to determine if the router is the cause of an issue, such as backing up the router's config and resetting it to default. – JW0914 Apr 05 '22 at 14:10
  • @LPChip My main point is why take such an enormous risk with a device's data, and user's privacy, for something so trivial when resetting the router to default will accomplish the same end result - ruling out the router as the source of the issue. There are a myriad of ways to solve a problem, but some solutions aren't always the best way to go about it _(e.g. guns should be cleaned regularly, but cleaning a gun while it's loaded isn't the best way to do so)_ – JW0914 Apr 05 '22 at 14:16
3

How do I best collect this data?

There is a similar to speedtest.net command line tool called speedtest-cli: https://github.com/sivel/speedtest-cli. It supports several output formats such as csv or json or default verbose human readable format. You can run speedtest-cli in a loop several times from terminal:

for i in `seq 1 2`; do speedtest-cli; done
Retrieving speedtest.net configuration...
Testing from ***...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by *** ms
Testing download speed................................................................................
Download: *** Mbit/s
Testing upload speed......................................................................................................
Upload: *** Mbit/s
Retrieving speedtest.net configuration...
Testing from ***...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by *** ms
Testing download speed................................................................................
Download: *** Mbit/s
Testing upload speed......................................................................................................
Upload: *** Mbit/s

When you done, you can send all this output to ISP as a proof of possibly bad connection.

ks1322
  • 445
  • 5
  • 19
  • 2
    Speedtest released their [own](https://www.speedtest.net/apps/cli) command line tool a while back – JW0914 Apr 02 '22 at 08:38
3

If you have a linux machine available or wsl on windows (or windows itself, but it's more tedious to automate a script) I would go with iperf3.

Here are some servers to use probably test some out of these or find your own.

(in this example the script is saved under /home/speedtest/speedtest.sh)

#!/bin/bash

date | tee -a /home/speedtest/log.txt
# testing upload
iperf3 -c speedtestserver.domain.tld | tee -a /home/speedtest/log.txt
# testing download
iperf3 -c speedtestserver.domain.tld -R | tee -a /home/speedtest/log.txt

And make a cronjob */30 * * * * /bin/bash /home/speedtest/speedtest.sh

You can easily fine tune your execution plan with this site

kevinSpaceyIsKeyserSöze
  • 440
  • 3
  • 6