To see this working, head to your live site.

Latency and Packet Loss in the Evening

I'm seeing a consistent pattern of increased latency and packet loss every evening between ~6pm and ~12am. I do pings every minute and record the latency and packet loss.

The graphs are from the last 24 hours and you can see this spike last night. If it was just last night I wouldn't be worried but looking at the last 30 days:

You can see that up and down that is happening every single night. I've seen this pattern since Novemember 8th. I don't believe this is on my end. This runs on a wired connection from my raspberry pi to my router and out to the internet. I've never seen this pattern before.

Do you see anything on your end? Is there something that is causing this type of behavior?

Thanks,

~Jeff

6 comments

6 Comments

Jeff Whiting

Nov 29, 2023

Thank you for following up. The last two days I haven't seen it happen. It seemed to stop on the 26th of November. It has maintained (or slightly improved) the ping times. I still see packet loss from time to time, which is still concerning. This is the last 24 hours:

How this manifests to me is that things will randomly not load (or seem to be stuck) on a web page but if I refresh then it will load just fine. If something is loading it loads really quickly.

I ran mtr to google.com while I was having the issue and while I wasn't, and nothing really stood out to me as being different between the two time periods. The only thing I thought was interesting was that ip 67.199.171.241 which is before cognet seems to have consitent intermittent packet loss. My first 2 ips are always 38.78.248.1 and 67.199.171.241.

But with mtr it is hard to know what the behavior should be at each hop. Running `traceroute 8.8.8.8` I see the same on and off behavior with 67.199.171.241, where it didn't get a response the first time or the thrid but did the second.

❯ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
 1  firewalla.lan (172.16.1.1)  1.362 ms  0.539 ms  0.456 ms
 2  38.78.248.1 (38.78.248.1)  1.931 ms  1.625 ms  1.448 ms
 3  * 67.199.171.241 (67.199.171.241)  9.974 ms *
 4  * * *
 5  be3110.ccr22.sfo01.atlas.cogentco.com (154.54.44.141)  19.477 ms  19.084 ms  18.683 ms

If there is anything I should be doing specifically on my side to look at something, just let me know.

Thanks,

~Jeff

Jeff Whiting

Dec 04, 2023

Replying to

For now lets consider this issue resolved. I've seen a massive improvement since November 27th. I still think there is a something going. However it is small and infrequent. I really appreciate you looking into this and running tests on your end.

Cheers,

Jeff

Most recent graph showing the improvement. Since the 27th it has been way better.

Forum: Forum