MetadataService banner downloads failing from tvdb

Help & Support
#1

Sonarr version 2.0.0.5085:
Mono version (if Sonarr is not running on Windows):
Windows 10:
**
MetadataService
Couldn’t download image http://thetvdb.com/banners/episodes/305574/5579554.jpg for [305574][The Crown]. The operation has timed out: The operation has timed out
8:32pm

HttpClient
Failed to get response from: http://thetvdb.com/banners/episodes/305574/5579554.jpg The operation has timed out
8:32pm
**:

Description of issue: Am I the only one having this problem? Existing Sonarr setup, has been working no problem. New episodes continue to pop up and process and can add new series, so most of tvdb connectivity appears to be working.

However, all poster/banner download connections seem to be failing similar to the log excerpt. Every link in the log downloads fine in browsers but redirects to https first. Grasping at straws based on forum search, I deleted contents of MediaCover directory and updated library. Approx 5% of show posters downloaded and log fills with failures again.

I’ve tried letting it sit for a couple of days in case it was a tvdb ip throttle, but no dice and all other tvdb connections work.

There are a handful of other similar old posts, but they either realized they had been messing with DNS (I haven’t) or tvdb was down.

#2

I guess no responses does give me some information, that this is not a common problem.

Knowing that, it probably does not make sense to spend much effort troubleshooting the code and settings. Connectivity is not the issue because I can load the pages in other browsers and other tvdb functions (add shows, update episodes) work.

Does anyone have any suggestions for a uninstall/reinstall process that would give me a clean start while retaining as much of the shows database and settings as possible?

#3

Which browser did you use when you tested the image links? Try IE 11 and Edge as they use they use a similar path as Sonarr does.

Most of the TVDB functionality you see working doesn’t connect to TVDB’s servers at all, but goes through our proxy so that’s not a good indication that things are otherwise working, though in both cases, they sit behind CloudFlare.

You could backup Sonarr (and save the backup file), uninstall, remove C:\ProgramData\NzbDrone and then re-install and see, but I wouldn’t expect that to change anything.

Debug logs may be helpful as more detail is logged there (which is why we ask for them in the template).

#4

I was usually testing the links in Edge because the privacy extensions I use in Chrome and Firefox don’t play well with the log tab in the GUI. But I tested one or two in the other browsers as well. All of the links I’ve tried load immediately, but with the https prefix. That’s why I was wondering if it could be a redirection issue.

Below is a pastebin of a chunk of the debug log a few minutes after triggering the “Refresh Series” task. Sorry about the excerpt above, I wasn’t clear on what the template meant by debug logs until just now.
https://pastebin.com/J84inLUh

BTW, the issue is not significant most of the time but it causes manual imports to slow to a crawl. It waits for the banner timeout between each episode so a season import can take hours.

#5

I dug up a previous issue where the image URLs were causing issues for some people and made a change, not sure if it will help in this scenario, but those changes should roll out in the next hour or so, if you can try again.

#6

Should I be installing in a way that pulls from a dev branch to see this? I am still showing the same version (2.0.0.5085 - Dec 7 2017) with no updates available. Just in case there was something invisible to me, I did an app update/app restart/app update/refresh series cycle. It does not appear to have changed anything. Same timeout errors.

#7

No, it was a change on our TVDB proxy. The image URLs should all be http://www.thetvdb.com/... now.

#8

Not on this end. Still http://thetvdb.com/…

#9

Ahh right, the URLs are set when Skyhook (the proxy) pulls them in, so if the data isn’t changing Sonarr won’t see any changes.

Based on your logs I cleared the cache for a few series,so you may see some changes in the next hour or s on those. Try Top Gear to start with (it’s one I cleared the cache for).

#10

OK, my logs show that your update for Top Gear has taken effect (www.thedvdb.com) but the downloads are still failing. New links also work when pasted into edge.

https://pastebin.com/7NdyYGPu

#11

Have you tried loading the links in IE11 as well? IE11 shares a lot of same settings as Sonarr does, including Proxy settings, not sure if Edge does the same.

#12

Actually no, I’d never used IE11 on this machine at all until now (it asked me for configuration options when I opened it).

I just loaded one of each type of failing link (http://thetvdb.com and http://www.thetvdb.com) and got a delay followed by “Can’t connect securely to this page”. I then added both to the “Trusted Sites” zone in “Internet Options”. Then I hit reload and both images load.

So that seemed promising. But I restarted sonar, refreshed Top Gear, and I’m still getting the same download failures in the log.

#13

That’s disappointing, not sure what else to try beyond making sure IE11 doesn’t have a proxy enabled,but even then everything else works, just not this.

#14

I’ve gone down the rabbit hole with uninstalling and disabling things that could be interfering and I’m still having the problem. Below are the steps I’ve taken. After each step, I updated a series, waited for a download timeout error, and copied the link into Edge to make sure it loads there.

  • Uninstall IE + Restart Computer (left uninstalled after test)
  • Disable Windows Defender Firewall (re-enabled after test)
  • Set Internet Zone to medium (lowest option, default medium-high)
  • Add http://*.thedvdb.com to trusted zone (https was already there)
  • Internet Properties settings: uncheck “Check for server certificate revocation”, check “Use SSL 3.0” (restore all settings to default after this didn’t work)
  • Found this reference to cert issues with thetvdb on Plex forums, but the two cert authorities they use as a fix are already installed and trusted on my machine https://forums.plex.tv/discussion/252473/plex-media-server-not-downloading-artwork-yes-another-one/p2

Any ideas? Maybe the same change that pointed the proxy to www.thetvdb.com could be changed to point to https rather than http? Or would that break something somewhere else?

#15

Another data point: Within powershell, curl (alias for Invoke-WebRequest) is able to successfully download a failed link from the log and save it to a file, but there is a significant delay (2-3 minutes) before it returns to the prompt and the file is there.

I also found a few links for other software (MythTV and Plex) having similar issues but I didn’t see anything that I could try. The Sep30 date for the phase-out of the old TheTVDB API, does roughly coincide with when I noticed new shows missing artwork.
https://lists.gt.net/mythtv/users/613912
https://www.mythtv.org/wiki/TheTVDB_API_v2


https://forums.thetvdb.com/viewtopic.php?f=17&t=42599

I should also say that there are a very small number of posters slipping through and getting downloaded. Since I cleared out the MediaCover folder a couple of weeks ago, one out of every 5-10 shows now has a poster displayed in the series tab of the GUI. I want to say that half of them popped up soon after clearing the folder, with the rest coming at no discernable interval (not 1/day, one was a new series add but other new series fail). There is also no pattern in the shows with posters displayed (old vs. new, series age, date added to sonar, alphabetical).

#16

Checking in again. Logs are still filled with timeout errors. New posters are showing up sporadically. A little more than half of the series appear to have posters now. If there is an easy way to flag when a poster downloads and pull the relevant section of the log to post, I’m listening.

I also noticed I’m getting intermittent server timeout messages from my Kodi boxes during library update attempts. This leads me to think that my public IP may have been flagged somehow by thetvdb.com. I’ve tried rebooting my cable modem hoping for a new IP, but no dice. Is there a way to throttle Sonarr requests to that site for a while to see if it helps? I suppose I could block all requests at my firewall, but I wouldn’t have any way to see if the problem is gone without unblocking and triggering the flood of requests again.

Alternately, is there a way to increase the timeout period to see if the curl behavior described above is repeatable?

#17

You could try refreshing a single series, but it depends how long they are blocking for (if they are). TVDB is behind CloudFlare (as are Sonarr’s services), so it’s possible they’re blocking you for some reason.

Not without making a change and compiling Sonarr manually, as were not going to globally change the timeout for all users.

#18

Just tried refreshing a single series, with the same behavior. Requests appear to be timing out in approximately 1 minute. (Off-topic: It looks like it triggered a refresh of all series. Is it supposed to do that?)

Out of curiosity I tried the powershell invoke-webrequest method again for the URI’s. Two things:

  • When it eventually downloads the jpg, the headers indicate it is being served by CloudFlare as you said, with CF-Cache-Status: HIT
  • When I time the command, it takes exactly 210 seconds (+ 300-500ms execution) to complete every time. This holds when I repeat the download of the exact same URI, or when I change http to https. Always 210 seconds.

I tried googling for a few combinations of “210 seconds” “3.5 minutes” “timeout” “delay” “cloudflare” “ssl”, but I’m coming up empty.

I’m not looking for custom compile of Sonarr to solve this. Since I can’t know until I ask, how big a deal would it be to put in a feature request for thetvdb timeout to be configurable under an advanced setting menu?

#19

They’re scheduled, so maybe it was just time to run. Clicking it on the series details page will run it for that series, on the list of all series it will refresh all.

Not sure how much effort it’d be, but it’s not something we would do, it’s unnecessary in almost every setup, except yours for some reason and would lead to it being set without people understanding what it does (we already see that for other settings).

Since it loads at the 210 second mark every time I get the feeling there is something on your side causing that delay, because CF is unlikely to allow the connection to sit idle that long which would eat up resources needlessly (it’s a benefit for them to terminate it immediately). I don’t know what the significance of 210 seconds is, that doesn’t ring any bells.

Since you’ve replicated the behaviour in powershell are you able to replicate it on another system on your network? I’d also be curious if you can replicate it with another network connection (cellphone acting as a hotspot) to rule out your router/modem/provider’s infrastructure.

#20

OK, so I tried wget within an Ubuntu guest VM on the same machine and it loads instantly. However, it does give more verbose output and apparently is redirecting to https://www.thetvdb.com/…

When I load the above redirected uri within powershell, the delay drops to 105 seconds every time. That got me hunting again in “Internet Options” to see if there was something I could disable. No dice there. Enabling all of the riskiest settings I can find changed nothing.

On a hunch, I decided to try consecutive attempts with keepalive enabled and subsequent attempts loaded instantly. I’m not sure what to do with that information, because a series update still gets poster failures immediately afterward.

Anything else anyone can think to try?