Important: Zero Episodes and Old Episodes reappearing as Monitored and potentially downloaded unintentionally

TL;DR: There may be a lot of unexpected monitored shows and/or queued downloads. Check your Sonarr instance.

Timeline

Just under 2 days ago TVDB had another api ‘accident’ where they advertised many series as being without episodes. Given their existing 24h caching issue this problem persisted for far longer too.
This lead to many Sonarr instances removing the episodes from the UI (leaving the files in place, of course).

(I’m certain that at one point in the past we had a protection that would refuse removal of all episodes, but it evidently fell through the cracks somewhere. I’ll find out what happened with that safeguard later.)

Yesterday, given the 24h delay, I switched skyhook to another tvdb api that would allow us to bypass the 24h cache and get the episodes back quicker, hoping it would prevent more instances from being affected. This worked till about 6 hours later when the tvdb api started to experience serious performance issues. Long story short, we couldn’t guarantee that it wouldn’t happen again.
So earlier today, just before going to work, I decided to switch off tvdb sync entirely until I could deal with it in a more controlled manner.

Impact

The problem with this cycle of events is that Sonarr defaults ‘newly appeared’ episodes as monitored (although a series level monitored flag overrides that).
Also, any such episodes with airdates in the past are automatically searched if the series+episode level monitored flag is enabled. (Feature to search in case tvdb was ‘behind’)
Normally that would be fine, but in this case this leads to potentially numerous old and watched episodes from being redownloaded unintentionally, and the reason for me to pause tvdb sync for now.

Note that if the series was unmonitored, it will have stayed unmonitored and no searches or downloads should’ve been triggered as a result of this issue.

We apologize for this clearly undesirable behavior and we hope to remedy it minimizing further impact for users.

Plan

We’re considering the following actions:

  • Update Sonarr to prevent old reappearing episodes from being monitored, I’m thinking about anything older than 2 weeks.
    Done in v3, Done in v2 develop & master
  • Update Sonarr to prevent automatic search in that scenario as well.
    Done in v3, Done in v2 develop & master
  • Alternative: Preserve Monitored/Unmonitored flags for seasons and episodes in Sonarr in case they reappear. (More work, not realizable on short notice)
  • Investigate why the ‘no episodes’ safeguard disappeared in skyhook/sonarr and re-implement as needed. By for example only allowing removal of all episodes if they have no airdates or are all in the future (Upcoming shows).
  • Determine how many shows have no episodes in skyhook but have in tvdb. (To determine how many series are still affected)
    Done, 919 shows
  • Formulate a strategy to get the episodes back to the user without again causing the search. (The Sonarr updates above would do that, but not everyone gets application updates in time)
    Done, going to serve modified responses to unpatched sonarr instances

One possible strategy I’ve considered is to first have the episodes of affected shows reappear without airdates, and at a later time reintroduce the airdates. This should prevent the automatic search for such old episodes. It wouldn’t solve the monitored flag, but it should at least prevent a ton of indexer hits and possible downloads.

Updates

  • 2020-03-12: Patch went out to Sonarr v3 beta that: (3.0.3.727)

    1. Prevents the auto-search on episodes older than 14 days. This is to limit damage when tvdb completely reorders series in the future.
    2. Unmonitor old episodes when the series was completely empty. This is to prevent new monitored episodes when I turn tvdb sync back on.
  • 2020-03-12: Patched skyhook that:

    1. Prevents any future update which causes all episodes from a series to be removed.
    2. Temporarily blocks updates on shows with zero episodes currently. (basically this list) This is to allow us to turn on sync again, while giving sonarr instances time to update.
  • 2020-03-12: Re-enabled tvdb sync (except essentially, for those 919 shows)

    • (PS: Tvdb api is very very slow atm)
  • 2020-03-13: Patch went out to Sonarr v2 develop branch (2.0.0.5343) with the same fix as released for v3 beta

  • 2020-03-13: Updated skyhook to give it the capability to serve modified responses to unfixed sonarr instances. So unfixed instances will get empty episodes for the affected shows till they’re updated to the fixed sonarr versions.

  • 2020-03-13: After confirming the above fixes worked on v3 beta and v2 develop, we released Sonarr v2 master (2.0.0.5344)

Actions needed by you

Update to latest version of Sonarr (either v2 or v3 beta), Refresh all your shows, check your show monitored flags
Versions:

  • v3 beta: 3.0.3.727
  • v2 develop: 2.0.0.5343
  • v2 master: 2.0.0.5344

If you’re a ‘watch and delete’ user, then you’re very likely affected by this issue. Check your sonarr instance for unexpected downloads. Also check the shows and unmonitor as needed.

If you have a private tracker that’s now hit with an excess of downloads, please check your Ratio and HnR status. I hope that the staff of such trackers will be understanding, but please let me know if any particular tracker desires direct confirmation from us in the case of HnRs that may have resulted from this.

For myself under 1% of the shows were affected, but this likely varies per user. So any user with older unmonitored episodes should check their system.

Workarounds

  • Some users have noted that you can set the ‘Older Priority’/‘Initial Priority’ to Pause for some download clients (Sabnzbd/Nzbget/Qbittorrent/etc). This would also pause upgrades for older episodes. So it’s definitely a temporary stopgap measure you can take.
  • In Sonarr v3 you can disable the Auto Search for indexers. It’s what’s used to determine whether automatic backlog searches are allowed.
    In Sonarr v2 that option doesn’t exist separately and iirc Manual Search is used instead.
1 Like

Thanks for notifying us as I noticed a short while ago that sonarr added a list of previous past episodes to the queue. It’s a good thing I have it and the client set to start paused.

So am I understanding correctly that at the moment Sonarr is not fetching any new metadata from the TVDB? So we just have to put up with no new metadata until this is fixed?

And under the circumstances, are there any further thoughts from the dev team about the desirability of expediting implementation of an alternate metadata source, i.e., one that doesn’t regularly screw the data up?

Woke up to find out that all my disk space is used up and I went over my ISPs max bandwidth for the month so that will cost me overage fees. efffff.
I needed a good reminder that automation can run wild.

1 Like

@Hoodah ouch, setup a limit on your download client if possible.

PS: All I can say is sorry, and yes, automation can run wild.

1 Like

@olliebean It’ll be turned back on once we’ve mitigated the affected shows. tvdb is on a 24h cache anyway, so it’s not that drastic.

Will do. Looking into failsafe options now.
Looks like my isp is not charging for overages this month due to COVID-19… weird silver lining, but I’ll take it.

Sonarr has sent sabnzbd 391 previous episodes to download. Is there an easy way to delete these other than going one by one?

Bulk delete on SAB is probably best.

How on SAB? I can only find a way to delete one at a time.

On the line headed “Queue”, the far right icon; if you hover over it it says Multi Operations.

Click on it and another line will appear at the bottom of the Queue which will let you select all items and delete them.

1 Like

That was great. Thank you.

is this the indexer option?
image

is there a way to break that in two? or perhaps a global option so i dont have to edit every indexer?

i want automatic GUI searches to work but i dont want sonarr to ever be able to go off and look for things on its own (we’ve been told it doesnt so its a bit annoying to find out it actually does) - i only want rss feed “finds” to be downloaded by sonarr without any interaction from me.

its been constantly drummed into us that sonarr never automatically (back) searches, its just what turns up on the rss feeds that will get downloaded - now we know it actually will go off and do its own search under certain circumstances, so it would be nice to be able lobotomise that functionality to protect ourselves.

note - im not having a go, shit happens, but now that we know, i would like permanent protection from cases like this to be baked into the code so that even when tvdb screws up again, and they will, it can never trigger mass downloads because i will have configured sonarr to never run a non gui based auto search in the first place.

feel free to add extra precautions into skyhook or wherever, more never hurts, but ultimately users will want the ability to control their side such that no matter what happens on your side, we know that sonarr cannot run amok.

@rhom I hear you. I’ll have to think about it a bit.

Previously Sonarr would search once if a new already aired episode appears on tvdb. In case tvdb was behind. Which is what was getting triggered here.
I’ve already changed that to 14 days old max. I’m not opposed to giving the user control over that too, i’m just not sure about the right approach there.
Mostly because it’s a ‘monitor old episodes’ issue too and I rather address that.

Another user already suggested adding a permanent warning/issue about reappearing episodes, and keep them unmonitored till the user addresses the issue.
But we don’t have something like that yet, so it’s not quickly added.

There just a few edge cases, like what if tvdb is a month behind… now you have 2 episodes not being monitored. Gotta think about that.