Leftover files not being deleted

Sonarr version (exact version): 2.0.0.3953
Mono version (if Sonarr is not running on Windows):
OS: Windows 8.1 Enterprise x64
Debug logs (posted to hastebin or similar): N/A
Description of issue (if you think you’ve found a bug please include steps to reproduce):

So here’s my setup:

1.) SAB has a tv category that saves usenet files to D:\raw\TV
2.) Sonarr is pointed to the tv category of SAB to segregate the downloads from the rest

I was under the impression that when Sonarr does its post-processing/importing, it should not leave any file behind in the D:\raw\TV folder, right? From time to time, I keep getting multiple folders of the series that I downloaded there, most of them contain the MKV itself, the nfo for the release, and some par2 files. Now when I check the import destination folder of these series episode files, the downloaded files are completely post-processed and are there just fine. So why are there leftover files especially MKV files? Are they duplicates or what?

EDIT1: It would be worth it to note that most of the MKV files that were left over are of lower quality than those of the MKV files already present in the import destination folder of the series.

EDIT2: Some of them are samples files. So when I see an orange entry in Sonarr’s queue and then chose blacklist this release, it doesn’t delete the files themselves from D:\raw\TV ? If this is the case, then this behavior makes sense as Sonarr didn’t post process those files.

Lower quality files are not imported and are not automatically deleted.

Sonarr tells the download client to remove the download and the files when its removed via Sonarr’s queue.

When Sonarr successfully imports all the video files in a folder it will remove the folder (for usenet clients), or for torrents it will tell the download client to remove it once Sonarr sees that it has finished seeding.

What do you mean lower quality files are not imported? You mean when a lower quality and a higher quality file both downloads at the same time it won’t do any importing on the lower quality one?

Ok, so if it is removed from Sonarr’s queue why do I have these leftover files?

If it was removed through Sonarr manually, clicking remove (regardless if it was blacklisted), then Sonarr tells your download client to remove it, which it should remove it from it’s queue/history and remove the files from disk (Sonarr tells the download client to remove the files). If nothing was done and it fell out of Sonarr’s queue because its not one of the last 30 items in history then it gets left there.

When you say last 30 items in history, are you pertaining to SAB’s history? Correct me if I’m wrong but are you saying that if it stayed in Sonarr’s queue for a long time and nothing was done until it is not in the last 30 items of SAB’s history then it pretty much stays and limbo causing this issue?

Yes, SAB’s history. With SAB 1.0 thats the history for the specific category that Sonarr is using, prior to 1.0 (0.7.x) its all items in SAB’s history.

Your understanding is correct.

Ok, got it. That explains why this issue mostly happens when there are a lot of ongoing downloads in SAB. Thanks!

Why do you think that some of the leftover files are full-sized MKV files?

It can happen if multiple qualities are grabbed quickly and the better quality download finishes before the lower quality or at least Sonarr imports it first because it hadn’t tried processing the lower quality release.

Ok I just had a sample downloaded today by Sonarr/SAB and as expected stayed in the Sonarr’s queue and was orange. I manually deleted it from the queue and blacklisted it also. It did remove it from SAB’s history BUT did not remove the directory itself on D:\raw\TV . Any thoughts?

From Sonarr’s side it would be good to make sure that we’re asking SAB to remove the files, debug or trace logs should show the URL used to make the request. If Sonarr is making the request SAB should be honouring it.

Specifically its del_files=1 that needs to be set, from what I can tell it should be set properly.

Should I recreate the issue with trace logging turned on then?

You’ll need to to see exactly whats going on.

I was trying to recreate the situation while trace logging is enabled in Sonarr and debug logging enabled in Sab but Sab sees the sample release as “aborted, cannot be completed” this time around. Sonarr detects this as a failed download, gets automatically removed and blacklisted from Sonarr’s queue but it remains in Sab’s history. I’ve tried it twice already with the same results. This seems to be the problem on the old thread that I posted here. Here are the trace logs:

Sonarr: https://www.dropbox.com/s/le73zf0af06x6tw/logs_sonarr.zip?dl=0
Sab: https://www.dropbox.com/s/4cyw51h4f4skz2f/sabnzbd.log?dl=0

The release name is “Scorpion.S02E21.1080p.HDTV.X264-DIMENSION-Obfuscated”

SAB logs show that it can’t be removed:

2016-03-30 08:08:28,153::INFO::[misc:1283] Cannot remove folder \\?\D:\raw\Default\Scorpion.S02E21.1080p.HDTV.X264-DIMENSION-Obfuscate\__ADMIN__ Not sure why, but that looks like the cause.

Ok, so there are no problems from Sonarr’s side because it did send out the command to Sab. If they remain in Sab’s history, why were they successfully removed from Sonarr’s queue? I thought Sonarr’s queue is just querying Sab’s history?

I also don’t understand why that log is pointing to the “D:\raw\Default” folder. That folder is my temporary download folder set in Sab. I have a “TV” category set that has a folder path of “D:\raw\TV” and Sonarr is pointing to that category. As I understand, if a download in Sab is set to the TV category, it’s temporary download folder would be whatever folder path set for that category and in this case “D:\raw\TV”, correct?

I was just told in the SAB forum that all jobs start in the temp download folder and can only delete items from that folder alone. So if a sample file is downloaded, it starts from the temp download folder (D:\raw\Defualt) and ends up in the category folder path (D:\raw\TV). In this case, when you remove and blacklist the sample release from Sonarr’s queue, it will send a request to Sab to delete it from its history BUT it won’t be able to delete the item because it is no longer in the temp download folder. What are your thoughts?

Any updates on this?

I had a new occurrence of the sample problem and the same thing happened. After deleting it from Sonarr’s queue, it was deleted from SAB’s queue but the actual folder was never deleted in D:\raw\TV . As I mentioned above, I think this behavior is to be expected because SAB can only delete in the temp download folder (D:\raw\Default), right? Isn’t this an issue then?

There was another problem also wherein a certain release was downloaded with two qualities (720p HDTV and 720p WebDL) at the same time. The 720p HDTV folder was left and never deleted in the D:\raw\TV folder. I checked the completed download time of both and naturally the 720p HDTV was downloaded first and the 720 WebDL second. In this case, at the exact time the 720p WebDL was released and sent to SAB the 720p HDTV was still downloading. so in the end, when both were already downloaded Sonarr only post-processed the 720p WebDL because that’s what it is expecting and left the 720p HDTV alone. Isn’t this an issue also?

Do you have a link to the SAB forum post you’re referring to?

Definititely an issue, SAB Is telling Sonarr it did something, but in reality it didn’t. I think I know why they’re doing that, but they shouldn’t tell Sonarr the request was successful if it wasn’t done as requested, at least if it failed we’d know and not have some mystery to look into.

Both are viable to import, but it comes down to timing, if Sonarr saw the WEB-DL was available to post process before the HDTV release then it will import the best release.

In what sense? We’ve been hesitant to remove lower quality releases from the queue once they are started to give a better chance of success, especially when dealing with propagation issues and take downs.

Filed an issue for us to take a look:

Here’s the SAB forum post: http://forums.sabnzbd.org/viewtopic.php?f=2&t=19692&sid=bf4375bcc13e4e4834360eae2b3ea440&start=15#p105235

What do you mean if Sonarr saw the WEB-DL was available to post-process before the HDTV release? At what circumstance will that happen? The HDTV release completed 8 hours earlier than the WEB-DL in SAB.

Are you saying that when Sonarr finds a better quality WHILE a lower quality is still downloading, it won’t right away remove the lower quality from its queue? So in the end, what will it post-process?

Thanks for filing the issue. I think this had been the behavior of SAB even in the past. I noticed in the past that whenever a completed download is already moved out off of the temp download folder, the “remove NZB and delete files” button is greyed out.