Completed Download Handler on Season Packs only picking up first episode or two


#1

Sonarr version 3.0.1.467:
Mono version 5.20.1.19:
Ubuntu:
Debug logs:
My setup is that I run Sonarr on one server and connect to rtorrent/ruTorrent on another. I then use an lftp script to get the files and remote path mapping to make it all work. I have lftp set to download the files with a different extension until complete so that Sonarr doesn’t try to copy them before they’re complete. This all works for individual episodes/files, but if I download a season pack, once an episode or two completes, then Sonarr will grab those two, move them and then never go back to look for the rest as they complete. Is there some way to force it to recheck after a certain amount of time?

Thanks!


#2

It’d need to keep the filename different until all files are complete or you should transfer to a different directory and when all files are transferred.


#3

Thanks, I did this, created a temp folder and move the files after complete. Seems to be working now.


#4

I’m in the same situation but I download back files with Syncthing, wouldn’t it be better that sonarr process season pack as X individuals files for import?

I mean, it’s already showing season pack as a list of individual files in Activity tab, but once one or two is processed, the whole list disappears.

For individual file sonarr handles perfectly this kind of delay (downloading back the file from the seedbox after the download), but each time I grab a season pack only first few episodes are processed (while the rest is still being transfered), I have to manual import the rest if/when I see it.


#5

are there any options in your download client so it doesnt set the job to completed until after the files have been moved to the final download location?

sonarr will only start importing once the job is flagged as completed, which means the files should be in the location it passed to sonarr and ready for pickup - if they arent then the job isnt actually completed and shouldnt have been set that way until they were

as you said, the slowness meant that the other files had not been moved yet to the download location sonarr was told to look in, so sonarr starts importing because it was told the job was complete and runs out of files to import from that folder so it ends that import and cleans up. its not going to hang around waiting for more files, they should have been there already if the job was complete.

its showing the individual files because thats a better option visually than a single download job - we dont care about the jobs per se, just the files that are in them. sonarr just asks the download client for the list of files in the job so it can display them individually.

it cant, its a single download job, when its complete then sonarr can import - how would sonarr know when a completed job isnt actually completed? how long does it stick around waiting for files that are never going to turn up?

season packs work ok in every other download client where the job is marked as completed when all the files are in the folder - why doesnt the download client you are using do that?

realistically this all boils down to the download client saying the job was completed when it actually wasnt. thats what needs to be fixed.


#6

My download client is on a distant server. Once downloaded, Files are moved to a sync folder, and the download task is set as completed. There, Syncthing (like rsync) detect a change in the sync folder and transfer new files to it’s final destination.

realistically this all boils down to the download client saying the job was completed when it actually wasnt. thats what needs to be fixed.

I’ dont agree, this problem didn’t exist with drone factory due to it’s low level logic but it worked well with delays. Download Handler is the one that ask for a job to the client, so once the job is set to “completed” it should perform integrity check of downloaded data, if it knows how many files is to be expected when a torrent files is being download, it should wait for every pieces expected.

Sonarr already handle delays between completed state and availability of a single file (the yellow/orange inbox icon). Since it knows how many file is to be expected it may handle a delay until every piece is there.

I’m currently searching for a way to lock files while being transfer, it’s a degraded method but it may stay in the tolerance zone of sonarr since sonarr can wait an unlimited amount of time after the download is completed.


#7

so dont set it as completed until the files have been moved?

any possibility of telling sonarr where the files actually are and have it move them instead?

well the issue there is that the drone factory didnt have a download associated with it, it was just an import run at intervals on a specific folder - to be fair im not sure why it was deprecated as it does cover the “weird” download requirements like this

if the download is completed then the files are meant to be in the download location already, there shouldnt be any requirement to wait, if there was then the job would not actually be completed.

your issue is that you have an extra step in between job completion and the files being in the correct location for import.

actually it doesnt. that orange icon is because it had an issue with the file(s) in the specified completed location. that issue covers a couple of things (eg, missing or invalid file) but its tried to import and failed.

it wont try again, its left in manual mode for you to click on later to try again - if you try it when the files are still syncing it will do the exact same thing as now. import what is there and clean up the download job, which isnt what you want to happen.

except the import code dosnt. it appears to be generic in that all it knows is that the job is completed and to run an import on a specific folder (to pick up every file in there) or to import a single specific file. the simplicity makes the code easier and more robust but it wont handle edge cases like this.

it still comes back to the job being marked as completed when the files are not in the correct location. i would expect that rewriting the code to handle an exact number of files would end up causing issues. this is after all an edge case, not something standard. everything else works normally except the way you have it setup - and this only fails because you’ve gotten in between the job completion and the files being in the location you want to import them from

if you dont move the files does it fail and give you the orange icon or does it clear out the job? just wondering if it might be better to sync them to a different folder and have a script that uses the API to import the files from that folder as they sync? (sort of mimicking the drone factory import functionality)


#8

so dont set it as completed until the files have been moved?

On rtorrent that’s the completed event that trigger the movment toward its final directory (the sync directory) :
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.get_data_full_path=,$d.get_finished_dir="

any possibility of telling sonarr where the files actually are and have it move them instead?

I currently map the sync folder on rtorrent side to the final local path of this synced folder in sonarr, once file is moved by rtorrent, path is update so sonarr is able to pin point the local sync folder, but there’s a delay before local sync folder is properly synced with distant folder (transfer time)

well the issue there is that the drone factory didnt have a download associated with it, it was just an import run at intervals on a specific folder - to be fair im not sure why it was deprecated as it does cover the “weird” download requirements like this

I agree, I clearly understand the benefit of Download Handler, but as you mentioned the dumb drone factory did a good job in this unconventionnal situation.

if the download is completed then the files are meant to be in the download location already, there shouldnt be any requirement to wait, if there was then the job would not actually be completed.

your issue is that you have an extra step in between job completion and the files being in the correct location for import.

Exaclty, but at least sonar should wait for every files individualy (I know you told me it expect files to be there), I mean it search for a season, it knows how many file it has to gather, but if it moves only 1 file of the season, it considers the job as done, something is missing in its logic, at least an icon or a warning.

except the import code dosnt. it appears to be generic in that all it knows is that the job is completed and to run an import on a specific folder (to pick up every file in there) or to import a single specific file. the simplicity makes the code easier and more robust but it wont handle edge cases like this.

it still comes back to the job being marked as completed when the files are not in the correct location. i would expect that rewriting the code to handle an exact number of files would end up causing issues. this is after all an edge case, not something standard. everything else works normally except the way you have it setup - and this only fails because you’ve gotten in between the job completion and the files being in the location you want to import them from

I agree, but to some extend, a finish download is an event at rtorrent side, not sonarr. Sonarr should handle a life cycle for each of it’s request (it may actually), I mean if it search for a season, it may handle sub cycle for each episode. I don’t really know how people using seedbox with a local sonarr do there stuff, I should search for that, maybe through the sonarr API.

if you dont move the files does it fail and give you the orange icon or does it clear out the job? just wondering if it might be better to sync them to a different folder and have a script that uses the API to import the files from that folder as they sync? (sort of mimicking the drone factory import functionality)

if sync is off for exemple, and files are not in the local folder, it stays with the orange icon for each episode. but once 1 or 2 episodes are moved, it clear out the job with the whole list of episode. As mentioned above, using the API may be a lead. I’m also searching a way to lock files during transfer like append a dot before the name of the transfert folder, so once the transfert is done Sonarr should see all the files spawning instantly.

the main concern that I have, is the fact that only season pack is faulty, for individual episodes, I don’t have any issue at all.