Sonarr copying incomplete files to Plex

hmgnsd · February 5, 2018, 6:21pm

Sonarr version (exact version): 2.0.0.5085
Mono version (if Sonarr is not running on Windows): 5.0.0
OS: unRAID 6.4.0, (Docker)
Debug logs: https://www.dropbox.com/s/122ban846qri6ei/sonarr%20debug.txt?dl=0 https://www.dropbox.com/s/cvgt7ntjonm34ht/sonarr%20access%20incomplete.txt?dl=0
Description of issue:

Hello all! I had been quite happily using Sonarr & Radarr on unRAID for 6 months or so before noticing that downloads were being added to Plex with the the exact same timestamp, 2:00 or 1:59:59. When checking, Deluge has dowloaded the complete file but whats been copied is only a fraction. The second log file demonstrates this but I am unsure on how to proceed:

“18-2-4 16:49:22.4|Error|DownloadedEpisodesImportService|Import failed, path does not exist or is not accessible by Sonarr: /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]
18-2-4 16:50:55.7|Error|DownloadedEpisodesImportService|Import failed, path does not exist or is not accessible by Sonarr: /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]
18-2-4 16:51:00.7|Error|DownloadedEpisodesImportService|Import failed, path does not exist or is not accessible by Sonarr: /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]
18-2-4 16:52:21.3|Error|DownloadedEpisodesImportService|Import failed, path does not exist or is not accessible by Sonarr: /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]
18-2-4 16:53:51.3|Error|DownloadedEpisodesImportService|Import failed, path does not exist or is not accessible by Sonarr: /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]
18-2-4 16:56:58.1|Warn|DiskTransferService|Failed to completely transfer [/data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEB-DL.AAC2.0.x264-TVSmash.mkv] to [/media/TV Kids - Animation/Marvel’s Spider-Man/Marvel’s Spider-Man - Season 01/Marvel’s Spider-Man - S01E20 - Spider-Island- Part Two.mkv], retrying [1/2].
18-2-4 16:57:01.5|Warn|DiskTransferService|Failed to completely transfer [/data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEB-DL.AAC2.0.x264-TVSmash.mkv] to [/media/TV Kids - Animation/Marvel’s Spider-Man/Marvel’s Spider-Man - Season 01/Marvel’s Spider-Man - S01E20 - Spider-Island- Part Two.mkv], retrying [2/2].
18-2-4 16:57:05.0|Error|DiskTransferService|Failed to completely transfer [/data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEB-DL.AAC2.0.x264-TVSmash.mkv] to [/media/TV Kids - Animation/Marvel’s Spider-Man/Marvel’s Spider-Man - Season 01/Marvel’s Spider-Man - S01E20 - Spider-Island- Part Two.mkv], aborting.
18-2-4 16:57:05.0|Warn|ImportApprovedEpisodes|Couldn’t import episode /data/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEBRip.AAC2.0.x264-TVSmash[rarbg]/Marvels.Spider-Man.S01E20.Spider.Island.Part.II.720p.DSNY.WEB-DL.AAC2.0.x264-TVSmash.mkv”

I realise there may be some permission issue occurring, it isn’t apparent to me where I can view permissions on an app/docker/share basis, and this setup was working flawlessly for a long period.

I can take instruction and with the help of Google hopefully learn quite quickly if anyone has the patience to help me work through this?

Thanks in advance

markus101 · February 5, 2018, 10:59pm

We’re going to need to see Debug logs of the import process.

hmgnsd · February 6, 2018, 7:59pm

Is there a way of limiting the debug log to just imports? It should be in here https://www.dropbox.com/s/dzak86eku0xbofz/import.txt?dl=0 but it’s a large file… Thank you for the reply.

markus101 · February 6, 2018, 8:50pm

No way to filter it. I’m not seeing Failed to completely transfer in that log either.

hmgnsd · February 6, 2018, 9:42pm

Sorry, I thought that log had caught it, seems there’s only the inaccessible error. I’m trying to download something and import it now to capture in the log but it’s late here (London) so probably wont be posted until tomorrow.

hmgnsd · February 7, 2018, 9:23am

ok, this should included it

markus101 · February 7, 2018, 8:36pm

What file system are you using on unRAID?
Are /data and /media both local to the unRAID server? Are they on the same volume?

Typically we’ve seen cutoff files when Sonarr is copying to an SMB share (only when running under mono), in which case we recommend using NFS for shares.

hmgnsd · February 7, 2018, 9:17pm

My cache drives are a pool of 2 SSDs formatted as BTRFS, the array is 5 data 2 parity formatted as XFS. So /data and /media are both mapped to separate shares on my unRAID array, /data is cache only /media is cache then array, and yes the shares are set up as SMB. I use macOS mainly, can this read NFS or can I run SMB and NFS? How do I tell sonarr to use NFS shares?

markus101 · February 7, 2018, 11:12pm

Having SMB shares enabled on them doesn’t matter it only matters if Sonarr is forced to copy to/from an SMB share directly, if both /data and /media are local then it won’t matter.

Is Use Hardlinks instead of Copy enabled in Sonarr?
If enabled, try disabling it and see if the result changes.

Is that time significant on unRAID? IIRC there is a time of the day it copies from the cache to the array, is that it?

hmgnsd · February 7, 2018, 11:24pm

Sorry, when I said ‘time stamp’ I meant to say that that is the length Plex is interpreting the incomplete files at.

I’ve tried it with and without ‘hardlinks’.

hmgnsd · March 5, 2018, 5:29pm

OK, so I nuked the entire site from orbit and started again, backed up media, erased array, setup as new and put Plex, deluge and sonarr back on…

And it worked

For a week

After importing tens of completed files into Plex, I’m back to only having partial transfers.

Any and all help appreciated.

hmgnsd · March 5, 2018, 8:52pm

Important info I neglected to mention regarding my post processing:

Sonarr sends torrent to Deluge
Deluge downloads to /mnt/user/downloads/Incomplete
Deluge moves completed torrent file to /mnt/user/downloads/Processing
Handbrake in a Docker watches /mnt/user/downloads/Processing then converts them putting the finished file in /mnt/user/downloads/Completed
Sonarr monitors /mnt/user/downloads/Completed for completed files, this was working fine and importing whole episodes, but now is importing incomplete files

markus101 · March 5, 2018, 11:26pm

Does the file show up in /mnt/user/downloads/Completed before the conversion is complete? If it does, Sonarr may try to import it.

hmgnsd · March 5, 2018, 11:50pm

That’s what I was thinking, but that wouldn’t explain why I t was working and now isn’t, nor why the same setup works for radarr…

I was thinking about a script or another link that would move the completed file from handbrake to /mnt/usr/downloads/Import for example, but don’t have the chops to create it.

A feature request for sonarr download handling? Only import if file size remains constant for 5 min?

hmgnsd · March 9, 2018, 10:39am

OK, so what does this mean?

Import failed, path does not exist or is not accessible by Sonarr: /data/substitute file name

[/data/substitute filename has a runtime of 0, is it a valid video file?

first it says it doesn’t have access, then that it’s not long enough to be a video file, then imports it incomplete anyway, as the file is still being written?

markus101 · March 9, 2018, 2:54pm

It means exactly what it describes.

When it first looks it’s not there
It looks and sees a file, but it’s not a valid video file
It looks sees the file, it’s a “valid” video file because media info gets a result, Sonarr imports it

Happens because the file isn’t yet converting
It’s converting, but not well formed yet
It’s converting, but not yet done, but Sonarr sees it’s viable for import

Convert then move to where Sonarr expects to find the file, that’s the only viable option currently.

Possibly in the future, but no short term plans for such a thing.

hmgnsd · March 9, 2018, 7:03pm

Thanks for the nudge in the right direction, I found and modified an unRAID mover script, which seems to work:

#!/bin/bash

if [ -f /var/run/mymover.pid ]; then
if ps h cat /var/run/mymover.pid | grep mymover.sh ; then
echo "mymover already running, sorry."
exit 0
fi
fi

echo $$ >/var/run/mymover.pid

echo “Moving files from $1 to $2”
(cd “/mnt/user/downloads/Complete/”; find -depth -print ( ! -exec fuser -s ‘{}’ ; -exec rsync -i -qdIWRpEAXogt --numeric-ids --inplace --remove-source-files {} “/mnt/user/downloads/Import/” ; ))

Prune empty directories from source dir

find “/mnt/user/downloads/Complete/” -type d -empty -prune -exec rmdir --ignore-fail-on-non-empty {} ;

Prune empty directories from destination dir

find “/mnt/user/downloads/Import/” -type d -empty -prune -exec rmdir --ignore-fail-on-non-empty {} ;

sync

rm /var/run/mymover.pid

echo “Moving all done.”

Sonarr will also remove the torrent file once my ratio is reached, how would I create a Sonarr script to remove the transcoded file from /data/ after import?

Kinda forgot this isn’t the unRAID forum, that mover script runs every hour, it won’t move anything being currently written, then there’s another script that runs that removes anything from the transcoded directory more than 1 day old. Will see how that runs

system · May 8, 2018, 7:14pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.