Sonarr deleting files on transfer to Amazon Cloud Drive (via acd_cli FUSE)

[Fozz]

Sonarr version (exact version): 2.0.0.4326
Mono version (if Sonarr is not running on Windows): 4.2.1 (Debian 4.2.1.102+dfsg2-7ubuntu4)
OS: Ubuntu 16.04
((Debug logs)): https://drive.google.com/file/d/0B38b9TU5d5BSaHdSZm5jNWQzUnM/view?usp=sharing
Trace logs: https://drive.google.com/file/d/0B38b9TU5d5BSMjlKYjc5TU00LTA/view?usp=sharing
(Make sure debug logging is enabled in settings and post the full log to hastebin/pastebin/dropbox/google drive or something similar, do not post them directly here)
Description of issue:

I’ve been using Sonarr at home on OSX for a couple of years, but now trying to transition to a VPS setup. My Media root folder is on ACD mounted via FUSE (acd_cli). Sab downloads to a local ‘Downloads’ folder, a script converts to MP4, then Sonarr is meant to pick up and transfer MP4 to the ACD drive.

It looks like Sonarr is starting the process OK:

16-10-14 02:36:04.5|Debug|EpisodeFileMovingService|Moving episode file: /home/fozz/Downloads/Downloaded/The.Great.British.Bake.Off.S07E08.720p.HDTV.x264-C4TV.2/the.great.british.bake.off.s07e08.720p.hdtv.x264-c4tv.mp4 to /home/fozz/ACD/Media/TV/The Great British Bake Off/Season 07/The Great British Bake Off - S07E08 - Tudor Week HDTV-720p.mp4
16-10-14 02:36:04.5|Debug|DiskTransferService|Move [/home/fozz/Downloads/Downloaded/The.Great.British.Bake.Off.S07E08.720p.HDTV.x264-C4TV.2/the.great.british.bake.off.s07e08.720p.hdtv.x264-c4tv.mp4] > [/home/fozz/ACD/Media/TV/The Great British Bake Off/Season 07/The Great British Bake Off - S07E08 - Tudor Week HDTV-720p.mp4]

After some time Sonarr seems to give up and try and delete the file:

16-10-14 02:43:59.8|Debug|DiskTransferService|Rolling back incomplete file move [/home/fozz/Downloads/Downloaded/The.Great.British.Bake.Off.S07E08.720p.HDTV.x264-C4TV.2/the.great.british.bake.off.s07e08.720p.hdtv.x264-c4tv.mp4] to [/home/fozz/ACD/Media/TV/The Great British Bake Off/Season 07/The Great British Bake Off - S07E08 - Tudor Week HDTV-720p.mp4].

And this later. Not sure what’s going on here:

16-10-14 02:44:02.8|Warn|ImportApprovedEpisodes|Couldn’t import episode /home/fozz/Downloads/Downloaded/The.Great.British.Bake.Off.S07E08.720p.HDTV.x264-C4TV.2/the.great.british.bake.off.s07e08.720p.hdtv.x264-c4tv.mp4

[v2.0.0.4326] System.IO.IOException: Cannot create a file that already exist.
at System.IO.File.Move (System.String sourceFileName, System.String destFileName) <0x7fd5f455a640 + 0x002f0> in :0
at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, Boolean overwrite) [0x000e3] in M:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Common\Disk\DiskProviderBase.cs:209
at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileVerified (System.String sourcePath, System.String targetPath, Int64 originalSize) [0x00000] in M:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Common\Disk\DiskTransferService.cs:555

Trace logs also show this:

16-10-14 02:43:51.4|Trace|DiskProviderBase|Deleting file: /home/fozz/ACD/Media/TV/The Great British Bake Off/Season 07/The Great British Bake Off - S07E08 - Tudor Week HDTV-720p.mp4.partial~

Any idea what’s going on here? Is it the slow upload to ACD? Uploads to ACD work outside Sonarr, but only run at about 1MB/s.

Sometimes the Sonarr move does work as expected, but the item stays in my Wanted list. I have to manually refresh the Show in ‘Series’ in Sonarr to get it to pick up that the episode is correctly downloaded.

Any help welcome - thanks!

[markus101]

It could be slow upload, but more likely it’s being interrupted for some reason and not copying completely so Sonarr removes the partial file.

When Sonarr detects the file system it’s copying to is remote it tries to copy (as a partial file) to the destination then compares whether the source and destination file match, if Sonarr doesn’t see them as the same size then it removes the partial file (which is what you see in the trace logs).

System.IO.IOException: Cannot create a file that already exist.

I saw some posts on Reddit last week about this, seems to be something on the Amazon side if the file truly doesn’t exist.

Sometimes the Sonarr move does work as expected, but the item stays in my Wanted list. I have to manually refresh the Show in ‘Series’ in Sonarr to get it to pick up that the episode is correctly downloaded.

Sounds like it thinks it failed for some reason and rescanning the series folder finds the file that was uploaded.

[Fozz]

Thanks for the reply Marcus (and thanks to you and the Sonarr team generally - it’s a great project).

Sounds like acd_cli upload via FUSE is a bit flakey. To get round that, in the coming days I’ll try and set up a local Media folder (for Sonarr to directly move files to) and an ACD Media folder.

I’ll then join them together with UnionFS-FUSE, and use rclone in the background to sync local files over to the ACD volume (and then delete the local version to save space). Not my idea - using this guide https://amc.ovh/2015/08/15/uniting-encrypted-encfs-filesystems.html

Will report back if that goes well.

[rukuh]

@Fozz, been a while since the initial post but I have also been experiencing this issue. I was able to trace the issue back to root cause. Seems like Sonarr initiates the file move successfully and acd_cli is creating a “placeholder” file with size of 0 bytes until the copy is complete. Since Sonarr sees the file it tries to verify it and finds that the size is 0 and tries to rollback but can not. The “placeholder” file becomes the real file after the next “acd_cli sync” call, at least from what I have seen.

I have been looking for a solution, something along the lines of delaying import after copy but have had no luck.
An alternative is having the download complete folder on ACD also, but unpacking large files takes an extreme amount of time and the queue just backs up.

Any luck with the UnionFS route?

[Fozz]

Hi, apologies - I did mean to respond to this thread when I got things working. Yes, I took the UnionFS route and it works very well so far. Here’s the outline of the process from system boot:

1. Mount the acd_cli ACD drive using the init.d script from the project’s Github. I think for me that script only ran when a user actually logged in, so I had to modify it to run as root following this - https://github.com/yadayada/acd_cli/issues/264

2. Created a folder called “Media-local” which is my local “cached” version of my Media folder. You’ll see why in next step. My “Download” folder is also local.

3. I then use crontab to mount the UnionFS-FUSE drive. I think the “correct” way to do this is to use fstab, but on my VPS modifying that caused the whole system to fail to boot. Therefore, I use this crontab line:

@reboot sleep 20 && /usr/local/bin/unionfsmount

This waits 20 seconds after boot (to make sure ACD has mounted) and then pulls this script:

/usr/bin/unionfs-fuse -o cow,allow_other /home/fozz/Media-local=RW:/home/fozz/ACD/Media=RO /home/fozz/Media

That creates one “/home/fozz/Media” folder for ACD and the “Media-local” version. For clarity, the script above will really put all files copied to “/home/fozz/Media” into “/home/fozz/Media-local”. However, the contents of “Media-local” and “ACD” will appear as one unified “Media” folder from Sonarr / Plex / CP point of view.

4. Configure Sonarr to use “/home/fozz/Media” as it’s Media folder.

5. When downloads are complete Sonarr will put them in a nice chain of folders, via UnionFS, in “Media-local”. Sonarr will see them with all other episodes on ACD through the magic of UnionFS.

6. Now, I need to get those local files up to ACD. For that, I use an rclone script called via Settings -> Connect in Sonarr whenever a download / upgrade is complete. My rclone script just does this:

rclone move /home/fozz/Media-local/TV ACD:/Media/TV --exclude .@**
rclone move /home/fozz/Media-local/Movies ACD:/Media/Movies --exclude .@**
acd_cli sync

The act_cli sync command is there to force refresh the ACD FUSE mount. Because we’re using move rclone will delete the “Media-local” version, but the change will be invisible to Plex / Sonarr.

Hope this is some help, although I’ve probably missed some steps. I personally prefer rclone to acd_cli, but the FUSE implementation on the former is still experimental hence the mash-up.

I appreciate that this looks very amateurish! I’m not a programmer! I’m not claiming any of the above is the “best” way to set this up, but I first tried using Linux a month ago and it’s the best I can do! I’m sure others can improve.

[rukuh]

@Fozz, no worries. Appreciate the detailed walkthrough, was in the middle of getting something set up but I was manually calling acdcli upload, rclone seems to be a better route.

Also, FWIW from my research, seems like alot of other very experienced programmers are following a similar route so I would say it is a pretty good set up =) . Thanks again!