Detecting read only file systen

@Taloth spoke with Rootyrm so I won’t bother copying the conversation from gitter.

What else do you need from me? The file system is mounted read/write, sonarr is detecting it as read only. Radarr and Lidarr both are working.

First step is to get a list of how your mergerfs is configured. what volumes are involved. I don’t use mergerfs myself, so you’ll need to explain how it’s configured.
Second step is to enable trace level logs in sonarr and then reproduce, the trace log files will then contain the individual move/io operations sonarr performs in sequence.

The trick here is to determine from which mergerfs volume the file is being moved to which volume.
One of the operations is considered an RO operation by mergerfs, so the mergerfs configuration could shed some light on it.
You can also attempt to reproduce it from the cmdline. But identifying how the individual operations interact with mergerfs is important to be able to understand what’s going on.

It’s pretty basic - I think just default.

I think what you want is my /etc/fstab

# <file system>                                 <mount point>   <type>          <options>                               <dump>  <pass>
UUID=edb48d14-df73-4efb-8808-92d2c705741f       /               ext4            errors=remount-ro,defaults,acl          0       0
UUID=51b8dc62-a9e9-4c7e-9e94-32ae8109d62b       /home           ext4            errors=remount-ro,defaults,acl,nofail   0       0

UUID=f3ce7d6d-7543-4cd8-991c-4f6c427f18c5       /mnt/SG-1.5TB-1 ext4            errors=remount-ro,defaults,acl,nofail   0       0

UUID=4e16cf60-13ee-452f-a67f-e989032dcf7d       /mnt/SG-3TB-1   ext4            errors=remount-ro,defaults,acl,nofail   0       0

UUID=d834fa15-c81c-4330-b40e-b04e2cd5471f       /mnt/SG-4TB-1   ext4            errors=remount-ro,defaults,acl,nofail   0       0 # USB
UUID=ce1c1da3-fee5-4020-8361-e94f9a106c80       /mnt/SG-4TB-2   ext4            errors=remount-ro,defaults,acl,nofail   0       0 # USB

UUID=a8d31f58-54e6-4867-8833-c9a3c323f105       /mnt/SG-6TB-1   ext4            errors=remount-ro,defaults,acl,nofail   0       0
UUID=3ffb6832-f412-43bd-b984-0aad7dd16cd0       /mnt/SG-6TB-2   ext4            errors=remount-ro,defaults,acl,nofail   0       0

UUID=3602d0dc-8fa0-40bd-9ebe-b8dec16fc1da       /mnt/SG-8TB-1   ext4            errors=remount-ro,defaults,acl,nofail   0       0
UUID=59c19339-fdf1-40ef-b802-33c3c8f7e670       /mnt/SG-8TB-2   ext4            errors=remount-ro,defaults,acl,nofail   0       0
UUID=e9d6ec8c-c519-4d00-88e4-79b22d11b0ec       /mnt/SG-8TB-3   ext4            errors=remount-ro,defaults,acl,nofail   0       0
UUID=9a8febdb-27d9-4518-aac2-0566457a74cf       /mnt/SG-8TB-4   ext4            errors=remount-ro,defaults,acl,nofail   0       0 # USB

UUID=ba9d87b3-09c2-4cee-ab6a-eb73110d50f0       /mnt/WD-2TB-1   ext4            errors=remount-ro,defaults,acl,nofail   0       0

# Merger FS
/mnt/SG-*:/mnt/WD-2TB-1/                        /mnt/Drives/    fuse.mergerfs   allow_other,use_ino                     0       0

I mount everything individually just incase, and then I have MergerFS mounted as well with just the basic options from their setup guide. Here are what those two options mean from here:

allow_other : A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.

use_ino : Causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.

I don’t have trace logging on yet, however I noticed in my logs these two entries which might help track things down:

Couldn't rescan series [71862][Chappelle's Show]: Read-only file system
Couldn't rescan series [79275][Dragon Ball GT]: Read-only file system

Directories for those two don’t exist so Sonarr didn’t auto-create them. I manually created the Dragon B all GT directory, made Sonarr the owner and a new error appears in the log:

DiskScanService failed while processing [SeriesUpdatedEvent]: Access to the path '/media/Completed Televsion/Dragon Ball GT' is denied.

Anyways, I’ve enabled trace logging now - we’ll see what it spits out.

Is there a way to attach txt or zip files? They aren’t allowed - only image files it looks like. I’m not sure what relevant parts of the log file to post because there are a bunch of parts that might be useful.

The sections containing DiskTransferService is the relevant part. With a bit of context it should be no more than one or two dozen lines.
It’s possible that other stuff happens during the move attempt, but you should be able to see that in the log.

20-5-24 11:04:01.7|Debug|EpisodeFileMovingService|Moving episode file: /media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv to /media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv
20-5-24 11:04:01.7|Debug|DiskTransferService|Move [/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv] > [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv]
20-5-24 11:04:01.7|Trace|SymbolicLinkResolver|Checking path /mnt/Drives/Media//Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv for symlink returned error ENOENT, assuming it's not a symlink.
20-5-24 11:04:01.7|Trace|DiskTransferService|Attempting to move hardlinked backup.
20-5-24 11:04:01.7|Debug|DiskProvider|Hardlink '/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv' to '/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup~' failed.

[v2.0.0.5344] System.IO.IOException: Read-only file system ---> Mono.Unix.UnixIOException: Read-only file system [EROFS].
   --- End of inner exception stack trace ---
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in <0391fbdae99340f5a5d9b7181828ced5>:0 

20-5-24 11:04:01.7|Trace|DiskTransferService|Hardlink move failed, reverting to move.
20-5-24 11:04:01.7|Debug|DiskTransferService|Rolling back incomplete file move [/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv] to [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv].
20-5-24 11:04:04.7|Trace|DiskProviderBase|Deleting file: /media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv
20-5-24 11:04:04.7|Warn|ImportApprovedEpisodes|Couldn't import episode /media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv

[v2.0.0.5344] System.IO.IOException: Read-only file system
  at System.IO.FileSystem.LinkOrCopyFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d120 + 0x00116> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.FileSystem.MoveFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d370 + 0x000ce> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.File.Move (System.String sourceFileName, System.String destFileName) <0x7f19c2e98fd0 + 0x00217> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFileInternal (System.String source, System.String destination) [0x00000] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Mono.Disk.DiskProvider.MoveFileInternal (System.String source, System.String destination) [0x00076] in <0391fbdae99340f5a5d9b7181828ced5>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, System.Boolean overwrite) [0x000e3] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileVerified (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00047] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0018b] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x003ce] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x0017c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 


20-5-24 11:04:11.0|Debug|EpisodeFileMovingService|Moving episode file: /media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv to /media/Television/DC's Legends of Tomorrow/Season 05/DC's Legends of Tomorrow - S05E13 - I Am Legends.mkv
20-5-24 11:04:11.0|Debug|DiskTransferService|Move [/media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv] > [/media/Television/DC's Legends of Tomorrow/Season 05/DC's Legends of Tomorrow - S05E13 - I Am Legends.mkv]
20-5-24 11:04:11.0|Trace|SymbolicLinkResolver|Checking path /mnt/Drives/Media//Television/DC's Legends of Tomorrow/Season 05/DC's Legends of Tomorrow - S05E13 - I Am Legends.mkv for symlink returned error ENOENT, assuming it's not a symlink.
20-5-24 11:04:11.0|Trace|DiskTransferService|Attempting to move hardlinked backup.
20-5-24 11:04:11.0|Debug|DiskProvider|Hardlink '/media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv' to '/media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv.backup~' failed.

[v2.0.0.5344] System.IO.IOException: Read-only file system ---> Mono.Unix.UnixIOException: Read-only file system [EROFS].
   --- End of inner exception stack trace ---
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in <0391fbdae99340f5a5d9b7181828ced5>:0 

20-5-24 11:04:11.0|Trace|DiskTransferService|Hardlink move failed, reverting to move.
20-5-24 11:04:11.0|Debug|DiskTransferService|Rolling back incomplete file move [/media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv] to [/media/Television/DC's Legends of Tomorrow/Season 05/DC's Legends of Tomorrow - S05E13 - I Am Legends.mkv].
20-5-24 11:04:14.0|Trace|DiskProviderBase|Deleting file: /media/Television/DC's Legends of Tomorrow/Season 05/DC's Legends of Tomorrow - S05E13 - I Am Legends.mkv
20-5-24 11:04:14.0|Warn|ImportApprovedEpisodes|Couldn't import episode /media/Downloads/Complete/Television/DCs Legends of Tomorrow S05E13 720p HDTV x264-AVS/vayZ04BPfHNCdc9GptsBTvc7i4771MJ.mkv

[v2.0.0.5344] System.IO.IOException: Read-only file system
  at System.IO.FileSystem.LinkOrCopyFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d120 + 0x00116> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.FileSystem.MoveFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d370 + 0x000ce> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.File.Move (System.String sourceFileName, System.String destFileName) <0x7f19c2e98fd0 + 0x00217> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFileInternal (System.String source, System.String destination) [0x00000] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Mono.Disk.DiskProvider.MoveFileInternal (System.String source, System.String destination) [0x00076] in <0391fbdae99340f5a5d9b7181828ced5>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, System.Boolean overwrite) [0x000e3] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileVerified (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00047] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0018b] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x003ce] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x0017c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 

20-5-24 11:04:42.1|Debug|EpisodeFileMovingService|Moving episode file: /media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv to /media/Television/The Flash (2014)/Season 06/The Flash (2014) - S06E19 - Success Is Assured.mkv
20-5-24 11:04:42.1|Debug|DiskTransferService|Move [/media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv] > [/media/Television/The Flash (2014)/Season 06/The Flash (2014) - S06E19 - Success Is Assured.mkv]
20-5-24 11:04:42.1|Trace|SymbolicLinkResolver|Checking path /mnt/Drives/Media//Television/The Flash (2014)/Season 06/The Flash (2014) - S06E19 - Success Is Assured.mkv for symlink returned error ENOENT, assuming it's not a symlink.
20-5-24 11:04:42.1|Trace|DiskTransferService|Attempting to move hardlinked backup.
20-5-24 11:04:42.1|Debug|DiskProvider|Hardlink '/media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv' to '/media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv.backup~' failed.

[v2.0.0.5344] System.IO.IOException: Read-only file system ---> Mono.Unix.UnixIOException: Read-only file system [EROFS].
   --- End of inner exception stack trace ---
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in <0391fbdae99340f5a5d9b7181828ced5>:0 

20-5-24 11:04:42.1|Trace|DiskTransferService|Hardlink move failed, reverting to move.
20-5-24 11:04:42.1|Debug|DiskTransferService|Rolling back incomplete file move [/media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv] to [/media/Television/The Flash (2014)/Season 06/The Flash (2014) - S06E19 - Success Is Assured.mkv].
20-5-24 11:04:45.1|Trace|DiskProviderBase|Deleting file: /media/Television/The Flash (2014)/Season 06/The Flash (2014) - S06E19 - Success Is Assured.mkv
20-5-24 11:04:45.1|Warn|ImportApprovedEpisodes|Couldn't import episode /media/Downloads/Complete/Television/The Flash 2014 S06E19 720p HDTV x264-KILLERS/xCBKNxZ60sXXjzbR8N4JR9nKKpo.mkv

[v2.0.0.5344] System.IO.IOException: Read-only file system
  at System.IO.FileSystem.LinkOrCopyFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d120 + 0x00116> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.FileSystem.MoveFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d370 + 0x000ce> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.File.Move (System.String sourceFileName, System.String destFileName) <0x7f19c2e98fd0 + 0x00217> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFileInternal (System.String source, System.String destination) [0x00000] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Mono.Disk.DiskProvider.MoveFileInternal (System.String source, System.String destination) [0x00076] in <0391fbdae99340f5a5d9b7181828ced5>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, System.Boolean overwrite) [0x000e3] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileVerified (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00047] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0018b] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x003ce] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x0017c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 

I think that’s about it.

That’s the relevant log parts, thank you.

Can you try that with ln cmdline?

Based on the mounts above, you’re not using cloud storage, so this is just a ton of different disks merged together.
Can you find out on which of the drives /media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV is located?

That was a permission error - when it downloads from Sabnzbd the owner/group is sabnzbd:sabnzbd and it needs to be something different for Sonarr/Radarr/etc to work. I fixed the permissions and the ln worked.

Yup - nothing fancy. JBOD.

Here’s the source:

/mnt/SG-8TB-3/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/

This is /dev/sdj.

In case you also need, here’s the target: /mnt/SG-8TB-2/Media/Television/Ridiculousness This is /dev/sdg.

It was /dev/sdj going offline that caused all of this in the first place.

Ok, that’s good news, so if ln now works so should succeed the step that failed earlier. If it happens again post same section of the logs so we can see what else goes wrong.

PS. The error indicates that it didn’t yet make the attempt to move it to /dev/sdg it was still linking on /dev/sdj

Looks like it’s still failing at the hard link.

20-5-25 19:20:20.8|Debug|EpisodeFileMovingService|Moving episode file: /media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv to /media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv
20-5-25 19:20:20.8|Debug|DiskTransferService|Move [/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv] > [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv]
20-5-25 19:20:20.8|Trace|SymbolicLinkResolver|Checking path /mnt/Drives/Media//Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv for symlink returned error ENOENT, assuming it's not a symlink.
20-5-25 19:20:20.8|Trace|DiskTransferService|Attempting to move hardlinked backup.
20-5-25 19:20:20.8|Debug|DiskProvider|Hardlink '/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv' to '/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup~' failed.

[v2.0.0.5344] System.IO.IOException: Read-only file system ---> Mono.Unix.UnixIOException: Read-only file system [EROFS].
   --- End of inner exception stack trace ---
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <b605f6389d5c406e803fb711c6f542c7>:0 
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in <0391fbdae99340f5a5d9b7181828ced5>:0 

20-5-25 19:20:20.8|Trace|DiskTransferService|Hardlink move failed, reverting to move.
20-5-25 19:20:20.8|Debug|DiskTransferService|Rolling back incomplete file move [/media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv] to [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv].
20-5-25 19:20:23.8|Trace|DiskProviderBase|Deleting file: /media/Television/Ridiculousness/Season 17/Ridiculousness - S17E18 - JoJo.mkv
20-5-25 19:20:23.8|Warn|ImportApprovedEpisodes|Couldn't import episode /media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv

[v2.0.0.5344] System.IO.IOException: Read-only file system
  at System.IO.FileSystem.LinkOrCopyFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d120 + 0x00116> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.FileSystem.MoveFile (System.String sourceFullPath, System.String destFullPath) <0x7f19c2e9d370 + 0x000ce> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at System.IO.File.Move (System.String sourceFileName, System.String destFileName) <0x7f19c2e98fd0 + 0x00217> in <f2b3ab7dfff746f594d2ef5b16ec3c90>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFileInternal (System.String source, System.String destination) [0x00000] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Mono.Disk.DiskProvider.MoveFileInternal (System.String source, System.String destination) [0x00076] in <0391fbdae99340f5a5d9b7181828ced5>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.MoveFile (System.String source, System.String destination, System.Boolean overwrite) [0x000e3] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileVerified (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00047] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryMoveFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0018b] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x003ce] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <faeb8209b6ff4c15aabf93dc8df43c9f>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x0017c] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in <4d3cc4e5f7ed4ed2814d83bdaf94809e>:0

However I’m able to do it manually as the sonarr user.

What’s the output of (running as sonarr)

ln /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~
ls -al /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV

ln: target 'HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~' is not a directory

total 441628
drwxrwxr-x+ 2 sonarr mediausers 4096 May 26 19:14 .
drwxrwxr-x+ 26 mediauser mediausers 12288 May 26 00:01 …
-rw-rw-r–+ 1 sonarr mediausers 34 May 21 22:00 a6ff9dedb139bd64b57e46c88b547c44.txt
-rw-rw-r–+ 1 sonarr mediausers 452189063 May 21 21:36 puyf5SF4ceSMrgXYk.mkv

I’m sorry, I did ln wrong, should’ve been ln -T /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~

sonarr@Ubuntu:~$ ln -T /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~
ln: extra operand 'HDTV x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~'

I’m not sure the syntax you’re going for, I don’t really play around with hard links :slight_smile: but I know that I can make a symbolic link without issue.

It’s complaining about a missing \ to escape spaces. I missed one.

ln -T /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/puyf5SF4ceSMrgXYk.mkv.backup2~

I just want ln -T {src} {dest} to make a hardlink coz that’s what sonarr is doing.

Ahh my bad sorry.

It worked without issue.

Here’s ls -al:

total 883228
drwxrwxr-x+  2 sonarr    mediausers      4096 May 27 13:31 .
drwxrwxr-x+ 27 mediauser mediausers     12288 May 26 20:26 ..
-rw-rw-r--+  1 sonarr    mediausers        34 May 21 22:00 a6ff9dedb139bd64b57e46c88b547c44.txt
-rw-rw-r--+  2 sonarr    mediausers 452189063 May 21 21:36 puyf5SF4ceSMrgXYk.mkv
-rw-rw-r--+  2 sonarr    mediausers 452189063 May 21 21:36 puyf5SF4ceSMrgXYk.mkv.backup2~

Can you attempt to import in Sonarr again? Same file of course, if needed you can use Wanted->Manual Import.

I’m getting curious what’s different between ln and the low-level link syscall that mono does.

PS: I believe the + in ls means there are actually extended acl permissions involved.
Can you run getfacl <file>? (where file is the .mkv)

20-5-27 14:05:48.1|Debug|EpisodeFileMovingService|Moving episode file: /media/Downloads/Complete/Television/Ridiculousness S17E13 720p HDTV x264-YesTV/Vy1QueHqg9Wf02CeU2J57PtbobHU7rZ1DqyKUg.mkv to /media/Television/Ridiculousness/Season 17/Ridiculousness - S17E13 - Chanel and Sterling CLXXVIII.mkv
20-5-27 14:05:48.1|Debug|DiskTransferService|Move [/media/Downloads/Complete/Television/Ridiculousness S17E13 720p HDTV x264-YesTV/Vy1QueHqg9Wf02CeU2J57PtbobHU7rZ1DqyKUg.mkv] > [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E13 - Chanel and Sterling CLXXVIII.mkv]
20-5-27 14:05:48.1|Trace|SymbolicLinkResolver|Checking path /mnt/Drives/Media//Television/Ridiculousness/Season 17/Ridiculousness - S17E13 - Chanel and Sterling CLXXVIII.mkv for symlink returned error ENOENT, assuming it's not a symlink.
20-5-27 14:05:48.1|Trace|DiskTransferService|Attempting to move hardlinked backup.
20-5-27 14:05:48.1|Debug|DiskProvider|Hardlink '/media/Downloads/Complete/Television/Ridiculousness S17E13 720p HDTV x264-YesTV/Vy1QueHqg9Wf02CeU2J57PtbobHU7rZ1DqyKUg.mkv' to '/media/Downloads/Complete/Television/Ridiculousness S17E13 720p HDTV x264-YesTV/Vy1QueHqg9Wf02CeU2J57PtbobHU7rZ1DqyKUg.mkv.backup~' failed.

[v2.0.0.5344] System.IO.IOException: Read-only file system ---> Mono.Unix.UnixIOException: Read-only file system [EROFS].
   --- End of inner exception stack trace ---
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <b605f6389d5c406e803fb711c6f542c7>:0
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <b605f6389d5c406e803fb711c6f542c7>:0
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <b605f6389d5c406e803fb711c6f542c7>:0
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in <0391fbdae99340f5a5d9b7181828ced5>:0

20-5-27 14:05:48.1|Trace|DiskTransferService|Hardlink move failed, reverting to move.
20-5-27 14:05:48.1|Debug|DiskTransferService|Rolling back incomplete file move [/media/Downloads/Complete/Television/Ridiculousness S17E13 720p HDTV x264-YesTV/Vy1QueHqg9Wf02CeU2J57PtbobHU7rZ1DqyKUg.mkv] to [/media/Television/Ridiculousness/Season 17/Ridiculousness - S17E13 - Chanel and Sterling CLXXVIII.mkv].

Yes it is - I use ACL’s to try and make Sonarr, Radarr, Sabnzbd, Transmission, etc play nice with each other while all running with their own users. It hasn’t worked. :slight_smile: All of them belong to the group mediausers which I thought would deal with the problem originally, but it didn’t so I started adding on ACL’s. Sabnzbd will download and the owner of the file is sabnzbd and I regularlry have to change the owner to radarr or sonarr for them to be able to process them properly.

sudo getfacl puyf5SF4ceSMrgXYk.mkv
# file: puyf5SF4ceSMrgXYk.mkv
# owner: sonarr
# group: mediausers
user::rw-
user:debian-transmission:rwx    #effective:rw-
user:lidarr:rwx                 #effective:rw-
user:mylar:rwx                  #effective:rw-
user:radarr:rwx                 #effective:rw-
user:sonarr:rwx                 #effective:rw-
user:sabnzbd:rwx                #effective:rw-
group::rwx                      #effective:rw-
group:mediausers:rwx            #effective:rw-
mask::rw-
other::r--

Here’s the directory ACL just in case you want to see:

sudo getfacl /media/Downloads/Complete/Television/Ridiculousness\ S17E18\ 720p\ HDTV\ x264-YesTV/
getfacl: Removing leading '/' from absolute path names
# file: media/Downloads/Complete/Television/Ridiculousness S17E18 720p HDTV x264-YesTV/
# owner: sonarr
# group: mediausers
user::rwx
user:debian-transmission:rwx
user:lidarr:rwx
user:mylar:rwx
user:radarr:rwx
user:sonarr:rwx
user:sabnzbd:rwx
group::rwx
group:mediausers:rwx
mask::rwx
other::r-x
default:user::rwx
default:user:debian-transmission:rwx
default:user:lidarr:rwx
default:user:mylar:rwx
default:user:radarr:rwx
default:user:sonarr:rwx
default:user:sabnzbd:rwx
default:group::rwx
default:group:mediausers:rwx
default:mask::rwx
default:other::r-x

A bit of a detour, but what group is sonarr running as? you can see with ps axo pid,user,group,command | grep mono

It shouldn’t matter coz sonarr is owner of the file and folder, but as I said, slight detour. :slight_smile:

sonarr is the user and group.