Not sure if i’m doing something wrong here but here is the problem I am having. I am using drone on linux if that matters or not.
Drone grabs the show
Sends the NZB over to SABnzbd
SABnzbd downloads the show in folder called NAS/sabnzbd/tv/incomplete/
When download is finished, SABnzbd begins unpacking it into a temporary folder within the completed folder called NAS/sabnzbd/tv/complete/ _UNPACK_filename/
and i think this is where the problems start
NZBdrone is set to watch NAS/sabnzbd/tv/complete, and it can see that there’s a new video file (although it’s still in the process of unpacking) and KEEPS trying to rename and move it to the NAS/TV Shows/ location every minute. It doesn’t have permissions for this folder though.
At first I thought, well this isn’t a big deal, since eventually SABnzbd will move it out of the temporary UNPACK folder and move it to a regular folder… but I saw this wasn’t happening. File stayed in the UNPACK folder and drone could never get it. Initially suspected it was a problem with SABnzbd since the file wasn’t being moved out of the temp folder… But I did a test by telling drone to not watch the NAS/sabnzbd/tv/complete folder any more to see if SABnzbd would move it. Looks like it did move it out of the unpack folder afterall.
Do i need to change something in my setup to fix this issue?
Simple. You need to set SABnzdb to unpack in a folder that isn’t being watched by NZBdrone. Instead of NAS/sabnzbd/tv/complete/ _UNPACK_filename/ try something like NAS/sabnzbd/tv/incomplete/ _UNPACK_filename/
The easiest solution is to enable the Switch “Enable folder rename” in SAB’s config, this will prepend _UNPACK to the folder during the extraction and NzbDrone will wait for it to be removed.
@bobbintb said:
Simple. You need to set SABnzdb to unpack in a folder that isn’t being watched by NZBdrone. Instead of NAS/sabnzbd/tv/complete/ _UNPACK_filename/ try something like NAS/sabnzbd/tv/incomplete/ _UNPACK_filename/
There’s no setting in SABnzbd to tell it where to unpack as far as I can see. There are only options for temporary download folder and complete download folder. As soon as the download is completed, sabnzbd will begin to unpack into the completed download folder.
I could have missed something, but just did a double check at all the settings again. Also says so according to their wiki.
@markus101 said:
The easiest solution is to enable the Switch “Enable folder rename” in SAB’s config, this will prepend _UNPACK to the folder during the extraction and NzbDrone will wait for it to be removed.
Yes, I have this enabled. But that’s the problem, NzbDrone is trying to rename and move the file that is in the UNPACK folder.
It attempts to do this every minute, and ultimately breaks the process by preventing SABznbd from moving the final folder without the _UNPACK and with write permission. Think i must have something set up incorrectly if nobody else has this issue (which i didn’t see from a search).
NzbDrone is watching folder /mnt/network/NAS-001/SABnzbd/complete/TV Shows. Is it supposed to ignore folders with _UNPACK?
Just saw another post saying that nzbdrone is supposed to ignore _unpack… so I guess it answers that question. Wonder if it has anything to do with the fact that it’s all taking place on a network share?
I am using the enable folder rename switch and all my folder are on a Synology NAS and NZBDrone is running on my local pc C;\ drive and watches the folder on the NAS. No problems since I used the rename switch. Was having a problem before though
Drone also checks the last write time to attempt to deal with SAB failing to rename folders after extracting, which is sometimes an issue with windows.
I’m running it on linux, but i think the problem is that Drone keeps attempting to grab the file from _unpack folder even though it is still in the process of unpacking. I’m thinking it has to do with the fact that it’s on a network share. gonna run the scenario from local drive now and see.
@markus101 : So i changed the temp download and completed downlod folders in SABnzbd to local drives. It is working fine now. My guess is that Drone wasn’t able to read the write times when looking at the CIFS share?
Its something with the CIFS share either not reporting the last write time correctly or it being interpreted wrong. Not sure how or why that would be wrong, but the issue is outside of drone’s control.
If you look at the CIFS share is the last write time correct?
Is drone logging an error every minute? (if so, please put it on pastebin)
Why/how doesn’t drone have access to the file when its unpacking? Does SAB set/reset the permissions after moving it?
When it was setup like that, drone was logging an error every minute. I already cleared those logs, so will have to go back and reconfigure the folders and run another download it to provide the error to you. But basically it was saying it did not have access to the file.
Yes, it looks like SAB creates a new folder with different permission after it’s finished unpacking, and moves the file from the _unpack folder. It seems if Drone had access to it while it’s unpacking it would probably completely screw up the process by renaming the file and moving it 1 minute into the unpacking (since it doesn’t seem to be able to check the write time)
When i look at the mounted share from the terminal, it does show the correct write/modification time. Maybe it’s not being passed to Drone in a way it understands?
It might be another check its doing to see if the file in ready to move (it doesn’t a number of them before it makes the final decision to move it), not the write time as now that I think about it, it seems unlikely that the write time would be sent in an unknown way (though it could be saved as the time the file was zipped as is often seen). These checks are in place to prevent it from moving before the file is completely unpacked and only require read permissions on the folder and the files within, the actual move requires write permissions to move it.
Hopefully the actual error can shed some light on what the exact cause is, but it sounds like its a permissions issue.
But regardless it shouldn’t be trying to move from the _unpack folder, correct? SAB works fine with moving the final unpacked file to the regular folder as long as Drone isn’t trying to access it every minute.
Is it supposed to ignore any folder starting with UNPACK?
It does a number of checks before trying to import the files in the folder, including the _UNPACK_ check (so yes, it will ignore folders starting with _UNPACK_), all the checks have to pass before it will try to import the file. One of the checks attempts to lock the file for exclusive read-only access, which is intended to fail when another process has a lock on the file, the odds of it locking for the fraction of second it needs to do that check at the exact same time as when SAB is renaming the folder is slim, definitely not going to happen every single time.
Without logs I don’t know whats failing so I can’t fix anything, once I see the logs I should have a better idea of whats happening.
@markus101 , not really sure what happened here. I reverted back to the previous configuration of downloading to and unpacking on the CIFS share as opposed to the local machine. I didn’t get the same error this time. But actually drone moved that file from the _Unpack folder while SAB was still unpacking it. You can see from the log that it moved the file at 13:53 but SAB didnt’ finish unpacking until 2:13. Doesn’t appear to have disrupted the unpacking process however.
No idea why the behavior isn’t the same, I guess this is better than the problem I had before, but it’s creating another issue now that SAB will still create a final folder w/o _unpack, leaving a folder behind in the /completed/ downloads folder.
Yeah, thats still an issue, I think under Linux its safe to consider _UNPACK_ as truly unpacking, whereas on Windows it can sometimes get stuck in that folder. So I’ll make that check fail if its in the folder at all when running on Linux. The in use check properly fails because its via a CIFS share, we may want to disable that check on Linux, but for now I’ll leave it in place.
My only question now, is how did it take over 10 minutes for the files to get unpacked by SABnzbd? Thats either a big file and a slow network, a really slow network, or gigantic file.
Not really sure how long it’s supposed to take. Started unpacking at 1:45 -> 2:13, almost 30 minutes to unpack a 2GB file (30 packages). The linux machine is doing that work on an older Core 2 Solo processor over wifi, so maybe that has something to do with it.
Thanks for your support! Great tool you have here.
Probably wifi, my core2duo unpacks 1.2GB in about 6 seconds, to local disk. Unpacking is less CPU intensive than it is disk intensive, which explains the slow unpack speeds over wifi. Might explain why you saw this issue pop up as well.
I’ve pushed a fix for _UNPACK_ under Linux, should be able to follow the install instructions for drone on Linux to update it (via apt-get).
I am currently having this same issue, but with NZBget. I looked at the flow of the download and it goes from nzbdrone to nzbget which downloads it to a temporary folder, where it is also extracted then it is moved to the TV Shows folder which is the watched folder for nzbdrone. sometimes not always nzbdrone moves the file I am guessing before the move is complete which leave partial files to be moved to the TV Show directory. I should mention I am running NZBdrone, and NZBget on Ubuntu, and the final destination is a shared folder on a separate machine. any help will be greatly appreciated.