NZBDrone freezes during series processing

Help & Support
1

Hi,

I’m running SAB + NZBDrone on Ubuntu.

I recently had issues relating to speed caused by my SAB installation having a temp and completed folder on my NAS, instead of the local disk.

I’m now doing all Temp + Completed folders on the local SSD, which NZBDrone then picks up and moves to my NAS.

So, what’s my problem:

Since I fixed the speed issue, I’m getting download speeds of 10-25Mb/s, which is great, but what I’ve noticed is that I can download 20 episodes (using NZBDrone series search), and only 3 or 4 episodes get copied to my NAS, and some are partially completed (what I know is a 200Mb file for example, copies only 100Mb) before the NZBDrone freezes, I can’t access via the web at the normal IP 192.168.[x].[x]:8989.

I’m able to download the whole series of 20 episodes before 4 files have copied to my NAS.

I can see in my NAS folders that some files have copied, some partially, I can’t get access back to NZBDrone via the web, although SAB is responding fine. CPU usage on Linux is hovering at around 1%, and I can still reach the NAS folders from my Linux installation.

I never had this problem when my Temp + Completed folders were on the NAS, which leads me to think it is downloading everything too quick for NZBDrone and somewhere it is bottlenecking.

I’m after any advice at all you guys can give me!!!

TL;DR: NZBDrone freezes after downloading a series of episodes, as the series finishes downloading before NZBDrone has moved a few of the episodes. I’m unable to access NZBDrone via web, and have to restart my Linux box (hosting SAB+NZBDrone) in order to get access back to NZBDrone console. Need help finding the cause!

Thanks for reading!

2

Which version of drone?
Which version of mono?
Did more than one file only get copied partially?

3

I’m pretty shit with Linux, so bear with me.

Drone version 2.0.0.1690

Mono version 3.2.8+dfsg-4ubuntu1

Of a recent series of 20+ episodes (600Mb files), 2 copied fully to NAS, 2 got about 40% and they’ve stayed there for the past 20 minutes.

I did another series yesterday which were larger epidoes (3.5Gb files) and they all were ok apart from 1 or 2 which I re-did manually.

And just for more info, NZBDrone is still trying to connect via Web, for the same period of time.

4

I’ve restarted my Linux box and NZBDrone is still refusing to connect via http://localhost:8989, it just times out.

— EDIT 1 hour later —

Here is the log file URL:

http://pastebin.com/gqu9QkZs

More files have started to be slowly copied now, I can see them very slowly appearing on the NAS. However, I still have errors in the log file, and I still can’t connect to the web console for NZBDrone.

Advice still welcome!!!

5

First things first, you should update drone to the latest master release, there are a number of fixes in it. I’d also recommend installing mono 3.6.

While mono 3.2 or 3.4 will work, there may be some issues, crashing on 3.2 came up in another thread and seems to be fixed on 3.6. mono 3.4 has a pretty major issue that is resolved in 3.6. Not sure if there is an apt-get repo available that contains 3.6, but here is a guide on how to compile it that looks pretty straight forward: http://www.lovesmesomecode.com/20130719-compiling-mono-3-in-ubuntu/

6

I don’t really have an answer for you because i am using Windows server (tried and failed with linux too many hassles)

just wanted to curse you out for having such better download speeds than me 10mb/s - 25mb/s C’mon now that’s unfair :frowning:

My Internet sucks (Par for the course living in Australia i guess)

7

Ok, so here’s what I’ve done this evening!

I’ve updated Mono using the link you gave, it is now running 3.10.

I updated to latest NZBDrone.

I restarted my server and still couldn’t login to NZBDrone, it was timing out. However, I noticed that on restart I was getting some CIFS errors for my mapped NAS, so I started looking at that because restarts of Ubuntu were locking.

I deleted the mapping, and recreated the mapping in /etc/fstab, rebooted and I could log into NZBDrone finally. This tells me that Ubuntu/NZBDrone was trying to move data to the NAS but it was failing. Now I’ve tried downloading single episodes and it is working well, however the second episode I’ve tried NZBDrone isn’t picking it up, and there is nothing in the logs to say why, even though SAB has completely finished processing it and it is sitting waiting in the Completed folder.

I also deleted all the episodes that it tried to copy yesterday, since they were locked and half-size so I guess that was caused by the CIFS issue.

Any ideas on NZBDrone not picking up the finished download from SAB, with nothing in logs to suggest why?

–

I’ll start a new topic about that problem, I think it is completely unrelated.

8

Problem is back. It was downloading and moving individual episodes very nicely, so I tried to download 2 at once.

SAB downloaded both, but only one has copied to the NAS. NZBDrone is now timing out again when I try to access via web.

I’m back to square one… any help?

============ EDIT ===============

Here is the latest logfile, I can see that it parsed episode 7, the one that didn’t copy, but the log file just stoped 10 minutes ago at this step:

4-9-11 23:12:44.0|Trace|ConfigService|Unable to find config key ‘enablefaileddownloadhandling’ defaultValue:'True’
14-9-11 23:13:13.7|Trace|EventAggregator|Publishing UpdateQueueEvent
14-9-11 23:13:13.7|Trace|EventAggregator|UpdateQueueEvent -> QueueModule
14-9-11 23:13:13.7|Trace|CommandExecutor|Publishing BroadcastSignalRMessage
14-9-11 23:13:13.7|Trace|CommandExecutor|BroadcastSignalRMessage -> NzbDronePersistentConnection
14-9-11 23:13:13.7|Trace|EventAggregator|Publishing CommandExecutedEvent
14-9-11 23:13:13.7|Trace|EventAggregator|CommandExecutedEvent -> TaskManager
14-9-11 23:13:13.7|Trace|EventAggregator|CommandExecutedEvent <- TaskManager
14-9-11 23:13:13.7|Trace|CommandExecutor|BroadcastSignalRMessage <- NzbDronePersistentConnection [00:00:00.0002920]
14-9-11 23:13:13.7|Trace|EventAggregator|UpdateQueueEvent <- QueueModule
14-9-11 23:13:13.9|Debug|SabnzbdProxy|Url: https://localhost:9090/api?mode=queue&start=0&limit=0&apikey=&output=json
14-9-11 23:13:13.9|Trace|SabnzbdProxy|Response: {“queue”:{“active_lang”:“en”,“session”:“sessionID”,“slots”:[{“unpackopts”:“3”,“size”:“662 MB”,“index”:0,“script”:“None”,“msgid”:"",“filename”:“The Thick Of It.4x07.720p HDTV x264-FoV”,“priority”:“High”,“rating_avg_audio”:0,“percentage”:“94”,“status”:“Downloading”,“avg_age”:“684d”,“missing”:0,“has_rating”:true,“sizeleft”:“35 MB”,“nzo_id”:“SABnzbd_nzo_VtZRpr”,“mbleft”:“35.13”,“timeleft”:“0:00:20”,“mb”:“661.68”,“verbosity”:"",“rating_avg_video”:0,“cat”:“tv category”,“eta”:“23:13 Thu 11 Sep”}],“speed”:“1.7 M”,“size”:“662 MB”,“rating_enable”:false,“limit”:0,“start”:0,“diskspacetotal2”:“46.68”,“darwin”:false,“last_warning”:"",“have_warnings”:“0”,“noofslots”:1,“newzbin_url”:“www.newzbin2.es”,“pause_int”:“0”,“categories”:["*",“movies category”,“tv category”],“pp_pause_event”:false,“diskspacetotal1”:“46.68”,“mb”:“661.68”,“loadavg”:“2.53 | 1.50 | 0.71 | V=2429M R=88M”,“cache_max”:“0”,“speedlimit”:"",“webdir”:"",“left_quota”:“0 “,“uniconfig”:”/usr/share/sabnzbdplus/interfaces/Config/templates”,“paused”:false,“isverbose”:false,“restart_req”:false,“power_options”:true,“helpuri”:“http://wiki.sabnzbd.org/",“uptime”:“1h”,“refresh_rate”:"",“my_home”:"/home/username",“version”:“0.7.18”,“my_lcldata”:"/home/username/.sabnzbd",“color_scheme”:“gold”,“new_release”:"",“nt”:false,“status”:“Downloading”,“finish”:0,“cache_art”:“0”,“paused_all”:false,“finishaction”:null,“sizeleft”:"35 MB”,“quota”:“0 “,“cache_size”:“0 B”,“mbleft”:“35.13”,“diskspace2”:“22.99”,“diskspace1”:“22.99”,“scripts”:[],“timeleft”:“0:00:20”,“have_quota”:false,“nzb_quota”:””,“eta”:“23:13 Thu 11 Sep”,“kbpersec”:“1737.57”,“new_rel_url”:"",“queue_details”:“0”}}
14-9-11 23:13:13.9|Trace|ConfigService|Unable to find config key ‘downloadclienthistorylimit’ defaultValue:'30’
14-9-11 23:13:13.9|Debug|SabnzbdProxy|Url: https://localhost:9090/api?mode=history&start=0&limit=30&apikey=&output=json

=========== EDIT 2 ==============

Something I’ve just noticed. Episodes 1 - 3 which I grabbed individually are fine, but epiusodes 4 - 7 that I kicked off within minutes of each other are all corrupted in some way, although their sizes approximately match, but I guess they are only 9x% copied, not 100%.

I’m completely out of ideas what is going on here…

9

3.10? the latest release is 3.8 and that just came out a few days ago. What is the full output from mono --version?

The have copied files sound like a mono + CIFS problem, we’ve seen it before, but not sure we have a fix for it (@Taloth do you remember the outcome of CIFS copying issues?). What type of NAS are you copying to?

10

I’m copying to a WDMyCloud 4TB drive.

I’m thinking it is definitely a NAS issue now, but unfortunately I don’t have a big HD in my Linux box to test stuff moving there, although it sounds like I will need to forget a NAS and stock up on big HDs.

Is there another option for mounting my NAS that might be worth testing? Any alternative to CIFS?

Output from Mono is:

Mono JIT Compiler 3.10.1

11

Depends, can you connect it via USB? or does it support NFS?

You would only need enough space on your linux box to store a few files to test, 5-10GB given the frequent failures we’re seeing over to the NAS.

12

Ok, I’ve just setup a temp folder in my home drive and set off 2 downloads… let’s see what happens!

I can’t connect it via USB, but I’ll try NAS if it is possible to setup a permanent mount after reboot, which I didn’t think was possible?

====== EDIT =====

THat has worked nicely. 2 episodes downloaded and processed into NZBDrone without flaws.

Could I also setup my Completed folder so SAB copies processed files there, and let NZBDrone process them from the NAS, since it wouldn’t have to move it to the Network drive, but only move it which would be quicker?

13

If its in the same share then that should work, if it needs to move it to another share then its not going to be faster. It will be on the same system, but also needs to be on the same share.

/nas/share/completed/tv to /nas/share/tv works, but /nas/completed/tv to /nas/tv` wouldn’t.

14

Both are stored on /media/Plex/TV so I will try that!

If you have any advice on a permanent fix (or find the issue in the logs or whatever), I’d love to hear it, otherwise I’ll also start to think about HD storage, not NAS.

15

Tested with 2 more episodes, took a bit longer to unpack in SAB (because I guess it is unpacking onto the NAS), but NZBDrone worked perfectly and imported as planned.

So, that pretty much means it must be a CIFS issue. When I have some time on the weekend I’ll try mounting as NFS, although it seems you’re limited on NFS with a WDMyCloud, but I’ll try nonetheless!

Thanks so much for your help, once I’ve got it all figured out I’ll be sending a nice donation for your support :smile:

16

@markus101 The outcome back then was that the user affected switched to NFS.

The only thing I can think of is have drone detect if the target dir is a different fs, and then do a copy and verify file size to confirm a proper copy. But it’s kinda insane we would have to do that.