NzbDrone stops responding

Since the updates that were installed on 8/25/14 NzbDrone has stopped responding twice. A stop and start of NzbDrone resolves the issue. When this happens I am unable to connect to NzbDrone locally via the web interface or locally/remotely using nzb360. I am including the logs of the last time this occurred. This last event happened over several days while I was out of town. I have removed nothing from the logs. You will see several places in the logs where there is a reported SQLite error followed by many hours of nothing. The first time gap is over 9 hours in length, the second time gap is over 7 hours, the 3rd time gap is over 9 hours and the 4th time gap is almost 29 hours. That is when I restarted NzbDrone.

Logs: http://pastebin.com/ikd0NZVm

About
NzbDrone Version 2.0.0.1861
Mono Version 3.2.1 (Debian 3.2.1+dfsg-1~pre2)
Ubuntu Linux 12.04.4

Drone is unable to access its DB file, which is stored in the home directory, is that remotely mounted or symlinked somewhere other than a local disk?

No changes we made to how drone accesses its database in the last update.

I’ve been having the same thing happen. After a day or two of NZBDrone running, it stops responding. I too have only had this start happening since the last update.

Drone is unable to access its DB file, which is stored in the home
directory, is that remotely mounted or symlinked somewhere other than a
local disk?

It is not remotely mounted or symlinked.

I am having the exact same problem. I’m running OSX 10.9.4, Mono 3.2.4 and NZBDrone 2.0.0.1861.

Have you been able to figure out what’s causing this? I’m lucky if I can last 6 hours before it freezes.

@caleb @tuffcalc do you get the same error message?
@caleb please provide more information about your setup, mono version, etc.
@tuffcalc can you try updating to mono 3.6? Its available at: http://www.mono-project.com/download/
@fubduck Are you using updating from within drone? Do you have a database backup from before the update? It may be worthwhile to restore that backup and revert drone to a previous version, which you can get here: http://update.nzbdrone.com/v2/master/mono/NzbDrone.master.2.0.0.1690.mono.tar.gz

@markus101 I just updated mono to version 3.6. I will let you know if it solves the problem…

@fubduck Are you using updating from within drone? Do you have a database backup from before the update?

I do not use the update in drone. I install the updates using webmin.

edit: I upgraded to mono 3.6 as well.
Mono Version 3.6.0 (tarball Wed Aug 20 13:05:36 UTC 2014)

When I installed the new version of Mono it blew away my drone install. Luckily my AppData Directory was still intact, so I reinstalled the latest drone. It is up and running now. My apologies I forgot about the downgrade you mentioned earlier.

Same thing here. Started on 26-Aug several hours after autoupdating to 2.0.0.1861.

Mac OS X 10.9.4
nzbdrone Version 2.0.0.1861
Mono Version 3.2.4 ((no/294f999 Fri Oct 25 20:18:12 EDT 2013)
AppData directory /Users/me/.config/NzbDrone
Startup directory /Users/me/NzbDrone

Any chance of an update soon to correct this issue?

@emtayl00 no chance of an update until we understand why its happening. Considering we have received logs from one user and a bunch of “me too” comments, we have very little to go on, except that it seems to affect people with mono 3.2.x.

Please post trace logs so we can investigate. Trace logging will help us determine what operation is executing when it crashes. https://github.com/NzbDrone/NzbDrone/wiki/Log-Files

Yes, same message. My db is on local disc, the drone factory is on an external DAS connected via thunderbolt. I only started having these on the latest update to 2.0.0.1861, however, I never updated to 2.0.0.1855 or 2.0.0.1856 but went straight from 2.0.0.1690 so this issue may exist in those releases as well (I can’t say one way or another).

Currently I’m on mono 3.2.4,

mono -V
Mono JIT compiler version 3.2.4 ((no/294f999 Fri Oct 25 20:18:12 EDT 2013)
Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  x86
	Disabled:      none
	Misc:          softdebug 
	LLVM:          yes(3.3svn-mono)
	GC:            sgen

I’ll upgrade to 3.6 and see if that helps. I too don’t get more than a couple hours before it goes down, usually I check it every day or two which is when I notice it.

My system NZBDrone is running on is OS X 10.9.4 as well. I update manually using a shell script:

#!/bin/bash

cd ~/Downloads && wget http://update.nzbdrone.com/v2/master/osx/NzbDrone.master.osx.tar.gz
tar zxf ~/Downloads/NzbDrone.master.osx.tar.gz
if [ -d /Applications/Nzbdrone.bak ]
then
  rm -rf /Applciations/Nzbdrone.bak
fi
ps aux | grep mono | grep -v grep | awk {'print $2'} | xargs kill
mv /Applications/Nzbdrone /Applications/Nzbdrone.bak && mv ~/Downloads/Nzbdrone /Applications/
rm -rf NzbDrone.master.osx.tar.gz
launchctl start com.nzbdrone.launched.nzbdrone

I’ll downgrade to 1690 right now as well and see if the problem persists.

Looks like a downgrade to 1690 doesn’t work as something in the db was changed. The logs are spewing the following:

System.Data.SQLite.SQLiteException: SQL logic error or missing database
no such column: t0.QualityProfileId
  at System.Data.SQLite.SQLite3.Prepare (System.Data.SQLite.SQLiteConnection cnn, System.String strSql, System.Data.SQLite.SQLiteStatement previous, UInt32 timeoutMS, System.String& strRemain) [0x00000] in <filename unknown>:0 
  at System.Data.SQLite.SQLiteCommand.BuildNextCommand () [0x00000] in <filename unknown>:0 

It’s also not allowing nzbdrone to fully start. I’ve put my logs back in to Trace, but the error messages look identical to what was posted in the first post. Not sure what having additional logs saying the exact same thing will provide you, but I’ll do it.

You would have to restore an old version of the DB, since there were many changes.

I ask because the sqlite error and the crashing don’t happen at the same time, so they might not be related.

Upgrading mono and running the same version of drone makes sense because I want to know if we’re exposing a mono issue that’s now fixed in 3.6, downgrading drone isn’t going yo give us much.

When drone crashes is mono still running?
Did these errors appear before this version of drone?
Did drone crash in the same way before this version of drone?

So I’ve been running mono 3.6 for right about 24 hours, no crashes. The weird thing on the “crashes” is it’s still running, th epic is still an active pid, the process still shows active, but the interface just hangs. No response (but no 404, 500, etc…but no 200 either…just no response at all). So yes, everything still shows mono running, but drone shows running too. Never saw these errors prior to this version of drone. Also never saw a crash like this prior to this version of drone. FWIW, after upgrading to 3.6, I haven’t seen any more of these error either:

$ pwd
/Users/<me>/.config/NzbDrone/logs
$ grep -i database *
nzbdrone.2.txt:14-9-3 20:21:19.1|Info|Database|Vacuuming database
nzbdrone.2.txt:14-9-3 20:21:19.3|Info|Database|Database Compressed
$

I upgraded to Mono 3.6 a little over 24 hours ago as well and have not had any more problems.

Thanks for the feedback, please keep me posted over the next few days to see if it continues working well.

Upgraded to mono 3.6.0 - now approaching 48 hours with no problems. Looks like that took care of it. Thanks.

I’m coming up on 72 hours, after installing Mono 3.6, with no issues.

I noticed that there is a new version of Mono available, 3.8. Should we stay on 3.6 or update to 3.8?

Its a day old and I’m not sure what to expect, there might be issues, so I’d say stick with 3.6 for now.