I just moved over from SickBeard and really like NZBDrone. Great job!
I installed NZBDrone on my Ubuntu 12.04 LTS box flawlessly, but have noticed that it’s running with 100% cpu usage all the time (mono is #1 entry on ‘top’). If I restart it, it will sit at 0.2% CPU for a while until it fetches a release, etc., then it will just sit at 100% again.
Has anyone else experienced this? Any clues on how to fix/debug this?
I’ve noticed mine will occasionally climb, but I haven’t seen 100% in several releases. Honestly, I think I’ve had to restart it more due to updates than climbing cpu usage of late.
Yeah I updated yday but still have this issue. Everything fine on startup but as soon as it downloads/processes a release, mono pegs at 100% CPU until nzbdrone is restarted.
Bit of a deal breaker for switching from sickbeard.
Please enable trace logging and grab the log file from start to 100% (delete the log file then start drone). Want to see if anything weird is happening.
I’m getting the same issue on linux running v2.0.0.855
This is from the logs:
13-11-20 20:32:34.2|Info|EpisodeSearchService|Episode search completed. 1 reports downloaded.
13-11-20 20:36:58.6|Fatal|GlobalExceptionHandlers|EPIC FAIL: Object reference not set to an instance of an object
System.NullReferenceException: Object reference not set to an instance of an object
at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessCommand (Microsoft.AspNet.SignalR.Messaging.Command command) [0x00000] in :0
at Microsoft.AspNet.SignalR.Infrastructure.Connection.b__8 (System.Object state, Microsoft.AspNet.SignalR.Messaging.Message message) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBusExtensions.Enumerate[Object] (IList1 messages, System.Func2 filter, System.Action2 onMessage, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessResults (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.GetResponse (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection+ReceiveContext.InvokeCallback (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.MessageBusCallback (MessageResult result, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.<Receive>b__4 (MessageResult result, System.Object s) [0x00000] in <filename unknown>:0 at (wrapper delegate-invoke) System.Func3<Microsoft.AspNet.SignalR.Messaging.MessageResult, object, System.Threading.Tasks.Task1<bool>>:invoke_TResult__this___T1_T2 (Microsoft.AspNet.SignalR.Messaging.MessageResult,object) at Microsoft.AspNet.SignalR.Messaging.Subscription.Invoke (MessageResult result, System.Action1 beforeInvoke, System.Object state) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.Subscription.WorkImpl (System.Threading.Tasks.TaskCompletionSource1 taskCompletionSource) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Messaging.Subscription.Work () [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpImpl (System.Threading.Tasks.TaskCompletionSource1 taskCompletionSource, ISubscription subscription) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpAsync () [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.ProcessWork (System.Object state) [0x00000] in :0
[Info] Router: Application mode: Interactive
[Info] OwinHostController: starting server on http://*:8989/
[Info] NancyBootstrapper: Starting NzbDrone API
[Info] SceneMappingService: Updating Scene mapping
[Info] HousekeepingService: Running housecleaning tasks
[Warn] GlobalExceptionHandlers: SignalR Heartbeat error.
EPIC FAIL: System.NullReferenceException: Object reference not set to an instance of an object
at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessCommand (Microsoft.AspNet.SignalR.Messaging.Command command) [0x00000] in :0
at Microsoft.AspNet.SignalR.Infrastructure.Connection.b__8 (System.Object state, Microsoft.AspNet.SignalR.Messaging.Message message) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBusExtensions.Enumerate[Object] (IList1 messages, System.Func2 filter, System.Action2 onMessage, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessResults (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.GetResponse (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection+ReceiveContext.InvokeCallback (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.MessageBusCallback (MessageResult result, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.<Receive>b__4 (MessageResult result, System.Object s) [0x00000] in <filename unknown>:0 at (wrapper delegate-invoke) <Module>:invoke_Task1__this___MessageResult_object (Microsoft.AspNet.SignalR.Messaging.MessageResult,object)
at Microsoft.AspNet.SignalR.Messaging.Subscription.Invoke (MessageResult result, System.Action1 beforeInvoke, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Messaging.Subscription.WorkImpl (System.Threading.Tasks.TaskCompletionSource1 taskCompletionSource) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.Subscription.Work () [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpImpl (System.Threading.Tasks.TaskCompletionSource`1 taskCompletionSource, ISubscription subscription) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpAsync () [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.ProcessWork (System.Object state) [0x00000] in :0
[Fatal] GlobalExceptionHandlers: EPIC FAIL: Object reference not set to an instance of an object
System.NullReferenceException: Object reference not set to an instance of an object
at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessCommand (Microsoft.AspNet.SignalR.Messaging.Command command) [0x00000] in :0
at Microsoft.AspNet.SignalR.Infrastructure.Connection.b__8 (System.Object state, Microsoft.AspNet.SignalR.Messaging.Message message) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBusExtensions.Enumerate[Object] (IList1 messages, System.Func2 filter, System.Action2 onMessage, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.ProcessResults (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.GetResponse (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection+ReceiveContext.InvokeCallback (MessageResult result) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.MessageBusCallback (MessageResult result, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Infrastructure.Connection.<Receive>b__4 (MessageResult result, System.Object s) [0x00000] in <filename unknown>:0 at (wrapper delegate-invoke) <Module>:invoke_Task1__this___MessageResult_object (Microsoft.AspNet.SignalR.Messaging.MessageResult,object)
at Microsoft.AspNet.SignalR.Messaging.Subscription.Invoke (MessageResult result, System.Action1 beforeInvoke, System.Object state) [0x00000] in <filename unknown>:0 at Microsoft.AspNet.SignalR.Messaging.Subscription.WorkImpl (System.Threading.Tasks.TaskCompletionSource1 taskCompletionSource) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.Subscription.Work () [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpImpl (System.Threading.Tasks.TaskCompletionSource`1 taskCompletionSource, ISubscription subscription) [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.PumpAsync () [0x00000] in :0
at Microsoft.AspNet.SignalR.Messaging.MessageBroker.ProcessWork (System.Object state) [0x00000] in :0
Does the CPU spike up after that error? Haven’t seen that error cause any issues, it just logs periodically when running under mono and continues on. Though @r0b07 's error looks like its actually two.
Need some more information as well:
What version of drone?
What version of Ubuntu?
What version of mono?
For next time:
Its much easier to see logs that are posted on pastebin as opposed to the body of the reply, it also keeps the length of the thread down.
I’ve run into this problem on ubuntu 13.10 with mono v2 and compiled from source v3.2.3 and nzbdrone 740 through 854 (i update a lot since i don’t mind bugs or work in progress).
When i had some spare time i ran a profiler on the process to see what was up during the spike, here are the results (I forget how long of a time period i had the profiler running): http://pastebin.com/Nx2DvSkD
I’m not totally sure how SignalR -> nzbdrone works yet (haven’t had the time to look through it all), but hopefully this can give you a starting point as to what’s wrong.
Either way, i recompiled signalr.core.dll with a thread.sleep(100) instead of the spinwait and haven’t had an issue in 24hours: http://imgur.com/R94Ani2 (7day graph, those spikes to the left are earlier in week the mono nzbdrone process at 100%)
My setup is a bit out of the ordinary because i have nzbdrone and sabnzbd behind an nginx server, so this may have been part of the cause due to timeout settings on the passthrough and thought the cpu @ 100% was just me.
Will continue to keep an eye on it and notify back with more info if needed.
Awesome, look forward to having an updated NZBDrone with thread.sleep();
Moffisaurus, in the meantime, is there any way to get the updated signalr dll from you? Have been trying to build my own, but not at all familiar with C# and NuGet is driving me a bit bonkers.