Sonarr hanging on Ubuntu 16.04?

[tommycahir]

Sonarr version (exact version): version (2.0.0.5252)
Mono version (if Sonarr is not running on Windows):
Mono JIT compiler version 5.16.0.220 (tarball Mon Nov 26 17:22:46 UTC 2018)
Copyright © 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
Interpreter: yes
LLVM: yes(3.6.0svn-mono-/)
GC: sgen (concurrent by default)

OS: Ubuntu 16.04
Debug logs: https://pastebin.com/rwArt0zL
Description of issue:

Over the last few weeks Sonarr has been hanging for me and not opening, I usually have to reboot my server to get it to start up again and will work for a few hours to days before it screws up again. I see an error in the logs about the address already being in use but I have changed the port from 8989 to 8282 to try work around this but still I’m seeing these errors.
This seems to have got worse since a system update last week but not sure if that is related to a possible mono update (it doesn’t effect Radarr or Lidarr though)

I checked the network connections and see the following on the box

Any help at all would be appreciated.
Regards
Tommy

[markus101]

mono 5.16 has been problematic, but I haven’t seen it cause this. I’d still recommend downgrading to 5.14 or earlier though.

When it’s failing to start is there another instance running or is something else bound to that port?

[tommycahir]

No nothing else running when I check processes using htop and no other application is listening on that port as you can see from the above.
I’ve uninstalled, purged, rebooted and installed from scratch again and now seems to be working. Just slowly adding all the media back in again and configuring it.

[tommycahir]

Since the original post above I have installed a new ssd hdd in my server which has a fresh minimal install of ubuntu 18.04 and reinstalled sonarr v2 and mono v5.14 as per below

Mono JIT compiler version 5.14.0.177 (tarball Mon Aug  6 09:07:45 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          yes(3.6.0svn-mono-/)
	GC:            sgen (concurrent by default)

Checking my logs I start seeing an error appear at ~12:42 today (7 Jan 2019) - Link to Logs https://pastebin.com/Kd5JX9de

Any help would be appreciated as it is unusable at the moment, when I check the service status I get the below

tommy@harold:~$ sudo systemctl status sonarr.service 
● sonarr.service - Sonarr Daemon
   Loaded: loaded (/lib/systemd/system/sonarr.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-01-07 15:13:21 GMT; 2s ago
 Main PID: 21764 (mono)
    Tasks: 6 (limit: 4915)
   CGroup: /system.slice/sonarr.service
           ├─ 1127 /usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser
           └─21764 /usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser

Jan 07 15:13:21 harold systemd[1]: Started Sonarr Daemon.
Jan 07 15:13:22 harold mono[21764]: [Info] Bootstrap: Starting Sonarr - /opt/NzbDrone/NzbDrone.exe - Version 2.0.0.5252
Jan 07 15:13:24 harold mono[21764]: [Info] Router: Application mode: Interactive

[markus101]

You’re going to need to turn on trace logging to see exactly what Sonarr was doing in this gap:

19-1-7 12:42:58.0|Info|DiskScanService|Scanning disk for Altered Carbon
19-1-7 12:56:11.8|Info|Bootstrap|Starting Sonarr - /opt/NzbDrone/NzbDrone.exe - Version 2.0.0.5252

My guess is you have an ancient version of libmediainfo installed and a particular file in that series is causing libmediainfo to crash and takes out mono/Sonarr. Make sure you install the latest version of libmediainfo from their repository, not the latest in the Ubuntu repo.

[tommycahir]

I updated mediainfolib to latest version from their site as per below, it was previously on v17.12(iirc)

I am still seeing the same behaviour though, I enabled the trace logging @ 17:30 or so and rebooted earlier, it hung again, i then updated mediainfolib @ 20:40 and rebooted (you can see it coming online @ 20:47 in the below logs and it is still hanging trying to access the site.

Trace logs - https://pastebin.com/PhvMJgAu
Debug Logs - https://pastebin.com/m5GGmEj6
Standard logs - https://pastebin.com/N7Jqukp8

Mediainfolib version -

MediaInfo Command line,
MediaInfoLib - v18.12

Sonarr Status

`tommy@harold:~$ sudo systemctl status sonarr.service
● sonarr.service - Sonarr Daemon
   Loaded: loaded (/lib/systemd/system/sonarr.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-01-07 20:47:05 GMT; 15min ago
 Main PID: 848 (mono)
    Tasks: 16 (limit: 4915)
   CGroup: /system.slice/sonarr.service
           └─848 /usr/bin/mono /opt/NzbDrone/NzbDrone.exe -nobrowser

Jan 07 20:47:45 harold mono[848]: [Trace] EventAggregator: EpisodeInfoRefreshedEvent <- EpisodeAddedService
Jan 07 20:47:45 harold mono[848]: [Info] RefreshEpisodeService: Finished episode refresh for series: [268592][The 100].
Jan 07 20:47:45 harold mono[848]: [Debug] RefreshSeriesService: Finished series refresh for The 100
Jan 07 20:47:45 harold mono[848]: [Trace] EventAggregator: Publishing SeriesUpdatedEvent
Jan 07 20:47:45 harold mono[848]: [Trace] EventAggregator: SeriesUpdatedEvent -> DiskScanService
Jan 07 20:47:46 harold mono[848]: [Info] DiskScanService: Scanning disk for The 100
Jan 07 20:47:46 harold mono[848]: [Trace] EventAggregator: Publishing CommandUpdatedEvent
Jan 07 20:47:46 harold mono[848]: [Trace] EventAggregator: CommandUpdatedEvent -> CommandModule
Jan 07 20:47:46 harold mono[848]: [Trace] EventAggregator: CommandUpdatedEvent <- CommandModule
Jan 07 20:47:46 harold mono[848]: [Debug] DiskScanService: Scanning '/media/TVShows/The 100' for video files
`

[markus101]

Is Sonarr still running and writing to the logs and you can’t access the UI or that’s stopped as well?

If Sonarr isn’t logging anything it sounds like it’s crashed, which may not be in the logs, in which case you’d need to capture the standard output and error to a file to see what’s happening.

[tommycahir]

Sonarr is not responding at all (the site will remain up and can swap between sections but nothing will load from the DB such as list of episodes, logs, etc) and I need to reboot the server to get it started up again.

Below is the result of sending stdout and stderr to a file with trace level of logging enabled aswell
https://pastebin.com/Qf9wRK2t

*edited to add more detail

[tommycahir]

OK I’ve been testing for the last few days and only thing that I can see is the following error in the console output
The freezing/hanging only happens when moving the files to my NAS which is mounted as per below on fstab

[Debug] DiskTransferService: HardLinkOrCopy [/home/tommy/Downloads/Completed/TV Shows/The.Orville.S02E03.720p.WEB.x264-TBS[rarbg]/the.orville.s02e03.720p.web.x264-tbs.mkv] > [/media/TVShows/The Orville/Season 02/The Orville.S02E03.mkv] 
[Debug] DiskProvider: Hardlink '/home/tommy/Downloads/Completed/TV Shows/The.Orville.S02E03.720p.WEB.x264-TBS[rarbg]/the.orville.s02e03.720p.web.x264-tbs.mkv' to '/media/TVShows/The Orville/Season 02/The Orville.S02E03.mkv' failed. 

[v2.0.0.5252] Mono.Unix.UnixIOException: Invalid cross-device link [EXDEV].
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <6ecdf4972be240398b3e321fbad963dd>:0 
  at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <6ecdf4972be240398b3e321fbad963dd>:0 
  at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <6ecdf4972be240398b3e321fbad963dd>:0 
  at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Mono\Disk\DiskProvider.cs:182 

[Trace] SymbolicLinkResolver: Checking path /media/TVShows/The Orville/Season 02/The Orville.S02E03.mkv for symlink returned error ENOENT, assuming it's not a symlink. 

My fstab file has the following entry for the TV Shows folder where these are being moved to

//192.168.1.150/Multimedia/TV\040Shows /media/TVShows cifs credentials=/home/tommy/.smbcredentials,file_mode=0776,dir_mode=0776,gid=1001,iocharset=utf8 0 0

Sonarr is running under root user and the mediamnt group has the following members

tommy@harold:/media$ ls -al
total 140
drwxr-xr-x  14 root root      4096 Jan 10 15:35 .
drwxr-xr-x  24 root root      4096 Jan  3 22:32 ..
drwxrwxrwx   1 root root      4096 Sep 25 12:18 Fitness
drwxrwxrw-   2 root root         0 Jul 24  2017 HomeMovies
drwxrwxrwx   1 root root      8192 Jan 10 17:47 KidsCartoons
drwxrwxrwx   1 root root     36864 Jan  4 17:28 KidsMovies
drwxrwxrw-   2 root mediamnt     0 Jan  4 22:18 Movies
drwxrwxrwx   1 root root     69632 Dec 13 16:42 Music
drwxrwxrw-   2 root mediamnt     0 May  8  2017 Pictures
drwxrwxrwx   1 root root      4096 Jan  3 08:43 Random
drwxr-xr-x+  3 root root      4096 Jan  3 21:16 tommy
**drwxrwxrw-   2 root mediamnt     0 Jan 11 13:05 TVShows**
drwxr-xr-x   2 plex mediamnt  4096 Jan 10 15:35 TVShows2
drwxrwxrwx   1 root root      4096 Nov 25 18:11 Youtube

mediamnt:x:1001:plex,tommy,root

[markus101]

Sonarr is not responding at all (the site will remain up and can swap between sections but nothing will load from the DB such as list of episodes, logs, etc) and I need to reboot the server to get it started up again.

Once the UI is loaded you can move around the app and data will load from the server (assuming it is able to).

Nothing in the job looks abnormal and Sonarr/mono doesn’t appear to be crashing.

OK I’ve been testing for the last few days and only thing that I can see is the following error in the console output

That error shouldn’t be causing any issues, just Sonarr trying to hardlink a file and failing because it’s not possible, which is expected with a SMB/CIFS share.

The freezing/hanging only happens when moving the files to my NAS which is mounted as per below on fstab

And once a file transfer takes place you can’t access the UI again until you reboot?

Are you/have you tried using NFS instead of SMB/CIFS?
SMB shares with Sonarr running under mono have been problematic in many cases, mostly with truncated files, but we’ve seen other issues as well and if memory serves, we’ve seen this issue, though it’s been quite rare.

NFS on the other hand hasn’t exhibited any of these issues (that I recall at least).

[tommycahir]

Once the UI is loaded you can move around the app and data will load from the server (assuming it is able to).

Yes I can move around the UI but it will not load anything from the DB such as episode lists, won’t load the settings or even under the general tab won’t load the directory information.

And once a file transfer takes place you can’t access the UI again until you reboot?
Are you/have you tried using NFS instead of SMB/CIFS?

Yes correct that is what usually happens.
I haven’t tried NFS shares yet and will mess around with them to see if I can get them working.
Since replying above the service has been up and running with no issues for 3 days now and has downloaded and transferred files over that very cifs share.

I’ll have a look at NFS shares but for the moment i ain’t touching anything

regards
Tommy