Mono didn't support intermediate SSL certificate chains causing problems between nzbToMedia and NzbDrone/Sonarr [fix]

I am having difficulties getting nzbToMedia talk properly to my SSL enabled nzbdrone port… It says it has trouble connecting to it. I SUSPECT it may be because nzbdrone appears to attempt to do a certificate level authentication process… or perhaps because the python SSL code doesn’t trust my StartCom SSL key… I tried modifying the code to not verify the SSL cert, but that doesn’t seem to help… and testing with curl (w/ -k to ignore the untrusted cert) seems to throw a connection failure message too.

I’m running latest of everything, AFAIK:
NZBGet 14.0-testing-r1146
nzbdrone 2.0.0.2064 (on linux)
nzbToMedia commit c7d4160a06333aa99ebbefeacc0780dc7c84ec99 (latest one)

Curl test output from the local machine, and servername.removed.com resolves to the eth0 IP address, which nzbdrone is listening on:

[code]curl -v https://servername.removed.com:9898

However, when I load up https://servername.removed.com:9898 in Chrome, it happily connects but prompts me with a “Select a certificate” pop up. I typically hit cancel at that screen and then it prompts me for HTTP basic authentication. Then I login and it works fine.

If I switch nzbToMedia in NZBGET to NOT utilize SSL for nzbdrone (and switch to the other non-ssl port), then the post processing script seems to work.

So, my long winded question is, what is causing the failure, and how can I use SSL w/o errors? And a side question of how I can either utilize or silence the certificate authentication pop up of nzbDrone.

Thank you!

drone doesn’t do anything to require a client side certificate, so I’m not sure where that is coming from.

What flavour/version of Linux?
What version of mono?

That’s interesting… it could be a StartCom SSL thing, then… come to think of it, it seems to only ask for the cert from my laptop that I generated my StartCom user cert on (required to create the SSL cert later)… on my chromebook, it doesn’t ask which cert to use… But alas, my problem still exists… nzbToMedia scripts can’t seem to talk via SSL to nzbdrone… does that work for anyone else?

Ubuntu 14.04.1 LTS
Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1)

This was interesting output… 9898 is my nzbdrone SSL port now… I swapped it… 8989 is non-ssl… This is what openssl has to say about it, even though Chrome can access it just fine… though it does pop up a cert auth box…

# openssl s_client -host servername.removed.com -port 9898  
CONNECTED(00000003)
139875413264032:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1414014594
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Running that same openssl command on another port on my box that uses the same SSL cert, and the output is MUCH different… streaming the certificate, and cert description, TLSv1.2, etc… Something is wonky with nzbdrone… Perhaps TLSv1 isn’t supported anymore…

Certificate handling is done outside of drone/sonarr, it is done within mono/the OS (on Windows its done through a component of Windows). When you switched the port did you run the external config tool again?

https://github.com/NzbDrone/NzbDrone/wiki/SSL#linux--os-x - make sure its run by the user that will be running mono.

markus101,
Thanks for the suggestions… I’ve done the following, and unfortunately, no change occurred.

On a windows box:

pvk -in server.key -topvk -nocrypt -out server.pvk

That essentially turned a base64 key into a binary key… I don’t think the original one had a password on it.

Then I copied server.pvk back onto the Linux machine and ran mono’s

$ httpcfg -list
Port: 9898 Thumbprint: BA9457B1770D2450BEA6480E7F01A787562B9002
$ httpcfg -del -port 9898
$ httpcfg -add -port 9898 -pvk server.pvk -cert server.cert 
$ service nzbdrone restart
Restarting NzbDrone
$ httpcfg -list
Port: 9898 Thumbprint: BA9457B1770D2450BEA6480E7F01A787562B9002

As you can see, the thumbprints remained the same before and after.

Unfortunately, the openssl command output remained the same as my previous post…

Did I misunderstand you? Did I miss a necessary step? I sure hope so!
Btw, this cert is a startssl.com signed cert. They are free and work great w/ every other software I’ve thrown them at on this same machine. (sab/couchpotato/nzbget/etc)

Btw, my nzbdrone config.xml file looks like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Config>
  <Port>8989</Port>
  <SslPort>9898</SslPort>
  <EnableSsl>True</EnableSsl>
  <LaunchBrowser>False</LaunchBrowser>
  <ApiKey>removed</ApiKey>
  <AuthenticationEnabled>True</AuthenticationEnabled>
  <Branch>master</Branch>
  <Username>removed</Username>
  <Password>removed</Password>
  <LogLevel>Info</LogLevel>
  <SslCertHash>
  </SslCertHash>
  <UrlBase>
  </UrlBase>
  <UpdateMechanism>BuiltIn</UpdateMechanism>
  <UpdateAutomatically>True</UpdateAutomatically>
</Config>

I just noticed that SslCertHash tag in there… what populates that? Perhaps that’s what I’m missing!

Thanks again!

I’ve done some surfing, as you got me pointed in the right direction with regard to mono and httpcfg… It appears that mono’s httpcfg method of opening an SSL port does not support Intermediate CA certs… so I’m a bit out of luck… oddly, Chrome doesn’t seem to give a crap… but it does seem to cause it to ask for a client certificate when I hit it the first time. I saw someone else mention that…

The suggestion is to use an SSL proxy server to offload the SSL… specifically using nginx in this manner… https://www.startssl.com/?app=42

I guess I’ll keep using the http port of nzbdrone/sonarr and maybe someday mono/httpcfg will be fixed to support intermediate CA’s… Isn’t that sort of an important feature these days???

Doh! I totally forgot about this; to answer your later question, yes, they are pretty much required these days.

@hildebrau - can you use the startssl root PEMs instead of the intermediates?

https://www.startssl.com/certs/

Also, a side note: disable SSLv3 as it’s exploitable.

LJ

I’ve read that the Intermediate certs are supported in the latest code of “mono”… I’m having trouble finding any more recent Ubuntu packages for Mono, so I’m going to attempt to build from source and see what happens… Probably not worth all this hassle; honestly… but now I’m on a mission…

Ok… just to close the loop on this a little bit… I ended up NOT having to build mono from source… I found a repository that has up to date mono releases built… I did jump through some hoops before I found this out, so what I’m outlining below isn’t the exact process I went through… but it is what I think will work for someone else if anyone is having SSL troubles w/ nzbdrone.

Add this repo to debian/ubuntu:
sudo add-apt-repository ppa:inizan-yannick/mono
sudo apt-get update
sudo apt-get upgrade

That should update mono to 3.10 (or newer) (which looks older than the stock ubuntu package, but apparently the decimal system doesn’t mean anything in versioning… 3.10 is > 3.2 believe it or not…

Then you can TRY firing up nzbdrone and seeing if openssl reports a proper cert via this test:
openssl s_client -host localhost -port

As mentioned before, this is not the exact path I took… so if that doesn’t work right off the bat, you can try these two things…

  1. rerun the httpcfg step to see if the updated mono/httpcfg will fix your cert install
    OR
  2. follow the linux instructions on this page which is just another application that uses “mono” and had similar issues… they patched their version of mono (which you just did above) and then they run this exe which is a package of CMD scripts that needs to be run on a Windows box. To run that, drop your private key in your current directory (on windows) and name it ScreenConnectPrivateKey.key and have your third party signed cert in the same current directory named with a .CER extension (not .CERT or anything else)… then follow the script’s prompts to set your current directory and doing the linux path and then the 3rd step to build a linux package for your cert. I forget how they worded it exactly. You’ll end up with a ScreenConnectSslFiles.tar.gz tarball that you can transfer back to your linux box and untar it, then rsync (or whatever recursive copy/move method you desire) the .mono subdirectory out of the App_Runtime/etc/ path prefix over to where ever your nzbdrone/sonarr .mono directory tree is… if you run nzbdrone on a port other than 443, you’ll need to go rename the two port files in .mono/httplistener from 443.* to whatever.*…

That should be it… just restart nzbdrone/sonarr afterwards and rerun the openssl test… It worked for me… I did the ScreenConnect script step first and it didn’t work because my mono version was too old… then after upgrading mono, it works wonderfully. If you have/had this issue, please report back if you had to do the ScreenConnect step or not.

Yeah, its not decimal and definitely feels a bit weird, but version numbers are typically 3 or 4 digits long, like sonarr: 2.0.0.x (where x is now 2000+).

Thanks for sharing the steps, I’m sure it will help someone else in the future.