Torrent RSS Casting Error

jasonfxp · March 29, 2015, 7:04pm

Hi,

Using the bitmetv RSS feed, Sonarr is successfully matching items on the feed, but failing to download the .torrent and pass it to the download client. The debug log is below, but I’m guessing it’s some bad parsing of the URL from the RSS?

[Warn] ProcessDownloadDecisions: Couldn’t add report to download queue. REM
System.InvalidCastException: Cannot cast from source type to destination type.
at NzbDrone.Common.Http.HttpClient.Execute (NzbDrone.Common.Http.HttpRequest request) [0x0000c] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Common\Http\HttpClient.cs:36
at NzbDrone.Common.Http.HttpClient.Get (NzbDrone.Common.Http.HttpRequest request) [0x00007] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Common\Http\HttpClient.cs:164
at NzbDrone.Core.Download.TorrentClientBase1[NzbDrone.Core.Download.Clients.Transmission.TransmissionSettings].DownloadFromWebUrl (NzbDrone.Core.Parser.Model.RemoteEpisode remoteEpisode, System.String torrentUrl) [0x00021] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Download\TorrentClientBase.cs:106 at NzbDrone.Core.Download.TorrentClientBase1[NzbDrone.Core.Download.Clients.Transmission.TransmissionSettings].DownloadFromWebUrl (NzbDrone.Core.Parser.Model.RemoteEpisode remoteEpisode, System.String torrentUrl) [0x0008f] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Download\TorrentClientBase.cs:121
at NzbDrone.Core.Download.TorrentClientBase1[NzbDrone.Core.Download.Clients.Transmission.TransmissionSettings].Download (NzbDrone.Core.Parser.Model.RemoteEpisode remoteEpisode) [0x00093] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Download\TorrentClientBase.cs:85 at NzbDrone.Core.Download.DownloadService.DownloadReport (NzbDrone.Core.Parser.Model.RemoteEpisode remoteEpisode) [0x000fb] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Download\DownloadService.cs:44 at NzbDrone.Core.Download.ProcessDownloadDecisions.ProcessDecisions (System.Collections.Generic.List1 decisions) [0x00146] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Download\ProcessDownloadDecisions.cs:74

Any ideas?

Thanks!

markus101 · March 30, 2015, 5:40am

Please pastebin trace logs hopefully it will give a bit more information about the error, looks like the URL might be invalid, but not 100% certain.

jasonfxp · March 30, 2015, 7:05am

Thanks, log here: http://pastebin.com/5nXT78ZT

So it looks like it gets a valid HTTP url out of the RSS, and I can manually download that fine, but Sonarr gets a 302 redirect to login.php and it all goes wrong.

This sort of implies its not using the cookie data correct, but I have that all setup and it passes the indexer test fine…

Thanks!

markus101 · March 30, 2015, 4:59pm

Sonarr sends the cookie with the RSS feed request, but if memory serves its not actually required to get the RSS, so its possible the cookie is incorrect and causes it to fail when grabbing it, at least thats my educated guess.

Taloth · March 30, 2015, 7:11pm

That’s surprising coz we added the cookie to the rss after a bitmetv user helped us fix it. I assumed that that user also tested downloads. We didn’t add the cookie to the download url.

This whole cookie stuff at bitmetv is a pain

jasonfxp · April 1, 2015, 8:42am

Agreed! After some more digging the cookie is definitely not sent in the torrent GET request which is why things go wrong.

GET /download.php/…/NAME.torrent HTTP/1.1
User-Agent: Sonarr/2.0.0.3004 (Linux 3.13)
Accept: application/x-bittorrent
Content-Length: 0
Connection: close
Host: www.bitmetv.org
Accept-Encoding: gzip

And that’s it - but it does go out with the rss.php request.

The cookie I use (copied from real req) in is is of the form: _cfduid=X; PHPSESSID=X; uid=X; pass=X; LAST_BROWSE=X

Any ideas why it wouldn’t be sent?

jasonfxp · April 1, 2015, 9:58pm

Fairly certain the bug is:

Cookie is sent with rss.php query (where it’s not required) instead of to download.php where it is required.

Any chance of a fix in future versions?

Taloth · April 2, 2015, 7:14pm

It should be fixed now on develop 2.0.0.3037. Please lemme know if it works, coz I can’t test it.

jasonfxp · April 12, 2015, 8:00pm

Hi,

Sorry for the late response! Testing with latest development, a cookie now gets sent with the download.php request, however it is not the cookie defined in the indexer settings.

This is what happens:

GET rss.php with no cookie, which is all correct
The HTTP response to this, contains a Set-Cookie header, sonarr seems to save this and sends it out with the download.php request, rather than the one defined in index.php. It’s just a cloud hosting cache cookie type thing that gets set.

I hope that makes sense as a bug, so basically the set-cookie header is saved and used for download.php rather than the one defined in the indexer settings.

Thanks very much!

Taloth · April 13, 2015, 8:07pm

Ah, so the CookeContainer I created to preserve the stored cookie between rss and download link gets overridden by the cookie returned on the rss call? Didn’t see that one coming.

It’s odd though, wouldn’t that mess up your browser too when you access the rss feed? Could you confirm that?

jasonfxp · April 15, 2015, 5:45pm

Hi,

Ok done some more debugging…

If you visit the rss.php in the browser without being logged in, no cookie is sent. This all works because the pass keys are all in the GET request. A cloudfair cookie id is returned with the rss data in Set-Cookie header - Sonarr needs to ignore this.

Note that if you’re logged in, the browser will send the cookie anyway to rss.php, the Set-cookie is then appended to the existing cookie, rather than replacing it. i.e. the __cfduid part is added/updated in the cookie.

Sonarr currently sends no cookie to rss.php and gets a Set-Cookie back with __cfduid set (same as a browser).
When Sonnar visits download.php, only the __cfduid cookie is sent - nothing in the indexer settings cookie goes out. As a result, you get a redirect to login.php.

This means either:

Set-Cookie completely replaces the indexer defined cookie
Sonarr is using the new cookie and just ignoring the indexer defined cookie.

My best guess!

jasonfxp · April 15, 2015, 6:37pm

Sorry, I missed your fix checkin: https://github.com/Sonarr/Sonarr/commit/ee8753784838f5e3dc2c8f00be57f9f8e671fe03

Using latest code, no cookie at all now goes to download.php! Seems like it is maybe failing to lookup the cookie in the indexer settings?

I’ve tried deleting/creating the indexer again, cookie is definitely there but not hitting the wire.

scottcase · June 4, 2015, 12:11pm

Any update on this. Using the latest dev version of sonarr Version2.0.0.3211 and still can not get BitMeTV to work with either the BitMeTV indexer or the Torrent RSS feed

Taloth · June 4, 2015, 7:42pm

I checked it again, everything seems to be fine. Unless the download url is a diferrent host than the feed.

I don’t have BitMeTv, so all information has to come from you guys.

This includes stuff like a capture of the actual http traffic.

sam · June 27, 2015, 9:33am

I am having issues connecting to BitMeTV on Sonarr version 2.0.0.3243 and it throws the following error:

Message:

Unable to connect to indexer: Indexer API call resulted in an unexpected StatusCode [Redirect]: Indexer API call resulted in an unexpected StatusCode [Redirect]

Exception:

   NzbDrone.Core.Indexers.Exceptions.IndexerException: Indexer API call resulted in an unexpected StatusCode [Redirect]
  at NzbDrone.Core.Indexers.RssParser.PreProcess (NzbDrone.Core.Indexers.IndexerResponse indexerResponse) [0x000a1] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Indexers\RssParser.cs:105 
  at NzbDrone.Core.Indexers.RssParser.ParseResponse (NzbDrone.Core.Indexers.IndexerResponse indexerResponse) [0x00006] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Indexers\RssParser.cs:40 
  at NzbDrone.Core.Indexers.HttpIndexerBase`1[NzbDrone.Core.Indexers.BitMeTv.BitMeTvSettings].FetchPage (NzbDrone.Core.Indexers.IndexerRequest request, IParseIndexerResponse parser) [0x0005c] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Indexers\HttpIndexerBase.cs:204 
  at NzbDrone.Core.Indexers.HttpIndexerBase`1[NzbDrone.Core.Indexers.BitMeTv.BitMeTvSettings].TestConnection () [0x0000e] in m:\BuildAgent\work\6c3239faf2b92630\src\NzbDrone.Core\Indexers\HttpIndexerBase.cs:218

Any ideas? I have confirmed that my userid, RSS passkey and cookie have been entered correctly

Thanks

markus101 · June 27, 2015, 2:03pm

Either an extra redirect they have added or one from http to https.