Sonarr hammering

jayhawk · April 15, 2015, 1:47am

Hello creators…

I like your app, and I use it myself–but investigating what’s been causing some issues on my server–Ive been seeing a lot of hammering by your application. I’ll see API queries 3-4 per second. Is it possible to request that those be spaced out to one request every 2-3 seconds? IF not, sonarr users are going to see a lot of issues on ours servers…

here’s a small sample (obfuscated ip and apikey) --there’s a lot of this though, and it adds up…

8.8.300.300 - - [14/Apr/2015:21:40:40 -0400] "GET /getnzb/e55ef8f9c511acf836dacd1395f5ce8a.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:41 -0400] "GET /getnzb/73c276c8bf6d34d654d51a1bc8639e30.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:41 -0400] "GET /getnzb/0a5db4aa4e4ba7beaf788985f0e0ecbe.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:41 -0400] "GET /getnzb/6e120a4a2c6fc0b1a0cb418c95a0e8b2.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:42 -0400] "GET /getnzb/2f011f093616da38ff5342938de6c933.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:42 -0400] "GET /getnzb/9f57487bed933ce43f4ff1b493f67e99.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:42 -0400] "GET /getnzb/45e440ca64464ec1da249775a043a431.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:42 -0400] "GET /getnzb/394de27c8775a4d919d827757e0a4a25.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:43 -0400] "GET /getnzb/c847845f2f0e8a4a2f5dec12975d70db.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:43 -0400] "GET /getnzb/81ec168f611f927096c682f0b86553f3.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:43 -0400] "GET /getnzb/4ab5901b5797bcc43d470b05828fd4e4.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:43 -0400] "GET /getnzb/761796d057ad542e74536b81f1f85147.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:44 -0400] "GET /getnzb/a7338baf9c247179749e76cd65267db8.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:44 -0400] "GET /getnzb/0dd3e9a5ba559c0d601cb09b71612261.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:44 -0400] "GET /getnzb/6eca15a86df5ddd504d98344df708ecb.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:45 -0400] "GET /getnzb/902a3803906dd288878254480b3dfd01.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:45 -0400] "GET /getnzb/7107f2d4889752c55293c0d184844dab.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:45 -0400] "GET /getnzb/e866017fb4b8018308aaec886a25c345.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:45 -0400] "GET /getnzb/8e1197d39ebad623828612e319b3b4ef.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:46 -0400] "GET /getnzb/65132791ad7bd52bb13b2deb9b252399.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"
8.8.300.300 - - [14/Apr/2015:21:40:46 -0400] "GET /getnzb/19ba534f5a43f68826d239f9c811c193.nzb&i=501&r=APIKEY HTTP/1.1" 429 573 "-" "Sonarr/2.0.0.3004 (Windows 6.2)"

You can reply to me at steve@nzb.su - I can provide more logs or whatever you need to help diagnose whats happening…

Steve

markus101 · April 15, 2015, 6:55am

Hi Steve,

These requests here are all coming from the same user?
Is the 429 following the request the response code? Assuming it is, is that due to the number of requests or the API limit being hit?

The only way I can see requests for nzbs coming in that fast is when grabbing a full season of episodes, in which case, taking 40-60 seconds just to grab the nzbs after the results are processed is pretty lengthy.

We’re already looking at reducing the number of calls we make to indexers when they’re not available or the request limit has been reached, not exactly the same issue as here, but something we’ll try to look at.

//cc @Taloth

jayhawk · April 15, 2015, 12:37pm

Hi Marcus,

The 429 is either from hitting the api limits, or for hammering and is from the same user–I’ve recently adjusted the api to considering hammering anything that is 3 hits in 3 seconds.

If you look at what SABnzbd does–you’ll see a much longer pause between grabs:

255.255.255.255 - - [12/Apr/2015:20:34:30 -0400] "GET /getnzb/0429fe1985cb50f7db99de8d8e4a4111.nzb&i=userID&r=APIKEY HTTP/1.1" 200 43583 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:34:38 -0400] "GET /getnzb/a6a5a7aa91e77b7f95d5fe6512df7d63.nzb&i=userID&r=APIKEY HTTP/1.1" 200 37882 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:34:46 -0400] "GET /getnzb/7054ea46b9184df0243993d93fb17255.nzb&i=userID&r=APIKEY HTTP/1.1" 200 34743 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:34:53 -0400] "GET /getnzb/4c73ec375874769ec2b48d290824469d.nzb&i=userID&r=APIKEY HTTP/1.1" 200 38083 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:02 -0400] "GET /getnzb/642cb6c9c88da772dc08d968d9939c00.nzb&i=userID&r=APIKEY HTTP/1.1" 200 17763 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:09 -0400] "GET /getnzb/27ca3ca3581b490594f7a3e4d71e830d.nzb&i=userID&r=APIKEY HTTP/1.1" 200 16010 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:17 -0400] "GET /getnzb/c8baaf93dc4327b5f558683380bcf2a6.nzb&i=userID&r=APIKEY HTTP/1.1" 200 21715 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:25 -0400] "GET /getnzb/ff4e835fb051a1a2e7b1edc2e5e0c4c1.nzb&i=userID&r=APIKEY HTTP/1.1" 200 17962 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:32 -0400] "GET /getnzb/b13e2965e161418c92489f33a029f0c4.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18651 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:40 -0400] "GET /getnzb/bee446445772902383973fcab807d22c.nzb&i=userID&r=APIKEY HTTP/1.1" 200 19436 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:48 -0400] "GET /getnzb/c4c9be77242cf32002434b9b3ad74767.nzb&i=userID&r=APIKEY HTTP/1.1" 200 16236 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:35:56 -0400] "GET /getnzb/db685c738e4c4adce96a063b5a5aadc6.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18254 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:03 -0400] "GET /getnzb/e818a7defe90fc65c8df9fe43d2dc987.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18296 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:11 -0400] "GET /getnzb/6c680121779fdf46184e4688fde91cd4.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18980 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:19 -0400] "GET /getnzb/b1b49b77496fa07b1eedc6c4f3cdfb7d.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18923 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:27 -0400] "GET /getnzb/2e0a64df5babef1af0a86b3aa0cd2b64.nzb&i=userID&r=APIKEY HTTP/1.1" 200 16027 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:35 -0400] "GET /getnzb/0bfdc803998a1be2cb13ede31a6bd9d7.nzb&i=userID&r=APIKEY HTTP/1.1" 200 22438 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:43 -0400] "GET /getnzb/0c2212a42c91a9db0e0e4e2c2f1d070a.nzb&i=userID&r=APIKEY HTTP/1.1" 200 17010 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:50 -0400] "GET /getnzb/f1bcffe22d3de8cbed7ae2a3139c8dd2.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18781 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:36:58 -0400] "GET /getnzb/d04cce4f8a397c558c1fd0bff61236eb.nzb&i=userID&r=APIKEY HTTP/1.1" 200 18296 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:37:05 -0400] "GET /getnzb/6206df12c74436b3aebf66585353a267.nzb&i=userID&r=APIKEY HTTP/1.1" 200 17822 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:37:13 -0400] "GET /getnzb/d4ba6c7517aaf8772013680bad051ec6.nzb&i=userID&r=APIKEY HTTP/1.1" 200 16323 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:37:21 -0400] "GET /getnzb/eac9b38c8f39bdd951ba50954e3acadd.nzb&i=userID&r=APIKEY HTTP/1.1" 200 19430 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:37:28 -0400] "GET /getnzb/3bd73211955bae20f6a37099ec1e7cf2.nzb&i=userID&r=APIKEY HTTP/1.1" 200 17133 "-" "SABnzbd+/0.7.20"
255.255.255.255 - - [12/Apr/2015:20:37:36 -0400] "GET /getnzb/b1a7d8a346c0d6ebb1f904f10f12e885.nzb&i=userID&r=APIKEY HTTP/1.1" 200 20216 "-" "SABnzbd+/0.7.20"

If you get a 429 from the server, you should definitely be waiting a while though, I’d think…?

jayhawk · April 28, 2015, 1:12pm

I’ve instituted rate limiting on the server–hopefully you guys have found a way to space out your requests a bit?

thezoggy · April 28, 2015, 4:57pm

just fyi, sb obviously had to do something similar to fight people doing stupid things like massive backlogs… caching rss results and searching within these first… delays between grabs… etc. gives user system some time to recover from the data coming in (low performance) and also not trying to hammer sites.

2sec sleep off rss grab:

github.com

midgetspy/Sick-Beard/blob/development/sickbeard/search_queue.py#L140




    logger.log(u"Beginning search for new episodes on RSS")



    foundResults = search.searchForNeededEpisodes()



    if not len(foundResults):

        logger.log(u"No needed episodes found on the RSS feeds")

    else:

        for curResult in foundResults:

            search.snatchEpisode(curResult)

            time.sleep(2)



    generic_queue.QueueItem.finish(self)



def _changeMissingEpisodes(self):



    logger.log(u"Changing all old missing episodes (UNAIRED) to status WANTED")



    curDate = datetime.date.today().toordinal()



    myDB = db.DBConnection()

5sec delay after season search:

github.com

midgetspy/Sick-Beard/blob/development/sickbeard/search_queue.py#L216


def execute(self):



    generic_queue.QueueItem.execute(self)



    results = search.findSeason(self.show, self.segment)



    # download whatever we find

    for curResult in results:

        if curResult:

            search.snatchEpisode(curResult)

            time.sleep(5)



    logger.log(u"Finished searching for episodes from " + self.show.name + " season " + str(self.segment))

    self.finish()



def _need_any_episodes(self, statusResults, bestQualities):



    wantSeason = False



    # check through the list of statuses to see if we want any

    for curStatusResult in statusResults:

system · December 5, 2015, 10:02am