Guide: Automatically process .rar releases in Sonarr/Radarr while using usenet and torrents on a Windows Environment


#1

How to automatically process .rar releases in Sonarr and Radarr while using usenet and torrents on a Windows Enviroment.

For Deluge you can use the following plugin:

This is mostly geared toward windows, because let’s face it – all these python-based applications don’t work well on windows, and the devs really don’t give two shits about windows (which is cool, I love open source too).
This works for both Radarr and Sonarr on Windows. However, this should work on any system depending on your environment.

Noob level: 4 (1 being you think a CD rom drive is a cup holder, 10 being you are Steve Wozniak.) If you have installed Sonarr and Radarr, use NZBs and Torrents, and realize that there is a fucking annoying problem with Sonarr and Radarr NOT auto-extracting files, then you are probably good to go.
This guide will do the following:

1- Handle and process all .rar releases automatically and unmonitored with no intervention required by the user.
2- Ensure all .rar files are auto-extract and moved to final media destination
3- Ensure no data is left behind that doesn’t need to be and no data duplication.
4- Allow for torrents to still be seeded
5- Ensures that Sabnzbd+Deluge (and all other tools do not conflict)
6- Enables Sonarr and Radarr to Handle completed and failed downloads - HUGE

NOTE: I’m all ears if you have found a better way. However, I’ve been doing this for 2 years, and I’m a windows wizard. This is the best method. I’m a windows guy, I hate fucking CLI, and I like pretty, colorful GUIs, but most of all I like my shit to be automatic, so please don’t suggest other applications that can’t be run as a SERVICE on Windows. The goal of this project is that this all happens automatically, unmonitored and in a “server” environment (Windows).

My rig:
Windows 10E (Host/server)
VMbox (Seedbox): Windows 10P as a client to the host (Deluge is the ONLY tool the runs on my seedbox. Everything else is on the host) However, if you have some crazy remote seedbox crap - Let me tell you that “MKLNK /J” is your friend.

“Searchers:
Sonarr: https://github.com/Sonarr/Sonarr
Radarr: https://github.com/Radarr/Radarr
(I also run Jackett (Torrents) and NZBHydra (Usenet) as my indexers although they really don’t play a role here, but if you have a lot of top-tier indexers like myself, these tools are gold)

Downloaders:
Deluge (Torrents)
Sabanzbd (Usenet/NZBs)

Software required:

  1. Some type of torrent-client “plug-in” that monitors folders and moves the torrents around. I know this is baked into most clients, but for deluge, this is called Auto-add (NOT THE DEFAULT PLUGIN): https://github.com/gazpachoking/deluge-autoadd-plugin/

  2. NSSM (Freeware) https://nssm.cc/24

  • Enables you to run applications as if they were a “service” aka unmonitored. This tool is VERY handy when running python based applications on Windows) The ONLY downfall that I’ve found, is that the program that is “converted” to a service will not show up as it’s proper .exe in the processes list. You will only see NSSM. However, it will use whatever name you assign it in the services console list. So, you can easily remotely stop/restart it.
  1. SyncBackUpSE (The FREE version will NOT work since it does NOT keep a database of what has already been copied): https://www.2brightsparks.com/syncback/sbse.html
  • Copies files from one location to another
  1. ExtractNow (Freeware): http://www.extractnow.com/28
    -Extractor for .rar files

Basic environment/file structure: (PAY CLOSE ATTENTION TO THE FILE HIERARCHY, you can name your folders whatever, but if they are not in the correct structure, none of this will work and you’ll kill a gorilla trying to figure it out)

  1. Download Directory (Where you download client (Deluge and SABNZBd) download to.

X:\downloads\Incomplete
…\TV
…\movies

  1. Intermediate Directory: (This is where the download client moves the files after downloaded for post-processing (and seeding for torrents) by Radarr and Sonarr:

X:\downloads
…\TV (Sonarr)
…\Torrents
…\NZBs
…\Movies (Radarr)
…\Torrents
…\NZBs

Example: X:\downloads\TV\Torrents

  1. “Drone Factory”: (Yes, you MUST use the drone folder on Radarr and Sonarr for this to work – Keep reading, trust me.)

X:\downloads
…\DroneTV (Sonarr)
…\droneMovies (Radarr)

Example: X:\downloads\DroneTV

  1. Final Media Directory (where PLEX/Kodi look for your files):

x:\Media
…\TV
…\Movies


INSTRUCTIONS:

Note1: (I’m going to move in the order that a torrent/NZB would move thru this project for better understand. It can get confusing where the files are moving to/from. I literally had to get a fucking white-board out and map all this shit out…maybe I was drunk? Not sure.
Note2: I’m also going to reference Sonarr mostly, but it’s the same process for Radarr except the paths mentioned above (Obviously)

  1. Install all required software mentioned above (Don’t configure anything, just install it)

  2. Configure Sonarr. Sonarr -> settings -> Download client
    a. ENABLE completed download handling:
    b. ENABLE Remove
    c. Redownload Failed download: I recommend YES
    d. Remove Failed downloads: I recommend YES
    e. Drone factory options: Drone folder from above: X:\downloads\droneTV
    f. Drone factory interval: 15 min (Suggested)

  3. Configure SABNZBD: Settings -> Categories
    a. Create a category called TV
    b. Priority: whatever
    c. Processing +delete
    d. Folder path (where SAB DLs to: x:\downloads\TV\NZBs
    e. Indexer categories: Whatever, but TV is a good choice

  4. Configure Deluge (Or your preferred torrent client: (Screenshot: http://imgur.com/0MqJh9E)
    a. Preferences -> Downloads
    i. Download to: X:\downloads\Incomplete
    ii. DO NOT TOUCH ANYTHING ELSE. (The autoadd plugin will handle the rest)
    b. Preferences -> Plug-ins:
    i. Must have LABEL enabled (Default plug-in)
    ii. Create Labels: Movies, TV, etc.
    c. Preferences –> AutoAdd (must enable plugin)
    i. Add a “Watch Folder” path for each label you created from above
    ii. Path Options:

  5. Watch folder x:\plexdownloads\Incomplete\TV

  6. Set download location: x:\plexdownloads\Incomplete\TV

  7. Move completed: x:\plexdownloads\TV\torrents (this will be your seed DIR)

  8. Label: TV

  9. Open ExtractNow from START MENU, Configure ExtractNow: (Screenshot: http://imgur.com/jNDaQaN)
    a. General:
    i. Enable: Automatically extract archives
    ii. Enable: Allow multiple instances
    b. Destination:
    i. Automatically extract archives to (Sonarr drone path: Example: X:\downloads\DroneTV
    c. Process:
    i. DISABLE: Extract archives with-in archives
    ii. ENALBE: move to recycle bin
    iii. ENALBE: DELETE
    iv. DISABLE: Delete DIR contain archives IMPORTANT
    d. Monitor:
    i. Path (Torrent intermediate directory): X:\downloads\TV\Torrents
    ii. ENABLE: Auto Extract
    e. Close extract now.
    f. Test with simple .rar file:
    i. create a txt file,
    ii. Rar it.
    iii. Put rar in dir,
    iv. Drop dir into your “monitor” folder. example: X:\downloads\TV\Torrents\testdir\test.rar
    d. Within 3 minutes the contents of that .rar archive should appear in your Sonarr DRONE folder. With the test dir and .rar file deleted.
    g. Close ExtractNow
    h. move extractnow.ini from %LOCALAPPDATA%\ExtractNow\ To: C:\Program Files (x86)\ExtractNow

  10. Configure EXTRACT NOW to run as a service via NSSM
    a. From Admin CMD Prompt:
    i. “NSSM Install EXTRACT NOW”
    i. Path: C:\Program Files (x86)\ExtractNow\extractnow.exe
    ii. Startup DIR: C:\Program Files (x86)\ExtractNow
    iii. Example: http://imgur.com/Wfc0sB0
    j. EXTRACTNOW is now a windows service and can be stopped at any time from the Windows services console.
    k. Check if service installed correctly: Start -> RUN -> services.msc
    l. Start ExtractNow service, then Stop service
    m. Now run ExtractNow from start menu. Check to make sure all your settings are still there. Then exit ExtractNow. (this is necessary as there is NO GUI to manage the extract now service, you just trust it’s running and your settings are correct
    n. Go back to services. Start ExtractNow service, ensure service is set to Automatic.
    o. If you ever need to change settings to extract now, you must STOP the service FIRST, then open the GUI from the start menu, then close the GUI, then RESTART the service.

  11. Configure SyncBackupSE: (Screenshot: http://imgur.com/Fb2piLv)
    NOTE: You have two options to run SyncBackupSE: As a scheduled TASK or Service. I chose to run it as a scheduled task simply because this program has gnarly options, and if you run a service, you would have to ensure ALL your params are correct when installing it via NSSM. It would NOT be easy. There really is no downfall of doing it via task other than you can’t stop the scheduled task w/o logging onto the box, however, it will run automatically…because…. it’s a scheduled fucking task. So, I’ll use the Scheduled task method.

  12. Create a profile Call it “TV or Sonarr” whatever
    a. Type of backup: Synchronize (Smart Sync)
    b. Left: Name it torrents: x:\ downloads\tv\Torrents
    c. Right: Name it DroneTV: x:\downloads\Drone
    d. Click OPTIONS on top right, click EXPERT
    e. Profile options: CHANGE FILTER
    i. Files/folders to copy: IMPORTANT

  13. First line: " *\ " "

  14. Second line: " *.r "

  15. (without quots) (What this does is copy ONLY DIRs and their contents that have any files that have “R” in the extension)

  16. For fun: Exclude (on right): .mp4, MKV, etc.

  17. Click OK, this will bring up a summary of what will happen when you run the profile, click OTHER OPTIONS:
    a. Select Clean up automatically.
    b. (What this does is when the torrent client removes torrents after they are being seeded, SyncBackUpSE will remove those files/folders from the database.

  18. APPLY and close out of filter options
    f. Profile options -> WHEN
    i. This is where you configure the schedule. This will auto generate a windows task with whatever options you set here. Personally, I set a basic schedule, then used the actual windows Task Scheduler to tune the options
    ii. There are 1245 ways you can this. However, this part is key, so make sure you got your shit right, and test it.
    iii. Mine:

  19. Schedule Options:
    a. General: Run whether user is logged on or not (Duh)
    b. ENABLE Run with highest privileges
    c. Trigger:
    i. Daily
    ii. Recur every 1 day
    iii. Repeat task every 15 minutes
    iv. Stop task if runs longer than 30 minutes (This should be the MAX of how long would take for your rig to copy a file from the \torrent dir to your \drone dir)
    d. Conditions: SyncBackupSE should have created that for you
    e. Settings: If the task fails, restart every: 5 min.
    f. Stop the task if it runs longer than: 1 hour (This means if there is a file lock, it will stop the task and try again)
    g. If the task is already running: Queue a new instance.
    iv. Back to profile options on left:

  20. Decisions – Files IMPORTANT: Ensure all options are selected as: “Do nothing, skip the file and ignore changes EXCEPT: “A new file has been created on <\movies\torrents> or is only on <\tv\torrents>: COPY file to \DRONE\

  21. Destinations – FOLDERS: Ensure everything is “DO NOTHING”.
    This is how my final configuration looks for SyncBackUpSE (Please omit the root drive path as I’m using symlinking to my external drive pool)
    • Files will be synchronised (Smart Sync) between Left (C:\Media\PlexDownloads\tv\Torrents) and Right (C:\Media\PlexDownloads\Drone).
    • The following files/folders will be ignored: .mkv,.mp4,$RECYCLE.BIN,*.SBSECOR,"\AppData\Local\Temporary Internet Files",\AppData\Local\Temp,"\Application Data\Mozilla\Firefox\Profiles*\parent.lock",\desktop.ini,\DfsrPrivate,"\Local Settings\Temporary Internet Files","\Local Settings\Temp","\Microsoft\Windows\Temporary Internet Files",\RECYCLER,\SBSE____.,"\System Volume Information",\System32\Msdtc\MSDTC.LOG,\thumbs.db,\Windows\csc,\Windows\debug\NtFrs*,\Windows\ntfrs\jet,\Windows\Prefetch,\Windows\Registration*.crmlog,\Windows\sysvol\domain\DO_NOT_REMOVE_NtFrs_PreInstall_Directory,\Windows\sysvol\domain\NtFrs_PreExisting___See_EventLog,\Windows\sysvol\staging\domain\NTFRS_,\Windows\Temp,\hiberfil.sys,\pagefile.sys,\PGPWDE01
    • Only the following files/folders will be included: *,*.r*
    The profile is stored in C:\Users\Administrator\AppData\Local\2BrightSparks\SyncBack\

TEST IT!!! TEST IT!!!
Create a text file on your desktop. Put the text file in a rar file. Put the .rar file in a folder. Put the folder in your intermediate torrent folder: Example: x:\downloads\tv\torrents.
If you did everything that .txt file (and ONLY the text file will then appear in your drone folder.

In summary:
Process (A, B, C):
Sonarr searches for Debbie Does Dallas (1999)

A. NZB Flow:
1.Sonarr downloads NZB release appends Category TV
2. Sabnzbd receives download, appends category TV
3. SabNZBD downloads articles to x:\downloads\incomplete\TV
(BTW, Sonarr is tracking the download)
4. Sabnzbd finished download, moves extracted release to x:\downloads\tv\nzbs
5. Sonarr finds file, imports it, renames it, moves it to final destination
6. Sonarr removes file from x:\downloads\tv\nzbs, and client history.
4. Complete.

B: Torrent Flow: non .RARed release (Debbie.Does.dallas.1999.1080p.mkv)
1.Sonarr downloads .torrent file, sends to deluge with label TV
2. Deluge receives .torrent file, appends label TV
3. Deluge downloads torrent data to x:\downloads\incomplete\TV
4. Sonarr tracks torrent progress
5. Deluge finishes download, moves release to x:\downloads\tv\torrents
SyncBackupSE sees new file (Debbie.Does.dallas.1999.1080p.mkv) (sub-dir or not), but ignores it because the extension of the file doesn’t have .r in extension.
6. Sonarr detects torrent is finished, imports it, renames it, moves it to the final destination.
Deluge continues to seed torrent.
7. Complete.

C: Torrent Flow: RAR-ed release:
(Debbie.Does.dallas.1999.1080p)
\Debbie.Does.dallas.1999.1080p.r00
\Debbie.Does.dallas.1999.1080p.r01
\Etc, etc, etc
\Debbie.Does.dallas.1999.1080p.rar\ Debbie.Does.dallas.1999.1080p.mkv
1.Sonarr downloads a mother fucking *rar release sends .torrent to deluge with label TV
2. Deluge receives .torrent file, appends label TV
3. Deluge downloads torrent data to x:\downloads\incomplete\TV
4. Sonarr tracks torrent progress
5. Deluge finishes download, moves folder with .rar files to data to x:\downloads\tv\torrents\Debbie.Does.dallas.1999.1080p
6. Sonarr now gives you shit and says “no files eligible for import” (you ignore it)
7. SyncBackupSE sees new Directory (\Debbie.Does.dallas.1999.1080p), and its contents (\Debbie.Does.dallas.1999.1080p.r00, r01, etc) and flags it for COPY because it has .R in extension.
8. SyncBackupSE then COPIES new DIR (and all contents that have .r in extension) to x:\downloads\droneTV\Debbie.Does.dallas.1999.1080p
9. ExtractNow sees new DIR and .RAR files
10. ExtractNow extracts base .RAR file (Debbie.Does.dallas.1999.1080p.mkv) to root \DroneTV folder
11. ExtractNow then deletes Sub DIR and all containing files (.rar files) (Drone folder now looks like x:\plexdownloads\DroneTV\Debbie.Does.dallas.1999.1080p.mkv)
12. Sonarr now executes a Done Folder Scan
13. Sonarr sees the Debbie.Does.dallas.1999.1080p.MKV file and IMPORTS it to the final destination.
14. Sonarr Deletes .mkv file from \DroneTV DIR
15. Sonarr then marks the release as downloaded and removes it from history “Client download history”
16. Drone Factory folder is clean.
17. Original release is still being seeded.

FUCK YEAH.

Cheers


Importing extracted files without Drone Factory [unrar]
[UNRAR] [WIN]: **NEW** Guide to auto-process .rar torrents via Sonarr w/o Drone Factory [edit: 01 SEP 17]
[Auto-extract episodes] How to auto-extract TV shows when using windows with no scripts [Service]!
[UNRAR] [WIN]: **NEW** Guide to auto-process .rar torrents via Sonarr w/o Drone Factory [edit: 01 SEP 17]
Drone Factory is Deprecated [unrar]
[UNRAR] [WIN]: **NEW** Guide to auto-process .rar torrents via Sonarr w/o Drone Factory [edit: 01 SEP 17]
Importing extracted files without Drone Factory [unrar]
Drone Factory is Deprecated [unrar]
#2

Have you ever got the “run a program” function i ExtractNow to work? It gives me an error every time.

I am trying to make a similar setup as you have, but I want to run a batchfile when extract now is done so Sonarr/Radarr wont have to monitor a folder every minute.


#3

@ProphetSe7en

I have not. I’m not a huge fan of Batch/scripts. Too hard to control and monitor

That’s why I chose this way. The entire process is managed by actual services…rather than something happening in the background that you can’t monitor.

Just my 2 cents.


#4

The problem is that I am running rtorrent on a seedbox and the seedbox is a mapped nettwork drive on my server.

If rutorrent on my seedbox extract the files Sonarr/Radarr starts to prosess the mkv file as soon as it sees it in the folder. If only 20% of the file is extracted at that point I only get 20% of the movies on my plex server.

So I tried to set it up the way you have done, but Sonarr/Radarr never removes the error message of failed import, even after the file is imported with Drone Factory. It still keeps looking for the file were rutorrent reports it to be stored.

So I had to find a workaround. I dropped SyncBackup and extract now and replaced them with Dropit as a free alternative. Now I can sett different profiles for .rar, cleanup, move, extract and so on. Works really well :slight_smile:


#5

Thanks a lot for this guide first of all. I’ve gone through and set everything up all the rar tests were fine. Then I went to actually test it out, I just can’t get this auto-add plugin to work. I’m using deluge 1.3.15 with the 1.5 auto-add. Watch folder and download location are both x:\downloads\incomplete\tv but it still just downloads to x:\downloads\incomplete from the “Downloads” tab settings. The label matches properly as well. There is no move complete or any other options in the “Downloads” tab checked except “Use Full Allocation”.

Spent so many hours trying to get this rar crap automated. Really hope I can get your solution working I haven’t seen any better. Been working on this all night n it’s 4 am so time to call it quits for the night. Probably try it with qbit (I love deluge though) when I have time unless you have any advice for me, spent last hour trying to solve this problem after going through everything else :frowning:


#6

@demoncow

First:

If the auto add plugin isn’t working, I would suggest a few things:
1 - Make sure deluge has the proper permissions for that folder
2 - ensure you’re using the right python version of the plug-in…2.6 or 2.7?
3 - Post on the Deluge Forums. I can Assure you it DOES work, and very well.

Now for the kicker -

Unfortunately, for some reason, the devs are deciding to do away with the “drone factory” which completely invalidates this process…I have to look at how to reincorporate this with the new software…PLEASE make a complaint here:


#7

UPDATE: THIS WILL SOON NOT WORK WITH SONARR (Still does work great with Radarr)

The devs from Sonarr have decided to do away with the Drone factory for some unknown reason. Please make a complaint here:


#8

Yea I don’t know I tried newly created folders and still didn’t work. It’s using 2.7 like it should be. Couldn’t figure out how to do it on Qbit. I gotta go to work, probably for 12 hours. I might just give up on this as I’m getting extremely frustrated trying to get this working n just unrar myself like I have been. Sucks to do all that work to have one tiny problem though.


#9

Well it looks like inside core.conf this keeps switching to “autoadd_enable”: false

So far haven’t been able to make it stay true. I think this is my problem still trying to figure out how to stop it flipping to false.

EDIT: Can’t figure it out flips to false as soon as I open deluge. Could you post your deluge autoadd.conf and core.conf so I can compare and see if something is wrong.

EDIT2: Wait a second… do I have to use the blackhole downloadclient option with sonarr / radarr if so what do I set the watch folder and torrent folder to and anything else I need to know with using blackhole. I also posted in that thread as well. Might say something on radarr somewhere too. Wish I could just flick an option on in sonarr/radarr to do this all lol.

EDIT3: Yea it was the blackhole, you should add that part to your guide for newbs like me. I took out the backsync turned off the move to recycle and delete in extract now. TV show is seeding in x:\downloads\tv, torrent client will remove that when done seeding(autoremoveplus deluge plugin). Video file sent to droneTV folder. Video file grabbed by sonarr. All unwanted data gone!!! Working with sonarr and radarr.

I guess I’m not seeing the point of backsync. I guess extract now was deleting .rar n .r0s before backsync moved them, backsync did nothing and because extractnow was deleting the .rar n .r0s preventing me from seeding. Which is why I turned off the delete stuff in extractnow. Then I was like wait whats the point of backsync I can just leave this stuff here have my torrent client delete data with torrents when I’m done seeding.

Thank you so much for this guide man, without it I don’t think I would have got this going at all, even though I didn’t end up using backsync. Hopefully one day we can just hit an unrar enabled button in sonnar lol. :smile: :wink:


#10

Wait, “blackhole”? What do you mean?

The point of syncbackup is that sometimes the torrent client will lock the file, so that extract now can’t extract it.

You def CAN use this method without syncback, but I included it because I like to have a database to see what was copied… Depending on your windows environment and your torrent client, it is possible that extract now will not be able to extract files that are in use by your torrent client.

In my case it’s not.

Cheers.


#11

By blackhole I mean the option inside sonnar/radarr in download client settings it has to be Torrent Blackhole not Deluge (or w/e client) for autoadd to move stuff. EDIT: Download Client > add Torrent Blackhole > Name It > Enabled > Torrent folder = X:\Downloads\Incomplete\TV\ > Watch Folder = X:\Downloads\TV\

I’ve never had problems extracting stuff while it’s seeding either so guess I’ll just keep the backsync out. Hell I can even modify the files seeding and it’ll just error out in Deluge.

Thanks again <3


#12

All:

I ALMOST have this working perfectly on windows… No scripts, all automatic, no Drone factory, and NO Blackhole Directory!!!

Doing some testing. I’ll write a tutorial and post it here when I’m done.


#13

Interested to see what you come up with. I’m running Windows 10, Deluge/Sonarr/Radarr… hoping to not have to use ExtractNow.


[UNRAR] [WIN]: **NEW** Guide to auto-process .rar torrents via Sonarr w/o Drone Factory [edit: 01 SEP 17]
#14

all and @fromthewoods

worked with devs to put together a much easier and light-weight solution to this without using the drone factory.

See here:


#15

THis can now be done VERY easily with a BRAND NEW deluge plugin. See here:

( I will keep this topic open, but I will no longer be actively developing it)

Thanks all to helped especially @Taloth !!


#16

I don’t want to rain on anyone’s parade and I’ll admit I only skimmed the guide and topic but I feel you shouldn’t need to do almost any of this. I’ve been using Sonarr and Radarr (and Couchpotato and others) for a long time now without having ever set up anything to deal with rars or the drone factory folder.

The reason is that your downloader should be doing this work for you. I use nzbget and it’s always handled the extraction for me. Like someone else mentioned above there are ways to have torrent downloaders do this as well. I’ve use SAB before as well but nzbget is a major step forward.

Sonarr and Radarr support the nzbget API very well and have complete control over the downloads. They tell nzbget what to get and they even display the download status in your Activity view as well as the movie and TV episode lists. Nzbget tells Sonarr and Radarr right away when downloads finish or fail so that they can be dealt with appropriately. It’s simple and versatile. No reason to use the drone factory folder.

Maybe I missed something I’m my skimming, though, so I apologize of that’s the case.

EDIT: I see that you’ve actually made a new guide and that this issue really only seems to apply to Torrent clients which don’t support unRARing or PP scripts. Usenet FTW I guess. Hopefully the dev or someone else adds this functionality to a Torrent client. It doesn’t seem like it belongs in Sonarr or Radarr, though.


#17

I use Utorrent and have found this new script to work nicely. https://www.reddit.com/r/sonarr/comments/747dia/windows_powershell_script_for_qbittorrent_rar/


#18

@rewen As you can see from the title of this thread, this is only for torrent clients - specifically Deluge. However, this entire thread is obsolete, as a solution has finally been developed.


#19

Hey @markus101 I see this thread still gets A LOT of hits. I can’t edit the original post, but maybe you’d be willing to edit the OP or maybe “Pin” this tool linked below? It has direct support for Sonarr and Radarr and works like a charm. Trying to get the word out there.

Hope you’re well Sir. Thanks for all you do.


#20

Edited to OP to point out the tool, might be best open a new thread recommending it for deluge users.