How to automatically process .rar releases in Sonarr and Radarr while using usenet and torrents on a Windows Enviroment.
For Deluge you can use the following plugin:
This is mostly geared toward windows, because let’s face it – all these python-based applications don’t work well on windows, and the devs really don’t give two shits about windows (which is cool, I love open source too).
This works for both Radarr and Sonarr on Windows. However, this should work on any system depending on your environment.
Noob level: 4 (1 being you think a CD rom drive is a cup holder, 10 being you are Steve Wozniak.) If you have installed Sonarr and Radarr, use NZBs and Torrents, and realize that there is a fucking annoying problem with Sonarr and Radarr NOT auto-extracting files, then you are probably good to go.
This guide will do the following:
1- Handle and process all .rar releases automatically and unmonitored with no intervention required by the user.
2- Ensure all .rar files are auto-extract and moved to final media destination
3- Ensure no data is left behind that doesn’t need to be and no data duplication.
4- Allow for torrents to still be seeded
5- Ensures that Sabnzbd+Deluge (and all other tools do not conflict)
6- Enables Sonarr and Radarr to Handle completed and failed downloads - HUGE
NOTE: I’m all ears if you have found a better way. However, I’ve been doing this for 2 years, and I’m a windows wizard. This is the best method. I’m a windows guy, I hate fucking CLI, and I like pretty, colorful GUIs, but most of all I like my shit to be automatic, so please don’t suggest other applications that can’t be run as a SERVICE on Windows. The goal of this project is that this all happens automatically, unmonitored and in a “server” environment (Windows).
My rig:
Windows 10E (Host/server)
VMbox (Seedbox): Windows 10P as a client to the host (Deluge is the ONLY tool the runs on my seedbox. Everything else is on the host) However, if you have some crazy remote seedbox crap - Let me tell you that “MKLNK /J” is your friend.
“Searchers:
Sonarr: https://github.com/Sonarr/Sonarr
Radarr: https://github.com/Radarr/Radarr
(I also run Jackett (Torrents) and NZBHydra (Usenet) as my indexers although they really don’t play a role here, but if you have a lot of top-tier indexers like myself, these tools are gold)
Downloaders:
Deluge (Torrents)
Sabanzbd (Usenet/NZBs)
Software required:
-
Some type of torrent-client “plug-in” that monitors folders and moves the torrents around. I know this is baked into most clients, but for deluge, this is called Auto-add (NOT THE DEFAULT PLUGIN): https://github.com/gazpachoking/deluge-autoadd-plugin/
-
NSSM (Freeware) https://nssm.cc/24
- Enables you to run applications as if they were a “service” aka unmonitored. This tool is VERY handy when running python based applications on Windows) The ONLY downfall that I’ve found, is that the program that is “converted” to a service will not show up as it’s proper .exe in the processes list. You will only see NSSM. However, it will use whatever name you assign it in the services console list. So, you can easily remotely stop/restart it.
- SyncBackUpSE (The FREE version will NOT work since it does NOT keep a database of what has already been copied): https://www.2brightsparks.com/syncback/sbse.html
- Copies files from one location to another
- ExtractNow (Freeware): http://www.extractnow.com/28
-Extractor for .rar files
Basic environment/file structure: (PAY CLOSE ATTENTION TO THE FILE HIERARCHY, you can name your folders whatever, but if they are not in the correct structure, none of this will work and you’ll kill a gorilla trying to figure it out)
- Download Directory (Where you download client (Deluge and SABNZBd) download to.
X:\downloads\Incomplete
…\TV
…\movies
- Intermediate Directory: (This is where the download client moves the files after downloaded for post-processing (and seeding for torrents) by Radarr and Sonarr:
X:\downloads
…\TV (Sonarr)
…\Torrents
…\NZBs
…\Movies (Radarr)
…\Torrents
…\NZBs
Example: X:\downloads\TV\Torrents
- “Drone Factory”: (Yes, you MUST use the drone folder on Radarr and Sonarr for this to work – Keep reading, trust me.)
X:\downloads
…\DroneTV (Sonarr)
…\droneMovies (Radarr)
Example: X:\downloads\DroneTV
- Final Media Directory (where PLEX/Kodi look for your files):
x:\Media
…\TV
…\Movies
INSTRUCTIONS:
Note1: (I’m going to move in the order that a torrent/NZB would move thru this project for better understand. It can get confusing where the files are moving to/from. I literally had to get a fucking white-board out and map all this shit out…maybe I was drunk? Not sure.
Note2: I’m also going to reference Sonarr mostly, but it’s the same process for Radarr except the paths mentioned above (Obviously)
-
Install all required software mentioned above (Don’t configure anything, just install it)
-
Configure Sonarr. Sonarr -> settings -> Download client
a. ENABLE completed download handling:
b. ENABLE Remove
c. Redownload Failed download: I recommend YES
d. Remove Failed downloads: I recommend YES
e. Drone factory options: Drone folder from above: X:\downloads\droneTV
f. Drone factory interval: 15 min (Suggested) -
Configure SABNZBD: Settings -> Categories
a. Create a category called TV
b. Priority: whatever
c. Processing +delete
d. Folder path (where SAB DLs to: x:\downloads\TV\NZBs
e. Indexer categories: Whatever, but TV is a good choice -
Configure Deluge (Or your preferred torrent client: (Screenshot: http://imgur.com/0MqJh9E)
a. Preferences -> Downloads
i. Download to: X:\downloads\Incomplete
ii. DO NOT TOUCH ANYTHING ELSE. (The autoadd plugin will handle the rest)
b. Preferences -> Plug-ins:
i. Must have LABEL enabled (Default plug-in)
ii. Create Labels: Movies, TV, etc.
c. Preferences –> AutoAdd (must enable plugin)
i. Add a “Watch Folder” path for each label you created from above
ii. Path Options: -
Watch folder x:\plexdownloads\Incomplete\TV
-
Set download location: x:\plexdownloads\Incomplete\TV
-
Move completed: x:\plexdownloads\TV\torrents (this will be your seed DIR)
-
Label: TV
-
Open ExtractNow from START MENU, Configure ExtractNow: (Screenshot: http://imgur.com/jNDaQaN)
a. General:
i. Enable: Automatically extract archives
ii. Enable: Allow multiple instances
b. Destination:
i. Automatically extract archives to (Sonarr drone path: Example: X:\downloads\DroneTV
c. Process:
i. DISABLE: Extract archives with-in archives
ii. ENALBE: move to recycle bin
iii. ENALBE: DELETE
iv. DISABLE: Delete DIR contain archives IMPORTANT
d. Monitor:
i. Path (Torrent intermediate directory): X:\downloads\TV\Torrents
ii. ENABLE: Auto Extract
e. Close extract now.
f. Test with simple .rar file:
i. create a txt file,
ii. Rar it.
iii. Put rar in dir,
iv. Drop dir into your “monitor” folder. example: X:\downloads\TV\Torrents\testdir\test.rar
d. Within 3 minutes the contents of that .rar archive should appear in your Sonarr DRONE folder. With the test dir and .rar file deleted.
g. Close ExtractNow
h. move extractnow.ini from %LOCALAPPDATA%\ExtractNow\ To: C:\Program Files (x86)\ExtractNow -
Configure EXTRACT NOW to run as a service via NSSM
a. From Admin CMD Prompt:
i. “NSSM Install EXTRACT NOW”
i. Path: C:\Program Files (x86)\ExtractNow\extractnow.exe
ii. Startup DIR: C:\Program Files (x86)\ExtractNow
iii. Example: http://imgur.com/Wfc0sB0
j. EXTRACTNOW is now a windows service and can be stopped at any time from the Windows services console.
k. Check if service installed correctly: Start -> RUN -> services.msc
l. Start ExtractNow service, then Stop service
m. Now run ExtractNow from start menu. Check to make sure all your settings are still there. Then exit ExtractNow. (this is necessary as there is NO GUI to manage the extract now service, you just trust it’s running and your settings are correct
n. Go back to services. Start ExtractNow service, ensure service is set to Automatic.
o. If you ever need to change settings to extract now, you must STOP the service FIRST, then open the GUI from the start menu, then close the GUI, then RESTART the service. -
Configure SyncBackupSE: (Screenshot: http://imgur.com/Fb2piLv)
NOTE: You have two options to run SyncBackupSE: As a scheduled TASK or Service. I chose to run it as a scheduled task simply because this program has gnarly options, and if you run a service, you would have to ensure ALL your params are correct when installing it via NSSM. It would NOT be easy. There really is no downfall of doing it via task other than you can’t stop the scheduled task w/o logging onto the box, however, it will run automatically…because…. it’s a scheduled fucking task. So, I’ll use the Scheduled task method. -
Create a profile Call it “TV or Sonarr” whatever
a. Type of backup: Synchronize (Smart Sync)
b. Left: Name it torrents: x:\ downloads\tv\Torrents
c. Right: Name it DroneTV: x:\downloads\Drone
d. Click OPTIONS on top right, click EXPERT
e. Profile options: CHANGE FILTER
i. Files/folders to copy: IMPORTANT -
First line: " *\ " "
-
Second line: " *.r "
-
(without quots) (What this does is copy ONLY DIRs and their contents that have any files that have “R” in the extension)
-
For fun: Exclude (on right): .mp4, MKV, etc.
-
Click OK, this will bring up a summary of what will happen when you run the profile, click OTHER OPTIONS:
a. Select Clean up automatically.
b. (What this does is when the torrent client removes torrents after they are being seeded, SyncBackUpSE will remove those files/folders from the database. -
APPLY and close out of filter options
f. Profile options -> WHEN
i. This is where you configure the schedule. This will auto generate a windows task with whatever options you set here. Personally, I set a basic schedule, then used the actual windows Task Scheduler to tune the options
ii. There are 1245 ways you can this. However, this part is key, so make sure you got your shit right, and test it.
iii. Mine: -
Schedule Options:
a. General: Run whether user is logged on or not (Duh)
b. ENABLE Run with highest privileges
c. Trigger:
i. Daily
ii. Recur every 1 day
iii. Repeat task every 15 minutes
iv. Stop task if runs longer than 30 minutes (This should be the MAX of how long would take for your rig to copy a file from the \torrent dir to your \drone dir)
d. Conditions: SyncBackupSE should have created that for you
e. Settings: If the task fails, restart every: 5 min.
f. Stop the task if it runs longer than: 1 hour (This means if there is a file lock, it will stop the task and try again)
g. If the task is already running: Queue a new instance.
iv. Back to profile options on left: -
Decisions – Files IMPORTANT: Ensure all options are selected as: “Do nothing, skip the file and ignore changes EXCEPT: “A new file has been created on <\movies\torrents> or is only on <\tv\torrents>: COPY file to \DRONE\
-
Destinations – FOLDERS: Ensure everything is “DO NOTHING”.
This is how my final configuration looks for SyncBackUpSE (Please omit the root drive path as I’m using symlinking to my external drive pool)
• Files will be synchronised (Smart Sync) between Left (C:\Media\PlexDownloads\tv\Torrents) and Right (C:\Media\PlexDownloads\Drone).
• The following files/folders will be ignored: .mkv,.mp4,$RECYCLE.BIN,*.SBSECOR,"\AppData\Local\Temporary Internet Files",\AppData\Local\Temp,"\Application Data\Mozilla\Firefox\Profiles*\parent.lock",\desktop.ini,\DfsrPrivate,"\Local Settings\Temporary Internet Files","\Local Settings\Temp","\Microsoft\Windows\Temporary Internet Files",\RECYCLER,\SBSE____.,"\System Volume Information",\System32\Msdtc\MSDTC.LOG,\thumbs.db,\Windows\csc,\Windows\debug\NtFrs*,\Windows\ntfrs\jet,\Windows\Prefetch,\Windows\Registration*.crmlog,\Windows\sysvol\domain\DO_NOT_REMOVE_NtFrs_PreInstall_Directory,\Windows\sysvol\domain\NtFrs_PreExisting___See_EventLog,\Windows\sysvol\staging\domain\NTFRS_,\Windows\Temp,\hiberfil.sys,\pagefile.sys,\PGPWDE01
• Only the following files/folders will be included: *,*.r*
The profile is stored in C:\Users\Administrator\AppData\Local\2BrightSparks\SyncBack\
TEST IT!!! TEST IT!!!
Create a text file on your desktop. Put the text file in a rar file. Put the .rar file in a folder. Put the folder in your intermediate torrent folder: Example: x:\downloads\tv\torrents.
If you did everything that .txt file (and ONLY the text file will then appear in your drone folder.
In summary:
Process (A, B, C):
Sonarr searches for Debbie Does Dallas (1999)
A. NZB Flow:
1.Sonarr downloads NZB release appends Category TV
2. Sabnzbd receives download, appends category TV
3. SabNZBD downloads articles to x:\downloads\incomplete\TV
(BTW, Sonarr is tracking the download)
4. Sabnzbd finished download, moves extracted release to x:\downloads\tv\nzbs
5. Sonarr finds file, imports it, renames it, moves it to final destination
6. Sonarr removes file from x:\downloads\tv\nzbs, and client history.
4. Complete.
B: Torrent Flow: non .RARed release (Debbie.Does.dallas.1999.1080p.mkv)
1.Sonarr downloads .torrent file, sends to deluge with label TV
2. Deluge receives .torrent file, appends label TV
3. Deluge downloads torrent data to x:\downloads\incomplete\TV
4. Sonarr tracks torrent progress
5. Deluge finishes download, moves release to x:\downloads\tv\torrents
SyncBackupSE sees new file (Debbie.Does.dallas.1999.1080p.mkv) (sub-dir or not), but ignores it because the extension of the file doesn’t have .r in extension.
6. Sonarr detects torrent is finished, imports it, renames it, moves it to the final destination.
Deluge continues to seed torrent.
7. Complete.
C: Torrent Flow: RAR-ed release:
(Debbie.Does.dallas.1999.1080p)
\Debbie.Does.dallas.1999.1080p.r00
\Debbie.Does.dallas.1999.1080p.r01
\Etc, etc, etc
\Debbie.Does.dallas.1999.1080p.rar\ Debbie.Does.dallas.1999.1080p.mkv
1.Sonarr downloads a mother fucking *rar release sends .torrent to deluge with label TV
2. Deluge receives .torrent file, appends label TV
3. Deluge downloads torrent data to x:\downloads\incomplete\TV
4. Sonarr tracks torrent progress
5. Deluge finishes download, moves folder with .rar files to data to x:\downloads\tv\torrents\Debbie.Does.dallas.1999.1080p
6. Sonarr now gives you shit and says “no files eligible for import” (you ignore it)
7. SyncBackupSE sees new Directory (\Debbie.Does.dallas.1999.1080p), and its contents (\Debbie.Does.dallas.1999.1080p.r00, r01, etc) and flags it for COPY because it has .R in extension.
8. SyncBackupSE then COPIES new DIR (and all contents that have .r in extension) to x:\downloads\droneTV\Debbie.Does.dallas.1999.1080p
9. ExtractNow sees new DIR and .RAR files
10. ExtractNow extracts base .RAR file (Debbie.Does.dallas.1999.1080p.mkv) to root \DroneTV folder
11. ExtractNow then deletes Sub DIR and all containing files (.rar files) (Drone folder now looks like x:\plexdownloads\DroneTV\Debbie.Does.dallas.1999.1080p.mkv)
12. Sonarr now executes a Done Folder Scan
13. Sonarr sees the Debbie.Does.dallas.1999.1080p.MKV file and IMPORTS it to the final destination.
14. Sonarr Deletes .mkv file from \DroneTV DIR
15. Sonarr then marks the release as downloaded and removes it from history “Client download history”
16. Drone Factory folder is clean.
17. Original release is still being seeded.
FUCK YEAH.
Cheers