[UNRAR] [WIN]: **NEW** Guide to auto-process .rar torrents via Sonarr w/o Drone Factory [edit: 01 SEP 17]

Third Party Addons
#8

@Fragsrus

It’s a good question … and this is the hardest part of this configuring IF you are using a remote machine.

So, YES, my “seedbox” is on a remote machine within my LAN. However, it IS just a VM on the server that is running Sonarr, but for all intensive purposes, all applications see it as a separate machine, ie sonarr, radarr, - because it has a separate IP address.

so in your scenario - YES, I would recommend SyncThing. However, I’ve always used a local seedbox, I’ve never used a seedbox service or something outside my LAN. But I DO KNOW that this does work if you are using a something remote.

Also, I actually StableBit’s “Cloud Drive” to mount remote folders as “local drives”. I’m NOT using it for this scenario, but I know it would work. CloudDrive “spoofs” Windows (and all other appliaions) into seeing a remote folder as a local drive. It allows you to mount ANY remote folder, Google Drive, FTP, etc.

https://stablebit.com/CloudDrive

LMK what you figure out.

#9

So following the guide this is my setup:

Deluge download to \incomplete
Auto add watching to \incomplete\tv download to \incomplete\tv move to \tv\torrents
Label move to \tv\torrents

But all my downloaded files are in \incomplete so Label and autoadd will not move any files to tv\torrents :sweat:

#10

@ToRvaLDz - The “download to” or “download location” is not that important unless you are using NZBs and Torrents, which it looks like you are not. As longs as the torrent data is moved FROM one DIR to another DIR

Also, your WATCH folder is not as important as well…that only comes in use IF you manually download a torrent. So, if I go to a torrent website, download the actual .torrent file, and save it to my local computer, THEN, I move the .torrent file to the “WATCHED DIR”, then Deluge should automatically add the TV label. SINCE SOnarr automatically appends the TV label, it’s not as important.

So:

“But all my downloaded files are in \incomplete so Label and autoadd will not move any files to tv\torrents”

So, after it’s complete, the finished torrent data stays in \incomplete?!

When the torrent is downloading, is the label 'TV" added to it?

also, make sure that the plugin AUTOADD adds the same label (Case sensitive?) ‘TV’ to the torrent as Sonarr does.

So, there are three places you have to actually input the label “TV”

1 - In the AutoAdd plugin properties at the bottom
2 - in the Deluge main window under “labels” (right click and “add label”)
3 - in Sonarr -> settings -> Download client -> Deluge -> “category”

Finally, This MAY also be a permissions issue. Is your folder “\incomplete” and “\tv\torrents” on the same machine? are they on the same Hard drive?

#11

Yes my downloads remains in \incomplete, anyway Sonarr are processing them and files are copied to media folder, probably they are not finished because deluge is still seeding them.
They are on same drive.

Thank you

#12

I’m an analog guy living in a digital world, however, I use a program called “extractnow” that auto extracts all torrents and deletes what it extracted from and dumps to drone factory folder. I’m having to use the import feature.

I was hoping someone a lot smarter and technical than me could take a look at extractnow and see if it makes for easier workflow and create a guide based on that for the benefit of all.

I don’t remember where I downloaded extractnow but here’s a copy of it if you need it: https://dl.dropboxusercontent.com/u/1687202/Disks/extractnow.exe

If this isn’t helpful, I apologize. But from a windows perspective, extractnow has been entirely helpful to me.

#13

No, Deluge should move them even if they are still seeding.

I would check the deluge logs and find out why they aren’t being moved.

#14

@Sean_Reitmeyer

Yes, I was using extract now before, I wrote an in-depth guide here. :

However, the problem with extract now is that you still have to use the drone folder as ExtractNow it doesn’t have the ability to call Sonarr to ESPISODESCAN. So, once the drone folder is completely depreciated, extract now will no longer work.

#15

Here a simpler method for those who are on windows and have winrar already installed or wish to install it since its free.

Ok this should help for those who already have winrar installed on thier windows pc.

If not its a free download.

In utorrent go to options/preferences/advanced/run program and copy this into the run this program when a torrent is finished box.
“C:\Program Files\WinRAR\WinRAR.exe” x -ibck -inul “%D*.r00” “%D”

If your winrar is installed in a different directory mod the first part as needed but that the default install location for it.

This will run winrar and extract the file to the same folder as the download.

This should work in most torrent programs that support the run function which is most of them these days.

I couldn’t get this to work in deluge since most of its plugins are presently broken in windows along with the one used to run programs.

This should be a much simpler less of a headache way for the windows users tell sonarr adds the feature itself. :slight_smile:

#16

@Eric_M

So how does this method make the call to Sonarr for an EPISODESCAN ?

Also if you’re using your torrent client for more than just TV - how would you separate the completed torrents into diff directories based on their labels - IE, music, movies, etc?

#17

Sonnar can see and scans the episode without issue.

I use the client for more then just tv which doesn’t cause a issue.
Since basically the only things using rar is tv shows these days.
Sonnar is going to just ignore anything else anyway since it didn’t request or need them.

Different directories not needed as winrar unpacks the tv shows directly into the episode own directory which sonnar is happy with.

Set sonnar to monitor what ever download direct you client is using.
The client downloads the tv show into a folder of the same name in the download directory.
Once completed winrar upacks the show into that directory.
Sonnar sees all this and scans the shows over to the correct place.

Simple easy no messing with extra directories or the like.

Of course this is windows only.

PS: sorry for spelling and grammar autocarrot is killing me (gaaaaa)

#18

Example.

Sonarr grabbed torrent for episode from show last night uotrrent auto-added it.

Sonnar could see it was being download.

It finished and show was unpacked .
Sonarr was aware of this and scanned the show over.

Wife happy :wink:

#19

Hey @seanvree. I wanted to thank you for the work you have done so far on this - it definitely helped give me a better picture of how all the different pieces need to interplay to get this working. I will admit I was a bit frustrated by some lack of clarity in your guide however. I’m sure that it took you a few drinks to get this all working and I know putting a guide together like this is a chore mostly for the benefit of others. So again, thanks.

That being said I would like to offer some constructive criticism and suggestions on how to make this better and hopefully the next person won’t stumble through all the same blocks that I did.

I’ll try to go in order of the guide - though some things may be out of place.

  1. Most importantly…link this guide at the very top of your other guide so people don’t waste time with the other method.

  2. It isn’t really clear (in either guide) that all these steps you are taking are:
    a) because you have a separate seed box machine
    b) because you are trying to seed the files and want to avoid locks

Yes you do mention in passing that this guide solves all those issues, but not everyone who wants an “auto-process .rar” solution is going to have that configuration. Many pieces here are overly complex or simply don’t make much sense if you don’t fit the above criteria. It would be very helpful if you both explained clearly that is your setup at the onset AND commented on each piece as to what part of the puzzle it solves so that those who don’t have that particular challenge know that they can skip it/go an easier route.

  1. The directory names you have chosen are confusing. Calling the directory “drone” confuses the issue since we are specifically doing away with the drone folder. I know this is just semantics, but it does muddy the waters. Personally I call my folder “processing”, but anything to distinguish it as “not drone” would be better.

  2. You don’t stay consistent with your directory names. You show your tree structure up top, but then down in your autoproecssmedia.cfg and log files you have a different structure. This makes it very difficult for people to follow how you should set up the autoproecssmedia.cfg.

  3. On the same note, the section for configuration of the autoproecssmedia.cfg is difficult to follow. Part of it is the naming I mention in #4, part of it is the bad color coding the site here does to it. And part of it is the wall of text with no real indicator of what specific variables one needs to set.

  4. While we are on autoproecssmedia.cfg. I’m pretty sure the default_downloadDirectory is optional and probably will never come into play with how this is configured.

  5. In you section “6. Process explanation (How this shit works):” your two flow examples could use a little clarification. I believe that step #5 in the non-Rar flow “Sonarr doesn’t import because the remote path is EMPTY.” should also be step #5 in the rar flow. I know Sonarr wouldn’t import anyway because the files are archived, but that doesn’t mean that the workflow step isn’t there. You should try to mirror the text in both workflows to be exactly the same except where they actually differ.

Another example here is your step 10/9 in the two flows. In both cases sonarr “Sonarr imports, post-processes, and MOVES release to final destination.” Just because the file isn’t a rar doesn’t mean that the NTM script doesn’t act on it. For example it may download subtitles for a file that is already an .mkv. Likewise, in both cases sonarr does additional post processing in the form of moving and renaming the files.

I know this seems minor, but when you are looking at this through fresh eyes little omissions/differences like this are very confusing as to how they should be interpreted and what is responsible for what.

  1. The section on the code change to autoProcesstv.py. A change like this is a big deal (since as you state it may be lost on upgrade of source). It doesn’t hurt to point out here again exactly what this is being used for and why you probably don’t need it if on a single box config.

This is the one thing that I think I can offer some additional assistance on (rather than just nitpicking :wink:). You can actually get around this by writing your bat script as follows:

if /I %3==D:\queue\processing\tv\torrent\\ (
	"d:\Program Files (x86)\nzbToMedia\TorrentToMedia.py" %1 %2 %3
) ELSE (
	# do nothing
)

Ofc you would substitute your path for mine above (after the %3==). This way even though the deluge script will always execute, it will only actually run NTM for torrents in the appropriate directory. Less processing and cleaner logs!

Well, that’s about it for now…it is really late and I’ve been working on this whole configuration for too long today. I’ll probably read this over again in the next couple of days so would appreciate your feedback and if you care for any more of mine (if I have it).

thanks again.

#20

@bengalih ,

Hey, brother. Really appreciate the feedback, you’re right it did take a few drinks. But after using sonarr for a couple years now and ALWAYS having that single problem nagging me, when I figured it out…I had to share.

So, let me address each one:

  1. I actually tried, but it’s “Locked”, I can’t edit it. Maybe @Taloth can delete it? Not sure what the best way to address that is.

  2. Understood, I’ll consider editing, but to answer, it’s both. Depending on your enviroment…you MAY have either of those issues or none of those issues. However, the bottom line is that there is NO method to auto-extract RARs. I’ll compare it to CouchPotato - which DID have the auto-extract option, native. I guess I assumed you’re using this guide if BOTH of those things are true. I don’t see why you would even need this guide if you don’t have those issues? So, Yeah, good point - I may or may not edit it simply because I think there is something in the works that will auto-extract native.

  3. Yeah, I actually thought about this. I guess I couldn’t come up with a better name than “drone” HA.

  4. Got it, I’ll clean that up. I was trying to use “common” terms/names, while also copying and pasting from my environment. Good point.

  5. Humm… not sure I see what you mean of why it’s hard to follow…I guess I can change all the DIR names to be consistent. As far as formatting - yeah, that took me a good hour to figure out, I guess I could post it on gist.

  6. Actually, that is important. … I THINK. when you set that, AND you run the script manually - if it’s not set, it will try to post-process everything in all the DIRs you have enabled. So, as I understand it, that will PREVENT it from re-extracting the same files again. I may be wrong but this ONLY comes into play if you run the script manually…if you’re testing or something. But I also left some values in there so that the code I posted looked similar to what the .cfg file looks like on your machine.

  7. Yup, you’re right. Corrected.

  8. Yeah, you’re right. I guess I assumed that people are using this to COPY the release and not link it. I’ll add verbiage.

  9. HA, I was about to get pissed man! I was like “WHO IS THIS GUY”? Ha, jk, thanks for the input…yeah, I’ll add that in as well.
    Personally, I’m not going to use it as I forsee myself using this same method for Radarr (as it looks as if they are doing away with drone factory as well) That way all I don’t have to edit the bat script again, I just have to edit the .cfg file. But yeah, thanks!

Really appreciate your feedback. I’m interested to know IF you actually got this working for you. Sounds like you know the deal real well, so PLEASE reach out. Depending on the progress that the devs are making with including this native, I’ve been procrastinating editing this for Radarr as well, SO…if I get word that this is still useful for the time being, I’ll make some MAJOR edits once I figure out how to implement this with Radarr.

_SV

#21

So just to clarify some points/responses:

  1. The reason I am making the distinction is because as you state there is no built-in solution for archives. So everyone who uses torrents (or maybe even nzbs without a good post-processor) is going to need something to do the job. I came across your other thread first in searching for good solutions and then this one which led me to NTM - which I never had heard of before. I think it is a good solution because it isn’t another process running (like ExtractNow) and does all the processing and notifications back to sonarr. It also handles the failed downloads and monitoring/link of the downloads in sonarr, which not every other solution (like a simple drone folder) can do. So, in short the NTM solution is a good one but this guide is overly complicated if you just want to do that. For instance you don’t need the empty remote folder, you don’t need to make the changes to the NTM source code, etc if you don’t have all the issues around seeding and/or seedbox/remote share. If a built-in autoextract feature was coming out soon then you may be right and only people with your specific challenges will need this (although NTM still might be more flexible). However just in the meantime my point was that it took me a while to figure out exactly what/why you were doing certain things which made the guide overly complex for what I needed. Really, in the end the best thing I got out of it was learning about NTM.

  2. My main issue here (and maybe others…) is that I just wasn’t familiar with NTM at all. So it took me quite some time reading up on that to just understand how it is solving the problem that you are using it for. So with a big config file it is hard to pick out just those options that are important and need changing. For example you set “force_clean = 1”… but I don’t know why that would be because it wasn’t the default. Once you have more details on how NTM actually processes you can make this decision. In short your guide may be able to get followed in 15 minutes if you don’t care to know how it works AND you don’t run into a single issue. Otherwise, this was a much bigger time investment even for people familiar with these topics.

  3. We can research this more because somewhere in the NTM docs/wiki it says it is optional. What it supposedly does is if safe_mode=1 is not process anything in that directory. I don’t think it has anything to do with re-extraction, it simply prevents extraction/processing at all. And when we call the script normally it passes a specific torrent directory/file so it never even sees/knows about any other folder. If you run it manually without passing all the correct arguments then yes, it does seem to process everything in the folder you run it against. However you likely will be running it against your xxx\tv folder and never against your xxx folder root. That being said you setting it isn’t a bad thing per-se…just I think probably unnecessary which brings me back to all the points I mention in #5.

  4. Or just move it…which is what I’m doing.

  5. If you are just using your torrent client for radarr/sonarr then yes you would have no need. But, if you use your torrent client for anything else it will also trigger the execute script on any torrent you finish. You can change my example by using OR statements so that it covers both your tv and movies folders and anything else you torrent won’t execute the NTM script.
    One other correction on this topic. You state:

I think you meant Auto-Execute (or simply Execute) plugin here.

As far as me getting this working. I’m not a heavy user and I won’t know 100% until I actually get some files that are archived to know 100%. But I have done extensive testing including creating my own test torrents and loading them up into the appropriate labels manually and NTM processes them all and sonarr picks them up. The only thing not tested really is the monitoring from sonarr to deluge (i.e. activity), but I expect that to work as normal. I also have NTM setup to do my subtitle downloads. As I was setting NTM up I was planning on looking into how I could have it call another script to run sublminal when I realized it had it built-in!

Once again, my single box scenario isn’t as complex as yours and only time will tell if I experience file locks, etc. I like the idea of the NTM script and I am upgrading my SabNZB finally and am going to switch to that from the sickbeard script I think. I also plan to try and use NTM for processing downloads from other clients like maybe IRC and/or jdownloader2… so I like the idea of a single post-processing script that can work with anything as long as you can feed it the parameters it needs. I’m 90% sure at this point that the TorrentToMedia.py script doesn’t actually have any reliance on any torrent client…it will still do all the post-processing and simply ignore the pieces that have to do with torrents (pause/resume/etc).

I’ll be sure to chime back in with any issues or if I discover something worthwhile.

thanks

#22

So I’m sorry to say things aren’t working very well. I’m hoping you or @Taloth may be able to shed some light.

I was seeing success when I was running the TorrentToMedia.py script manually, like this:


TorrentToMedia.py 234004f6071ce63e5e912201d9a999885d59103a My.Show.S01E19.HDTV.x264 D:\queue\processing\tv\torrent

This cause the script to look for the directory D:\queue\processing\tv\torrent\My.Show.S01E19.HDTV.x264 and copy/extract it to the defined output directory which I have defined as D:\queue\pickup\tv.

This worked, and after extraction (or just copy in the case of no rar) it sent the API call to Sonarr. Sonarr then did 3 things:

  1. Imported from the D:\queue\pickup\tv
  2. Removed the files from D:\queue\pickup\tv (because #1 was successful)
  3. Removed the download from deluge, which also served to delete D:\queue\processing\tv\torrent\My.Show.S01E19.HDTV.x264

So the net effect was file got into my media folder through Sonarr and all other copies were deleted. Perfect!

However, when running an actual download through Sonarr, the problem arises where Sonarr will actually scan the D:\queue\processing\tv\torrent\My.Show.S01E19.HDTV.x264, seeing how that is the information it has been passed to it by deluge.

In the case of a rar file this shouldn’t cause a problem (I don’t think) because it won’t find anything to import there and so the call from NTM should trigger the import from the pickup directory.

However, for a file that isn’t archived what happens is that even thought the file gets copied from D:\queue\processing\tv\torrent\My.Show.S01E19.HDTV.x264 to the pickup folder, Sonarr uses the deluge path and grabs the copy from the former. This leaves the latter directory there with an extra copy of the data.

At this point I thought that I probably needed to implement the remote path hack that @taloth mentioned. But, it doesn’t appear to work when everything is on the same box. To be clear, I added “10.10.10.102” and “d:\empty” to both path locations. But, the logs show it always still going out to pickup from the processing directory.

I think I can probably fudge this by setting force_clean = 1 in the NTM script. But this doesn’t seem like a graceful solution, and I’m not sure how safe it is either.

Pretty discouraged that there is no standardized way to do this and the only real option is to use the drone folder which we are discouraged from doing.

:sleeping:need sleep

UPDATE:

Ok, so I got this working again though I don’t know how ideal it is.
It turns out I have some of the same issues as you do with your data on a separate seed box because my download location and final media location are actually on separate drives on the same box - which means hard-linking won’t work.

What this means is that the intended pickup directory (what you called “drone”) will still keep a copy of the files in them unless you either edit the NTM code to add the "importMode": "Move" option or use force_clean = 1. The “Move” is probably safer, because what is unclear is this:

When NTM sends the process command to sonarr, how long does it wait? Does it actually wait for a successful response that Sonarr has copied/moved/linked/whatever the file? Or does NTM just wait a certain time after the process command is send. If the latter case, and NTM just continues on processing after so many seconds, then force_clean = 1 could be very dangerous as it might wipe out a file (or at least try and then fail…depending on locks) before Sonarr fully gets it.

So I’m a little suspicious of just using force_clean = 1 by itself. However I must use it because as I am on a single box, this hack with the remote path mapping doesn’t seem to work at all. This means that Sonarr will always scan the final torrent location (after move) and get the copy from there. Since it will have already processed the movie (or at least begun to) by the time NTM sends its notification, Sonarr will not then try to process it from the NTM pickup location. This means that copy of the data will simply remain unless you use force clean.

I’m considering changing my label to move the finished torrent directory to the same drive the final media (seasons) are on. I have a concern though that Sonarr may start scanning that location before the file finishes copying. My guess is that Sonarr doesn’t know that a “move completed to” is taking place when it checks to see if the torrent is complete…so it might start processing before a move to another drive has a chance to happen. I hope @taloth or maybe @markus101 can chime in on that front. I’m going to open another thread maybe on this question because I see another weird issue that might relate to a race condition here.

In the end, I think this will work…but it is kludgey and I’m not crazy about it. I’ll have to let it run for a few weeks and see how clean it is.

#23

I’m back from vacation, so a bit rusty.

You should at least use ImportMode=Move, yes. force_clean is on optional addition. NzbtoMedia polls the Sonarr Command api to find out if Sonarr finished processing but it probably won’t do that forever.
Remote Path mapping should work, even locally, but I’ll need more details about the path and settings you used to be able to tell where it’s wrong.

The torrent state is indeed a bit tricky when it comes to ‘move completed to’, torrent clients have different behavior and I don’t know which does what and whether they still have the ‘downloading’ state while moving.

#24

hey ive made another topic but i’ll try to write here, I have another windows setup that extracts files to an import folder but i dont know enough about all this coding to make a script that tells sonarr to import, can i just use the import bit from this and make it work with another setup i would like to just call it from a bat file

hope you can help

#25

HUGE UPDATE!

THis can now be done VERY easily with a BRAND NEW deluge plugin. See here:

( I will keep this topic open, but I will no longer be actively developing it)

Thanks all to helped especially @Taloth !!

#26

you can use this.

#27

Found an awesome script on reddit a few weeks back that seems to handle this along with radarr support. works for Qbittorrent and Utorrent