Drone Factory is Deprecated [unrar]

Will Manually Triggering Imports also be removed in v3 because it refers to the drone factory folder?

A few of us that use a smaller seedbox service are investigating the best method of using Sonarr once v3 comes along and we have confirmed that this method works. Importantly it is an Event Driven method (executed by the seedbox’s utility on completion of file download) so shouldn’t lead to the same issues with Drone Factory.

Thanks for the awesome app BTW :smile:

@Lokanna

What surprises me is that we’ve repeated on several occasions that nothing has changed on the 4th, we simply added a warning to get new users to stop using the Drone Factory. Yet people react like the world is ending and we’re just a bunch of assholes. I expected some fallout from users that misinterpreted the announcements, but not that much.

The decision to phase out the Drone Factory periodic scan was made years ago and with good reasons. It was always the plan to only do that on the next backward-incompat major release, v3. We’re not there yet, not by a long shot, it’s not gonna be an automatic update either. This has been mentioned several times. Yet so many users go into full on panic mode or lash out.
The reason why we made the deprecation more obvious is because we’re run into dozen and dozens of users that complained about import issues caused by incorrect usage of the Drone Factory, such as partial import, truncated files, etc. Often solved by simply disabling the Drone Factory coz CDH was already functional in their situation. So our nr one goal here is to make sure new users don’t use the Drone Factory.
It’s entirely possible we come up with a solution to integrating unrar prior to or at the v3 release, but it isn’t that easy to cover all the edge-cases. I rather not open the can of worms of explaining those technical complications.
It’s however entirely possible to do unrar and then call the Sonarr API, if you can’t write that script, somebody else can… and they have. But it depends on your particular setup and download client.

[quote=“Lokanna, post:39, topic:15063, full:true”]
I read comments above by sonarr badged posters that seem somewhat defensive, perhaps even at times, condescending, towards the people who are asking for help. I apologize in advance if those two words put anyone on the defensive, as they’re not meant as an affront. However, I believe you folks should understand that your responses can sometimes come to be taken as such, whether intended to or not.[/quote]

I admit I may sound somewhat defensive or even aggressive at times, it isn’t my intention to offend people, but in my defense: You try to remain neutral if that many users misread stuff, jump to conclusions and then elect to take a dump on you on several different fora instead of asking constructive questions. Certain things also doesn’t translate well over the internet, especially since English isn’t the native language for many.
Several others have been cordial and asked how they particular workflow should work, usually with a positive result, which is exactly why we put up the warning in the first place. But most of those are in different topics, so obviously less visible.

Your post didn’t mention how exactly you use the Drone Factory and which download client, so I can’t advice your on how to proceed, but as I said, there isn’t any hurry.

I thank you for your constructive criticism, and I hope my response addresses some of the concerns you have.

2 Likes

@spurf No it won’t be removed entirely. Only the Periodic Scan in Sonarr (because it’s not event-driven and prone to partial/truncated files being imported).
However, in v3 that command should have a “path” parameter. See https://github.com/Sonarr/Sonarr/wiki/Command#downloadedepisodesscan
One main difference is that the standard Drone Factory scan is performed on a single ‘root’ directory, where everything inside is treated as somewhat of an unrelated mix of directories, files, subdirectories etc.
But the ‘path’ parameter must point to the directory for individual download ‘jobs’, which is assumed to be related to a single job. So could be a season pack, or single episode together with subs. And particularly the downloadClientId parameter, which allows Sonarr to pull info from the original grab event.

However, if you have a remote seedbox, isn’t mounting the drive remotely an option, for example with sshfs? What i’d like to see is a community wiki article about the various seedbox providers on how to set them up properly.

@Taloth

Thank you for the response. Having read this entire thread (and others) I realize you folks have good reasons to nix drone factory. The one new bit of information in the reply after my post that sparked curiosity is the bit about torrent clients not supporting unrar or post processing like Usenet clients.

Have you considered approaching some of the more popular torrent client devs to see if that is their feature map? Maybe you could pick 1 or 2 to partner with and recommend as it would integrate better with Sonarr, due to having post processing, etc. I know the two I see mentioned quite often are qbittorrent and deluge. Just a thought.

As for my set up, I found a new program last night that will act like drone factory and monitor a folder to extract files. I had one or two shows download and it appeared to work so I’m hopeful this work around will suffice until something proper is implemented. I’m hesitant to mention the program because my level of expertise and knowledge is low and I want to verify results before I give anyone else a false sense of hope.

Again, thank you for your time with both replying and developing Sonarr. It truly is a great tool that I rely on.

Some clients do have plugins for it, like Deluge ‘Extractor’ plugin. But it’s an afterthought for most clients.
That isn’t surprising. rar files are far from ideal for long term seeding, since you end up with double the disk space if you want to keep seeding. There are plenty of trackers that ban rar for that very reason.
Official scene rar files aren not actually compressed, they’re only used to ‘split’ the data up in parts. Quite literally the original video is split up in parts, then a header and footer is tacked on and voila… rar files. Which makes it even more ridiculous from a torrent pov.
It would be so much easier if Sonarr simply rejected any torrent with rar inside, but I might not survive the fallout… :wink:

The reverse is possible too: A virtual wrapper so that Sonarr can see what’s inside the rars, parse titles and do mediainfo on them without actually unraring. There are even fuse filesystem wrappers that do that on the fly.
Such a virtualization layer in Sonarr is one of the options we’ve been considering for dealing with rar, but it’s not the only option. Another is to have a pre-processing stage in Sonarr instead, but it’s fairly complicated because it requires the import logic to be ‘stateful’. But personally I think it’s the best approach, since it will allow better integration and handling of lftp/btsync/rsync etc.

I’ve mentioned this above, but I now have my setup using the DownloadEpisodesScan API, so am not using Complete Download Handling or the Drone Factory, so can we have a way of turning off the health check warning “Enable Completed Download Handling or configure Drone factory” please.

For new users I think its a good waning, but for existing, there should be a way of switching it off, as with myself I’m not using any deprecated sonarr features now but am getting this waning.

Unfortunately remote drive mounting isn’t available with this particular seedbox as it is a bit unconventional but suits the needs from a price / utility standpoint.

The seedbox does have a nice utility that can be run on the local machine which transfers torrent pieces as they arrive on the server. When all the pieces have been received the utility assembles them into the final files / directories and places them in a folder (which Drone Factory has been monitoring).

The utility has this option which I’m guessing should be able to provide the path parameter:

file_finished_command: specify a command/script to execute whenever a file finishes. 3 parameters will be passed: the file path, the info hash for the torrent, and the label. it is possible the label has been subjected to cleanup if you enabled cleanup_filenames and include_label_in_file_names.

What I am not sure is how to make a script to achieve this as I have zero experience :confused:

Would you be able to give me some pointers, preferably for powershell as I’m running windows.

Thanks!

OK, Reading all this and all related things made me few things to conclude.

  1. Drone Factory is very useful, quick to setup, slice food, blend, sort, cook, serve on plate, … but occasionally cut few fingers and mix them in smoothy so you can’t be sure until you try the end product.
  2. It will go
  3. I really need it due the facts/settings/stated below in appendix
  4. Solution needs to be made.

so regarding above statements:

  1. Currently I can’t live without it. I’m missing only few fingers (read episodes over the time I use Sonarr) as I carefully used/set my clients and scripts to minimize probability of that happening. eg. I have Download client setup but do not use it as I use DirectDownload client which is neither NZB not Torrent.
  2. Huh, well even hard at times it was loved and I will come to funeral
  3. Ahh, read appendix
  4. Ok. Solution found but needs to be polished.

Appendix:
I have custom made client that handles both DL + Unpack + Move to Join folder (eg. drone folder) and say I’m done. Just wrote Jscript to tell me that, so I have an DL client Event that contain path+filename where the file is saved, but do not know which show it belongs simply can’t know that and this even I can use to trigger downloadedepisodesscan. Hooray, 50% done.

I need help for next 50 percent which involves making Jscript that will work both on Win and linux.
I was thinking about something like this:

// Sending and receiving data in JSON format using POST mothod
//
var xhr = new XMLHttpRequest();
var url = "url";
xhr.open("POST", url, true);
xhr.setRequestHeader("Content-type", "application/json");
xhr.setRequestHeader("X-Api-Key","MyKey");
xhr.onreadystatechange = function () {
    if (xhr.readyState === 4 && xhr.status === 200) {
        var json = JSON.parse(xhr.responseText);
    }
};
var data = JSON.stringify({"name": "downloadedepisodesscan","path": "/serverA/var/temp/Download.S01E01","importMode":"Move","downloadClientId":"{infohash}"});
xhr.send(data);

So anyone help Please?

Windows 10 Pro/ Sonarr/ altbinz/ drone factory/ 4TB NAS. My set up has been working great for a long time. I hope that I can figure out how to replace drone factory. Subscribed.

@dhcrees Why not simply keep Completed Download Handling enabled, assuming you provide the downloadClientID in the API command, it should be properly handled. And that way you can see which downloads have not yet been imported.

@spurf Something like this https://gist.github.com/Taloth/e02e76bcb29942f791cc2e8595192846
No warranties, and you’ll need to tweak and test it.

@Peke I’d say it’s a good start, but I’m no JScript expert. You probably don’t need to set downloadClientID since you’re using a DirectDownload client.

@wtfchuck What’s special about your setup that it requires a Drone Factory? If you’re insistent on using alt.binz, consider using the Usenet Blackhole client in Sonarr to interface with it. But I’d recommend using sabnzbd or nzbget for better integration and failed download handling.

1 Like

@ everybody else, If you have a specific usecase or problems with switching away from using the Drone Factory, or would like advice on how to do so.
Please open a new topic, preferably with the format “Import without Drone Factory [usecase]” where ‘usecase’ is something unique with your setup, ‘seedbox’, ‘btsync’, ‘unrar’ etc.
Or post in a topic that already mentions your exact usecase.
It’ll help to keep threads specific to a particular usecase, or I’ll lose track of it all and end up making replies like you see above here. :smiley:

1 Like

@seanvree To get back to the topic OP posted, could you take a look at nzbToMedia? I thought it was usenet only but clinton hall appears to have quite a bit of torrent support, transcoding and unrar, and has built in support for the DownloadedEpisodesScan command. It might match your particular usecase for unrar.
At least, if you still want to spend time on it, coz I can imagine you’ve had enough…

Thanks for the help. Drone factory is gone and completed DL handling is working. Black Hole.

@taloth ,

Yeah, looks good. I was using it the past, but I can’t remember why I bailed on it.

Got it working mostly…the only thing I’m not seeing is how to envoke the DownloadedEpisodesScan which would tell Sonarr to look in the directory for the extracted file.

Fantastic work @Taloth will test ASAP

All:

I ALMOST have this working perfectly on windows… No scripts, all automatic and NO Blackhole Directory!!!

.Doing some testing. I’ll write a tutorial and post it here when I’m done.

Yo dudes,

We ALMOST have this working. If you want to join the discussion, see below:

All:

After many hours of jameson and formatting, I’ve completed a new guid which is MUCH MORE lightweight and streamlined to get this problem done, it’s tested, and it works. HERE:

Hi @Taloth. I can confirm your powershell script works exactly as expected :smile:

I added an extra bit of code after your filter “#TODO: Filter on Label?” to end the script if the torrent doesn’t match the ‘tv’ label that I use for Sonarr

if ($Label -NotMatch ‘tv’) {
Write-Host "Not for Sonarr"
break
}

I couldn’t get the utility I use to download from my seedbox provider to execute the powershell script directly but found I could get it to run a batch file instead with this command inside:

start /min “” powershell.exe -ExecutionPolicy Bypass -noexit -file “D:\Down\Sonarr.ps1” D:\Down%1 %2 %3

I needed the ‘D:\Down%1’ as the utility doesn’t pass the full windows directory for the downloaded file location.

Thanks again!

Tnx for the update, I appreciate that.

Please note that -NotMatch is a Regex match and, since it lacks anchors, would match ‘fdfdtvdjf’ as well. I suspect you intended to use -ne (not equal).

Anyway, nice work.