Also @Taloth -
Do you have an ETA when you guys are going to discontinue the drone factory?
That should largely be a non-issue, you shouldnāt mix the output folder for you nzb and torrent client. In that case it doesnāt use the mapping nor nzbToMedia.
So no ETA, but as mentioned on the wiki, it wonāt automatically disappear since v3 includes a collection of backwards incompatible changes and will be a manual update.
YAS! It worked! (logs below)
Tested with a .rar release and non-.rar release.
non-Rar release flow:
Sonarr sends release to deluge -> Deluge downloads, begins seeding in finished DIR-> Sonarr doesnāt import because remote path is EMPTY. -> NTM gets finished flag from DELUGE, COPIES to ādrone/tempā dir -> NTM sends episodescan command to sonarr -> Sonarr finds media in ātemp/drone folderā -> Sonarr MOVES release to final destination . Temp DIR is clean, original .rar torrent is still being seeding in itās original finished dir.
Rar-Release:
Sonarr send release (rar) to deluge -> Deluge downloads, sends .RARs to āfinished/seeding dirā begins seeding -> NTM extracts release, sends extract media file it to ātemp/drone folderā -> NTM sends episodescan command to sonarr -> Sonarr finds extracted media file in ātemp/drone folderā -> Sonarr MOVES release to final destination . Temp DIR is clean, original .rar torrent is still being seeding in itās original finished dir.
So to recap:
1. Sonarr settings -> Download Client -> disable drone factory
2. Sonarr settings -> Download Client -> Enable Remote & local paths, however, ensure these directories exist on disk, but are empty
3. Config NzbToMedia autoprocessmedia.cfg to the following:
[NzbDrone]
[[tv]]
enabled = 1
apikey = KEY
host = sonarrhost
port = sonarrport
username = ""
password = ""
###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
web_root = /sonarr
ssl = 0
delete_failed = 0
# Enable/Disable linking for Torrents
Torrent_NoLink = 0
keep_archive = 1
extract = 1
nzbExtractionBy = Downloader
wait_for = 2
# Set this to minimum required size to consider a media file valid (in MB)
minSize = 0
# Enable/Disable deleting ignored files (samples and invalid media files)
delete_ignored = 1
##### Enable if NzbDrone is on a remote server for this category
remote_
[Torrent]
clientAgent = deluge
useLink = hard
outputDirectory = C:\Media\PlexDownloads\drone (This is the old "Drone factory folder / temp folder)
default_downloadDirectory = C:\Media\PlexDownloads\tv\torrents (Finished DIR / Seeding DIR)
categories = tv
noFlatten = ""
DelugeHost = IP
DelugePort = PORT
DelugeUSR = admin
DelugePWD = PW
###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
deleteOriginal = 0
chmodDirectory = 0
resume = 1
resumeOnFailure = 1
[Extensions]
compressedExtensions = .zip, .rar, .7z, .gz, .bz, .tar, .arj, .1, .01, .001
mediaExtensions = .mkv, .avi, .divx, .xvid, .mov, .wmv, .mp4, .mpg, .mpeg, .vob, .iso, .m4v, .ts
audioExtensions = ""
metaExtensions = ""
4. Change the \core\NzbToMedia autoProcesstv.py lines 251 - 256 to this:
if remote_path:
logger.debug("remote_path: {0}".format(remoteDir(dirName)), section)
data = {"name": "DownloadedEpisodesScan", "path": remoteDir(dirName), "downloadClientId": download_id, "importMode": "Move"}
else:
logger.debug("path: {0}".format(dirName), section)
data = {"name": "DownloadedEpisodesScan", "path": dirName, "downloadClientId": download_id, "importMode": "Move"}
Iāll write this up in another post to make it all pretty, but does this sound right @taloth ?
Reference logs:
** RARed release / no modification to .py**
Result: Release extracted, post-processing complete, orphaned in drone folder
2017-07-19 05:06:40 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status = ?, client_agent = ?, last_update = ?, input_id = ?, input_hash = ?, input_name = ? WHERE input_directory = ? with args [0, u'deluge', 736529, u'edde03257c2712865a888e18349313dc083327a9', u'edde03257c2712865a888e18349313dc083327a9', u'Tosh.0.S09E16.1080p.WEB.x264-TBS', u'C:\\Media\\Plexdownloads\\tv\\Torrents']
2017-07-19 05:06:40 INFO ::MAIN: SEARCH: Found torrent directory Tosh.0.S09E16.1080p.WEB.x264-TBS in input directory directory C:\Media\Plexdownloads\tv\Torrents
2017-07-19 05:06:40 INFO ::MAIN: SEARCH: Setting inputDirectory to C:\Media\Plexdownloads\tv\Torrents\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:40 INFO ::MAIN: Auto-detected SECTION:NzbDrone
2017-07-19 05:06:45 INFO ::MAIN: Output directory set to: C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:46 INFO ::MAIN: Extracting C:\Media\Plexdownloads\tv\Torrents\Tosh.0.S09E16.1080p.WEB.x264-TBS\tosh.0.s09e16.1080p.web.x264-tbs.rar to C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:48 INFO ::MAIN: EXTRACTOR: Extraction was successful for C:\Media\Plexdownloads\tv\Torrents\Tosh.0.S09E16.1080p.WEB.x264-TBS\tosh.0.s09e16.1080p.web.x264-tbs.rar to C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:51 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:51 INFO ::MAIN: Found 1 media files in C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:51 INFO ::MAIN: Calling NzbDrone:tv to post-process:Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:51 INFO ::MAIN: Attempting to verify tv fork
2017-07-19 05:06:51 INFO ::MAIN: NzbDrone:tv fork set to default
2017-07-19 05:06:51 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS
2017-07-19 05:06:51 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request
2017-07-19 05:07:11 DB ::DB: Updating status of our download Tosh.0.S09E16.1080p.WEB.x264-TBS in the DB to 1
2017-07-19 05:07:11 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status=?, last_update=? WHERE input_name=? with args [1, 736529, u'Tosh.0.S09E16.1080p.WEB.x264-TBS']
2017-07-19 05:07:16 INFO ::CLEANDIRS: Directory C:\Media\PlexDownloads\drone\tv\Tosh.0.S09E16.1080p.WEB.x264-TBS still contains 1 unprocessed file(s), skipping ...
2017-07-19 05:07:16 INFO ::MAIN: The C:\nzbToMedia\TorrentToMedia.py script completed successfully.
----------
*** non-RARed release // Altered autoProcesstv.py file:**
Result: Release is copied to drone folder and orphaned.
2017-07-19 05:37:49 INFO ::MAIN: #########################################################
2017-07-19 05:37:49 INFO ::MAIN: ## ..::[TorrentToMedia.py]::.. ##
2017-07-19 05:37:49 INFO ::MAIN: #########################################################
2017-07-19 05:37:49 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status = ?, client_agent = ?, last_update = ?, input_id = ?, input_hash = ?, input_name = ? WHERE input_directory = ? with args [0, u'deluge', 736529, u'd4d6a85fa26ac6e6ab22c2b638ad6bda4fde80e4', u'd4d6a85fa26ac6e6ab22c2b638ad6bda4fde80e4', u'Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv', u'C:\\Media\\Plexdownloads\\tv\\Torrents']
2017-07-19 05:37:50 INFO ::MAIN: SEARCH: Found torrent file Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv in input directory directory C:\Media\Plexdownloads\tv\Torrents
2017-07-19 05:37:50 INFO ::MAIN: SEARCH: Setting inputDirectory to C:\Media\Plexdownloads\tv\Torrents\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:37:50 INFO ::MAIN: Auto-detected SECTION:NzbDrone
2017-07-19 05:37:55 INFO ::MAIN: Output directory set to: C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:37:55 INFO ::COPYLINK: MEDIAFILE: [Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv]
2017-07-19 05:37:55 INFO ::COPYLINK: SOURCE FOLDER: [C:\Media\Plexdownloads\tv\Torrents]
2017-07-19 05:37:55 INFO ::COPYLINK: TARGET FOLDER: [C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv]
2017-07-19 05:37:55 INFO ::COPYLINK: Hard linking SOURCE MEDIAFILE -> TARGET FOLDER
2017-07-19 05:37:55 WARNING ::COPYLINK: Error: Incorrect function.
, copying instead ...
2017-07-19 05:37:55 INFO ::COPYLINK: Copying SOURCE MEDIAFILE -> TARGET FOLDER
2017-07-19 05:38:23 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:38:23 INFO ::MAIN: Found 1 media files in C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:38:23 INFO ::MAIN: Calling NzbDrone:tv to post-process:Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:38:23 INFO ::MAIN: Attempting to verify tv fork
2017-07-19 05:38:23 INFO ::MAIN: NzbDrone:tv fork set to default
2017-07-19 05:38:23 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv
2017-07-19 05:38:23 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request
2017-07-19 05:38:43 DB ::DB: Updating status of our download Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv in the DB to 1
2017-07-19 05:38:43 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status=?, last_update=? WHERE input_name=? with args [1, 736529, u'Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv']
2017-07-19 05:38:48 INFO ::CLEANDIRS: Directory C:\Media\PlexDownloads\drone\tv\Frontline.S2017E14.Life.on.Parole.1080p.iT.WEB-DL.AAC2.0.H.264-QUEENS.mkv has been processed, removing ...
2017-07-19 05:38:48 INFO ::MAIN: The C:\nzbToMedia\TorrentToMedia.py script completed successfully.
----------
*** non Rared file, modified .py, remote path removed**
result: copied to drone folder, PP completed by Sonarr, DIR clean. BOOM!
2017-07-19 06:49:16 INFO ::MAIN: #########################################################
2017-07-19 06:49:16 INFO ::MAIN: ## ..::[TorrentToMedia.py]::.. ##
2017-07-19 06:49:16 INFO ::MAIN: #########################################################
2017-07-19 06:49:16 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status = ?, client_agent = ?, last_update = ?, input_id = ?, input_hash = ?, input_name = ? WHERE input_directory = ? with args [0, u'deluge', 736529, u'3756fdc9bab0c77fe3a79918f7b70293c1cc29f0', u'3756fdc9bab0c77fe3a79918f7b70293c1cc29f0', u'The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv', u'C:\\Media\\Plexdownloads\\tv\\Torrents']
2017-07-19 06:49:16 INFO ::MAIN: SEARCH: Found torrent file The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv in input directory directory C:\Media\Plexdownloads\tv\Torrents
2017-07-19 06:49:16 INFO ::MAIN: SEARCH: Setting inputDirectory to C:\Media\Plexdownloads\tv\Torrents\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:16 INFO ::MAIN: Auto-detected SECTION:NzbDrone
2017-07-19 06:49:21 INFO ::MAIN: Output directory set to: C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:21 INFO ::COPYLINK: MEDIAFILE: [The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv]
2017-07-19 06:49:21 INFO ::COPYLINK: SOURCE FOLDER: [C:\Media\Plexdownloads\tv\Torrents]
2017-07-19 06:49:21 INFO ::COPYLINK: TARGET FOLDER: [C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv]
2017-07-19 06:49:21 INFO ::COPYLINK: Hard linking SOURCE MEDIAFILE -> TARGET FOLDER
2017-07-19 06:49:21 WARNING ::COPYLINK: Error: Incorrect function.
, copying instead ...
2017-07-19 06:49:21 INFO ::COPYLINK: Copying SOURCE MEDIAFILE -> TARGET FOLDER
2017-07-19 06:49:30 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:30 INFO ::MAIN: Found 1 media files in C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:30 INFO ::MAIN: Calling NzbDrone:tv to post-process:The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:30 INFO ::MAIN: Attempting to verify tv fork
2017-07-19 06:49:30 INFO ::MAIN: NzbDrone:tv fork set to default
2017-07-19 06:49:30 INFO ::MAIN: FLATTEN: Flattening directory: C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv
2017-07-19 06:49:30 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request
2017-07-19 06:49:50 DB ::DB: Updating status of our download The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv in the DB to 1
2017-07-19 06:49:50 DB ::MAIN: nzbtomedia.db: UPDATE downloads SET status=?, last_update=? WHERE input_name=? with args [1, 736529, u'The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv']
2017-07-19 06:49:55 INFO ::CLEANDIRS: Directory C:\Media\PlexDownloads\drone\tv\The.Jim.Jefferies.Show.S01E05.1080p.WEB.x264-TBS.mkv has been processed, removing ...
2017-07-19 06:49:55 INFO ::MAIN: The C:\nzbToMedia\TorrentToMedia.py script completed successfully.
At 2) Instructions seem to be a bit unclear. The Remote Path Mapping must have an appropriate Remote & Local path specified. I assume you meant to say that the Local Directory specified is empty on disk.
At 3) Some cleanup might be a good idea. the block unintentionally ends with āremote_ā.
At 4) youāre not actually replacing any code. Might want to update in the issue you posted for clinton as well.
Otherwise Iād say it looks correct, I canāt comment on all the nzbToMedia specific settings but Iāve seen nothing weird.
However, you might want to make it more obvious which fields a user is supposed to update, paths, usernames/pw, etc.
"youāre not actually replacing any code. Might want to update in the issue you posted for clinton as well."
we arenāt?? Iām confused. I thought we had to edit that autoprocesstv.py file?
Iāll clean up the instructions for sure and start a new thread. Gimme a day or so.
I didnāt mess with any of the NZB settings either. Only using this as an extractor tool. This is pretty simple 10-minute setup thatās a very good resolution for b***h of a problem.
Iām glad we got it done . Thanks for your patience and willingness to help us out!
re: codeā¦ I meant you accidentally copy-pasted the unedited version. itās missing the āimportModeā part.
And thank you for sticking with it, I know itās not particularly easy (without built in unrar support).
Ahh, sorry, long nightā¦too much jameson. Corrected.
No problem man, Appreciate you .
Iāll write this up in the next 24 hours and blast it out.
Feel free to correct it if itās lacking something somewhere.
Draft is here:
If you say, ābut bro, just download a Linux docker and youāll be golden bro.ā Iām going to find you and poop in your beer.
It wouldnāt be better/easier anyway. How about a Windows Container? ** ducks **
Pretty thorough document Sean, I like how youāre clear about which directories are involved and their purpose.
I wish I could do something about that dummy Remote Path Mapping coz itās a bit of a hack, I had an idea earlier to do that fairly easily but that would apply to both usenet and torrent clients and we donāt want that.
ha, RIGHT?!
I mean, I intended this guide to cover the most complicated setup and then be backward compitible. If you donāt use usenet, simply remove it from this guide. Ya know?
I should be done in a couple hours.
I have updated these changes in NTM nightly branch.
Thanks for all the testing.
I now see the remote path settings relate to Sonarr and not NTMā¦ so that clarifies that one.
I still think you should be able to use hard-link in NTM to save processing time and disk usage for non-rared archivesā¦ Not sure what this is falling back to copy.
Hey guys. Iām trying to following the methods that @seanvree laid out in his guide.
Iām planning on posting some comments in that thread as well because for all the work he put into it there are some issues with clarityā¦but Iāll cover that later.
The main thing I want to check on here is this whole importmode:move that is stated must be updated in the NTM source. If Iām understanding this all correctly this is only needed when using a separate system as a seedbox, correct?
Without the āmoveā, then the call NTM makes to sonarr tells it just to hardlinkā¦but it canāt because it is on a different drive/system, so it defaults to copying. AND, the reason we donāt want that is because it leaves us with two copies of the same file (one on the seedbox and one on the media box). We are ok with two copies if the source is archived because we obviously need an archived one to seed (at least temporarily) and a non archived one to watch.
By setting the importMode to āmoveā, NTM tells sonarr not to try to hardlink(copy), but just to move all the files. Is this accurate?
What I am still confused about on the single box solution here (which seems to be working for me right now without the importMode changes to the NTM code) is once NTM sends the DownloadedEpisodesScan to sonarr, how/why is Sonarr moving from my temp/ādroneā directory to the proper season folder? I see something like this in my logs"
17-8-30 22:06:04.5|Debug|EpisodeFileMovingService|Moving episode file: D:\queue\pickup\tv\xxxx.S02E01.avi\xxxx.avi to E:\Data\Media\Video\TV\xxxx\Season 02\xxxx - 2x01 - episode.avi
17-8-30 22:06:04.5|Debug|DiskTransferService|Move [D:\queue\pickup\tv\xxxx.S02E01.avi\xxxx.S02E01.avi] > [E:\Data\Media\Video\TV\xxxx\Season 02\xxxx - 2x01 - episode.avi]
How is sonarr making the decision to move instead of attempting a hardlink or copy? In the hidden Media Management settings I see āUse Hardlinks instead of Copyā (which I have enabled), so I would assume it would either link or copyā¦not move. Now donāt get me wrongā¦Move is my desired behavior here because otherwise I would have a link (or worse a copy) in my temp/ādroneā folderā¦ I just donāt understand how it makes that call.
thanks
@Taloth - on a related note you might want to consider putting in a configurable switch in settings for DownloadedEpisodesScan to always do one or the other regardless of what is sent in the DownloadedEpisodesScan command. You could also add an extra option to DownloadedEpisodesScan command to override that configurable setting. This way the user can control what behavior they want without having to modify code. AND, if a dev wants they can override that behavior because they know 100% they need to. You may have a better idea in mind, but thought I would mention it.
Me againā¦wanted to get some more clarification on these things, and why they are needed particularly (Iāve read all the referenced threads).
The issue here is really only a problem if you still want to seed a torrent after completion, correct? I realize that a lot of the hoops to jump through here are precisely because we want to continue to do that and not have any file locks while we do. But, letās assume Iām not continuing to seed after completion (e.g. you pay $$ for private trackers because you donāt/canāt use the additional bandwidth to seed). Moralizing on the no-seeding aside, this remote file hack wouldnāt be necessary because you could either:
Am I understanding this correctly?
Assuming we donāt use the remote path hack, what exactly is the danger of Sonarr trying to import the file twice?
thanks seanā¦saw your post there although it doesnāt actually answer all my questions.
Iām going to assume that all my assumptions above were correct (if not please correct).
What I still need answered specifically:
andā¦
thanks guys