Files corrupting during move process (uTorrent)

Help & Support
#1

Having looked this up I’ve found mentions of nzb clients running post processing scripts and issues around that but don’t think it’ll apply here.

Noticed this happening several days ago and after a restart it stopped doing it, but now it’s cropped up again.

Set up:

Sonarr passes download to uTorrent
uTorrent downloads to E:\Downloads\In Progress
uTorrent completes download and moves to E:\Downloads\Completed\tv
uTorrent runs WinRAR on the download, extracts files and removes all archive files - At this point the video file is good matches the CRC from the archive
Sonarr picks up the completion around 30 seconds later (for this download in particular and checked from Date Created of unrar’d mkv and compared to log from Sonarr)
Sonar correctly identifies (multiple times from the looks of the log) the episode and proceeds to move it to D:\TV<showname> (a different drive)
The renamed episode exists now in the TV<showname> dir but the original release named file is still in the completed download dir.

It’s at this point that something goes wrong. When played back the renamed file ends prematurely and comparing CRC’s of renamed to not renamed they differ. The file sizes are exactly the same

Log of file move: http://pastebin.com/GH7Vs9ag

Putting the working video file in the drone sorter folder (different folder to completed downloads), it correctly overwrites the corrupted file and renames.

I suspect restarting the Sonarr service will fix this again for a short time.

#2

Do you use CDH (completed download handling) or the drone factory folder?
I don’t know uTorrent that well, but “uTorrent runs WinRAR on the download”: uTorrent will report the download as complete after this action or is it more like “when download is complete, run this script”?

#3

CDH is used, but I do have a drone factory folder for manually importing files, it’s outside of where uTorrent stores downloads so shouldn’t interfere with CDH - correct?

uTorrent has a “Run this program when the torrent finishes” option that I’ve set up to use the winrar.exe with some arguments to unrar the files.
I mentioned:

Sonarr picks up the completion around 30 seconds later

And I meant that Sonarr’s log file states that handling the completed download occurs around 30 seconds after the mkv’s date created attribute. I believe the extraction process should have completed well within this time frame, however, if it were to encroach on it surely the file size would be less than when fully extracted?

#4

if it were to encroach on it surely the file size would be less than when fully extracted?

I’d expect that as well, but its a fairly easy test, if you turn off Completed Download Handling, then turn it on after the download completes and extracts, turn CDH on and let it process the download.

Drone Factory and CDH use similar process, except that for torrents that are still seeding CDH will copy or hard link (in your case copy, because they are different drives).

Are the two drives local?
Would you be able to temporarily change the path of of one series to the E drive, enable hard linking in Sonarr and report on the result?

#5

Should’ve mentioned I had previously tried that, turning CDH off, but did so by closing Sonarr instead of just turning CDH off. And as I mentioned previously after restarting Sonarr the issue seemed to disappear for a couple of days.
But now I’m not sure if that was because some downloads aren’t in archives - and if it works when it’s not archived then the cause is that it’s more than likely attempting to move before the extraction completes (which I’ve now seen occurs after moving the files to the completed directory instead of before.

Both drives are local, they’re Windows Spaces, and I was thinking I could move the download folder to be located on the same drive as the TV folder to rule this out (having read it’s a potential cause of issue on other threads).

I’d restarted my computer since my last post and it appears to no longer be copying across a corrupted file, however, it is leaving the folder (and subsequent extracted file) in the download directory instead of deleting them.

Monitoring the files I’ve seen that seeding occurs while the file is in the completed directory so I tried both moving the downloads directory to the same drive as final destination and enabled hard linking. But still the folder is left over after it’s removed from uTorrent by Sonarr. Everything else is removed (.sfv, .nfo) just not the .mkv.

Here the log of another episode I tried, http://pastebin.com/SRSzWhSQ
And further down in the log the only other mention of it

 15-5-2 14:52:53.0|Debug|DownloadEventHub|[The.Simpsons.S26E19.HDTV.x264-LOL] Removing download from htpc history
15-5-2 14:52:53.0|Debug|UTorrentProxy|Url: http://localhost:8080/gui/?token=<removed>&action=removedata&hash=2D0AFFF3C237359F667843DAAC92FA8E4AE56B17
15-5-2 14:52:53.0|Debug|RestSharpExtensions|Validating Responses from [GET] [http://localhost:8080/gui/?token=<removed>&action=removedata&hash=2D0AFFF3C237359F667843DAAC92FA8E4AE56B17] status: [OK]
#6

Folders not being removed may be a permissions issue.
Doing some testing now to check.

Edit: nope, doesn’t look like it.

@markus101 does the removal process of a completed download only involve telling uTorrent that it needs to delete the entry in the client and the data associated with the torrent? If that’s the case it may not be removing the directory because the mkv inside is seen as a foreign file.
But if that’s the case I feel like this wasn’t an issue for me for a while and directories were being removed. But that might’ve been before I started using torrents that were archived and were only the video files, therefore adding no new foreign (as far as the torrent client is concerned) files!
I’ll test with a torrent that only contains a video file.

This looks to be a side issue, I have a feeling the corrupted video issue will come up again and I’ll post here with my findings if it does.

#7

Okay another 3 tests done:

  1. Torrent containing no archives- imported correctly, deleted the folder and entry from uTorrent
  2. Torrent containing archives, but removed the command to extract using WinRAR - completed, Sonarr couldn’t import with no video file so gave the option to force import. Extracted manually and forced import - deleted folder and entry from uTorrent
  3. Torrent containing archives, command set to extract using WinRAR - completed, WinRAR did its thing, Sonarr imported the video as expected, did not delete folder, did delete entry from uTorrent

What’s the difference between forcing import and normal import that results in normal import not deleting everything in the folder?

Edit: Started a download for a 720p HDTV episode using test method 3, left and when I came back it had completed (but still seeding) and imported it. File was corrupt in the TV directory. Note this was back to copying between hard drives using CDH.

May have to look at either only using trackers that don’t rar releases or using drone factory if Sonarr cannot either be told to wait until the extraction is complete (with a post processing script like nzb clients use?) or can do the extraction itself.

Edit2: OR look at downloading to the same drive as the final destination and hard linking - when I tried this earlier there was no corruption, only the folder and file left over which could be its own issue?

Edit3: Aaaaand it looks like hard links on the same drive has worked. No corruption and original folder and contents removed. I hope this is it but I guess I’ll have to wait a couple of days to see.

#8

It just bypasses some checks, but post import (deleting the folder) is the same.

It comes down to when the client reports that the torrent is done, in the usenet world SAB/get have scripts as part of the process for each download, only saying its complete when it has completed executing the script, for torrents its possible it completes and is marked as complete, then the script is triggered, so the unrarring and import by Sonarr trigger in parallel. We will likely look at unrar support on our side, but its not something we’re going to look at immediately.

#9

I moved a post to a new topic: Completed Toreent being remived before it is imported

#10

Can you provide me with the script you are using in Utorrent to unrar and send to a different folder and deletes the archives? I’ve been trying to get this done for a while now and seem to never get it right. I appreciate your help in advance!

#11

Sure.
Firstly you have to open a .rar file (any will do) using WinRAR on the user login that’ll be running uTorrent (usually your own).
Click “Extract To” then click on the Advanced tab in the new window.
Under “Delete Archive” choose “Always”.
Now whenever you unrar files, the rars will be deleted upon completion.

In uTorrent open Options, + Advanced and then "Run program"
Enter "C:\Program Files\WinRAR\WinRAR.exe" x -ibck "%D\*" "%D" in the field labeled “Run this program when a torrent finishes”

Done.

#12

Thank you, it worked! But there is an issue. Maybe you know how to fix it.

So I have downloads goto D:\Downloads when in progress and when finished it gets moved to D:\Drone (which is my drone factory that will pick up the files and import it into Sonarr).

Once its in D:\Drone it unrars it but it unrars it into the a folder.

So I believe the Drone factory is only able to pick up files from the root folder (D:\Drone)
And as an example I don’t think it can search D:\Drone\AwesomeTVshow\TVshow.mkv
Edit: Just ran a test and it will only pick up from Root and not inside a folder

So is there a way to have all the files extract directly to root?

Or maybe my setup isn’t the best.

Here is what I have:

Sonarr controlling all downloads and searching (using CDH and Drone Factory)
Utorrent as my torrent client
Put new downloads in D:\Downloads
Move completed downloads to D:\Drone

Thanks in advance

Edit 2: Also when Utorrent downloads a file that isn’t a RAR and it tries to execute the WinRAR command it just comes up w/ “No archive found”. I know this is normal but have you figured any way around it?

#13

Sonarr will only look inside a folder in the Drone Factory if it can match the folder to a series.

You should be able to extract downloads in the same folder as where they were downloaded and avoid Drone Factory entirely though.

#14

So when the Drone factory isn’t used it would be using the CDH right? If so, are you are saying the CDH will pull the file out provided that Sonarr initiated the download and passed the info on to Utorrent?
Follow up question, it will only work that way if Sonarr intitiated the donwload right? If I saw I was missing a file and downloaded it manually via Utorrent without Sonarr’s help it would not get picked up, correct?

#15

as long as the utorrent label matches sonarr will import it

#16

Thanks for the clarification.

#17