NZB Drone Post-Processor, end of file gets cut off when moving across network

If anyone is interested, I put together this little script to find potentially corrupted files:

#!/usr/bin/python

import os
import sys

def checkfile(filename):
    command = "echo 'seek 95 1' | mplayer -slave -msglevel all=4 -benchmark -vo null -nosound \"" + filename + "\" 2>&1"
    output = os.popen(command).read()
    if output.lower().find('error') == -1:
        return True
    else:
        return False

if __name__ == "__main__":
    for filename in sys.argv[1:]:
        if checkfile(filename):
            print('OK|' + filename)
        else:
            print('ERROR|' + filename)

It uses mplayer to seek to 95% of the file and attempts to play it. If any errors are reported it prints out ERROR| followed by the filename, otherwise it prints out OK|.

You can use it either by doing:

checkfile.py *.mkv

Or if you want to scan for recently created files, this will find all files created in the last 90 days and check them (assuming the script is in your home directory and you are in your TV directory). Output will be written to /tmp/filecheck.txt

find . -ctime -90 -type f -iname "*.jpg" ! -iname "*.nfo" ! -iname "*.tbn" -exec  ~/checkfile.py \{\} \; > /tmp/filecheck.txt

Output will look something like this:

OK|./The Strain/Season 01/s01e03 Gone Smooth 720p WEB-DL.mkv
ERROR|./The Strain/Season 01/s01e07 For Services Rendered WEBDL-720p.mkv
OK|./The Strain/Season 01/s01e06 Occultation HDTV-720p.mkv

My stats after running this:

Total files: 1332
Corrupted: 85 (6.4%)

I was thinking about getting a Synology NAS and running nzbdrone from it, will this issue affect that version of drone? Will I have to use NFS on the windows machine that I use as my htpc or will it work with samba shares without cutting off the files? I am wondering because hanewin NFS stopped worked for me recently and I haven’t been able to figure out why so I switched back to using samba shares.

The issue is when drone is running in mono and copying files to a SMB share, so no you won’t have an issue, unless you copy off the syno with drone, which would be the opposite of what you want.

So you’re saying the way to go would be to run nzbdrone from the windows htpc and not run nzbdrone on the synology since that would be a using a mono version of drone and copying to a windows smb share? Ideally I was trying to move as many background processes as I could to the synology.

Wh are you copying off the synology to a share to another device? Isn’t the synology a NAS?

What’s the actual setup you want to use? The more information the better as the hypotheticals are confusing.

Ah, yes, I see where the confusion came from, my tv libraries are on the windows box currently. But I guess I will be moving them to the NAS when I set it up. Sorry it was me that was confused :wink: At this point for me it is hypothetical since I don’t actually have the NAS yet, I am just planning it’s setup. Thanks for walking through it with me.

I’d like to know how everyone got on with this issue, ultimately.
I was working fine for a while, and then i think i must have updated some packages. after a reboot - All of my TV episodes ( which download in their entirety) now get truncated at random points when being transferred to my drobo (via cifs mount).
I tried updating to the latest mono (4.01) but to no avail.
It’s not quite a showstopper as i can save them locally on my download server and then manually copy them over at a later point. a standard cp command has no issue transferring the whole file (leading me to believe it’s definitely the mono environment that’s the culprit).
My setup is:
Ubuntu 12.04 (64bit)
Mono 4.01

example mount string from my fstab:
//x.x.x.x/kidsmedia /media/my/mount/point cifs username=bionicdude,password=xxx,iocharset=utf8,auto,user,uid=root,gid=users,file_mode=0777,dir_mode=0777,defaults,_netdev 0 0

Lots of other processes transfer data to my drobo without issue, so I’ve ruled out issues with my NAS (and assume the mountpoint is fine too).

Any help would be greatly appreciated (I undestand the mono development is out of the control of anyone here).

Can you switch to the develop branch? We implemented feature there to detect such truncation and handle it. (The real problem is samba/cifs, fyi, not something we can fix, so we just work around it with some magic)

I suppose most ppl just dumped CIFS :slight_smile: But check out the develop branch first.

switched to the development branch and can see the attempts to fix the MONO/CIFS issue.
whilst transferring, i ran an ls command. you can see where it goes wrong and suddently the file is smaller…
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 405004
-rwxrwxrwx 1 root users 632766464 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 406796
-rwxrwxrwx 0 root users 634601472 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 592203776 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 612614144 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 632799232 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 664363008 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 688865280 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 710828032 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 733364224 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 760229888 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 777283559 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 777283559 Jul 10 13:38 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 777283559 Jul 10 13:40 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 302784
-rwxrwxrwx 0 root users 310050816 Jul 10 13:40 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 681779200 Jul 10 13:40 cifse71e
autolog@DOWNNIX:/home/bionicdude/Downloads/TV/Workaholics/Season04$ ls -l
total 0
-rwxrwxrwx 0 root users 681779200 Jul 10 13:40 cifse71e

Incidentally - DANG! it slows my machine something crazy whilst doing this…
Not sure if this gives you anything to go by or will help in any way.
For now, I’ll go back to downloading locally and copying later.

I would appreciate getting a tracelog of sonarr while it’s doing that.

Sorry for the delay.

I cleared all the log files, restarted Sonnar and tried again.
things started hanging. Sonnar just didn’t recover at all and had to be restarted.
one NzbDone process would not die (signal 9 from root wouldn’t kill it).
I tried to restart the machine (via ssh) but it won’t even shutdown…

I’ll try when i get home…

Incidentally:
the log file had no mention of the transfer attempt.
I had a partial~ file in the destination directory, which seemed to be the right size.

after i restarted the machine and started up NzbDrone, it retried to copy the file.
the outcome was a failed attempt to copy (didn’t copy whole file). followed by a pushbullet to my phone telling me it was successful, and the below log (everything after the restart blurb):
I didn’t feel the need to obfuscate any of the paths…

15-7-23 18:37:38.3|Info|Database|Vacuuming Log database
15-7-23 18:37:38.4|Info|Database|Log database compressed
15-7-23 18:38:05.7|Warn|DiskTransferService|Failed to completely transfer [/home/autolog/Downloads/.temp_anime/Workaholics.S04E01.720p.BluRay.X264-REWARD/workaholics.s04e01.720p.bluray.x264-reward.mkv] to [/media/bionas/media/TV/Workaholics/Season04/Workaholics - S04E01 - Orgazmo Birth.mkv], retrying [1/2].
15-7-23 18:39:39.2|Warn|DiskTransferService|Failed to completely transfer [/home/SharedDownloads/.temp_anime/Workaholics.S04E01.720p.BluRay.X264-REWARD/workaholics.s04e01.720p.bluray.x264-reward.mkv] to [/media/bionas/media/TV/Workaholics/Season04/Workaholics - S04E01 - Orgazmo Birth.mkv], retrying [1/2].
15-7-23 18:39:40.2|Info|Database|Vacuuming Main database
15-7-23 18:39:41.4|Info|Database|Main database compressed
15-7-23 18:41:10.3|Warn|ImportApprovedEpisodes|Couldn’t import episode /home/autolog/Downloads/.temp_anime/Workaholics.S04E01.720p.BluRay.X264-REWARD/workaholics.s04e01.720p.bluray.x264-reward.mkv

System.IO.IOException: Win32 IO returned ERROR_GEN_FAILURE. Path: /media/bionas/media/TV/Workaholics/Season04/Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
at System.IO.File.Delete (System.String path) [0x00000] in :0
at NzbDrone.Common.Disk.DiskProviderBase.DeleteFile (System.String path) [0x00000] in :0
at NzbDrone.Common.Disk.DiskTransferService.TryCopyFile (System.String sourcePath, System.String targetPath) [0x00000] in :0
at NzbDrone.Common.Disk.DiskTransferService.TryMoveFile (System.String sourcePath, System.String targetPath) [0x00000] in :0
at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, TransferMode mode, Boolean overwrite, Boolean verified) [0x00000] in :0
at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List1 episodes, System.String destinationFilePath, TransferMode mode) [0x00000] in <filename unknown>:0 at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.MoveEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x00000] in <filename unknown>:0 at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, Boolean copyOnly) [0x00000] in <filename unknown>:0 at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List1 decisions, Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem) [0x00000] in :0

whilst it was attempting, i was checking progress from the living room.
Again, you can see where it suddenly seems to start to transfer the file again.
Interestingly, the log i posted shows both attempts to transfer the file as 1/2.

bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 13312
-rwxrwxrwx 1 root users 633798656 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 13312
-rwxrwxrwx 1 root users 717881344 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 3072
-rwxrwxrwx 1 root users 2752512 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 21504
-rwxrwxrwx 1 root users 21168128 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 34816
-rwxrwxrwx 1 root users 35454976 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 80896
-rwxrwxrwx 1 root users 82575360 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 102400
-rwxrwxrwx 1 root users 104398848 Jul 23 18:39 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 223232
-rwxrwxrwx 1 root users 333250560 Jul 23 18:40 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 223232
-rwxrwxrwx 1 root users 333250560 Jul 23 18:40 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 292864
-rwxrwxrwx 1 root users 643497984 Jul 23 18:40 Workaholics - S04E01 - Orgazmo Birth.mkv.partial~
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 305152
-rwxrwxrwx 1 root users 737673216 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth.mkv
-rwxrwxrwx 1 root users 566 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth.nfo
-rwxrwxrwx 1 root users 140744 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth-thumb.jpg
bionicdude@LivingNix:/media/bionas/media/TV/Workaholics/Season04$ ls -l
total 305152
-rwxrwxrwx 1 root users 737673216 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth.mkv
-rwxrwxrwx 1 root users 566 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth.nfo
-rwxrwxrwx 1 root users 140744 Jul 23 18:41 Workaholics - S04E01 - Orgazmo Birth-thumb.jpg

Enable Debug/Trace logging in Settings->General. Gives you a bit more info, although not much in this specific case.

A few questions:

Found this in the log:
/home/autolog/Downloads/.temp_anime/Workaholics.S04E01.720p.BluRay.X264-REWARD/workaholics.s04e01.720p.bluray.x264-reward.mkv
vs
/home/SharedDownloads/.temp_anime/Workaholics.S04E01.720p.BluRay.X264-REWARD/workaholics.s04e01.720p.bluray.x264-reward.mkv

Explain that, two paths to the same file? Do you have Drone Factory enabled? And at the same time Completed Download Handling with some remote path mapping?

I have the same issue since beginning of my raspbery pi and ubuntu setup. I have about 1 to 10 files missing the end, the video plays fine until the 96% or something and then it ends forcibly. I have the same almost setup files are being downloaded with sabnzbd and stored in ubuntu and then sonarr picks them up and copied to a NTFS share on another NAS server over a mount directory on ubuntu.

Does anyone find any permanent solution on this ? or what to look for ??

many thanks @megatron for your script, i tweak it a bit for my needs

same issue about 10-15% of the files are truncated last 10% or so of the file. Any resolution? Latest rev of Ubuntu and Sonarr, full updates done.

thanks,

Man, I’m glad I stumbled on this thread - I’ve been trying to figure out why all my episodes would only play through the first 8-12 minutes after post-processing.

Latest Sonarr is running on latest Ubuntu, and I have my NAS shares automount in fstab via CIFS. The cut-off problem was driving me nuts!

After some trial and error with getting my QNAP NAS set up with permissions for NFS, I can happily say that “move” is working again and my files are transferring in full.

Hate to be one of those “me too” guys, but I’ve had the same problem. Switching to NFS helped.

Anecdotally, I think the problem was more to do with the file copying process in combination with the 1 minute timer to see if the file was copied. I noticed that if that if the copy was slow and the “Downloaded Episodes Scan” task triggered (although it could have been the “Check For Finished Download” task) at the time it was copying, there was a higher chance that the end would be cut off.

Not sure if that’s the same as other people, or even that’s related or not (as it wasn’t a perfect correlation), but would be good to know if other people found the same.

this problem was the reason i switch my installation from my ubuntu to mac, the issue was unbearable

Add my name to the list of users affected by this bug. I recently rebuilt my NAS and ever since the number of failures has shot up. To the point that I no longer trust Sonarr. My drives are shared over CIFS on ubuntu and are being written to from sab/sonarr running on Debian.

Helpfully, Sonarr marks the ‘failed’ transfers as purple because as far as it’s concerned it never receives a success flag from the OS that the transfer has completed successfully. It also doesn’t appear to receive a fail either so it never retries. In fact it can’t retry because the file has been unsuccessfully moved, not copied.

Would it be possible to change the move command to copy-verify-delete/retry? I appreciate that this will require more code but since this bug has been around for several years and no-one has ever identified a solution, there does not appear to be an alternative.

As it stands I’ve gone back to downloading missing files manually and bypassing Sonarr completely. Not ideal.