FTP downloads from ISA fail

[suedtiroler]

I use ncftpget.exe (ver 3.1.9.10) from a script that is started via a scheduled task for downloading files directly at the isa server, every night.

This procedure went fine for years, until now i installed BS. Now the download fails every time, sometimes before starting to download the first file, sometimes over 200mb get copied before the download fails. In this case, large files (50mb), very small files, everything gets downloaded. At some point, ncftpget.exe just waits forever for the download of the current file to complete.

There is a rule permitting FTP downloads from local host. I tried configuring the ftp protocol to disable the inclusion of the BS application filter. The error still occurred. The only way i could get rid of the error was to completely disable the BS application filter under "Configuration-Add Ins", restarting the services and then try again.

OS: Win Server 2003 Standard SP2
ISA: 2004 Standard Edition, SP3
BS: 1.30

[Evgeny]

Was any shaping or quota rule applied to traffic of these downloads?
Do you have any antivirus software on this server?
Do you have any other 3rd-party ISA add-ons or filters on this server?
Are there any errors in Event Viewer on this server?

[suedtiroler]

- no this download is not affected by any bs rule. as i am new to bs, i have only created one rule for testing purposes. and this one rule does not affect the ftp download.
- yes antivirus software is installed, same product that has been on the server for years now. no recent changes here.
- no, bs is the only add-on / filter /whatever that is installed on the isa
- i see no relevant errors in the event log that might give a clue

thanks for the response!

[Evgeny]

Probably it is the combination of bsplitter and antivirus software causes the issue. Can you temporary disable the antivirus and see if problem still remains?

[suedtiroler]

i deactivated the whole antivirus thing and it not help the situation. the symptoms remained the same. thanks for the suggestion though

[Evgeny]

I see that there is a newer version 3.2.3 available of ncftpget program: http://www.ncftp.com/download/
Maybe it contains fixes that are related to this issue. Can you try it?
Do you experience this issue with any other clients? For example, if you try to download from ISA Server computer using another ftp client?

[suedtiroler]

thanks for the suggestions. using the newest version ncftp 3.2.3.16 did not make any difference.

a gui ftp client like filezilla was able to download the whole ftp structure (it contains subfolders, many files etc) without problems. but filezilla is not automatable within a batch file. i guess ncftp fails at some time because there is only one thread which downloads one file after another. filezilla downloads files simultaneoulsy, i guess using multithreading.

i have to use a command line based utility able to download whole ftp directory structures with subfolders etc because this whole thing runs as a scheduled task every night.

the bottom line remains that the bs application filter has some influence... maybe the bs developers can reproduce this behaviour using ncftpget? i can publish the name of the server adn the command line i use if that helps to reproduce behaviour...

[Evgeny]

maybe the bs developers can reproduce this behaviour using ncftpget? i can publish the name of the server adn the command line i use if that helps to reproduce behaviour...

Ok, if it is a public server, we can try to reproduce it.

[suedtiroler]

yes its the public ftp server network associates (nai) uses for antivirus updates. the command line is

REM -E Use regular (PORT) data connections.
REM -R Recursive mode; copy whole directory trees.
REM -t XX Timeout after XX seconds.
REM -d XX Use the file XX for debug logging.
ncftpget.exe -E -R -t 6000 -d ftplogfile.txt ftp://ftp.nai.com/commonupdater

[Evgeny]

I tried it on a server with Bandwidth Splitter, and it seems to run without errors. About 500 MB were downloaded.
I think maybe there is still interference from your antivirus software. Even if it is disabled, it can still filter the downloaded files. Can you try to completely uninstall it, reboot the server, and then try again to download the files?

[suedtiroler]

thanks for the prompt response and your effort. about 500mb is perfectly normal.

i'm only afraid i can't just uninstall our antivirus and reboot the server as it is our production environment (of course now one might say why install bs on a production system without prior testing anyway, but our resources here are kinda limited ;-).

then again, i'm sure the av does not affect the files once they are downloaded, without bs the av did never interfere. in fact, it's the actual downloading that fails, the files that are downloaded until that point seem perfectly normal.

of course i understand the necessity to eliminate all variables one after another in order to get to the bottom of this issue. it's just that for me, to eliminate the av component from the equation, it'll take some time to set up a proper test environment for installing an isa, domain controller etc... in the meantime, could you think of anything else that might be of influence? of course, since you are not able to reproduce the behaviour i'm witnessing, i understand it's difficult to go on from this point...

in the meantime, thanks again!

[Evgeny]

Can you say what antivirus (and what version, is it latest?) you are using? Maybe we can try to install it (if there is a trial version).
Do you have any other software on this server that could affect the system?

[suedtiroler]

yesterday i've set up a virtual test environment containing only a dc and a isa server, using the same os and isa versions my production system has.

i set up my ftp rule and was able to download that whole nai site normally.

i then installed bs. of course i didn't go through the lengthy steps of registering it on the virtual box, i just used that trial version. the ftp job still went ok.

then i installed the same antivirus i use in my production system (its mcafee virusscan enterprise 8.7i) and updated it with actual definitions. still, the download completed successfully in the virtual isa with bs.

finally i recreated the only bs test rule i had in my production system on the virtual test lab with the same parameters. as said before, this rule does not affect the isa ftp download, but whatever. even then, the download completed successfully on the virtual box. so i was not able to reproduce the error in the testing environment.

i then restarted my production isa twice, but the error there always remained the same. as long as the add in is enabled in the production isa, the download comes to a standstill after a certain time. but today morning i checked the download and it seemed okay. i retried the download some times, it failed once but once it also went okay. i guess i have to monitor the situation a few days since this task runs every night...

[suedtiroler]

until now, the download job seems to succeed... it does not work EVERY night, but that has always been the case with this ftp site... but it should never fail for about 5 days in succession and in fact it didn't so far... i'll keep monitoring

[suedtiroler]

it seems to work so far... i can't draw any conclusion since no particular action seemed to solve the problem.. maybe it was just the issue of restarting the server once or twice... i'll keep you posted here in the next couple of days/weeks if there are any changes...