How to retrieve a file from Internet via HTTP? - windows

Question

I want to download a file from Internet and InternetReadFile seem a good and easy solution at the first glance. Actually, too good to be true. Indeed, digging a bit I have started to see that actually there are a lot of issues with it. People are complaining about all kinds of problems when using this code.
Problems could appear because:
the application freezes temporarily until the HTTP server responds
the application freezes temporarily because the Internet connections breaks
the application locks up because the HTTP server never responds
the InternetOpen (I just discovered this recently) MUST be called only once during application life time
I could not find a complete example about how to use it properly and robustly. Does anybody have an idea about how to implement it in a separate thread and with a time out? There is another SIMPLE way to robustly download a file from Internet. Though I don't want to complicate my life with very large libraries like Jedi or even Indy.
function GetFileHTTP (const fileURL, FileName: String): boolean;
CONST
BufferSize = 1024;
VAR
hSession, hURL: HInternet;
Buffer: array[1..BufferSize] of Byte;
BufferLen: DWORD;
f: File;
sAppName: string;
begin
// result := false;
sAppName := ExtractFileName(Application.ExeName) ;
hSession := InternetOpen(PChar(sAppName), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0) ; { be aware that InternetOpen need only be called once in your application!!!!!!!!!!!!!! }
TRY
hURL := InternetOpenURL(hSession, PChar(fileURL), nil, 0, 0, 0) ;
TRY
AssignFile(f, FileName) ;
Rewrite(f, 1) ;
REPEAT
InternetReadFile(hURL, #Buffer, SizeOf(Buffer), BufferLen);
BlockWrite(f, Buffer, BufferLen)
UNTIL BufferLen = 0;
CloseFile(f) ;
Result:= True;
FINALLY
InternetCloseHandle(hURL)
end
FINALLY
InternetCloseHandle(hSession)
END;
END;
Edit:
This functions checks if Internet connection is available. It seems to work on Win98 also.
{ Are we connected to the Internet? }
function IsConnectedToInternet: Boolean; { Call SHELL32.DLL for Win < Win98 otherwise call URL.dll }
var InetIsOffline: function(dwFlags: DWORD): BOOL; stdcall;
begin
Result:= FALSE;
if IsApiFunctionAvailable('URL.DLL', 'InetIsOffline', #InetIsOffline)
then Result:= NOT InetIsOffLine(0)
else
if IsApiFunctionAvailable('SHELL32.DLL', 'InetIsOffline', #InetIsOffline)
then Result:= NOT InetIsOffLine(0)
end;
I am using Delphi 7. Many thanks.
Edit:
Losing customers because the application hangs at the first start up is the perfect recipe for losing money.
Writing your code to be Microsoft platform dependent is bad. You never know if the customer has the IE version x.x installed.
Installing stuff into a user's computer is like playing with guns. It will backfire.
(see more about this here: http://thesunstroke.blogspot.com/2010/06/programmig-like-there-is-no-ms-windows.html)

Answer 1

I basically do the same as you do. For me it works fairly flawlessly.
The only differences between my code and your code is I have an INTERNET_FLAG_RELOAD parameter to force a download from the file and not the cache. You can try that and see if it works better:
hURL := InternetOpenURL(hSession, PChar(fileURL), nil, 0, INTERNET_FLAG_RELOAD, 0) ;
Also check for an internet connection before downloading. Do this:
dwConnectionTypes := INTERNET_CONNECTION_MODEM
+ INTERNET_CONNECTION_LAN
+ INTERNET_CONNECTION_PROXY;
InternetConnected := InternetGetConnectedState(#dwConnectionTypes, 0);
if InternetConnected then ...

Answer 2

Here's some sample code that uses Indy. This code is for Delphi 2010 (with Indy 10?), but the code for Delphi 7 would be similar. I've used Indy for years with D7 and have been very happy with it. I think in D7 we use Indy 9. Check if you need to download a new version...
You can use OnWork and OnWorkBegin to add a progress meter if you need to.
This code I excerpted from a bigger piece, editing it a bit. I did not try compiling it, but it will give you a good starting place.
function Download( const aSourceURL: String;
const aDestFileName: String;
out aDownloadResult: TDownloadResult;
out aErrm: String): boolean;
var
Stream: TMemoryStream;
IDAntiFreeze: TIDAntiFreeze;
begin
aDownloadResult := DROther;
Result := FALSE;
fIDHTTP := TIDHTTP.Create;
fIDHTTP.HandleRedirects := TRUE;
fIDHTTP.AllowCookies := FALSE;
fIDHTTP.Request.UserAgent := 'Mozilla/4.0';
fIDHTTP.Request.Connection := 'Keep-Alive';
fIDHTTP.Request.ProxyConnection := 'Keep-Alive';
fIDHTTP.Request.CacheControl := 'no-cache';
IDAntiFreeze := TIDAntiFreeze.Create;
Stream := TMemoryStream.Create;
try
try
fIDHTTP.Get(aSourceURL, Stream);
if FileExists(aDestFileName) then
DeleteFile(PWideChar(aDestFileName));
Stream.SaveToFile(aDestFileName);
Result := TRUE;
aDownloadResult :=drSuccess;
except
On E: Exception do
begin
Result := FALSE;
aErrm := E.Message + ' (' + IntToStr(fIDHTTP.ResponseCode) + ')';
end;
end;
finally
Stream.Free;
IDAntiFreeze.Free;
fIDHTTP.Free;
end;
end; { Download }

Answer 3

My personal favorite is using the WebHttpRequest component from importing the "Microsoft WinHTTP Services" type library: http://yoy.be/item.asp?i142
var
w:IWebHttpRequest;
f:TFileStream;
os:TOleStream;
begin
w:=CoWebHttpRequest.Create;
w.Open('GET',SourceURL,false);
w.Send(EmptyParam);
os:=TOleStream.Create(IUnknown(w.ResponseStream) as IStream);
f:=TFileStream.Create(DestinationFilePath,fmCreate);
os.Position:=0;
f.CopyFrom(os,os.Size);
f.Free;
os.Free;
w:=nil;
end;

Answer 4

I recommend Synapse. It's small, stable and easy-to-use (no need of any external libraries).
Example from httpsend.pas
function HttpGetText(const URL: string; const Response: TStrings): Boolean;
var
HTTP: THTTPSend;
begin
HTTP := THTTPSend.Create;
try
Result := HTTP.HTTPMethod('GET', URL);
if Result then
Response.LoadFromStream(HTTP.Document);
finally
HTTP.Free;
end;
end;

Answer 5

Instead of fiddling with the WinAPI, the ExtActns unit provides just what you need for downloading to a file.
procedure TMainForm.DownloadFile(URL: string; Dest: string);
var
dl: TDownloadURL;
begin
dl := TDownloadURL.Create(self);
try
dl.URL := URL;
dl.FileName := Dest;
dl.ExecuteTarget(nil); //this downloads the file
dl.Free;
except
dl.Free;
end;
end;
Under the hood, it uses URLDownloadToFile from the URLMon library - which is part of IE, and therefore part of Windows.
TDownloadURL doesn't handle any timeout for you - it doesn't look like such a thing is supported in URLMon at all, although there could be some default timeout that causes the call to fail - but you could use the OnProgress event on TDownloadURL to get notified when something happens, and then do something in another thread if it's been too long since the last callback.

Answer 6

Solved using improved version of the above code.
(it still does not solve all issues - MS does not actually implemented full support for server time out)
The connection does not timeout while downloading file from internet