I've got a "send" routine in Delphi 6 that accepts a variable-sized block of data (a fixed-size header followed by varying amounts of data) and the routine eventually calls sendto() in Winsock. I've coded it two ways, once where the passed block is a var (somewhat misleading, but it works) and once where a pointer to the block is passed. A simple version used for benchmarking looks something like:
type
header = record destination, serialnumber: integer end;
pheader = ^header;
var
smallblock: record h: header; data: array[1..5] of integer end;
bigblock: record h: header; data: array[1..100] of integer end;
procedure send1(var h: header; size: integer);
begin
h.destination := 1; // typical header adjustments before sendto()
h.serialnumber := 2;
sendto(sock, h, size, 0, client, sizeof(client))
end;
procedure send2(p: pheader; size: cardinal);
begin
p^.destination := 1;
p^.serialnumber := 2;
sendto(sock, p^, size, 0, client, sizeof(client))
end;
procedure doit1;
begin
send1(smallblock.h, sizeof(smallblock));
send1(bigblock.h, sizeof(bigblock));
end;
procedure doit2;
begin
send2(#smallblock, sizeof(smallblock));
send2(#bigblock, sizeof(bigblock));
end;
The "send" routine will be called often, with many different block sizes, and should be as fast as possible. After doing a few runs of some simple benchmarks (by timing calls with gettickcount), the pointer technique (doit2) seems to run about 3% faster on my machine than the var technique (doit1), although I don't see any real difference between the two techniques in the object code (not that I'm an assembler guru).
Is the 3% an illusion due to my crude benchmarks, or is the pointer technique really beating the var technique?
There is no performance difference passing a var parameter versus a pointer parameter. They do exactly the same thing (pass a memory address), and compile to similar, if not identical, assembly code. So any benchmarking differences are likely to be caused by issues in the benchmarking itself, not in the code that is being benchmarked. GetTickCount() is not exactly the best benchmarking tool, for instance. The best way to time your code is to use an external profiler, like AQTime.
BTW, your doit2() test should be like this instead:
procedure doit2;
begin
send2(#(smallblock.h), sizeof(smallblock));
send2(#(bigblock.h), sizeof(bigblock));
end;
Related
I don't work with Pascal very often so I apologise if this question is basic. I am working on a binary file program that writes an array of custom made records to a binary file.
Eventually I want it to be able to write multiple arrays of different custom record types to one single binary file.
For that reason I thought I would write an integer first being the number of bytes that the next array will be in total. Then I write the array itself. I can then read the first integer type block - to tell me the size of the next blocks to read in directly to an array.
For example - when writing the binary file I would do something like this:
assignfile(f,MasterFileName);
{$I-}
reset(f,1);
{$I+}
n := IOResult;
if n<> 0 then
begin
{$I-}
rewrite(f);
{$I+}
end;
n:= IOResult;
If n <> 0 then
begin
writeln('Error creating file: ', n);
end
else
begin
SetLength(MyArray, 2);
MyArray[0].ID := 101;
MyArray[0].Att1 := 'Hi';
MyArray[0].Att2 := 'MyArray 0 - Att2';
MyArray[0].Value := 1;
MyArray[1].ID := 102;
MyArray[1].Att1:= 'Hi again';
MyArray[1].Att2:= MyArray 1 - Att2';
MyArray[1].Value:= 5;
SizeOfArray := sizeOf(MyArray);
writeln('Size of character array: ', SizeOfArray);
writeln('Size of integer var: ', sizeof(SizeOfArray));
blockwrite(f,sizeOfArray,sizeof(SizeOfArray),actual);
blockwrite(f,MyArray,SizeOfArray,actual);
Close(f);
Then you could re-read the file with something like this:
Assign(f, MasterFileName);
Reset(f,1);
blockread(f,SizeOfArray,sizeof(SizeOfArray),actual);
blockread(f,MyArray,SizeOfArray,actual);
Close(f);
This has the idea that after these blocks have been read that you can then have a new integer recorded and a new array then saved etc.
It reads the integer parts of the records in but nothing for the strings. The record would be something like this:
TMyType = record
ID : Integer;
att1 : string;
att2 : String;
Value : Integer;
end;
Any help gratefully received!!
TMyType = record
ID : Integer;
att1 : string; // <- your problem
That field att1 declared as string that way means that the record contains a pointer to the actual string data (att1 is really a pointer). The compiler manages this pointer and the memory for the associated data, and the string can be any (reasonable) length.
A quick fix for you would be to declare att1 something like string[64], for example: a string which can be at maximum 64 chars long. That would eliminate the pointer and use the memory of the record (the att1 field itself, which now is a special static array) as buffer for string characters. Declaring the maximum length of the string, of course, can be slightly dangerous: if you try to assign the string a string too long, it will be truncated.
To be really complete: it depends on the compiler; some have a switch to make your declaration "string" usable, making it an alias for "string[255]". This is not the default though. Consider also that using string[...] is faster and wastes memory.
You have a few mistakes.
MyArray is a dynamic array, a reference type (a pointer), so SizeOf(MyArray) is the size of a pointer, not the size of the array. To get the length of the array, use Length(MyArray).
But the bigger problem is saving long strings (AnsiStrings -- the usual type to which string maps --, WideStrings, UnicodeStrings). These are reference types too, so you can't just save them together with the record. You will have to save the parts of the record one by one, and for strings, you will have to use a function like:
procedure SaveStr(var F: File; const S: AnsiString);
var
Actual: Integer;
Len: Integer;
begin
Len := Length(S);
BlockWrite(F, Len, SizeOf(Len), Actual);
if Len > 0 then
begin
BlockWrite(F, S[1], Len * SizeOf(AnsiChar), Actual);
end;
end;
Of course you should normally check Actual and do appropriate error handling, but I left that out, for simplicity.
Reading back is similar: first read the length, then use SetLength to set the string to that size and then read the rest.
So now you do something like:
Len := Length(MyArray);
BlockWrite(F, Len, SizeOf(Len), Actual);
for I := Low(MyArray) to High(MyArray) do
begin
BlockWrite(F, MyArray[I].ID, SizeOf(Integer), Actual);
SaveStr(F, MyArray[I].att1);
SaveStr(F, MyArray[I].att2);
BlockWrite(F, MyArray[I].Value, SizeOf(Integer), Actual);
end;
// etc...
Note that I can't currently test the code, so it may have some little errors. I'll try this later on, when I have access to a compiler, if that is necessary.
Update
As Marco van de Voort commented, you may have to do:
rewrite(f, 1);
instead of a simple
rewrite(f);
But as I replied to him, if you can, use streams. They are easier to use (IMO) and provide a more consistent interface, no matter to what exactly you try to write or read. There are streams for many different kinds of I/O, and all derive from (and are thus compatible with) the same basic abstract TStream class.
I have an application which can log a stacktrace, which can be later used for debugging.
On Windows, I've gotten by using the excellent JCLDebug unit provided by the JEDI project.
Now that my application is running on OSX, I've hit a bit of a hitch - I don't know how to obtain the correct stacktrace when an exception occurs.
I have got the basics down -
1) I can get a stacktrace using 'backtrace' (found in libSystem.dylib)
2) The resulting backtrace can be converted into line numbers using the .map file provided by Delphi's linker
The issue I'm left with is - I don't know where to call backtrace from. I know that Delphi uses Mach exceptions (on a separate thread), and that I cannot use posix signals, but that's all that I've managed to sort out.
I can get a backtrace in the 'try...except' block, but unfortunately, by that point the stack has already wound down.
How can I install a proper exception logger which will run right after the exception occurs?
Update:
As per 'Honza R's suggestion, I've taken a look at the 'GetExceptionStackInfoProc' procedure.
This function does get me 'inside' of the exception handling process, but unfortunately leaves me with some of the same issues I had previously.
First of all - on desktop platforms, this function 'GetExceptionStackInfoProc' is just a function pointer, which you can assign with your own exception info handler. So out of the box, Delphi doesn't provide any stack information provider.
If I assign a function to 'GetExceptionStackInfoProc' and then run a 'backtrace' inside of it, I receive a stacktrace, but that trace is relative to the exception handler, not the thread which caused the exception.
'GetExceptionStackInfoProc' does contain a pointer to a 'TExceptionRecord', but there's very limited documentation available on this.
I might be going beyond my depth, but how can I get a stacktrace from the correct thread? Would it be possible for me to inject my own 'backtrace' function into the exception handler and then return to the standard exception handler from there?
Update 2
Some more details. One thing to clear up - this question is about exceptions that are handled by MACH messages, not software exceptions that are handled entirely within the RTL.
Embarcadero has laid out some comments along with these functions -
System.Internal.MachExceptions.pas -> catch_exception_raise_state_identity
{
Now we set up the thread state for the faulting thread so that when we
return, control will be passed to the exception dispatcher on that thread,
and this POSIX thread will continue watching for Mach exception messages.
See the documentation at <code>DispatchMachException()</code> for more
detail on the parameters loaded in EAX, EDX, and ECX.
}
System.Internal.ExcUtils.pas -> SignalConverter
{
Here's the tricky part. We arrived here directly by virtue of our
signal handler tweaking the execution context with our address. That
means there's no return address on the stack. The unwinder needs to
have a return address so that it can unwind past this function when
we raise the Delphi exception. We will use the faulting instruction
pointer as a fake return address. Because of the fencepost conditions
in the Delphi unwinder, we need to have an address that is strictly
greater than the actual faulting instruction, so we increment that
address by one. This may be in the middle of an instruction, but we
don't care, because we will never be returning to that address.
Finally, the way that we get this address onto the stack is important.
The compiler will generate unwind information for SignalConverter that
will attempt to undo any stack modifications that are made by this
function when unwinding past it. In this particular case, we don't want
that to happen, so we use some assembly language tricks to get around
the compiler noticing the stack modification.
}
Which seem to be responsible for the issue I'm having.
When I do a stacktrace after this exception system has handed control over to the RTL, it looks like this - (bearing in mind, the stack unwinder has been superseded by a backtrace routine. The backtrace will hand control over to the unwinder once it is completed)
0: MyExceptionBacktracer
1: initunwinder in System.pas
2: RaiseSignalException in System.Internal.ExcUtils.pas
Since RaiseSignalException is called by SignalConverter, I'm led to believe that the backtrace function provided by libc is not compatible with the modifications made to the stack. So, it's incapable of reading the stack beyond that point, but the stack is still present underneath.
Does anyone know what to do about that (or whether my hypothesis is correct)?
Update 3
I've finally managed to get proper stacktraces on OSX. Huge thanks to both Honza and Sebastian. By combining both of their techniques, I found something that works.
For anyone else who could benefit from this, here's the basic source. Bear in mind that I'm not quite sure if it's 100% correct, if you can suggest improvements, go ahead. This technique hooks onto an exception right before Delphi unwinds the stack on the faulting thread, and compensates for any stack frame corruption that might have taken place beforehand.
unit MyExceptionHandler;
interface
implementation
uses
SysUtils;
var
PrevRaiseException: function(Exc: Pointer): LongBool; cdecl;
function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin : NativeUInt;
begin
SPMin:=base;
Result:=0;
while (size > 0) and (base >= SPMin) and (base <> 0) do begin
buffer^:=PPointer(base + 4)^;
base:=PNativeInt(base)^;
//uncomment to test stacktrace
//WriteLn(inttohex(NativeUInt(buffer^), 8));
Inc(Result);
Inc(buffer);
Dec(size);
end;
if (size > 0) then buffer^:=nil;
end;
procedure UnInstallExceptionHandler; forward;
var
InRaiseException: Boolean;
function RaiseException(Exc: Pointer): LongBool; cdecl;
var b : NativeUInt;
c : Integer;
buff : array[0..7] of Pointer;
begin
InRaiseException := True;
asm
mov b, ebp
end;
c:=backtrace2(b - $4 {this is the compiler dependent value}, #buff, Length(buff));
//... do whatever you want to do with the stacktrace
Result := PrevRaiseException(Exc);
InRaiseException := False;
end;
procedure InstallExceptionHandler;
var
U: TUnwinder;
begin
GetUnwinder(U);
Assert(Assigned(U.RaiseException));
PrevRaiseException := U.RaiseException;
U.RaiseException := RaiseException;
SetUnwinder(U);
end;
procedure UnInstallExceptionHandler;
var
U: TUnwinder;
begin
GetUnwinder(U);
U.RaiseException := PrevRaiseException;
SetUnwinder(U);
end;
initialization
InstallExceptionHandler;
end.
You can use GetExceptionStackInfoProc, CleanUpStackInfoProc and GetStackInfoStringProc in Exception class you need to save stack trace in GetExceptionStackInfoProc and then retrieve it with GetStackInfoStringProc which will get called by RTL if you use StackTrace property of the Exception. Maybe you could also take look at https://bitbucket.org/shadow_cs/delphi-arm-backtrace which demonstrates this on Android.
To do this properly on Mac OS X the libc backtrace function cannot be used because Delphi will corrupt stack frame when calling the GetExceptionStackInfoProc from Exception.RaisingException. Own implementation must be used that is capable of walking the stack from different base address which can be corrected by hand.
Your GetExceptionStackInfoProc would then look like this (I used XE5 for this example the value added to EBP bellow may differ based on which compiler you use and this example was only tested on Mac OS X, Windows implementation may or may not differ):
var b : NativeUInt;
c : Integer;
buff : array[0..7] of Pointer;
begin
asm
mov b, ebp
end;
c:=backtrace2(b - $14 {this is the compiler dependent value}, #buff, Length(buff));
//... do whatever you want to do with the stacktrace
end;
And the backtrace2 function would look like this (note that stop conditions and other validations are missing in the implementation to ensure that AVs are not caused during stack walking):
function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin : NativeUInt;
begin
SPMin:=base;
Result:=0;
while (size > 0) and (base >= SPMin) and (base <> 0) do begin
buffer^:=PPointer(base + 4)^;
base:=PNativeInt(base)^;
Inc(Result);
Inc(buffer);
Dec(size);
end;
if (size > 0) then buffer^:=nil;
end;
You could hook yourself into the Exception Unwinder. Then you can call backtrace where the exception happens. Here's an example. The unit SBMapFiles is what I use for reading the mapfiles. It is not required to get the exception call stack.
unit MyExceptionHandler;
interface
implementation
uses
Posix.Base, SysUtils, SBMapFiles;
function backtrace(result: PNativeUInt; size: Integer): Integer; cdecl; external libc name '_backtrace';
function _NSGetExecutablePath(buf: PAnsiChar; BufSize: PCardinal): Integer; cdecl; external libc name '__NSGetExecutablePath';
var
PrevRaiseException: function(Exc: Pointer): LongBool; cdecl;
MapFile: TSBMapFile;
const
MaxDepth = 20;
SkipFrames = 3;
procedure ShowCurrentStack;
var
StackLog: PNativeUInt; //array[0..10] of Pointer;
Cnt: Integer;
I: Integer;
begin
{$POINTERMATH ON}
GetMem(StackLog, SizeOf(Pointer) * MaxDepth);
try
Cnt := backtrace(StackLog, MaxDepth);
for I := SkipFrames to Cnt - 1 do
begin
if StackLog[I] = $BE00EF00 then
begin
WriteLn('---');
Break;
end;
WriteLn(IntToHex(StackLog[I], 8), ' ', MapFile.GetFunctionName(StackLog[I]));
end;
finally
FreeMem(StackLog);
end;
{$POINTERMATH OFF}
end;
procedure InstallExceptionHandler; forward;
procedure UnInstallExceptionHandler; forward;
var
InRaiseException: Boolean;
function RaiseException(Exc: Pointer): LongBool; cdecl;
begin
InRaiseException := True;
ShowCurrentStack;
Result := PrevRaiseException(Exc);
InRaiseException := False;
end;
procedure InstallExceptionHandler;
var
U: TUnwinder;
begin
GetUnwinder(U);
Assert(Assigned(U.RaiseException));
PrevRaiseException := U.RaiseException;
U.RaiseException := RaiseException;
SetUnwinder(U);
end;
procedure UnInstallExceptionHandler;
var
U: TUnwinder;
begin
GetUnwinder(U);
U.RaiseException := PrevRaiseException;
SetUnwinder(U);
end;
procedure LoadMapFile;
var
FileName: array[0..255] of AnsiChar;
Len: Integer;
begin
if MapFile = nil then
begin
MapFile := TSBMapFile.Create;
Len := Length(FileName);
_NSGetExecutablePath(#FileName[0], #Len);
if FileExists(ChangeFileExt(FileName, '.map')) then
MapFile.LoadFromFile(ChangeFileExt(FileName, '.map'));
end;
end;
initialization
LoadMapFile;
InstallExceptionHandler;
end.
I have a room with 60 computers/devices (40 computers and 20 oscilloscopes Windows CE based) and I would like to know which and every one is alive using ping. First I wrote a standard ping (see here Delphi Indy Ping Error 10040), which is working fine now but takes ages when most computers are offline.
So what I am trying to do is to write a MultiThread Ping but I am quite struggling with it. I have seen only very few examples over the internet and no one was matching my needs, that's why I try to write it myself.
I use XE2 and Indy 10 and the form is only constitued of a memo and a button.
unit Main;
interface
uses
Winapi.Windows, System.SysUtils, System.Classes, Vcl.Forms,
IdIcmpClient, IdGlobal, Vcl.StdCtrls, Vcl.Controls;
type
TMainForm = class(TForm)
Memo1: TMemo;
ButtonStartPing: TButton;
procedure ButtonStartPingClick(Sender: TObject);
private
{ Private declarations }
public
{ Public declarations }
end;
type
TMyPingThread = class(TThread)
private
fIndex : integer;
fIdIcmpClient: TIdIcmpClient;
procedure doOnPingReply;
protected
procedure Execute; override;
public
constructor Create(index: integer);
end;
var
MainForm: TMainForm;
ThreadCOunt : integer;
implementation
{$R *.dfm}
constructor TMyPingThread.Create(index: integer);
begin
inherited Create(false);
fIndex := index;
fIdIcmpClient := TIdIcmpClient.Create(nil);
fIdIcmpClient.ReceiveTimeout := 200;
fIdIcmpClient.PacketSize := 24;
fIdIcmpClient.Protocol := 1;
fIdIcmpClient.IPVersion := Id_IPv4;
//first computer is at adresse 211
fIdIcmpClient.Host := '128.178.26.'+inttostr(211+index-1);
self.FreeOnTerminate := true;
end;
procedure TMyPingThread.doOnPingReply;
begin
MainForm.Memo1.lines.add(inttostr(findex)+' '+fIdIcmpClient.ReplyStatus.Msg);
dec(ThreadCount);
if ThreadCount = 0 then
MainForm.Memo1.lines.add('--- End ---');
end;
procedure TMyPingThread.Execute;
begin
inherited;
try
fIdIcmpClient.Ping('',findex);
except
end;
while not Terminated do
begin
if fIdIcmpClient.ReplyStatus.SequenceId = findex then Terminate;
end;
Synchronize(doOnPingReply);
fIdIcmpClient.Free;
end;
procedure TMainForm.ButtonStartPingClick(Sender: TObject);
var
i: integer;
myPing : TMyPingThread;
begin
Memo1.Lines.Clear;
ThreadCount := 0;
for i := 1 to 40 do
begin
inc(ThreadCount);
myPing := TMyPingThread.Create(i);
//sleep(10);
end;
end;
end.
My problem is that it "seems" to work when I uncomment the "sleep(10)", and "seems" not to be working without it. This for sure means I am missing a point in the threading I have written.
In other words. When Sleep(10) is in the code. Every time I clicked the button to get to check the connections the result was correct.
Without the sleep(10), it is working "most" of the time but some times the result is wrong giving me a ping echo on offline computers and no ping echo on online computer, as is the ping reply was not assigned to the correct thread.
Any comment or help is welcome.
----- EDIT / IMPORTANT -----
As a general follow up of this question, #Darian Miller started a Google Code project here https://code.google.com/p/delphi-stackoverflow/ which is a working basis. I mark his answer as the "accepted answer" but users should refer to this open source project (all the credit belongs to him) as it will surely be extended and updated in the future.
The root problem is that pings are connectionless traffic. If you have multiple TIdIcmpClient objects pinging the network at the same time, one TIdIcmpClient instance can receive a reply that actually belongs to another TIdIcmpClient instance. You are trying to account for that in your thread loop, by checking SequenceId values, but you are not taking into account that TIdIcmpClient already does that same check internally. It reads network replies in a loop until it receives the reply it is expecting, or until the ReceiveTimeout occurs. If it receives a reply it is not expecting, it simply discards that reply. So, if one TIdIcmpClient instance discards a reply that another TIdIcmpClient instance was expecting, that reply will not get processed by your code, and that other TIdIcmpClient will likely receive another TIdIcmpClient's reply instead, and so on. By adding the Sleep(), you are decreasing (but not eliminating) the chances that pings will overlap each other.
For what you are attempting to do, you won't be able to use TIdIcmpClient as-is to have multiple pings running in parallel, sorry. It is simply not designed for that. There is no way for it to differentiate reply data the way you need it. You will have to serialize your threads so only one thread can call TIdIcmpClient.Ping() at a time.
If serializing the pings is not an option for you, you can try copying portions of TIdIcmpClient's source code into your own code. Have 41 threads running - 40 device threads and 1 response thread. Create a single socket that all of the threads share. Have each device thread prepare and send its individual ping request to the network using that socket. Then have the response thread continuously reading replies from that same socket and routing them back to the appropriate device thread for processing. This is a bit more work, but it will give you the multiple-ping parallelism you are looking for.
If you don't want to go to all that trouble, an alternative is to just use a third-party app that already supports pinging multiple machines at the same time, like FREEPing.
Remy explained the problems... I've wanted to do this in Indy for a while so I posted a possible solution that I just put together to a new Google Code project instead of having a long comment here. It's a first-stab sort of thing, let me know if you have some changes to integrate:
https://code.google.com/p/delphi-vault/
This code has two ways to Ping...multi-threaded clients as in your example, or with a simple callback procedure. Written for Indy10 and later versions of Delphi.
Your code would end up using a TThreadedPing descendant defining a SynchronizedResponse method:
TMyPingThread = class(TThreadedPing)
protected
procedure SynchronizedResponse(const ReplyStatus:TReplyStatus); override;
end;
And to fire off some client threads, the code becomes something like:
procedure TfrmThreadedPingSample.butStartPingClick(Sender: TObject);
begin
TMyPingThread.Create('www.google.com');
TMyPingThread.Create('127.0.0.1');
TMyPingThread.Create('www.shouldnotresolvetoanythingatall.com');
TMyPingThread.Create('127.0.0.1');
TMyPingThread.Create('www.microsoft.com');
TMyPingThread.Create('127.0.0.1');
end;
The threaded response is called in a synchronized method:
procedure TMyPingThread.SynchronizedResponse(const ReplyStatus:TReplyStatus);
begin
frmThreadedPingSample.Memo1.Lines.Add(TPingClient.FormatStandardResponse(ReplyStatus));
end;
I did not try your code, so that is all hypothetical, but i think you messed the threads and got classic race condition. I restate my advice to use AsyncCalls or OmniThreadLibrary - they are much simpler and would save you few attempts at "shooting your own foot".
Threads are made to minimize main-thread load. Thread constructor should do minimal work of remembering parameters. Personally i'd moved idICMP creation into .Execute method. If for any reason it would want to create its internal synchronization objects, like window and message queue or signal or whatever, i'd like it to happen already in a new spawned thread.
There is no sense for "inherited;" in .Execute. Better remove it.
Silencing all exceptions is bad style. You probably have errors - but have no way to know about them. You should propagate them to main thread and display them. OTL and AC help you in that, while for tThread you have to do it manually. How to Handle Exceptions thrown in AsyncCalls function without calling .Sync?
Exception logic is flawed. There is no point to have a loop if exception thrown - if no succesful Ping was set - then why waiting for response ? You loop should go within same try-except frame as issuing ping.
Your doOnPingReply executes AFTER fIdIcmpClient.Free yet accesses fIdIcmpClient's internals. Tried changing .Free for FreeAndNil ?
That is a classic mistake of using dead pointer after freeing it.
The correct approach would be to:
5.1. either free the object in doOnPingReply
5.2. or copy all relevant data from doOnPingReply to TThread's private member vars before calling both Synchronize and idICMP.Free (and only use those vars in doOnPingReply )
5.3. only do fIdIcmpClient.Free inside TMyThread.BeforeDestruction or TMyThread.Destroy. Afterall, if you chosen to create the object in constructor - then you should free it in the matching language construct - destructor.
Since you do not keep references to the thread objects - that While not Terminated loop seems redundant. Just make usual forever-loop and call break.
The aforementioned loop is CPU-hungry, it is like spin-loop. Please call Sleep(0); or Yield(); inside loop to give other threads better chance to do their work. Don't work agaisnt OS scheduler here - you are not in a speed-critical path, no reason to make spinlock here.
Overall, i consider:
4 and 5 as critical bugs for you
1 and 3 as a potential gotcha maybe influencing or maybe not. You'd better 'play safe' rather than doing risky things and investigating if they would work or not.
2 and 7 - bad style, 2 regarding language and 7 regarding platform
6 either you have plans to extend your app, or you broke YAGNI principle, dunno.
Sticking with complex TThread instead of OTL or AsyncCalls - strategic errors. Don't you put rooks on your runway, use simple tools.
Funny, this is example of the bug that FreeAndNil could expose and make obvious, while FreeAndNil-haters are claiming it "conceals" bugs.
// This is my communication unit witch works well, no need to know its work but your
// ask is in the TPingThread class.
UNIT UComm;
INTERFACE
USES
Windows, Messages, SysUtils, Classes, Graphics, Controls, ExtCtrls, Forms, Dialogs,
StdCtrls,IdIcmpClient, ComCtrls, DB, abcwav, SyncObjs, IdStack, IdException,
IdTCPServer, IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, IdContext,
UDM, UCommon;
TYPE
TNetworkState = (nsNone, nsLAN, nsNoLAN, nsNet, nsNoNet);
TDialerStatus = (dsNone, dsConnected, dsDisconnected, dsNotSync);
{ TBaseThread }
TBaseThread = Class(TThread)
Private
FEvent : THandle;
FEventOwned : Boolean;
Procedure ThreadTerminate(Sender: TObject); Virtual;
Public
Constructor Create(AEventName: String);
Property EventOwned: Boolean Read FEventOwned;
End;
.
.
.
{ TPingThread }
TPingThread = Class(TBaseThread)
Private
FReply : Boolean;
FTimeOut : Integer;
FcmpClient : TIdIcmpClient;
Procedure ReplyEvent(Sender: TComponent; Const AReplyStatus: TReplyStatus);
Protected
Procedure Execute; Override;
Procedure ThreadTerminate(Sender: TObject); Override;
Public
Constructor Create(AHostIP, AEventName: String; ATimeOut: Integer);
Property Reply: Boolean Read FReply;
End;
.
.
.
{ =============================================================================== }
IMPLEMENTATION
{$R *.dfm}
USES
TypInfo, WinSock, IdGlobal, UCounter, UGlobalInstance, URemoteDesktop;
{IdGlobal: For RawToBytes function 10/07/2013 04:18 }
{ TBaseThread }
//---------------------------------------------------------
Constructor TBaseThread.Create(AEventName: String);
Begin
SetLastError(NO_ERROR);
FEvent := CreateEvent(Nil, False, False, PChar(AEventName));
If GetLastError = ERROR_ALREADY_EXISTS
Then Begin
CloseHandle(FEvent);
FEventOwned := False;
End
Else If FEvent <> 0 Then
Begin
FEventOwned := True;
Inherited Create(True);
FreeOnTerminate := True;
OnTerminate := ThreadTerminate;
End;
End;
//---------------------------------------------------------
Procedure TBaseThread.ThreadTerminate(Sender: TObject);
Begin
CloseHandle(FEvent);
End;
{ TLANThread }
.
.
.
{ TPingThread }
//---------------------------------------------------------
Constructor TPingThread.Create(AHostIP: String; AEventName: String; ATimeOut: Integer);
Begin
Inherited Create(AEventName);
If Not EventOwned Then Exit;
FTimeOut := ATimeOut;
FcmpClient := TIdIcmpClient.Create(Nil);
With FcmpClient Do
Begin
Host := AHostIP;
ReceiveTimeOut := ATimeOut;
OnReply := ReplyEvent;
End;
End;
//---------------------------------------------------------
Procedure TPingThread.Execute;
Begin
Try
FcmpClient.Ping;
FReply := FReply And (WaitForSingleObject(FEvent, FTimeOut) = WAIT_OBJECT_0);
Except
FReply := False;
End;
End;
//---------------------------------------------------------
Procedure TPingThread.ReplyEvent(Sender: TComponent; Const AReplyStatus: TReplyStatus);
Begin
With AReplyStatus Do
FReply := (ReplyStatusType = rsEcho) And (BytesReceived <> 0);
SetEvent(FEvent);
End;
//---------------------------------------------------------
Procedure TPingThread.ThreadTerminate(Sender: TObject);
Begin
FreeAndNil(FcmpClient);
Inherited;
End;
{ TNetThread }
.
.
.
My setting:
OS: Windows 7 SP1 (32 bits)
Ram: 4 Go
Processor: Intel Pentium D 3.00 GHz
Delphi XE
My simple test:
I performed a test running the following program:
program TestAssign;
{$APPTYPE CONSOLE}
uses
SysUtils,
Diagnostics;
type
TTestClazz = class
private
FIntProp: Integer;
FStringProp: string;
protected
procedure SetIntProp(const Value: Integer);
procedure SetStringProp(const Value: string);
public
property IntProp: Integer read FIntProp write SetIntProp;
property StringProp: string read FStringProp write SetStringProp;
end;
{ TTestClazz }
procedure TTestClazz.SetIntProp(const Value: Integer);
begin
if FIntProp <> Value then
FIntProp := Value;
end;
procedure TTestClazz.SetStringProp(const Value: string);
begin
if FStringProp <> Value then
FStringProp := Value;
end;
var
i, j: Integer;
stopw1, stopw2 : TStopwatch;
TestObj: TTestClazz;
begin
ReportMemoryLeaksOnShutdown := True;
//
try
TestObj := TTestClazz.Create;
//
try
j := 10000;
while j <= 100000 do
begin
///
/// assignement
///
stopw1 := TStopwatch.StartNew;
for i := 0 to j do
begin
TestObj.FIntProp := 666;
TestObj.FStringProp := 'Hello';
end;
stopw1.Stop;
///
/// property assignement using Setter
///
stopw2 := TStopwatch.StartNew;
for i := 0 to j do
begin
TestObj.IntProp := 666;
TestObj.StringProp := 'Hello';
end;
stopw2.Stop;
///
/// Log results
///
Writeln(Format('Ellapsed time for %6.d loops: %5.d %5.d', [j, stopw1.ElapsedMilliseconds, stopw2.ElapsedMilliseconds]));
//
Inc(j, 5000);
end;
//
Writeln('');
Write('Press Return to Quit...');
Readln;
finally
TestObj.Free
end
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
end.
My (provisionnal) conclusion:
It seems that:
It's worth using Setter with property under some condition
The overhead of calling a method and performing a conditional test take less time than an assignement.
My question:
Are those findings valid under any other diffrent setting or just localized ones (exception)?
I would make the following observations:
The decision as to whether or not to use a setter should be based on factors like code maintenance, correctness, readability rather than performance.
Your benchmark is wholly unreasonable since the if statements evaluate to False every time. Real world code that sets properties would be likely to modify the properties a reasonable proportion of the time that the setter runs.
I would expect that for many real world examples, the setter would run faster without the equality test. If that test were to evaluate to True every time then clearly the code would be quicker without it.
The integer setter is practically free and in fact the setter is slower than the direct field access.
The time is spent in the string property. Here there is some real performance benefit due to the optimisation of the if test which avoids string assignment code if possible.
The setters would be faster if you inlined them, but not by a significant amount.
My belief is that any real world code would never be able to detect any of these performance differences. In reality the bottleneck will be obtaining the values passed to the setters rather than time spent in the setters.
The main situation where such if protection is valuable is where the property modification is expensive. For example, perhaps it involves sending a Windows message, or hitting a database. For a property backed by a field you can probably take it or leave it.
In the chatter in the comments Premature Optimization wonders why the comparison if FStringProp <> Value is quicker than the assignment FStringProp := Value. I investigated a little further and it wasn't quite as I had originally thought.
It turns out that if FStringProp <> Value is dominated by a call to System._UStrEqual. The two strings passed are not in fact the same reference and so each character has to be compared. However, this code is highly optimised and crucially there are only 5 characters to compare.
The call to FStringProp := Value goes to System._UStrAsg and since Value is a literal with negative reference count, a brand new string has to be made. The Pascal version of the code looks like this:
procedure _UStrAsg(var Dest: UnicodeString; const Source: UnicodeString); // globals (need copy)
var
S, D: Pointer;
P: PStrRec;
Len: LongInt;
begin
S := Pointer(Source);
if S <> nil then
begin
if __StringRefCnt(Source) < 0 then // make copy of string literal
begin
Len := __StringLength(Source);
S := _NewUnicodeString(Len);
Move(Pointer(Source)^, S^, Len * SizeOf(WideChar));
end else
begin
P := PStrRec(PByte(S) - SizeOf(StrRec));
InterlockedIncrement(P.refCnt);
end;
end;
D := Pointer(Dest);
Pointer(Dest) := S;
_UStrClr(D);
end;
The key part of this is the call to _NewUnicodeString which of course calls GetMem. I am not at all surprised that heap allocation is significantly slower than comparison of 5 characters.
Put 'Hello' const into a variable and use it for setting then do a test again
I'm experiencing a memory leak when using WMI from Delphi 7 to query a (remote) pc. The memory leak only occurs on Windows 2003 (and Windows XP 64). Windows 2000 is fine, and so is Windows 2008. I'm wondering if anyone has experienced a similar problem.
The fact that the leak only occurs in certain versions of Windows implies that it might be a Windows issue, but I've been searching the web and haven't been able to locate a hotfix to resolve the issue. Also, it might be a Delphi issue, since a program with similar functionality in C# doesn't seem to have this leak. The latter fact has led me to believe that there might be another, better, way to get the information I need in Delphi without getting a memory leak.
I've included the source to a small program to expose the memory leak below. If the line sObject.Path_ below the { Leak! } comment is executed, the memory leak occurs. If I comment it out, there's no leak. (Obviously, in the "real" program, I do something useful with the result of the sObject.Path_ method call :).)
With a little quick 'n dirty Windows Task Manager profiling on my machine, I found the following:
Before N=100 N=500 N=1000
With sObject.Path_ 3.7M 7.9M 18.2M 31.2M
Without sObject.Path_ 3.7M 5.3M 5.4M 5.3M
I guess my question is: has anyone else encountered this problem? If so, is it indeed a Windows issue, and is there a hotfix? Or (more likely) is my Delphi code broken, and is there a better way to get the information I need?
You'll notice on several occasions, nil is assigned to objects, contrary to the Delphi spirit... These are COM objects that do not inherit from TObject, and have no destructor I can call. By assigning nil to them, Windows's garbage collector cleans them up.
program ConsoleMemoryLeak;
{$APPTYPE CONSOLE}
uses
Variants, ActiveX, WbemScripting_TLB;
const
N = 100;
WMIQuery = 'SELECT * FROM Win32_Process';
Host = 'localhost';
{ Must be empty when scanning localhost }
Username = '';
Password = '';
procedure ProcessObjectSet(WMIObjectSet: ISWbemObjectSet);
var
Enum: IEnumVariant;
tempObj: OleVariant;
Value: Cardinal;
sObject: ISWbemObject;
begin
Enum := (wmiObjectSet._NewEnum) as IEnumVariant;
while (Enum.Next(1, tempObj, Value) = S_OK) do
begin
sObject := IUnknown(tempObj) as SWBemObject;
{ Leak! }
sObject.Path_;
sObject := nil;
tempObj := Unassigned;
end;
Enum := nil;
end;
function ExecuteQuery: ISWbemObjectSet;
var
Locator: ISWbemLocator;
Services: ISWbemServices;
begin
Locator := CoSWbemLocator.Create;
Services := Locator.ConnectServer(Host, 'root\CIMV2',
Username, Password, '', '', 0, nil);
Result := Services.ExecQuery(WMIQuery, 'WQL',
wbemFlagReturnImmediately and wbemFlagForwardOnly, nil);
Services := nil;
Locator := nil;
end;
procedure DoQuery;
var
ObjectSet: ISWbemObjectSet;
begin
CoInitialize(nil);
ObjectSet := ExecuteQuery;
ProcessObjectSet(ObjectSet);
ObjectSet := nil;
CoUninitialize;
end;
var
i: Integer;
begin
WriteLn('Press Enter to start');
ReadLn;
for i := 1 to N do
DoQuery;
WriteLn('Press Enter to end');
ReadLn;
end.
I can reproduce the behaviour, the code leaks memory on Windows XP 64 and does not on Windows XP. Interestingly this occurs only if the Path_ property is read, reading Properties_ or Security_ with the same code does not leak any memory. A Windows-version-specific problem in WMI looks like the most probable cause of this. My system is up-to-date AFAIK, so there probably isn't a hotfix for this either.
I'd like to comment on your resetting all variant and interface variables, though. You write
You'll notice on several occasions, nil is assigned to objects, contrary to the Delphi spirit... These are COM objects that do not inherit from TObject, and have no destructor I can call. By assigning nil to them, Windows's garbage collector cleans them up.
This is not true, and consequently there is no need to set the variables to nil and Unassigned. Windows does not have a garbage collector, what you are dealing with are reference-counted objects, which are immediately destroyed once the reference count reaches 0. The Delphi compiler does insert the necessary calls to increment and decrement the reference count as necessary. Your assignments to nil and Unassigned decrement the reference count, and free the object when it reaches 0.
A new assignment to a variable, or the exiting of the procedure take care of this as well, so additional assignments are (albeit not wrong) superfluous and decrease the clarity of the code. The following code is completely equivalent and does not leak any additional memory:
procedure ProcessObjectSet(WMIObjectSet: ISWbemObjectSet);
var
Enum: IEnumVariant;
tempObj: OleVariant;
Value: Cardinal;
sObject: ISWbemObject;
begin
Enum := (wmiObjectSet._NewEnum) as IEnumVariant;
while (Enum.Next(1, tempObj, Value) = S_OK) do
begin
sObject := IUnknown(tempObj) as SWBemObject;
{ Leak! }
sObject.Path_;
end;
end;
I'd say one should explicitly reset interfaces only if this does actually free the object (so the current ref count has to be 1) and the destruction itself should really happen exactly at this point. Examples for the latter are that a large chunk of memory can be freed, or that a file needs to be closed or a synchronization object to be released.
you should store the return value of
sObject.Path_;
in a variable and make it SWbemObjectPath. This is necessary to make the reference counting right.