I have class with 8 byte fields. I can put them in record and put record in 1 class field (record is packed and efficient).
I have 100000 objects of this class. So need efficient variant- which is efficient, less memory: byte fields or record field?
It's FPC 2.6.
While nobody of FPC gurus have no time to answer, lets explore this question. Here is some code for this:
program Project1;
type
TByteArray = packed array[Low(Word)..High(Word)] of Byte;
PByteArray = ^TByteArray;
TMyRec = packed record
f1, f2, f3, f4, f5, f6, f7, f8: Byte;
end;
{ TMyClass1 }
TMyClass1 = class
mark1: Byte;
f1, f2, f3, f4, f5, f6, f7, f8: Byte;
mark2: Byte;
r: TMyRec;
mark3: Byte;
constructor Create;
procedure ShowMe; // Dump of the object's data
end;
{ TMyClass1 }
constructor TMyClass1.Create;
begin
mark1 := 66;
mark2 := 77;
mark3 := 88;
end;
procedure TMyClass1.ShowMe;
var
data: PByteArray;
i: Word;
begin
data := Pointer(Self);
for i := 0 to 15 + 4 + 3 do // 4 - some unknown data at the beginning of the class, 3 - marks
Writeln(data^[i]);
end;
var
test: TMyClass1;
begin
test := TMyClass1.Create;
try
test.ShowMe;
Readln;
finally
test.Free;
end;
end.
And output is:
0
192
64
0
66 <- Data starts, simple fields
0
0
0
0
0
0
0
0
77 <- Second data portion, record
0
0
0
0
0
0
0
0
88 <- Data ends
As we can see in both cases 8 fields takes 8 bytes.
Good luck.
Or, if you think that packing is really important, simply declare the class packed:
TMyClass1 = packed class
mark1: Byte;
f1, f2, f3, f4, f5, f6, f7, f8: Byte;
mark2: Byte;
r: TMyRec;
mark3: Byte;
constructor Create;
procedure ShowMe; // Dump of the object's data
end;
Classes will always have some overhead, but that is not necessarily the packing of the fields, but hidden administration. And over time that overhead has gotten bigger.
But 100000 records is peanuts. If you waste a few bytes, it is like 10MB or so, a fraction of the memory of the cheapest PC available nowadays.
Related
i'm trying to sort TStringList of integers from a text file with Insertion and Selection Sort .Selection Sort works ok , but the Insertion Sort doesnt work with my code . Can someone tell me where i'm wrong ? My 'numbers.txt' has 5000 lines of numbers. Thanks in advance
UPDATE : I have edited my code a bit , it works now with Insertion-Sort but it sorts just 4 indexes of integer as on the image
var
i, Position, n: integer;
Value: string;
begin
n := Items.Count;
for i := 1 to n - 1 do
begin
Value := Items[i];
Position := i-1;
while (Position >0) and (Items[Position]>Value) do
begin
Items[Position+1]:= Items[Position] ;
Position := Position -1 ;
end;
Items[Position+1] := Value;
end;
end;
Your data in the image is sorting exactly as it should, because you're sorting on string values, and based on the comparison you're making the order is perfect. "1143" falls exactly between the string values "11413" and "11443", because the comparison is made character by character out to the length of the shortest of the values. "1141" < "1143" < "1144", based on the first four characters of each string.
If you want an actual integer sort, then you need to convert the two values to integer before comparing them. Something like this should work (note I did not test your overall sort logic - I just used values that demonstrate the concept):
program Project1;
{$APPTYPE CONSOLE}
{$R *.res}
uses
System.SysUtils, System.Classes;
var
i, Position, n: integer;
Value: integer;
Items: TStringList;
begin
Items := TStringList.Create;
try
Items.DelimitedText := '1116,11170,11178,11206,1122,11221,11228';
n := Items.Count;
for i := 1 to n - 1 do
begin
Value := StrToInt(Items[i]);
Position := i - 1;
while (Position > 0) and (StrToInt(Items[Position]) > Value) do
begin
Items[Position + 1]:= Items[Position];
Position := Position - 1 ;
end;
Items[Position+1] := IntToStr(Value);
end;
for i := 0 to Items.Count - 1 do
WriteLn(Items[i]);
finally
Items.Free;
end;
ReadLn;
end.
The output I got from the code above in a console window:
1116
1122
11170
11178
11206
11221
11228
I am working on my school project and I would like to use Dynamic (not static) array. I worked with ObjectPascal, so I am used to some syntax. But now I am programming in the old TurboPascal (I am using Turbo Pascal 7 for Windows).
It doesn't seem to know the ObjectPascal, so I thought, that you Turbo Pascal doesn't know dynamic arrays.
Could anyone tell me, if my theory is right or not? I tried to google, but I was not succesfull.
Basicly I am asking "how is it with dynamic arrays in Turbo Pascal 7" ?
Thank you for all reactions.
As MartynA says, there is no dynamic array type in Turbo Pascal. You need to manually allocate memory using pointers, and be careful if you use rangechecks.
Typically you define an array type
TYPE
TArrayT = array[0.. ((65535-spillbytes) div sizeof(T))-1] of T;
where spillbytes is a constant for a small deduction because you can't use the whole 64k, see what the compiler accepts. (Probably this deduction is for heapmanager structures inside the 64k block)
Then you define a pointer
PArrayT= ^TArrayT;
and a variable to it
var
P : PArrayT;
and you allocate nrelement elements using getmem;
getmem(P,SizeOf(T) * nrelements);
and optionally fill them with zero to initialize them:
fillchar(p^,SizeOf(T) * nrelements,#0);
You can access elements using
p^[index]
to free them, use freemem using the exact opposite of the getmem line.
freemem(P,Sizeof(T)*nrelements);
Which means you have to save the allocated number of elements somewhere. This was fixed/solved in Delphi and FPC.
Also keep in mind that you can't find bugs with rangechecking anymore.
If you want arrays larger than 64k, that is possible, but only with constraints, and it matters more which exact TP target (dos, dos-protected or Windows you use) I advise you to search for the online SWAG archive that has many examples. And of course I would recommend to go to FreePascal/Lazarus too where you can simply do:
var x : array of t;
begin
setlength(x,1000000);
and be done with it without additional lines and forget about all of this nonsense.
I'm using Turbo Pascal 5.5 and to create a dynamic array, perhaps the trick is to declare an array with zero dimension as follows:
dArray = array [0..0] of integer;
And then declare a pointer to that array:
pArray = ^dArray ;
And finally, create a pointer variable:
ArrayPtr : pArray;
You can now reference the pointer variable ArrayPtr as follows:
ArrayPtr^[i]; { The index 'i' is of type integer}
See the complete example below:
{
Title: dynarr.pas
A simple Pascal program demonstrating dynamic array.
Compiled and tested with Turbo Pascal 5.5.
}
program dynamic_array;
{Main Program starts here}
type
dArray = array [0..0] of integer;
pArray = ^dArray ;
var
i : integer;
ArrayPtr : pArray;
begin
for i := 0 to 9 do { In this case, array index starts at 0 instead of 1. }
ArrayPtr^[i] := i + 1;
writeln('The Dynamic Array now contains the following:');
writeln;
for i := 0 to 9 do
writeln(ArrayPtr^[i]);
end.
In this example, we have declared the array as:
array[0..0] of integer;
Therefore, the index starts at 0 and if we have n elements, the last element is at index n-1 which is similar to array indexing in C/C++.
Regular Pascal arrays start at 1 but for this case, it starts at 0.
unit Vector;
interface
const MaxVector = 8000;
// 64 k div SizeOf(float); number of float-values that fit in 64 K of stack
VectorError: boolean = False;
// toggle if error occurs. Calling routine can handle or abort
type
VectorStruc = record
Length: word;
Data: array [1..MaxVector] of float;
end;
VectorTyp = ^VectorStruc;
procedure CreateVector(var Vec: VectorTyp; Length: word; Value: float);
{ Generates a vector of length Length and sets all elements to Value }
procedure DestroyVector(var Vec: VectorTyp);
{ release memory occupied by vector }
procedure SetVectorElement(var Vec: VectorTyp; n: word; c: float);
function GetVectorElement(const Vec: VectorTyp; n: word): float;
implementation
var ch: char;
function WriteErrorMessage(Text: string): char;
begin
Write(Text);
Read(WriteErrorMessage);
VectorError := True; // toggle the error marker
end;
procedure CreateVector(var Vec: VectorTyp; Length: word; Value: float);
var
i: word;
begin
try
GetMem(Vec, Length * SizeOf(float) + SizeOf(word) + 6);
except
ch := WriteErrorMessage(' Not enough memory to create vector');
exit;
end;
Vec^.Length := Length;
for i := 1 to Length do
Vec^.Data[i] := Value;
end;
procedure DestroyVector(var Vec: VectorTyp);
var
x: word;
begin
x := Vec^.Length * SizeOf(float) + SizeOf(word) + 6;
FreeMem(Vec, x);
end;
function VectorLength(const Vec: VectorTyp): word;
begin
VectorLength := Vec^.Length;
end;
function GetVectorElement(const Vec: VectorTyp; n: word): float;
var
s1, s2: string;
begin
if (n <= VectorLength(Vec)) then
GetVectorElement := Vec^.Data[n]
else
begin
Str(n: 4, s1);
Str(VectorLength(Vec): 4, s2);
ch := WriteErrorMessage(' Attempt to read non-existent vector element No ' +
s1 + ' of ' + s2);
end;
end;
procedure SetVectorElement(var Vec: VectorTyp; n: word; C: float);
begin
if (n <= VectorLength(Vec)) then
Vec^.Data[n] := C
else
ch := WriteErrorMessage(' Attempt to write to non-existent vector element');
end;
end.
As long as your data fit on the stack, i.e., are smaller than 64 kB, the task is relatively simple. The only thing I don't know is where the 6 bit of extra size go, they are required, however.
I have two types and two arrays of that types in file.ads
type Ebit is mod 2**8;
type Sbit is mod 2**6;
type Data_Type is array (Positive range <>) of Ebit;
type Changed_Data_Type is array (Positive range <>) of Sbit;
and function:
function ChangeDataType (D : in Data_Type) return Changed_Data_Type
with
Pre => D'Length rem 3 = 0 and D'Last < Positive'Last / 4,
Post => ChangeDataType'Result'Length = 4 * (D'Length / 3)
Ok i can understand all of this.
For example we have arrays of:
65, 66, 65, 65, 66, 65 in 8bit values function should give to us 16, 20, 9, 1, 16, 20, 9, 1 in 6bit values.
I dont know how i can build a 6bit table from 8 bit table.
My idea of sollutions is for example taking bit by bit from type:
fill all bites in 6bit type to 0 (propably default)
if first bit (2**1) is 1 set bit (2**1) in 6bit type to 1;
and do some iterations
But i dont know how to do this, always is a problem with types. Is this good idea or i can do this with easier way? I spend last nigt to try write this but without success.
Edit:
I wrote some code, its working but i have problem with array initialization.
function ChangeDataType (D: in Data_Type) return Changed_Data_Type
is
length: Natural := (4*(D'Length / 3));
ER: Changed_Data_type(length);
Temp: Ebit;
Temp1: Ebit;
Temp2: Ebit;
Actual: Ebit;
n: Natural;
k: Natural;
begin
n := 0;
k := 0;
Temp := 2#00000000#;
Temp1 := 2#00000000#;
Temp2 := 2#00000000#;
Array_loop:
for k in D'Range loop
case n is
when 0 =>
Actual := D(k);
Temp1 := Actual / 2**2;
ER(k) := Sbit(Temp1);
Temp := Actual * ( 2**4);
n := 2;
when 2 =>
Actual := D(k);
Temp1 := Actual / 2**4;
Temp2 := Temp1 or Temp;
ER(k) := Sbit(Temp2);
Temp := Actual * ( 2**2);
n := 4;
when 4 =>
Actual := D(k);
Temp1 := Actual / 2**6;
Temp2 := Temp1 or Temp;
ER(k) := Sbit(Temp2);
n := 6;
when 6 =>
Temp1 := Actual * ( 2**2);
Temp2 := Actual / 2**2;
ER(k) := Sbit(Temp2);
n := 0;
when others =>
n := 0;
end case;
end loop Array_Loop;
return ER;
end;
IF I understand what you're asking... it's that you want to re-pack the same 8-bit data into 6-bit values such that the "leftover" bits of the first EBit become the first bits (highest or lowest?) of the second Sbit.
One way you can do this - at least for fixed size arrays, e.g. your 6 words * 8 bits, 8 words * 6 bits example, is by specifying the exact layout in memory for each array type, using packing, and representation aspects (or pragmas, before Ada-2012) which are nicely described here.
I haven't tested the following, but it may serve as a starting point.
type Ebit is mod 2**8;
type Sbit is mod 2**6;
for Ebit'Size use 8;
for Sbit'Size use 6;
type Data_Type is array (1 .. 6) of Ebit
with Alignment => 0; -- this should pack tightly
type Changed_Data_Type is array (1 .. 8) of Sbit
with Alignment => 0;
Then you can instantiate the generic Unchecked_Conversion function with the two array types, and use that function to convert from one array to the other.
with Ada.Unchecked_Conversion;
function Change_Type is new Ada.Unchecked_Conversion(Data_Type, Changed_Data_Type);
declare
Packed_Bytes : Changed_Data_Type := Change_Type(Original_Bytes);
begin ...
In terms of code generated, it's not slow, because Unchecked_Conversion doesn't do anything, except tell the compile-time type checking to look the other way.
I view Unchecked_Conversion like the "I meant to do that" look my cat gives me after falling off the windowledge. Again...
Alternatively, if you wish to avoid copying, you can declare Original_Bytes as aliased, and use a similar trick with access types and Unchecked_Access to overlay both arrays on the same memory (like a Union in C). I think this is what DarkestKhan calls "array overlays" in a comment below. See also section 3 of this rather dated page which describes the technique further. It notes the overlaid variable must not only be declared aliased but also volatile so that accesses to one view aren't optimised into registers, but reflect any changes made via the other view. Another approach to overlays is in the Ada Wikibook here.
Now this may be vulnerable to endian-ness considerations, i.e. it may work on some platforms but not others. The second reference above gives an example of a record with exact bit-alignment of its members : we can at least take the Bit_Order aspect, as in with Alignment => 0, Bit_Order => Low_Order_First; for the arrays above...
-- code stolen from "Rationale" ... see link above p.11
type RR is record
Code: Opcode;
R1: Register;
R2: Register;
end record
with Alignment => 2, Bit_Order => High_Order_First;
for RR use record
Code at 0 range 0 .. 7;
R1 at 1 range 0 .. 3;
R2 at 1 range 4 .. 7;
end record;
One thing that's not clear to me is if there's a formulaic way to specify the exact layout of each element in an array, as is done in a record here - or even if there's a potential need to. If necessary, one workaround would be to replace the arrays above with records. But I'd love to see a better answer if there is one.
I have this function called IntToStrLen which converts an integer to a string of a padded size. For example, an integer value of 42 with a size of 4 would result in a string __42 (padded with a given character, space by default).
The problem is when this function is used in bulk (for example 1,000,000 times in a loop), it adds extra weight onto the loop. The loop I'm using this in, without this function, takes about 20 seconds, but with this function, I'm still waiting as of right now for this function to complete, after about 5 minutes.
How can I speed up the following function?
function IntToStrLen(const Value: Integer; const Len: Integer;
const Fill: String = ' '): String;
var
T: String;
begin
Result:= IntToStr(Value); //convert result
if Length(Result) > Len then
Result:= Copy(Result, 1, Len) //forcefully truncate
else if Length(Result) < Len then begin
T:= '';
while Length(T) < (Len - Length(Result)) do //fill space with character
T:= T + Fill;
Result:= T + Result; //return combination
end;
end;
The absolute number one change that you can make here is to avoid heap allocations. Your code presently has multiple heap allocations. Your goal here is to write a function with zero heap allocations.
That means you need the caller to allocate and provide the buffer. Typically they will do so with a stack allocated buffer, which hence costs nothing to allocate (or deallocate). If the caller really needs a string, which is always allocated on the heap, then the caller can wrap your function with a call to SetString.
The function prototype might look like this:
procedure IntToStrLen(const Value, Len: Integer; var Buffer: array of Char;
const Fill: Char = ' ');
The first point to stress here is that Fill must be a Char. Your use of string is inefficient and allows the caller to call a fill "character" with length not equal to one. Doing so would of course break your function because it would return a value with length not equal to Len.
Do note also that the implementation must not call IntToStr because that involves heap allocation. So you need to write your own heap allocation free integer to decimal text conversion code because, astonishingly, the RTL does not offer such functionality. When I do this I use code like so:
procedure DivMod(Dividend, Divisor: Cardinal;
out Quotient, Remainder: Cardinal);
{$IFDEF CPUX86}
asm
PUSH EBX
MOV EBX,EDX
XOR EDX,EDX
DIV EBX
MOV [ECX],EAX
MOV EBX,Remainder
MOV [EBX],EDX
POP EBX
end;
{$ELSE IF Defined(CPUX64)}
asm
.NOFRAME
MOV EAX,ECX
MOV ECX,EDX
XOR EDX,EDX
DIV ECX
MOV [R8],EAX
MOV [R9],EDX
end;
{$ELSE}
{$Message Error 'Unrecognised platform.'}
{$ENDIF}
function CopyIntegerToCharBuffer(const Value: Integer;
var Buffer: array of Char): Integer;
var
i, j: Integer;
val, remainder: Cardinal;
negative: Boolean;
tmp: array [0..15] of Char;
begin
negative := Value<0;
val := abs(Value);
Result := 0;
repeat
DivMod(val, 10, val, remainder);
tmp[Result] := Chr(remainder + ord('0'));
inc(Result);
until val=0;
if negative then begin
tmp[Result] := '-';
inc(Result);
end;
Assert(Result<=Length(Buffer));
i := 0;
j := Result-1;
while i<Result do begin
Buffer[i] := tmp[j];
inc(i);
dec(j);
end;
end;
Now you can make decimal text representations of integers without touching the heap. From there it's a short way to your function.
procedure IntToStrLen(const Value, Len: Integer; var Buffer: array of Char;
const Fill: Char = ' ');
var
tmp: array [0..15] of Char;
i, N: Integer;
begin
Assert(Length(Buffer)>=Len);
N := CopyIntegerToCharBuffer(Value, tmp);
if N>=Len then begin
Move(tmp, Buffer, SizeOf(Char)*Len);
end else begin
for i := 0 to Len-N-1 do begin
Buffer[i] := Fill;
end;
Move(tmp, Buffer[Len-N], SizeOf(Char)*N);
end;
end;
At this point you'll have gained the bulk of the available performance benefits. From here on in you will be into diminishing returns. You could micro-optimise CopyIntegerToCharBuffer as is done in SysUtils._IntToStr32 for instance. Beyond that I'm sure the implementation of IntToStrLen could be optimised with judicious use of assembler. But such optimisations will yield nothing like the benefit you have gained so far from avoiding the heap.
Of course, all this assumes that you have correctly identified your performance bottleneck. It's all too easy to assume that you know where the performance bottleneck is by statically analysing the code. Unless you've actually profiled it carefully expect to find that your intuition is a poor judge of where to invest optimisation effort.
Try this variant
(and you don't treat negative numbers)
function IntToStrLen(const Value: Integer; const Len: Integer;
const Fill: Char = ' '): String;
var
T: String;
begin
Result:= IntToStr(Value);
if Length(Result) > Len then
SetLength(Result, Len) //forcefully truncate
else if Length(Result) < Len then
Result := StringOfChar(Fill, Len - Length(Result)) + Result;
end;
P.S. So strange truncation (2014=>20) - is what you really want?
How to implement the following algorithm in Delphi (Object Pascal) in a way that:
Each alpha numeric item will be a single object (in my case a reference to file strings).
to be possible to enumerate (to output) each pick state/combination.
imagine the columns of an abacus; all have the same size (according to its base). I need columns with different sizes. (in my case, sets of file strings with different sizes)
Last EDIT: Please, see Python intertools implementation.
Similar algorithms in other languages: c#, ruby, java, php
ALGORITHM
Consider the following sets and its members:
S1 = {a1, a2, a3, a4, a5}
S2 = {b1, b2, b3, b4}
S3 = {c1, c2, c3, c4, c5}
Pick the first member of each set (P = Pick States):
P1 = {a1, b1, c1}
Then, increment the first until its limit:
P2 = {a2, b1, c1} P3 = {a3, b1, c1} P4 = {a4, b1, c1} P5 = {a5, b1,
c1}
Then, reset the first Set, and increment 'one' the second set;
P6 = {a1, b2, c1}
Increment the first set again... and so on... reseting the first and the second set for each 'plus one' on third set.
P7 = {a2, b2, c1}
Regarding the fundamental principle of counting or principle of multiplication, this algorithm would generate 100 pick states/combinations.
P100 = {a5, b4, c5}
You are counting.
Each bead is a digit and you're working in base 5 in your example, since each bead can have one of 5 positions.
To determine what position of beads corresponds to a given integer, it is enough to write that integer in the relevant base. Here's how to do in ruby, for 17:
>> 17.to_s(5).rjust(3, '0')
=> "032"
Here I left-padded to 3 beads to be clear where each is, and I'm using a convention that beads begin in position 0, not position 1.
I've reached a solution using recursive and conditional calls. I've created a record with this structure:
TGridKey = record
ID : Integer;
Index : Integer;
Files : TStringList;
end;
TTrialSet = record
theGrid : array of TGridKey;
end;
With this name in a TForm class.
TfRandStimuliSet = class (TForm)
//...
lst1: TListBox;
dlgOpenPic: TOpenPictureDialog;
private
FTrialSet: TTrialSet;
procedure TfRandStimuliSet.SubSetsMountEngine;
//...
end;
Set Length of the Grid Array:
NumComp := 3;
SetLength(FTrialSet.theGrid, NumComp);
for I := Low(FTrialSet.theGrid) to High(FTrialSet.theGrid) do
begin
FTrialSet.theGrid[I].ID := I;
FTrialSet.theGrid[I].Index:= -1;
FTrialSet.theGrid[I].Files := TStringList.Create;
end;
Put some strings in each 'I' grid:
if dlgOpenPic.Execute then
begin
if dlgOpenPic.Files.Count > 0 then
for K := 0 to (dlgOpenPic.Files.Count - 1) do
begin
FTrialSet.theGrid[I].Files.Add(dlgOpenPic.Files.Strings[K]);
end;
dlgOpenPic.Files.Clear;
end;
Then the procedure:
procedure TfRandStimuliSet.SubSetsMountEngine;
var ID: integer; s1 : string;
procedure AddStmFromGrid(Grid, Stm : Integer);
begin
s1 := s1 + ExtractFileName(FTrialSet.theGrid[Grid].Files.Strings[Stm]) + ',';
end;
procedure AddTrialFromIndex; //each trial is the current index's
var I: Integer;
begin
for I:= Low(FTrialSet.theGrid) to High(FTrialSet.theGrid) do
AddStmFromGrid(I,FTrialSet.theGrid[I].Index);
lst1.Items.Add(s1);
s1:= '';
end;
procedure IndexReset(aGrid : Integer);
var i : Integer;
begin
for I := aGrid to (High(FTrialSet.theGrid)) do
FTrialSet.theGrid[I].Index := 0
end;
procedure IndexInc(aGrid : Integer);
begin
AddTrialFromIndex; //Save
Inc(FTrialSet.theGrid[aGrid].Index);
end;
function MoveGrid(var ID:integer): Boolean; //begin from right most, the highest grid
var IDMaxIndex, IDCurIndex, LowID, HighID: Integer;
begin
Result := True;
LowID := Low(FTrialSet.theGrid);
HighID := High(FTrialSet.theGrid);
//Application.ProcessMessages;
if (ID < LowID) then
begin
//ShowMessage('False');
AddTrialFromIndex;
Result := False;
end
else
begin
IDMaxIndex:= FTrialSet.theGrid[ID].Files.Count -1;
IDCurIndex := FTrialSet.theGrid[ID].Index;
if IDCurIndex = IDMaxIndex then
begin
ID := ID - 1;
Result:= MoveGrid(ID);//moveleft
Exit;
end;
if (ID < HighID)
and (IDCurIndex < IDMaxIndex) then
begin
IndexInc(ID); //increment/move donw
IndexReset(ID + 1); //reset everything on the right
MoveGrid(HighID); //move to the most right/next down
Exit;
end;
if (ID = (HighID))
and (IDCurIndex < IDMaxIndex) then
begin
IndexInc(ID); //increment/move down
MoveGrid(ID) //next increment/move down
end;
end;
end;
begin
ID := High(FTrialSet.theGrid);
IndexReset(Low(FTrialSet.theGrid)); //0's for everyone
MoveGrid(ID); //begin from the most right
end;