Binary file error in Lazarus Pascal with custom records - error SIGSEGV - binaryfiles

I don't work with Pascal very often so I apologise if this question is basic. I am working on a binary file program that writes an array of custom made records to a binary file.
Eventually I want it to be able to write multiple arrays of different custom record types to one single binary file.
For that reason I thought I would write an integer first being the number of bytes that the next array will be in total. Then I write the array itself. I can then read the first integer type block - to tell me the size of the next blocks to read in directly to an array.
For example - when writing the binary file I would do something like this:
assignfile(f,MasterFileName);
{$I-}
reset(f,1);
{$I+}
n := IOResult;
if n<> 0 then
begin
{$I-}
rewrite(f);
{$I+}
end;
n:= IOResult;
If n <> 0 then
begin
writeln('Error creating file: ', n);
end
else
begin
SetLength(MyArray, 2);
MyArray[0].ID := 101;
MyArray[0].Att1 := 'Hi';
MyArray[0].Att2 := 'MyArray 0 - Att2';
MyArray[0].Value := 1;
MyArray[1].ID := 102;
MyArray[1].Att1:= 'Hi again';
MyArray[1].Att2:= MyArray 1 - Att2';
MyArray[1].Value:= 5;
SizeOfArray := sizeOf(MyArray);
writeln('Size of character array: ', SizeOfArray);
writeln('Size of integer var: ', sizeof(SizeOfArray));
blockwrite(f,sizeOfArray,sizeof(SizeOfArray),actual);
blockwrite(f,MyArray,SizeOfArray,actual);
Close(f);
Then you could re-read the file with something like this:
Assign(f, MasterFileName);
Reset(f,1);
blockread(f,SizeOfArray,sizeof(SizeOfArray),actual);
blockread(f,MyArray,SizeOfArray,actual);
Close(f);
This has the idea that after these blocks have been read that you can then have a new integer recorded and a new array then saved etc.
It reads the integer parts of the records in but nothing for the strings. The record would be something like this:
TMyType = record
ID : Integer;
att1 : string;
att2 : String;
Value : Integer;
end;
Any help gratefully received!!

TMyType = record
ID : Integer;
att1 : string; // <- your problem
That field att1 declared as string that way means that the record contains a pointer to the actual string data (att1 is really a pointer). The compiler manages this pointer and the memory for the associated data, and the string can be any (reasonable) length.
A quick fix for you would be to declare att1 something like string[64], for example: a string which can be at maximum 64 chars long. That would eliminate the pointer and use the memory of the record (the att1 field itself, which now is a special static array) as buffer for string characters. Declaring the maximum length of the string, of course, can be slightly dangerous: if you try to assign the string a string too long, it will be truncated.
To be really complete: it depends on the compiler; some have a switch to make your declaration "string" usable, making it an alias for "string[255]". This is not the default though. Consider also that using string[...] is faster and wastes memory.

You have a few mistakes.
MyArray is a dynamic array, a reference type (a pointer), so SizeOf(MyArray) is the size of a pointer, not the size of the array. To get the length of the array, use Length(MyArray).
But the bigger problem is saving long strings (AnsiStrings -- the usual type to which string maps --, WideStrings, UnicodeStrings). These are reference types too, so you can't just save them together with the record. You will have to save the parts of the record one by one, and for strings, you will have to use a function like:
procedure SaveStr(var F: File; const S: AnsiString);
var
Actual: Integer;
Len: Integer;
begin
Len := Length(S);
BlockWrite(F, Len, SizeOf(Len), Actual);
if Len > 0 then
begin
BlockWrite(F, S[1], Len * SizeOf(AnsiChar), Actual);
end;
end;
Of course you should normally check Actual and do appropriate error handling, but I left that out, for simplicity.
Reading back is similar: first read the length, then use SetLength to set the string to that size and then read the rest.
So now you do something like:
Len := Length(MyArray);
BlockWrite(F, Len, SizeOf(Len), Actual);
for I := Low(MyArray) to High(MyArray) do
begin
BlockWrite(F, MyArray[I].ID, SizeOf(Integer), Actual);
SaveStr(F, MyArray[I].att1);
SaveStr(F, MyArray[I].att2);
BlockWrite(F, MyArray[I].Value, SizeOf(Integer), Actual);
end;
// etc...
Note that I can't currently test the code, so it may have some little errors. I'll try this later on, when I have access to a compiler, if that is necessary.
Update
As Marco van de Voort commented, you may have to do:
rewrite(f, 1);
instead of a simple
rewrite(f);
But as I replied to him, if you can, use streams. They are easier to use (IMO) and provide a more consistent interface, no matter to what exactly you try to write or read. There are streams for many different kinds of I/O, and all derive from (and are thus compatible with) the same basic abstract TStream class.

Related

Delphi Function that takes Integer and returns a not-so-easily decoded Integer

Can anyone share what are some common Delphi examples of a function that takes a number
and returns a number that is not so obvious?
For example :
function GetNumber(const aSeed: Integer): Integer;
begin
Result := ((aSeed+5) * aSeed) + 15;
end;
So let's say the user knows that sending aSeed = 21 gives 561
and aSeed = 2, gives 29
and so on...
is there a function that makes it hard to reverse engineer the code,
even if one can generate a large number sets of Seed/Result ?
(hard : I do not mean impossible, just need to be non-trivial)
Preferably a function that does not allow in function result exceeding the
Integer result as well.
In any case, if you are not sure whether it's hard/impossible to reverse,
do feel free to share what you have.
some other requirements:
the same input always results in the same output; cannot have Random output
the same output regardless of platform: windows/android/mac/ios
won't result in some extraordinary big number (fit in Integer)
Using a hash is a very good way to achieve what you want. Here is an example that takes an integer, converts it to a string, appends it to a salt, computes the MD5 and returns the integer corresponding to the first 4 bytes:
uses
System.Hash;
function GetHash(const s: string): TBytes;
var
MD5: THashMD5;
begin
MD5 := THashMD5.Create;
MD5.Update(TEncoding.UTF8.GetBytes(s));
Result := MD5.HashAsBytes;
end;
function GetNumber(Input: Integer): Integer;
var
Hash: TBytes;
p: ^Integer;
begin
Hash := GetHash('secret' + IntToStr(Input));
P := #Hash[0];
Result := Abs(P^);
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
ShowMessage(IntToStr(GetNumber(1))); // 996659739
ShowMessage(IntToStr(GetNumber(2))); // 939216101
ShowMessage(IntToStr(GetNumber(3))); // 175456750
end;

Reading record from a text file into array in Pascal

1.this is my code i want to read a record from a text file into array in pascal my program is about making a hotel helper and i already have a text file with the data of the hotel then i should read it from the text file and store it in array .. but i am facing error 103 exit code (file not open).... any help Please . :)
program Hotel1(input,output);
const max =10; MaxFloor =10;
type
Date = record
day :1..31;
month:1..12;
year:integer;
end;
Booking = record
Guest:string[20];
S_Date:date;
E_date:date;
end;
Booking_Mat= array[1..max] of Booking;
History_Booking = record
B_num:integer;
B_Mat:Booking_Mat;
end;
Room = record
Num:integer;
Bed_num:integer;
Price:integer;
Status:Boolean;
H:History_Booking;
end;
Data = record
Ro:Room;
m:integer;
end;
Data_mat= array [1..max] of Data;
Procedure Read_Data(filename:string; var table:Data_mat);
var df:text; i,j :integer;
n,m,num,GN:integer;
Bed_num,Price:integer;
f:text;
s,e:Date;
Gname:string[20];
ok:boolean;
a:Data_mat;
c:char;
Begin
writeln('Reading ',filename,' records into array.... ');
assign(df,filename);
reset(df);
i:=0;
while (not eof) do
begin
i:=i+1;
Read (f,num);
a[i].Ro.num:=num;
Read (f,Bed_num);
a[i].Ro.Bed_num:=Bed_num;
Read (f,Price);
a[i].Ro.Price:=Price;
Read(f,c);
if (c ='Y') then
a[i].Ro.status:= true
else
a[i].Ro.status:= false;
readln;
End; {while eof}
close(df);
End; {Read_Data}
You've declared two variables of type Text, (df and f) in your var block.
You open df with these lines:
assign(df,filename);
reset(df);
You then read from f (which is not the file you opened above) in several lines, such as this one:
Read (f, num);
It's interesting to note that you actually manage to close the file you really opened, even though you never use it in your loop:
close(df);
The solution to all of these issues is to delete the declaration of either f or df, and then fix the compiler errors you get by correcting the code to use the remaining text variable. (Two important lessons here are
Only declare the variables you actually need.
Use the variables you declare.
Your loop is also invalid, because you're using while not eof with no file provided for which to test the end. Your loop should read while not Eof(df) do instead.
It's also much better to follow the typical naming convention of prefixing types with a T. It makes it clear that it's a type and not a variable, and allows you to read the code more easily. For instance, I'd change your definition of Data to TRoomData, and change the other type declarations accordingly. Here's an example - note that TRoomData now has a field (member) named Room of type TRoom:
TRoomData = record
Room: TRoom;
m: Integer;
end;
TRoom is defined as
TRoom = record
Num: Integer;
Bed_num: Integer;
Price: Integer;
Status: Boolean;
H: THistory_Booking;
end;
And so forth. This allows you to write code more clearly:
var
RoomData: TRoomData;
begin
RoomData.Room.Num := 1;
RoomData.Room.Price := 50;
// etc.
end;
With all that being said, your file does not contain text, and therefore you're using the wrong file type by using df: Text in the first place. You should use a File of TRoomData, allowing you to read and write entire records at a time. Here's an example of doing so:
var
DF: File of TRoomData;
RoomData: TRoomData;
i: Integer;
const
DataFileName = 'D:\TempFiles\RoomData.dat';
Writing it:
// Put some data into the record
RoomData.Room.Num := 1;
RoomData.Room.Bed_num := 1;
RoomData.Room.Price := 40;
RoomData.Room.Status := True;
RoomData.Room.H.B_num := 1;
for i := 1 to Max do
begin
RoomData.Room.H.B_Mat[1].Guest := Format('Guest %d', [i]);
RoomData.Room.H.B_Mat[1].S_Date.Year := 2014;
RoomData.Ro.H.B_Mat[1].S_Date.Month := i;
RoomData.Ro.H.B_Mat[1].S_Date.Day := i;
end;
// Write it out to the file
AssignFile(DF, DataFileName);
try
Rewrite(DF);
Write(DF, RoomData);
finally
CloseFile(DF);
end;
Reading it back in:
AssignFile(DF, DataFileName);
try
Reset(DF);
Read(DF, RoomData);
finally
CloseFile(DF);
end;
(Or, better yet: If the version of Pascal you're using supports it, move away from the old file I/O routines and start using TFileStream instead.)
Last but not least, learn to properly format your code. It makes it much easier to debug and maintain, and it's much easier to read when you can follow the execution path clearly.

Looking for second opinion on the validity of findings drawn from this simple localized performance test under any other different setting

My setting:
OS: Windows 7 SP1 (32 bits)
Ram: 4 Go
Processor: Intel Pentium D 3.00 GHz
Delphi XE
My simple test:
I performed a test running the following program:
program TestAssign;
{$APPTYPE CONSOLE}
uses
SysUtils,
Diagnostics;
type
TTestClazz = class
private
FIntProp: Integer;
FStringProp: string;
protected
procedure SetIntProp(const Value: Integer);
procedure SetStringProp(const Value: string);
public
property IntProp: Integer read FIntProp write SetIntProp;
property StringProp: string read FStringProp write SetStringProp;
end;
{ TTestClazz }
procedure TTestClazz.SetIntProp(const Value: Integer);
begin
if FIntProp <> Value then
FIntProp := Value;
end;
procedure TTestClazz.SetStringProp(const Value: string);
begin
if FStringProp <> Value then
FStringProp := Value;
end;
var
i, j: Integer;
stopw1, stopw2 : TStopwatch;
TestObj: TTestClazz;
begin
ReportMemoryLeaksOnShutdown := True;
//
try
TestObj := TTestClazz.Create;
//
try
j := 10000;
while j <= 100000 do
begin
///
/// assignement
///
stopw1 := TStopwatch.StartNew;
for i := 0 to j do
begin
TestObj.FIntProp := 666;
TestObj.FStringProp := 'Hello';
end;
stopw1.Stop;
///
/// property assignement using Setter
///
stopw2 := TStopwatch.StartNew;
for i := 0 to j do
begin
TestObj.IntProp := 666;
TestObj.StringProp := 'Hello';
end;
stopw2.Stop;
///
/// Log results
///
Writeln(Format('Ellapsed time for %6.d loops: %5.d %5.d', [j, stopw1.ElapsedMilliseconds, stopw2.ElapsedMilliseconds]));
//
Inc(j, 5000);
end;
//
Writeln('');
Write('Press Return to Quit...');
Readln;
finally
TestObj.Free
end
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
end.
My (provisionnal) conclusion:
It seems that:
It's worth using Setter with property under some condition
The overhead of calling a method and performing a conditional test take less time than an assignement.
My question:
Are those findings valid under any other diffrent setting or just localized ones (exception)?
I would make the following observations:
The decision as to whether or not to use a setter should be based on factors like code maintenance, correctness, readability rather than performance.
Your benchmark is wholly unreasonable since the if statements evaluate to False every time. Real world code that sets properties would be likely to modify the properties a reasonable proportion of the time that the setter runs.
I would expect that for many real world examples, the setter would run faster without the equality test. If that test were to evaluate to True every time then clearly the code would be quicker without it.
The integer setter is practically free and in fact the setter is slower than the direct field access.
The time is spent in the string property. Here there is some real performance benefit due to the optimisation of the if test which avoids string assignment code if possible.
The setters would be faster if you inlined them, but not by a significant amount.
My belief is that any real world code would never be able to detect any of these performance differences. In reality the bottleneck will be obtaining the values passed to the setters rather than time spent in the setters.
The main situation where such if protection is valuable is where the property modification is expensive. For example, perhaps it involves sending a Windows message, or hitting a database. For a property backed by a field you can probably take it or leave it.
In the chatter in the comments Premature Optimization wonders why the comparison if FStringProp <> Value is quicker than the assignment FStringProp := Value. I investigated a little further and it wasn't quite as I had originally thought.
It turns out that if FStringProp <> Value is dominated by a call to System._UStrEqual. The two strings passed are not in fact the same reference and so each character has to be compared. However, this code is highly optimised and crucially there are only 5 characters to compare.
The call to FStringProp := Value goes to System._UStrAsg and since Value is a literal with negative reference count, a brand new string has to be made. The Pascal version of the code looks like this:
procedure _UStrAsg(var Dest: UnicodeString; const Source: UnicodeString); // globals (need copy)
var
S, D: Pointer;
P: PStrRec;
Len: LongInt;
begin
S := Pointer(Source);
if S <> nil then
begin
if __StringRefCnt(Source) < 0 then // make copy of string literal
begin
Len := __StringLength(Source);
S := _NewUnicodeString(Len);
Move(Pointer(Source)^, S^, Len * SizeOf(WideChar));
end else
begin
P := PStrRec(PByte(S) - SizeOf(StrRec));
InterlockedIncrement(P.refCnt);
end;
end;
D := Pointer(Dest);
Pointer(Dest) := S;
_UStrClr(D);
end;
The key part of this is the call to _NewUnicodeString which of course calls GetMem. I am not at all surprised that heap allocation is significantly slower than comparison of 5 characters.
Put 'Hello' const into a variable and use it for setting then do a test again

String to byte array in UTF-8?

How to convert a WideString (or other long string) to byte array in UTF-8?
A function like this will do what you need:
function UTF8Bytes(const s: UTF8String): TBytes;
begin
Assert(StringElementSize(s)=1);
SetLength(Result, Length(s));
if Length(Result)>0 then
Move(s[1], Result[0], Length(s));
end;
You can call it with any type of string and the RTL will convert from the encoding of the string that is passed to UTF-8. So don't be tricked into thinking you must convert to UTF-8 before calling, just pass in any string and let the RTL do the work.
After that it's a fairly standard array copy. Note the assertion that explicitly calls out the assumption on string element size for a UTF-8 encoded string.
If you want to get the zero-terminator you would write it so:
function UTF8Bytes(const s: UTF8String): TBytes;
begin
Assert(StringElementSize(s)=1);
SetLength(Result, Length(s)+1);
if Length(Result)>0 then
Move(s[1], Result[0], Length(s));
Result[high(Result)] := 0;
end;
You can use TEncoding.UTF8.GetBytes in SysUtils.pas
If you're using Delphi 2009 or later (the Unicode versions), converting a WideString to a UTF8String is a simple assignment statement:
var
ws: WideString;
u8s: UTF8String;
u8s := ws;
The compiler will call the right library function to do the conversion because it knows that values of type UTF8String have a "code page" of CP_UTF8.
In Delphi 7 and later, you can use the provided library function Utf8Encode. For even earlier versions, you can get that function from other libraries, such as the JCL.
You can also write your own conversion function using the Windows API:
function CustomUtf8Encode(const ws: WideString): UTF8String;
var
n: Integer;
begin
n := WideCharToMultiByte(cp_UTF8, 0, PWideChar(ws), Length(ws), nil, 0, nil, nil);
Win32Check(n <> 0);
SetLength(Result, n);
n := WideCharToMultiByte(cp_UTF8, 0, PWideChar(ws), Length(ws), PAnsiChar(Result), n, nil, nil);
Win32Check(n = Length(Result));
end;
A lot of the time, you can simply use a UTF8String as an array, but if you really need a byte array, you can use David's and Cosmin's functions. If you're writing your own character-conversion function, you can skip the UTF8String and go directly to a byte array; just change the return type to TBytes or array of Byte. (You may also wish to increase the length by one, if you want the array to be null-terminated. SetLength will do that to the string implicitly, but to an array.)
If you have some other string type that's neither WideString, UnicodeString, nor UTF8String, then the way to convert it to UTF-8 is to first convert it to WideString or UnicodeString, and then convert it back to UTF-8.
var S: UTF8String;
B: TBytes;
begin
S := 'Șase sași în șase saci';
SetLength(B, Length(S)); // Length(s) = 26 for this 22 char string.
CopyMemory(#B[0], #S[1], Length(S));
end.
Depending on what you need the bytes for, you might want to include an NULL terminator.
For production code make sure you test for empty string. Adding the 3-4 LOC required would just make the sample harder to read.
I have the following two routines (source code can be downloaded here - http://www.csinnovations.com/framework_utilities.htm):
function CsiBytesToStr(const pInData: TByteDynArray; pStringEncoding: TECsiStringEncoding; pIncludesBom: Boolean): string;
function CsiStrToBytes(const pInStr: string; pStringEncoding: TECsiStringEncoding;
pIncludeBom: Boolean): TByteDynArray;
widestring -> UTF8:
http://www.freepascal.org/docs-html/rtl/system/utf8decode.html
the opposite:
http://www.freepascal.org/docs-html/rtl/system/utf8encode.html
Note that assigning a widestring to an ansistring in a pre D2009 system (including current Free Pascal) will convert to the local ansi encoding, garbling characters.
For the TBytes part, see the remark of Rob Kennedy above.

saving a records containing a member of type string to a file (Delphi, Windows)

I have a record that looks similar to:
type
TNote = record
Title : string;
Note : string;
Index : integer;
end;
Simple. The reason I chose to set the variables as string (as opposed to an array of chars) is that I have no idea how long those strings are going to be. They can be 1 char long, 200 or 2000.
Of course when I try to save the record to a type file (file of...) the compiler complains that I have to give a size to string.
Is there a way to overcome this? or a way to save those records to an untyped file and still maintain a sort of searchable way?
Please do not point me to possible solutions, if you know the solution please post code.
Thank you
You can't do it with a typed file. Try something like this, with a TFileStream:
type
TStreamEx = class helper for TStream
public
procedure writeString(const data: string);
function readString: string;
procedure writeInt(data: integer);
function readInt: integer;
end;
function TStreamEx.readString: string;
var
len: integer;
iString: UTF8String;
begin
self.readBuffer(len, 4);
if len > 0 then
begin
setLength(iString, len);
self.ReadBuffer(iString[1], len);
result := string(iString);
end;
end;
procedure TStreamEx.writeString(const data: string);
var
len: cardinal;
oString: UTF8String;
begin
oString := UTF8String(data);
len := length(oString);
self.WriteBuffer(len, 4);
if len > 0 then
self.WriteBuffer(oString[1], len);
end;
function TStreamEx.readInt: integer;
begin
self.readBuffer(result, 4);
end;
procedure TStreamEx.writeInt(data: integer);
begin
self.WriteBuffer(data, 4);
end;
type
TNote = record
Title : string;
Note : string;
Index : integer;
procedure Save(stream: TStream);
end;
procedure TNote.Save(stream: TStream);
var
temp: TMemoryStream;
begin
temp := TMemoryStream.Create;
try
temp.writeString(Title);
temp.writeString(Note);
temp.writeInt(Index);
temp.seek(0, soFromBeginning);
stream.writeInt(temp.size);
stream.copyFrom(temp, temp.size);
finally
temp.Free;
end;
end;
I'll leave the Load procedure to you. Same basic idea, but it shouldn't need a temp stream. With the record size in front of each entry, you can read it and know how far to skip if you're looking for a certain record # instead of reading the whole thing.
EDIT: This was written specifically for versions of Delphi that use Unicode strings. On older versions, you could simplify it quite a bit.
Why not write this out as XML? See my session "Practical XML with Delphi" on how to get started with this.
Another possibility would be to make your records into classes descending form TComponent and store/retreive your data in DFM files.
This Stackoverflow entry shows you how to do that.
--jeroen
PS: Sorry my XML answer was a bit dense; I'm actually on the road for two conferences (BASTA! and DelphiLive! Germany).
Basically what you need to do is very simple: create a sample XML file, then start the Delphi XML Data Binding Wizard (available in Delphi since version 6).
This wizard will generate a unit for you that has the interfaces and classes mapping XML to Delphi objects, and a few helper functions for reading them from file, creating a new object, etc. My session (see the first link above) actually contains most of the details for this process.
The above link is a video demonstrating the usage of the Delphi XML Data Binding Wizard.
You could work with two different files, one that just stores the strings in some convenient way, the other stores the records with a reference to the strings. That way you will still have a file of records for easy access even though you don't know the size of the actual content.
(Sorry no code.)
TNote = record
Title : string;
Note : string;
Index : integer;
end;
could be translated as
TNote = record
Title : string[255];
Note : string[255];
Index : integer;
end;
and use Stream.writebuffer(ANodeVariable, sizeof(TNode), but you said that strings get go over 255 chars in this case IF a string goes over 65535 chars then change WORD to INTEGER
type
TNodeHeader=Record
TitleLen,
NoteLen: Word;
end;
(* this is for writing a TNode *)
procedure saveNodetoStream(theNode: TNode; AStream: TStream);
var
header: TNodeHeader;
pStr: PChar;
begin
...
(* writing to AStream which should be initialized before this *)
Header.TitleLen := Length(theNode.Title);
header.NodeLen := Length(theNode.Note);
AStream.WriteBuffer(Header, sizeof(TNodeHeader);
(* save strings *)
PStr := PChar(theNode.Title);
AStream.writeBuffer(PStr^, Header.TitleLen);
PStr := PChar(theNode.Note);
AStream.writebuffer(PStr^, Header.NoteLen);
(* save index *)
AStream.writebuffer(theNode.Index, sizeof(Integer));
end;
(* this is for reading a TNode *)
function readNode(AStream: TStream): TNode;
var
header: THeader
PStr: PChar;
begin
AStream.ReadBuffer(Header, sizeof(TNodeHeader);
SetLength(Result.Title, Header.TitleLen);
PStr := PChar(Result.Title);
AStream.ReadBuffer(PStr^, Header.TitleLen);
SetLength(Result.Note, Header.NoteLen);
PStr := PChar(Result.Note);
AStream.ReadBuffer(PStr^, Header.NoteLen);
AStream.ReadBuffer(REsult.Index, sizeof(Integer)(* 4 bytes *);
end;
You can use the functions available in this Open Source unit.
It allows you to serialize any record content into binary, including even dynamic arrays within:
type
TNote = record
Title : string;
Note : string;
Index : integer;
end;
var
aSave: TRawByteString;
aNote, aNew: TNote;
begin
// create some content
aNote.Title := 'Title';
aNote.Note := 'Note';
aNote.Index := 10;
// serialize the content
aSave := RecordSave(aNote,TypeInfo(TNote));
// unserialize the content
RecordLoad(aNew,pointer(aSave),TypeInfo(TNote));
// check the content
assert(aNew.Title = 'Title');
assert(aNew.Note = 'Note');
assert(aNew.Index = 10);
end;

Resources