Ada Endianness Float/Integer - endianness

Question

I'm a new developper in Ada so forgive me if I not clear enough.
I am confronted with a problem and I do not know where from the fault can come. I explain first of all the context:
I possess a set of tests which work on a qemu (BE). I wished execute them on PC native (x86) with the pragma Default_Scalar_Storage_Order ( High_Order_First). I noticed that some of my test worked perfectly but it was not the case for the tests including float. To make simple I have write a test including a FLOAT and an INT.
with AUNIT.ASSERTIONS; use AUNIT.ASSERTIONS;
with BASIC_TYPES;
with BASIC_TYPES.STREAM;
with INTERFACES;
with ADA.INTEGER_TEXT_IO;
with ADA.FLOAT_TEXT_IO;
with ADA.TEXT_IO;
with STREAMS;
with SYSTEM;
package body TEST.TEST is
function Integer2Hexa(Hex_Int : Integer; Bits_Nbr : Integer) return String is
Hexa : String(1..Bits_Nbr);
begin
Ada.Integer_Text_IO.Put(Hexa,Hex_Int,16);
return Hexa;
end Integer2Hexa;
function NAME (T : TEST) return AUNIT.MESSAGE_STRING is
pragma UNREFERENCED (T);
begin
return AUNIT.FORMAT ("Test package");
end NAME;
IntegerNbr : BASIC_TYPES.INT32_T;
FloatNbr : INTERFACES.IEEE_Float_32;
procedure RUN_TEST (T : in out TEST) is
PACKED_ARRAY : BASIC_TYPES.UINT8_ARRAY_NC_T (1 .. 8) := (others => 0);
MY_STREAM : STREAMS.STREAM_T;
use type BASIC_TYPES.UINT8_ARRAY_NC_T;
begin
IntegerNbr := 479037433;
FloatNbr := 2.0012151e+09;
ADA.TEXT_IO.PUT_LINE ("Default bit order: " & SYSTEM.Default_Bit_Order'IMG);
ADA.TEXT_IO.PUT_LINE ("Integer size : " & INTEGER'IMAGE (INTEGER'SIZE));
ADA.TEXT_IO.PUT ("16#4EEE903D#"); -- 2.0012151e+09 in FLOAT BIG ENDIAN
ADA.TEXT_IO.PUT (Integer2Hexa(Integer(IntegerNbr),32)); -- 16#1C8D87F9# in INT BIG ENDIAN
ADA.TEXT_IO.NEW_LINE;
-- Init the stream
STREAMS.INIT (MY_STREAM => MY_STREAM,
STREAM_ADDRESS => PACKED_ARRAY (PACKED_ARRAY'FIRST)'ADDRESS,
STREAM_SIZE => PACKED_ARRAY'LENGTH);
BASIC_TYPES.STREAM.WRITE_FLOAT_T (MY_STREAM => MY_STREAM,
ITEM => FloatNbr,
ALIGN_MODE => STREAMS.PACK);
BASIC_TYPES.STREAM.WRITE_INT32_T (MY_STREAM => MY_STREAM,
ITEM => IntegerNbr,
ALIGN_MODE => STREAMS.PACK);
if (not ASSERT(PACKED_ARRAY = (16#4e#, 16#ee#, 16#90#, 16#3d#, 16#1c#, 16#8d#, 16#87#, 16#f9#), "PACKED_ARRAY incorrect")) then
for I in PACKED_ARRAY'RANGE loop
ADA.TEXT_IO.PUT (Integer2Hexa(Integer(PACKED_ARRAY (I)),8));
end loop;
ADA.TEXT_IO.NEW_LINE;
end if;
end RUN_TEST;
end TEST.TEST;
I noticed that the writing of the INT is correctly made but it is not the case of the FLOAT (it is written in Little Endian). Indeed in exit I should have
16#4e#, 16#ee#, 16#90#, 16#3d#, 16#1c#, 16#8d#, 16#87#, 16#f9#
but I get
16#3d#, 16#90#, 16#ee#, 16#4e#, 16#1c#, 16#8d#, 16#87#, 16#f9#
I used this site to confirm my results: https://www.scadacore.com/tools/programming-calculators/online-hex-converter/
I don't know if the conversion thanks to the pragma is correctly used for the FLOAT. I call it in my gpr file in the package Compiler with this text in the PRAGMA.txt : pragma Default_Scalar_Storage_Order(High_Order_First);
package Compiler is
for Local_Configuration_Pragmas use "PRAGMAS.txt";
for Switches ("ada") use ("-g");
end Compiler;
Does the problem come to my way to use the pragma?
Here are the called procedures:
procedure WRITE_FLOAT_T
(MY_STREAM : in out STREAMS.STREAM_T;
ITEM : in BASIC_TYPES.FLOAT_T;
ALIGN_MODE : in STREAMS.ALIGN_MODE_T)
is
pragma UNREFERENCED (ALIGN_MODE);
-- Temporary types for non pack case
type TMP_TYPE_T is new STANDARD.FLOAT;
for TMP_TYPE_T'VALUE_SIZE use FLOAT_T_SIZE_C;
TMP_TYPE : TMP_TYPE_T;
subtype BITS_FIELD_T is STREAMS.BIT_FIELD_ARR_NC_T (1 .. STREAMS.SIZE_T (FLOAT_T_SIZE_C));
function TO_BITS_ARRAY is new UNCHECKED_CONVERSION (TMP_TYPE_T,
BITS_FIELD_T);
begin
-- Convert item to a temporary type
TMP_TYPE := TMP_TYPE_T(ITEM);
STREAMS.WRITE (MY_STREAM => MY_STREAM,
DATA => TO_BITS_ARRAY(TMP_TYPE));
end WRITE_FLOAT_T;
procedure WRITE (MY_STREAM : in out STREAM_T;
DATA : in BIT_FIELD_ARR_NC_T) is
begin
if (MY_STREAM.ERROR_CODE = NO_ERROR)
and then (MY_STREAM.WRITE_OFFSET + DATA'LENGTH - 1 <= MY_STREAM.STREAM_SIZE * 8) then
if (MY_STREAM.WRITE_OFFSET mod 8 = 1) and then (DATA'LENGTH mod 8 = 0) then
-- Byte mode
WRITE_BYTES(MY_STREAM => MY_STREAM,
DATA => DATA);
else
-- Bit mode
WRITE_BITS(MY_STREAM => MY_STREAM,
DATA => DATA);
end if;
elsif (MY_STREAM.ERROR_CODE = NO_ERROR) then
-- Update ERROR_CODE on first error
MY_STREAM.ERROR_CODE := END_ERROR;
end if;
end WRITE;
procedure WRITE_BYTES (MY_STREAM : in out STREAM_T;
DATA : in BIT_FIELD_ARR_NC_T) is
BYTE_FIELD_ARR : BYTE_FIELD_ARR_NC_T (1 .. MY_STREAM.STREAM_SIZE);
for BYTE_FIELD_ARR'ADDRESS use MY_STREAM.STREAM_ADDRESS;
TMP_BYTE_FIELD_ARR : BYTE_FIELD_ARR_NC_T (1 .. DATA'LENGTH / 8);
for TMP_BYTE_FIELD_ARR'ADDRESS use DATA'ADDRESS;
begin
-- Write byte field
BYTE_FIELD_ARR ((MY_STREAM.WRITE_OFFSET + 7) / 8 .. (MY_STREAM.WRITE_OFFSET + 7) / 8 + (DATA'LENGTH / 8) - 1) := TMP_BYTE_FIELD_ARR;
MY_STREAM.WRITE_OFFSET := MY_STREAM.WRITE_OFFSET + DATA'LENGTH;
end WRITE_BYTES;
Thank you in advance!
Q.Dherb

Answer 1

According to documentation of Scalar_Storage_Order:
This implementation defined attribute only apply to Array and Record. This means it has no effect for the memory layout of scalar type such as Float or Integer. Whatever the value of the Default_Scalar_Storage_Order attribute, on a big endian machine a 16#12345678# integer would be represented as 12 34 56 78 and on a low endian machine it would be represented as 78 56 34 12.
For array it determines the order of storage_element (that is usually byte) of each scalar component. In your case, all of your array component have a size which is inferior or equal to a storage element which means the Scalar_Storage_Order clause has no effect.
Here is an example that show the effect of this clause for array:
with Ada.Text_IO;
with System;
with Interfaces;
with Ada.Streams;
with Ada.Integer_Text_IO;
procedure Scalar_Storage_Element_Exemple is
type T_U16_Arr_Le is array (Positive range <>) of Interfaces.Unsigned_16
with Component_Size => 16, Scalar_Storage_Order => System.Low_Order_First;
type T_U16_Arr_Be is array (Positive range <>) of Interfaces.Unsigned_16
with Component_Size => 16, Scalar_Storage_Order => System.High_Order_First;
type T_U8_Arr_Le is array (Positive range <>) of Interfaces.Unsigned_8
with Component_Size => 8, Scalar_Storage_Order => System.Low_Order_First;
type T_U8_Arr_Be is array (Positive range <>) of Interfaces.Unsigned_8
with Component_Size => 8, Scalar_Storage_Order => System.High_Order_First;
Arr_16_LE : T_U16_Arr_Le (1 .. 2) := (16#1234#, 16#5678#);
Arr_16_BE : T_U16_Arr_Be (1 .. 2) := (16#1234#, 16#5678#);
Arr_8_LE : T_U8_Arr_Le (1 .. 4) := (16#12#, 16#34#, 16#56#, 16#78#);
Arr_8_BE : T_U8_Arr_Be (1 .. 4) := (16#12#, 16#34#, 16#56#, 16#78#);
Sea_16_LE : Ada.Streams.Stream_Element_Array (1 .. 4) with Address => Arr_16_LE'Address;
Sea_16_BE : Ada.Streams.Stream_Element_Array (1 .. 4) with Address => Arr_16_BE'Address;
Sea_8_LE : Ada.Streams.Stream_Element_Array (1 .. 4) with Address => Arr_8_LE'Address;
Sea_8_BE : Ada.Streams.Stream_Element_Array (1 .. 4) with Address => Arr_8_BE'Address;
function byte2Hexa(byte : Integer) return String is
Hexa : String(1..8);
begin
Ada.Integer_Text_IO.Put(Hexa,byte,16);
return Hexa;
end byte2Hexa;
begin
for byte of Sea_16_LE loop
Ada.Text_IO.Put(byte2Hexa(Integer(byte)));
end loop;
-- display 16#34# 16#12# 16#78# 16#56#
-- each item of the array is in LE
Ada.Text_IO.New_Line;
for byte of Sea_16_BE loop
Ada.Text_IO.Put(byte2Hexa(Integer(byte)));
end loop;
-- 16#12# 16#34# 16#56# 16#78#
-- each item of the array is in BE
Ada.Text_IO.New_Line;
for byte of Sea_8_LE loop
Ada.Text_IO.Put(byte2Hexa(Integer(byte)));
end loop;
-- 16#12# 16#34# 16#56# 16#78#
-- no effect as size of component is inferior or equal to storage_element size
Ada.Text_IO.New_Line;
for byte of Sea_8_BE loop
Ada.Text_IO.Put(byte2Hexa(Integer(byte)));
end loop;
-- 16#12# 16#34# 16#56# 16#78#
-- no effect as size of component is inferior or equal to storage_element size
end Scalar_Storage_Element_Exemple;
Your float serialization works on your QEMU because you are already on BE. Therefore the Scalar_Storage_Order is only confirming and has no effect.
It doesn't works on x86 because the native endianess is LE and as explained previously the BE Scalar_Storage_Order clause have no effect for the types that are involved. So the end result is a LE float.
Provided you use the same logic for serialization (the relevant code is not provided so I assume it's different), Integer or Float should have behaved similarly here.

Answer 2

It's not entirely clear, because you've included a lot of confusing detail, but I think you're trying to write to streams in an endianness-independent way in order to communicate (over the net?) between machines of different endianness.
The issue with your procedure WRITE_FLOAT_T is that its ITEM is a plain float, so Scalar_Storage_Order has no effect.
The way I've used Scalar_Storage_Order is to declare the record I wanted to send,
type SNTP_Packet is record
-- contents
end record
with
Bit_Order => System.High_Order_First,
Scalar_Storage_Order => System.High_Order_First,
Size => 48 * 8;
for SNTP_Packet use record
-- placement of content
end record;
subtype Net_Packet is Ada.Streams.Stream_Element_Array (1 .. 48);
-- This is what actually gets streamed
function To_Net_Packet
is new Ada.Unchecked_Conversion (SNTP_Packet, Net_Packet);
function To_SNTP_Packet
is new Ada.Unchecked_Conversion (Net_Packet, SNTP_Packet);
You could use pragma Default_Scalar_Storage_Order, but then I'm not sure what happens about the need to make Bit_Order match.
Alternatively, if you want to be able to use e.g. Float'Write, you can alter the way that GNAT streams fundamental types.
The Ada runtime handles streaming for fundamental types using the package System.Stream_Attributes, in files s-stratt.ads, s-stratt.adb, and provides an alternative implementation in s-stratt__xdr.adb (in the latest compilers; older compilers may use a different file name, but there'll be an xdr in there).
Getting the compiler to use this alternate version isn't very straightforward, but this worked for me:
copy s-stratt__xdr.adb to s-stratt.adb in your working directory
use gnatmake -a to compile the necessary parts of the runtime locally (-gnatpg says "compile for the runtime"):
gnatmake -a -f s-stratt.adb -gnatpg
build your program:
gprbuild main.adb
Note, gprbuild doesn’t support -a. It might be possible to use a project file to allow you to make a library containing the modified runtime components.

Answer 3

You are trying to encode data in bigendian (probably for network transmission) independently of your hot endianess.
You expect both arguments of your UNCHECKED_CONVERSION to be Scalar_Storage_Order=System.High_Order_First defined here
If the opposite storage order is specified, then whenever the value of a scalar component of an object of type S is read, the storage elements of the enclosing machine scalar are first reversed.
Your problem comes from the use of an old gcc version.
I tried to decompose the problem by testing the conversion of a FLOAT_T.Scalar_Storage_Order from System.Default_Bit_Order to System.High_Order_First thanks to the UNCHECKED_CONVERSION with following code:
inc.ads:
with SYSTEM;
package inc is
type BITS32_T is mod (2 ** 32);
for BITS32_T'SIZE use 32;
subtype UINT32_T is BITS32_T;
subtype INT32_UNSIGNED_T is UINT32_T;
type SIZE_T is new UINT32_T;
subtype INDEX_T is SIZE_T range 1 .. SIZE_T'LAST;
type BIT_T is mod (2 ** 1);
for BIT_T'SIZE use 1;
type BITS8_T is mod (2 ** 8);
for BITS8_T'SIZE use 8;
-- 64-bit signed integer
type INT64_T is range -(2 ** (64 - 1)) .. (2 ** (64 - 1) - 1);
for INT64_T'SIZE use 64;
subtype INT64_SIGNED_T is INT64_T;
type BIT_FIELD_ARR_NC_T is array (INDEX_T range <>) of BIT_T;
for BIT_FIELD_ARR_NC_T'COMPONENT_SIZE use 1;
for BIT_FIELD_ARR_NC_T'Scalar_Storage_Order use System.High_Order_First;
--Low_Order_First
type BYTE_FIELD_ARR_HOST_ENDIANNESS_NC_T is array (INDEX_T range <>) of BITS8_T;
for BYTE_FIELD_ARR_HOST_ENDIANNESS_NC_T'COMPONENT_SIZE use 8;
type BIT_FIELD_ARR_HOST_ENDIANNESS_NC_T is array (INDEX_T range <>) of BIT_T;
for BIT_FIELD_ARR_HOST_ENDIANNESS_NC_T'COMPONENT_SIZE use 1;
end inc;
test_types.adb:
with inc;
with INTERFACES;
with Ada.Text_IO;
with Ada.Integer_Text_IO;
with UNCHECKED_CONVERSION;
procedure TEST_TYPES is
longfloat : INTERFACES.IEEE_FLOAT_64 := INTERFACES.IEEE_FLOAT_64(1e11);
--float64 : inc.INT64_T := 16#1122334455667788#;
int64 : inc.INT64_T := 16#1122334455667788#;
---------------- TYPE used to print represnentation in memory ------------------------------
subtype BYTES_ARRAY_T is inc.BYTE_FIELD_ARR_HOST_ENDIANNESS_NC_T (1 .. 8);
-------- tableau de bits -------
subtype BITS_FIELD_T is inc.BIT_FIELD_ARR_NC_T (1 .. 64);
subtype BITS_FIELD_HOST_ENDIANNESS_T is inc.BIT_FIELD_ARR_HOST_ENDIANNESS_NC_T (1 .. 64);
---------------- FLOAT with BIG ENDIAN encoding ------------------------------
type TMP_TYPE_T is new STANDARD.LONG_FLOAT;
for TMP_TYPE_T'VALUE_SIZE use 64;
TMP_TYPE : TMP_TYPE_T;
function TO_BYTES_ARRAY is new UNCHECKED_CONVERSION (TMP_TYPE_T, BITS_FIELD_T);
bytes: BITS_FIELD_T;
---------------- FLOAT with host ENDIANNESS ------------------------------
function TO_BYTES_HOST_ENDIANNESS_ARRAY is new UNCHECKED_CONVERSION (TMP_TYPE_T, BITS_FIELD_HOST_ENDIANNESS_T);
bytesNoEndian: BITS_FIELD_HOST_ENDIANNESS_T;
---------------- INTEGER with ENDIAN CONVERSION ------------------------------
type TMP_Integer_T is new STANDARD.LONG_LONG_INTEGER;
for TMP_Integer_T'VALUE_SIZE use 64;
TMP_Integer : TMP_Integer_T;
function TO_BYTES_ARRAY_Integer is new UNCHECKED_CONVERSION (TMP_Integer_T, BITS_FIELD_T);
bytes_integer: BITS_FIELD_T;
---------------- INTEGER without ENDIAN CONVERSION ------------------------------
function TO_BYTES_ARRAY_HOST_ENDIANNESS_Integer is new UNCHECKED_CONVERSION (TMP_Integer_T, BITS_FIELD_HOST_ENDIANNESS_T);
bytes_no_endian_integer: BITS_FIELD_HOST_ENDIANNESS_T;
-- representation in memory
float_rep: BYTES_ARRAY_T;
float_bits_field_rep: BYTES_ARRAY_T;
int_rep: BYTES_ARRAY_T;
int_bits_field_rep: BYTES_ARRAY_T;
for float_rep'ADDRESS use bytesNoEndian'ADDRESS;
for float_bits_field_rep'ADDRESS use bytes'ADDRESS;
for int_rep'ADDRESS use bytes_no_endian_integer'ADDRESS;
for int_bits_field_rep'ADDRESS use bytes_integer'ADDRESS;
------------------ FUNCTION FROM STACKOVERFLOW-----------------------
function byte2hexa(byte : Integer) return String is
Hexa : String(1..8);
begin
Ada.Integer_Text_IO.Put(Hexa, byte, 16);
return Hexa;
end byte2hexa;
procedure array2hexa(bytes : BYTES_ARRAY_T) is
begin
Ada.Integer_Text_IO.Put(Integer(bytes(1)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(2)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(3)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(4)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(5)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(6)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(7)), Base => 16);
Ada.Integer_Text_IO.Put(Integer(bytes(8)), Base => 16);
Ada.Text_IO.New_line;
end array2hexa;
begin
-- test serialisation on float
TMP_TYPE := TMP_TYPE_T(longfloat);
bytesNoEndian := TO_BYTES_HOST_ENDIANNESS_ARRAY(TMP_TYPE);
Ada.Text_IO.Put_line("float in native endianess ");
array2hexa(float_rep);
Ada.Text_IO.New_line;
Ada.Text_IO.Put_line("float into BigEndian Bit array");
TMP_TYPE := TMP_TYPE_T(longfloat);
bytes := TO_BYTES_ARRAY(TMP_TYPE);
array2hexa(float_bits_field_rep);
Ada.Text_IO.New_line;
-- test serialisation on integer
TMP_Integer := TMP_Integer_T(int64);
bytes_no_endian_integer := TO_BYTES_ARRAY_HOST_ENDIANNESS_Integer(TMP_Integer);
Ada.Text_IO.Put_line("Integer in native endianess ");
array2hexa(int_rep);
Ada.Text_IO.New_line;
Ada.Text_IO.Put_line("Integer into BigEndian Bit array");
TMP_Integer := TMP_Integer_T(int64);
bytes_integer := TO_BYTES_ARRAY_Integer(TMP_Integer);
array2hexa(int_bits_field_rep);
end TEST_TYPES;
In my proposed code, the problem comes from that endianess of BITS_FIELD_T'element is clearly defined but behavior of UNCHECKED_CONVERSION is undefined (regarding the convertion from Endianess of the Float toward the Endianess of BITS_FIELD_T )
Surprisingly, using gcc (GCC) 6.2.1 20161010 (for GNAT Pro 17.2 20170606)
the UNCHECKED_CONVERSION convert the endianess of the integer but not the floating point :
float in native endianess
16#0# 16#0# 16#0# 16#E8# 16#76# 16#48# 16#37# 16#42#
float into BigEndian Bit array
16#0# 16#0# 16#0# 16#E8# 16#76# 16#48# 16#37# 16#42#
Integer in native endianess
16#88# 16#77# 16#66# 16#55# 16#44# 16#33# 16#22# 16#11#
Integer into BigEndian Bit array
16#11# 16#22# 16#33# 16#44# 16#55# 16#66# 16#77# 16#88#
but with gcc (GCC) 7.3.1 20181018 (for GNAT Pro 20.0w 20181017)
Floating point values are corectly swapped:
float in native endianess
16#0# 16#0# 16#0# 16#E8# 16#76# 16#48# 16#37# 16#42#
float into BigEndian Bit array
16#42# 16#37# 16#48# 16#76# 16#E8# 16#0# 16#0# 16#0#
One solution (for old compiler) is to pass through an intermediate BigEndian structure before UNCHECKED_CONVERSION:
procedure WRITE_LONG_FLOAT_T
(MY_STREAM : in out STREAMS.STREAM_T;
ITEM : in BASIC_TYPES.LONG_FLOAT_T;
ALIGN_MODE : in STREAMS.ALIGN_MODE_T)
is
pragma UNREFERENCED (ALIGN_MODE);
-- Temporary types for non pack case
type TMP_TYPE_T is new STANDARD.LONG_FLOAT;
for TMP_TYPE_T'VALUE_SIZE use LONG_FLOAT_T_SIZE_C;
TMP_TYPE : TMP_TYPE_T;
subtype BITS_FIELD_T is STREAMS.BIT_FIELD_ARR_NC_T (1 .. STREAMS.SIZE_T (LONG_FLOAT_T_SIZE_C));
function TO_BITS_ARRAY is new UNCHECKED_CONVERSION (TMP_TYPE_T,
BITS_FIELD_T);
type ITEM_ENDIAN_T is
record
TMP_TYPE_ENDIAN : TMP_TYPE_T;
end record;
for ITEM_ENDIAN_T'Bit_Order use System.High_Order_First;
for ITEM_ENDIAN_T'Scalar_Storage_Order use System.High_Order_First;
ITEM_ENDIAN : ITEM_ENDIAN_T;
--subtype LONG_FLOAT_TO_ARRAY_T is PUS.TYPES.BYTE_FIELD_NC_T (1 .. 8);
function TO_BITS_ARRAY_ENDIAN is new UNCHECKED_CONVERSION (ITEM_ENDIAN_T, BITS_FIELD_T);
begin
-- Convert item to a temporary type
TMP_TYPE := TMP_TYPE_T(ITEM);
ITEM_ENDIAN.TMP_TYPE_ENDIAN := TMP_TYPE;
STREAMS.WRITE (MY_STREAM => MY_STREAM,
--DATA => TO_BITS_ARRAY(TMP_TYPE));
DATA => TO_BITS_ARRAY_`enter code here`ENDIAN(ITEM_ENDIAN));
end WRITE_LONG_FLOAT_T;

Answer 4

One way (or another source of ideas...) is to use the IEEE 754 representations packages: http://www.dmitry-kazakov.de/ada/components.htm#IEEE_754
An endian-neutral use of those packages is done here: http://excel-writer.sf.net/