I'm creating a 4 bit and 8 bit colour encoding for font. This includes a foreground, background, style, and format. My hope is to use the following struct to represent the data in a 4-byte pack. My intent is to extract it as a single uint32_t that can be converted to binary data and saved in a file.
This is what I currently have:
struct font_pack {
uint8_t : 8;
struct {
uint8_t format : 4;
uint8_t style : 4;
} header;
uint8_t foreground;
uint8_t background;
}
The header contains two half-bytes. format says the colour codes are either 4-bit or 8-bit colour. style is a bit flag set that declares formatter such as bolding and underline.
I am then using the following union to get the raw binary for both writing to file, as well to set or print the data as hex.
union font_raw {
font_pack pack;
uint32_t data;
}
Unfortunately when I print out the hex I get 0x04032100 when I was expecting 0x00120304. Which makes me feel like byte alignment is not guaranteed within a union and endianness is catching me. I was really just hoping to have simple method of packing and unpacking the data into 3 bytes.
Is there any other easy way to do it or am I stuck with making a more traditional function that does the packing and unpacking?
This looks like an endianness issue, for sure. I'm guessing you're on x86/x64 (Intel-like) architecture, which is little-endian and will pack bytes from least-significant to most significant. Provided you are writing and reading the data on the same architecture (so little-endian systems) the endianness will ensure you get byte packing read back in the same order, so your font_pack members should still come out correctly. However if you'll be loading these files on a big-endian system you'll need to go the more traditional route. But if you're on the same endian-ness guaranteed, I'd go with your method. It's nicely elegant :)
edit: If you are reading or writing between different endianness machines, then you could always do something like the following:
#ifdef LITTLE_ENDIAN
struct font_pack {
uint8_t : 8;
struct {
uint8_t format : 4;
uint8_t style : 4;
} header;
uint8_t foreground;
uint8_t background;
}
#else
struct font_pack {
uint8_t background;
uint8_t foreground;
struct {
uint8_t format : 4;
uint8_t style : 4;
} header;
uint8_t : 8;
}
#endif
And then define LITTLE_ENDIAN on your x86 or similar system, and not on the big-endian system. Hope that helps.
Related
I am trying to write a module that uses the AXI4 streaming protocol to communicate with the previous and next modules. The modules use the following communication signals:
TDATA, which is 16 bits,
TKEEP, which is 2 bits,
TUSER, which is 1 bit,
TVALID, which is 1 bit,
TREADY, which is 1 bit and goes towards the previous module, and
TLAST, which is 1 bit.
These all need to be separate signals. I tried to implement it using the following code:
#include "core.h"
void core_module(hls::stream<ap_axis_str> &input_stream, hls::stream<ap_axis_str> &output_stream){
#pragma HLS INTERFACE axis port=input_stream
#pragma HLS INTERFACE axis port=output_stream
#pragma HLS INTERFACE s_axilite port=return bundle=CTRL
ap_axis_str strm_val_in;
ap_axis_str strm_val_out;
for (int i = 0; i<NDATA; i++){
strm_val_in = input_stream.read();
strm_val_out.data = strm_val_in.data * 2;
strm_val_out.keep = 3;
strm_val_out.valid = 1;
strm_val_in.ready = 1;
strm_val_out.user = ((i%2)==0);
strm_val_out.last = (i == NDATA-1) ? 1:0;
output_stream.write(strm_val_out);
}
}
where the header file is
#ifndef core_h
#define core_h
#include <ap_int.h>
#include <ap_axi_sdata.h>
#include <hls_stream.h>
typedef ap_uint<16> word;
#define NDATA 10
struct ap_axis_str {
word data;
ap_uint<2> keep;
bool user;
bool last;
bool ready;
bool valid;
};
void core_module(hls::stream<ap_axis_str> &input_stream, hls::stream<ap_axis_str> &output_stream);
#endif
The problem is that this doesn't separate the signals. When I synthesise it and run it in the co-simulation (giving it values from 0 to 9), even if the result is what I expect it to be, the waveform produced looks like this:
We can see that TREADY, TVALID, and TDATA are there, but not the other 3. Furthermore, looking at the contents of TDATA (which for some reason are 64 bits) we notice that they contain all the signals. They are the following:
0001000001030000,
0001000000030002,
0001000001030004,
0001000000030006,
...
000100000003000c, (they are in base 16)
0001000001030010,
0001000100030012.
From which we can see that the 3 in position 12 is probably what was intended to be TKEEP, the 1 in position 8 which only appears in the last case is probably what was intended to be TUSER, the last 4 digits are what was supposed to be TDATA, etc. Additionally, the program drops TREADY when it isn't ready to receive data, which is what is intended of TREADY, but I didn't program it to work this way, which means that it's automatically generated and probably has nothing to do with the TREADY I told it to have.
So my question is: How do I make a module that gives out the correct 6 separate signals for the version of the AXI4 protocol that we are using?
Well, according to the Xilinx Documentation,
If you specify an hls::stream object with a data type other than ap_axis or ap_axiu, the tool will infer an AXI4-Stream interface without the TLAST signal, or any of the side-channel signals. This implementation of the AXI4-Stream interface consumes fewer device resources, but offers no visibility into when the stream is ending.
Now I had already imported the needed module with#include <ap_axi_sdata.h>, all I needed to do was actually use it by removing
struct ap_axis_str {
word data;
ap_uint<2> keep;
bool user;
bool last;
bool ready;
bool valid;
};
and replacing it with
typedef ap_axiu<16, 1, 0, 0> ap_axis_str;
Additionally, I needed to remove my manual attempt to control TREADY and TVALID, as those are done automatically.
How can I improve the following code, that is, make it more robust with respect to type safety and endianness using the functions and macros in the Linux kernel's API? For instance, in the following example src_data is an array of two 16-bit signed integers (typically stored in little endian order) and is to be sent out via UART in big endian byte order.
s16 src_data[2] = {...}; /* note: this is signed data! */
u8 tx_data[4];
u8* src_data_u8 = (u8*)src_data;
tx_data[0] = src_data_u8[1];
tx_data[1] = src_data_u8[0];
tx_data[2] = src_data_u8[3];
tx_data[3] = src_data_u8[2];
I think the functions cpu_to_be16 and cpu_to_be16p should play a role in doing this conversion. Although I'm not sure how I can use them in a way that is safe and robust to endianness.
As I understand, two 16-bit words, to be sent, one after another, after converting each into bigendian format.
I think following should be ok.
s16 src_data[2] = {...}; /* note: this is signed data! */
s16 tx_data[2];
tx_data[0] = cpu_to_be16(src_data_u8[0]);
tx_data[1] = cpu_to_be16(src_data_u8[1]);
Your issue with safety seems to be that the htons(x) function/macro expects an unsigned integer, but you possess a signed one. Not an issue:
union {
int16_t signed_repr;
uint16_t unsigned_repr;
} data;
data.signed_repr = ...;
u16 unsigned_big_endian_data = htons(data.unsigned_repr);
memcpy(tx_data, &unsigned_big_endian_data,
min(sizeof tx_data, sizeof unsigned_big_endian_data));
PS. Type-punning via unions is perfectly well-defined.
I believe the following is one of the best answers to my question. I have used the links provided by #0andriy to existing examples in the kernel source code.
Converting a signed 16-bit value for transmitting
s16 src = -5;
u8 dst[2];
__be16 tx_buf;
*(__be16*)dst = cpu_to_be16(src);
Converting multiple signed 16-bit values for transmitting
s16 src[2] = {-5,-2};
u8 dst[4];
s16* psrc = src;
u8* pdst = dst;
int len = sizeof(src);
for ( ; len > 1; len -= 2) {
*(__be16 *)pdst = cpu_to_be16p(psrc++);
pdst += 2;
}
A quick disclaimer, I still need to check if this code is correct / compiles.
Overall, I'm a bit unsatisfied with the solution for copying and converting the endianness of multiple values since it is prone to typos and could easily be implemented into a macro.
If the Linux machine will always be little endian, and the protocol will always be big endian, then the code works fine and you don't need to change anything.
If you for some reason need to make the Linux code endian-independent, then you'd use:
tx_data[0] = ((unsigned int)src_data[0] >> 8) & 0xFF;
tx_data[1] = ((unsigned int)src_data[0] >> 0) & 0xFF;
tx_data[2] = ((unsigned int)src_data[1] >> 8) & 0xFF;
tx_data[3] = ((unsigned int)src_data[1] >> 0) & 0xFF;
Where the cast is there to ensure that the right shifts are not carried out on a signed type, which would invoke non-portable implementation defined behavior.
The advantage of bit shifts compared to any other version is that they work on an abstraction level above the hardware and endianess, letting the specific compiler generate the instructions for the underlying memory access. Code such as u16 >> 8 always means "give me the least significant byte" regardless of where that byte is stored in memory.
I used the 'ntQuerySystemInformation' to get all the handle information like:
NtQuerySystemInformation(SystemHandleInformation, pHandleInfor, ulSize,NULL);//SystemHandleInformation = 16
struct of pHandleInfor is:
typedef struct _SYSTEM_HANDLE_INFORMATION
{
ULONG ProcessId;
UCHAR ObjectTypeNumber;
UCHAR Flags;
USHORT Handle;
PVOID Object;
ACCESS_MASK GrantedAccess;
} SYSTEM_HANDLE_INFORMATION, *PSYSTEM_HANDLE_INFORMATION;
It works well in xp 32bit, but in Win7 64bit can only get the right pid that less than 65535. The type of processId in this struct is ULONG, I think it can get more than 65535. What's wrong with it? Is there any other API instead?
There are two enum values for NtQuerySystemInformation to get handle info:
CNST_SYSTEM_HANDLE_INFORMATION = 16
CNST_SYSTEM_EXTENDED_HANDLE_INFORMATION = 64
And correspondingly two structs: SYSTEM_HANDLE_INFORMATION and SYSTEM_HANDLE_INFORMATION_EX.
The definitions for these structs are:
struct SYSTEM_HANDLE_INFORMATION
{
short UniqueProcessId;
short CreatorBackTraceIndex;
char ObjectTypeIndex;
char HandleAttributes; // 0x01 = PROTECT_FROM_CLOSE, 0x02 = INHERIT
short HandleValue;
size_t Object;
int GrantedAccess;
}
struct SYSTEM_HANDLE_INFORMATION_EX
{
size_t Object;
size_t UniqueProcessId;
size_t HandleValue;
int GrantedAccess;
short CreatorBackTraceIndex;
short ObjectTypeIndex;
int HandleAttributes;
int Reserved;
}
As You can see, the first struct really can only contain 16-bit process id-s...
See for example ProcessExplorer project's source file ntexapi.h for more information.
Note also that the field widths for SYSTEM_HANDLE_INFORMATION_EX in my struct definitions might be different from theirs (that is, in my definition some field widths vary depending on the bitness), but I think I tested the code both under 32-bit and 64-bit and found it to be correct.
Please recheck if necessary and let us know if You have additional info.
From Raymond Chen's article Processes, commit, RAM, threads, and how high can you go?:
I later learned that the Windows NT folks do try to keep the numerical values of process ID from getting too big. Earlier this century, the kernel team experimented with letting the numbers get really huge, in order to reduce the rate at which process IDs get reused, but they had to go back to small numbers, not for any technical reasons, but because people complained that the large process IDs looked ugly in Task Manager. (One customer even asked if something was wrong with his computer.)
I am running out of ideas here. I have a piece of code adapted from http://thetechnofreak.com/technofreak/keylogger-visual-c/ to convert keycodes to unicode chars. It works fine in all situations except when you try to run the 32-bit version from 64-bit Windows. For some reason pKbd->pVkToWcharTable keeps returning NULL. I have tried __ptr64 as well as explicitly specifying SysWOW64 and System32 for the kbd dll path. I have found several items across the internet referring to this exact or very similar problem but I cannot seem to get any of the solutions to work (See: KbdLayerDescriptor returns NULL at 64bit architecture) The following is my test code that was compiled with mingw-32 on Windows XP (gcc -std=c99 Wow64Test.c) and then executed on Windows 7 64-bit. On Windows XP I am getting a valid pointer, however on Windows 7 I am getting NULL.
***Update: So it looks like the problems I am having are due to mingw not implementing __ptr64 correctly as the sizeof operation gives 4 bytes instead of the 8 bytes given by visual studio. So the real solution would be figuring out a way to make the size of KBD_LONG_POINTER dynamic or at least 64-bits but I am not sure if thats possible. Any ideas?
#include <windows.h>
#include <stdio.h>
#define KBD_LONG_POINTER __ptr64
//#define KBD_LONG_POINTER
typedef struct {
BYTE Vk;
BYTE ModBits;
} VK_TO_BIT, *KBD_LONG_POINTER PVK_TO_BIT;
typedef struct {
PVK_TO_BIT pVkToBit;
WORD wMaxModBits;
BYTE ModNumber[];
} MODIFIERS, *KBD_LONG_POINTER PMODIFIERS;
typedef struct _VK_TO_WCHARS1 {
BYTE VirtualKey;
BYTE Attributes;
WCHAR wch[1];
} VK_TO_WCHARS1, *KBD_LONG_POINTER PVK_TO_WCHARS1;
typedef struct _VK_TO_WCHAR_TABLE {
PVK_TO_WCHARS1 pVkToWchars;
BYTE nModifications;
BYTE cbSize;
} VK_TO_WCHAR_TABLE, *KBD_LONG_POINTER PVK_TO_WCHAR_TABLE;
typedef struct {
DWORD dwBoth;
WCHAR wchComposed;
USHORT uFlags;
} DEADKEY, *KBD_LONG_POINTER PDEADKEY;
typedef struct {
BYTE vsc;
WCHAR *KBD_LONG_POINTER pwsz;
} VSC_LPWSTR, *KBD_LONG_POINTER PVSC_LPWSTR;
typedef struct _VSC_VK {
BYTE Vsc;
USHORT Vk;
} VSC_VK, *KBD_LONG_POINTER PVSC_VK;
typedef struct _LIGATURE1 {
BYTE VirtualKey;
WORD ModificationNumber;
WCHAR wch[1];
} LIGATURE1, *KBD_LONG_POINTER PLIGATURE1;
typedef struct tagKbdLayer {
PMODIFIERS pCharModifiers;
PVK_TO_WCHAR_TABLE pVkToWcharTable;
PDEADKEY pDeadKey;
PVSC_LPWSTR pKeyNames;
PVSC_LPWSTR pKeyNamesExt;
WCHAR *KBD_LONG_POINTER *KBD_LONG_POINTER pKeyNamesDead;
USHORT *KBD_LONG_POINTER pusVSCtoVK;
BYTE bMaxVSCtoVK;
PVSC_VK pVSCtoVK_E0;
PVSC_VK pVSCtoVK_E1;
DWORD fLocaleFlags;
BYTE nLgMax;
BYTE cbLgEntry;
PLIGATURE1 pLigature;
DWORD dwType;
DWORD dwSubType;
} KBDTABLES, *KBD_LONG_POINTER PKBDTABLES;
typedef PKBDTABLES(CALLBACK *KbdLayerDescriptor) (VOID);
int main() {
PKBDTABLES pKbd;
HINSTANCE kbdLibrary = NULL;
//"C:\\WINDOWS\\SysWOW64\\KBDUS.DLL"
//"C:\\WINDOWS\\System32\\KBDUS.DLL"
kbdLibrary = LoadLibrary("C:\\WINDOWS\\SysWOW64\\KBDUS.DLL");
KbdLayerDescriptor pKbdLayerDescriptor = (KbdLayerDescriptor) GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if(pKbdLayerDescriptor != NULL) {
pKbd = pKbdLayerDescriptor();
printf("Is Null? %d 0x%X\n", sizeof(pKbd->pVkToWcharTable), pKbd->pVkToWcharTable);
}
FreeLibrary(kbdLibrary);
kbdLibrary = NULL;
}
It might be late for you, but here is a solution for anyone having the same problem. This demo and incomplete explanation helps, but only works in Visual Studio:
http://www.codeproject.com/Articles/439275/Loading-keyboard-layout-KbdLayerDescriptor-in-32-6
The pointers in the structures in kbd.h all have the KBD_LONG_POINTER macro, which is defined as *__ptr64* on 64 bit operating systems. In Visual Studio, this makes the pointers take up 8 bytes instead of the usual 4 of 32 bit programs. Unfortunately in MinGW, *__ptr64* is defined to not do anything.
As written in the linked explanation, the KbdLayerDescriptor function returns pointers differently on 32 bit and 64 bit Windows. The size of pointers seem to depend on the operating system and not on the running program. Actually, the pointers are still 4 bytes on a 64 bit operating system for a 32 bit program, but in VS, the __ptr64 keyword lies that they are not.
For example some structures look like this in kbd.h:
typedef struct {
BYTE Vk;
BYTE ModBits;
} VK_TO_BIT, *KBD_LONG_POINTER PVK_TO_BIT;
typedef struct {
PVK_TO_BIT pVkToBit;
WORD wMaxModBits;
BYTE ModNumber[];
} MODIFIERS, *KBD_LONG_POINTER PMODIFIERS;
This can't work neither in MinGW nor in VS for 32 bit programs on 64 bit Windows. Because the pVkToBit member in MODIFIERS is only 4 bytes without __ptr64. The solution is to forget about KBD_LONG_POINTER (you could even remove them all) and define structures similar to the above. i.e. :
struct VK_TO_BIT64
{
BYTE Vk;
BYTE ModBits;
};
struct MODIFIERS64
{
VK_TO_BIT64 *pVkToBit;
int _align1;
WORD wMaxModBits;
BYTE ModNumber[];
};
(You could use VK_TO_BIT and not define your own VK_TO_BIT64, as they are the same, but having separate definitions help understanding what's going on.)
The member pVkToBit still takes up 4 bytes, but KbdLayerDescriptor pads pointers to 8 bytes on a 64 bit OS, so we have to insert some padding (int _align1).
You have to do the same thing for the other structures in kbd.h. For example this will replace KBDTABLES:
struct WCHARARRAY64
{
WCHAR *str;
int _align1;
};
struct KBDTABLES64
{
MODIFIERS64 *pCharModifiers;
int _align1;
VK_TO_WCHAR_TABLE64 *pVkToWcharTable;
int _align2;
DEADKEY64 *pDeadKey;
int _align3;
VSC_LPWSTR64 *pKeyNames;
int _align4;
VSC_LPWSTR64 *pKeyNamesExt;
int _align5;
WCHARARRAY64 *pKeyNamesDead;
int _align6;
USHORT *pusVSCtoVK;
int _align7;
BYTE bMaxVSCtoVK;
int _align8;
VSC_VK64 *pVSCtoVK_E0;
int _align9;
VSC_VK64 *pVSCtoVK_E1;
int _align10;
DWORD fLocaleFlags;
byte nLgMax;
byte cbLgEntry;
LIGATURE64_1 *pLigature;
int _align11;
DWORD dwType;
DWORD dwSubType;
};
(Notice that the _align8 member does not come after a pointer.)
To use this all, you have to check whether you are running on 64 bit windows with this: http://msdn.microsoft.com/en-us/library/ms684139%28v=vs.85%29.aspx
If not, use the original structures from kbd.h, because the pointers behave correctly. They take up 4 bytes. In case the program is running on a 64 bit OS, use the structures you created. You can achieve it with this:
typedef __int64 (CALLBACK *LayerDescriptor64)(); // Result should be cast to KBDTABLES64.
typedef PKBDTABLES (CALLBACK *LayerDescriptor)(); // This is used on 32 bit OS.
static PKBDTABLES kbdtables = NULL;
static KBDTABLES64 *kbdtables64 = NULL;
And in some initialization function:
if (WindowsIs64Bit()) // Your function that checks the OS version.
{
LayerDescriptor64 KbdLayerDescriptor = (LayerDescriptor64)GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if (KbdLayerDescriptor != NULL)
kbdtables64 = (KBDTABLES64*)KbdLayerDescriptor();
else
kbdtables64 = NULL;
}
else
{
LayerDescriptor KbdLayerDescriptor = (LayerDescriptor)GetProcAddress(kbdLibrary, "KbdLayerDescriptor");
if (KbdLayerDescriptor != NULL)
kbdtables = KbdLayerDescriptor();
else
kbdtables = NULL;
}
This solution does not use __ptr64 at all, and works both in VS and MinGW. The things you have to watch out for are:
The structures should be aligned on 8 byte boundaries. (This is the default in current VS or MinGW, at least for C++.)
Don't define KBD_LONG_POINTER to __ptr64, or remove it from everywhere. Although you are better off not changing kbd.h.
Understand how alignment for structure members work. (I have compiled this as C++ and not C. I'm not sure whether alignment rules would be any different for C.)
Use the correct variable (either kbdtables or kbdtables64) depending on the OS.
This solution is obviously not needed when compiling a 64 bit program.
I am using MPLABx and the HI Tech PICC compiler. My target chip is a PIC16F876. By looking at the pic16f876.h include file, it appears that it should be possible to set the system registers of the chip by referring to them by name.
For example, within the CCP1CON register, bits 0 to 3 set how the CCP and PWM modules work. By looking at the pic16f876.h file, it looks like it should be possible to refer to these 4 bits alone, without change the value of the rest of the CCP1CON register.
However, I have tried to refer to these 4 bits in a variety of ways with no success.
I have tried;
CCP1CON.CCP1M=0xC0; this results in "error: struct/union required
CCP1CON:CCP1M=0xC0; this results in "error: undefined identifier "CCP1M"
but both have failed. I have read through the Hi Tech PICC compiler manual, but cannot see how to do this.
From the pic16f876.h file, it looks to me as though I should be able to refer to these subsets within the system registers by name, as they are defined in the .h file.
Does anyone know how to accomplish this?
Excerpt from pic16f876.h
// Register: CCP1CON
volatile unsigned char CCP1CON # 0x017;
// bit and bitfield definitions
volatile bit CCP1Y # ((unsigned)&CCP1CON*8)+4;
volatile bit CCP1X # ((unsigned)&CCP1CON*8)+5;
volatile bit CCP1M0 # ((unsigned)&CCP1CON*8)+0;
volatile bit CCP1M1 # ((unsigned)&CCP1CON*8)+1;
volatile bit CCP1M2 # ((unsigned)&CCP1CON*8)+2;
volatile bit CCP1M3 # ((unsigned)&CCP1CON*8)+3;
#ifndef _LIB_BUILD
volatile union {
struct {
unsigned CCP1M : 4;
unsigned CCP1Y : 1;
unsigned CCP1X : 1;
};
struct {
unsigned CCP1M0 : 1;
unsigned CCP1M1 : 1;
unsigned CCP1M2 : 1;
unsigned CCP1M3 : 1;
};
} CCP1CONbits # 0x017;
#endif
You need to access the bitfield members through an instance of a struct. In this case, that is CCP1CONbits. Because it is a bitfield, you only need to have the number of significant bits as defined in the bitfield, not the full eight bits in your code.
So:
CCP1CONbits.CCP1M = 0x0c;
Should be the equivalent of what you are trying to do. If you want to set all eight bits at once you can use CCP1CON = 0xc0. That would set the CCP1M bits to 0x0c and all the other bits to zero.
The header you gave also has individual bit symbols, so you could do this too:
CCP1M0 = 1;
CCP1M1 = 1;
CCP1M2 = 0;
CCP1M3 = 0;
Although the bitfield approach is cleaner.