Explanation of a macro in kernel - linux-kernel

In kernel 2.4.37, there is a macro in page.h like this:
struct page *mem_map;
struct page *page;
#define VALID_PAGE(page) ((page - mem_map) < max_mapnr)
I know mem_map is an array of struct page, page is a struct, so what does page - mem_map mean?

It will compute the index of corresponding page in mem_map array means which number of page it is in mem_map array, let say it as pfn or page frame number for linux (linux assumes that mem_map array starts with 0th pfn to the max pfn) , adding a PHYS_PFN_OFFSET to pfn will give you the actual physical page frame in your memory map.
__page_to_pfn
max_mapnr is the limit of maximum number of mapped pages or maximum page frame number.
set_max_mapnr
I hope it clears your doubts.

Humm, I'm not sure but maybe a pointer adresses comparaison ?
I mean, if one of them is a array and it's not dereferenced the operations are apply on adresses I suppose.
Edit: (precision)
So, in this case I think this operation is for check if "page" is in range of the adresses array "mem_map".
We can represent like this: Graphic representation
Utility of Macro:
So, "mem_map" is adresses of the begin of array, suppose: 0x0...5.
The size of "mem_map"(max_mapnr) array is: 5.
We want to know if "page" adresses is in the range of "mem_map" array.
True Case:
Suppose "page" is in "mem_map", 2e element. We can suppose his adresses is something like: 0x0...7;
Now we do operation: ((0x0...7 - 0x0...5) < 5).
We obtain 2. So "page" adresse is in mem_map.
False Case:
Otherwise if "page" is out of the array (0x0...D): We the result will be 8. So, 8 is not less than "max_mapnr"(5). So this page is not in the "mem_map" array.
And if the adresses is bellow the array adresse (0x0...2):
The result of ((0x0...2 - 0x0...2)) will be a negative value. And in that case they comparaison with "max_mapnr"(unsigned long) is not possible.
I found this topic explain why better than me:
Signed/unsigned comparisons
So for resume:
You canno't do operations between negative(signed) and unsigned value in C cause he cast them automaticaly. In others terms, when you do (-3 - U_nbr), it's same if you do: (((unsigned)-3) - U_nbr). And in option, normaly if you compile with gcc -Wall flags, and you don't cast manually your value you will have an compilation Warning message.
For testing I tried to run this code:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
int main(void)
{
unsigned long test = 0x0000F;
unsigned long test2 = 0x0000A;
unsigned long weird = 0x00002;
char* pt1 = "This is first test string !";
char* pt2 = "This is a test string";
printf("Try to make operation on two unsigned long result must be 5: %lu\n", (test - test2));
printf("Try to make operation between unsigned long result must be negative, so he will be cast: %lu\n", (weird - test2));
printf("Let's try the same with real adresses: %lu\n", (pt2 - pt1));
printf("And this is what happens with negative value: %lu\n", (pt1 - pt2));
printf("For be sure, this is the lenght of string 1. %lu\n", strlen(pt1));
return (0);
}
The ouput is:
Try to make operation on two unsigned long result must be 5: 5
Try to make operation between unsigned long result must be negative, so he will be cast: 18446744073709551608
Let's try the same with real adresses: 28
And this is what happens with negative value: 18446744073709551588
For be sure, this is the lenght of string 1. 27
So, as we can see, the negative value is casted in Unsigned long and return a overflowed one. And if you make this comparaison with max_mapnr you will see he is "out of range".
Thank's to AnshuMan Gupta for the "weird case".

Related

How to change a boost::multiprecision::cpp_int from big endian to little endian

I have a boost::multiprecision::cpp_int in big endian and have to change it to little endian. How can I do that? I tried with boost::endian::conversion but that did not work.
boost::multiprecision::cpp_int bigEndianInt("0xe35fa931a0000*);
boost::multiprecision::cpp_int littleEndianInt;
littleEndianIn = boost::endian::endian_reverse(m_cppInt);
The memory layout of boost multi-precision types is implementation detail. So you cannot assume much about it anyways (they're not supposed to be bitwise serializable).
Just read a random section of the docs:
MinBits
Determines the number of Bits to store directly within the object before resorting to dynamic memory allocation. When zero, this field is determined automatically based on how many bits can be stored in union with the dynamic storage header: setting a larger value may improve performance as larger integer values will be stored internally before memory allocation is required.
It's not immediately clear that you have any chance at some level of "normal int behaviour" in memory layout. The only exception would be when MinBits==MaxBits.
Indeed, we can static_assert that the size of cpp_int with such backend configs match the corresponding byte-sizes.
It turns out that there's even a promising tag in the backend base-class to indicate "triviality" (this is truly promising): trivial_tag, so let's use it:
Live On Coliru
#include <boost/multiprecision/cpp_int.hpp>
namespace mp = boost::multiprecision;
template <int bits> using simple_be =
mp::cpp_int_backend<bits, bits, mp::unsigned_magnitude>;
template <int bits> using my_int =
mp::number<simple_be<bits>, mp::et_off>;
using my_int8_t = my_int<8>;
using my_int16_t = my_int<16>;
using my_int32_t = my_int<32>;
using my_int64_t = my_int<64>;
using my_int128_t = my_int<128>;
using my_int192_t = my_int<192>;
using my_int256_t = my_int<256>;
template <typename Num>
constexpr bool is_trivial_v = Num::backend_type::trivial_tag::value;
int main() {
static_assert(sizeof(my_int8_t) == 1);
static_assert(sizeof(my_int16_t) == 2);
static_assert(sizeof(my_int32_t) == 4);
static_assert(sizeof(my_int64_t) == 8);
static_assert(sizeof(my_int128_t) == 16);
static_assert(is_trivial_v<my_int8_t>);
static_assert(is_trivial_v<my_int16_t>);
static_assert(is_trivial_v<my_int32_t>);
static_assert(is_trivial_v<my_int64_t>);
static_assert(is_trivial_v<my_int128_t>);
// however it doesn't scale
static_assert(sizeof(my_int192_t) != 24);
static_assert(sizeof(my_int256_t) != 32);
static_assert(not is_trivial_v<my_int192_t>);
static_assert(not is_trivial_v<my_int256_t>);
}
Conluding: you can have trivial int representation up to a certain point, after which you get the allocator-based dynamic-limb implementation no matter what.
Note that using unsigned_packed instead of unsigned_magnitude representation never leads to a trivial backend implementation.
Note that triviality might depend on compiler/platform choices (it's likely that cpp_128_t uses some builtin compiler/standard library support on GCC, e.g.)
Given this, you MIGHT be able to pull of what you wanted to do with hacks IF your backend configuration support triviality. Sadly I think it requires you to manually overload endian_reverse for 128 bits case, because the GCC builtins do not have __builtin_bswap128, nor does Boost Endian define things.
I'd suggest working off the information here How to make GCC generate bswap instruction for big endian store without builtins?
Final Demo (not complete)
#include <boost/multiprecision/cpp_int.hpp>
#include <boost/endian/buffers.hpp>
namespace mp = boost::multiprecision;
namespace be = boost::endian;
template <int bits> void check() {
using T = mp::number<mp::cpp_int_backend<bits, bits, mp::unsigned_magnitude>, mp::et_off>;
static_assert(sizeof(T) == bits/8);
static_assert(T::backend_type::trivial_tag::value);
be::endian_buffer<be::order::big, T, bits, be::align::no> buf;
buf = T("0x0102030405060708090a0b0c0d0e0f00");
std::cout << std::hex << buf.value() << "\n";
}
int main() {
check<128>();
}
(Changing be::order::big to be::order::native obviously makes it compile. The other way to complete it would be to have an ADL accessible overload for endian_reverse for your int type.)
This is both trivial and in the general case unanswerable, let me explain:
For a general N-bit integer, where N is a large number, there is unlikely to be any well defined byte order, indeed even for 64 and 128 bit integers there are more than 2 possible orders in use: https://en.wikipedia.org/wiki/Endianness#Middle-endian.
On any platform, with any native endianness you can always extract the bytes of a cpp_int, the first example here: https://www.boost.org/doc/libs/1_73_0/libs/multiprecision/doc/html/boost_multiprecision/tut/import_export.html#boost_multiprecision.tut.import_export.examples shows you how. When exporting bytes like this, they are always most significant byte first, so you can subsequently rearrange them how you wish. You should not however, rearrange them and load them back into a cpp_int as the class won't know what to do with the result!
If you know that the value is small enough to fit into a native integer type, then you can simply cast to the native integer and use a system API on the result. As in endian_reverse(static_cast<int64_t>(my_cpp_int)). Again, don't assign the result back into a cpp_int as it requires native byte order.
If you wish to check whether a value is small enough to fit in an N-bit integer for the approach above, you can use the msb function, which returns the index of the most significant bit in the cpp_int, add one to that to obtain the number of bits used, and filter out the zero case and the code looks like:
unsigned bits_used = my_cpp_int.is_zero() ? 0 : msb(my_cpp_int) + 1;
Note that all of the above use completely portable code - no hacking of the underlying implementation is required.

Why the following code prints garbage values for input strings greater than 128 bytes?

This is a problem of codechef that I recently came across. The answer seems to be right for every test case where the value of input string is less than 128 bytes as it is passing a couple of test cases. For every value greater than 128 bytes it is printing out a large value which seems to be a garbage value.
std::string str;
std::cin>>str;
vector<pair<char,int>> v;
v.push_back(make_pair('C',0));
v.push_back(make_pair('H',0));
v.push_back(make_pair('E',0));
v.push_back(make_pair('F',0));
int i=0;
while(1)
{
if(str[i]=='C')
v['C'].second++;
else if (str[i]=='H')
{
v['H'].second++;
v['C'].second--;
}
else if (str[i]=='E')
{
v['E'].second++;
v['C'].second--;
}
else if (str[i]=='F')
v['F'].second++;
else
break;
i++;
Even enclosing the same code within
/*reading the string values from a file and not console*/
std::string input;
std::ifstream infile("input.txt");
while(getline(infile,input))
{
istringstream in(input);
string str;
in>>str;
/* above code goes here */
}
generates the same result. I am not looking for any solution(s) or hint(s) to get to the right answer as I want to test the correctness of my algorithm. But I want to know why this happens as I am new to vector containers`.
-Regards.
if(str[i]=='C')
v['C'].second++;
You're modifying v[67]
... which is not contained in your vector, and thus either invalid memory or uninitialized
You seem to be trying to use a vector as an associative array. There is already such a structure in C++: a std::map. Use that instead.
With using this v['C'] you actually access the 67th (if 'A' is 65 from ASCII) element of a container having only 4 items. Depending on compiler and mode (debug vs release) you get undefined behavior for the code.
What you probably wanted to use was map i.e. map<char,int> v; instead of vector<pair<char,int>> v; and simple v['C']++; instead of v['C'].second++;

Crash when casting the result of arc4random() to Int

I've written a simple Bag class. A Bag is filled with a fixed ratio of Temperature enums. It allows you to grab one at random and automatically refills itself when empty. It looks like this:
class Bag {
var items = Temperature[]()
init () {
refill()
}
func grab()-> Temperature {
if items.isEmpty {
refill()
}
var i = Int(arc4random()) % items.count
return items.removeAtIndex(i)
}
func refill() {
items.append(.Normal)
items.append(.Hot)
items.append(.Hot)
items.append(.Cold)
items.append(.Cold)
}
}
The Temperature enum looks like this:
enum Temperature: Int {
case Normal, Hot, Cold
}
My GameScene:SKScene has a constant instance property bag:Bag. (I've tried with a variable as well.) When I need a new temperature I call bag.grab(), once in didMoveToView and when appropriate in touchesEnded.
Randomly this call crashes on the if items.isEmpty line in Bag.grab(). The error is EXC_BAD_INSTRUCTION. Checking the debugger shows items is size=1 and [0] = (AppName.Temperature) <invalid> (0x10).
Edit Looks like I don't understand the debugger info. Even valid arrays show size=1 and unrelated values for [0] =. So no help there.
I can't get it to crash isolated in a Playground. It's probably something obvious but I'm stumped.
Function arc4random returns an UInt32. If you get a value higher than Int.max, the Int(...) cast will crash.
Using
Int(arc4random_uniform(UInt32(items.count)))
should be a better solution.
(Blame the strange crash messages in the Alpha version...)
I found that the best way to solve this is by using rand() instead of arc4random()
the code, in your case, could be:
var i = Int(rand()) % items.count
This method will generate a random Int value between the given minimum and maximum
func randomInt(min: Int, max:Int) -> Int {
return min + Int(arc4random_uniform(UInt32(max - min + 1)))
}
The crash that you were experiencing is due to the fact that Swift detected a type inconsistency at runtime.
Since Int != UInt32 you will have to first type cast the input argument of arc4random_uniform before you can compute the random number.
Swift doesn't allow to cast from one integer type to another if the result of the cast doesn't fit. E.g. the following code will work okay:
let x = 32
let y = UInt8(x)
Why? Because 32 is a possible value for an int of type UInt8. But the following code will fail:
let x = 332
let y = UInt8(x)
That's because you cannot assign 332 to an unsigned 8 bit int type, it can only take values 0 to 255 and nothing else.
When you do casts in C, the int is simply truncated, which may be unexpected or undesired, as the programmer may not be aware that truncation may take place. So Swift handles things a bit different here. It will allow such kind of casts as long as no truncation takes place but if there is truncation, you get a runtime exception. If you think truncation is okay, then you must do the truncation yourself to let Swift know that this is intended behavior, otherwise Swift must assume that is accidental behavior.
This is even documented (documentation of UnsignedInteger):
Convert from Swift's widest unsigned integer type,
trapping on overflow.
And what you see is the "overflow trapping", which is poorly done as, of course, one could have made that trap actually explain what's going on.
Assuming that items never has more than 2^32 elements (a bit more than 4 billion), the following code is safe:
var i = Int(arc4random() % UInt32(items.count))
If it can have more than 2^32 elements, you get another problem anyway as then you need a different random number function that produces random numbers beyond 2^32.
This crash is only possible on 32-bit systems. Int changes between 32-bits (Int32) and 64-bits (Int64) depending on the device architecture (see the docs).
UInt32's max is 2^32 − 1. Int64's max is 2^63 − 1, so Int64 can easily handle UInt32.max. However, Int32's max is 2^31 − 1, which means UInt32 can handle numbers greater than Int32 can, and trying to create an Int32 from a number greater than 2^31-1 will create an overflow.
I confirmed this by trying to compile the line Int(UInt32.max). On the simulators and newer devices, this compiles just fine. But I connected my old iPod Touch (32-bit device) and got this compiler error:
Integer overflows when converted from UInt32 to Int
Xcode won't even compile this line for 32-bit devices, which is likely the crash that is happening at runtime. Many of the other answers in this post are good solutions, so I won't add or copy those. I just felt that this question was missing a detailed explanation of what was going on.
This will automatically create a random Int for you:
var i = random() % items.count
i is of Int type, so no conversion necessary!
You can use
Int(rand())
To prevent same random numbers when the app starts, you can call srand()
srand(UInt32(NSDate().timeIntervalSinceReferenceDate))
let randomNumber: Int = Int(rand()) % items.count

How to compute SHA1 of an array in Linux kernel

I'm trying to compute SHA1 of an integer array in the Linux kernel. I have gone through crypto.c/crypto.h and security/integrity/ima/ima_crypto.c but I can't figure out how to init and then update the SHA1 computer. Can someone point me to a tutorial or guide on how to go about doing this?
There's a pretty good introduction to the linux cryptography api in Documentation/crypto/api-intro.txt. Also check out fs/ecryptfs/crypto.c for a real-life example of how the functions are used.
Here's a quick summary though to get you start:
Step 1: Declaration
Create some local variables:
struct scatterlist sg;
struct hash_desc desc;
char *plaintext = "plaintext goes here";
size_t len = strlen(plaintext);
u8 hashval[20];
A struct scatterlist is used to hold your plaintext in a format the crypto.h functions can understand, while a struct hash_desc is used to configure the hashing.
The variable plaintext holds our plaintext string, while hashval will hold the hash of our plaintext.
Finally, len holds the length the plaintext string.
Note that while I'm using ASCII plaintext in this example, you can pass an integer array as well -- just store the total memory size in len and replace every instance of plaintext with your integer array:
int myarr[4] = { 1, 3, 3, 7 };
size_t len = sizeof(myarr);
Be careful though: an int element generally has a size greater than a byte, so storing integer values in an int array won't have the same internal representation as a char array -- you may end up with null bytes as padding in between values.
Furthermore, if your intention is to hash the ASCII representation of your integers, you will have to first convert the values in your array to a string character sequence (perhaps using sprintf).
Step 2: Initialization
Initialize sg and desc:
sg_init_one(&sg, plaintext, len);
desc.tfm = crypto_alloc_hash("sha1", 0, CRYPTO_ALG_ASYNC);
Notice that "sha1" is passed to crypto_alloc_hash; this can be set to "md5" for MD5 hashing, or any other supported string in order to use the respective hashing method.
Step 3: Hashing
Now perform the hashing with three function calls:
crypto_hash_init(&desc);
crypto_hash_update(&desc, &sg, len);
crypto_hash_final(&desc, hashval);
crypto_hash_init configures the hashing engine according to the supplied struct hash_desc.
crypto_hash_update performs the actual hashing method on the plaintext.
Finally, crypto_hash_final copies the hash to a character array.
Step 4: Cleanup
Free allocated memory held by desc.tfm:
crypto_free_hash(desc.tfm);
See also
how to use CryptoAPI in the linux kernel 2.6

What is the meaning of the __attribute__ vector_size(16)?

I got a C program were I saw the __attribute__ keyword for the 1st time. It seems like it is a GNU keyword. In GCC's this page, they explain its use with the (vector_size(16)) attribute, saying:
int foo __attribute__ ((vector_size (16)));
causes the compiler to set the mode for foo, to be 16 bytes, divided into int sized units. Assuming a 32-bit int (a vector of 4 units of 4 bytes), the corresponding mode of foo will be V4SI.
What does this mean? Is foo now declared as a 4-element array of ints? If so, what is wrong with just:
int foo[4];
?
It's for use with SIMD vectorization. (No, it doesn't make foo an array.)
It's documented here.
No, foo isn't be declared as an array. In this statement the int type specifies the base type, while the attribute specifies the vector size for the variable, measured in bytes.

Resources