How to split blob into Byte Array In shell script? - bash

I have a blob in postgresql database. Have inserted a C structure into it.
struct temp {
uint64_t a,
uint64_t b,
uint64_t c
};
Now when I write q query in shell for retrieving it.
select resource,.....,blob_column from rtable where rId is=1
I got the result as a blob from database. the result is
x00911ee3561ac801cb0783462586cf01af00000000000000
But now in shell script I need to iterate on this and display the result on console. Tried different things like awe,split , convert_from ,convert function but nothing is helping me.
Can someone tell me how can I read this hex string and get back the integers?

Is this some kind of exersise in programmer-torture? I can't imagine why you'd possibly do this. Not least because your struct-as-a-blob could be subject to padding and alignment that will vary from compiler to compiler and platform to platform. Even then, it'll vary between architectures because of endianness differences. At least you used fixed-width types.
Assuming you only care about little-endian and your compilers don't add any padding or alignment (likely for a struct that's just 3 64-bit fields) it's possible. That doesn't make a great idea.
My preferred approach would be to use some Python code with struct, e.g.
python - "x00911ee3561ac801cb0783462586cf01af00000000000000" <<__END__
import sys
import struct
print "{} {} {}".format(*struct.unpack('#QQQ', sys.argv[1][1:].decode("hex")))
__END__
as this can even handle endianness and packing using appropriate modifiers, and you can easily consume the output in a shell script.
If that's not convenient/suitable, it's also possible in bash, just absolutely horrible. For little-endian, unpadded/packed-unaligned:
To decode each value (adapted from https://stackoverflow.com/a/3678208/398670):
$ x=00911ee3561ac801
$ echo $(( 16#${x:14:2}${x:12:2}${x:10:2}${x:8:2}${x:6:2}${x:4:2}${x:2:2}${x:0:2} ))
so, for the full deal:
x=x00911ee3561ac801cb0783462586cf01af00000000000000
uint64_dec() {
echo $(( 16#${1:14:2}${1:12:2}${1:10:2}${1:8:2}${1:6:2}${1:4:2}${1:2:2}${1:0:2} ))
}
uint64_dec ${x:1:16}
uint64_dec ${x:17:16}
uint64_dec ${x:33:16}
produces:
128381549860000000
130470408871937995
175
Now, I feel dirty and need to go wash. I strongly suggest the following:
CREATE TYPE my_struct AS (a numeric, b numeric, c numeric);
then using my_struct instead of a bytea field. Or just use three numeric columns. You can't use bigint because Pg doesn't have a 64-bit unsigned integer.

Related

Using termios in Swift

Now that we've reached Swift 2.0, I've decided to convert my, as yet unfinished, OS X app to Swift. Making progress but I've run into some issues with using termios and could use some clarification and advice.
The termios struct is treated as a struct in Swift, no surprise there, but what is surprising is that the array of control characters in the struct is now a tuple. I was expecting it to just be an array. As you might imagine it took me a while to figure this out. Working in a Playground if I do:
var settings:termios = termios()
print(settings)
then I get the correct details printed for the struct.
In Obj-C to set the control characters you would use, say,
cfmakeraw(&settings);
settings.c_cc[VMIN] = 1;
where VMIN is a #define equal to 16 in termios.h. In Swift I have to do
cfmakeraw(&settings)
settings.c_cc.16 = 1
which works, but is a bit more opaque. I would prefer to use something along the lines of
settings.c_cc.vim = 1
instead, but can't seem to find any documentation describing the Swift "version" of termios. Does anyone know if the tuple has pre-assigned names for it's elements, or if not, is there a way to assign names after the fact? Should I just create my own tuple with named elements and then assign it to settings.c_cc?
Interestingly, despite the fact that pre-processor directives are not supposed to work in Swift, if I do
print(VMIN)
print(VTIME)
then the correct values are printed and no compiler errors are produced. I'd be interested in any clarification or comments on that. Is it a bug?
The remaining issues have to do with further configuration of the termios.
The definition of cfsetspeed is given as
func cfsetspeed(_: UnsafeMutablePointer<termios>, _: speed_t) -> Int32
and speed_t is typedef'ed as an unsigned long. In Obj-C we'd do
cfsetspeed(&settings, B38400);
but since B38400 is a #define in termios.h we can no longer do that. Has Apple set up replacement global constants for things like this in Swift, and if so, can anyone tell me where they are documented. The alternative seems to be to just plug in the raw values and lose readability, or to create my own versions of the constants previously defined in termios.h. I'm happy to go that route if there isn't a better choice.
Let's start with your second problem, which is easier to solve.
B38400 is available in Swift, it just has the wrong type.
So you have to convert it explicitly:
var settings = termios()
cfsetspeed(&settings, speed_t(B38400))
Your first problem has no "nice" solution that I know of.
Fixed sized arrays are imported to Swift as tuples, and – as far as I know – you cannot address a tuple element with a variable.
However,Swift preserves the memory layout of structures imported from C, as
confirmed by Apple engineer Joe Groff:. Therefore you can take the address of the tuple and “rebind” it to a pointer to the element type:
var settings = termios()
withUnsafeMutablePointer(to: &settings.c_cc) { (tuplePtr) -> Void in
tuplePtr.withMemoryRebound(to: cc_t.self, capacity: MemoryLayout.size(ofValue: settings.c_cc)) {
$0[Int(VMIN)] = 1
}
}
(Code updated for Swift 4+.)

I need a data type big enough to hold pow(80,7)

Im using pow from cmath.h for the power of function. Im using the line of code pow(80,7). Int and long are to small to give me the right value. Im going to use printf or cout to display the result
what data type can I use
Maybe you should have a look at GNU GMP (https://gmplib.org/).
HTH!

run time error '6' Overflow in visual basic 6.0

I am getting a run time error '6' Over Flow in vb 6
The "Overflow" error means that you are trying to put a number into a variable (or property etc), and the data type of the variable doesn't allow numbers that large.
Make sure that numbers used in calculations that are coerced into integers do not have results larger than integers.
What is the type of the data in the database?
My guess is that ADO returns it as either a String or a Decimal, and Decimal values only "fit into" a Variant in VB6.
VB6 has no syntax for a Decimal literal, however you can use something like:
CDec(111010114289#)
... inline, or declare a Const as in:
Private Const BigVal As Currency = 111010114289#
I you have to put a large number in a small variable, like C, check Remove integer bound check in project properties (if you are not compiling as PCode)

Format statement with unknown columns

I am attempting to use fortran to write out a comma-delimited file for import into another commercial package. The issue is that I have an unknown number of data columns. My output needs to look like this:
a_string,a_float,a_different_float,float_array_elem1,float_array_elem2,...,float_array_elemn
which would result in something that might look like this:
L1080,546876.23,4325678.21,300.2,150.125,...,0.125
L1090,563245.1,2356345.21,27.1245,...,0.00983
I have three issues. One, I would prefer the elements to be tightly grouped (variable column width), two, I do not know how to define a variable number of array elements in the format statement, and three, the array elements can span a large range--maybe 12 orders of magnitude. The following code conceptually does what I want, but the variable 'n' and the lack of column-width definition throws an error (of course):
WRITE(50,900) linenames(ii),loc(ii,1:2),recon(ii,1:n)
900 FORMAT(A,',',F,',',F,n(',',F))
(I should note that n is fixed at run-time.) The write statement does what I want it to when I do WRITE(50,*), except that it's width-delimited.
I think this thread almost answered my question, but I got quite confused: SO. Right now I have a shell script with awk fixing the issue, but that solution is...inelegant. I could do some manipulation to make the output a string, and then just write it, but I would rather like to avoid that option if at all possible.
I'm doing this in Fortran 90 but I like to try to keep my code as backwards-compatible as possible.
the format close to what you want is f0.3, this will give no spaces and a fixed number of decimal places. I think if you want to also lop off trailing zeros you'll need to do a good bit of work.
The 'n' in your write statement can be larger than the number of data values, so one (old school) approach is to put a big number there, eg 100000. Modern fortran does have some syntax to specify indefinite repeat, i'm sure someone will offer that up.
----edit
the unlimited repeat is as you might guess an asterisk..and is evideltly "brand new" in f2008
In order to make sure that no space occurs between the entries in your line, you can write them separately in character variables and then print them out using theadjustl() function in fortran:
program csv
implicit none
integer, parameter :: dp = kind(1.0d0)
integer, parameter :: nn = 3
real(dp), parameter :: floatarray(nn) = [ -1.0_dp, -2.0_dp, -3.0_dp ]
integer :: ii
character(30) :: buffer(nn+2), myformat
! Create format string with appropriate number of fields.
write(myformat, "(A,I0,A)") "(A,", nn + 2, "(',',A))"
! You should execute the following lines in a loop for every line you want to output
write(buffer(1), "(F20.2)") 1.0_dp ! a_float
write(buffer(2), "(F20.2)") 2.0_dp ! a_different_float
do ii = 1, nn
write(buffer(2+ii), "(F20.3)") floatarray(ii)
end do
write(*, myformat) "a_string", (trim(adjustl(buffer(ii))), ii = 1, nn + 2)
end program csv
The demonstration above is only for one output line, but you can easily write a loop around the appropriate block to execute it for all your output lines. Also, you can choose different numerical format for the different entries, if you wish.

C++: What would be an appropriate solution to forming a collection of different data types?

I'm writing a command line interpreter and I'm trying to setup formats for individual commands. I have things like the name of the command, the maximum amount of parameters and the minimum amount of parameters. I want to have sort of collection, a sort of prototype of what kind of types the parameters are. My first thought was just to declare a vector without the generics, but then I realized this isn't Java.
Let's say I have a command such as "read test.dat 2"
I would want a structure showing that the typical read command has a string and then an integer.
Any ideas?
I'm not really clear about what you are asking so I may be getting this wrong.
From your description ,it sounds as if you have an abstract notion of commands, and that they have names and an expected structure in terms of parameters. From your description, it sounds like you would want a list of type identifiers and indications on whether they are optional.
Then, you could just instantiate a command object for each command you expect to work with, and add all of them to a collection of commands.
Alternatively, use a map collection to map from command names to the actual command objects.
Each command object can also hold a reference to a handler object to actually execute the command.
Heterogenous Container
If you want to have a container that can store a fixed set of types, you can use one of boost::variant:
typedef boost::variant<std::string, int> optval;
typedef std::vector<optval> options;
Now, you can push_back into the vector either strings or integers, and the variant will note what it contains:
options opts;
opts.push_back(10);
opts.push_back("hello");
You can read about it in its documentation at Boost variant, including how to get the right value out of the variant. You can of course also have a map from argument names to such variants, if you have already set up your command line parsing, and don't need libraries for that anymore:
std::map<std::string, optval> map;
map["--max-foo"] = 10;
map["--title"] = "something fun";
Command line parsing
If you want to parse the command line arguments of your program, you can look into the Boost.Program Options library, which will greatly assist you doing that.
Mostly, however, i end up using the posix getopt function, which can also parse the command line. I recommend you to look into boost program options first, and if you feel it's too heavy, you can look into getopt (see man 3 getopt)
Something like this, perhaps:
enum ParameterType
{
Int,
String
};
struct Command
{
string name;
vector<ParameterType> maxParameters;
int minParameters;
};

Resources