i want to create a file with only 2 bytes and having as a text "00"
when I run the command echo -n ok > myFile to have just 2 bytes in 'myFile' I get 6 bytes after running the ls -l
In fact it shows me in the byte section that I have 6 bytes but I want to just 2 bytes.
Use printf:
printf '%s' ok >myFile
...or more simply (but less reliably, as this doesn't always work right if your ok is instead a string with %s, literal backslashes, etc):
printf ok >myFile
You can use dd utility
dd if=/dev/zero of=twobytes.txt count=2 bs=1
2+0 records in
2+0 records out
2 bytes copied, 0.000128232 s, 15.6 kB/s
cat twobytes.txt | hexdump
0000000 0000
0000002
ls -l twobytes.txt
-rw-r--r-- 1 lmc users 2 Aug 4 17:39 twobytes.txt
This would also work (thanks #CharlesDuffy)
tr -d '\n' <<<'00' > twobytes.txt
Not recommended as echo behavior might differ in some implementation (see comments below)
echo -ne "\060\060" > twobytes.txt
Related
I found a lot of questions/answers related to this issue but I failed to choose what exactly what I need.
How can I generate specified number of integers?
#!/bin/sh
count=$1
# TODO: print $count integers to the console
which prints random integers one at a line.
$ generate.sh 32768
1
32
-47
9
-2
31
...
$
So that I can pipe the output to a file.
sh ./generate.sh 32768 > random_intengers.txt
I'm currently doing this.
#!/bin/sh
count=$1
for i in $(seq $count); do
echo $SRANDOM
done
Is it good enough? Any other more efficient way using od or some?
Integer range is not a problem as long as not bigger than 32 bit, signed will be better
I would use hexdump and /dev/urandom if they are available:
$ count=5
$ hexdump -n $((4*count)) -e '/4 "%d\n"' /dev/urandom > random_integers.txt
$ cat random_integers.txt
535142754
455371601
-323071241
-1154116530
1841746137
Explanation:
-n $((4*count)) reads 4*count bytes from /dev/urandom.
-e '/4 "%d\n"' specifies a format string that uses 4 bytes at a time (/4) and prints them as an integer followed by a newline ("%d\n").
If you have the jot command available you could do this:
$ jot -r 10 0 1000
89
664
188
588
785
426
723
221
833
265
Prints 10 random (-r) numbers between 0 and 1000. And it can also generate decimal numbers:
$ jot -r 10 0 1000.0
189.9
15.6
989.6
139.9
777.0
18.1
285.3
574.8
201.0
312.1
Or signed:
$ jot -r 10 -1000 1000.0
-551.5
-65.0
634.3
-814.6
881.2
-713.7
422.1
390.3
674.2
-750.9
python based solution below
import sys
from random import randrange
count = int(sys.argv[1]) if len(sys.argv) > 1 else 1000
for x in range(0,count):
print(randrange(100000))
I have this function:
function convert_ascii_string_to_decimal {
ascii=$1
unset converted_result
while IFS="" read -r -n 1 char; do
decimal=$(printf '%d' "'$char")
echo $decimal
converted_result="$converted_result $decimal"
done < <(printf %s "$ascii")
converted_result=$(echo $converted_result | xargs) #strip leading and trailing
}
It is meant to take an ascii string variable, loop through every character, and concatenate the ascii decimal representation to a string. However, this while loop seems to ignore null chars, ie characters with ascii 0. I want to be able to read every single ascii there is, including null.
To get all characters of a string as decimal number, you can use hexdump to parse a string:
echo -e "hello \x00world" | hexdump -v -e '1/1 "%d "'
104 101 108 108 111 32 0 119 111 114 108 100 10
This also works for parsing a file:
echo '05 04 03 02 01 00 ff' | xxd -r -ps > file
hexdump --no-squeezing --format '1/1 "%d "' file
5 4 3 2 1 0 255
hexdump explanation:
options -v and --no-squeezing prints all bytes (without skipping duplicated bytes)
options -e and --format allows giving a specific format
format is 1/1 "%d " which means
Iteration count = 1 (process the byte only once)
Byte count = 1 (apply this format for each byte)
Format = "%d" (convert to decimal)
You can't store the null character in a bash variable, which is happening in your script with the $char variable.
I suggest using xxd instead of writing your own script:
echo -ne "some ascii text" | xxd -p
If we echo a null charcter:
$ echo -ne "\0" | xxd -p
00
If I have a file on a file system I can do something like this with dd:
dd if=/my/filewithaheader.bin bs=32k skip=1 | gunzip | tar tvf
however if I try something like this:
./commandthatputsstuffonstdout | dd bs=32k skip=1 | gunzip | tar tvf
I get the error:
dd: 'standard input': cannot skip to specified offset.
How can I fix this, can it be done with dd, or is there another unix command I can use
You could use tail. Say:
./commandthatputsstuffonstdout | tail -c +1025 ...
to skip the first 1024 bytes of output produced by your command.
From man tail:
-c, --bytes=K
output the last K bytes; alternatively, use -c +K to output
bytes starting with the Kth of each file
I've just run into this too, and using the fullblock iflag prevents the short read and subsequent abort.
Example:
gzip -d < ./disk_image.dd.gz | \
dd bs=4M skip=32768 iflag=fullblock,skip_bytes of=./partial_image.dd
Bit late answer but this dd example worked for me.
Create example source file:
$ dd if=/tmp/somefile of=/tmp/test skip=50 bs=100 count=1
Skip 50 bytes and copy 10 bytes after it to test_skip file:
$ dd if=/tmp/test of=/tmp/test_skip skip=50 bs=1 count=10
10+0 records in
10+0 records out
10 bytes (10 B) copied, 8.2091e-05 s, 122 kB/s
Or data from stdin:
cat /tmp/test| dd of=/tmp/stdin bs=1 skip=50 count=1
Verify output:
$ hexdump /tmp/test
0000000 ebf3 e8fd df1b 0aa1 faa3 1fba 1817 1267
0000010 1402 f539 fb69 f263 f319 084b 0b26 1150
0000020 182a f98d 030c e0b0 e47c f13d ef3b 1146
0000030 0b7e 0f72 0e58 f2bd f403 ee95 e529 0567
0000040 f88e 1994 0e83 12e5 11e7 fd4b 032f f4f0
0000050 fc9d 010a 0ab6 06b6 1224 f5cb 01e4 e67a
0000060 ebe0 f1a0
0000064
$ hexdump /tmp/test_skip
0000000 0f72 0e58 f2bd f403 ee95
Source file offset 50 is byte: 0x0f
How can I create a binary file with consequent binary values in Bash?
Like:
hexdump testfile
0000000 0100 0302 0504 0706 0908 0b0a 0d0c 0f0e
0000010 1110 1312 1514 1716 1918 1b1a 1d1c 1f1e
0000020 2120 2322 2524 2726 2928 2b2a 2d2c 2f2e
0000030 ....
In C, I do:
fd = open("testfile", O_RDWR | O_CREAT);
for (i=0; i< CONTENT_SIZE; i++)
{
testBufOut[i] = i;
}
num_bytes_written = write(fd, testBufOut, CONTENT_SIZE);
close (fd);
This is what I wanted:
#! /bin/bash
i=0
while [ $i -lt 256 ]; do
h=$(printf "%.2X\n" $i)
echo "$h"| xxd -r -p
i=$((i-1))
done
There's only one byte you cannot pass as an argument in a Bash command line: 0
For any other value, you can just redirect it. It's safe.
echo -n $'\x01' > binary.dat
echo -n $'\x02' >> binary.dat
...
For the value 0, there's another way to output it to a file
dd if=/dev/zero of=binary.dat bs=1c count=1
To append it to file, use
dd if=/dev/zero oflag=append conv=notrunc of=binary.dat bs=1c count=1
Take a look at xxd:
xxd: creates a hex dump of a given file or standard input. It can
also
convert a hex dump back to its original binary form.
If you don't mind to not use an existing command and want to describe you data in a text file, you can use binmake. That is a C++ program that you can compile and use like following:
First get and compile binmake (the binary will be in bin/):
git clone https://github.com/dadadel/binmake
cd binmake
make
Create your text file file.txt:
big-endian
00010203
04050607
# Separated bytes not concerned by endianness
08 09 0a 0b 0c 0d 0e 0f
Generate your binary file file.bin:
./binmake file.txt file.bin
hexdump file.bin
0000000 0100 0302 0504 0706 0908 0b0a 0d0c 0f0e
0000008
Note: you can also use it with standard input and standard output.
Use the below command,
i=0; while [ $i -lt 256 ]; do echo -en '\x'$(printf "%0x" $i)'' >> binary.dat; i=$((i+1)); done
I would love some help with a Bash script loop that will show all the differences between two binary files, using just
cmp file1 file2
It only shows the first change I would like to use cmp because it gives a offset an a line number of where each change is but if you think there's a better command I'm open to it :) thanks
I think cmp -l file1 file2 might do what you want. From the manpage:
-l --verbose
Output byte numbers and values of all differing bytes.
The output is a table of the offset, the byte value in file1 and the value in file2 for all differing bytes. It looks like this:
4531 66 63
4532 63 65
4533 64 67
4580 72 40
4581 40 55
[...]
So the first difference is at offset 4531, where file1's decimal octal byte value is 66 and file2's is 63.
Method that works for single byte addition/deletion
diff <(od -An -tx1 -w1 -v file1) \
<(od -An -tx1 -w1 -v file2)
Generate a test case with a single removal of byte 64:
for i in `seq 128`; do printf "%02x" "$i"; done | xxd -r -p > file1
for i in `seq 128`; do if [ "$i" -ne 64 ]; then printf "%02x" $i; fi; done | xxd -r -p > file2
Output:
64d63
< 40
If you also want to see the ASCII version of the character:
bdiff() (
f() (
od -An -tx1c -w1 -v "$1" | paste -d '' - -
)
diff <(f "$1") <(f "$2")
)
bdiff file1 file2
Output:
64d63
< 40 #
Tested on Ubuntu 16.04.
I prefer od over xxd because:
it is POSIX, xxd is not (comes with Vim)
has the -An to remove the address column without awk.
Command explanation:
-An removes the address column. This is important otherwise all lines would differ after a byte addition / removal.
-w1 puts one byte per line, so that diff can consume it. It is crucial to have one byte per line, or else every line after a deletion would become out of phase and differ. Unfortunately, this is not POSIX, but present in GNU.
-tx1 is the representation you want, change to any possible value, as long as you keep 1 byte per line.
-v prevents asterisk repetition abbreviation * which might interfere with the diff
paste -d '' - - joins every two lines. We need it because the hex and ASCII go into separate adjacent lines. Taken from: Concatenating every other line with the next
we use parenthesis () to define bdiff instead of {} to limit the scope of the inner function f, see also: How to define a function inside another function in Bash?
See also:
https://superuser.com/questions/125376/how-do-i-compare-binary-files-in-linux
https://unix.stackexchange.com/questions/59849/diff-binary-files-of-different-sizes
The more efficient workaround I've found is to translate binary files to some form of text using od.
Then any flavour of diff works fine.