Bash: substitute some bytes in a binary file - bash

I have a binary file zero.bin which contains 10 bytes of 0x00, and a data file data.bin which contains 5 bytes of 0x01. I want to substitute the first 5 bytes of the zero.bin with data.bin. I have tried
dd if=data.bin of=zero.bin bs=1 count=5
but, the zero.bin is truncated, finally it becomes 5 bytes of 0x01. I want to keep the tailing 5 bytes of 0x00.

No problem, just add conv=notrunc:
dd if=data.bin of=zero.bin bs=1 count=5 conv=notrunc

You have half of the solution; do that into a temporary file tmp.bin instead of zero.bin, then
dd if=zero.bin bs=1 seek=5 skip=5 of=tmp.bin
mv zero.bin old.bin # paranoia
mv tmp.bin zero.bin

Don't get stuck on using dd(1). There are other tools, eg:
(cat data.bin && tail -c +5 zero.bin) > updated.bin

Related

Why does "echo -n ok >file" write a 6-byte file?

i want to create a file with only 2 bytes and having as a text "00"
when I run the command echo -n ok > myFile to have just 2 bytes in 'myFile' I get 6 bytes after running the ls -l
In fact it shows me in the byte section that I have 6 bytes but I want to just 2 bytes.
Use printf:
printf '%s' ok >myFile
...or more simply (but less reliably, as this doesn't always work right if your ok is instead a string with %s, literal backslashes, etc):
printf ok >myFile
You can use dd utility
dd if=/dev/zero of=twobytes.txt count=2 bs=1
2+0 records in
2+0 records out
2 bytes copied, 0.000128232 s, 15.6 kB/s
cat twobytes.txt | hexdump
0000000 0000
0000002
ls -l twobytes.txt
-rw-r--r-- 1 lmc users 2 Aug 4 17:39 twobytes.txt
This would also work (thanks #CharlesDuffy)
tr -d '\n' <<<'00' > twobytes.txt
Not recommended as echo behavior might differ in some implementation (see comments below)
echo -ne "\060\060" > twobytes.txt

bash script: Appending bytes from an input file to a preexisting file at a given byte offset location

What I am trying to accomplish:
I have a file that I need to copy certain bytes from a certain location
and append them to a file at a given location of that file.
I am thinking something along this lines:
xxd -s $startOffset -l $numBytes inFile | dd of=fileToModify seek=$location conv=notrunc
I have this as well but it will only work for appending at the beginning of a file.
read -p "Enter target file :> " targetFile
read -p "Enter source file to append at the end of target file :> " inputFile
dd if=$inputFile of=$targetFile oflag=append conv=notrunc
Thank you in advance!
contents of first file
$ cat first
fskasfdklsgdfksdjhgf sadjfsdjfhf
dsfghkasdfg sadfhsdfh hskdjfksdfgkfg
jhfksjdafhksdjfh
ksdjhfsdjfh
contents of sceond file
$ cat second
jfhasjdhfjskdhf dshfjsdfh3821349832749832]
87348732642364
]yfisdfhshf936494
sdfisdfsdfsa;dlf
9346934623984
contents of shell script
$ cat cppaste.sh
dd if=$1 of=$2 bs=1 count=$3 status=noxfer
dd if=$4 of=$2 bs=1 seek=$3 status=noxfer
finsize=$(stat -c%s $2)
dd if=$1 of=$2 bs=1 skip=$3 seek=$finsize oflag=append status=noxfer
executing shell script with proper arguments
$ bash cppaste.sh first third 10 second
10+0 records in
10+0 records out
107+0 records in
107+0 records out
92+0 records in
92+0 records out
contents of the resultant file
$ cat third
fskasfdklsjfhasjdhfjskdhf dshfjsdfh3821349832749832]
87348732642364
]yfisdfhshf936494
sdfisdfsdfsa;dlf
9346934623984
gdfksdjhgf sadjfsdjfhf
dsfghkasdfg sadfhsdfh hskdjfksdfgkfg
jhfksjdafhksdjfh
ksdjhfsdjfh
Try this:
# copy certain bytes from a certain location
file=$1
certainlocation=$2
certainbytes=$3
# Append them to a file at a given location of that file
givenlocation=$4
dd if=$file of=$file iflag=skip_bytes oflag=seek_bytes,append conv=notrunc skip=$certainlocation seek=$givenlocation count=1 bs=$certainbytes
Usage:
> printf "1\n2\n3\n4\n" > /tmp/1; ./1.sh /tmp/1 4 2 2; cat /tmp/1;
1+0 records in
1+0 records out
2 bytes copied, 0.000378992 s, 5.3 kB/s
1
2
3
4
3
{
dd if=inFile iflag=count_bytes count="$targetByteLocation" status=none
cat -- "$fileToAppend"
dd if=inFile iflag=skip_bytes skip="$targetByteLocation" status=none
} >outFile

Hex Editing From A Bash File

I need to know how to hex edit from a bash file. For example,
hedit 0x0 A
This would write A to offset 0x0
You should be able to use dd to overwrite parts of a file like this:
printf '\x0a' | dd of=filetopatch bs=1 seek=0 count=1 conv=notrunc
The meaning of the arguments are:
of=filetopatch : the file to patch (the output file)
bs=1 : change 1 byte at a time
seek=0 : move to offset 0 (decimal) before writing
count=1 : copy only 1 input block
conv=notrunc : do not truncate the output after the edit

skip the first 32k of stdin with dd?

If I have a file on a file system I can do something like this with dd:
dd if=/my/filewithaheader.bin bs=32k skip=1 | gunzip | tar tvf
however if I try something like this:
./commandthatputsstuffonstdout | dd bs=32k skip=1 | gunzip | tar tvf
I get the error:
dd: 'standard input': cannot skip to specified offset.
How can I fix this, can it be done with dd, or is there another unix command I can use
You could use tail. Say:
./commandthatputsstuffonstdout | tail -c +1025 ...
to skip the first 1024 bytes of output produced by your command.
From man tail:
-c, --bytes=K
output the last K bytes; alternatively, use -c +K to output
bytes starting with the Kth of each file
I've just run into this too, and using the fullblock iflag prevents the short read and subsequent abort.
Example:
gzip -d < ./disk_image.dd.gz | \
dd bs=4M skip=32768 iflag=fullblock,skip_bytes of=./partial_image.dd
Bit late answer but this dd example worked for me.
Create example source file:
$ dd if=/tmp/somefile of=/tmp/test skip=50 bs=100 count=1
Skip 50 bytes and copy 10 bytes after it to test_skip file:
$ dd if=/tmp/test of=/tmp/test_skip skip=50 bs=1 count=10
10+0 records in
10+0 records out
10 bytes (10 B) copied, 8.2091e-05 s, 122 kB/s
Or data from stdin:
cat /tmp/test| dd of=/tmp/stdin bs=1 skip=50 count=1
Verify output:
$ hexdump /tmp/test
0000000 ebf3 e8fd df1b 0aa1 faa3 1fba 1817 1267
0000010 1402 f539 fb69 f263 f319 084b 0b26 1150
0000020 182a f98d 030c e0b0 e47c f13d ef3b 1146
0000030 0b7e 0f72 0e58 f2bd f403 ee95 e529 0567
0000040 f88e 1994 0e83 12e5 11e7 fd4b 032f f4f0
0000050 fc9d 010a 0ab6 06b6 1224 f5cb 01e4 e67a
0000060 ebe0 f1a0
0000064
$ hexdump /tmp/test_skip
0000000 0f72 0e58 f2bd f403 ee95
Source file offset 50 is byte: 0x0f

Random generation of variable sized test files

Here is script i am planning to use to generate 500 test files populated with random data.
for((counter=1;counter<=500;counter++));
do
echo Creating file$counter;
dd bs=1M count=10 if=/dev/urandom of=file$counter;
done
But what i need the script to do is make those 500 files to be of variable size as in let say between 1M and 10M; ie, file1=1M, file2=10M, file3=9M etc …
any help?
This will generate 500 files each containing between 1 and 10 megabytes of random bytes.
#!/bin/bash
max=10 # number of megabytes
for ((counter=1; counter<=500; counter++))
do
echo Creating file$counter
dd bs=1M count=$(($RANDOM%max + 1)) if=/dev/urandom of=file$counter
done
The second line could instead be:
for counter in {1..500}
set MAX=10
for((counter=1;counter<=500;counter++));
do
echo "Creating file$counter"
dd bs=$(( ($RANDOM%$MAX)+1 ))M count=10 if=/dev/urandom of=file$counter
done
Try $((1+$RANDOM%$MAX))

Resources