I took a look into the iso images (ISO 9660) of small Linux distributions. I found 16 empty sectors, followed by a sector describing the Primary Volume Description. Next sector is commonly a Boot Record containing only descriptive information such as System and Version Identifier and a little endian integer inside the unspecified byterange. Then comes the Supplementary Volume Descriptor and finally the Volume Descriptor Set Terminator.
I only guess it's a little endian integer in the Boot Record. I found no more Information about this. In all the images I used was the little endian integer smaller than the value for Sector Count from the Primary Volume Descriptor, so I further guess it's pointing to a sector inside the Image. Could someone provide more detailed informations about this?
The "El Torito Bootable CD-ROM Format Specification" describes the format of bootable CDs.
Related
I'm trying to understand the Arduino bootloader. I've found some intro here: Arduino Bootloader, but I need more details and because of it I'm asking for help.
During my research i found a start point: OK, Arduino (atmega family) has a specific block of flash dedicated to bootloader. Once the mcu has a bootloader, it can download the new program via serial and store it on flash, at address 0x00.
Let's asign the atmega328p for this question.
#1 - If you take a look at datasheet page 343, you will see a table, showing some info about the bootloader size:
By this table I understood: if i set BOOTSZ1/0 to 0/0, I can have a 2K bootloader and it will be stored in flash stack: 0x3800 ~ 0x3FFF.
#2 - If you open the hex file of ATMEGA328_BOOTLOADER generated by Arduino, you will see the bootloader stored at:
:10**7800**000C94343C0C94513C0C94513C0C94513CE1
to
:10**7FF0**00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF91
If you consider 7FF0 - 7800 you will get 7F0 (2K bytes of program)
#3 - If you open the makefile (C:\Program Files (x86)\Arduino\hardware\arduino\avr\bootloaders\atmega) you will see this argument for atmega328:
atmega328: LDSECTION = --section-start=.text=0x7800
Where 0x7800 matches the hexa file from bootloader.
Questions:
1- Why the datasheet tells me that I have an special place for bootloader and the makefile of Arduino force it to be stored in a different place?
2- What means the line of an hexa file?
:10E000000D9489F10D94B2F10D94B2F10D94B2F129
:10 : (?)
E000 : Address
00 : (?)
0D9489F10D94B2F10D94B2F10D94B2F1 : data (?)
29 : CRC (?)
First of all, I'm not sure we have the same atmel datasheet; your table was 27-7 on my one, but anyway the data is the same.
Disclaimer: I'm not 100% sure about this, but I'm pretty confident.
I think that the "problem" here is that "words". Reading in the overview of the core they wrote
Most AVR instructions have a single 16-bit word format. Every program memory address contains a 16- or 32-bit instruction
This means that instructions are 2 byte wide. So your address 0x3800 corresponds to 0x7000 byte from the start of the memory, while the end of the bootloader 0x3FFF corresponds to the bytes 0x7FFE and 0x7FFF.
This is confirmed by the memory size: 0x7FFF corresponds to 32k, which is the available flash. Since the bootloader is at the end of the memory (as the pictures from the DS imply), this means that 0x3FFF should be the last memory cell.
EDIT:
When I answered, I did not read the second part of the question. The format is the so-called Intel Binary format. See Wikipedia for the detailed information. Here is a brief summary.
You correctly identified the field division:
: : Start code
10 : Number of data bytes (in this case: 16)
E000 : Starting address of the data bytes
00 : Record type (in this case: data)
0D9489F10D94B2F10D94B2F10D94B2F1 : data payload (in this case: 16 bytes)
29 : CRC
The record type can be
00: data
01: end of file
02: extended segment address
03: start segment address
04: extended linear address
05: start linear address
The CRC is obtained summing all the bytes (starting from the length one, so the very beginning), then getting the LSB and finally making the 2 complement. In this case the sum is 2519, which is 0x9D7 in hex. The lower byte, D7, corresponds to 215. Its 2 complement can be calculated as 256 - x, so 41, which means 0x29 in hex.
Can anyone explains how CD/DVD Boot Sector works to me? I've extracted some boot sectors from ISO images and found out that some of them are 6 sectors long and some are 8. I tried to look it up but no results. What is the minimum(maximum) length of a CD/DVD boot sector? Does it have to end with 0x55 0xAA?
Bootable ISO 9660 images are very different from other media such as floppies and hard drives. In the latter case, the BIOS loads a single, 512-byte sector, verifies the final 55 AA bytes, and then jumps to what it loaded.
El Torito, the extension that defines bootable ISO 9660 images for PCs, supports various methods of booting. Four of the methods emulate floppy (1.2M, 1.44M, 2.88M) and hard disk boot sectors; the BIOS will map the first floppy or hard disk to the CD-ROM so that you can take bootable floppies or small, bootable hard disks and convert them to ISO images. The last method is called native booting. Native boot sectors can be anywhere from 1-65535 sectors in length, or up to 32 MiB. Native boot sectors do not have to end in 55 AA.
ISO 9660 native sectors are almost always 2048 bytes, so native boot sectors will usually be 4 sectors long (512 * 4 == 2048).
You can find more info including a link to the El Torito standard here:
http://wiki.osdev.org/ISO_9660
http://wiki.osdev.org/El-Torito
https://en.wikipedia.org/wiki/El_Torito_(CD-ROM_standard)
Also, this document shows the binary structure of El Torito:
http://fossies.org/linux/xorriso/doc/boot_sectors.txt
While trying to data-recovery of a flash-drive, I am trying to write a tool that can search for FAT directory entries. Since I cannot rely on the FAT to tell me where to look, I am doing a simple scan of the drive sectors (actually an image dump of the drive).
The problem is that I cannot find any information about how to detect if a sector/cluster contains FAT directory entries. I know the structure of a directory entry, but not how to detect if a bunch of given bytes actually comprise one.
Finding the start of a sub-directory is simple enough since you can just search for . at byte 0x00 and .. at byte 0x20, but this only helps with the first sector of a sub-directory, not subsequent sectors, nor the root directory or sub-directory fragments in other locations.
I tried using date ranges, file sizes, cluster ranges, invalid filename characters as rough guides, but of course, that’s not too reliable.
If I open the image in a disk-editor and hold down the PgDn key, my brain can detect when a sector containing valid directory entries passes through my field of vision, but how can this be implemented in a program? Is there any way to detect FAT directory entries?
It's unlikely that you can do a perfect job of identifying the directory entries, but you should be able to get reasonable results by using some simple heuristics.
As you said, you can start by looking for a . character at offset 0x00. If it's not there, then the entry is definitely not a directory.
Bit 4 of the file attributes (offset 0x0B) is set if it's a directory entry. If that bit is not set, then it's definitely not a directory. Also, the documentation says that bit 6 will never be set for a disk device. So if bit 6 is set, then it's almost certainly not a valid FAT entry. Although be careful, because a value of 0x0F designates a VFAT long file name entry.
The two bytes at 0x0E are the creation time. If the decoded hours are > 23, or decoded minutes > 59, or decoded seconds > 29, then you can view it as suspicious. It could be a directory entry that somebody mucked with or was somehow corrupted, but it's unlikely.
The access rights at 0x14 says that bits 12-15 must be set to 0. If any of those bits are set, consider it suspicious.
The four bytes at 0x1C give the file size. Those are supposed to be 0 for a directory entry. If they aren't, consider it suspicious.
It appears that there are other such indications in that structure. What you'll have to do is have your code identify the ones that it can, and then make a decision based on the evidence. It won't be 100% correct (i.e. you can probably fool it), but I suspect it would be quite good.
I've got a 3TB drive partitioned like so:
TimeMachine 800,000,000,000 Bytes
TELUS 2,199,975,890,944 Bytes
I bought an identical drive so that I could mirror the above in case of failure.
Using DiskUtility, partitioning makes the drives a different size than the above by several hundreds of thousands of bytes, so when I try to add them to the RAID set, it tells me the drive is too small.
I figured I could use terminal to specify the exact precise sizes I needed so that both partitions would be the right size and I could RAID hassle-free...
I used the following command:
sudo diskutil partitionDisk disk3 "jhfs+" TimeMachine 800000000000b "jhfs+" TELUS 2199975886848b
But the result is TimeMachine being 799,865,798,656 Bytes and TELUS being 2,200,110,092,288 Bytes. The names are identical to the originals and I'm also formatting them in Mac OS Extended (Journaled), like the originals. I can't understand why I'm not getting the same exact sizes when I'm being so specific with Terminal.
Edit for additional info: Playing around with the numbers, regardless of what I do I am always off by a minimum of 16,384 bytes. I can't seem to get the first partition, TimeMachine to land on 800000000000b on the nose.
So here's how I eventually got the exact sizes I needed:
Partitioned the Drive using Disk Utility, stating I wanted to split it 800 GB and 2.2 TB respectively. This yielded something like 800.2GB and 2.2TB (but the 2.2 TB was smaller than the 2,199,975,890,944 Bytes required, of course).
Using Disk Utility, I edited the size of the first partition to 800 GB (from 800.2GB), which brought it down to 800,000,000,000 bytes on the nose, as required.
I booted into GParted Live so that I could edit the second partition with more accuracy than Terminal and Disk Utility and move it around as necessary.
In GParted, I looked at the original drive for reference, noting how much space it had between partitions for the Apple_Boot partitions that Disk Utility adds when you add a partition to a RAID array (I think it was 128 MB in GParted).
I deleted the second partition and recreated it leaving 128 MB before and after the partition and used the original drive's second partition for size reference.
I rebooted into OS X.
Now I couldn't add the second partition to the RAID because I think it ended up being slightly larger than the 2,199,975,890,944 Bytes required (i.e., it didn't have enough space after it for that Apple_Boot partition), I got an error when attempting it in Disk Utility.
I reformatted the partition using Disk Utility just so that it could be a Mac OS Extended (journaled) rather than just HSF+, to be safe (matching the original).
I used Terminal's diskutil resizeVolume [drive's name] 2199975895040b command to get it to land on the required 2,199,975,890,944 Bytes (notice how I had to play around with the resize size, making it bigger than my target size to get it to land where I wanted).
Added both partitions to their respective RAID arrays using Disk Utility and rebuilt them successfully.
... Finally.
Hello fellow programmers.
I'm trying to dump the contents of the USN Journal of a NTFS partition using WinIoCtl functions. I have the *USN_JOURNAL_DATA* structure that tells me that it has a maximum size of 512 MB. I have compared that to what fsutil has to say about it and it's the same value.
Now I have to read each entry into a *USN_RECORD* structure. I do this in a for loop that starts at 0 and goes to the journal's maximum size in increments of 4096 (the cluster size).
I read each 4096 bytes in a buffer of the same size and read all the USN_RECORD structures from it.
Everything is going great, file names are correct, timestamps as well, reasons, everything, except I seem to be missing some recent records. I create a new file on the partition, I write something in it and then I delete the file. I run the app again and the record doesn't appear. I find that the record appears only if I keep reading beyond the journal's maximum size. How can that be?
At the moment I'm reading from the start of the Journal's data to the maximum size + the allocation delta (both are values stored in the *USN_JOURNAL_DATA* structure) which I don't believe it's correct and I'm having trouble finding thorough information related to this.
Can someone please explain this? Is there a buffer around the USN Journal that's similar to how the MFT works (meaning it's size halves when disk space is needed for other files)?
What am I doing wrong?
That's the expected behaviour, as documented:
MaximumSize
The target maximum size for the change journal, in bytes. The change journal can grow larger than this value, but it is then truncated at the next NTFS file system checkpoint to less than this value.
Instead of trying to predetermine the size, loop until you reach the end of the data.
If you are using the FSCTL_ENUM_USN_DATA control code, you have reached the end of the data when the error code from DeviceIoControl is ERROR_HANDLE_EOF.
If you are using the FSCTL_READ_USN_JOURNAL control code, you have reached the end of the data when the next USN returned by the driver (the DWORDLONG at the beginning of the output buffer) is the USN you requested (the value of StartUsn in the input buffer). You will need to set the input parameter BytesToWaitFor to zero, otherwise the driver will wait for the specified amount of new data to be added to the journal.