I wrote a firmware to run our ESP32-based custom PCB.
The firmware holds a unique S/N (serial number) in the NVS thru Preferences API which is set thru the bluetooth app I wrote.
But now I have to produce tens of PCB and it takes time to connect and set the S/N thru the app.
The current process is a 2-step process which I want to streamline:
flash the generic firmware
set the unique S/N
I am wondering if I could write a script that could do both steps, providing the S/N as the script argument.
I could take advantage of Espressif esp tool write_flash for example.
How could I do that?
You can pre-generate the NVS data partition and flash it together with firmware. ESP IDF provides the NVS Partition Generator Utility specifically for for that purpose. Here's an overview of the process.
First you create a CSV file (let's name it mfgdata.csv) with your pre-generated data, e.g. serial number, product ID, keypair, whatever. Assume the NVS namespace is "mfgdata_ns".
key,type,encoding,value
mfgdata_ns,namespace,,
serial,data,string,"ABC1234"
private_key,file,string,/path/key.pri
Then you generate the binary partition with NVS data on it (named mfgdata.bin) from this CSV file. Assume the NVS partition starts at address 0x3000.
$IDF_PATH/components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py generate mfgdata.csv mfgdata.bin 0x3000
Finally you flash the NVS partition together with your firmware. Here's a sample flashing command for the NVS partition alone (assuming the partition is named "mfgdata_part" in your partition table).
$ $IDF_PATH/components/partition_table/parttool.py -p /dev/ttyUSB0 -b 921600 write_partition --partition-name=mfgdata_part --input=mfgdata.bin
Firmware starts up, loads the NVS namespace and finds all values that you specified in mfgdata.csv.
You have to be careful when creating the CSV file. Some 2 years ago they were using the python CSV module to parse this file and it worked perfectly, as expected. Then some not very bright person decided to ditch the python module and replace it by splitting each line on comma character, silently ignoring all parsing problems. I don't know if they've unf*cked it yet.
Related
I'm using Intel's Pin Tool to do some binary instrumentation, and was wondering if there an API to get the instruction byte code at a given address.
Something like:
instruction = getInstructionatAddr(addr);
where addr is the desired address.
I know the function Instruction (used in many of the simple/manual examples) given by Pin gets the instruction, but I need to know the instructions at other addresses. I perused the web with no avail. Any help would be appreciated!
CHEERS
wondering if there an API to get the instruction byte code at a given
address
Yes, it's possible but in a somewhat contrived way: with PIN you are usually interested in what is executed (or manipulated through the executed instructions), so everything outside the code / data flow is not of any interest for PIN.
PIN is using (and thus ships with) Intel XED which is an instruction encoder / decoder.
In your PIN installation you should have and \extra folder with two sub-directories: xed-ia32 and xed-intel64 (choose the one that suits your architecture). The main include file for XED is xed-interface.h located in the \include folder of the aforementioned directories.
In your Pintool, given any address in the virtual space of your pintooled program, use the PIN_SafeCopy function to read the program memory (and thus bytes at the given address). The advantage of PIN_SafeCopy is that it fails graciously even if it can't read the memory, and can read "shadowed" parts of the memory.
Use XED to decode the instruction bytes for you.
For an example of how to decode an instruction with XED, see the first example program.
As the small example uses an hardcoded buffer (namely itext in the example program), replace this hardcoded buffer with the destination buffer you used in PIN_SafeCopy.
Obviously, you should make sure that the memory you are reading really contains code.
AFAIK, it is not possible to get an INS type (the usual type describing an instruction in PIN) from an arbitrary address as only addresses in the code flow will "generate" an INS type.
As a side note:
I know the function Instruction (used in many of the simple/manual
examples) given by Pin gets the instruction
The Instruction routine used in many PIN example is called an "Instrumentation routine": its name is not relevant in itself.
Pin_SafeCopy may help you. This API could copy memory content from the address space of target process to one specified buffer.
I am working on a boot loader for TMS320DM6437. The idea is to create 2 independent firmware that one will update another. In firmware1 I will download firmware2 file and write it to NOR flash in a specified address. Both firmware are stored in NOR flash in ais format. Now I have two applications in flash. One is my custom boot loader and the second one is my main project. I want to know how I can jump from the first program to the second program located at a specified address. I also expect information about documents which may help me to create custom bootloader
Any recommendations?
You can jump to the entry point. I'm using this approach on TMS320 2802x and 2803x, but it should be the same.
The symbol of the entry point is c_int00.
To get to know the address of c_int00 in the second application, you have to fix the Run-Time Support (RTS) library at a specific address, by modifying the linker command file.
Otherwise you can leave the RTS unconstrained, and create a C variable (at a fixed address) that is initialized with the value of cint_00. Using this method your memory map is more flexible and you can add, togheter with the C variable, a comprehensive data structure with other information for your bootloader, e.g. CRC, version number, etc.
Be carefull with the (re)initialization of the peripherals in the second application, since you are not starting from a hardware reset, and you may need to explicity reset some more registers, or clear interrupt requests.
How to get CPU/Processor serial number or unique ID in perl for windows machine without using any modules, probably using syscalls or something?
http://metacpan.org/pod/Win32::SystemInfo. Without modules you can learn source code of this module and use it. You will see that module imports function GetSystemInfo from kernel32.dll and than uses it, you can do it in the same way
You can't get a CPU serial number. For a brief time, Intel issued a series of Pentium III processors where each chip had a unique serial number. It became a PR mess for them over privacy concerns and they discontinued that set of instructions in subsequent processor releases.
But if you are looking for a "unique id" for a Windows install that is mostly guaranteed to by unique, you can try any of the following:
Generate a unique GUID (uuidgen, CoCreateGuid), save it out to the registry/disk, then read it back later.
Get the Machine SID. http://www.perlmonks.org/?node_id=645077
I'm writing custom firmware for a SparkFun Logomatic V2 that records binary data to a file on a 2GB micro-SD card. The data file size will range from 100 MB to 1 GB.
The format of the binary data is in flux as the board's firmware evolves (it will actually be dynamically reconfigurable at run-time). Rather than create and maintain a separate decoder/converter program for each version of firmware/configuration, I'd much rather make the data files self-converting to CSV format by starting the data file with a Bash script that is written to the data file before data recording starts.
I know how to create a Here Document, but I suspect Bash would be unable to quickly parse and convert a gigabyte of binary data, so I'd like to make the process run much faster by having the script first compile some C code (assume GCC is present and in the path), then run the resulting program, passing the binary data to stdin.
To make the problem more concrete, assume the firmware will create binary data consisting of 4 16-bit integer values: A timestamp (unsigned) followed by 3 accelerometer axes (signed). There is no separator between records (mainly because I'm saturating the SPI interface to the uSD card).
So, I think I need a script with TWO here documents: One for the C code (parameterized by expanded Bash variables), and another for the binary data. Here's where I am so far:
#! env bash
# Produced by firmware version 0.0.0.0.0.1 alpha
# Configuration for this data run:
header_string = "Time, X, Y, Z"
column_count = 4
# Create the converter executable
# Use "<<-" to permit code to be indented for readability.
# Allow variable expansion/substitution.
gcc -xc /tmp/convertit - <<-THE_C_CODE
#include <stdio.h>
int main (int argc, char **argv) {
// Write ${header_string} to stdout
while (1) {
// Read $(column_count} shorts from stdin
// Break if EOF
// Write $(column_count} comma-delimited values to stdout
}
// Close stdout
return 0;
}
THE_C_CODE
# Pass the binary data to the converter
# Hard-quote the Here tag to prevent subsequent expansion/substitution
/tmp/convertit >./$1.csv <<'THE_BINARY_DATA'
...
... hundreds of megabytes of semi-random data ...
...
THE_BINARY_DATA
rm /tmp/convertit
exit 0
Does that look about right? I don't yet have a real data file to test this with, but I wanted to verify the idea before going much further.
Will Bash complain if the closing lines are missing? This may happen if data capture terminates unexpectedly due to a shock knocking loose the battery or uSD card. Or if the firmware borks.
Is there a faster or better method I should consider? For example, I wonder if Bash will be too slow to copy the binary data as fast as the C program can consume it: Should the C program open the data file directly?
TIA,
-BobC
You may want to have a look at makeself. It allows you to change any .tar.gz archive into a self-extracting file which is platform independent (something like a shell script that contains a here document). This will allow you to easily distribute your data and decoder. It also allows you to configure a script contained within the archive to be run when the container script is run. This way you can use makeself for packaging and inside the archive you can put your data files and decoder written in C or bash or whatever language you find suitable.
While it is possible to decode binary data using shell tools (e.g. using od), it's very cumbersome and ineffective. I'd recommend using either a C program or perl which is also likely to be found on almost any machine (check this page).
I needed to recover the partition table I deleted accidentally. I used an application named TestDisk. Its simply mind blowing. I reads each cylinder from the disk. I've seen similar such applications which work with MBR & partitioning.
I'm curious.
How do they read
clusters/cylinders/sectors from the
disk? Is there some kind of API for this?
Is it again OS dependent? If so whats the way to for Linux & for windows?
EDIT:
Well, I'm not just curious I want a hands on experience. I want to write a simple application which displays each LBA.
Cylinders and sectors (wiki explanation) are largely obsoleted by the newer LBA (logical block addressing) scheme for addressing drives.
If you're curious about the history, use the Wikipedia article as a starting point. If you're just wondering how it works now, code is expected to simply use the LBA address (which works largely the same way as a file does - a linear array of bytes arranged in blocks)
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
For *nix, there have been already answers (/dev directory); for Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To actually perform reads or writes you have then to use the DeviceIoControl function.
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
I'm the OP. I'm combining Eric Seppanen's & Matteo Italia's answers to make it complete.
*NIX Platforms:
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
Windows Platform
For Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To perform reads or writes simply call ReadFile and WriteFile (buffer must be aligned on sector size).
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
Alternatively you can also you DeviceIoControl function which sends a control code directly to a specified device driver, causing the corresponding device to perform the corresponding operation.
On linux, as root, you can save your MBR like this (Assuming you drive is /dev/sda):
dd if=/dev/sda of=mbr bs=512 count=1
If you wanted to read 1Mb from you drive, starting at the 10th MB:
dd if=/dev/sda of=1Mb bs=1Mb count=1 skip=10