How to determine the size of an PE executable file from headers and or footers - windows

Assuming you have a stream of data or a block of bytes you want to carve, how can you determine the size of the executables?
There are numerous headers inside the PE executable format, but what header sections do I use to determine (if possible) the total length of the executable?
Here is a picture of the file format.

If the PE file is well formed, the calculation can be simplified as (pseudo-code):
size = IMAGE_NT_HEADERS.OptionalHeader.SizeOfHeaders
foreach section_header in section_headers:
size += section_header.SizeOfRawData
Where:
SizeOfHeaders is a member of IMAGE_OPTIONAL_HEADER structure.
(IMAGE_OPTIONAL_HEADER structure is part of IMAGE_NT_HEADERS)
SizeOfHeaders field gives the length of all the headers (note: including the 16-bit stub).
Each section header is an IMAGE_SECTION_HEADER structure
SizeOfRawData field gives the length of each section on disk.
Example with notepad (Windows 10):
SizeOfHeaders : 0x400
SizeOfRawDataof each sections :
.text: 0x15400
.data: 0x800
.idata: 0x1A00
.rsrc: 0x19C00
.reloc: 0x1600
(note: SizeOfRawData is called Raw Size in the below picture):
Sum everything:
>>> size_of_headers = 0x400
>>> sec_sizes = [0x15400, 0x800, 0x1a00, 0x19c00, 0x1600]
>>> size_of_headers + sum(sec_sizes)
207872
>>>
Total size: 207872 bytes.
Verification:
Note: the above calculation doesn't take into account if the PE is badly formed or if there is an overlay.

Related

How to send color images with ESC/POS?

I have an Epson CW-C6000 that I'm trying to control with ESC commands. I've gotten text to print, so I know I have the IP address, port, etc correct but cannot for the life of me get an image printed.
Here is my code (running from a Ruby on Rails server, with most of the image truncated):
streamSock = TCPSocket.new( "X.X.X.X", 9100 )
str = "~DYR:PRODIMG,B,P,183208,0,89504E470D...4AE426082" + "^XA" + "^FO150,150^IMR:PRODIMG.PNG^FS" + "^XZ"
streamSock.send( str , 0)
streamSock.close
The image is a .png I converted to hexadecimal with this site:
http://tomeko.net/online_tools/file_to_hex.php?lang=en
I'm mostly using page 10 of this PDF for reference:
https://files.support.epson.com/pdf/pos/bulk/esclabel_apg_en_forcw-c6000series_reve.pdf
Does anyone have a hint? Epson support staff was spectacularly unhelpful.
Also I'm sorry if my formatting is bad; I'm new here and will happily edit my post if something is wrong.
Alright I finally got it working. The command for printing a color .PNG is this:
~DYE:[Image Name].PNG,p,p,[Image Size],0,:B64:[Base64 String]:[CRC]
Things that tripped me up:
-You seem to need the .PNG extension on the file name, even though the Epson manual doesn't show that.
-[Image Size] is the number of characters in the Base64 string, even though the Epson manual says it should be the size of the original .PNG image file. If this is wrong the printer will hang and no longer accept input of any kind until restarted.
-There may be other options, but I could only get it working with a CRC of the hex CRC-16/XMODEM type.
Thanks to K J for his/her suggestions and coming along with me!
Perhaps this material can be used as an additional reference.
They seem to have a completely different command/data format than ESC/POS.
ESC/Label Command Reference Guide
Page 12
1.3.4 About Saving the Graphics and Label Formats in the Printer
With ESC/Label command, you can save graphics and label formats in the printer. The printer has a file system. Data saved in the printer is handled as files and is managed in the following way.
The file system does not have a hierarchy.
The printer has a non-volatile saving device, such as Flash ROM, and a volatile saving device, such as RAM, and different drive letters are allocated for each device.
Files are designated as
"<drive letter> colon <:> <file name> dot <.> <extension>".
Page 40-41
2.8 Printing Graphics
...Details have been omitted. Please refer to the actual document...
2.8.1 Registering a Graphic in a Printer and Printing It
...Pick up some from the content. Please refer to the actual document...
Delete the files that remain in the printer (^ID command).
Register the graphic in the printer (~DY command).
When registering a color graphic, you can use the PNG format. When registering a monochrome graphic, you can register the PNG format or the GRF format.
PNG format Monochrome and color graphics
GRF format Monochrome graphics
The reason to execute the step 1.
To ensure capacity of the storage memory necessary for print which application will perform.
2.8.2 Embedding a Graphic in the Field and Printing It
...Details have been omitted. Please refer to the actual document...
In Addition:
Page 104-106
~DY
[Name]
Save File
[Format]
~DY d: o ,f ,x ,t ,w ,data
...A table detailing the parameters is due, but omitted...
[Function]
...Further detailed explanations and figures of functions and parameters are due, but omitted...
Graphic data is handled as follows.
If the data format is binary, you can use any binary data as Parameter data. At this time, the size of Parameter data must be matched to the size specified in Parameter t.
If the data format is a hexadecimal character string, one character from 1. to 3. below is used as Parameter data. At this time, the size of Parameter data written in binary must be matched to the size specified in Parameter t.
0 to 9, A to F, and a to f in ASCII can be used as hexadecimal graphic data.
ASCII comma <,>, the parameter separator character, is used to separate lines. If a comma is input, processing is carried out as if ASCII 0 was input for the remainder of the line.
G to Y and g to z in ASCII can be used as repetition characters. For example, if I9 is input, processing is carried out as if 999 were input. The following table indicates the number of repetitions.
...Characters and repeat specified number of times table omitted...
Looking at the contents of this Technical Reference Guide, it seems that you can register images with tools instead of commands.
CW-C6000/C6500 Series Technical Reference Guide
Page 173-174
And page 288 outlines the Epson Inkjet Label Printer SDK and also describes the existence of sample programs.
#Farmbot26. I have been attempting this same using vb.Net and as you noted Epson support is not helpful. I'm not sure if it's the actual image data that is wrong, CRC, or the ZPL code as nothing helps. Here's 2 examples that have not worked.
`Dim binaryData As Byte() = System.IO.File.ReadAllBytes(txtPNGFile.Text)
zplImageData = Convert.ToBase64String(binaryData)
crc = calcrc(binaryData, binaryData.Length).ToString("X4")
Dim zplToSend As String = "~DYE:" & Path.GetFileName(txtPNGFile.Text).ToUpper & ",P,P," & zplImageData.Length & ",0,:B64:" & zplImageData & ":" & crc & "^XZ"`
`Dim binaryData As Byte() = System.IO.File.ReadAllBytes(txtPNGFile.Text)
crc = calcrc(binaryData, binaryData.Length).ToString("X4") 'Calculate CRC
zplImageData = BitConverter.ToString(binaryData).Replace("-", "")
Dim zplToSend As String = "~DYE:" & Path.GetFileName(txtPNGFile.Text).ToUpper & ",A,P," & zplImageData.Length & ",0,:B64:" & zplImageData & ":" & crc & "^XZ"`
This is the CRC example I have.
`Function calcrc(ByVal data() As Byte, ByVal count As Integer) As Integer
Dim crc As Integer = 0
For Each b As Byte In data
Dim d As Integer = CInt(b)
crc = crc Xor (d << 8)
For j = 0 To 7
If ((crc And &H8000) <> 0) Then
crc = (crc << 1) Xor &H1021
Else
crc = (crc << 1)
End If
Next
Next
Return crc And &HFFFF
End Function`
I have figured out another solution. Save the PNG Image using the Binary data. I found this when reading the Saved Backup file of Image data using the Epson Settings Utility.
~DYE:FILENAME.PNG,B,P,BINARYFILESIZE,0, BINARYIMGDATA
` Try
Dim binaryData As Byte() = System.IO.File.ReadAllBytes(txtPNGFile.Text)
Dim client As System.Net.Sockets.TcpClient = New System.Net.Sockets.TcpClient()
client.Connect(IP_TextBox1.Text.Replace(" ", ""), txtPort.Text)
Dim writer As System.IO.StreamWriter = New System.IO.StreamWriter(client.GetStream(), Encoding.UTF8)
Using mStream As New MemoryStream(binaryData)
Dim zplToSend As String = "~DYE:" & Path.GetFileName(txtPNGFile.Text).ToUpper & ",B,P," & mStream.Length & ",0,"
writer.Write(zplToSend)
writer.Flush()
mStream.WriteTo(client.GetStream())
writer.Flush()
End Using
writer.Close()
client.Close()
MsgBox("Send Complete", MsgBoxStyle.OkOnly, "Complete")
Catch ex As Exception
MsgBox(ex.Message.ToString, MsgBoxStyle.OkOnly, "ERROR")
End Try`
You can also open the image file in an IMAGE object and resize it as needed. I had to do this for the label size of the printer.

what does it mean files overflow_xxxx.bin while training glove

I'm training a word embedding model based on Glove method. While the algorith shows a logger like:
$ build/cooccur -memory 4.0 -vocab-file vocab.txt -verbose 2 -window-size 8 < /home/ignacio/data/GUsDany/corpus/GUs_regulon_pubMed.txt > cooccurrence.bin
COUNTING COOCCURRENCES
window size: 8
context: symmetric
max product: 13752509
overflow length: 38028356
Reading vocab from file "vocab.txt"...loaded 145223095 words.
Building lookup table...table contains 228170143 elements.
Processing token: 5478600000
The home directory of Glove is filled with files caled overflow_0534.bin. Can someone tell whether all is going well?
Thanks
Everything is OK.
You can view the source code of Glove cooccur program at Github.
At the line 57 of the file:
long long overflow_length; // Number of cooccurrence records whose product exceeds max_product to store in memory before writing to disk
If your corpus has too many co-occurrence records, then there will be some data to be written into some temp bin disk files.
while (1) {
if (ind >= overflow_length - window_size) { // If overflow buffer is (almost) full, sort it and write it to temporary file
qsort(cr, ind, sizeof(CREC), compare_crec);
write_chunk(cr,ind,foverflow);
fclose(foverflow);
fidcounter++;
sprintf(filename,"%s_%04d.bin",file_head,fidcounter);
foverflow = fopen(filename,"w");
ind = 0;
}
The variable overflow_length depends on your memory settings.
Line 463:
if ((i = find_arg((char *)"-memory", argc, argv)) > 0) memory_limit = atof(argv[i + 1]);
Line 467:
rlimit = 0.85 * (real)memory_limit * 1073741824/(sizeof(CREC));
Line 470:
overflow_length = (long long) rlimit/6; // 0.85 + 1/6 ~= 1

Reading freely available UVR data using gfortran on mac OSX

I would like to use fortran to read ultraviolet radiation data that has been produced by the Japan Aerospace Exploration Agency. This data is at a daily and monthly temporal resolution from 2000-2010 at a ~5 km spatial resolution. This question is worth answering as the data could be useful for a number of environment/health projects and is freely available, with proper acknowledgement of source and sharing of preprint of any subsequent publications, from:
ftp://suzaku.eorc.jaxa.jp/pub/GLI/glical/Global_05km/monthly/uvb/
There is a readme file available, which provides instructions on how to read data using fortran as follows:
Instructions for _le files
Header
Read header (size= pixel size *2byte):
character head*14400
read(10,rec=1) head
read(head,'(2i6,2f8.2,f8.4,2e12.5,a1,a8,a1,a40)')
& npixel,nline,lon_min,lat_max,reso,slope,offset,',',
& para,',',outfile
Read data (e.g., fortran77)
parameter(nl=7200, ml=3601)
... open file by "unformatted", "recl=nl*2(byte)" (,"bytereclen")
integer*2 i2buf(nl,ml)
do m=1,ml
read(10,rec=1+m) (i2buf(n,m), n=1,nl)
do n=1,nl
par=i2buf(n,m)*slope+offset
write(6,*) 'PAR[Ein/m^2/day]=',par
enddo
enddo
slope values
par__le : daily PAR [Ein/m^2/day] = DN * 0.01
dpar_le : direct PAR = DN * 0.01
swr__le : daily mean shortwave radiation [W/m^2] = DN * 0.01
tip__le : transmittance of instantaneous PAR at noon = DN * 0.0001
uva__le : daily mean UVA [W/m^2] = DN * 0.001
uvb__le : daily mean UVB [W/m^2] = DN * 0.0001
rpar_le : PAR-range surface reflectance (TOP of canopy/solid surfaces) = DN * 0.0001 (monthly data only)
error values
-1 as signed short integer (int16)
65535 as unsigned short integer (uint16)
Progress so far
I have downloaded and installed gfortran successfully on mac OSX. I have downloaded a test file (MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le.gz) and decompressed it. I have created a program file:
PROGRAM readuvr
IMPLICIT NONE
!some code
END PROGRAM
I will then type the following into the command line to create an executable and run it to extract the data.
gfortran -o executable
./executable
As a complete beginner to fortran, my question is: how can I use the instructions provided to build a program that can read the data and output it into a text file?
Well, that file expands to 51,868,800 bytes. The comments imply the header is 14,400 bytes, which leaves 51,854,400 bytes of actual data payload.
There seem to be 7200 lines of data, so that means there are 7202 bytes per line. There seem to be 2 bytes (16-bit samples) so if we assume 2 bytes/sample, that means there are 3601 samples per line, which matches the ml=3601.
So basically, you need to read 14,400 bytes of header, then 7200 lines of data, each line consisting of 3601 values, each of those being 2 bytes wide...
Actually, if you are that unfamiliar with FORTRAN, you may like to extract the data with Perl which is already installed and available on OS X anyway. I have started a VERY SIMPLISTIC Perl program that reads the dat and prints the first 2 values on each line:
#!/usr/bin/perl
use strict;
use warnings;
# Read 14,400 bytes of header
my $buffer;
my $nBytes = 14400;
my $bytesRead = read (STDIN, $buffer, $nBytes) ;
my ($npixel,$nline,$lon_min,$lat_max,$reso,$slope,$offset,$junk)=split(' ',$buffer);
print "npixel:$npixel\n";
print "nline:$nline\n";
print "lon_min:$lon_min\n";
print "lat_max:$lat_max\n";
print "reso:$reso\n";
print "slope:$slope\n";
$offset =~ s/,.*//; # strip trailing comma and junk
print "offset:$offset\n";
# Read actual lines of data
my $line;
for(my $m=1;$m<=$nline;$m++){
read(STDIN,$line,$npixel*2);
my $x=$npixel*2;
my #values=unpack("S$x",$line);
printf "Line: %d",$m;
for(my $j=0;$j<2;$j++){
printf ",%f",$values[$j]*$slope+$offset;
}
printf "\n"; # newline
}
Save it as go.pl and then in the Terminal, type the following once to make it executable
chmod +x go.pl
and then run it like this
./go.pl < MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le
Sample output extract:
npixel:7200
nline:3601
lon_min:0.00
lat_max:90.00
reso:0.0500
slope:0.10000E-03
offset:0.00000E+00
...
...
Line: 3306,0.099800,0.099800
Line: 3307,0.099900,0.099900
Line: 3308,0.099400,0.074200
Line: 3309,0.098900,0.098900
Line: 3310,0.098400,0.098400
Line: 3311,0.074300,0.074200
Line: 3312,0.071300,0.071200
fortran (f2003 or so) solution. (The linked instructions are awful by the way )
implicit none
character*80 para,outfile
character(len=:),allocatable::header,infile
integer npixel,nline,blen,i
c note kind=2 is not standard. This needs to be a 2-byte integer.
integer(kind=2),allocatable :: data(:,:)
real lon_min,lat_max,reso,slope,off
c header is plain text, so first open formatted and
c directly read header data
infile='MOD02SSH_A20000224Av6_v601_7200_3601_uvb__le'
open(10,file=infile)
read(10,*)npixel,nline,lon_min,lat_max,reso,slope,off,
$ para,outfile
close(10)
write(*,*)npixel,nline,lon_min,lat_max,reso,slope,off,
$ trim(para),' ',trim(outfile)
blen=2*npixel
allocate(character(len=blen)::header)
allocate(data(npixel,nline))
if( sizeof(data(1,1)).ne.2 )then
write(*,*)'error kind=2 did not give a 2 byte integer'
stop
endif
c now close and reopen for binary read.
c direct access approach:
open(20,file=infile,access='direct',recl=blen/4)
c note the granularity of the recl= specifier is not standard.
c ifort uses 4 bytes. (note this will break if npixel is not even )
read(20,rec=1)header
write(*,*)trim(header)
do i=1,nline
read(20,rec=i+1)data(:,i)
enddo
c note streams if available is simpler: (we don't need to know rec len )
c open(20,file=infile,access='stream')
c read(20)header,data
end
This is not actually validated because I don't have known file content to compare against.

reading file with UPC

I'm starting to learn UPC, and I have the following piece of code to read a file:
upc_file_t *fileIn;
int n;
fileIn = upc_all_fopen("input_small", UPC_RDONLY | UPC_INDIVIDUAL_FP , 0, NULL);
upc_all_fread_local(fileIn, &n, sizeof(int), 1, UPC_IN_ALLSYNC | UPC_OUT_ALLSYNC);
upc_barrier;
printf("%d\n", n);
upc_all_fclose(fileIn);
However, the output (value of n) is always 808651319, which means something is wrong, and I can't find what is it. The first line of the file I'm giving as input is '7', so the result of the printf should be 7...
Any idea why this happens?
Thanks in advance!
UPC Parallel I/O library performs unformatted (binary) input/output, not formatted one like what you get with (f)printf(3)/(f)scanf(3) from the standard C library. Parallel I/O cannot handle text files because of their intrinsic properties like variable-length records.
upc_all_fread_local(fileIn, &n, sizeof(int), 1, UPC_IN_ALLSYNC | UPC_OUT_ALLSYNC)
behaves like the following call to the standard C library function for unformatted read from a file:
fread(&n, sizeof(int), 1, fh)
You are just reading 1 element of sizeof(int) bytes from the file (4 bytes on most platforms) into the address of n. The number you got 808651319 in hexadecimal is 0x30330A37. On little endian systems like x86/x64 this is stored in memory and on disk as 0x37 0x0A 0x33 0x30 (reversed byte order). These are the ASCII codes of the first 4 bytes of the string 7\n30 (\n or LF is the line feed/new line symbol) so I'd guess your input_small file looked like:
7
30...
...
You should prepare your input data in binary format using fwrite(3) instead of using (f)printf(3) or your text editor of choice.

The size of a PE Header

Is there a way to find out the size of a PE Header without reading all of it or the entire file?
You can calculate the total size of the PE header like this:
sizeof(Signature) + sizeof(FileHeader) + sizeof(OptionalHeader) + sizeof(SectionTable)
The file header always has the same size but the OptionalHeader's size can differ, as can the section table size.
The OptionalHeader's size is stored in FileHeader.SizeOfOptionalHeader, and the section table size equals FileHeader.NumberOfSections * sizeof(IMAGE_SECTION_HEADER)
And some C code:
DWORD SizeOfPEHeader(const IMAGE_NT_HEADERS * pNTH)
{
return (offsetof(IMAGE_NT_HEADERS, OptionalHeader) + pNTH->FileHeader.SizeOfOptionalHeader + (pNTH->FileHeader.NumberOfSections * sizeof(IMAGE_SECTION_HEADER)));
}
All you have to do is read the DOS header, get the PE offset (e_lfanew) and read PE.Signature + PE.FileHeader into memory. That's two reading operations of fixed size and you have all the info you need.

Resources