I'm hoping somebody out there can help me with this. I'm attempting to extract some barcode data from some .dat files. Its a B Tree file system with groups of three files .dat .ix. .dia. The company that wrote the software (a long time ago) say that the program is written in Pascal. I have no experience in reverse engineering but from what I read its most likely the only way to extract the data as the structure of the database is contained in the code of the program. I'm looking for advice on where to start.
I suppose the first thing you need to do is to see if the exe you've got was written with Delphi. You can check with this: http://cc.embarcadero.com/Item/15250
Then, to see if the exe that creates those .dat files were made with 'TurboPower B-Tree Filer', the I'd suggest you download and take a look at this: http://sourceforge.net/projects/tpbtreefiler/
At this step, looking at these sources is needed to familiarize yourself with the class names used in 'TurboPower B-Tree Filer' to help determine if any of those classes were used in your exe.
Then, using 'XN Resource Editor' [search the Internet for this] or, probhably better, 'MiTeC Portable Executable Reader' [ http://www.mitec.cz/pe.html ], see if any class names are relevant.
If they are, then you're in luck --sort of. All you will need to do is to write an app using 'TurboPower B-Tree Filer' to import the data in your dat files to export or manipulate as you wish.
At that point, you might find this link useful.
TurboPower B-Tree Filer and Delphi XE2 - Anyone done it?
If, OTOH, none of the above applies; I fear the only option is to reverse engineer the exe you have.
Related
I like reading the PoC||GTFO issues and one thing I found remarkable when I first discovered it, was the "polyglot" nature of their PDF files.
Let met explain: when you consider for example their 8th issue, you may unzip files from it; execute the encryption they are talking about by running it as a script and even better(worse?) with their 9th issue you can even play it as a music file!
I'm currently in the process of writing small scripts every week and writing each time a little one page PDF in LaTeX to explain the said scripts. So I would really enjoy being able to create the same kind of PDF files. Sadly they explained (partly) in their first issue how to include zip files, but they did so through three small sketches of cmd lines without actual explanations.
So my question is basically :
how can one create such a polyglot PDF file containing stuff like a zip as well as being a shell script which may be run using arguments just like normal scripts?
I'm asking here about the process of creation, not just an explanation of how this is possible. The ideal way for me would that there are already some scripts or programs allowing to create easily such PDF files.
I've tried to search the net for the keywords "polyglot files" and others of the kind and wasn't able to find any useful matches. Maybe this process has another name?
I've already read the presentation by Julia Wolf which explains how things works, but I sadly haven't had time to apply the knowledge there to real world, because I'm sadly not used to play with file headers and the way a PDF is constructed.
EDIT:
Okay, I've read more and found the 7th edition of PoC||GTFO to be really informative concerning this subject. I may end up being able to create my own scripts to do such polyglot PDF files if I have some more time to consider it.
I played around with polyglots myself after attending Ange's talks and also talking to him in person. You really need to understand the file formats to be able to nest them into each other.
However, long story short, here are some links I found extremely useful for creating polyglots:
Some older Google Code Trunk
PoC of the polyglot stuff
Especially the second link (to github) will help you creating polyglots, but also understanding how they are working and how they are implemented. Since it is mostly Python stuff and very well / clean written, it is very useful and easy to follow.
I feel dissecting some file formats would be a good place to start. You can find many file format specifications for different file types through Google, but they can be a tough read and will likely take you some time to translate into whatever language you are using.
PDF: https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf
ELF: https://www.cs.cmu.edu/afs/cs/academic/class/15213-s00/doc/elf.pdf
ZIP: http://kat.sdf.org/zip_file_format.txt
The language(s) you select will need a way to read and write raw bytes (not just ascii alphanumeric), so perhaps C would be good for more direct access to memory. Some Python tricks could help with open sourcing the scripts easily.
To dissect the files, you may want to build a tool kinda like https://github.com/kvesel/zipbrk/ to take them apart, then put them all back together in a polyglot format. For example, zip does not require the section headers to be at the start (or even contiguous for that matter), and PDF magic number can appear in multiple places within the file as well. I also believe I recall a polyglot tool being included in one of the PoC||GTFO publishings (maybe issue 8 or 2??) as a polyglot in the pdf file.
Don't forget the hackers bible! :)
https://nostarch.com/gtfo
I am new to programming and F# is my first .NET language as well as my first functional language. As a beginner's project, I would like to try implementing my own duplicate file finder, and I am looking for some tips on the F# tools that are relevant to my project. I apologise in advance if my question doesn't meet StackOverflow's standards: I will gladly make changes to it as required.
Here is the rough idea I have come up with: I will retrieve all files from a desired folder, read the file contents into byte arrays, and then use a hash-table to store the byte arrays and remove duplicates. Will more experienced programmers tell me whether this is a good approach? What improvements can I make? Additionally, as asked earlier, what are the relevant F# tools to look at? MSDN has a huge list of libraries and namespaces and etc., and it is really overwhelming for a newbie like me.
Thank you warmly in advance for your help!
I'd recommend starting with a console application.
There are a couple of relevant .Net APIs:
System.IO.Directory.GetFiles
System.IO.Directory.EnumerateFiles
GetFiles returns an easy to use array of all file paths but blocks until all files are found, where as EnumerateFiles lets you enumerate the files one-by-one and give feedback to the user.
For performance when finding duplicates, the file length can be used to find potential duplicates before comparing the data. Here you could use the Length property of System.IO.FileInfo.
If you create a sequence of tuple of file name and file length, you could use Seq.groupBy to group potential matches. Finally for groups of 2 or more you can read the files and compare the bytes to find exact duplicates.
Is there a way to extract the size and address of elements within a structure using an elf file? I am hoping there is a tool available that can do this and export it to a more readable format.
My end goal is to convert the ELF file to a ASAM A2L file. A open source/free tool that could do this would even be better but most companies that do this charge alot for their tools.
I don't know offhand of anything pre-canned, but it isn't very hard to modify an existing tool to do it.
The "pahole" program from the "dwarves" project does something similar. It prints a structure definition in a certain way.
There's also a "pahole.py" script for gdb that does pretty much the same thing. This would be trivial to modify to print things however you like.
If you want to get a little deeper you could write it yourself using one of the existing DWARF libraries. I like the one in elfutils, but YMMV.
I am working on reverse engineering and i wanted to give my own file path without using open dialog box like writing a fixed file path and read a file from that every time the software start.i tried to give the file name in a .data section of the software but it retrieve only the hex number not the string.
Is there any way hooking windows API for file open?or any way to write in memory and to read it every time the software starts?Any advice or direction would be greatly appreciated.
I think that what you may be looking for is "intercepting system call", quick google search came up with this link:
http://jbremer.org/intercepting-system-calls-on-x86_64-windows/ so it should be possible and not too difficult.
Basically if you manage to intercept a call to GetOpenFileName and replace it with custom implementation you should be able to do what you want (that is put hardcoded file path to appropriate buffer in LPOPENFILENAME structure)
As for your attempt to modify compiled code (I assume that what you are referring as reverse engineering is disassembling and modifying binaries) it should be possible to do, but it will require deep knowledge of windows binary architecture and assembly language.
Sadly, a project that I have been working on lately has a large amount of copy-and-paste code, even within single files. Are there any tools or techniques that can detect duplication or near-duplication within a single file? I have Beyond Compare 3 and it works well for comparing separate files, but I am at a loss for comparing single files.
Thanks in advance.
Edit:
Thanks for all the great tools! I'll definitely check them out.
This project is an ASP.NET/C# project, but I work with a variety of languages including Java; I'm interested in what tools are best (for any language) to remove duplication.
Check out Atomiq. It finds code that is duplicate that is prime for extracting to one location.
http://www.getatomiq.com/
If you're using Eclipse, you can use the copy paste detector (CPD) https://olex.openlogic.com/packages/cpd.
You don't say what language you are using, which is going to affect what tools you can use.
For Python there is CloneDigger. It also supports Java but I have not tried that. It can find code duplication both with a single file and between files, and gives you the result as a diff-like report in HTML.
See SD CloneDR, a tool for detecting copy-paste-edit code within and across multiple files. It detects exact copyies, copies that have been reformatted, and near-miss copies with different identifiers, literals, and even different seqeunces of statements.
The CloneDR handles many languages, including Java (1.4,1.5,1.6) and C# especially up to C#4.0. You can see sample clone detection reports at the website, also including one for C#.
Resharper does this automagically - it suggests when it thinks code should be extracted into a method, and will do the extraction for you
Check out PMD , once you have configured it (which is tad simple) you can run its copy paste detector to find duplicate code.
One with some Office skills can do following sequence in 1 minute:
use ordinary formatter to unify the code style, preferably without line wrapping
feed the code text into Microsoft Excel as a single column
search and replace all dual spaces with single one and do other replacements
sort column
At this point the keywords for duplicates will be already well detected. But to go further
add comparator formula to 2nd column and counter to 3rd
copy and paste values again, sort and see the most repetitive lines
There is an analysis tool, called Simian, which I haven't yet tried. Supposedly it can be run on any kind of text and point out duplicated items. It can be used via a command line interface.
Another option similar to those above, but with a different tool chain: https://www.npmjs.com/package/jscpd