I would like to create a file format for my app like Quake, OO, and MS Office 07 have.
Basically a uncompressed zip folder, or tar file.
I need this to be cross platform (mac and windows).
Can I do something via command prompt and bash?
If you want a single file that is portable to all platforms and which contain structured data, consider using sqlite. You'll get a full featured ACID compliant database that exists on disk as a single file.
There are libraries you can link against to directly access the file, and there is a command line tool you can use as well. No matter what language you are using, most likely there is support for it.
http://www.sqlite.org
Have a look at the open source 7Zip compression format. For your specific needs, you can use it in an "Archive" mode, zero compression but very fast.
It provides a powerful SDK, LZMA, from the site:
"LZMA is the default and general compression method of 7z format in the 7-Zip program. LZMA provides a high compression ratio and very fast decompression, so it is very suitable for embedded applications. For example, it can be used for ROM (firmware) compressing.
The LZMA SDK provides the documentation, samples, header files, libraries, and tools you need to develop applications that use LZMA compression."
Zip is supported everywhere. If a container is all you need, than those are surely good options.
SQLite is great.
A single file, crossplatform, a tiny library, SQL access to data, transactions, the whole enchilada.
you can use transactions to guarantee consistent return points in case of crashing. check uses for sqlite, they specifically advocate using it as a data model layer for desktop applications.
also, there's a command-line tool to manually access the data.
First thing you should ask yourself is, "Do I really need to make my own?"
Depending on what you want to use it for, you are probably better off using a common format and some pre-made libraries which already handle one of those formats very well.
Good places to start:
http://www.destructor.de/libtar/index.htm (tar -- a the 'container' format)
http://www.zlib.net/ (zlib -- a method of compressing data before or after you put it in the container)
If you still really think you need to make your own, I would suggest studying something very simple first, like tar's format:
http://en.wikipedia.org/wiki/Tar_(file_format)
or
http://schmidt.devlib.org/file-formats/tar-archive-file-format.html
Instead of making a format, I'd just decide on a convention. One or more named files within the container have the metadata you need to access the rest of the files, and know what to do with them. The container itself, though, should just be some ubiquitous format, such as zip. No need to reinvent the wheel, here.
Related
How to check (programmatically) if a video/audio file has DRM protection? Can FFmpeg do this?
As noted in the comments, this heavily depends on the file format. However, since the most common format is likely to be ISOBMFF, that will likely be the case where you most often need to detect it.
For ISOBMFF, it's quite easy, you can simply loop through the "atoms" in the file, and look for a pssh atom.
Tools like mp4dump will be able to dump the whole structure for you. Alternatively i built a small tool that will find the pssh box, and tell you which DRM systems it signals.
I would like to write a small program, or script, to extract a set of pictures from a pdf.
I have several PDFs, they each have a table of pictures. I would link to have one picture per file. Therefore I need a way to extract them. Due to the nature of the PDF (A table/grid), it seems that it would be much easier to write a program, than do some manual method. However I have no idea what tools are available.
What libraries are available?
Preference Python, then C# or Java, then maybe some other language (My C and C++ is rusty, I have not done them for years).
I am on Debian Gnu/Linux, so have a wide choice of tools.
I went with pdfbox (an Apache project, so Free Software) it is a java library and a command line tool (the app module). I then scripted it with a bit of python to process the extracted text (yes it did that as well), and rename the image files.
I am wondering if there is a way of version control in vivado for a VHDL project.
One way would be to add the version number to the bitstream file name.
Would that be possible?
What other options are there if that isn't possible?
Thanks in advance
Think about what you require
Version information format and size: What is the version? Is a natural number? String? Real? Do you save date or time stamp? Do you need a build number? Need extra space for sub-unit versions?
Version source and design flow: What sets the version? Is it a source control tool? Do you manually set the version? Do you need automation to update parts of the version information (timestamp, build number, etc.)?
Version accessibility/observability: Where do you want to read the version number and have it viewable? In the source code? External file? The bitstream file name? The flash file contents? Do you need the FPGA logic to be able to access the version information (to be read via communication channel, embedded processor, etc.)?
Vendor Tools/Device/System/Source dependency: Implementation and automation of version information may depend on several things: Each vendor has different tools (even within the same vendor's). Different devices may allow you to do different things. The system where you work may allow different forms of automation (OS, installed script tools, etc.). The source files (VHDL, Verilog, System C, ...) and directory structure may influence the applicability of your solution.
Automation options
Makefile (builder script) automation
The most flexible way is to have some kind of builder script which may or may not be hooked to your source control mechanism (makefile, perl, python, etc.). Think about the transportability of your project between designers and even design platforms (Windows, Linux, etc.).
Pros:
Complete control of what you want to do.
Can be hooked to any source control automation.
Can be modified to work on any platform when using standard tools (be sure to take special care of file paths which differ between OS).
Cons:
Does not flow well with GUI vendor tools (see Vivado TCL hooks).
Vivado TCL hooks
Vivado has TCL hooks which you can use to run when you synthesize, implement, etc.
Using a pre-synthesis TCL hook will allow you to connect a script to run just before synthesis, and implement your version information update mechanism.
Using a post-bitstream TCL hook will allow you to connect a script to run just after bitstream file creation, and create a file name matching your updated version information, or other final operation you may require.
More information:
http://www.xilinx.com/support/documentation/sw_manuals/xilinx2014_4/ug894-vivado-tcl-scripting.pdf
http://www.xilinx.com/support/documentation/sw_manuals/xilinx2014_4/ug835-vivado-tcl-commands.pdf
Pros:
Works with a GUI flow (automatically takes care of everything when you click on Synthesize).
Cons:
(Vivado) tool dependent
Need to remember to create the hooks in your Vivado project
Implementation options
Source code modification
Version number, build number, timestamp, and basically everything you need can be modified (within the VHDL/Verilog code) automatically using a script which parses a known file format. I've done this with Vivado TCL hooks and a well formatted package VHDL file. Works like a charm.
Note: When updating source files using pre-synthesis TCL hooks, Vivado will think the files were modified after, and will consider the implementation as not updated. I added to the script a command to save the current timestamp of the package file, updated it, and then wrote back the timestamp (meaning the package file will forever hold its system date and time).
Pros:
Version information easily viewable accessible to designer (package file).
Version information easily accessible for logic usage.
Device independent.
Cons:
Version information cannot be read by examining the bitstream file.
Writing the script may take some work and may force you to a standardisation of your project sources directory structure [which may not be a totally bad thing ;) ]
Top level entity/module generic
Generics at the top level design module/entity can be modified by the Synthesis tool and accessed by the design's logic.
Pros:
Less complicated scripting than modifying the source code.
Cons:
Version information cannot be read by examining the bitstream file.
You still need to save your version/build number somewhere (a file).
For VHDL/Verilog, using generics is more complicated than using a package, since you need to wire the generics through your hierarchies until you reach the design code which writes the value to a register.
Synthesis tool dependency. Different tools have different commands to update the top level generic values.
JTAG USERCODE
Most FPGAs support the IEEE Std. 1149.1 JTAG feature USERCODE, which gives the user a 32-bit value to identify the bitstream. This value can be set using the Vendor tools when creating the bitstream file. However, this value cannot be read by the FPGA's logic (unless you implement logic to read the flash and track the correct address for this value, which is a big overkill).
Pros:
Fit for most devices.
Version information is accessible by examining the file.
Cons:
Version information cannot be read by the logic without extreme procedures.
Limited information size (only 32 bits).
Xilinx USR_ACCESS Register
Similar to the JTAG USERCODE, Xilinx have a implemented a USR_ACCESS Register for Virtex-5, Virtex-6, and all 7 series devices. They provided a primitive for VHDL/Verilog to allow you to read the value via logic.
More information here: http://www.xilinx.com/support/documentation/application_notes/xapp497_usr_access.pdf
Pros:
Version information can be read by the design logic.
Version information is accessible by examining the file.
Cons:
Solution for specific devices (not even all Xilinx devices support this).
Limited information size (only 32 bits).
ROM static information
You can implement a version information ROM and use an external file to set its value. (see Verilog : Memory block Instantiation)
Pros:
Can include large version information (even ASCII code).
Version information viewable by the designer (external file).
Fit for most devices and tools which accept the VHDL/Verilog code for ROM.
Cons:
A bit more complicated logic may be required to read all the ROM (depends on how you implement it).
It is difficult (yet possible) to examine version information by examining the file (You need to extract ROM bitstream locations).
Flash image modification
Many systems have their FPGAs load from flash devices. Usually there is more room than just the FPGA bitstream. It is possible to create a large flash image file to include extra information, among which is version information. This is probably to worst option of all I've written above, and mentioned this for completeness.
Pros:
Potentially large information cache.
Flash image file can be examined for version information.
Cons:
Difficult to implement.
Dependent on vendor tools.
Does anyone know a good tool for compression files using Windows API function called RtlCompressBuffer? I want to compress an executable using this method.
RtlCompressBuffer is a fairly low-level tool. You'd normally use it to create some higher-level tool before actually compressing an executable.
Windows has a couple such higher level tools, depending on exactly what you want to accomplish. If you want an executable that's stored on a file system in compressed form, then you probably want to use NTFS compression. To compress your executable with this, you call DeviceIoControl with the FSCTL_SET_COMPRESSION flag.
If you want to compress a file for distribution (e.g., so a user will be able to download it faster) you normally want to put it into a cabinet file, which you might then ship by itself, or (generally preferred) package it up into an MSI package. As far as pre-packaged tools to do this, you'd be looking at Microsoft's cabarc. If you want to do the job in your own code, you can use the File Compression Interface (FCI).
Obligatory side note: although both of these do compression (and also support matching decompression), no I'm not really sure whether either (not to mention both) is actually implemented using RtlCompressBuffer. I don't believe Microsoft's documentation specifies their implementation in nearly that level of detail.
Hello I need to load NSKeyedArchiver classes to C++/CLI counterparts. Is there any way to get internal format of NSKeyedArchiver?
Another option is to rewrite whole saving and opening code into pure C++ for both Mac and Windows.
Thanks a lot.
So your constraints seem to be:
Something easy to parse on both Windows and Mac.
3D modeling data, so the format needs to be efficient
If you want something quick and dirty, go with a binary plist (and possibly gzip it). You could also try Google's protocol buffers but I have no experience with that. There are also a couple open source implementations of Foundation out there -- Cocotron might be most useful for you.
If you're not serializing anything Objective-C specific then just use XML property lists.