Is there a Linux equivalent of Windows' "resource files"? - gcc

I have a C library, which I build as a shared object for Linux and a DLL for Windows with MinGW32. The API depends on a couple of data files (statistical models) which I'd really like to roll in with the SO/DLL so that deployment is just one file.
It looks like I can achieve this for Windows with a "resource file" compiled with windres, but then I've got to write a bunch of resource-handling code for Windows, and I'm still stuck with the files on Linux.
Is there a way to achieve the same functionality on Linux?
Even better, is there a portable solution?

It's actually quite simple on Linux and other ELF systems: http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967
OS X has bundles, so you just build your library as a framework and put the file in the bundle.

Two potential solutions:
Phong Vo's sfio library, which is part of the AT&T Advanced Software Technology toolset, is a wonderful replacement for C stdio.h, and it will allow you to open either files or memory blocks using a single API. So you can easily convert your existing files to C initialized data to include in your DLL or SO file.
This is a good cross-platform solution, but the penalty is that the learning curve to get started is pretty high. They don't make it easy to figure out how stuff works or to take one part of their toolset and split it out for use independent of the other parts. But the good news is that if you want to adopt their U/Win system for running Unix codes on windows (all part of the same toolset), you can create DLLs and SOs using the same system.
For this kind of problem I often fall back on Lua; I can stored Lua data either in external files or within C as initialized data. This is great for distributing everything in one .so file; I do this for my students.
Again the downside is that you have to master and incorporate a new technology.
In my own work I use Lua over the AT&T stuff for these reasons:
Lua has a much smaller footprint and is designed to play well with others; with AST you really have do adopt their way of doing things.
The learning curve with Lua is much less steep; you can be productive very quickly.
Lua is dead easy to install and it's easy to get information about it. AST has its own quirky installation process shared by nobody else in the world; it's often hard to make the installation work; and it's harder to get information about it.
Using Lua has a lot of other payoffs, so the effort spent learning Lua and learning how to incorporate Lua into C codes is easy to amortize over multiple projects.

Related

Why is it that we can't write a program that will run on both mac and pc?

Programming languages are platform independent, so why is it that we can't write a program that will run on both a PC and a Mac?
I want to develop a software and I'm on a mac, but I want it to run on a PC also, is it possible to develop such a software without having to require the user to download a special program that will make my program compatible with their computer?
The problem with this is that most software is dependent on the OS to handle some tasks. Yes, most programming languages are compatible with many platforms, but the OS provides a lot of support. When software uses the OS, it is sometimes called making a system call. If you want here is some more information.
Theoretically if you write your program in a 'high level language' it should be portable between two operating systems.
Practically however, the differences start from the very beginning - the API of choice, which works on one and does not on another(Such as, Mac's BSD API is incompatible with Win32 API) and boils down to the very last, which is, executable format, linker and loader. Each operating system has its own quirks.
Then comes the difference between the underlying architecture. Previously Macs ran on PowerPC architecture and Motorola architectures, while PCs used Intel. Since Macs have switched to Intel, there have been attempts at making cross platform executables inside Apple. Most attempts have failed.
There is however a way around your problem. You can use a very high level language such as Python to code and then distribute your python code to your PC friends.(But remember remember, you need a Python interpreter in your PC friends' computers for your program to run). I have successfully ported Python programs from Mac to PC with 0 code changes, and sometimes requiring only 2-4% code changes.
Simple answer: because language per se is not enough to make an application cross-platform. Also the framework it uses must be cross-platform too, frameworks are required for everything: handling data, displaying things, communicate with the hardware, multi threading, etc
This can usually be done:
by choosing a complete solution like Java, which will actually run on both platforms seamlessly and even with the same binary
by using C/C++ and cross platform libraries so that the same program can be compiled for both platforms (keep in mind that you can't distribute the real same binary, you need to compile two in any case)
by writing the logic of your program just using standard libraries and a standard language and then attach whatever you need for a specific platform just to build two different libraries. Of course you will have to wrap as much as you need so that the cross-platform part of your program doesn't know it
Mind that developing cross-platform applications which are not trivial examples like a game (for which there are plenty of cross-platform APIs) without using a complete solution like Java is not an easy task at all. Especially because most of the GUI you can build are strictly platform specific and relies on their own frameworks.
If you want an application to run "anywhere" your best option is a JIT type language which means that it compiles as it runs (Just In Time) for the platform that it's on. Really the language that stands out in my mind is Java (there's others and personally I don't like java). However, it's not quite that simple. For example a Window on a Mac computer has pieces and functions that a Window on a PC doesn't have and vice versa. And other operating systems don't even have windows or anything equivilant yet still run Java like Android or iOS for example or countless Linux Distros. And that's just a very basic example it gets MUCH MUCH harrier. Really the best way to build an application that can be used by anyone on just about any device is going web based.
The lesson is that if it was that simple a lot of people wouldn't have jobs and it never will be that simple, things will always progress and change and not everyone is going to want to do the same thing with their OS as someone else. There's a million ways to skin a cat and there's many more ways to implement something in an OS.
Yes, it is possible. But it is quite tricky. You need to:
Use a cross platform language (this is the easy part, many languages run on different plaftforms)
Avoid using any platform-specific features (usually not too hard, but needs testing)
Ensure you have cross platform libraries for all your dependencies (hard!)
Because of the library issue in particular, there are very few options that work across platforms. Your best options are probably:
A JVM language (like Java, Scala or Clojure) - because the JVM abstracts away from platform specific features, pure Java applications and libraries will run on any platform. Java probably has the best ecosystem of cross platform libraries and tools as a result.
JavaScript - quite a good option if you don't mind running in a browser. There are lots of quirks to deal with, but JavaScript is one of the best cross-platform options because of it's ubiquity.

Quick and simple programming language

Is there someone that can suggest me a programming language that allows you to write quickly GUI programs (on windows platform)?
P.S. I am interested on only languages that do not rely on virtual machines and then have a compiler that produces executable code directly on the machine
I would go with AutoIT, it's a very easy to learn windows scripting language with tons of functionalities: http://www.autoitscript.com/autoit3/index.shtml
I'm using it to automate some tasks, but it can do way more than that.
EDIT
Just to make things a little bit clearer for everybody:
You can create new applications using AutoIT and the Aut2Exe compiler provided. The .exe files created are stand-alone, thus require no other files but the files that you might need in your app. Everything is free and the AutoIT scripting language has a BASIC-like syntax.
The GUI that you'll use are standard Windows controls. Among the functionalities you have the possibility to automate keystrokes/mouse movements, call the Windows API and external .dlls, manipulate windows and processes and through user created libraries (called UDFs) you can even acces local databases, manage networking tasks, encryption, archiving and many more.
All I can say is that it's worth take a look and the first app I built with AutoIT was done in roughly 8 hours since I started learning. It took a folder as the source and copied everything in a chosen directory, copying files in folders named as the date when the files were created. So the destination directory would have a series of subfolders like:
11.11.2010
whatever.txt
whatever.png
12.11.2010
archive.zip
and so on. Just 8 hours and got me rid of a lot of effort ordering the files myself.
Any .NET will probably be what you're after.
Start with VB.NET which is now called Visual Basic CCYY eg (Visual Basic 2005, Visual Basic 2008, Visual Basic 2010).
If you want something not using .NET framework, you might as well go back to older version of VB and if you want something compilable that'd be like C++ with their MFC (Microsoft Foundation Class).
You need to give more info on the type of gui and what you're using it for. This could be accomplished with Microsoft Access forms and VBA, or you could make an HTML Application (.hta).
I'll put in a vote for Delphi. You can easily write applications by dragging and dropping components on to a form and doing minimal coding in Pascal, which isn't hard to learn. Later, if you decide to go deeper, you can do pretty much whatever you want. And it compiles to native executable code.
Is an executable bundler, that combines the script with the framework/interpreter, good enough?
If so, you might look at Tcl/Tk or Lua.
http://www.powerbasic.com/
http://www.powerbasic.com/aboutpb.asp
Seems like it has a RAD GUI and of course it's BASIC, plus it compiles down to .exe (as I understand it.) Might be worth checking out.
A 'quick and simple' language will only allow you to do 'quick and simple' things - and for those, having a VM or not wont make much of a difference to you.
For quick and simple & native code, about all I can think of is RealBasic. Its cross platform Windows/Mac/Linux. I find their IDE to be difficult to work with due to its inflexibility and the help system last I looked wasnt that great, but the underlying language isnt bad and does compile to native code. So it might do the trick for you.

Portable scripting language for a multi-server admin?

Please Note: Portable as in portableapps.com, not in the traditional sense of a language that can be used on multiple architectures or operating systems. Whoever coined this usage of the word portable should be whacked. :)
I'm a DBA and sysadmin, mostly for Windows machines running SQL Server. I'm looking for a programming/scripting language for Windows that doesn't require Admin access or an installer, needing no install process other than expanding it into a folder. My intent is to have a language for automation around which I can standardize.
Up to this point, I've been using a combination of batch files and Unix shell, using sh.exe from UnxUtils but it's far from a perfect solution.
I've evaluated a handful of options, all of them have at least one serious shortcoming or another. I have a strong preference for something open source or dual license, but I'm more interested in finding the right tool than anything else. Not interested that anything that relies on Cygwin or Java, but at this point I'd be fine with something that needs .NET.
Requirements:
Manageable footprint (1-100 files, under 30 MB installed)
Run on Windows XP and Server (2003+)
No installer (exe, msi)
No reliance on a JVM or Cygwin install
Works with external pipes, processes, and files
Support for MS SQL Server or ODBC connections
Bonus Points:
Open Source
FFI for calling functions in native DLLs
GUI support (native or gtk, wx, fltk, etc)
Linux, AIX, and/or OS X support
Dynamic, object oriented and/or functional, interpreted or bytecode compiled; interactive development
Able to package or compile scripts into executables
So far I've tried:
Ruby: 148 MB on disk, 23000 files
Portable Python: 54 MB on disk, 2800 files
Strawberry Perl: 123 MB on disk, 3600 files
REBOL: Great, except closed source and no MSSQL or ODBC in free version
Squeak Smalltalk: Great, except poor support for scripting
I urge you to try Lua. Regarding your requirements:
Tiny footprint (56 source files, under 150K compiled)
Runs everywhere (uses only ANSI C)
No installer needed; you compile from source (there's also a "batteries included" package that I haven't explored
Doesn't need JVM and works with any ANSI C compiler, so you can compile with Visual Studio, not Cygwin
Works with external processes and files but only to the extent supported by ANSI C. If POSIX popen is provided then that is supported also.
And your bonus points:
Open source (MIT license)
FFI to C is brilliantly conceived and executed—not quite as simple as Tcl but loads more powerful. Much better integration with C than Python or Ruby.
GUI support is mixed but there are good bindings for wx widgets. QT support was there at one time but I don't know if it has been maintained.
Linux is supported
Language/compiler features:
Dynamic
Functional
Prototype-based objects and inheritance through metamethods (you'll want to see examples in the book below
Fastest bytecode compiler in the West
Interactive read-eval-print loop; load new code dynamically
Able to package scripts into executables; either use Luiz de Figueiredo's srlua, or I can send you a 120-line Lua script that converts Lua source to a .c file that you link in with your app and the interpreter to make an executable.
Additional bonus points:
Very crisp, clean, well-designed language.
Small enough to master in its entirety and to be productive within a day.
Superb book Programming in Lua (check out the previous edition free online)
There are a couple of options for Python that might fit your bill:
The first is IronPython, which can be run without an installer and will play nicely with .net APIs. This gives you access to anything with a .net API or a COM typelib that you could build a PIA for. I've used at as a scripting mechanism for precisely this reason - it could be dropped into a directory within the system and did not need to be explicitly installed..You will have to have an appropriate .Net runtime installed, but .Net 2.0 is installed with SQL Server 2005. SQL Server can be accessed through ADO.net and building GUIs with Winforms is fairly straightforward.
The second is Portable Python which is designed to be run off a USB key. Although I see you've already tried it, you might elaborate on what the shortcomings were. If something isn't available in the basic install you could always look into building a custom version with it included. TkInter (at least) is bundled.You can also use Py2EXE to generate standalone python applications with all superfluous junk stripped out. This will give you about 10 files or so (depending on the number of DLLs) that can be run from a single directory, possibly on a USB key.
Running local python installs on Unix-oid OS's is pretty straightforward, so that's pretty much a no brainer. Also, python comes with most linux distros and is available as 'contributed software' from most if not all trad unix vendors. IIRC it's also bundled with MacOS.
Tclkit is a single-file, self-contained Tcl/Tk system. The mac version I have is about 3.8 megs. You can get a version for just about any modern OS. I carry around a thumb drive that has mac, windows and linux binaries so I can run my scripts on any platform. No install is required, just copy one file wherever you want.
The only thing it's missing from your original spec is MS SQL Server / ODBC support out of the box. I know people use tcl for that but I think you'll have to add an extra library or something. See the Tcl'ers wiki entry on MS SQL Server for more information.
For tcl, apart from Tclkit, freewrap is another small portable, self-contained interpreter for tcl.
Just rename the freewrap executable to something else will convert it to a stand-alone interpreter. Renaming it back to freewrap will convert it to a script wrapper.
Also, freewrapped apps contain a tcl interpreter. In dire emergencies you can try opening the app as a zip file and edit/replace the tcl code contained within (just remember to make a copy first). This has saved me several times when I'm at a client site without development tools but need to troubleshoot something. I just make a copy of one of my deployed app and presto - instant development environment!
Looking at wikipedia's exhaustive list of portable software
There's Tiny C compiler, again on Wikipedia here, and its own homepage here.
To summarize by quoting from wikipedia's list of features:
Small - can compile and execute C code everywhere, for example on rescue disks (about 100KB for x86 TCC executable, including C preprocessor, C compiler, assembler and linker).
Fast - tcc generates optimized x86 code. No byte code overhead. It compiles, assembles and links about 9 times faster than GCC.
Any C dynamic library can be used directly. TCC is heading towards full ISOC99 compliance. TCC can of course compile itself.
Includes an optional memory and bound checker. Bound checked code can be mixed freely with standard code.
Compile and execute C source directly. No linking or assembly necessary. Full C preprocessor and GNU-like assembler included.
C script is supported: just add '#!/usr/local/bin/tcc -run' at the first line of your C source, and execute it directly from the command line.
With libtcc, you can use TCC as a backend for dynamic code generation.
Few dependencies. It includes its own hand-written lexer, and it is implemented using a recursive descent parser. Thus, building TCC requires few other libraries.
Its LGPL license permits anyone to use, modify, and/or redistribute the software, and it can be used to develop either open source or proprietary software.
Hope this helps and would be of use,
Best regards,
Tom.
Every somewhat modern Windows version comes pre-installed with both VBScript and JScript. The doesn't meet all your features (compile to an executable comes to mind), but they certainly have an unbeatable advantage with the installation size: it's hard to beat 0.
In addition to the Lua suggestion, there is also Idle. It is basically a superset of Lua 5.1, with both the language (and libraries) and the implementation based on Lua. It was originally created to be a more complete scripting solution for Windows: because Lua is primarly intended for embedding, it has a rather small standard library and it is usually expected that the embedding application provides a rich library to Lua.
This makes sense for an embedded language, because, after all, there isn't much common functionality between, say Adobe Lightroom, Nginx and World of Warcraft, so there simply is nothing you can put in a standard library. But for a more general purpose OS scripting language, one would want a slightly larger library. Thus, Idle bundles a couple of libraries that are third-party (and sometimes hard to get to work on Windows) in Lua in its standard library.
Some of the things that the Idle standard library adds over Lua are tight Win32 integration, SQLite3 support, networking support, a PEG parser generator and archive support.
Also, Idle has support for embedding Perl and C code into your Idle programs.

Is it worth learning GNU Make?

I'm lately feeling the need to learn a build tool. I'm looking through StackOverflow for recommendations and Gnu Make gets barely mentioned. Instead I see Ant, Maven, CMake, Scon and many others. However, when I look at the little "rogue sources" (as in not-in-the-repo) that I sometimes have to compile, they all require the make && make install steps.
Is learning Make a worse investment of my time than learning another tool?
If so why is Make still so popular?
Make is the standard build tool for everything C/C++. Many others have stepped to the plate, but even when they were useful and successful, they never achieved the ubiquity of make.
Make is installed on virtually every Unix-like machine out there. No matter if you're working with AIX, Solaris, Irix, BSD, or Linux, if there's a compiler installed, there's also make.
Some of the "replacements" (like Automake, CMake) even create Makefiles, which are in turn executed by make.
I would definitely recommend becoming familiar with make. If handled by someone who took the time to learn about make, it is a powerful tool, which can be used in a number of ways not even necessarily related to software development.
Even if you end up using a different build tool in the end, you will be able to "recycle" the lessons learned with make, as the underlying concepts are quite similar. And the sheer number of make-built projects means that there will always be the chance that you have to figure out an existing Makefile.
One thing, though. Get it right from the beginning.
I think the reason you don't see (GNU) make mentioned is that it's often the default; if you have a GNU toolchain, you will have make already. Thus, most people that start talking about build tools, talk about something else.
In my experience, make is fine, but it can be kind of tricky to get it to do exactly what you want to. It's maybe slightly arcane, but it's proven and works.
Make is popular because it's used (mainly) for C/C++ sources in Linux/*nix projects, and is far older than any of the other tools you've mentioned, thus it has stood the test of time and is mature. Kinda like tar.
To be honest with you, I only know make. Those other tools above may be better, but so many projects just use a basic Makefile that you're best off knowing at least a little bit of it. Not only for your own projects at work but most of the open-source ones you find on the net.
It really depends how much you will use it.
If yoy work a lot with C/C++ make projects, then yes, I would recommend learning more about it as a large make file has a steeper learning curve than other build tools you mention.
If you don't work with make, or work in other languages such as C#, Java or PHP then you'd be better off learning build tools relevant to those languages.
Like all tools, if you use it at all, you should put some time
into becoming reasonably adept at it. Also, some tools (like CMake, for example) generate makefiles and you may one day need to mess with those generated files.
GNU make has an excellent manual - it's certainly worth spendin an hour or two reading it.
Make is the de-facto standard on Linux systems for example. It is a very complex tool, and also a very powerful tool.
It is well suited to learn if you are developing C or C++, particularly if targeting Linux/*nix.
One of the features of make, is that you can set up dependencies for when to rebuild a file. E.g. each c or c++ file is build into an .obj file, and in the end, all .obj files are linked to an executable. But maybe the executable is a statically linked library, that is linked into another executable with other .obj files.
Make can make sure that you compilation time is as short as possible, because you can define that a c file should only be compiled if it, or any dependent header files, are newer that the .obj file. So any compilation or linking step is only executed if the current source files for the step is newer that the target file.
If you are developing in for example C#, you don't need this kind of dependency checking because all .cs files are compiled at once into a single executable.
So the conclusion is that you should use a build tool that is well suited for your choice of programming language.
Even if you end up preferring another build tool (personally I'm fond of VS... I know...) knowing make will probably prove more useful.
Make has many applications and whilst it is not always ideal for a single task, when dealing with new technologies it is stalwart and flexible.
I guess where you work is probably different, but I know that everywhere I've worked I would have been a far less valuable employee if I hadn't at least learned how to read Makefiles. Even in all Windows-VisualStudio environments, it comes up every now and then.
For instance, we just got a job that involves porting a bunch of old CX/UX code to Windows. The old code was built with makefiles. There's no way to understand their old system without knowing how to read those old makefiles.

Scripting language that can produce a small, independent, Windows EXE?

I'd like to do some light data processing - a little binary data manipulation followed by conversion to text serialization. The result is written to a file, and processed by an external program (run by my program). The data processing is more than I'd care to consider doing in batch files.
I'd prefer to use a scripting language, but not have to install the language first. The target computers are mostly older Windows boxes, which are disconnected from the network (no updates, such as PowerShell)
I'm not familiar with the various language's tools for creating EXE files. Which ones have solutions that work well and don't produce huge files? (i.e., whole interpreter package plus my script.)
For my money (its free) AutoIt 3 is exactly what your looking for. AutoIt produces relatively (250k is the standard overhead) small stand alone exes. It has a full perl like regex engine so your light data processing should be a breeze (I've written some pretty heavy data processing scripts in it myself). When downloading autoit be sure to get the full version including Scite this makes compile to exe a one click operation.
I know I might get flamed for this, but VB 6 is a viable option. Since XP SP2 (I think, possibly earlier), Windows has come with its runtimes installed. Not sure about vista.
Theres also the Windows Scripting Host that uses VBScript and JScript.
http://en.wikipedia.org/wiki/Windows_Script_Host
Lua is an excellent choice for that kind of stuff. You can integrate it in your executable or use the standalone Lua interpreter to run your scripts.
While waiting for answers I ran across Shoes, which can make Ruby .exe (I'm most familiar with Ruby) I got it mostly working, although the size of 2.4MB was a bit larger than I'd like. However, I found that it would crash when changing application focus.
I switched to a 'regular' terminal script, and found rubyscript2exe, which, after working around a problem with rubygems, seems to work, and creates a ~700kb file.
I did rather like some of the options presented, but it's not worth redeveloping at this point.
Python with py2exe. Depends on what you mean by small though.
Would using PowerShell script be something you've considered. The data processing might be richer there.
Why not knock up a .NET application? There are free editions of the IDE, and the Framework comes with Windows as a standard component (which also includes a C# compiler, as it happens.)

Resources