I'm developing a set of shell scripts and for ease of development, the functions are often split across various files.
So the final binary scripts which I expect the end user to use require them to have all the relevant "library" scripts installed in the right location.
I am trying to find a way that allows me to develop the scripts with the same logical split in files, but then I can merge them all into a single binary script.
In the naive case, it would recursively go through all the sourced files and include them in the same file (similar to the pre-processing step in C compilers). The more involved version would also identify which functions are unused and trim them out.
Does anything like this exist? If not, I might consider writing it, but would be happy to hear about potential pitfalls that I should account for
I have seen this before, in Arch Linux' devtools repo. They use m4 to process .in files.
It's just templating though. And you might not need anything more.
Related
This feels too obvious to be unanswered, but if the answer is out there, I haven't found it. For context, I'm incorporating someone else's existing code into a Bazel build, so I'm really not looking for "just don't do it that way"-type answers.
The code produces man dozen related files: Libraries, compiled binaries (from C and C++, if that matters), python and shell scripts, etc. Those files expect to find each other in specific locations (e.g. shell scripts reference binaries by relative or absolute path), and I need to package up and install the whole lot.
Is there a way to do that in Bazel? To pick out a bunch of bazel-generated files (and, in this case, a bunch of input files that we pass through unmodified) and put them in a tarball, or a standard package format (e.g. .deb) or even just place them in the local file system in known locations?
The closest ideas I've seen involve basically doing it by hand (e.g. writing a shell script to go into Bazel's output directory and copy out the files of interest) but that seems easy to get wrong. There has to be a way to use the intelligence of the build system to bundle up a bunch of targets and data files, right?
Naturally, I find what's probably the answer shortly after posting the question: https://docs.bazel.build/versions/master/be/pkg.html. If anyone has further insight, though, I'm definitely happy to hear it!
I am developing an OS for embedded devices that runs bytecode. Basically, a micro JVM.
In the process of doing so, I am able to compile and run Java applications to bytecode(ish) and flash that on, for instance, an Atmega1284P.
Now I've added support for C applications: I compile and process it using several tools and with some manual editing I eventually get bytecode that runs on my OS.
The process is very cumbersome and heavy and I would like to automate it.
Currently, I am using makefiles for automatic compilation and flashing of the Java applications & OS to devices.
All steps, roughly, for a C application are as follows and consist of consecutive manual steps:
(1) Use Docker to run a Linux container with lljvm that compiles a .c file to a .class file (see also https://github.com/davidar/lljvm/tree/master)
(2) convert this c.class file to a jasmin file (https://github.com/davidar/jasmin) using the ClassFileAnalyzer tool (http://classfileanalyzer.javaseiten.de/)
(3) manually edit this jasmin file in a text editor by replacing/adjusting some strings
(4) convert the modified jasmin file to a .class file again using jasmin
(5) put this .class file in a folder where the rest of my makefiles (the ones that already make and deploy the OS and class files from Java apps) can take over.
Current options seem to be just keep using makefiles but this is a bit unwieldly (I already have 5 different makefiles and this would further extend that chain). I've also read a bit about scons. In essence, I'm wondering which are some recommended tools or a good approach for complicated builds.
Hopefully this may help a bit, but the question as such could probably be a subject for a heated discussion without much helpful results.
As pointed out in the comments by others, you really need to automate the steps starting with your .c file to the point you can integrated it with the rest of your system.
There is generally nothing wrong with make and you would not win too much by switching to SCons. You'd get more ways to express what you want to do. Among other things meaning that if you wanted to write that automation directly inside the build system and its rules, you could also use Python and not only shell (should that be of a concern though, you could just as well call that Python code from make). But the essence of target, prerequisite, recipe is still there. And with that need for writing necessary automation for those .c to integration steps.
If you really wanted to look into alternative options. bazel might be of interest to you. The downside being the initial effort to write the necessary rules to fit your needs could be costly. And depending on size of your project, might just be too much. On the other hand once done with that, it'd be very easy to use (apply those rules on growing code base) and you could also ditch the container and rely on its more lightweight sand-boxing and external rules to get the tools and bits you need for your build... all with a single system for build description.
Can I make a ruby file (e.g script.rb) unreadable to a user?
The file is on an Ubuntu (offline) machine. The user will use a local Sinatra app that will use some ruby files. I don't want the user to see the code in some of those files.
How can I do it?
EDIT:
Can I setup the project in a way that the user will be able to start the app but won't have access to specific files in it?
Thanks
Does that correspond to what you are searching for ?
chmod yourfile.rb 711
As I said in my comment it is literally almost impossible to hide the content of your ruby source file, many people try this in many different ways but it is almost always trivial to reverse engineer. There are some "suggestions" for making your code hidden but they never really work still, here are a few;
Obfuscation - The process of making your code executable but unreadable, using a tool like ProGuard for Java (there are ones for most major languages) will try to make your code a mess, and as unreadable as possible while still maintaining execution speed. Normally this consists of renaming variables, using strange characters and generally hiding, moving or wrapping functions in complicated structures.
Package the interpreter - You can use a tool like ocra to package the script up inside an executable with the interpreter and standard library, but anyone with even a tiny bit of experience in reverse engineering will be able to easily tear out the source code given a small amount of time
Write a custom interpreter - Now we are getting somewhere with making it harder. Writing a custom interpreter will allow you to compile your script to a "bytecode" that can then be executed. This is of course a very time consuming, expensive and incompatible solution when it comes to working with other code bases.
Write most of your code in C and then call out to it via extensions - Again this mostly moves the problem but its still there. It will take more time but anyone can easily pull apart the machine code of the C library you load in and bob is your uncle they have the source code.
Many more alternatives - This isn't a comprehensive list, I am probably missing a few ideas or suggestions.
As far as it goes making code unreadable is hard a better solution might just to be consider providing a licence agreement with your code. That way, someone reads or modifies the source file you can take them to court for a legal settlement.
Extract your code and its functionality to an external API. And then provide it as a service. This way you don't have to expose your source code to your 'users'.
I work on software that requires access to DLLs, usually wants environment variables set a certain way, and can take command line parameters. I'm generally opposed to setting/modifying system-wide environment variables for the purpose of launching these apps, since I might want to use different dependencies (different dlls), etc. and I don't want to accidentally get the wrong DLL loaded.
Up until now, I've been generating the visual studio .vcproj.user files and matching batch files from cmake ( see here for my script ) that extends the path to include the path to my dlls, sets any other environment variables as needed, and launches the application, forwarding all the command line arguments. (I do the same on Linux, but it's simpler because of RPATH). When we want to launch with a command line argument that wasn't just a file we could drop on the batch file, what we did is copy the batch file and edit the command to add our argument. (It's all GUI applications, but config files/flags can be passed on the command line)
This has become quite a hassle for me and my colleagues, and we end up with a ton of batch files, named similarly and difficult to maintain. It's not really a great interface for starting but there's a lot of apps we either use or develop that are like this, so it's a common task.
My question is this: I'd like a nicer way to configure dynamic library search paths, process-local environment variables, working directory, executable, and arguments for starting a program than hand-editing a batch file every time. I've done some digging to try to find one, but haven't been successful - "launcher" tends to bring me to "search-as-you-type" tools for frequently-used apps, and "command line argument" tends to find recommendations to use batch files. (It's not a windows-specific issue, either, but the dynamic library path stuff is less problematic during the code/compile/run cycle)
I've made a quick mockup of what I'm envisioning: it seems like the kind of thing that has to exist somewhere. Do you know of a tool similar to this (preferably cross-platform, and open source is even better)? It doesn't need all bells and whistles I put in the mockup, but that's what I'd build if I had time to build it myself. Thanks!
A sysadmin teacher told me one day that I should learn to use "make" because I could use it for a lot of other things that just triggering complilations.
I never got the chance to talk longer about it. Do you have any good example ?
As a bonus, isn't it this tool deprecated, and what are modern alternatives (for the compilation purpose and others) ?
One excellent thing make can be used for besides compilation is LaTeX. If you're doing any serious work with LaTeX, you'll find make very handy because of the need to re-interpret .tex files several times when using BibTex or tables of contents.
Make is definitely not deprecated. Although there are different ways of doing the same thing (batch files on Windows, shell scripts on Linux) make works the best, IMHO.
Make can be used to execute any commands you want to execute. It is best used for activities that require dependency checking, but there is no reason you couldn't use make to check your e-mail, reboot your servers, make backups, or anything else.
Ant, NAnt, and msbuild are supposedly the modern alternatives, but plain-old-make is still used extensively in environments that don't use Java or .NET.
isn't it this tool deprecated
What?! No, not even slightly. I'm on Linux so I realise I'm not an average person, but I use it almost daily. I'm sure there are thousands of Linux devs who do use it daily.
I remember seeing an article on Slashdot a few years ago describing a technique for optimising Linux boot sequence by using make.
edit:
Here's an article from IBM explaining the principle.
Make performs a topological sort, which is to say that given a bunch of things, and a set of requirements that one thing be before another thing, it finds a way to order all of the things so that all of the requirements are met. Building things (programs, documents, distribution tarballs, etc.) is one common use for topological sorting, but there are others. You can create a Makefile with one entry for every server in your data center, including dependencies between servers (NFS, NIS, DNS, etc.) and make can tell you what order in which to turn on your computers after a power outage, or what order to turn them off in before a power outage. You can use it to figure out what order in which to start services on a single server. You can use it to figure out what order to put your clothes on in the morning. Any problem where you need to find an order of a bunch of things or tasks that satisfies a bunch of specific requirements of the form A goes before B is a potential candidate for being solved with make.
The most random use I've ever seen is make being used in place of bash for init scripts on BCCD. It actually worked decently, once you got over the wtf moment....
Think of make as shell scripts with added oomph.
Well, I sure that the UNIX tool "make" is still being used a lot, even if it's waning in the .Net world. And while more people may be using MSBUILD, Ant, nAnt, and others tools these days, they are essentially just "make" with a different file syntax. The basic concept is the same.
Make tools are handy for anything where there's an input file which is processed into an output file. Write your reports in MSWord, but distribute them as PDFs? -- use make to generate the PDFs.
Configuration file changes through crontab, if needed.
I have examples for postfix maps, and for squid external tables.
Example for /etc/postfix/Makefile:
POSTMAP=/usr/sbin/postmap
POSTFIX=/usr/sbin/postfix
HASHES=transport access virtual canonical relocated annoying_senders
BTREES=clients_welcome
HASHES_DB=${HASHES:=.db}
BTREES_DB=${BTREES:=.db}
all: ${BTREES_DB} ${HASHES_DB} aliases.db
echo \= Done
${HASHES_DB}: %.db: %
echo . Rebuilding $< hash...
${POSTMAP} $<
${BTREES_DB}: %.db: %
echo . Rebuilding $< btree...
${POSTMAP} $<
aliases.db: aliases
echo . Rebuilding aliases...
/usr/bin/newaliases
etc