Organizing asset files in a Go project - go

I have a project that contains a folder to manage file templates, but it doesn't look like Go provides any support for non-Go-code project files. The project itself compiles to an executable, but it needs to know where this template folder is in order to operate correctly. Right now I do a search for $GOPATH/src/<templates>/templates, but this feels like kind of a hack to me because it would break if I decided to rename the package or host it somewhere else.
I've done some searching and it looks like a number of people are interested in being able to "compile" the asset files by embedding them in the final binary, but I'm not sure how I feel about this approach.
Any ideas?

Either pick a path (or a list of paths) that users are expected to put the supporting data in (/usr/local/share/myapp, ...) or just compile it into the binary.
It depends on how you are planning to distribute the program. As a package? With an installer?
Most of my programs I enjoy just having a single file to deploy and I just have a few templates to include, so I do that.
I have an example using go-bindata where I build the html template with a Makefile, but if I build with the 'devel' flag it will read the file at runtime instead to make development easier.

I can think of two options, use a cwd flag, or infer from cwd and arg 0:
-cwd path/to/assets
path/to/exe -cwd=$(path/to/exe/assets)
Internally, the exectable would chdir to wherever cwd points to, and then it can use relative paths throughout the application. This has the added benefit that the user can change the assets without having to recompile the program.
I do this for config files. Basically the order goes:
process cmd arguments, looking for a -cwd variable (it defaults to empty)
chdir to -cwd
parse config file
reparse cmd arguments, overwriting the settings in the config file
I'm not sure how many arguments your app has, but I've found this to be very useful, especially since Go doesn't have a standard packaging tool that will compile these assets in.
infer from arg 0
Another option is to use the first argument and get the path to the executable. Something like this:
here := path.Dir(os.Args[0])
if !path.IsAbs(os.Args[0]) {
here = path.Join(os.Getwd(), here)
}
This will get you the path to where the executable is. If you're guaranteed the user won't move this without moving the rest of your assets, you can use this, but I find it much more flexible to use the above -cwd idea, because then the user can place the executable anywhere on their system and just point it to the assets.
The best option would probably be a mixture of the two. If the user doesn't supply a -cwd flag, they probably haven't moved anything, so infer from arg 0 and the cwd. The cwd flag overrides this.

Related

Unable to load/require file from Lua running from Atom in Windows

I'm trying to use Atom to run a Lua script. However, when I try to load files via the require() command, it always says it's unable to locate them. The files are all in the same folder. For example, to load utils.lua I have tried
require 'utils'
require 'utils.lua'
require 'D:\Users\Mike\Dropbox\Lua Modeling\utils.lua'
require 'D:\\Users\\Mike\\Dropbox\\Lua Modeling\\utils.lua'
require 'D:/Users/Mike/Dropbox/Lua Modeling/utils.lua'
I get errors like
Lua: D:\Users\Mike\Dropbox\Lua Modeling\main.lua:12: module 'D:\Users\Mike\Dropbox\Lua Modeling\utils.lua' not found:
no field package.preload['D:\Users\Mike\Dropbox\Lua Modeling\utils.lua']
no file '.\D:\Users\Mike\Dropbox\Lua Modeling\utils\lua.lua'
no file 'D:\Program Files (x86)\Lua\5.1\lua\D:\Users\Mike\Dropbox\Lua Modeling\utils\lua.lua'
no file 'D:\Program Files (x86)\Lua\5.1\lua\D:\Users\Mike\Dropbox\Lua Modeling\utils\lua\init.lua'
no file 'D:\Program Files (x86)\Lua\5.1\D:\Users\Mike\Dropbox\Lua Modeling\utils\lua.lua'
The messages says on the first line that 'D:\Users\Mike\Dropbox\Lua Modeling\utils.lua' was not found, even though that is the full path of the file. What am I doing wrong?
Thanks.
The short answer
You should be able to load utils.lua by using the following code:
require("utils")
And by starting your program from the directory that utils.lua is in:
cd "D:\Users\Mike\Dropbox\Lua Modeling"
lua main.lua
The long answer
To understand what is going wrong here, it is helpful to know a little bit about how require works. The first thing that require does is to search for the module in the module path. From Programming in Lua chapter 8.1:
The path used by require is a little different from typical paths. Most programs use paths as a list of directories wherein to search for a given file. However, ANSI C (the abstract platform where Lua runs) does not have the concept of directories. Therefore, the path used by require is a list of patterns, each of them specifying an alternative way to transform a virtual file name (the argument to require) into a real file name. More specifically, each component in the path is a file name containing optional interrogation marks. For each component, require replaces each ? by the virtual file name and checks whether there is a file with that name; if not, it goes to the next component. The components in a path are separated by semicolons (a character seldom used for file names in most operating systems). For instance, if the path is
?;?.lua;c:\windows\?;/usr/local/lua/?/?.lua
then the call require"lili" will try to open the following files:
lili
lili.lua
c:\windows\lili
/usr/local/lua/lili/lili.lua
Judging from your error message, your Lua path seems to be the following:
.\?.lua;D:\Program Files (x86)\Lua\5.1\lua\?.lua;D:\Program Files (x86)\Lua\5.1\lua\?\init.lua;D:\Program Files (x86)\Lua\5.1\?.lua
To make that easier to read, here are each the patterns separated by line breaks:
.\?.lua
D:\Program Files (x86)\Lua\5.1\lua\?.lua
D:\Program Files (x86)\Lua\5.1\lua\?\init.lua
D:\Program Files (x86)\Lua\5.1\?.lua
From this list you can see that when calling require
Lua fills in the .lua extension for you
Lua fills in the rest of the file path for you
In other words, you should just specify the module name, like this:
require("utils")
Now, Lua also needs to know where the utils.lua file is. The easiest way is to run your program from the D:\Users\Mike\Dropbox\Lua Modeling folder. This means that when you run require("utils"), Lua will expand the first pattern .\?.lua into .\utils.lua, and when it checks that path it will find the utils.lua file in the current directory.
In other words, running your program like this should work:
cd "D:\Users\Mike\Dropbox\Lua Modeling"
lua main.lua
An alternative
If you can't (or don't want to) change your working directory to run the program, you can use the LUA_PATH environment variable to add new patterns to the path that require uses to search for modules.
set LUA_PATH=D:\Users\Mike\Dropbox\Lua Modeling\?.lua;%LUA_PATH%;
lua "D:\Users\Mike\Dropbox\Lua Modeling\main.lua"
There is a slight trick to this. If the LUA_PATH environment variable already exists, then this will add your project's folder to the start of it. If LUA_PATH doesn't exist, this will add ;; to the end, which Lua fills in with the default path.

autoconf: how do I substitute the library prefix?

CLISP's interface to PARI is configured with the configure.in containing AC_LIB_LINKFLAGS([pari]) from lib-link.m4.
The build process also requires the Makefile to know where the datadir of PARI is located. To this end, Makefile.in has
prefix = #LIBPARI_PREFIX#
DATADIR = #datadir#
and expects to find $(DATADIR)/pari/pari.desc (normally
/usr/share/pari/pari.desc or /usr/local/share/pari/pari.desc).
This seems to work on Mac OS X where PARI is installed by homebrew in /usr/local (and LIBPARI_PREFIX=/usr/local), but not on Ubuntu, where PARI is in /usr, and LIBPARI_PREFIX is empty.
How do I insert the location of the PARI's datadir into the Makefile?
PS. I also asked this on the autoconf mailing list.
PPS. In response to #BrunoHaible's suggestion, here is the meager attempt at debugging on Linux (where LIBPARI_PREFIX is empty).
$ bash -x configure 2>&1 | grep found_dir
+ found_dir=
+ eval ac_val=$found_dir
+ eval ac_val=$found_dir
You are trying to use $(prefix) in an unintended way. In an Autotools-based build system, the $(prefix) represents a prefix to the target installation location of the software you're building. By setting it in your Makefile.in, you are overriding the prefix that configure will try to assign. However, since you appear not to have any installation targets anyway, at least at that level, that's probably more an issue of poor form than a cause for malfunction.
How do I insert the location of the PARI's datadir into the Makefile?
I'd recommend computing or discovering the needed directory in your configure script, and exporting it to the generated Makefile via its own output variable. Let's take the second part first, since it's simple. In configure.in, having in some manner located the wanted data directory and assigned it to a variable
DATADIR=...
, you would make an output variable of that via the AC_SUBST macro:
AC_SUBST([DATADIR])
Since you are using only Autoconf, not Automake, you would then manually receive that into your Makefile by changing the assignment in your Makefile.in:
DATDIR = #DATADIR#
Now, as for locating the data directory in the first place, you have to know what you're trying to implement before you can implement it. From your question and followup comments, it seems to me that you want this:
Use a data directory explicitly specified by the user if there is one. Otherwise,
look for a data directory relative to the location of the shared library. If it's not found there then
(optional) look under the prefix specified to configure, or specifically in the specified datadir (both of which may come from the top-level configure). Finally, if it still has not been found then
look in some standard locations.
To create a configure option by which the user can specify a custom data directory, you would probably use the AC_ARG_WITH macro, maybe like this:
AC_ARG_WITH([pari-datadir], [AS_HELP_STRING([--with-pari-datadir],
[explicitly specifies the PARI data directory])],
[], [with_pari_datadir=''])
Thanks to #BrunoHaible, we see that although the Gnulib manual does not document it, the macro's internal documentation specifies that if AC_LIB_LINKFLAGS locates libpari then it will set LIBPARI_PREFIX to the library directory prefix. You find that that does work when the --with-libpari option is used to give it an alternative location to search, so I suggest working with that. You certainly can try to debug AC_LIB_LINKFLAGS to make it set LIBPARI_PREFIX in all cases in which the lib is found, but if you don't want to go to that effort then you can work around it (see below).
Although the default or specified installation prefix is accessible in configure as $prefix, I would suggest instead going to the specified $datadir. That is slightly tricky, however, because by default it refers to the prefix indirectly. Thus, you might do this:
eval "datadir_expanded=${datadir}"
Finally, you might hardcode a set of prefixes such as /usr and /usr/local.
Following on from all the foregoing, then, your configure.in might do something like this:
DATADIR=
for d in \
${with_pari_datadir} \
${LIBPARI_PREFIX:+${LIBPARI_PREFIX}/share/pari} \
${datadir_expanded}/pari \
/usr/local/share/pari \
/usr/share/pari
do
AS_IF([test -r "$[]d/pari.desc"], [DATADIR="$[]d"; break])
done
AS_IF([test x = "x$DATADIR"], [AC_MSG_ERROR(["Could not identify PARI data directory"])])
AC_SUBST([DATADIR])
Instead of guessing the location of datadir, why don't you ask PARI/GP where its datadir is located? Namely,
$ echo "default(datadir)" | gp -qf
"/usr/share/pari"
does the trick.

Trying to open a file in C++, but the file cannot be found

I have an algorithm in C++ (main.cpp) and I use CLion to compile and run it. Algorithm would read strings from text file, but there is a mistake:
Could not open data.txt (file exists and placed in one folder with main.cpp)
How can I fix it and make this file "visible" to CLion?
If you are using fopen or something similar and just passing "data.txt", it is assumed that that file is in the current working directory of the running program (the one you just compiled).
So, either
Give a full path instead, like fopen("/full/path/to/data.txt"), where you use the actual full path
(not preferable), Move data.txt to the directory where CLion runs its compiled programs from.
(for #2, here's a hacky way to get that directory)
char buf[1024]; // hack, but fine for this
printf("%s\n", getcwd(buf, 1024));
Run/Edit configurations...
Select your application (on the lefthandside of the window)
Specify Working directory
Apply
Now you can fopen relatively from working directory.
I found another way to solve this problem.
#Lou Franco's solution may affect the project structure. For example, if I deploy code on a server, I should move the resource file to specific directory.
What I do is modify the CmakeLists.txt, on Windows, using
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "D:\\science\\code\\English-Prediction")
CMAKE_RUNTIME_OUTPUT_DIRECTORY is a CMake variable, it assigns the work directory of CLion work directory.
Continuing with the CMAKE_RUNTIME_OUTPUT_DIRECTORY CMakeLists variables, I do the following. In the root directory of my project, I create a directory, e.g., out. Then, in my CMakeLists.txt I set the CMAKE_RUNTIME_OUTPUT_DIRECTORY to that directory:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR}/out)
Note, that must come before you have
add_executable(YourProject ${SOURCE_FILES})
I might also add that instead of using fopen() I would keep it more object-oriented by using std::ifstream:
std::ifstream inFile("data.txt");
// check if it opened without issue...
if (!inFile) {
processError(); // a user-defined function to deal with the issue
} else {
// All is good, carry on...
// and when you're done don't forget
inFile.close();
}

Get result of compilation as single file with ASDF

Is it possible to tell ASDF that it should produce only one fas(l) file for entire system? This file should be concatenation (in right order) of all compiled files of the system, including all files of systems on which target system depends.
Yes, with compile-bundle-op (ASDF 3.1): http://common-lisp.net/project/asdf/asdf/Predefined-operations-of-ASDF.html
edit: Actually, monolithic-compile-bundle-op seemes to be asked for (as shown in other answers).
If you have to predict the extension, use uiop:compile-file-type.
And/or you can just call (asdf:output-files 'asdf:monolithic-compile-bundle-op :my-system) to figure out what is actually used.
Option monolithic-compile-bundle-op will create single compiled file which includes all dependencies, while compile-bundle-op creates a file for every system.
Example of use:
(asdf:operate 'asdf:monolithic-compile-bundle-op :my-system)
This command will create file my-system--all-systems.fas(l) in output directory of target project, as well as "bundle" files for every system, named like my-system--system.fas(l).

Join multiple Coffeescript files into one file? (Multiple subdirectories)

I've got a bunch of .coffee files that I need to join into one file.
I have folders set up like a rails app:
/src/controller/log_controller.coffee
/src/model/log.coffee
/src/views/logs/new.coffee
Coffeescript has a command that lets you join multiple coffeescripts into one file, but it only seems to work with one directory. For example this works fine:
coffee --output app/controllers.js --join --compile src/controllers/*.coffee
But I need to be able to include a bunch of subdirectories kind of like this non-working command:
coffee --output app/all.js --join --compile src/*/*.coffee
Is there a way to do this? Is there a UNIXy way to pass in a list of all the files in the subdirectories?
I'm using terminal in OSX.
They all have to be joined in one file because otherwise each separate file gets compiled & wrapped with this:
(function() { }).call(this);
Which breaks the scope of some function calls.
From the CoffeeScript documentation:
-j, --join [FILE] : Before compiling, concatenate all scripts together in the order they were passed, and write them into the specified file. Useful for building large projects.
So, you can achieve your goal at the command line (I use bash) like this:
coffee -cj path/to/compiled/file.js file1 file2 file3 file4
where file1 - fileN are the paths to the coffeescript files you want to compile.
You could write a shell script or Rake task to combine them together first, then compile. Something like:
find . -type f -name '*.coffee' -print0 | xargs -0 cat > output.coffee
Then compile output.coffee
Adjust the paths to your needs. Also make sure that the output.coffee file is not in the same path you're searching with find or you will get into an infinite loop.
http://man.cx/find |
http://www.rubyrake.org/tutorial/index.html
Additionally you may be interested in these other posts on Stackoverflow concerning searching across directories:
How to count lines of code including sub-directories
Bash script to find a file in directory tree and append it to another file
Unix script to find all folders in the directory
I've just release an alpha release of CoffeeToaster, I think it may help you.
http://github.com/serpentem/coffee-toaster
The most easy way to use coffee command line tool.
coffee --output public --join --compile app
app is my working directory holding multiple subdirectories and public is where ~output.js file will be placed. Easy to automate this process if writing app in nodejs
This helped me (-o output directory, -j join to project.js, -cw compile and watch coffeescript directory in full depth):
coffee -o web/js -j project.js -cw coffeescript
Use cake to compile them all in one (or more) resulting .js file(s). Cakefile is used as configuration which controls in which order your coffee scripts are compiled - quite handy with bigger projects.
Cake is quite easy to install and setup, invoking cake from vim while you are editing your project is then simply
:!cake build
and you can refresh your browser and see results.
As I'm also busy to learn the best way of structuring the files and use coffeescript in combination with backbone and cake, I have created a small project on github to keep it as a reference for myself, maybe it will help you too around cake and some basic things. All compiled files are in www folder so that you can open them in your browser and all source files (except for cake configuration) are in src folder. In this example, all .coffee files are compiled and combined in one output .js file which is then included in html.
Alternatively, you could use the --bare flag, compile to JavaScript, and then perhaps wrap the JS if necessary. But this would likely create problems; for instance, if you have one file with the code
i = 0
foo = -> i++
...
foo()
then there's only one var i declaration in the resulting JavaScript, and i will be incremented. But if you moved the foo function declaration to another CoffeeScript file, then its i would live in the foo scope, and the outer i would be unaffected.
So concatenating the CoffeeScript is a wiser solution, but there's still potential for confusion there; the order in which you concatenate your code is almost certainly going to matter. I strongly recommend modularizing your code instead.

Resources