I have TopoJSON file and want to add some external fields in it by id (and preserve old) without any other changes (like simplification or quantization) . So I used command like in the example in TopoJSON wiki. The command was like:
topojson -o world-110m_ext.json -e world-country-names.tsv --id-property +id -p nameENG=name world-110m.json.
But I got file smaller than the source and without any id properties. I tried to preserve id's like so:
topojson -o world-110m_ext.json -p -e world-country-names.tsv --id-property +id -p nameENG=name world-110m.json.
It did not work for me and many other ways too. What am I doing wrong?
For input I am using Mike's files from this gist.
Related
I have some configuration templates which use FMPP to generate the
real runtime config files based upon info in a csv and properties
file (defined in config.fmpp).
I want to be able to configure a second cluster server for the same task using the same set of templates and config.fmpp information. However, there are slight differences needed in the generated runtime config and I can do this if I know which server instance I am on ("serverA" or "serverB") using a standard fmpp variable like ${myserver}.
But there must only be one set of templates and FMPP config files so I need to somehow get the value of "myserver" from the runtime
environment in each server.
Some of the options I might have are:
pass value of myserver on the command line tool invocation (best way); or
get it from an environment variable.
Does anyone have an example of the code to do any of these and any suggestions of the best approach? Online reference would be great.
fmpp -S /home/me/sample-project/src -Param myserver:serverA
Environment settings:
fmpp v0.9.14
freemarker v2.3.19
Use the -D command line option (see --help):
-D, --data=<TDD> Creates shared data that all templates will see. <TDD> is the
Textual Data Definition, e.g.:
-D "properties(style.properties), onLine:true"
Note that paths like "style.properties" are relative to the
data root directory.
Like:
fmpp -S /home/me/sample-project/src -D myserver:serverA
Note that there's a space after the -D. (It's not like the java command line syntax, but rather like the standard GNU command line syntax.
This -D has nothing to do with Java's -D option.
The documentation shows onLine:true, but such Boolean values are legacy and no longer accepted. Use online:yes to parse Boolean values.
For example:
fmpp \
-S /path/ \
--verbose \
-D "online:yes"
Then, within the template:
<p>
online: ${online}
</p>
Will result in:
online: yes
The --verbose command-line parameter is useful to show any errors when parsing the template.
I would like to combine three geojson files into one topojson file. two of these files are pretty straightforward shapefiles. they need no simplifying.
one of them is the 10m cultural natural earth vector file. it's geojson is 24.8mb. I need the 10m because i need small islands to remain on the map (though i'll likely simplify them out but retain them with --allow-empty).
is it possible to simplify the 10m cultural file with topojson, then combine it with the other two geojson files using topojson but without simplifying?
or is this a totally crooked approach, and i should be approaching in another way?
thanks for any help
You may simplify them separately and merge them.
Given you have your 3 shapefiles in your folder, you could use a makefile a bit such :
#Merge:
final.json: map1.json map2.json world10m.json
topojson --id-property none --allow-empty -o final.json -- map1.json map2.json world10m_s.json
# Simplify:
world10m_s.json -- world10m.json
topojson --id-property none --allow-empty --simplify 0.5 -p name=Name -o world10m_s.json -- world10m.json
# SHP2JSON
world10m.json: world10m.shp
ogr2ogr -f GeoJSON world10m.json world10m.shp
map1.json: map1.shp
ogr2ogr -f GeoJSON map1.json map1.shp
map2.json: map2.shp
ogr2ogr -f GeoJSON map2.json map2.shp
Note: There are plenty of good reasons Why Use Make
Note2: Since topojson script also accept .shp for input, we could skip the #SHP2JSON task.
Okay so here is what I want to do. I want to add a print option that prints whatever the user's document is to a PDF and adds some headers before sending it off to a device.
I guess my questions are: how do I add a virtual "printer" driver for the user that will launch the application I've been developing that will make the PDF (or make the PDF and launch my application with references to the newly generated PDF)? How do I interface with CUPS to generate the PDF? I'm not sure I'm being clear, so let me know if more information would be helpful.
I've worked through this printing with CUPS tutorial and seem to get everything set up okay, but the file never seems to appear in the appropriate temporary location. And if anyone is looking for a user-end PDF-printer, this cups-pdf-for-mac-os-x is one that works through the installer, however I have the same issue of no file appearing in the indicated directory when I download the source and follow the instructions in the readme. If anyone can get either of these to work on a mac through the terminal, please let me know step-by-step how you did it.
The way to go is this:
Set up a print queue with any driver you like. But I recommend to use a PostScript driver/PPD. (A PostScript PPD is one which does not contain any *cupsFilter: ... line.):
Initially, use the (educational) CUPS backend named 2dir. That one can be copied from this website: KDE Printing Developer Tools Wiki. Make sure when copying that you get the line endings right (Unix-like).
Commandline to set up the initial queue:
lpadmin \
-p pdfqueue \
-v 2dir:/tmp/pdfqueue \
-E \
-P /path/to/postscript-printer.ppd
The 2dir backend now will write all output to directory /tmp/pdfqueue/ and it will use a uniq name for each job. Each result should for now be a PostScript file. (with none of the modifications you want yet).
Locate the PPD used by this queue in /etc/cups/ppd/ (its name should be pdfqueue.ppd).
Add the following line (best, near the top of the PPD):
*cupsFilter: "application/pdf 0 -" (Make sure the *cupsFilter starts at the very beginning of the line.) This line tells cupsd to auto-setup a filtering chain that produces PDF and then call the last filter named '-' before it sends the file via a backend to a printer. That '-' filter is a special one: it does nothing, it is a passthrough filter.
Re-start the CUPS scheduler:sudo launchctl unload /System/Library/LaunchDaemons/org.cups.cupsd.plist
sudo launchctl load /System/Library/LaunchDaemons/org.cups.cupsd.plist
From now on your pdfqueue will cause each job printed to it to end up as PDF in /tmp/pdfqueue/*.pdf.
Study the 2dir backend script. It's simple Bash, and reasonably well commented.
Modify the 2dir in a way that adds your desired modifications to your PDF before saving on the result in /tmp/pdfqueue/*.pdf...
Update: Looks like I forgot 2 quotes in my originally prescribed *cupsFilter: ... line above. Sorry!
I really wish I could accept two answers because I don't think I could have done this without all of #Kurt Pfeifle 's help for Mac specifics and just understanding printer drivers and locations of files. But here's what I did:
Download the source code from codepoet cups-pdf-for-mac-os-x. (For non-macs, you can look at http://www.cups-pdf.de/) The readme is greatly detailed and if you read all of the instructions carefully, it will work, however I had a little trouble getting all the pieces, so I will outline exactly what I did in the hopes of saving someone else some trouble. For this, the directory with the source code is called "cups-pdfdownloaddir".
Compile cups-pdf.c contained in the src folder as the readme specifies:
gcc -09 -s -lcups -o cups-pdf cups-pdf.c
There may be a warning: ld: warning: option -s is obsolete and being ignored, but this posed no issue for me. Copy the binary into /usr/libexec/cups/backend. You will likely have to the sudo command, which will prompt you for your password. For example:
sudo cp /cups-pdfdownloaddir/src/cups-pdf /usr/libexec/cups/backend
Also, don't forget to change the permissions on this file--it needs root permissions (700) which can be changed with the following after moving cupd-pdf into the backend directory:
sudo chmod 700 /usr/libexec/cups/backend/cups-pdf
Edit the file contained in /cups-pdfdownloaddir/extra/cups-pdf.conf. Under the "PDF Conversion Settings" header, find a line under the GhostScript that reads #GhostScript /usr/bin/gs. I did not uncomment it in case I needed it, but simply added beneath it the line Ghostscript /usr/bin/pstopdf. (There should be no pre-cursor # for any of these modifications)
Find the line under GSCall that reads #GSCall %s -q -dCompatibilityLevel=%s -dNOPAUSE -dBATCH -dSAFER -sDEVICE=pdfwrite -sOutputFile="%s" -dAutoRotatePage\
s=/PageByPage -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dPDFSETTINGS=/prepress -c .setpdfwrite \
-f %s Again without uncommenting this, under this I added the line GSCall %s %s -o %s %s
Find the line under PDFVer that reads #PDFVer 1.4 and change it to PDFVer, no spaces or following characters.
Now save and exit editing before copying this file to /etc/cups with the following command
sudo cp cups-pdfdownloaddir/extra/cups-pdf.conf /etc/cups
Be careful of editing in a text editor because newlines in UNIX and Mac environments are different and can potentially ruin scripts. You can always use a perl command to remove them, but I'm paranoid and prefer not to deal with it in the first place.
You should now be able to open a program (e.g. Word, Excel, ...) and select File >> Print and find an available printer called CUPS-PDF. Print to this printer, and you should find your pdfs in /var/spool/cups-pdf/yourusername/ by default.
*Also, I figured this might be helpful because it helped me: if something gets screwed up in following these directions and you need to start over/get rid of it, in order to remove the driver you need to (1) remove the cups-pdf backend from /usr/libexec/cups/backend (2) remove the cups-pdf.conf from /etc/cups/ (3) Go into System Preferences >> Print & Fax and delete the CUPS-PDF printer.
This is how I successfully set up a pdf backend/filter for myself, however there are more details, and other information on customization contained in the readme file. Hope this helps someone else!
I tried deploying 20- news group example with mahout, it seems working fine. Out of curiosity I would like to dig deep into the model statistics,
for example: bayes-model directory contains the following sub directories,
trainer-tfIdf trainer-thetaNormalizer trainer-weights
which contains part-0000 files. I would like to read the contents of the file for better understanding, cat command doesnt seems to work, it prints some garbage.
Any help is appreciated.
Thanks
The 'part-00000' files are created by Hadoop, and are in Hadoop's SequenceFile format, containing values specific to Mahout. You can't open them as text files, no. You can find the utility class SequenceFileDumper in Mahout that will try to output the content as text to stdout.
As to what those values are to begin with, they're intermediate results of the multi-stage Hadoop-based computation performed by Mahout. You can read the code to get a better sense of what these are. The "tfidf" directory for example contains intermediate calculations related to term frequency.
You can read part-0000 files using hadoop's filesystem -text option. Just get into the hadoop directory and type the following
`bin/hadoop dfs -text /Path-to-part-file/part-m-00000`
part-m-00000 will be printed to STDOUT.
If it gives you an error, you might need to add the HADOOP_CLASSPATH variable to your path. For example, if after running it gives you
text: java.io.IOException: WritableName can't load class: org.apache.mahout.math.VectorWritable
then add the corresponding class to the HADOOP_CLASSPATH variable
export HADOOP_CLASSPATH=/src/mahout/trunk/math/target/mahout-math-0.6-SNAPSHOT.jar
That worked for me ;)
In order to read part-00000 (sequence files) you need to use the "seqdumper" utility. Here's an example I used for my experiments:
MAHOUT_HOME$: bin/mahout seqdumper -s
~/clustering/experiments-v1/t14/tfidf-vectors/part-r-00000
-o ~/vectors-v2-1010
-s is the sequence file you want to convert to plain text
-o is the output file
How to use gettext in bash script?
I only found this page, but I don't understand it.
Localization
My script is written like this:
#!/bin/bash
. lang_file.sh
echo $LANG_HELLO_WORLD
And lang_file.sh look like that:
#!/bin/bash
LANG_HELLO_WORLD="Hello World"
I want to change lang_file.sh to something using gettext, like this:
#!/bin/bash
LANG_HELLO_WORLD=`some gettext command to get string in user language`
I want to use the code in Launchpad, so other users can translate it (.po, .pot files)
Sorry for bad English, any suggestions?
You need to preform following steps:
Determine what's your project name, gettext calls it textdomain, you will need it to retrieve the translations for your project. Let's call it "PRJ".
Mark the strings to be translated. Following snippet gives example:
Let's call it PRJ.sh:
#!/bin/sh
alias GETTEXT='gettext "PRJ"'
## Use GETTEXT to mark the string you want to translate
HELLO_WORLD=$(GETTEXT "Hello world")
echo "$HELLO_WORLD"
Produce .pot file so translators can work on it.
Run the following command, it only looks for GETTEXT, the ones you actually want to translate.
xgettext -o PRJ.pot -L Shell --keyword --keyword=GETTEXT PRJ.sh
Optional: Generate .po files.
For each locale you want to cover.
msginit -i PRJ.pot -l fr.UTF-8
Note that "UTF-8" is sugggested, otherwise msginit may mistakenly choose some obsoleted encoding for you.
Retrieve completed .po files, and convert them to .mo file (the file that machine can read it)
msgfmt -v fr.po -o fr.mo
Install .mo files. Run:
sudo install fr.mo /usr/share/locale/fr/LC_MESSAGES/PRJ.mo
Now you can try the result:
LANGUAGE=fr ./PRJ.sh
and you should see French translation for Hello world.
There is a long-lost, never-documented and almost-deprecated builtin solution in bash.
LANG=foo_BAR.utf8
TEXTDOMAIN="test"
TEXTDOMAINDIR="/usr/share/locale"
echo $"fooMsgid"
# bash --dump-po-strings <scriptfile>
The gettext translation make use of an editable format *.po to store translation, and a compiled format *.mo for loading.
For info of the files format, reference here: section "3 The Format of PO Files" and "10 Producing Binary MO Files" of https://www.gnu.org/software/gettext/manual/html_node/index.html
Here I focus on how to try the gettext command to get a translation in short.
After you prepared a folder /path/to/your/locale with inside-hierarchy like <lang>/LC_MESSAGES/<textdomain>.mo (where <lang> is e.g. ko_KR for Korean), use the following code in your lang_file.sh:
#!/bin/bash
export LC_ALL=ko_KR.UTF-8 # if LC_ALL not work, you could try also "LANG" and "LANGUAGE"
export TEXTDOMAINDIR=/path/to/your/locale
# export TEXTDOMAIN="<textdomain>" # <- optional, set this to save the "<textdomain>" argument for `gettext` below
LANG_HELLO_WORLD="$( gettext "<textdomain>" "Your message to translate" )"
What you are looking to do is I think ask the user in the appropriate language? You would probably want the user to pick the language first. The other part of what you are asking is simply just embedding commands like $(get_some_str_func) inside of your variable.
I did not write this code but it might be along the lines of what you are trying to do? I'm not sure, i don't understand fully what you are asking.
function _configure_locale() { # [profile]
local profile=${1:-EN}
case ${profile} in
DE|DE_DE|de_DE)
LC_ALL="de_DE.UTF-8"
LANG="de_DE.UTF-8"
LANGUAGE="de_DE:de:en_US:en"
;;
EN|EN_US|en|en_US)
LC_ALL="en_US.UTF-8"
LANG="en_US.UTF-8"
LANGUAGE="en_US:en"
;;
*)
echo "ALERT" "${FUNCNAME}: unknown profile '${profile}'"
;;
esac
LC_PAPER="de_DE.UTF-8"; # independent from locale
LESSCHARSET="utf-8"; # independent from locale
MM_CHARSET="utf-8" # independent from locale
echo "locale settings" "${LANG}";
export LC_ALL LANG LANGUAGE LC_PAPER LESSCHARSET MM_CHARSET
}
Here is my attempt to create a demo example code for Creating Internationalized Scripts section from Bash Reference Manual.
Create a script file script.sh:
#!/bin/bash
# script.sh
TEXTDOMAIN=script
TEXTDOMAINDIR="$PWD"
LANG=$1
# or use following
LC_MESSAGES=$1
echo $"hi"
Make it executable chmod +x script.sh.
Create template(.pot) and PO(.po) files:
bash --dump-po-strings script.sh > script.pot
# spanish
cp script.pot es.po
# french
cp script.pot fr.po
Edit the es.po and fr.po files and add translations.
For example es.po:
#: script.sh:8
msgid "hi"
msgstr "hola"
Compile PO files to create .mo files
msgfmt -o es.mo es.po
msgfmt -o fr.mo fr.po
# copy to following folder structure in pwd
cp es.mo es/LC_MESSAGES/script.mo
cp fr.mo fr/LC_MESSAGES/script.mo
Run the script to use the locales:
# find valid locale names
locale -a | egrep 'es|fr'
# use the valid locales to execute script
./script.sh spanish
hola
./script.sh french
salut
./script.sh es_ES
hola
./script.sh fr_FR
salut
NOTE: The example used the translation files from local directory. And installation to other common place is not required.