USA and Canada with jvectormap - converters

I'm looking for a map of (US + Canada) together with states/provinces respectively.
This's what I've done so far:
Downloaded jVectorMap 1.2.2 from here;
After reading this, installed GDAL and Shapely;
Downloaded 10m Admin 1 package from Natural Earth;
Than, according to this thread, it is possible to do what I need using following:
python converter.py --width 900 --country_name_index 12 --country_code_index 18 --longitude0 -100 --where="iso_a2 = 'CA' OR iso_a2 = 'US'" --projection lcc --name us_ca ne_10m_admin_1_states_provinces_shp/ne_10m_admin_1_states_provinces_shp.shp ../jquery-jvectormap-us-ca-lcc-en.js
where --country_name_index 12 --country_code_index 18 part doesn't make any sense to me, since I'm trying to convert 2 countries.
Anyways, after running suggested code I get:
Traceback (most recent call last):
File "converter.py", line 296, in <module>
converter.convert(args['output_file'])
File "converter.py", line 144, in convert
self.loadData()
File "converter.py", line 89, in loadData
self.loadDataSource( sourceConfig )
File "converter.py", line 130, in loadDataSource
shapelyGeometry = shapely.wkb.loads( geometry.ExportToWkb() )
AttributeError: 'module' object has no attribute 'wkb'
I find this really odd, unless I missed something in installation.
After adding import shapely.wkb to converter.py I get Alaska with name State and Yukon as Territory, and that's it.
What am I missing here?
Thanks for your time.

I had the same problem as you. Solved it by using the shapefile, 10m_cultural/ne_10m_admin_1_states_provinces_shp.shp from the package naturalearth all vector themes.
But only downside is that the output JS file is too big. It comes up to 2MB easily. I'll try using shapefile from different source next time and let you know. But for now at least this works.

I had the same problem when building continent maps. The fix was to use an older convert.py version (1.1.1 rather than 1.2.2). You still need to input --country_name_index and --country_code_index flag so give whatever you want as values. The map produced is fine.
convert.py 1.1.1 can be found here :
https://github.com/jfhovinne/jvectormap-maps-builder

Related

Faster way of Appending/combining thousands (42000) of netCDF files in NCO

I seem to be having trouble properly combining thousands of netCDF files (42000+) (3gb in size, for this particular folder/variable). The main variable that i want to combine has a structure of (6, 127, 118) i.e (time,lat,lon)
Im appending each file 1 by 1 since the number of files is too long.
I have tried:
for i in input_source/**/**/*.nc; do ncrcat -A -h append_output.nc $i append_output.nc ; done
but this method seems to be really slow (order of kb/s and seems to be getting slower as more files are appended) and is also giving a warning:
ncrcat: WARNING Intra-file non-monotonicity. Record coordinate "forecast_period" does not monotonically increase between (input file file1.nc record indices: 17, 18) (output file file1.nc record indices 17, 18) record coordinate values 6.000000, 1.000000
that basically just increases the variable "forecast_period" 1-6 n-times. n = 42000files. i.e. [1,2,3,4,5,6,1,2,3,4,5,6......n]
And despite this warning i can still open the file and ncrcat does what its supposed to, it is just slow, at-least for this particular method
I have also tried adding in the option:
--no_tmp_fl
but this gives an eror:
ERROR: nco__open() unable to open file "append_output.nc"
full error attached below
If it helps, im using wsl and ubuntu in windows 10.
Im new to bash and any comments would be much appreciated.
Either of these commands should work:
ncrcat --no_tmp_fl -h *.nc
or
ls input_source/**/**/*.nc | ncrcat --no_tmp_fl -h append_output.nc
Your original command is slow because you open and close the output files N times. These commands open it once, fill-it up, then close it.
I would use CDO for this task. Given the huge number of files it is recommended to first sort them on time (assuming you want to merge them along the time axis). After that, you can use
cdo cat *.nc outfile

How to pretty print a matrix in Octave?

I want to create a pretty printed table from a matrix (or column vector).
For Matlab there are several available functions that can do this (such as printmat, array2table, and table), but for Octave I cannot find any.
So instead of:
>> a = rand(3,2)*10;
>> round(a)
ans =
2 10
1 3
2 1
I would like to see:
>> a = rand(3,2)*10;
>> pretty_print(round(a))
THIS THAT
R1 2 10
R2 1 3
R3 2 1
How can I produce a pretty printed table from a matrix?
(Any available package to do so?)
UPDATE
After trying to follow the extremely obtuse package installation instruction from Octave Wiki, I kept getting the error pkg: failed to read package 'econometrics-1.1.1.tar.gz': Couldn't resolve host name. Apparently the windows version isn't able to use the direct installation command (as given on their Wiki). The only way I managed to get it, was by first downloading the package manually into the current working directory of Octave. (See pwd output.) Only then did the install command work.
pkg install econometrics-1.1.1.tar.gz
pkg load econometrics
Yes, there is a prettyprint function in the econometrics package. Once the package is installed and loaded, you can use it like this:
>> a = rand(3,2)*10;
>> prettyprint(round(a),['R1';'R2';'R3'],['THIS';'THAT'])
THIS THAT
R1 2.000 3.000
R2 3.000 4.000
R3 10.000 3.000

cannot edit ghostprint ppd in Windows 10

I had some difficulty posing my problem in a way that the Title filter found pleasing. The real problem is that modifying only the GhostPDF.PDD file in the GS9.26 installation in Windows 10 doesn't seem to affect the output after a re-installation using Windows 10 Device Installer.
I print to a networked Sun SPARCprinter 1 which is controlled by Ghostprint (script?) compiled to run on SunOS 4.1.4. This has worked successfully for some years printing output from Windows XP using Adobe's PS driver and a SPARCstation PPD cobbled together from samples found on the net.
I've installed Artifex's 9.26 on Windows 10 and output to an LPR printer (The Sun). The output works, is recognized as PS output by the Sun, but produces a number of FATAL errors.
I need to edit the Windows Ghostscript installation to output PS files which are more suitable for the Sun.
So to my simple question: Do I need to modify anything in the Ghostscript Windows 10 installation other than the Ghostpdf.PPD file?
additional info:
SPARCstation 10 information:
SunOS 4.1.4
arcad# gcc -dumpversion
2.95.2 Note: I had to bootstrap this version up from the early GCC which could be compiled with the SunOS 4.1.4 C compiler. I had the impression I couldn't bring it up any further but could be mistaken.
arcad# gs --help
Aladdin Ghostscript 6.01 (2000-03-17)
Copyright (C) 2000 Aladdin Enterprises ...
Usage: gs [switches] [file1.ps file2.ps ...]
Most frequently used switches: (you can use # in place of =)
-dNOPAUSE no pause after page | -q `quiet', fewer messages
-g<width>x<height> page size in pixels | -r<res> pixels/inch resolution
-sDEVICE=<devname> select device | -dBATCH exit after last file
-sOutputFile=<file> select output file: - for stdout, |command for pipe,
embed %d or %ld for page #
Input formats: PostScript PostScriptLevel1 PostScriptLevel2 PDF
.....
For more information, see /usr/local/share/ghostscript/6.01/doc/Use.htm.
Note: I think this is the most recent GS version I can compile with this gcc version
printcap section:
gp|GhostPrinter:\
:lp=/dev/lpvi0:sd=/var/spool/gsprintspool:lf=/var/spool/gsprintspool/log:\
:mx#0:sh:if=/usr/local/libexec/lpfilter-gps:
Typical spool file - "....." indicates stuff not included here"
arcad# more dfA004DESKTOP-M8C5I86
%!PS-Adobe-3.0
%%Title: Document
%%Creator: PScript5.dll Version 5.2.2
%%CreationDate: 12/14/2018 19:56:8
%%For: jferg
%%BoundingBox: (atend)
%%Pages: (atend)
%%Orientation: Portrait
%%PageOrder: Special
%%DocumentNeededResources: (atend)
%%DocumentSuppliedResources: (atend)
%%DocumentData: Clean7Bit
%%TargetDevice: (Ghostscript) (3010) 815
%%LanguageLevel: 3
%%EndComments
%%BeginDefaults
%%PageBoundingBox: 0 0 612 792
%%ViewingOrientation: 1 0 0 1
%%EndDefaults
.....
%%EndResource
userdict /Pscript_WinNT_Incr 230 dict dup begin put
%%BeginResource: file Pscript_FatalError 5.0 0
userdict begin/FatalErrorIf{{initgraphics findfont 1 index 0 eq{exch pop}{dup
length dict begin{1 index/FID ne{def}{pop pop}ifelse}forall/Encoding
{ISOLatin1Encoding}stopped{StandardEncoding}if def currentdict end
/ErrFont-Latin1 exch definefont}ifelse exch scalefont setfont counttomark 3 div
cvi{moveto show}repeat showpage quit}{cleartomark}ifelse}bind def end
%%EndResource
userdict begin/PrtVMMsg{vmstatus exch sub exch pop gt{[
quires more memory than is available in this printer.)100 500
more of the following, and then print again:)100 485
put format, choose Optimize For Portability.)115 470
ce Settings page, make sure the Available PostScript Memory is accur--More--(2%)
ce the number of fonts in the document.)115 440
ocument in parts.)115 425 12/Times-Roman showpage
Error: Low Printer VM ]%%)= true FatalErrorIf}if}bind def end
2016 ge{/VM?{pop}bind def}{/VM? userdict/PrtVMMsg get def}ifelse
.....
SPARCprinter PDD file which works with Adobe PS in Windows XP:
john#hp2:~/sun-stuff/cups-sparc$ more SPARCprinter2.ppd
*PPD-Adobe: "4.1"
*% PostScript(R) Printer Description File for SPARCprinter
*% Date: 94/01/14
*% Copyright 1994 Sun Microsystems, Inc. All Rights Reserved.
*% Permission is granted for redistribution of this file as
*% long as this copyright notice is intact and the contents
*% of the file is not altered in any way from its original form.
*% End of Copyright statement
*% Changed margins on SPARCprinter JAF 3-3-2017
*FormatVersion: "4.1"
*FileVersion: "1.10"
*LanguageEncoding: ISOLatin1
*LanguageVersion: English
*PCFileName: "SPRN.PPD"
*Product: "(SPARCprinter)"
*PSVersion: "(3.000) 0"
*ModelName: "SPARCprinter"
*ShortNickName: "SPARCprinter"
*NickName: "SPARCprinter"
*% ==== Device Capabilities ===============
*LanguageLevel: "3"
*Extensions: CMYK Composite
*FreeVM: "4194304"
*ColorDevice: False
*DefaultColorSpace: Gray
*VariablePaperSize: False
*TTRasterizer: None
*FileSystem: False
..... more of the usual stuff
I don't really understand why you have installed Ghostscript on Windows. Windows is perfectly capable of producing PostScript files all of its own. In addition, the PPD file doesn't actually do very much, it is simply a text file with descriptions of the capabilities of the printer.
So the real problem is, or seems to be, that your SUN setup doesn't like the PostScript being produced by the new version of Windows.
You don't say how you are printing the PostScript file. not how your printer is 'controlled by Ghostscript' (I'm not aware of any product called Ghostprint, there is a GSPrint as part of GSView, but that's really for Windows).
Assuming you are using Ghostscript on your Sparc workstation to drive the pritner, then the most likely problem I would say is that you are using an old version of Ghostscript on the workstation, and it doesn't like the PostScript being generated by the newer version of Windows.
If you had included the transcript from the workstation Ghostscript installation it might be possible to say more but without that I'm rather guessing.
Another possibility is that you are using the ps2write device in Ghostscript to produce PostScript files on Windows. I can't think why you would be doing that, but it sort of fits your description. In that case editing the PPD file will have no effect, because Ghostscript doesn't use it.
Now the ps2write device emits level 2 PostScript, the clue is in the name, and its possible again that your Sparc setup is so elderly that it doesn't understand level 2, or doesn't fully implement it. In which case you will probably get errors. Again, if you were to provide the text of the error messages this would help!
In the latter case, you are frankly out of luck. We dropped support for level 1 PostScript output some time ago, what with level 2 being 28 years old now and level 3 coming up on 20. If you need language level 1 output you will have to go back to a very old version of Ghostscript. Something like 9.07 (from 5 and a half years ago) was the last version that included the pswrite device.
With effort you could take the pswrite device and upgrade it so that it works with the current version of Ghostscript
[EDIT]
My word, that's a really old version of Ghostscript!
You could try building a new version to replace it, but I also don't know if current code will compile on gcc 2.95. It 'should' because we only expect C89, but the third party libraries (which are essential) may very well not compile.
The PostScript file you quoted has been produced by Windows, not by Ghostscript (%%Creator: PScript5.dll Version 5.2.2). So it seems likely to me that your problem is the PostScript being produced by the newer version of Windows doesn't work with your 18 year old version of Ghostscript. That's not actually entirely surprising.
If you look at the DSC comments it says:
%%LanguageLevel: 3
And your Ghostscript information says that it supports language levels 1 and 2. At the time the level 3 spec had only just been published (1999), and clearly the maintainers back then hadn't had time to fully implement it.
Note that the ghostpdf.ppd file is intended for use with Ghostscript as a 'print to PDF' printer along with the RedMon port monitor.
Now its not obvious to me which PPD file you are using, but..... Both the ghostpdf.ppd file and the sparcprinter ppd file have :
*LanguageLevel: "3"
That tells the PostScript driver that it can use language level 3, which your Sparc Ghostscript doesn't support. You could try changing that to:
*LanguageLevel: "2"
and see if that makes a difference (you will have to uninstall the printers from Windows and re-install them with the modified PPD file).
If it doesn't work, the only other thing I can think of is to use the Ghostscript you installed on the Windows system, and preprocess the PostScript file produced by Windows before you send it on. You can use the ps2write device in Ghostscript 9.26 to take in the level 3 file, and produce a level 2 file. It might be a bit bigger, but it ought to work.
To do that on Windows you would use something like:
gswin64c -sDEVICE=ps2write -sOutputFile=out.ps <input.ps>
The file 'out.ps' should then be a level 2 PostScript file. I can't guarantee that the output will then work the old version of Ghostscript on your Sparc, but you stand a chance!

using ipdb to debug python code in one cell (jupyter or Ipython)

I'm using jupyter (or Ipython) notebook with firefox, and want to debug some python code in the cell. I am using 'import ipdb; ipdb.set_trace()' as kind of breakpoint, for example my cell has the following code:
a=4
import ipdb; ipdb.set_trace()
b=5
print a
print b
which after execution with Shift+Enter gives me this error:
--------------------------------------------------------------------------
MultipleInstanceError Traceback (most recent call last)
<ipython-input-1-f2b356251c56> in <module>()
1 a=4
----> 2 import ipdb; ipdb.set_trace()
3 b=5
4 print a
5 print b
/home/nnn/anaconda/lib/python2.7/site-packages/ipdb/__init__.py in <module>()
14 # You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
15
---> 16 from ipdb.__main__ import set_trace, post_mortem, pm, run, runcall, runeval, launch_ipdb_on_exception
17
18 pm # please pyflakes
/home/nnn/anaconda/lib/python2.7/site-packages/ipdb/__main__.py in <module>()
71 # the instance method will create a new one without loading the config.
72 # i.e: if we are in an embed instance we do not want to load the config.
---> 73 ipapp = TerminalIPythonApp.instance()
74 shell = get_ipython()
75 def_colors = shell.colors
/home/nnn/anaconda/lib/python2.7/site-packages/traitlets/config/configurable.pyc in instance(cls, *args, **kwargs)
413 raise MultipleInstanceError(
414 'Multiple incompatible subclass instances of '
--> 415 '%s are being created.' % cls.__name__
416 )
417
MultipleInstanceError: Multiple incompatible subclass instances of TerminalIPythonApp are being created.
The same error appears if I use this code not in the jupyter notebook in the browser, but in jupyter qtconsole.
What does this error mean and what to do to avoid it?
Is it possible to debug code in the cell step-by-step, using next, continue, etc commands of pdb debugger?
Had this problem also and it seems to be related to versions of jupyter and ipdb.
Solution is to use this instead of the ipdb library set_trace call:
from IPython.core.debugger import Tracer
Tracer()() #this one triggers the debugger
Source: http://devmartin.com/blog/2014/10/trigger-ipdb-within-ipython-notebook/
Annotated screenshot:
Tracer() is deprecated.
Use:
from IPython.core.debugger import set_trace
and then place set_trace() where breakpoint is needed.
from IPython.core.debugger import set_trace
def add_to_life_universe_everything(x):
answer = 42
set_trace()
answer += x
return answer
add_to_life_universe_everything(12)
This works fine and brings us a little bit more comfort (e.g. syntax highlighting) than just using the built-in pdb.
source
If using Jupyter Notebook
begin your cell with magic command "%%debug".
Then a ipdb line will be shown at the bottom of the cell which will help you navigate through the debugging session. Following commands should get you started:
n- execute current line and go to next line.
c- continue execution until next break point.
Make sure you restart the kernel each time you decide on debugging, so that all variables are freshly assigned.You can check the value of each variable through the ipdb line and you will see that the variable is undefined until you execute the line that assigns a value to that variable.
%%debug
import pdb
from pdb import set_trace as bp
def function_xyz():
print('before breakpoint')
bp() # This is a breakpoint.
print('after breakpoint')
My version of Jupyter is 5.0.0 and my corresponding ipython version is 6.1.0. I am using
import IPython.core.debugger
dbg = IPython.core.debugger.Pdb()
dbg.set_trace()
Tracer is listed as deprecated.
Update:
I tried using the method from another answer https://stackoverflow.com/a/43086430/8019692 below but got an error:
MultipleInstanceError: Multiple incompatible subclass instances of TerminalIPythonApp are being created.
I prefer my method to the %%debug magic since I can set breakpoints in functions defined in other cells and run the function in another cell. Jupyter/IPython drops into the debugger in my function where the breakpoint is set, and I can use the usual pdb commands. To each his own...
#lugger1, the accepted answer is deprecated.

Computing Sky View Factor in GrassGis

Hy community,
I´m currently working on my Master Thesis and I have to compute the "sky view factor". Since ESRI Arcmap is not a helpful choice to do that, I found that it is fairly easy to compute with GrassGIS (V.7) using the r.skyview command.
But I get an error message in the logfile i can´t really deal with. Hope that someone of you is experienced with that kind of problem and can help me out with this.
Here is what the GrassGIS output says:
*(Fri Jan 09 16:17:10 2015)
r.skyview input=Subset#PERMANENT output=Subset_SVF ndir=16 maxdistance=15.0
Unknown module parameter "keyword" at line 21
Unknown module parameter "keyword" at line 22
FEHLER: Value <rast> ambiguous for parameter <type>
Valid options: raster,raster_3d,vector,old_vector,ascii_vector,labels,region,group,all
Traceback (most recent call last):
File "C:\Users\Axel-HP\AppData\Roaming\GRASS7\addons/scripts/r.skyview.py", line 120, in <module>
sys.exit(main())
File "C:\Users\Axel-HP\AppData\Roaming\GRASS7\addons/scripts/r.skyview.py", line82, in main
old_maps = _get_horizon_maps()
File "C:\Users\Axel-HP\AppData\Roaming\GRASS7\addons/scripts/r.skyview.py", line 114, in_get_horizon_maps
pattern=TMP_NAME + "*")[gcore.gisenv()['MAPSET']]
File "C:\Temp\GRASSGIS7\etc\python\grass\script\core.py", line 1176, in list_grouped
type=types, pattern=pattern,
exclude=exclude).splitlines():
File "C:\Temp\GRASSGIS7\etc\python\grass\script\core.py", line 425, in read_command
return handle_errors(returncode, stdout, args, kwargs)
File "C:\Temp\GRASSGIS7\etc\python\grass\script\core.py", line 308, in handle_errors
returncode=returncode)
grass.exceptions.CalledModuleError: Module run None
['g.list', '--q', '-m', 'type=rast', 'pattern=tmp_horizon_2340*'] ended with error
Process ended with non-zero return code 1. See errors in the (error) output.
(Fri Jan 09 16:17:11 2015) Befehl ausgeführt (1 Sek)*
I just tested r.skyview and it is working. There were big changes in GRASS module parameter names recently which caused the trouble, but now it should work without problems.

Resources