(OS is Windows 7 Professional. jq is version 1.5.)
I've been using jq to automate prettifying some JSON files (with Python). It seems to me after some time trying to determine why it wasn't working that jq fails silently when working with a file path string that's length 28, or simply stops working if the file path string is length 29 or more.
E.g. on cmd (and it's worth pointing out that I made a kind of shortcut so that jq calls jq-win64.exe, and tested the latter directly as well, so that's not the source of the issue):
C:\jq>jq . 123456789012345678901234567
displays prettified content of the file;
C:\jq>jq . 1234567890123456789012345678
displays nothing; and
C:\jq>jq . 12345678901234567890123456789
causes a "jq-win64.exe has stopped working" window.
(I also tested this on JSON files within folders; the common point was that the input string be of length 28 or more including slashes to fail.)
Is this a bug? If it's not, what can I do to work around it Okay, I admit that was a stupid question, I can work around it by copying content into a temp file in the base folder, prettify it, and then save it back to wherever I want it to be. More on-point question: is this the best workaround available for me to take?
There was a Windows-specific bug in jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1094). It was fixed shortly after the release of jq 1.5.
To obtain a post-1.5 .exe for Windows, see any of:
https://chocolatey.org/packages?q=jq
https://stedolan.github.io/jq/download
https://github.com/stedolan/jq/wiki/Installation#windows-using-appveyor
Related
I able to load libpcap.dylib which is confusing cause I can't figure out the actual file location. Doing find / -name libpcap.A.dylib or libpcap.dylib says no such file.
Also finder search with libpcap just results in libpcap.A.tbd and libpcap.rb.
libpcap.A.tbd shows "Install location /usr/lib/libpcap.A.dylib", but it does not actually exist there.
I wanted to locate the actual dylib file cause I running into issue with being able to import function, So I wanted to check file to make sure I have function names correct.
So I wanted to check file to make sure I have function names correct.
The first thing to check is the pcap man page - from the command line, it'd be
man pcap
It's a bit long, but it should mention all the functions available in libpcap; it may be easier than
nm /usr/lib/libpcap.dylib | egrep ' T '
(and doesn't require you to remember that the leading underscores in the output of that command are NOT part of the name of the function, they're a leftover from ancient UNIX history).
Where is libpcap.dylib?
/usr/lib/libpcap.A.dylib. /usr/lib/libpcap.dylib is a symbolic link to it.
I have several folders of video files where, due to the download manager I use, they are all named in the following format "FILENAME.mp4; filename= FILENAME.mp4" All I've been trying to do is to remove everything after (and including) ".mp4; filename". However, I haven't found a way to do this.
I have tried some free software (such as Renamer, Namechanger, Name Munger for Mac, Transnomino) but I failed to do what I need to.
I'm working on Mac OSX 10.13.6.
Any help with this issue would be appreciated.
You can achieve it using Terminal. Go to the folder where you want to rename files using this cd command, for example:
cd ~/Documents/Videos
And run this command to rename all files recursively:
find . -iname "*.mp4;*" | sed -E 's/(\.[^\.]*)(\.mp4)(.*)/mv "\1\2\3" "\1\2"/' | sh
This command will keep only FILENAME.mp4 part from FILENAME.mp4; filename= FILENAME.mp4 file name
I used to extensively use a windows Rename tool called Renamer 6.0, and it had a "pattern rename" facility called "Multi change" that could have handled this.
In the context of that tool it would be asking for a source pattern like %a= %b and a destination pattern (like %b), everything after the = would be stored in %b variable and then renaming the file to just %b would lose everything after the =
See if your preferred rename tool has a similar facility?
If your tool supports regex, then find: .*?=(.*) and replace with $1
I'm also minded that asking this question on https://unix.stackexchange.com/ might elicit some help crafting a shell script that will perform this rename (though also plenty of shell capable people here, one of them may see it - it's just that it's not quite as hardcore programmer-y a question as most).
If you're willing to learn/use java, then that could be another good way to get the problem solved. It would (at a guess) look something like this:
for (final File f : new File("C:\\temp").listFiles()) {
if (f.isFile()) {
string n = f.getName();
if (n.contains("=")) {
f.renameTo(new File(n.substring(n.indexOf("=")+1));
}
}
}
I have a script in python that does the following:
For a folder of XML files (each file lacks a docroot) :
Read the first 7 lines of source file, but do nothing with them as they need to "not be in the output"
Write a new file (in separate directory) that starts with XML tag & opening Docroot /
Parent Tag
While still reading source file at line 8, go line-by-line and append same new file
Append a closing Docroot / Parent tag to end of new file
Inspiration from John Machin Feb 1 2011
I have a similar solution using bash and sed.
The project sponsor is looking to have the script get called by AWS Lambda, and as such, is leaning towards python as the script's language.
I'm looking for a performance boost and scaling (the source files range in size from 2 MB to 241 MB, and may be larger in the future).
Is it better to stick to Pure Python solution, or use Python, but call out to the sed routines or run the bash script using the subprocess module? Thanks.
Per the advice from Jean-François Fabre , I used subprocess.call() and it worked great. Thanks again for your help
Okay so here is what I want to do. I want to add a print option that prints whatever the user's document is to a PDF and adds some headers before sending it off to a device.
I guess my questions are: how do I add a virtual "printer" driver for the user that will launch the application I've been developing that will make the PDF (or make the PDF and launch my application with references to the newly generated PDF)? How do I interface with CUPS to generate the PDF? I'm not sure I'm being clear, so let me know if more information would be helpful.
I've worked through this printing with CUPS tutorial and seem to get everything set up okay, but the file never seems to appear in the appropriate temporary location. And if anyone is looking for a user-end PDF-printer, this cups-pdf-for-mac-os-x is one that works through the installer, however I have the same issue of no file appearing in the indicated directory when I download the source and follow the instructions in the readme. If anyone can get either of these to work on a mac through the terminal, please let me know step-by-step how you did it.
The way to go is this:
Set up a print queue with any driver you like. But I recommend to use a PostScript driver/PPD. (A PostScript PPD is one which does not contain any *cupsFilter: ... line.):
Initially, use the (educational) CUPS backend named 2dir. That one can be copied from this website: KDE Printing Developer Tools Wiki. Make sure when copying that you get the line endings right (Unix-like).
Commandline to set up the initial queue:
lpadmin \
-p pdfqueue \
-v 2dir:/tmp/pdfqueue \
-E \
-P /path/to/postscript-printer.ppd
The 2dir backend now will write all output to directory /tmp/pdfqueue/ and it will use a uniq name for each job. Each result should for now be a PostScript file. (with none of the modifications you want yet).
Locate the PPD used by this queue in /etc/cups/ppd/ (its name should be pdfqueue.ppd).
Add the following line (best, near the top of the PPD):
*cupsFilter: "application/pdf 0 -" (Make sure the *cupsFilter starts at the very beginning of the line.) This line tells cupsd to auto-setup a filtering chain that produces PDF and then call the last filter named '-' before it sends the file via a backend to a printer. That '-' filter is a special one: it does nothing, it is a passthrough filter.
Re-start the CUPS scheduler:sudo launchctl unload /System/Library/LaunchDaemons/org.cups.cupsd.plist
sudo launchctl load /System/Library/LaunchDaemons/org.cups.cupsd.plist
From now on your pdfqueue will cause each job printed to it to end up as PDF in /tmp/pdfqueue/*.pdf.
Study the 2dir backend script. It's simple Bash, and reasonably well commented.
Modify the 2dir in a way that adds your desired modifications to your PDF before saving on the result in /tmp/pdfqueue/*.pdf...
Update: Looks like I forgot 2 quotes in my originally prescribed *cupsFilter: ... line above. Sorry!
I really wish I could accept two answers because I don't think I could have done this without all of #Kurt Pfeifle 's help for Mac specifics and just understanding printer drivers and locations of files. But here's what I did:
Download the source code from codepoet cups-pdf-for-mac-os-x. (For non-macs, you can look at http://www.cups-pdf.de/) The readme is greatly detailed and if you read all of the instructions carefully, it will work, however I had a little trouble getting all the pieces, so I will outline exactly what I did in the hopes of saving someone else some trouble. For this, the directory with the source code is called "cups-pdfdownloaddir".
Compile cups-pdf.c contained in the src folder as the readme specifies:
gcc -09 -s -lcups -o cups-pdf cups-pdf.c
There may be a warning: ld: warning: option -s is obsolete and being ignored, but this posed no issue for me. Copy the binary into /usr/libexec/cups/backend. You will likely have to the sudo command, which will prompt you for your password. For example:
sudo cp /cups-pdfdownloaddir/src/cups-pdf /usr/libexec/cups/backend
Also, don't forget to change the permissions on this file--it needs root permissions (700) which can be changed with the following after moving cupd-pdf into the backend directory:
sudo chmod 700 /usr/libexec/cups/backend/cups-pdf
Edit the file contained in /cups-pdfdownloaddir/extra/cups-pdf.conf. Under the "PDF Conversion Settings" header, find a line under the GhostScript that reads #GhostScript /usr/bin/gs. I did not uncomment it in case I needed it, but simply added beneath it the line Ghostscript /usr/bin/pstopdf. (There should be no pre-cursor # for any of these modifications)
Find the line under GSCall that reads #GSCall %s -q -dCompatibilityLevel=%s -dNOPAUSE -dBATCH -dSAFER -sDEVICE=pdfwrite -sOutputFile="%s" -dAutoRotatePage\
s=/PageByPage -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dPDFSETTINGS=/prepress -c .setpdfwrite \
-f %s Again without uncommenting this, under this I added the line GSCall %s %s -o %s %s
Find the line under PDFVer that reads #PDFVer 1.4 and change it to PDFVer, no spaces or following characters.
Now save and exit editing before copying this file to /etc/cups with the following command
sudo cp cups-pdfdownloaddir/extra/cups-pdf.conf /etc/cups
Be careful of editing in a text editor because newlines in UNIX and Mac environments are different and can potentially ruin scripts. You can always use a perl command to remove them, but I'm paranoid and prefer not to deal with it in the first place.
You should now be able to open a program (e.g. Word, Excel, ...) and select File >> Print and find an available printer called CUPS-PDF. Print to this printer, and you should find your pdfs in /var/spool/cups-pdf/yourusername/ by default.
*Also, I figured this might be helpful because it helped me: if something gets screwed up in following these directions and you need to start over/get rid of it, in order to remove the driver you need to (1) remove the cups-pdf backend from /usr/libexec/cups/backend (2) remove the cups-pdf.conf from /etc/cups/ (3) Go into System Preferences >> Print & Fax and delete the CUPS-PDF printer.
This is how I successfully set up a pdf backend/filter for myself, however there are more details, and other information on customization contained in the readme file. Hope this helps someone else!
I'm trying to load hunchentoot via quicklisp in slime, and getting the following error:
READ error during COMPILE-FILE:
:ASCII stream decoding error on
#<SB-SYS:FD-STREAM
for "file [redacted]/dists/quicklisp/software/rfc2388-20120107-http/rfc2388.asd"
{100607B723}>:
the octet sequence #(196) cannot be decoded.
(in form starting at line: 29, column: 29,
file-position: 1615)
[Condition of type ASDF:LOAD-SYSTEM-DEFINITION-ERROR]
I get this when trying to run either:
(ql:quickload "hunchentoot")
Or simply:
(ql:quickload "rfc2388")
It seems that others are getting this too. I found one hint at a possible answer, saying:
The system file is encoded as UTF-8.
I'm not sure how to configure things so that SBCL on Windows starts with
UTF-8 as its default encoding for loading sources, but that's what you
need to do.
From there, I've tried (based on e.g. [this] adding the following to my emacs config:
(set-language-environment "UTF-8")
(setq slime-lisp-implementations
'((sbcl ("/opt/local/bin/sbcl") :coding-system utf-8-unix)))
(setq slime-net-coding-system 'utf-8-unix)
But... I still get the same error, even after completely re-starting emacs, to make sure I had a fresh Slime that was reading the above config.
So, what am I missing, and/or otherwise how can I get this to load?
Thanks in advance! (More thanks to come for a successful answer. ;)
Have you checked your locale settings? Emacs configuration only tells it what coding systems to set for communication between SLIME and SWANK.
You can check for locale settings with /usr/bin/locale, for example:
navi ~ » locale
LANG=pl_PL.UTF-8
LC_CTYPE=pl_PL.UTF-8
LC_NUMERIC=pl_PL.UTF-8
LC_TIME=pl_PL.UTF-8
LC_COLLATE="pl_PL.UTF-8"
LC_MONETARY=pl_PL.UTF-8
LC_MESSAGES=C
LC_PAPER=pl_PL.UTF-8
LC_NAME="pl_PL.UTF-8"
LC_ADDRESS="pl_PL.UTF-8"
LC_TELEPHONE="pl_PL.UTF-8"
LC_MEASUREMENT=pl_PL.UTF-8
LC_IDENTIFICATION=pl_PL.UTF-8
LC_ALL=
navi ~ »
Mine is setup for UTF-8 everywhere, as you can see, except for displaying 'C' messages.
Try this:
change into the .../quicklisp/dists/quicklisp/software/rfc2388* directory and load rfc2388.asd into a text editor.
Move down to the :author parameter of the defsystem form. Replace the author's name by the name given at the top of the file.
Store file using ASCII encoding.
Of course, when a new version of the library is published, the workaround gets lost. Or else store the modified project in local-projects.
With the original UTF-8 encoding still in effect, the DEBUGGER should present an INPUT-REPLACEMENT option to replace offending input characters by a replacement string. Choose that option, type "?" or "x" or any string you like at the prompt and then ENTER. The load then completes. Of course, that is not something you would like to do every time.
So the best idea is probably to send an email to the author and ask to provide an ascii version for quicklisp.
There should be a .cache directory in your HOME that contains all the fasl files. Sometimes removing those old fasl files seems to work for me when something goes wrong with compilation.