Good afternoon.
I am running pcl6.exe version 9.15 on Windows 8.1.
I am running into a problem where pcl6.exe in silently converting any APOSTROPHE characters into RIGHT SINGLE QUOTATION MARK characters using the 16602 typeface in a PCL5 file.
Here is the command line I am using:
pcl6.exe -dNOPAUSE -sDEVICE=txtwrite -sOutputFile=test.txt test.prn
test.prn input (hex)
1B 28 30 55 1B 28 73 31 70 31 30 76 31 36 36 30 32 54 1B 26 61 30 76 30 48 3E 27 3C
test.prn input (text ['.' is the escape character])
.(0U.(s1p10v16602T.&a0v0H>'<
test.txt output (hex)
20 20 3E E2 80 99 3C 0D 0A
test.txt output (text)
>’<..
expected test.txt output (hex)
20 20 3E 27 3C 0D 0A
expected test.txt output (text)
>'<..
Is there a flag or an option somewhere that can disable this conversion?
Thank you for your time.
txtwrite does the best it can with the input, PCL tends not to have much information in the PCL file to allow us to determine what the glyph should be (PostScript is better, and PDF often still better).
If you think there is a real problem I would suggest your best bet is to open a bug report. Apart from anything else, I would need to see the PCL file to determine what's going on. Most lkely the character code you are using corresponds to an apostrophe, which in the specific font is a right quote. There is no way for the text extraction device to know what shape the font will draw in response to a character code. At least, not in PCL
The problem was caused by the symbolset.
The sample was using the PCL ISO 6: ASCII symbolset (code 0U )
http://www.pcl.to/symbolset/pcl_0u.pdf
According to the 0U symbolset reference, APOSTROPHE (0x27) is replaced with RIGHT SINGLE QUOTATION MARK (0x2019). Pcl6.exe then converts these UTF-16 bytes into their UTF-8 equivalent: 0xE28099
Related
I have been working on some device that allowed login via telnet and I extracted some data from devices and made some reports, without any problems. recently, I had to switch to SSH while rest of the script is all the same, only login procedure has been changed from telnet to SSH. after switching to SSH, I am facing some problem with the data extracted that there are some invalid characters in some of the lines, below is an example: as can be seen, there is an invalid character after PON7 in the line:
OLT:LT6.PON7.ONT1,ALARM,Date time,
problem is that this invalid character is not even visible in the bash/csv file, but it was discovered when I copied the line in notepad++ or while posting it here.
now I have two problems:
1st: if someone knows what is causing these invalid characters while switching between telnet/ssh.
2nd: how to deal with this invalid character in BASH as it is not even visible in BASH, but this report is being used somewhere and these invalid characters are causing problems.
Edit:
Pasting the text into a text-to-hex converter produces this:
4f 4c 54 3a 4c 54 36 2e 50 4f 4e 37 11 2e 4f 4e 54 31 2c 41 4c 41 52 4d 2c 44 61 74 65 20 74 69 6d 65 2c
It looks like there's a DC1 character (hex 11) between the "7" and the ".".
Unfortunately, this edit also has the side effect of removing the character from the sample text.
Passing your text through a text to hexadecimal converter shows that the invisible character is an ASCII DC1 character (hex 11, octal 021). This character is also known as Ctrl-Q or XON. It's sometimes used in flow control.
In a bash script, you could filter it out using the tr program:
echo $badtext | tr -d '\021'
SSH doesn't inherently insert DC1 characters into text streams. If you're getting a DC1 character in the output from a device, presumably the device sent that character.
Running this command:
echo "foo" > test.txt
I get strange results in test.txt. I actually can't even copy-paste the contents directly here in stack overflow but need to show the hex output of the file. Which looks like this-
ff fe 66 00 6f 00 6f 00 0d 00 0a 00
Not sure what FF and FE are but it looks like it's also placing NULL between each character.
Can any Windows people guide me in the right direction as to why this is happening and how I can resolve it? I just want to the contents "foo" to be placed in that file unmolested.
0xFFFE is the Unicode byte order mark for UTF-16 LE (little endian), which is normal Windows Unicode.
The contents of your file are the two byte BOM, 0x6600 ("f"), 0x6f00 ("o"), 0x6f00 ("o"), 0x0d00 ("`r" or carriage return), 0x0a00 ("`n" or new line)
Try:
echo "foo" | Out-File -FilePath test.txt -Encoding utf8
I am not an experienced ruby programmer, so bear with me. I have a problem with this specific text file containing two lines ( this issue shows up only on occasions) :
trim(0, 15447)
0, 15447
I am trying to read these two lines with the following code:
File.open(trim).each do |line|
puts line
end
I normally obtain the normal output, but here, I get only one line, with some characters missing:
0, 1544715447)
If I want to check the character codes, I get this:
irb(main):120:0> File.open(trim).each do |line|
irb(main):121:1* puts '========================'
irb(main):122:1> puts line
irb(main):123:1> puts '........................'
irb(main):124:1> puts line.each_byte {|c| print c, ' ' }
irb(main):125:1> end
========================
0, 1544715447)
........................
116 114 105 109 40 48 44 32 49 53 52 52 55 41 13 48 44 32 49 53 52 52 55 trim(0,0, 15447
=> #<File:E:\Public\Public_videos\Soccer\1995_0129_odp_es\950129-ODP_&m3_trim30.txt>
I frankly don't understand what is going on, as I don't see any hidden character, and this happen randomly, but consistently with some files.
Any suggestion to help me understand or avoid this issue would be greatly appreciated.
What happened is that your file had two "lines" separated by a carraige return character, and not a linefeed.
You showed the bytes in your file as
116 114 105 109 40 48 44 32 49 53 52 52 55 41 13 48 44 32 49 53 52 52 55
That 13 is a carriage return, which is sometimes "displayed" by the writer going back to the start of the line it is writing.
So first it wrote out
trim(0, 15447)
then it went back to the start of the same line and wrote
0, 15447
overlaying the initial line! What do you end up with?
0, 1544715447)
Your "problem" is probably best fixed by reencoding that text file of yours to use a better way to separate lines. On Unix systems, including OSX these days, the line terminator is character 10 - known as LINE FEED. Windows uses the two-character combination 13 10 (CR LF). Only old Mac systems to my knowledge used the 13.
Many text editors today will allow you to select a "line ending" option, so you might be able to just open that file, then save it using a different line ending option. FWIW my guess is that you are using Windows now, which is known for rendering CRs and LFs differently than *Nix systems.
I am writing a script that takes a UTF-16 encoded text file as input and outputs a UTF-16 encoded text file.
use open "encoding(UTF-16)";
open INPUT, "< input.txt"
or die "cannot open > input.txt: $!\n";
open(OUTPUT,"> output.txt");
while(<INPUT>) {
print OUTPUT "$_\n"
}
Let's just say that my program writes everything from input.txt into output.txt.
This WORKS perfectly fine in my cygwin environment, which is using "This is perl 5, version 14, subversion 2 (v5.14.2) built for cygwin-thread-multi-64int"
But in my Windows environment, which is using "This is perl 5, version 12, subversion 3 (v5.12.3) built for MSWin32-x64-multi-thread",
Every line in output.txt is pre-pended with crazy symbols except the first line.
For example:
<FIRST LINE OF TEXT>
㈀ Ⰰ ㈀Ⰰ 嘀愀 ㌀ 䌀栀椀愀 䐀⸀⸀⸀ 儀甀愀渀最 䠀ഊ<SECOND LINE OF TEXT>
...
Can anyone give some insight on why it works on cygwin but not windows?
EDIT: After printing the encoded layers as suggested.
In Windows environment:
unix
crlf
encoding(UTF-16)
utf8
unix
crlf
encoding(UTF-16)
utf8
In Cygwin environment:
unix
perlio
encoding(UTF-16)
utf8
unix
perlio
encoding(UTF-16)
utf8
The only difference is between the perlio and crlf layer.
[ I was going to wait and give a thorough answer, but it's probably better if I give you a quick answer than nothing. ]
The problem is that crlf and the encoding layers are in the wrong order. Not your fault.
For example, say you do print "a\nb\nc\n"; using UTF-16le (since it's simpler and it's probably what you actually want). You'd end up with
61 00 0D 0A 00 62 00 0D 0A 00 63 00 0D 0A 00
instead of
61 00 0D 00 0A 00 62 00 0D 00 0A 00 63 00 0D 00 0A 00
I don't think you can get the right results with the open pragma or with binmode, but it can be done using open.
open(my $fh, '<:raw:encoding(UTF-16):crlf', $qfn)
You'll need to append a :utf8 with some older version, IIRC.
It works on cygwin because the crlf layer is only added on Windows. There you'd get
61 00 0A 00 62 00 0A 00 63 00 0A 00
You have a typo in your encoding. It should be use open ":encoding(UTF-16)" Note the colon. I don't know why it would work on Cygwin but not Windows, but could also be a 5.12 vs 5.14 thing. Perl seems to make up for it, but it could be what's causing your problem.
If that doesn't do it, check if the encoding is being applied to your filehandles.
print map { "$_\n" } PerlIO::get_layers(*INPUT);
print map { "$_\n" } PerlIO::get_layers(*OUTPUT);
Use lexical filehandles (ie. open my $fh, "<", $file). Glob filehandles are global and thus something else in your program might be interfering with them.
If all that checks out, if lexical filehandles are getting the encoding(UTF-16) applied, let us know and we can try something else.
UPDATE: This may provide your answer: "BOMed UTF files are not suitable for streaming models, and they must be slurped as binary files instead." Looks like you have to read the file in as binary and do the encoding as a string. This may have been a bug fixed in 5.14.
UPDATE 2: Yep, I can confirm this is a bug that was fixed in 5.14.
The Problem
I'm having a problem with a Windows batch file and labels.
I keep getting this error:
The system cannot find the batch label
specified
What I've tried
Two computers; a WindowsXP and a 2003 Server.
Made sure it was encoded as ASCII
Editted the hex code for the line continuation characters. Tried replacing all with CR , LF, and CRLF in turn. All combinations give me the same error.
Tried inserting extra characters before the label to make the label past 512 characters.
Here is the code:
cls
#echo off
SET zip=7za a dependencies.7z
call:dozip "c:\temp\dir.txt"
pause
goto exit
:dozip
echo Testing 1.2.3...
%zip% %1
goto:eof
:exit
Here's the hex with CRLF (0d 0a).
63 6c 73 0d 0a 53 45 54 20 7a 69 70 3d 37 7a 61 20 61 20 64 65 70 65 6e 64 65 6e 63 69 65 73 2e 37 7a 0d 0a 63 61 6c 6c 3a 64 6f 7a 69 70 20 22 63 3a 5c 74 65 6d 70 5c 64 69 72 2e 74 78 74 22 0d 0a 0d 0a 70 61 75 73 65 0d 0a 67 6f 74 6f 20 65 78 69 74 0d 0a 0d 0a 3a 64 6f 7a 69 70 0d 0a 20 20 65 63 68 6f 20 54 65 73 74 69 6e 67 20 31 2e 32 2e 33 2e 2e 2e 0d 0a 20 20 25 7a 69 70 25 20 25 31 0d 0a 67 6f 74 6f 3a 65 6f 66 0d 0a 3a 65 78 69 74
Here's the console's output (when I remove #echo off):
C:\>SET zip=7za a dependencies.7z
C:\>call:dozip "c:\temp\dir.txt"
C:\>echo Testing 1.2.3...
Testing 1.2.3...
C:\>7za a dependencies.7z "c:\temp\dir.txt"
The system cannot find the batch label specified - dozip
C:\>pause
Press any key to continue . . .
It never actually creates the 7zip file, so I think I can assume that its crashing on this line;
7za a dependencies.7z "c:\temp\dir.txt"
If I run that line by itself from a command prompt, it works fine and creates the dependencies.7z, so I don't think its necessarily a problem with 7za.exe.
I've already read this stackoverflow question:
stackoverflow.com/questions/232651/why-the-system-cannot-find-the-batch-label-specified-is-thrown-even-if-label-ex
and the link from that post;
help.wugnet.com/windows/system-find-batch-label-ftopict615555.html
The Answer
So, I found the problem guys.
I was using a technique I commonly use that I could only really describe as "proxy" batch files. I have a folder called c:\scripts, and I put several bat files in there to target commonly used exes. This saves my PATH variable from becoming absolutely massive with all of my command line tools. This way I only need to add c:\scripts to my PATH, and create a proxy batch file when I need something.
I had 7za.bat in c:\scripts, containing only this;
#echo off
"C:\Program Files\7-zip\7za.exe" %*
I changed my script to this;
SET zip="c:\program files\7-zip\7za.exe" a dependencies.7z
instead of this;
SET zip=7za a dependencies.7z
and it worked flawlessly.
The moral of the story...
Avoid calling other batch files from within a batch file. If you do, you will need to prefix them with "call".
I would point out that the "Testing 1.2.3..." and "Press any key to continue . . ." lines indicate that execution has successfully gone to the :dozip label and then successfully returned to the caller.
Is the "7za" executable actually a batch file? If I modify my test script to have the helper be a batch file, I get the same error. The fix is to do 'call %zip% %1'
the moral of the story: when calling external programs/batch files in a batch file, use call
call foo.bat
and/or
call %foo%
(Calling one batch from another has been done since the days of DOS, just remember to call)
The "The system cannot find the batch label specified" error message brought me here (10 years later!) and the problem turned out to be the CRLF was incorrect.
The cause, in our case, was a Git Repository that didn't recognize BAT files as being text files which needed to be CRLF on a Windows 7 machine.
Our solution was to create a .gitattributes file containing the line:
*.bat text eol=crlf
Then deleting the BAT files and checking out from the repository re-wrote our BAT files with the correct line endings. The BAT labels are now working again.
One possibility, although it seems unlikely, is that command extensions aren't enabled, or up-to-date, and this is interfering with call/goto/label behaviour.
Try:
echo [%cmdextversion%]
and if it's less than [2] (or empty -- []) then check to see if cmd.exe is being invoked with /e:off, or just run
cmd /e:on
in the console window where you will run this batch file.
In my case the reasons of "The system cannot find the batch label specified" were:
No CALL before jar- and unzip-commands
No ENDLOCAL for every SETLOCAL DisableDelayedExpansion
Some other batch-magic, which I didn't completely understood near SETLOCAL DisableDelayedExpansion, it was important to place SET param=1 after SETLOCAL DisableDelayedExpansion or something like that
Are you using Windows NT 4/Windows 2000? Only there can you use CALL to call subroutines within the same batch file.
Looking closely at your hex, it does not actually have all CRLF (0d 0a). Several lines end in LF-only (0a without a preceding 0d).
Check in your hex editor to make sure every 0a is preceded by 0d (exactly one).
Or just cut and paste your file into a blank Notepad document and re-save it.