Is it possible to send arabic sms with at-command in text mode (not pdu) and get a delivery report?
It depends on what the device supports. The AT interface itself is ASCII only, so if you want to do anything other than ASCII text you need a device that provides you a way to put Arabic text over that interface - effectively an encoding scheme, at which point you might as well be using PDU mode anyway.
You could put the modem in HEX mode with AT+CSCS="HEX", turn on delivery report with AT+CNMI command, and encode your message in unicode format for AT+CMGS command. Each character should be represented by four hexadecimal digits.
Related
I have data in SQLite that is text including a curly apostrophe (U-2019 ’) but the character is typed as in work’s. A browser makes the request for the data over a web socket connection which is configured as chan configure $sock -encoding iso8859-1 -translation crlf -buffering full simply because that is what I set it to before sending the headers to the browser in response to the request to upgrade the connection to a web socket and failed to consider it again thereafter.
I was mistaken it is endoded as chan configure $sock -buffering full -blocking 0 -translation binary.
If I use replace( ..., '’', '’') in the SQL then the browser displays the text with the curly apostrophe rendered normally. And if I follow the example in this SO question and encode the result of the SQL query as set encoded [encoding convertto utf-8 $SQLResult] it renders correctly also.
What is the correct way to handle this? Should the SQL results be endoded to utf-8 or should the web socket be configured differently? If the socket is encoded as utf-8 then it fails on the first message sent from the browser/client.
Thank you.
The data in SQLite should be Unicode (probably UTF-8 but that shouldn't matter; the package binding handles that). That means that the data arrives in Tcl correctly encoded. Tcl's internal encoding schemes are complicated and should be ignored for the purposes of this discussion: pretend that's all Unicode too.
With a websocket, you need to say what sort of messages ("frames") you are sending. For normal payloads, they can be either binary frames or text frames. Binary frames are a bunch of bytes. Text frames are UTF-8 (as specified by the standard). STOMP frames are text frames where the content is JSON (with a few basic rules and some higher-order things over what operations can be stated).
If you are sending text, you should use UTF-8. Ideally by configuring the socket to do that in the part of the websocket client that mediates between the point that receives the text to send (maybe JSON) and the part that writes the frame onto the underlying socket itself. Above that point, you should just work with ordinary Tcl strings. If you're forming JSON messages in Tcl, the rl_json package is recommended, or you can get SQLite to produce JSON directly. (JSON's serialized form is always UTF-8, and it is conceptually Unicode.)
If you are sending binary frames, you handle all the details yourself. Using encoding convertto utf-8 may help with forming the message frame body, or you can use any other encoding scheme you prefer (such as the one you mention in your question).
Why should we distinguish between text file and binary files when transmitting them? Why there are some channels designed only for textual data? At the bottom level, they are all bits.
At the bottom level, they are all bits... true. However, some transmission channels have seven bits per byte, and other transmission channels have eight bits per byte. If you transmit ASCII text over a seven-bit channel, then all is fine. Binary data gets mangled.
Additionally, different systems use different conventions for line endings: LF and CRLF are common, but some systems use CR or NEL. A text transmission mode will convert line endings automatically, which will damage binary files.
However, this is all mostly of historical interest these days. Most transmission channels are eight bit (such as HTTP) and most users are fine with whatever line ending they get.
Some examples of 7-bit channels: SMTP (nominally, without extensions), SMS, Telnet, some serial connections. The internet wasn't always built on TCP/IP, and it shows.
Additionally, the HTTP spec states that,
When in canonical form, media subtypes of the "text" type use CRLF as the text line break. HTTP relaxes this requirement and allows the transport of text media with plain CR or LF alone representing a line break when it is done consistently for an entire entity-body.
All files are saved in one of two file formats - binary or text. The two file types may look the same on the surface, but their internal structures are different.
While both binary and text files contain data stored as a series of (bits (binary values of 1s and 0s), the bits in text files represent characters, while the bits in binary files represent custom data.
Distinguishing between the two is important as different OSs treat text files differently. For example in *nix you end your lines with just \n while in MS OSs you use \r\n and in Macs you use \n\r. Software such as FTP clients try to change the line endings on text files to match the destination OS by adding/removing the characters. This is to make sure that the text file will look properly on the destination OS.
for example, if you create a text file in *nix with line breaks and try to copy it to a windows box as a binary file and open it in notepad, you will not see any of the line endings, but just a clog of text.
Important to add to the answers already provided is that text files and binary files both represent bytes but text files differ from binary files in that the bytes are understood to represent characters. The mapping of bytes to characters is done consistently over the file using a certain code page or Unicode. When using 7 or 8-bit code pages you can spin the dial when reading these files and interpret them with an English alphabet, a German alphabet, Russian alphabet, or others. This spinning the dial doesn't affect the bytes, it does affect which characters are chosen to correspond to the bytes.
As others have stated, there is also the issue of the encoding of line break separators which is unique to text files and which may differ from platform to platform. The "line break" is not a letter in our alphabet or a symbol you can write, so other rules apply to it.
With binary files there is no implicit convention on character encoding or on the definition of a "line".
All machine language files are actually binary files.
For opening a binary file, file mode has to be mentioned as "rb"or "wb"in fopen command. Otherwise all files are opened in default mode, which is text mode.
It may be noted that text files can also be stored and processed as binary files but not viceversa.
The binary files differ from text file in 2 ways:
The storage of newline characters
The EOF character
Eg:
wt-t stands for textfile
Wb-b stands for binaryfile
Binary files do not store any special character at the end either file end is verified by ueing their size itself.
I am working on reading the barcode using a scanner and I have the following questions:
Is there a way to distinguish between the input from the keyboard and input from the scanner ? This is required for me because the barcode reader will include a preamble, terminator and a prefix to the barcode value. For ex, if the value is 12345 the barcode will have \pre12345\ where \ and \ indicate the start and end and pre identifies the type of data.
If I know that the data is being typed from keyboard and not from a scanner then I would show whatever the user types.
If I know that the data is from a scanner then I will strip of the \pre and \ and show only the data.
Thanks,
My answer assumes that you are using a barcode scanner in "keyboard wedge" mode where it simply emulates a keyboard and does not have some special connection. These devices are often USB and work this way out of the box without any further configuration.
Would the user ever have occasion to actually type your \pre text? Often the prefix is chosen specifically to be a set of characters that the user will never enter or will at least very rarely want to enter. Thus, the mere presence of the prefix indicates a scanned barcode.
Another option would be to time the input. For example, the input from the barcode scanner will probably all be complete within a very short time (e.g. 50ms). If a user were typing the value via the keyboard, it would be impossible for him/her to type the prefix, the data and the postfix all within 50ms. (Of course, assumption here is that input will be made by an actual human and not by automated tools)
well yes! input from the barcode scanner should have first initialized itself through a COM port or serial connection. I know that many apps restrict input to certain field through specific ports, but beyond that I'm not sure.
One solution that I came up with was to start all of our barcodes with a unique sequence of characters not known to the user. I check for this sequence on the input to tell me how the data was entered.
This is a real shot in the dark, however maybe someone had a similar issue. Some console apps are being invoked by either SQL Server 2008, or Autosys (job schedule) under Windows Server 2008; output results of execution are being saved into .txt files. Every so often, with no definite pattern as far as I can tell saved output is displayed as a series of what I presume are Chinese characters. Have anyone encountered phenomenon above?
Typically when you discover chinese characters in output unexpectedly, it's because someone passed a 7-bit or 8-bit character array to an API which expected an array of unicode characters. The system interprets the 8-bit characters as 16 bit unicode characters and they end up being interpreted as unicode characters. At some point later the unicode characters are converted back to 8-bit characters, probably just before they're saved to the text file.
Note: This is an oversimplification but it should be enough to help you figure it out.
I am using a FTPS connection to send a text file
[this file will contain EDI(Electronic Data Interchange) information]to a mailbox INOVIS.I have configured the system to open a FTPS connection and using the PUT command I write the file to a folder on the FTP server.
The problem is: what mode of file transfer should I use? How do I switch between modes?
Moreover which mode is the 'best-practice' to use when transferring file over FTPS connection.
If some one can provide me a small ftp script it would be helpful.
Many of the other answers to this question are a collection of nearly correct to outright wrong information.
ASCII mode means that the file should be converted to canonical text form on the wire. Among other things this means:
NVT-ASCII character set. Even if the original file is in some other character set, such as ASCII, EBCDIC or UTF-8. Technically this disallows characters with the 8th bit set, but most implementations won't enforce this.
CRLF line endings.
EBCDIC mode means a similar set of rules, except that the data on the wire should be in EBCDIC.
LOCAL mode allows sending data with a size other than 8 bits per byte.
IMAGE (or BINARY) mode means that the data should be send without any changes. It is up to the user to ensure that the target system can understand the data once it arrives.
Among other things, this means that the recommendation to use BINARY mode to send text data will fail if one of the systems involved doesn't use a ASCII based character set.
ASCII mode changes new line characters between unix and DOS formats. \n to \r\n and viceversa.
Actually, ASCII/BINARY has nothing to do with the 8th bit. It's a convention for translating line endings.
When you are on a windows machine talking to a Unix FTP server (FTPS or FTP - doesn't matter - the protocol is the same), the server will replace any <CR><LF>-Combination with <LF> before storing the file and consequently do the translation in reverse in case you get the file from the unix server.
The idea behind ASCII mode is to convert the line endings to the respective endings of the target platform.
As todays world seems to be converging to the unix convention (<LF>) and as nearly all of todays editors (aside of notepad) can easily handle Unix-Line-Endings, the days of ASCII mode are, indeed, numbered and I would by all means recommend to always use BINARY transfer mode.
The prospect of having data altered in mid-transfer is somewhat frightening anyways.
ASCII mode also makes file sharing of text files across different platforms more straightforward for end users. They won't have to worry about the default line ending (cr/lf versus just lf for example) since the ASCII mode will do that translation for them on the fly.
For most file types you will ALWAYS want to use BINARY mode though.
ACSII mode converts text files between UNIX and Windows formats based on the server and client platform (CR/LF vs LF), Binary doesn't. Of course, if you transfer nearly anything in ASCII mode that isn't text, it will probably be corrupted for that reason.
If you want an exact copy the data use binary mode - using ascii mode will assume the data is 7bit text (chars 0-127) and truncate any data outside of this range. Dates back to arcane 7bit networking days where ascii mode could save you time.
In a globalized environment that we live in - such that it is quite common to find non-ascii characters e.g. foreign languages, currency symbols etc. - you should always use BINARY mode.
For the FTP protocol, the ASCII transfer mode will consider the 8th bit of each of your character as insignificant and will use it for error checking. As for binary transfer mode, your data will be sent as is. Note that sending binary data in ASCII mode will (almost) always end up in data corruption. However, transferring ASCII data in binary mode will work as long as the sending and receiving systems use the 8th bit in the same way (in modern system the 8th bit should stay at 0 to prevent collision with extended ASCII charsets).