I'm looking for some input on witch file transfer protocol to use over a serial line. I want to be able to transfer files of max 200 Mb size over a serial line (RS232) in both directions, but only one of the machines needs to be able to initiate the get/put (think master-slave).
The protocol also needs to be:
Easy/simple to implement since I would need to write both client and server myself (limited, embedded hardware)
Fairly robust, fault checking/recovery etc
At least somewhat standardized, in case I need to get a third party to implement it on some other hardware
Kermit? TFTP? Simplest possible home brew? What do you think?
In the beginning was the Xmodem, which was very simple to implement. Chuck Forsberg looked at the xmodem and decided it was inefficient, so he begat the Ymodem, but it's implementations were buggy and both x and ymodem were replaced with Zmodem.
Kermit followed on later. Kermit would probably be the "Standard" way to implement this. Do you have access to libraries for Kermit that will run on your embedded platform? If not I would probably consider one of the other options.
If ease of implementation is your primary concern then Xmodem wins hands down.
Related
I've been researching the creation of custom packets, and from what I've seen it is easy enough on Linux, only requiring sudo and the combination of SOCK_RAW and IPPROTO_RAW in the call to socket().
On Windows, on the other hand, very surprisingly, I have found little information on the creation of custom sockets, and most solutions use one of two librarys: WinPcap or NPcap, which both seem less than ideal for what I need, which is only one or two functions out of these librarys.
Therefore I'm curious if there is any more simpler way of doing accomplishing this task than what would otherwise be time-consuming (compiling NPcap from source, trying to understand the entire program in order to butcher it down, remove constraints, etc).
Surely there must be an easier way to use raw sockets.
Simply put, I need to take results from a DAQ and display them visually in a UI (no interaction needed) that gets information updated in real time. The DAQ I am using has an "utility" to plug into Labview, so it seems that the easiest way is to grab this data from Labview and then transmit that data to some UI using one of these methods.
I am using Windows 10 (although I could boot to Ubuntu), just not sure what UI application would be best / easiest to use.
You can use this National Instrument's tool for DAQ UI visualization. As it is native it should be quite straightforward to use.
You may want to use the DAQExpress VI in LabVIEW as #MateoRandwolf suggested. The neat thing about it is that it almost creates your first programm automatically -- besides the configuration of your NI modules.
There are just two things missing:
a waveform chart, and
a write to a TDMS file
Here is a snippit of a simple program doing this (the stop button is important to actually close the TMDS file before aborting the program)
If you really want to stream the data to a different device, you I suggest to use TCP/IP. There exist good examples in the documentation from which you can start (Help > Find Examples... > Search-tab). If you cannot accept the roughly 40ms buffer that TCP/IP has (because of shake-hands etc.), have a look on UDP.
You can use Dewesoft's DAQ systems which use dual mode capability. They use dual data buses (EtherCAT and USB). USB for high-speed buffered data storage to the PC's SSD hard drive and the EtherCAT bus for low latency real-time stream to any 3rd party EtherCAT master.
The DAQ systems are also capable of visualising data in real-time on the display using various pre-build visual displays like recorders, XY graphs, 3D graphs, osciloscopes, FFTs, GPS, video, and numerous other...
As read on topic here How to find the unique serial number of a flash device? and especially here How to get manufacturer serial number of an USB flash drive? I know it is possible to get properties of hardware devices (particularly hard drives and usb drives...) using WMI Win32_PhysicalMedia and Win32_DiskDrive, which I'm getting done successfully.
However, I really want to know about the safety of these informations.
PhysicalMedia property SerialNumber returns the actual serial number of the main hard drive, while using other Win32_LogicalDisk and other calls we can map the drive letter of flash storage to actual Win32_DiskDrive device, and from there read properties like Name, Model, FirmwareRevision, SerialNumber, DeviceID, Manufacturer...
Now, DeviceID is generated by Windows / Pc itself, while SerialNumber should be the one that manufacturer added to the physical flash drive.
Manufacturer in most cases returns "Standard" something, Name is also of no use, while SerialNumber actually gets me a something that looks like unique ID, (I've read that in some cases this is not returned, so PNPDeviceID should be used instead? , Model gives the actual model of the flash drive, and FirmwareRevision just a number that could be used to add safety switch to the licensing, but is not vital.
However, the only one of these that seems / should be actually safe to use is SerialNumber, right?
So, the question here goes: Which level is Win32_DiskDrive actually reading this info from? Is it possible to fake that at all (Ok, letalone the actual lowlevel hacking stuff or driver injection etc...(??)), and if so, how hard it is?
If there's a known way / guide / example, I'd be also happy to read it. (not necessary info looking for here though.)
This is not for intention of bypassing some licensing. I'm making licensing for my SW, and am curious, whether it would be safe enough to use USB drive's SerialNumber property, and lock license against the presence of that USB flash, for which the license was bought for? Basically to use it as kind of a dongle, but not like the dongles actually work (using communication with the actual hardware inside the dongle...)
I know it may not seem as a safe solution, as flash drives dies quite often these days, or get lost etc, but this is just to add an option to my licensing from "Per PC" to "Portable - per USB device".
Thanks for any info!!!
EDIT:
I am completely aware that bypassing these kind of safety switches is very possible. Of course, even Windows itself is not licensed in a way that couldn't be hacked, nor Adobe, ProTools etc, (software that is widely used and costs a lot!).
But that wasn't a real question, and also, that's not the case for me -> the software will not be that expensive and not used by that much people, that I'd be afraid to drag interest in someone who will do extensive programming to make a patch/crack for it. Regular debugger use and workaround is pretty unlikely to be used by regular client who would need the software, ( and also, since it is something to be used in business environment, where stability is vital, I doubt they will really play around that...).
Main point here:
It is possible for sure, but: HOW hard is it to do for a regular person? (I know, the answer is: depending on your code.)
Main question of the post: Is it possible to change the ID on the USB itself, OR to make an app that will fake that data to my app? If it is, I'm sure it might be easier than making a crack/patch, that's why I wanted to know, whether WMI reads explicitly from hardware, or could one make an app that would pass fake data to it?
WMI just returns what the hardware tells it. It's as unique as the hardware. Which ultimately depends on the vendor.
But...
If someone has an administrator account to the computer†, then there are very few things that can be done to keep them from just hooking up the kernel debugger to your program and overriding your checks, or recording the raw USB communication session and replaying it on an unauthorized system. The real dongles do some to mitigate this, by having the hardware generate a response to a particular challenge. The challenge/response changes for each request, so it's not as susceptible to replay attacks, but the debugger tricks still work.
This is the real problem with the serial number approach. Uniqueness is not the primary concern for dongled software. The primary concern is unpredictability.
An illustrative example-
Let's say that I'm a bouncer at an exclusive night club. We're so exclusive that you have to answer a question to get in. You really want to get in, but no one will tell you the answer to the question. One night, you hatch a plan. You hang out in the alley and listen to the conversations that I'm having with the patrons trying to enter the club. It doesn't take you long to realize that I'm asking everyone the exact same question, and you're in. (This is the serial number approach)
After a while, I notice that there are a lot of people coming into the club that I've never seen before, and begin to suspect something. The people we really want to allow in are all given a card with a formula‡ on it. Whenever they come to the door of the club, I give them a number and they apply their formula and tell me the result. Since I also know the formula, I can tell if they are really allowed in. Now, even if you hear the entire challenge and response, without the formula, you aren't getting in. (This is one common approach taken by dongles.)
But what about the debugger? The debugger just made herself the club's owner, fired me, and can come and go as she pleases.
†Or has physical access to the machine and a password reset disk.
‡Stop laughing, this could totally happen. :)
Photo credit: Guillaume Paumier, CC-BY. Found on the Wikimedia Commons 7-Oct-15
Edit to address the question edit:
HOW hard is it to do for a regular person? (I know, the answer is: depending on your code.)
The question is how skilled is the 'regular person'? If you're talking about software/electrical engineers, then this is a trivial task. If you're talking about sales/marketing then it's a challenging task.
Is it possible to change the ID on the USB itself, OR to make an app that will fake that data to my app?
It depends and Yes. Changing the ID on the device itself is possible with some devices, and impossible with others. Software to spoof/man-in-the-middle the USB communication, or to create a virtual USB device is possible.
If it is, I'm sure it might be easier than making a crack/patch, that's why I wanted to know, whether WMI reads explicitly from hardware, or could one make an app that would pass fake data to it?
As I led with above, WMI reads from the hardware. This can be intercepted or bypassed.
Some ways to bypass the check:
Make a virtual USB device
Modify the USB MSD device driver to report the same serial number for all devices.
Build hardware using commercially available cheap host controllers that identifies with the same information as the authorized device. ($10 worth of raw components and a little bit of time.)
Redirect the system calls to/from USB to a compromised library.
Note also that:
Some places have restrictions on USB storage devices, ranging from discouraging their use, to outright bans. This would prevent your software from being used in sensitive computing environments processing private data, like credit cards, PII, trade secrets, classified information, etc. (In the US many governmental agencies have outright bans on USB storage devices, and block the install of any MSD.)
The Mass Storage specification doesn't require serial numbers. They are usually there, but they don't have to be, and many low-cost vendors
A USB PKI token costs a little bit more, but would probably do what you want. Here's an example from Safenet (Disclaimer: I am in no way affiliated with Safenet Inc, and you should evaluate all the possible options from all vendors. I suggested this because it was the first thing that came up through CDW, and the price was ~$30)
I've got a project to create a roguelike that in some way abstracts the UI from the engine and the engine from map creation, line-of-site, etc. To narrow the focus, i first want to just get the UI (player's client) and engine working.
My current idea is to make the client basically a program that decides what one character (player, monsters) will do for its turn and waits until it can move again. So each monster has a client, and so does the player. The player's client prints the map, waits for input, sends it to the engine, and tells the player what happened. The monster's client does the same except without printing the map and using AI instead of keyboard input.
Before i go any futher, if this seems somehow an obfuscated way of doing things, my goal is to learn, not write a roguelike. It's the journy, not the destination.
And so i need to choose what form of ipc fits this model best.
My first attempt used pipes because they're simplest and i wrote a
UI for the player and a program to pipe in instructions such as
where to put the map and player. While this works, it only allows
one client--communicating through stdin and out.
I've thought about making the engine a daemon that looks in a spool
where clients, when started, create unique-per-client temp files to
give instructions to the engine and recieve feedback.
Lastly, i've done a little introductory programing with sockets.
They seem like they might be the way to go, and would allow the game
to perhaps someday be run over a net. I'd like to, if possible, use
a simpler solution, and since i'm unfamiliar with them, it's more
error prone.
I'm always open to suggestions.
I've been playing around with using these combinations for a similar problem (multiple clients talking via a single daemon on the local box, with much of the intelligence shoved off into the clients).
mmap for sharing large data blobs, with unix domain sockets, messages queues, or named pipes for notification
same, but using individual files per blob instead of munging them all together in an mmap
same, but without the files or mmap (in other words, more like conventional messaging)
In general I like the idea of breaking things up into separate executables this way -- it certainly makes testing easier, for instance. I think the choice of method comes down to usage patterns -- how large are messages, how persistent does the data in them need to be, can you afford the cost of multiple trips through the network stack for a socket-based message, that sort of thing. The fact that you're sticking to Linux makes things easy in terms of what's available -- you don't need to worry about portability of message queues, for instance.
This one's also applicable: https://stackoverflow.com/a/1428542/1264797
Has anyone tried to create a log file of interprocess communications? Could someone give me a little advice on the best way to achieve this?
The question is not quite clear, and comments make it less clear, but anyway...
The two things to try first are ipcs and strace -e trace=ipc.
If you want to log all IPC(seems very intensive), you should consider instrumentation.
Their are a lot of good tools for this, check out PIN in perticular, this section of the manual;
In this example, we show how to do
more selective instrumentation by
examining the instructions. This tool
generates a trace of all memory
addresses referenced by a program.
This is also useful for debugging and
for simulating a data cache in a
processor.
If your doing some heavy weight tuning and analysis, check out TAU (Tuning and analysis utilitiy).
Communication to a kernel driver can take many forms. There is usually a special device file for communication, or there can be a special socket type, like NETLINK. If you are lucky, there's a character device to which read() and write() are the sole means of interaction - if that's the case then those calls are easy to intercept with a variety of methods. If you are unlucky, many things are done with ioctls or something even more difficult.
However, running 'strace' on the program using the kernel driver to communicate can reveal just about all it does - though 'ltrace' might be more readable if there happens to be libraries the program uses for communication. By tuning the arguments to 'strace', you can probably get a dump which contains just the information you need:
First, just eyeball the calls and try to figure out the means of kernel communication
Then, add filters to strace call to log only the kernel communication calls
Finally, make sure strace logs the full strings of all calls, so you don't have to deal with truncated data
The answers which point to IPC debugging probably are not relevant, as communicating with the kernel almost never has anything to do with IPC (atleast not the different UNIX IPC facilities).