Does it possible debug Tcl/tk application for Windows, without the source code? The application have no support and there is no source code available. There is error shown during one of operation. Is it possible to find what cause an error and whether error can be fixed by patching code?
It's very difficult without the source code, as it is at the level of the source code where you'd need to make the patch. If the code is exposing itself via the send command mechanism (or the comm package) you can probably make a bit of progress (as there's quite good introspection capabilities in Tcl by default, so info commands, info vars and info body may help, along with many other info commands and some introspectors that are elsewhere too), but it's still really difficult, particularly if you don't understand the internal structure of the code. OTOH, I wouldn't expect a production application to expose itself this way; typically you disable that sort of thing when outside development.
A standard debugger like gdb won't help, especially with the Tcl 8.6 non-recursive runtime. (Tcl applications in that environment just tend to show up as effectively “doing stuff”; there's nothing to really indicate how it hangs together.)
But the first thing to check is whether you actually have the source code. It's possible that the code has just been packaged together as a Starpack and that you can use a tool like sdx to extract the things you need to make changes in. But you aren't guaranteed to succeed at that; if the code was compiled/obscured with a commercial tool like the TDK, you really don't have the source and can't really do much about it. (By the same general principles that mean that DRM doesn't work well, it's possible to decompile the .tbc files that the TDK produces, but that's really a last-ditch thing to do as it is technically illegal in some jurisdictions, morally rather dodgy, and inclined to produce really awful output.) Can you contact the original author(s) of the code instead? If that works, it'll be cheaper and simpler…
If you've actually got the human-readable code, even if packaged with sdx, then you can do a lot more.
Related
I wrote script in tcl and want to protect it.
Is there any way to compile the script so no one could see the code inside?
What is the conventional way to protect tcl script?
There's a few ways, depending on the level of sophistication of protection you want:
Against the least sophisticated users, just changing the extensions on the filenames may be enough (e.g., foobar.tcl to foobar.private).
You can wrap the files up inside an archive of some sort (e.g., a starkit, or a ZIP archive which is natively supported in Tcl 8.7); the archive might be combinable with a suitable runtime to make a single-file executable.
You can use the Tcl Dev Kit's “compiler” (Tcler's wiki page has many links) to create an obfuscated bytecode file. This requires a small extension to load, but the obfuscation is enough to defeat all but the most sophisticated of attackers. This can be mixed with the previous two techniques.
The ultimate in protecting a file is to use a full compiler like tclquadcode (experimental, but might do good things for you). The output of such programs is exceptionally hard to reverse engineer; it looks like machine code from another planet!
If even that's not enough, you have to go down the route of running the code as a web service and only allowing users to use it remotely; stealing code is (theoretically) impossible in that scenario, or at least (practically) incredibly difficult. This is a very large topic of its own.
You need to obfuscate your TCL file. Check this.
I am using a rather obsure, proprietary langauge called WIL/Winbatch that had an awful IDE (winbatch studio).
I would like to develop an alternative environment; however, without the ability to set breakpoints, step, and examine variables, there is really no point. How does one begin even researching how to implementing a debugger for a proprietary language? Is it even legal?
I guess I'm kind of locked in a mindset that the debugger portion must be able to examine the statements that are provided to it in WIL as they are executed, right? So somehow i have to trap the output of the interpreter? Or is it just a matter of reading locations in memory using whatever language?
Thanks in advance.
Having been there and successfully completed the task, here are the things to keep in mind:
Build it as a plug-in/extension to an IDE your team is already familiar with and likes. They'll thank you for providing an interface consistent with what they really know how to use, plus you can focus entirely on the features that make your language different from others.
You'll have to learn the debugging protocol for your language. In our case, we had source access to the runtime for the interpreted language. In other cases, you may find documentation for GDB local or remote debugging interface, a library you can link to for the language's debugging protocols, or maybe even figure out what the call stacks look like and wrap the Windows Debugging API to analyze it behind the scenes.
Don't build in excess of what the language provides. Adding debugging features takes a lot of time, and they have a rather annoying habit of needing to be significantly altered or completely rewritten as versions of the target language are updated.
Why are you tied so closely to this language? If it's not well supported, there are many others you can use. Anyway, to actually answer your question, the difficulty depends on whether it is a compiled or interpreted language and whether or not you have access to any source code (which it seems of course, that you don't). That said, this would be a very challenging project as you would have to reverse engineer the compiled code for it to have any meaning. Your time would be better spent learning another (better) language.
Perhaps if you can give us an idea of why you want to use this language we could give you some help?
I am trying to help a client with a problem, but I am running out of ideas. They have a custom, written in house application that runs on a schedule, but it crashes. I don't know how long it has been like this, so I don't think I can trace the crashes back to any particular software updates. The most unfortunate part is there is no longer any source code for the VB6 DLL which contains the meat of the logic.
This VB6 DLL is kicked off by 2-3 function calls from a VB Script. Obviously, I can modify the VB Script to add error logging, but I'm not having much luck getting quality information to pinpoint the source of the crash. I have put logging messages on either side of all of the function calls and determined which of the calls is causing the crash. However, nothing is ever returned in the err object because the call is crashing wscript.exe.
I'm not sure if there is anything else I can do. Any ideas?
Edit: The main reason I care, even though I don't have the source code is that there may be some external factor causing the crash (insufficient credentials, locked file, etc). I have checked the log file that is created in drwtsn32.log as a result of wscript.exe crashing, and the only information I get is an "Access Violation".
I first tend to think this is something to do with security permissions, but couldn't this also be a memory access violation?
You may consider using one of the Sysinternals tools if you truly think this is a problem with the environment such as file permissions. I once used Filemon to figure out all the files my application was touching and discovered a problem that way.
You may also want to do a quick sanity check with Dependency Walker to make sure you are actually loading the DLL files you think you are. I have seen the wrong version of the C runtime being loaded and causing a mysterious crash.
Depending on the scope of the application, your client might want to consider a rewrite. Without source code, they will eventually be forced to do so anyway when something else changes.
It's always possible to use a debugger - either directly on the PC that's running the crashing app or on a memory dump - to determine what's happening to a greater or lesser extent. In this case, where the code is VB6, that may not be very helpful because you'll only get useful information at the Win32 level.
Ultimately, if you don't have the source code then will finding out where the bug is really help? You won't be able to fix it anyway unless you can avoid that code path for ever in the calling script.
You could use the debugging tools for windows. Which might help you pinpoint the error, but without the source to fix it, won't do you much good.
A lazier way would be to call the dll from code (not a script) so you can at least see what is causing the issue and inspect the err object. You still won't be able to fix it, unless the problem is that it is being called incorrectly.
The guy of Coding The Wheel has a pretty interesting series about building an online poker bot which is full of serious technical info, a lot of which is concerned with how to get into existing applications and mess with them, which is, in some way, what you want to do.
Specifically, he has an article on using WinDbg to get at important info, one on how to bend function calls to your own code and one on injecting DLLs in other processes. These techniques might help to find and maybe work around or fix the crash, although I guess it's still a tough call.
There are a couple of tools that may be helpful. First, you can use dependency walker to do a runtime profile of your app:
http://www.dependencywalker.com/
There is a profile menu and you probably want to make sure that the follow child processes option is checked. This will do two things. First, it will allow you to see all of the lib versions that get pulled in. This can be helpful for some problems. Second, the runtime profile uses the debug memory manager when it runs the child processes. So, you will be able to see if buffers are getting overrun and a little bit of information about that.
Another useful tool is process monitor from Mark Russinovich:
http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx
This tool will report all file, registry and thread operations. This will help you determine if any you are bumping into file or registry credential issues.
Process explorer gives you a lot of the same information:
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
This is also a Russinovich tool. I find that it is a bit easier to look at some data through this tool.
Finally, using debugging tools for windows or dev studio can give you some insight into where the errors are occurring.
Access violation is almost always a memory error - all the more likely in this case because its random crashing (permissions would likely be more obviously reproducible). In the case of a dll it could be either
There's an error in the code in the dll itself - this could be something like a memory allocation error or even a simple loop boundary condition error.
There's an error when the dll tries to link out to another dll on the system. This will generally be caused by a mismatch between dll versions on the machine.
Your first step should be to try and get a reproducible crash condition. If you don't have a set of circumstances that will crash the system then you cannot know when you have fixed it.
I would then install the system on a clean machine and attempt to reproduce the error on that. Run a monitor and check precisely what other files (dlls etc) are open when the program crashes. I have seen code that crashes on a hyperthreaded Pentium but not on an earlier one - so restoring an old machine as a testbed may be a good option to cover that one. Varying the amount of ram in the machine is also worthwhile.
Hopefully these steps might give you a clue. Hopefully it will be an environment problem and so can be avoided by using the right version of windows, dlls etc. However if you're still stuck with the crash at this point with no good clues then your options are either to rewrite or attempt to hunt down the problem further by debugging the dll at assembler lever or dissassembling it. If you are not familiar with assembly code then both of these are long-shots and it's difficult to see what you will gain - and either option is likely to be a massive time-sink. Myself I have in the past, when faced with a particularly low-level high intensity problem like this advertised on one of the 'coder for hire' websites and looked for someone with specialist knowledge. Again you will need a reproducible error to be able to do this.
In the long run a dll without source code will have to be replaced. Paying a specialist with assembly skills to analyse the functions and provide you with flowcharts may well be worthwhile considering. It is good business practice to do this sooner in a controlled manner than later - like after the machine it is running on has crashed and that version of windows is no longer easily available.
You may want to try using Resource Hacker you may have luck de-compiling the in house application. it may not give you the full source code but at least maybe some more info about what the app is doing, which also may help you determine your culrpit.
Add the maximum possible RAM to the machine
This simple and cheap hack has work for me in the past. Of course YMMV.
Reverse engineering is one possibility, although a tough one.
In theory you can decompile and even debug/trace a compiled VB6 application - this is the easy part, modifying it without source, in all but the most simple cases, is the hard part.
Free compilers/decompilers:
VB decompilers
VB debuggers
Rewrite would be, in most cases, a more successful and faster way to solve the problem.
Is it possible to remotely debug a process started outside VB6?
The application is a VB6 application with quite a few dll/ocx resources. I am attempting to setup a ClickOnce deployment, using Registration-Free COM, of the VB6 app but have been getting errors when it executes.
My understanding of the way that VB6 redirects COM registerations will probably mean that this is not possible but I thought someone might have a better idea.
To support Darryl's answer suggesting Windbg - here's a 2006 blog post by a Microsoft guy about using Windbg with VB6, and 2004 blog post by another Microsoft guy with a brief introduction to Windbg.
EDIT: Just to make it totally clear. Windbg is a free standalone debugger from Microsoft. Compile your VB6 EXEs, DLLs and OCXs into native code with symbols (create PDB files) and you will be able to debug your ClickOnce application.
Key excerpt from the blog:
If you have limited access to the server machine then you can use the
remote debugging facilities of WinDbg. Attach a copy of WinDbg to the
process in the usual way and then turn it in to a debugging server
(check out .server in the WinDbg help). You can then connect to it
remotely from the File menu of WinDbg. It will be just like being
there except for the lack of noise from the server room fans. When
debugging a remote, your copy of WinDbg is just a very smart terminal
so all extensions, symbols and so on have to be on the remote server.
You set this up the exact same way for any DLL, VB6 or .NET.
The symbols for your component will not load until your component does
and so you have to let the server run at least that long. You can put
a break in early in your VB code if you want to stop the debugger at
that point but if you do, remember that it will stop there every time
through the code. Let’s assume that you let it run and then break in.
If you list the loaded symbols for your module with "x MyModule!*"
then you will see all of your functions together with a lot of symbols
bundled in there for you. VB adds interfaces and symbols quite
unashamedly but you don’t generally need to worry about those. One
thing that will probably look strange is that all class/method syntax
with the C++ double colon convention instead of the friendly little
dot. WinDbg doesn’t understand that VB is different and it is treated
just like any DLL with symbols.
From here, you can set breakpoints in the usual way (bp etc) and step
through code. You can also open up VB source code modules and set
breakpoints in them with F9 although the VB file extensions are not in
the source file type dropdown. Stepping through the code is revealing
but might be a little alarming if you have not seen the code that VB
generates for you before. You will be stepping through the assembler
and there is a lot of COM goo in there. Hresults get checked a lot.
You will probably need to refer to the source often to work out where
you are since it takes a bit of practice to be able to know what the
source code looked like. Variants are especially challenging because
VB does a lot of work for you there and what looks like a simple
equation can result in a great deal of code. Optimised code is even
harder because the order of execution is often very different from
what you might expect and it is harder than usual to see the data.
Data is not easy to get at this way. When you look at local variables
(dv is the command) then you may see that variables are simply listed
as eclipsed which means that the memory is being used for something
else as well within the function lifetime or that the name is not
unique in this context. Enums just show as integers or longs and
objects show as pointers. In fact, they always were exactly that but
the VB IDE hides that from you. VB strings are COM BSTRs (and
accordingly Unicode) under the covers and byte arrays are really char
arrays. You might be surprised to discover that VB strings are Unicode
as VB appears to have no support for anything but ANSI. That is
because the Ruby forms engine was ANSI only. The runtime converts the
Unicode strings to ANSI for Ruby and API calls although there are ways
to pass Unicode if you want.
You are not going to be able to get at the Err, App or Printer objects
since you would need to go through a lot of internal and completely
undocumented structures to get at them. Even if you could get there,
they would just be raw data without the accessor functions that you
use in VB. If you need to look at any of those fields, your best bet
is to embed debug code in the source code to copy their values to
somewhere that you can get at.
You can step in to the VB runtime if you want but it probably won’t be
very revealing if you are trying to debug your application. If you do,
you will notice that VB’s internals are very COM influenced. The
influence was actually two way since some COM ideas came from VB
originally.
You may see exceptions when running your code. Null reference
exceptions (i.e dereferencing a null pointer) are not uncommon or
anything to worry about. They will show up as first chance C000005
exceptions with a 0 or almost 0 address. The runtime will sometimes do
that if there are objects set to nothing but that is safe because the
only possible values are null or a valid value. You will also see
exceptions if your code does lookups in collections and the value is
not there. Because exceptions are now so expensive, you probably want
to avoid doing that if you can. Another exception that you will
commonly see is c000008f. If you look the number up then you will find
that it is a floating point inexact result exception. It is used in a
different meaning here – since we don’t generate real floating point
inexact result exceptions, they can safely be thrown to indicate VB
errors of the normal trappable type.
Debugging hangs and crashes in VB components is done very much in the
same way as with any other unmanaged component but it is just a little
harder because of the compilations described above. If you have to try
debugging VB code this way, I would strongly recommend that you start
on a "Hello world" application and work your way up. All the things
that may VB an easy language to code in make it a terrible language to
debug.
I believe that when debugging in VB6, it does not attach to a running binary but instead interprets the code within it's own process. This is why the Task Manager and Win32 APIs show VB6.exe as the running app when debugging.
Also as you say, VB6 sometimes short-circuits calls to COM libraries so intercepting these calls is not always possible.
You're probably going to have to resort to intelligent logging (i.e. log the values of variables around the points where the errors you are getting occur in the hope of locating the line of code it occurs on, and/or the state of relevant variables.)
Good luck
Have you tried windbg? Just make sure you have pdb files for the project.
What would be the best way of inserting functionality into a binary application (3d party, closed source).
The target application is on OSX and seems to have been compiled using gcc 3+. I can see the listing of functions implemented in the binary and have debugged and isolated one particular function which I would like to remotely call.
Specifically, I would like to call this function - let's call it void zoomByFactor(x,y) - when I receive certain data from a complex HIDevice.
I can easily modify or inject instructions into the binary file itself (ie. the patching does not need to occur only in RAM).
What would you recommend as a way of "nicely" doing this?
Edit:
I do indeed need to entire application. So I can't ditch it and use a library. (For those who need an ethical explanation: this is a proprietary piece of CAD software whose company website hasn't been updated since 2006. I have paid for this product (quite a lot of money for what it is, really) and have project data which I can not easily migrate away from it. The product suits me just fine as it is, but I want to use a new HID which I recently got. I've examined the internals of the application, and I'm fairly confident that I can call the correct function with the relevant data and get it to work properly).
Here's what I've done so far, and it is quite gheto.
I've already modified parts of the application through this process:
xxd -g 0 binary > binary.hex
cat binary.hex | awk 'substitute work' > modified.hex
xxd -r modified.hex > newbinary
chmod 777 newbinary
I'm doing this kind of jumping through hoops because the binary is almost 100 megs large.
The jist of what I'm thinking is that I'd jmp somewhere in the main application loop, launch a thread, and return to the main function.
Now, the questions are: where can I insert the new code? do I need to modify symbol tables? alternatively, how could I make a dylib load automatically so that the only "hacking" I need to do is inserting a call to a normally loaded dylib into the main function?
For those interested in what I've ended up doing, here's a summary:
I've looked at several possibilities. They fall into runtime patching, and static binary file patching.
As far as file patching is concerned, I essentially tried two approaches:
modifying the assembly in the code
segments (__TEXT) of the binary.
modifying the load commands in the
mach header.
The first method requires there to be free space, or methods you can overwrite. It also suffers from extremely poor maintainability. Any new binaries will require hand patching them once again, especially if their source code has even slightly changed.
The second method was to try and add a LC_ LOAD_ DYLIB entry into the mach header. There aren't many mach-o editors out there, so it's hairy, but I actually modified the structures so that my entry was visible by otool -l. However, this didn't actually work as there was a dyld: bad external relocation length at runtime. I'm assuming I need to muck around with import tables etc. And this is way too much effort to get right without an editor.
Second path was to inject code at runtime. There isn't much out there to do this. Even for apps you have control over (ie. a child application you launch). Maybe there's a way to fork() and get the initialization process launched, but I never go that.
There is SIMBL, but this requires your app to be Cocoa because SIMBL will pose as a system wide InputManager and selectively load bundles. I dismissed this because my app was not Cocoa, and besides, I dislike system wide stuff.
Next up was mach_ inject and the mach_star project. There is also a newer project called
PlugSuit hosted at google which seems to be nothing more than a thin wrapper around mach_inject.
Mach_inject provides an API to do what the name implies. I did find a problem in the code though. On 10.5.4, the mmap method in the mach_inject.c file requires there to be a MAP_ SHARED or'd with the MAP_READ or else the mmap will fail.
Aside from that, the whole thing actually works as advertised. I ended up using mach_ inject_ bundle to do what I had intended to do with the static addition of a DYLIB to the mach header: namely launching a new thread on module init that does its dirty business.
Anyways, I've made this a wiki. Feel free to add, correct or update information. There's practically no information available on this kind of work on OSX. The more info, the better.
In MacOS X releases prior to 10.5 you'd do this using an Input Manager extension. Input Manager was intended to handle things like input for non-roman languages, where the extension could popup a window to input the appropriate glyphs and then pass the completed text to the app. The application only needed to make sure it was Unicode-clean, and didn't have to worry about the exact details of every language and region.
Input Manager was wildly abused to patch all sorts of unrelated functionality into applications, and often destabilized the app. It was also becoming an attack vector for trojans, such as "Oompa-Loompa". MacOS 10.5 tightens restrictions on Input Managers: it won't run them in a process owned by root or wheel, nor in a process which has modified its uid. Most significantly, 10.5 won't load an Input Manager into a 64 bit process and has indicated that even 32 bit use is unsupported and will be removed in a future release.
So if you can live with the restrictions, an Input Manager can do what you want. Future MacOS releases will almost certainly introduce another (safer, more limited) way to do this, as the functionality really is needed for language input support.
I believe you could also use the DYLD_INSERT_LIBRARIES method.
This post is also related to what you were trying to do;
I recently took a stab at injection/overriding using the mach_star sources. I ended up writing a tutorial for it since documentation for this stuff is always so sketchy and often out of date.
http://soundly.me/osx-injection-override-tutorial-hello-world/
Interesting problem. If I understand you correctly, you'd like to add the ability to remotely call functions in a running executable.
If you don't really need the whole application, you might be able to strip out the main function and turn it into a library file that you can link against. It'll be up to you to figure out how to make sure all the required initialization occurs.
Another approach could be to act like a virus. Inject a function that handles the remote calls, probably in another thread. You'll need to launch this thread by injecting some code into the main function, or wherever else is appropriate. Most likely you'll run into major issues with initialization, thread safety, and/or maintaining proper program state.
The best option, if its available, is to get the vendor of your application to expose a plugin API that lets you do this cleanly and reliably in a supported manner.
If you go with either hack-the-binary route, it'll be time consuming and brittle, but you'll learn a lot in the process.
On Windows, this is simple to do, is actually very widely done and is known as DLL/code injection.
There is a commercial SDK for OSX which allows doing this: Application Enhancer (free for non-commercial use).