Is there a tool for refactoring using semantic patches for shell scripts, just like Coccinelle for C?
An example modification would be to switch from
command > file
syntax to "sticky" one
command >file
Is there a tool for refactoring shell scripts? I doubt it.
However, you could build one using a general program transformation system (PTS).
These are tools that accept language descriptions (you'd need a grammar for POSIX shell scripts), will parse said langauge building ASTs, and then allow you to apply transformations to those ASTs, finally prettyprinting the AST back to valid source text.
Good PTSes let you express code changes using source code patterns (Cocinelle is not a genearl purpose PTS since it only works for C, but it falls into this category of source-pattern driven) rather than writing procedure code to modify the trees.
A problem with most of them is they do not go beyond matching on (context-free) ASTs, while real constraints require the tool to understand "context" (e.g., how information from far away in the source text affects the meaning of a particular point in the text). Cocinelle also does this, which is why it is an interesting tool; this kind of capability is necessary to transform traditional programming languages.
Our DMS Software Reengineering Toolkit is a general PTS that provide support for context analysis (symbol tables, control and data flow analysis, ...). I think (Unix) shell scripting langauge like Posix, all have various macro like capabilities that make processing them much harder than macro-free langauges such as Java. DMS provides built-in support for capturing and handling preprocessor conditionals and macros; it presently uses these to handle C and C++.
But, a Posix transformation is not out of the box. You have to define its grammar, and the various context analysis support to DMS. At that point you can start to write context-dependent transformations using source patterns. This work is doable with DMS, but isn't a weekend exercise. So the real question is,
how much automated patching do you intend to do; is it enough to justify configuring a PTS?
Related
Vimscript is difficult. Ruby is not quite so diffiuclt. Customizing Vim with Ruby scripts can be done, and I am trying to learn how. This is a useful presentation about it which covers the basics, but meaningful examples are scarce (and these are rather complicated), so I'm wondering if anyone with experience in this area can offer some smaller examples of Vim mappings and shortcuts written in Ruby.
As a specific example of the kind of scale I'm looking for, let's suppose I want to create section headers for my documentation or something, as in
----------------------------------------------
------------------- SECTION ------------------
----------------------------------------------
where the section name is centered in the set of hyphens, and to achieve this I visually select the word
SECTION
on it's own line, and hit leader <arbitrary keystroke>.
Counterargument: Vimscript isn't difficult, maybe a bit different; after all, much of it is modeled after Python.
I do agree that for certain, complex tasks (especially anything that requires interaction with the "outside world", be it file systems, web service calls, etc.), or stuff that benefits from library functions, a different programming language (and fortunately one can choose among powerhorses like Perl, Python, and Ruby) has undeniable benefits.
But the example task you're giving is just a simple sequence of yanking, simple string manipulation, followed by paste. There's little meat, and the interaction with the Vim buffer isn't that different when done in an integration language. That's my main point: You still have to integrate with Vim, and for that, some knowledge of Vim's structure (and that means Vimscript) is necessary.
So iOS 6 deprecates presentModalViewController:animated: and dismissModalViewControllerAnimated:, and it replaces them with presentViewController:animated:completion: and dismissViewControllerAnimated:completion:, respectively. I suppose I could use find-replace to update my app, although it would be awkward with the present* methods, since the controller to be presented is different every time. I know I could handle that situation with a regex, but I don't feel comfortable enough with regex to try using it with my 1000+-files-big app.
So I'm wondering: Does Xcode have some magic "update deprecated methods" command or something? I mean, I've described my particular situation above, but in general, deprecations come around with every OS release. Is there a better way to update an app than simply to use find-replace?
You might be interested in Program Transformation Systems.
These are tools that can automatically modify source code, using pattern-directed source-to-source transformations ("if you see this source-level pattern, replace it by that source-level pattern") that operate on code structures rather than text. Done properly, these transformations can be reliable and semantically correct, and they're a lot easier to write than low-level procedural code that navigates and smashes nanoscopic actual tree structures.
It is not the case that using such tools is easy; such tools have to know how to parse the language of interest into compiler data structures, (e.g., ObjectiveC), process the patterns, and regenerate compilable source code from the modified structures. Even with the basic transformation engine, somebody needs to carefully define parsers (and unparsers!) for the dialects of the languages of interest. And it takes time to learn how to use such a even if you have such parsers/unparsers. This is worth it if the changes you need to make are "regular" (in the program transformation sense, not the regexp sense) and widespread (as yours seem to be).
Our DMS Software Reengineering toolkit has an ObjectiveC front end, and can carry out such transformations.
no there is no magic like that
I am aware that by default Java does not have the so-called eval (what I pronounce as "evil") method. This sounds like a bad thing—knowing you do not have something which so many others do. But even worse seems being notified that you can't have it.
My question is: What is solid reasoning behind it? I mean, Google'ing this just returns a massive amount of old data and bogus reasons—even if there is an answer that I'm looking for, I can't filter it from people who are just throwing generic tag-words around.
I'm not interested in answers that are telling me how to get around that; I can do that myself:
Using Bean Scripting Framework (BSF)
File sample.py (in py folder) contents:
def factorial(n):
return reduce(lambda x, y:x * y, range(1, n + 1))
And Java code:
ScriptEngine engine = new ScriptEngineManager().getEngineByName("jython");
engine.eval(new FileReader("py" + java.io.File.separator + "sample.py"));
System.out.println(engine.eval("factorial(932)"));
Using designed bridges like JLink
This is equivalent to:
String expr = "N[Integrate[E^(2 y^5)/(2 x^3), {x, 4, 7}, {y, 2, 3}]]";
System.out.println(MM.Eval(expr));
//Output: 1.5187560850359461*^206 + 4.2210685420287355*^190*I
Other methods
Using Dijkstras shunting-yard algorithm or alike and writing an expression evaluator from scratch.
Using complex regex and string manipulations with delegates and HashMultimaps.
Using Java Expressions Library
Using Java Expression Language
Using JRE compliant scripting language like BeanShell.
Using the Java Assembler and approach below or direct bytecode manipulation like Javaassist.
Using the Java Compiler API and reflections.
Using Runtime.getRuntime().exec as root
"eval" is only available in scripting languages, because it uses the same interpreter that runs the rest of the code; in such languages the feature is free and well integrated, as in scripting environment it makes little difference if you run a string or a "real" function.
In copiled languages, adding "eval" would mean bundling the whole compiler - which would defy the purpose of compiling. No compiled language I know (even dynamic ones, like ActionScrip3) has eval.
Incidentally, the easiest way to eval in Java is the one you forgot to mention: JRE 1.6 comes with Javascript engine, so you can eval any Javascript in two lines of code. You could even argue that the presuposition of your question is false. Java 1.6 bundles a very advanced expression evaluator.
As Daniel points out there is at least one limitation that eval-solutions face in java. The php eval for example executes the code as if it was part of the surrounding method with complete access to local variables, this is not possible to do in standard java. Without this feature eval alternatives require a lot more work and verbosity, which makes them a lot less attractive for "quick" and "easy" solutions.
eval() is mostly part of interpreted languages where the names of local variables and code structure(scopes) are available at runtime, making it possible to "insert" new code. Java bytecode no longer contains this information leaving eval() alternatives unable to map access to local variables. (Note: I ignore debug information as no program should rely on it and it may not be present)
An example
int i = 0;
eval("i = 1");
System.out.println(i);
required pseudocode for java
context.put("i",new Integer(0));
eval(context,"i = 1");
System.out.println(context.get("i"));
This looks nice for one variable used in the eval, try it for 10 in a longer method and you get 20 additional lines for variable access and the one or other runtime error if you forget one.
Because evaluation of arbitrary Java expressions depends on the context of it, of variable scopes etc.
If you need some kind of variable expression, just use the scripting framework, and badamm! you have lots of different kinds of expression evaluation. Just take one kind like JavaScript as a default, and there is your eval()!
Enterprisy as Java is, you are not constrained to one choice.
But even worse seems being notified that you can't have it.
I think you are misunderstanding what (most of) those articles are saying. Clearly, there are many ways to do expression evaluation in a Java application. They haven't always been available, but at least some of them have been around for a long time.
I think what people are trying to say is that expression evaluation is not available as native (i.e. as an intrinsic part of Java or the standard libraries) and is unlikely to be added for a number of good reasons. For example:
Native eval would have significant security issues if used in the wrong place. (And it does for other languages; e.g. you shouldn't use eval in Javascript to read JSON because it can be a route for injecting bad stuff into the user's browser.)
Native eval would have significant performance issues, compared with compiled Java code. We are talking of 100 to 10,000 times slower, depending on the implementation techniques and the amount of caching of "compiled" eval expressions.
Native eval would introduce a whole stack of reliability issues ... much as overuse / misuse of type casting and reflection to.
Native eval is "not Java". Java is designed to be a primarily static programming language.
and of course ...
There are other ways to do this, including all of the implementation approaches that you listed. The Java SE platform is not in the business of providing every possible library that anyone could possibly want. (JRE downloads are big enough already.)
For these reasons, and probably others as well, the Java language designers have decided not to support expression evaluation natively in Java SE. (Even so, some expression support has officially made it into Java EE; e.g. in the form of JSP Expression Language. The classes are in the javax.el package ... or javax.servlet.jsp.el for an older / deprecated version.)
I think you already put the solution to your answer - bundle the BeanShell jar with your application (or lobby for it to be included in the JRE sometime), and you have your Java expression evaluator. It will still need a Binding of the input variables, though.
(What I'm more curious about: How does sandboxing of such a script/expression work? I don't want my web users to execute dangerous code in my server.)
As a pet project, I was thinking about writing a program to migrate applications written in a language A into a language B.
A and B would be object-oriented languages. I suppose it is a very hard task : mapping language constructs that are alike is doable, but mapping libraries concepts will be a very long task.
I was wondering what tools to use, I know this has to do with compilation, but I'm a bit afraid to use Lex and Yacc and all that stuff.
I was thinking of maybe using the Eclipse Modeling Framework, which would help me write models (of application code) transformations in a readable form.
But first I would have to write parsers for creating the models (and also create the metamodel from the language grammar).
Are there tools that exist that would make my task easier?
You can use special transformation tools/languages for that TXL or Stratego/XT.
Also you can have a look and easily try Java to Python and Java to Tcl migrating projects made by me with TXL.
You are right about mapping library concepts. It is rather hard and long task. There are two ways here:
Fully migrate the class library from language A to B
Migrate classes/functions from language A to the corresponding concepts in language B
The approach you will choose depends on your goals and time/resources available. Also in many cases you wont be doing a general A->B migration which will cover all possible cases, you will need just to convert some project/library/etc. so you will see in your particular cases what is better to do with classes/libraries.
I think this is almost impossibly hard, especially as a personal project. But if you are going to do it, don't make life even more difficult for yourself by trying to come up with a general solution. Choose two specific real-life programming languages ind investigate the possibities of converting between them. I think you will be shocked by the number of problems and issues this will expose.
There are some tools for direct migration for some combinations of A and B.
There are a variety of reverse engineering and code generation tools for different languages and platforms. It's fairly rare to see reverse engineering tools which capture all the semantics of the source language, and the semantics of UML are not well defined ( since it's designed to map to different implementation languages, it itself doesn't define a complete execution model for its behavioural representations ), so you're unlikely to be able to reverse engineer and generate code between tools. You may find one tool that does full reverse engineering and full code generation for your A and B languages, and so may be able to get somewhere.
In general you don't use the same idioms on different platforms, so you're more likely to get something which emulates A code on B rather than something which corresponds to a native B solution.
If you want to use Java as the source language(that language you try to convert) than you might use Checkstyle AST(its used to write Rules). It gives you tree structure with every operation done in the source code. This will be much more easier than writing your own paser or using regex.
You can run com.puppycrawl.tools.checkstyle.gui.Main from checkstyle-4.4.jar to launch Swing GUI that parse Java Source Code.
Based on your comment
I'm not sure yet, but I think the source language/framework would be Java/Swing and the target some RIA language like Flex or a Javascript/Ajax framework. – Alain Michel 3 hours ago
Google Web Toolkit might be worth a look.
See this answer: What kinds of patterns could I enforce on the code to make it easier to translate to another programming language?
I'm working on a project which will do some complicated analyzing on some user-supplied input. There will be 3 parts of the code:
1) Input supplied by user, such as keywords
2) Rules, such as if keyword 1 is repeated 3 times in keyword 5, do this, etc.
3) And the analyzing itself which executes the rules and processes the user input, and generates the output necessary based on the processing.
Naturally this will lead to a lot of spaghetti code and many, many if statements in the processing code. I want to avoid that, and keep the rules (i.e. the if statements) separately from the code which loops through the user input and generates the output.
How can I do that, i.e. what is the best way?
If you have enough rules that you want to externalize, you could try using a business rules engines, like Drools in Java.
A business rules engine is a software system that executes one or more business rules in a runtime production environment. The rules might come from legal regulation ("An employee can be fired for any reason or no reason but not for an illegal reason"), company policy ("All customers that spend more than $100 at one time will receive a 10% discount"), or other sources. (Wikipedia)
It could be a little bit overhead depending of what you're trying to do. In my company we're using such kind of tools for our quality analysis tool.
Store it in XML. Easy to parse and update.
I had designed a code generator, which can be controllable from a xml file.
For each command I had a entry in the xml. I was processing the node to generate the opcode for that command. Node itself contains the actions I need to do for getting the opcode. For some commands I had to look into database, all those things I had put in this xml file.
Well, i doubt that it is necessary to have hughe if statements if polymorphism is applied correctly.
Actually, you need a proper domain model for your rules. This goes somehow into the direction of the command pattern, depending on the complexitiy of your code maybe in combination with the state machine pattern.
Once you have your model, defining rules is instantiate them correctly.
This could be done by having an xml definition, which is parsed and transformed into your model. But the new modern and even more fancy way would be using DSLs. If you program in Java and have a certain freedom about your libraries, this would be a proper use case for Embedded DSLs with Groovy. Basically you would need a Builder which constructs your model, that's all.
You always can implement factory that will create certain strategies according to passed parameters. And then you will use those strategies in your code without any if.
If it's just detecting keywords, a finite state machine or similar. If it's doing more, then other pattern matching systems, such as rules engines.
Adding an embedded scripting language to your application might help. The rules would then be expressed in scripts, executed by the applications on processing.
The idea is that scripts are easy to change and contain high level logic that will be executed by your application in details.
There are a lot of scripting languages available to do this : lua, Python, Falcon, squirrel, angelscript, etc.
Have a look at rule engines!
The approach from Lars may also be arguable.