How Is Syntax Highlighting Built? - syntax

I'm now building a developers editor using Lazarus and as every good developers editor, it needs to have a syntax highlighting, because of this that I want to know some things:
How is syntax highlight normally built(using many ifs to change the font on a TextField?
TSynEdit is very nice to do this?
How to use TSynEdit(links and resources)?

You can actually see an excellent example for yourself if you look at the editor for SharpDevelop. It's implemented in C# but uses great OOP that should be readily portable to other platforms or languages.
Most home-brewed syntax highlighters tend to use the MS Windows RichText control but they're usually slow and clunky and really slow down when a lot of code is added.

You could read the source code of Eclipse or VIM. In Eclipse you will find an Abstract Syntax Tree base. On top of it are parsers that are carefully written to be tolerant of errors, since code in progress often does not satisfy the full grammar of the language. The highlighting uses the information in the syntax tree.

Related

Is it possible to apply refactoring using OmniPascal?

I've installed OmniPascal extension in Visual Studio Code to use it for coding in Delphi and I am loving it. Code completion, parameters hint, go to declarations, all working 100%! Nonetheless, I am missing the refactor commands like rename (CTRL+SHIFT+E). Is that already implemented in OmniPascal?
There are currently no refactoring tools implemented in OmniPascal. Rename and Extract method are planned for the future but there are other features that need to be implemented before OmniPascal can turn into a reliable refactoring tool.
As long as OmniPascal doesn't handle all language features correctly (like with-blocks or class helpers) the rename command would produce wrong results. You don't want to work with such a refactoring tool.

Advanced code style in Clion

I want to have google-style like code style checker that would automatically run within Clion.
However, what I found as solutions (predeclared code style for google and others, direct Editor settings and EditorConfig support in Clion help) are all rather primitive. For example, I want to use snake case with final underscore for class member fileds (e.g. my_class_member_) and usual snake case for function arguments (e.g. some_argument), and none of the suggested options would do the trick as far as I am concerned. Furthermore, some politics associated with endless loops and so are to be added, which is even more context-specific.
I consider creating cpplint.py-like script for this, but it is going to be very time-consuming and is likely to be run outside Clion. Are there any elegant ways to solve my problem?
Yeah, you able to do this! Look into Clion plug-in Clion-cpplint and use with cpplint.py script, provided by Google. You will get highlights on the fly when you are editing C++ source code.
You able to install add-on through Plugins tab in settings. In the end you will get something like:

Syntax-Highlighting for lex and yacc in Visual-Studio

I want to add support for syntax highlighting for lex and yacc-files in Visual Studio 2010.
How can I do this?
Following the given link to Syntax Coloring leads to another, more pertinent page Implementing Syntax Coloring, where it is noted
Visual Studio does not specify a parser interface, and parser implementation is completely up to you. However, a default parser implementation is provided in the Visual Studio Language Package project. For managed code, the managed package framework (MPF) provides complete support for colorizing text.
Depending on what you want:
trivial, coloring only the C code portions of the lex/yacc files
much harder, coloring the patterns so that one can make sense of them
you could in principle make a parser using just lex (yacc isn't necessary). For yacc files, that's not so hard, but for lex there's the complication of regular expressions. vi-like-emacs does that and while the interface differs in detail, conceptually it's similar. Reading the lexers might give you some ideas how to apply that approach:
lex syntax filter
yacc syntax filter
There is documentation from Microsoft on integrating yacc/bison & flex/lex with Visual Studio. Although a URL only link is discouraged on SO, its best to go to the source for this kind of detailed information:
https://msdn.microsoft.com/en-us/library/aa730877(VS.80).aspx#vccustombr_topic3
It lays out all the steps necessary for integrating with the build tools. Syntax highlighting is covered elsewhere as described in Add a new language to Visual Studio 2010 with syntax highlighting and intellisense. In particular the guidance for syntax colouring can be found here: https://msdn.microsoft.com/en-us/library/bb166778(v=vs.100).aspx.
However, I'm not aware of anyone who has published extra colouring rules for the grammar specific components of flex and yacc. However, most of the body of flex and yacc files are written in C or C++ for which there are syntax highlighting rules that can be applied, and suit most peoples needs.

CoffeeScript Intellisense

I use Visual Studio for development and I am quite used to Intellisense. But when writting CoffeeScript you don't really get any Syntax Checking or Intellisense.
Is there a plugin for VS that would allow this?
Thanks
You can't have more than syntax checking/coloring with coffeescript (on any IDE) AFAIK.
As an alternative, you can use TypeScript to get the full Visual Studio tooling support (and stay close to the javascript), or some transcompilers that transform code to javascript, for C# there is Saltarelle and for CIL (.NET bytecode, so compatible with any .NET language), I just found JSIL but I have no idea how well it works.
Note than the generated code "look" can be important for debugging and using external libraries get a bit of work to be included in Typed languages.
For js code readability I would recommand TypeScript (similar to coffeescript, even easier, but less powerfull as a language).
Saltarelle code looks readable (didn't tried a lot), for JSIL I have no idea.
A list of languages that compiles to JS: https://github.com/jashkenas/coffee-script/wiki/List-of-languages-that-compile-to-JS
Webstorm and all of the JetBrains IDEs recently added some code completion and refactoring support. It knows what methods are in my own classes and will prompt to complete as I type. The red squiggly underline compile as you type is not quite there yet but it is coming. I bet Webstorm is the first to implement CS source map also. They have added so much in the last 11 months.
Check this out: http://visualstudiogallery.msdn.microsoft.com/2b96d16a-c986-4501-8f97-8008f9db141a
Here is another: http://chirpy.codeplex.com/
I think this should do what you want.

How does intellisense work in Visual Studio?

I hope this is a valid question: how does intellisense work in VS2008? I'm after what is known about the algorithm it uses to find the suggestions, when exactly it pops up (the "." is just one obvious trigger), how its behavior can be modified if at all possible, etc.
To put this question into context: The main issue I'm trying to resolve is how to activate and deactivate intellisense in portions of the editor screen and how to modify where it searches to populate the suggestion box.
All information is welcome.
Take a look at this DIY Intellisense article on CodeProject.
It's more fun to reverse-engineer it, though. Let's consider the problem:
you need to identify the words of interest
you need to find the options possible
you need to present them
Now, the first step means you have to parse the code. You've got the C/C** keywords, you pre-parse the various function and class declarations, and load them into some kind of data structure. Then you parse the code and store the class, variable, etc names and put them in the same data structure.
The second step means you want a data structure which efficiently can search for a partial word and get all the words that have that prefix. You can do that with regular expressions, but that's not very efficient. An efficient data structure for that kind of search is a trie, which is discussed here on SO .
Once you have the list of possibilities, you just present it. You probably want to keep a reference to the root of the tree of possibilities so you can search them out in real time as someone types more letters.
Eclipse also has this feature and it is an open source project. Why not check out how Eclipse does it by actually looking at the code?
This question is too broad. Since there are a number of different languages the VS IDE supports out of the box AND there are N number of DSL and IDE enhancements that support alternative intellisense this implies a number of answers. If you are speaking about C# specifically then See the Tools | Options | Text Editor | C# | Intellisense area to see the available options of completion options. As far as the algorithm[s] used, you would be looking for the metadata of assemblies, copious caching of type members, MRU list for last member chosen for specific type, etc. If you have a more specific question, I'd suggest you clarify.
See the example of a DSL (ironpython) and its implementation here.
I haven't seen any text editor in VS that limits where IntelliSense shows up. It's all language specific. If your cursor is located at a point where IntelliSense might contribute to a valid token, that's when it will be used.
I believe there is some interaction with the project system being used, but that's as far as I know. I also believe there is a sample project system in the Visual Studio SDK, and that might give you an idea.
For such cases I sometimes use my own version of InteliSense that I developed for AutoHotKey when I want specific behavior. The point of this script is that it can be used with any editor, or basically any control accepting text. It works by recording text input and interpreting it upon syntax file.
You can perhaps use it as a base for the thing you want to achieve. I used ISense succesifully with several languages that don't have such thing, like Csound or even batch scripts. It will be possible to extend it to support C# using input monitoring in combination with Reflection.
Anyway, with AHK you can even control VS intelissense by "taking" the list of items it presents and filter it, or similar things. You may have some small problems with process boundaries but nothing that cant be fixed.
The intellisense ius generally, AFAIK, implemented using different methods. I read that Delphi is so fast that it implements isense by recompiling the project on each token and thats the reason C++ Builder didn't have isense as its compiling very slow.
As to your how to change where it looks question, short answer is, you can't. Intellisense for the most part is provided by reflection of assemblies included in your project ( and some other tricks with C++ ). What you are getting is a result of VS processing through all the assemblies you have included and all assemblies from the GAC.
That said, if you want to provide explicit intellisense results from a project you are working on, look into IVsContextualIntellisenseFilterProvider
Finally, for some insight into the behind the scenes process, check this blog post

Resources