The documentation and general recommendations are that the abstract syntax tree should omit tokens that do not matter. ("Record significant input tokens (and only significant tokens" is the final ANTLR Reference) IE: in C ++ AST, you omit curly braces at the beginning and end of the class, since they have no meaning and are just a mechanism for determining the beginning and the end of the class for parsing I understand that for a quick and efficient walk in a tree, selecting useless token nodes like this is useful, but in order to color the code appropriately, I need this information, even if it doesn't help the code . A) is there a reason there why should I not use the AST for several purposes and do not miss these tokens?
It seems to me that the output of the ANTLRWorks interpreter is what I'm looking for. In the ANTLRWorks interpreter, it displays a tree diagram where a node is created for each agreed rule, as well as a child node for each token and / or sub steering. The parse tree, I suppose, is called.
If you walk manually in a tree, would it not be more useful to have nodes designating the rule? Having a node designating a rule, with its sub-controls and tokens as children, the manual walker does not need to look at several nodes to find out the context of the fact that it is a node. Grammar trees seem redundant to me. Given the AST node tree, the tree grammar “analyzes” the nodes again to get some other result. B) Considering that the parser grammar is responsible for the formation of properly formed ASTs and considering the inclusion of valid AST nodes, should the manual walker avoid excessive mapping of AST templates to the node grammar tree?
I am afraid that I am madly misunderstanding the purpose of the tree grammar mechanism. The grammar of a tree more or less determines the set of methods that will be executed through the tree, look for a pattern of nodes that matches the grammar rule of the tree, and perform some action on it. I can’t depend on shaping my AST output based on what is neat and tidy for tree grammar (omitting meaningless markers for pattern matching speed), but using AST for color coding even meaningless tokens. I am writing an IDE; I also can’t write any possible AST node pattern that the plugin author may need, and I don’t want them to use ANTLR to write the grammar of the tree. If the authors of the plugins go through the tree according to their criteria, the rule nodes will be very useful,to avoid the need for pattern matching.
? , "" SO, , .