-
Jun 6th, 2014, 10:41 AM
#1
Lexical Analyzer
For those of you who do not know, I've been trying to create my own compiler. The first step in this is the lexical analyzer. Basically what a lexical analyzer does is separate everything in a source code and converts certain words/characters into tokens. The way that my lexical analyzer works in this GUI program is that it stores the tokens in an XML file in this format:
Code:
<source>
<token>
<name>token class name here</name>
<value>token value here</value>
</token>
....
....
</source>
Here is an image of the program in action:
The scanner scans in order, so some important ordering in the case above is that I have my number class above my decimal class. I also have the identifier very last in the order. The reason for the number before the decimal is because any decimal(in my case) can match a number, but not every number can match a decimal. The reason why I have the identifier last is because almost everything(in my case) can be interpreted as an identifier.
Here is the program(minus the binaries):
lexical_analyzer.zip
Here is the template that I use for my language in case you don't want to create the token classes:
custom template.xml
Now this program is not practical in creating a custom compiler, it just simply shows a technique of a scanner.
-
Jun 7th, 2014, 02:33 AM
#2
Re: Lexical Analyzer
Keep it up, very much interested to see this to completion.
-
Oct 31st, 2014, 02:52 AM
#3
-
Oct 31st, 2014, 08:57 AM
#4
Re: Lexical Analyzer
It's funny that you mention something, because I've recently picked it back up about two/three days ago.
-
Nov 1st, 2014, 12:13 AM
#5
Re: Lexical Analyzer
Actually, even though I have downloaded you example I don't know how to use it, could you provide a sample code that it can 'parse'?
-
Nov 3rd, 2014, 11:00 AM
#6
Re: Lexical Analyzer
Sorry for the late reply, busy weekend.
could you provide a sample code that it can 'parse'?
This example is just of the lexical analyzer or lexer. The syntax analyzer or parser is a separate component of a compiler, but it parses the output of the lexical analyzer. I did have a working parser, but I was unhappy with it so I scrapped it. Now I'm working on a new one.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|