How do I create my own compiler?

[ChuckB]

Hi,
I am interested in learning more about creating my own compiler. I can do the following.

1. I can parse a test file that contains commands, keywords or script.
2. I can convert these parsed lines to assembler code. Meaning, I can take an expression such as CLS or clear(); and convert it to something like MOV AX,600.....etc.
3. I can take each of these lines in assembler and build the appropriate machine code expressions and dump them into a binary file with a COM extension.

I learned most of this with DEBUG and QBASIC.

So, am I on the right track to creating a simple compiler (I have no ambition at creating a monster)? Are there online resources that would guide me to create a compiler in VB or C++?

My interest is in taking an AI program and allowing the program to modify itself by adding executable code...You can read about our AI efforts and discussion in the Games and Graphics forum.

Any info would be appreciated.

Regards,
ChuckB

[jian2587]

Anyone with XPerience in QB and ASM would know how to do
that...
However, do u know how to parse maths operations?
This process involves converting infix to postfix using Stack data
structure...

FYI, I am also creating a compiler now, it can run without MSDOS,
so u can create an OS out of it. I want to make a new language,
which u can create tight and fast machine codes with high level
language

Creating a new language involves lots of steps, these are a few
to be mentioned:
1. Define ur language's syntax rules
2. The platform ur language is going to function
3. Fundamental statements, such as IF, SWITCH, FOR, WHILE
4. Writing the compiler
- Statement parsing
- Nested structure parsing
- Maths expression conversion
- Etc,Etc
5. Turning them into machine codes

I recommend u to read C HOW TO PROGRAM second edition

This is a COMPLETE guide of both C and C++ and numerous
projects which also includes creating ur own compiler

[si_the_geek]

It's also worth checking out Yacc and Bison - they are well know compiler generators

I'm sure there will be lots of tutorials/downloads around for them both

[ChuckB]

Hi,
I feel comfortable parsing statements. Nesting and math operations are a bit puzzling. However, I understand the need for a stack if I will be nesting. I have parsed a very simple math expression before such as 7=3*x...nothing more difficult. Not sure about prefix..postfix. Are prefix and postfix associated with the fixed point for floating point calculations...or something like that?

What platform are you developing a compiler for? I am interested in the standard PC...8088 instruction set (I am not really interested in taking advantage of the Pentium processing features). The goal is to write a fairly simple compiler so I can say I have done it...and then I want to teach the technique to my sons.

Now, you intrigued me by saying your compiler does not require MSDOS. I take that to mean that you do not utilize any of the DOS interrupts...but you simply based everything upon the mechanics of the CPU. Is that correct?

I stumbled upon a "How to Create a Compiler" by a fellow named Crenshaw. It was written from 1988 to 1995...using Turbo Pascal. Found that through Google.

Are you writing the compiler for school, work or general interest? It is a significant undertaking.

Thanks for responding. Incidently my wife's name is Grace...how did you know she was cute? She says thank you. :-)

Regards,
ChuckB

[jian2587]

I can parse things like this:

Abc = ((6 + 2) * 5 - 8 / 4) * Cba - 6 * 3

Yes, my compiler use the CPU's instruction sets only, no MSDOS
interrupts, however, you can also use it to write MSDOS programs
Both COM and EXE, and pure BINARY as well.

Now just in the beginning of it, I am doing this compiler just 4 my
interest...

BTW, C HOW TO PROGRAM is really good, it teaches u
how to write compiler...

Thanx, Grace's my girlfriend...

[ChuckB]

Hi,
I've read 10 chapters this week from my assembler tutorial I downloaded a couple of years ago. I downloaded NASM and need to figure out switching to assemble little program.

The book you suggest I have written down. I did find another online tutorial about writing a Compiler....still a bit over my head.

After reading dozens of sites, all of this terminology is starting to sink in....

Your math expression is complex to me...not sure if I could do it even in C++/VB very easily. How big is your code.

BTW, I'll tell my wife Grace you were referring to her afterall.. ;-)

Regards,
ChuckB

[jian2587]

Grace's the one in my avatar, beautiful,huh?

I've done the math expression to asm converter in QuickBasic,
and it does its job pretty good, however, recently, I found a bug,
now rewriting it...

Parsing a math expression takes lots of operations, I wouldn't be
surprised if the code is fairly large, however, if properly optimised,
it can be lightning fast.

I've seen ur post in ...Writing OS..., just 2 tell u some good
news. If my compiler is ready, you can do that kind of OS that u've
requested in that thread easily with my compiler...

Somethin' like this:

INCLUDE "standard.si"
FUNCTION VOID MAIN ()
CHAR STATEMENT(255) = 0
DO
PRINT ("COMMAND:")
STATEMENT = INPUT ()
SWITCH (LEFTSTR(STATEMENT, INSTR(1,STATEMENT," "))
CASE "ECHO"
//Process it//
EXITSWITCH
CASE ELSE
//Process it
ENDSWITCH
LOOP
ENDFUNCTION

[abdul]

ChuckB, which Assembly tutorial did you read those "10 chapters" from? I am reading Art of Assembly right now it's too long so I guess it'll take some time.
I tried to parse a math function before but gave up after using a recursive function to get all the expresions in between each "(" and ")" in a list. I guess I'll try it again sometime...it was in C++, BTW.

[ChuckB]

Hi Abdul,
I read http://www.datainstitute.com/debug1.htm It is written by Fran Golden and has 11 chapters. I used it to get into DEBUG to write some basic programs. From there, I found that the other assembler tutorials made much better sense.

Regards,
ChuckB

[abdul]

Thanks Chuck, I'll check it out.

[Zaei]

Art of Assembly is a textbook =).

You might want to check out Compilers - Principles, Techniques and Tools by Aho, Sethi and Ullman (ISBN 0-201-10194-7). I have taken a look through it at the book store at one point, and it seems to be quite in depth. The only problem is that it is a bit expensive... Like 70 bucks on Amazon.

Z.

[abdul]

Originally posted by Zaei
Art of Assembly is a textbook =).

Yep, I should have mentioned that. I don't think there is any tutorial which is like 1500 pages long.

[Zaei]

Though it is incredibly complete =). For just straight assembly programming, all you really need is the list of opcodes (chapter 6, or 9, if I remember correctly), and perhaps the list of interrupt codes in one of the appendicies. The rest is just really good stuff to know...

Z.

[pskyboy]

I love this

I thought i was one of the very few who wanted to create my own langauge and i come on here and find people trying to the same this is great.

Anyway im gonna bore you all by describing my two langauges im developing so that i can get some feedback because all my friends are not advanced enough computer wise to give me much input or they get bored of me talking bout it.


Anyway i have 2 langauges in the works codenamed Orion and Pegasus

Orion is a Low Level langauge and leads to direct translation just like assembler but allows you to use mathmatical syntax. For instance to assign a value to axxumulator

in asm:-
MOV EAX,0x56

in Orion
AX = 0x56

etc it just makes it a little bit easier and so people with a background in algebra will find it easier to pick up the syntax in all other terms it will be identical to assembler.



The second languge im working on is Pegasus this is a much bigger task and requires a complicated IDE in order to get it to fully work well thats the idea. You will be able to write it in notepad but it will be much easier in the IDE.

I came up with teh idea for pegasus because of the large amount of visual interface required nowdays which means a proggrammer spends more time getting thsi working then making teh actuall program even with MFC etc.

So pegasus is ment to allow for a more visual aspect of proggraming while not sacraficing the power, flexibility, speed and what i call mid level access that is available in something like pascal or c.


In Pegasus to define a memory block like a byte you would do

memblock(1) Onebyte = 8;

What do you think. Pegasus will also allow for things i call advanced structures and arrays. Thsi will mean that rather then having to specify the size of an array you can specify an end and it will size the array accordingly.


Any feedback will be greatly appreciated as im still trying to fully scope out the language.

Cheers Peter

[snakeeyes1000]

very cool. I am in the process of making a pascal like langauge that, like yours, people with algebra will be able to make but retian the power and speed of ASM(or at least most of it). It will in actuallity be a simplified ASM, here is a sample of some code.

<hello world - this is a comment>

begin var CHAR;
     string1 = "Hello, World!",13d,10d,"Press any key to continue:",00d
end var CHAR;

begin rettype void;
     begin function Main();
          StdOut(&string1);
          WaitForCharInput();
     end function Main();
end rettype void;

[ChuckB]

Writing your own languages? I think that is pretty neat. Especially when you have a target audience such as math students who want to avoid the complexity of programming too many details.

Have you heard of DarkBasic? I believe this basic-like language was written in C++ and simplified the complexities of working with C++ and DirectX graphic/audio functionality by allowing the user to create pretty decent apps quickly. I played with the demo and think this is a great option for basic programmers who want to program neat stuff without spending a year learning C++ and DirectX.

Here is my programming language concept...use basic like keywords, create a program...and then translate to C/C++ and compile/link with mingw32 or some other compiler. Programmer selects IDE to write in format of basic or C++ or both. Add in simplified assember stuff like you guys mention AX=0x56 and voila!

There is a standard called IEC 61131 (I think) that allows developers to build machine control software in one of five languages...or a combination of them. Now this is really a cool idea. It combines graphic approach (ladder diagrams,sequence diagrams and functional block diagrams) with structured text (looks like basic/c) and instruction list (looks like assembler).

Have you thought about having different windows that allow 2 or more language formats to be used..that compile into a single EXE?

Keep it up. :-)
Regards,
ChuckB