Compiling VB (and C#) projects

[NickThissen]

Hey,

I am trying to compile a VB or C# project manually, so that I can use Reflection on the compiled assembly to figure out the types in the assembly and their members.

I am running into a whole load of trouble however, and I cannot find much information about this. Perhaps I haven't looked hard enough, but all I can find is some basic compiling information, like how to compile a mathematical expression to be evaluated dynamically. I am not dealing with 'code snippets' like this, I am dealing with real projects, containing several source files, references, and resources.


At the moment, I am reading a project file (either VB Project or C# Project as their format seems to be the same) using XML and storing the information in a Project class I've created. The Project class isn't very interesting, all you need to know for now is that it has a bunch of properties that I need. I started with these:
- Files: the paths to the source files included in this project
- References: the references (dll name) in this project

In a UserControl (which is going to be used to display the types and their members, in a TreeView), I have a method that accepts a Project instance and then compiles this project. This compilation returns an array of Types which I use to populate the TreeView:

Public Sub ParseProject(ByVal prj As Project)
            Me.TreeView.Nodes.Clear()
 
            ' Get the source code from the paths
            Dim sources As New List(Of String)
            For Each file As String In prj.Files
                sources.Add(System.IO.File.ReadAllText(file))
            Next
          
            ' Compile the project (code later)
            Dim types = CodeParsingLibrary.CodeParser.GetTypes(prj.Language, sources, prj.References)
            If types IsNot Nothing Then
                ' Add types to treeview (code omitted)
            End If
        End Sub

The compilation is done using a few shared methods in the CodeParser class:

Imports System.CodeDom.Compiler
Imports System.Reflection
Imports System.Text
 
Public Class CodeParser
 
    Public Shared Function GetTypes(ByVal language As Languages, _
                                    ByVal sources As IEnumerable(Of String), _
                                    ByVal references As IEnumerable(Of String)) As Type()
 
        ' Compile the source files
        Dim asm As Assembly = CodeParser.CompileSource(language, sources, references)
        If asm IsNot Nothing Then
            Return asm.GetTypes
        Else
            Return Nothing
        End If
    End Function
 
    ' Gets the CodeDomProvider from the language (VB or CSharp for now)
    Private Shared Function GetCodeDomProvider(ByVal language As Languages) As CodeDomProvider
        Dim languageString = String.Empty
        If language = Languages.CSharp Then
            languageString = "CSharp"
        ElseIf language = Languages.VB Then
            languageString = "VB"
        End If
 
        If languageString = String.Empty OrElse Not CodeDomProvider.IsDefinedLanguage(languageString) Then
            Throw New ArgumentException("Provided language is invalid.", "language")
        End If
 
        Dim provider = CodeDomProvider.CreateProvider(languageString)
        Return provider
    End Function
 
    Private Shared Function CompileSource(ByVal language As Languages, _
                                          ByVal sources As IEnumerable(Of String), _
                                          ByVal references As IEnumerable(Of String)) As Assembly
 
        ' Get the CodeDomProvider used to compile
        Dim provider = CodeParser.GetCodeDomProvider(language)
 
        ' Set up some parameters
        Dim params As New CompilerParameters()
        params.GenerateInMemory = True
        params.GenerateExecutable = False
 
        ' Add the referenced assemblies found in the project xml file
        params.ReferencedAssemblies.AddRange(references.ToArray())
 
        ' Finally compile the sources and return the compiled assembly
        Dim result = provider.CompileAssemblyFromSource(params, sources.ToArray())
        Return result.CompiledAssembly
    End Function
 
End Class

Here is where the trouble starts.

It only works correctly when I try to compile an extremely simple C# project, containing just a single class with some base types. This proofs that the concept works.

However, when I try to compile a VB project, I run into trouble. I am getting all kinds of errors returned, three which seem common:

1. If the project uses multiple files, they can never seem to 'see' each other. If Class1 uses an instance of Class2 then compilation tells me that Class2 is not declared. I think this is easy to fix: Class2 cannot compile due to points 2 and 3 below, hence Class1 cannot find Class2 either.

2. I am getting errors like the fact that 'My.Settings' is not defined:

Type 'HatchBrushTest.My.MySettings' is not defined.

Also, I keep seeing the '<Default>' namespace returning, in errors such as:

'HatchBrushTest' is not a member of '<Default>'.

(HatchBrushTest is the project name). I've no idea what causes this and how to fix it...

3. I am getting errors that types such as 'Color' and 'Rectangle' and 'Control' are not declared. This seemed strange to me at first: I had already added 'System.Drawing.dll' and 'System.Windows.Forms.dll' to the referenced assemblies..? But then it occurred to me that this is not enough: you also need to use the Imports statement for these namespaces. Since VB can automatically ("implicitly"?) import some namespaces for you (System.Drawing is one of them by default), they do not appear in the source code files, and hence the types aren't found.

For point 3, I had another look in a VB project file, and luckily the file does list these 'invisible' imports. So, I extended my Project class to extract the imports statements as well.

But now I'm "stuck". I now have a list of source files, and a list of imports statements to add to them. At first I thought that wasn't too hard: just add the imports statement to the front of every source file manually. So I passed the imported namespace around to the compile method and did this:

Private Shared Function CompileSource(ByVal language As Languages, _
                                          ByVal sources As IEnumerable(Of String), _
                                          ByVal references As IEnumerable(Of String), _
                                          ByVal [imports] As IEnumerable(Of String)) As Assembly
 
        ' Get the CodeDomProvider used to compile
        Dim provider = CodeParser.GetCodeDomProvider(language)
 
        ' Set up some parameters
        Dim params As New CompilerParameters()
        params.GenerateInMemory = True
        params.GenerateExecutable = False
 
        ' Add the referenced assemblies found in the project xml file
        params.ReferencedAssemblies.AddRange(references.ToArray())
 
        ' Add the imports statements to the front of every source file
        Dim sourceFiles As List(Of String)
        sourceFiles = New List(Of String)
        For Each source As String In sources
            Dim sb As New StringBuilder()
            For Each imp As String In [imports]
                sb.AppendLine("Imports " & imp)
            Next
            sb.AppendLine()
            sb.Append(source)
            sourceFiles.Add(sb.ToString())
        Next
 
        Dim result = provider.CompileAssemblyFromSource(params, sourceFiles.ToArray())
        Return result.CompiledAssembly
    End Function

But that gives me more problems:
- I can't always add them to the top of the file, because there might be 'Option' statements that need to precede the Imports statements.
- I can't add all of them 'blindly', I have to check if they are imported already, because the compilation fails if there is a duplicate imports statement.

So I would have to do quite some string manipulation (or regex possibly) to first figure out which imports are there already, and where I need to put them. That doesn't seem like a very good approach... But I can't find any other way. There doesn't seem to be a way to set the implicit imports via the CompilerParameters or something (like I can set the references)...


Should I really use this cumbersome approach, is it the only way?

Also, am I correct about point 1 resolving itself if points 2 and 3 are resolved, and how can I solve point 2?

Finally, if anyone knows a good 'tutorial' or resource on this subject feel free to post it. I've looked around MSDN obviously but apart from some very simple examples I found nothing to help me.


Thanks!