Reusing a very specific project for multiple solutions

[NickThissen]

Hi,


First of all, I wasn't sure whether this was VB.NET or General Developer content... It's not technically about VB.NET code, but it's not very general either since it is specifically about Visual Studio. Feel free to move it if required.


A while back I realized that all of my database connection code was getting very biolerplate. I had 'developed' my own method of connecting to an Access database and storing the results. Basically, for each table I create one 'Manager' class and one 'Entity' class. The Entity class represents a single record in the database and has properties for each field. The Manager class has methods to save and load instances of this Entity class.

Writing this code became tedious quickly though, and when I eventually found out about T4 Text Templating in Visual Studio, I decided to go with that. T4 is basically a very simple way to generate source files automatically, on the fly during design-time. You can view the form designer as a special case of the T4 files; you edit something visually (drag controls on a form) and the source code for that is generated behind the scenes (in the Form.Designer.vb file).
T4 works very much the same, except in my case there is no 'designer' but rather a simple XML file that I write. The XML file contains an enumeration of the tables and their fields (and types), as well as relations between tables. The T4 file reads this XML file and generates all the Manager and Entity classes for me.

All I have to do now is write the XML file (this takes seconds) and hit 'generate', and I'm done.

Another important detail is the fact that all generated classes are partial classes (just like the Designer classes for the form designer). This allows me to extend the generated classes with more methods and properties. I cannot put that code in the generated class file because it will be overwritten the next time I change the XML file.

Suppose I generate a file PersonEntity (from a People table) which has a Birthdate property (database field). You can imagine that having an Age property on the PersonEntity is useful, so I can do that by extending the Person class: I add another PersonEntity file and declare the class Partial, then add my method:

Public Partial Class PersonEntity
   Public ReadOnly Property Age As Integer
      Get
         Return DateTime.Now.Years - Me.Birthdate.Years
      End Get
   End Property
End Class

This, as well as the fact that I generate my source code in the designer, now poses a major problem however:
I cannot create a class library (DLL) for this project!
A DLL contains 'fixed' classes that don't change; my project contains T4 files that generate source code on the fly. For every project that uses a database, it will contain different classes.

Furthermore, the extending Partial classes MUST be in the same project (a requirement for partial classes, unfortunately), so that's another reason I cannot compile it into a DLL and just reference it (there would be no way to add new (partial) classes to it).


Why is this a problem? Well... I cannot easily reuse this project in different solutions.

If I want to use this project for a solution with a database about my pets, and also for a solution about my work hours, I have to make a copy of the entire project and 'carry it around' to the new solution.

The most obvious thing one would do in such a scenario, usually, is just keep a single database generating project and use 'Add Existing Project' to the two solutions. The two solutions would then use the same project, and when I make a change from one solution, it also applies to the other.

This is usually desired (I am still making changes to the generating code constantly, it's not finished at all and probably never will be), but in my case I do not want this at all. In one solution the XML file defines completely different tables than in the other solution, and completely different classes will be generated. Furthermore, extending partial classes in one solution won't need to be in the other solution.




So, as far as I can see I must make a copy of the database generating project every single time I start a new solution. This now poses a new problem, namely the fact that I'm still changing it. Each time I discover an error or mistake or an improvement to make, I apply a fix to the project in the solution I happen to be working on at that time. However, the (possibly) 38 other copies of this project won't get that fix, and over time I will (and have...) end up with 38 different copies of the database generating project, some very old, some up to date, some containing fixes that others do not, etc... It's turning into a real mess!


Does anyone know what I can do about this?

One possible solution, that still has the last problem kind of, is to create a new Project Template for it. I already tried that, I can add the XML file, the T4 generating files and all, and when I create a new 'Database Generator Project' (which is what I named it), I get a fresh copy of my database generating code project.

All I need to do now is 'compile' any changes I make back into the Project Template, so that the Project Template always has the most up to date version. But that is basically the same problem as I had before, and it's not really feasible to keep updating the Project Template because I have to 'clean' the project first then (remove the extending partial classes, clean out the XML file, etc) and then my code is lost...

So I'm still really looking for a solution to this problem.


I know, very long post, but I hope some of you at least get the gist of what I'm trying to say... Thanks!!

[NickThissen]

Bump

[techgnome]

Never heard of T4 until... I can see this as a very handy tool. I'm assuming you're using the Design-Time version of the templates, yes? So I'm not sure why you can't compile your dll... unless for some reason you're including the template file in the dll output (which you shouldn't be doing). The resulting generated code should be compilable.

I poked around MSDN a little reading up on it... so I'm not sure if this will help or not.
http://msdn.microsoft.com/en-us/library/ee847423.aspx
On second thought, no it won't... that's about using the automated build process...

What about this?
http://msdn.microsoft.com/en-us/library/dd820614.aspx

-tg

[NickThissen]

See, the problem is that the database code files are generated during design-time.

Say I am developing an application that keeps track of contacts. I'll create a database (Access, SQL Server or SQLite are supported so far) with a table Contacts and fields like the FirstName, LastName, Birthdate, etc.

Then I'll write my XML file to reflect these tables:

<Database Type="SQLite" Language="VB">
   <CustomConnectionString>Datasource=&#37;PROJECT%\db.s3db;</CustomConnectionString>

   <Table Name="Contacts" EntityName="ContactEntity">
      <Field Name="Firstname" Type="String" />
      <Field Name="Lastname" Type="String" />
      <Field Name="Birtdate" Type="DateTime" />
   </Table>
</Database>

I hit 'Run custom tool' on the T4 file (or save it), and the generated code now includes:

Public Partial Class ContactEntity
    Inherits Entity
    
    Public Property Id As Integer
    Public Property Firstname As String
    Public Property Lastname As String
    Public Property Birthdate As DateTime

End Class

Public Partial Class ContactEntityManager
    Inherits EntityManager(Of ContactEntity)
 
    Public Function Load() As EntityCollection(Of ContactEntity)
        Dim sql As String = "select * from Contacts"  'also auto-generated
        Return Me.LoadSql(sql)
    End Function

    Public Sub Save(contact As ContactEntity)
        '...
    End Sub
End Class

(The actual implementation is much more detailed, each property for example calls partial methods when changed and raises the INotifyPropertyChanged.PropertyChanged event, but that's details)

And off we go. I can now just use methods from the ContactEntityManager (singleton class);

DataGridView1.DataSource = ContactEntityManager.Instance.Load()

Dim c As New ContactEntity()
c.Firstname = "Nick"
c.Lastname = "Thissen"
c.Birthdate = DateTime.Now.AddYears(-22)
ContactEntityManager.Instance.Save(c)

The code in the Entity and Manager classes has all been generated like 5 minutes ago, it wasn't there when I started the project. For a different project, I don't need these entities and managers, I might need completely different ones (maybe about my pets, or about my work hours, I dunno), and they will be generated on the fly as well.

I can compile these classes into a DLL, of course, and that also happens obviously when I compile / run the application, but I cannot keep using that same DLL in the future (I might need to add tables, fields), and certainly not for other projects (I need other tables!).

So what I meant with being unable to create a class library (DLL) for it was that I have to have this project as an actual 'live project' in my solution each time. I cannot build it into a DLL and reference it from various solutions, and I cannot reuse the same project in different solutions. So, I have to make a copy of the project every single time I start a new solution, and when I make changes to the generating T4 code (I am still updating it to include stuff I didn't need before, like many-to-many relations just recently) that change will only be made to the project that I happen to be working on currently. The other copies of the project don't get that change (unless I hunt them all down and apply the change to all of them, and it's likely I'll be too lazy to do that or simply forget).

[techgnome]

" but I cannot keep using that same DLL in the future (I might need to add tables, fields), and certainly not for other projects (I need other tables!). ... [snip] ... So what I meant with being unable to create a class library (DLL) for it was that I have to have this project as an actual 'live project' in my solution each time. I cannot build it into a DLL and reference it from various solutions, and I cannot reuse the same project in different solutions. " -- I guess this is where you lose me...

-tg

[NickThissen]

Ok, here's a screenshot of one of my solutions that uses this project (DatabaseLibrary):

(C# example, but that's not the point)

As you can see there are two 'tt' files, these are the T4 Text Templating files. As children of those files, the generated classes (.cs files, would be .vb for VB) that contain the code.

The DatabaseLibrary project thus merely contains the information to generate the classes, nothing more, until I write the data.config (XML) file and let the tt files do their job. Only then are the actual database classes I'm going to use generated.

Just for completion, here's a screenshot of the contents of Entities.tt and Entities.cs side by side, so you can see how the generating works basically:
http://i55.tinypic.com/33cu974.jpg

If I build this project into a DLL and reference it (so that there is only one project in the solution, which references a DLL of DatabaseLibrary), then there is no way to generate the classes anymore; they are 'fixed' in the DLL. There is no way to change them.

Do you understand what I'm talking about now?

[techgnome]

Sure there is... based on everything else I'm reading... if you make a change, you'd re-run the template files generating new classes... re-compile... done.

I've used templating systems before (just not T4) and that's how they work... any time you make a change to the underlying data that the templates are based on, you have to re-run the template and re-generate your resulting classes.

-tg

[NickThissen]

Sure there is... based on everything else I'm reading... if you make a change, you'd re-run the template files generating new classes... re-compile... done.

I've used templating systems before (just not T4) and that's how they work... any time you make a change to the underlying data that the templates are based on, you have to re-run the template and re-generate your resulting classes.

-tg

How can I change the underlying data when it is built into a DLL? How can I re-run the template when it is built into a DLL?
I can't, that's the whole problem.

At the moment, the project is a 'live project', by which I mean it is actually a project in my solution that I can edit, as you can see from the screenshot. When I built it into a DLL and reference it, I cannot edit it anymore, and there is no way to change the underlying data or to re-run the templates.

That is exactly the problem. It works now because the templates and the data are in a project, separate from all other projects. For each solution with a different database, I have a separate copy of this DatabaseLibrary project, each with its own data.config file, and (therefore automatically) with its own generated entity classes.

I cannot use one project for two solutions; as soon as I add a table 'Pets' in the data.config file, the templates will generate classes for the Pets table in that project and they will be in both solutions, whereas the other solution doesn't have anything to do with pets as it's a completely different database for a completely different application.

The database library project is 'dynamic' in that it doesn't always have the same classes (because it can be used for any database application, that's the point).

Compare it to any ordinary Windows Forms project. Surely for every application you have a separate project? You are not going to use (and can't use) the same project for a business application and for a photo album of your pets. It simply doesn't work that way, forms you need in the business application shouldn't be in the pets application and vice versa; they are two separate projects, but both have the same goal: they 'generate' forms that deal with one specific application. But you need two projects, one with the business app forms and one with the pet album forms.

My database library is exactly the same, except it doesn't generate forms but database access classes.

[techgnome]

Ahhh... I think I'm catching on... you want your cake and eat it too... you want something that is a one-size fits all... something that doesn't exist, and yet exists.

Yeah, I don't think that's going to work.

"My database library is exactly the same, except it doesn't generate forms but database access classes." -- and yet, you're trying to put... wait now hold on a set... using your analogy ... I don't create forms from scratch either... I inherit... so what if your database lib was inheritable... then you would have a DBLib in each project that could inherit from your main db lib, and would also contain your templates and subsequently the generated files... would that not work?

-tg

[NickThissen]

I don't see how I could 'inherit' my database library. My entities already inherit a base Entity class, the rest is generated.

In the past I've tried to create this same library using such inheriting approach, where one simply inherits Entity and defines the properties manually; then each property would have an attribute that pointed to the database field it belonged too. It worked, but it used a lot of Reflection and was therefor very slow so I abandoned the approach when I found out I could generate the classes instead.