Best way to store/retrieve text values for multilingual sites

[StrangerInBeijing]

The following is a few ideas for storing text for a multilingual site.

1) One version of the site for each language.
(not scaleable. hard to maintain.)
2) All text values in language tables in database
(many database hits will slow things down)
3) All text values into arrays in seperate files. Include file according to language and get value from array.
(hard to maintain....but works pretty well-this is what I'm doing now.)
4) Create xml file. Each node is value key, and value in each language. Use some php function to read value according to language from xml file.

I'm pretty much like the last one (xml file) as this will be the closest I can get to resource files in asp.net (one of the few things that's really cool about asp.net).

But I'm not sure what is the fastest, easiest best way to find specific values in xml files using php. Had a quick look at some sample code, and seems mostly one will loop through nodes? Is there some code that one can create to easily and quicly retrieve node values?

Other suggestions are welcome.

[RudiVisser]

On multi language sites I have always used what I think is #3.

I'd have a en.lang.php that would contain for example:

<?php
// English Language file for x
$lang['welcome'] = "Welcome";

And a nl.lang.php that would contain for example:

<?php
// English Language file for x
$lang['welcome'] = "Welkom";

That always seems to work just fine for me

[StrangerInBeijing]

Just for the heck of it, I created these two today (It's the start for a new version in php of my asp.net website)
This one use 2 versions (not sure what it's called). Separate English and Japanese pages (although now the text is still English).
This one uses values in an array in an included file.

Neither very fast from here, but then the server is in Kuala Lumpur, and the speed in and out of china can be unpredictable. Also running php on a win2k3 server and iis. But it's clear the second is a tad slower, even though now theres only about 12 values in the array. Worried how it will be when that get to 1000 entries

Still feel xml is way to go. Not that I know much, but this fit perfectly into what xml is made for in the first place.

[penagate]

I wouldn't use XML. It's slow, and since language tables do not require a hierarchical structure there's no need for it.

Use microtime() to benchmark code. Doing it from the client side is disastrously unpredictable.

[CornedBee]

The way I use on a large, multi-lingual project is a rather complex system.

1) There are templates that define the structure of the pages.
2) A new page is created by choosing a template and filling in the content.
3) The CMS then generates a .php file from the template by filling in the content. This process is called publishing.
4) In addition to the content of the page itself, navigation menus etc. are generated from the various pages in the system.
5) To get the page in another language, you must of course translate the content. You create a new page for this translated version, but you link it to the other one, saying that it's an alternate language version.
6) Very little text remains that is part of the template. The language-specific version of this text is chosen at runtime from resource files (very simple name-value pair files, or even CDB files).
7) If a user requests a resource in a language that doesn't exist, he gets an error page listing the languages that are available.

Don't fool yourself. Multiple languages are always hard to maintain; effort always scales linearly to the number of languages. Using a template system like I do minimizes the effort for structural changes, but in the long run, this effort is minimal compared to the effort of maintaining the content, and unless you find the output of machine translators acceptable, this always means writing the content once for each and every language you support.

[dclamp]

if you really wanted to, just create another site translating the whole page your self, and putting it in a subdomain and setting it in cookies to go to that subdomain when your site loades

ex:
en.yoursite.com (English)
sp.yoursite.com (Spanish)
...so on...

[StrangerInBeijing]

CornBee advice sounds interesting, even though I cannot think exactly how it's done (I'm a php noob, coming from asp.net where everything is done for you).

Is there any way you can extend on what you said, or/and if possible give some sample code?

Going with a subdomain for each language is defo a no-go in this case dclamp. If your goal is a small, but really fast site, one that basically just provide some information about the client, contact page, etc, yep, I go with that. But with a big site like the one I started to convert...huh uh.

I just started out, but this seems to work fine for now. Sounds like CornBee's way will require starting from scratch, but that's ok. I just started and dont mind starting over

[kows]

well, I believe what dclamp said was entirely anti-productive.

however, making a bunch of sub domains handle your languages is very nice looking (using query strings is ugly) -- and it works well. you just have to make all of the sub domains go to the same root directory, and in your index file you have to parse your URI to find out what language to use. it's quite simple, really.

<?php
  //$url will hold something like: http://en.example.tld/page.php
  $url = $_SERVER['REQUEST_URI'];
  list(, $host) = parse_url($url);
  //find the language by looking for the subdomain
  
  list($language) = explode(".", $host);
  echo $language; //should echo "en";
?>

or, you can just use make-believe directories and then use htaccess to redirect. also a nice way to make pretty URLs!

but, alas, this has nothing much to do with how you could implement your feature; but rather, how to do it with style.

[penagate]

Well, both language-specific subdomains and query strings are ugly, really. You should use some kind of content negotiation.

[CornedBee]

Content Negotiation is not the end-all. It's good for an initial choice of language (but even then, remember that the Accept-Language spec is complicated and few web apps properly follow it (MS's knowledge base is an example of a very annoying violator)), but it is unsuitable for keeping to a language once it's been chosen. The main problem is that the user might want to select a different language for this particular session - CN doesn't provide for this case. Or the particular page is not available in this language - how to choose a different one?

We use CN only for an initial choice, and cookies/session variables to remember after that. Also, since we actually have one file per language, the language is encoded in the file name, i.e.
http://example.com/index.de.php
http://example.com/index.en.php
http://example.com/index.fr.php

We can then override any prior language choice simply by linking to the appropriate language-specific version.