Just a note in case someone else tries to force the Publish to do what they want. Here is what I have learned so far. I am going to post the relevant VBPROJ file pieces and then explain below.

Note: This relates to VB 2005.

Code:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    ...
    <AssemblyName>MyApp</AssemblyName>(1)
    <ApplicationIcon>Images\MyApp.ico</ApplicationIcon>(2)
    <PublishUrl>\\Server\MyApp\</PublishUrl>(3)
    <ProductName>MyApp - Something</ProductName>(4)
    <PublisherName>Publisher</PublisherName>(5)
    ...
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
     ...
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release-City1|AnyCPU' ">
    <AssemblyName>MyApp - City1</AssemblyName>(1)
    <ApplicationIcon>Images\City1.ico</ApplicationIcon>(2)
    <PublishUrl>IGNORED BY ClickOnce</PublishUrl>
    <ProductName>IGNORED BY ClickOnce</ProductName>
    ...
    <DefineConstants>Location="CITY1"</DefineConstants>
    ...
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release-City2|AnyCPU' ">
    <AssemblyName>MyApp - City2</AssemblyName>(1)
    <ApplicationIcon>Images\City2.ico</ApplicationIcon>(2)
    <PublishUrl>IGNORED BY ClickOnce</PublishUrl>
    <ProductName>IGNORED BY ClickOnce</ProductName>
    ...
    <DefineConstants>Location="CITY2"</DefineConstants>
    ...
  </PropertyGroup>
Note: The four lines of code under each Configuration section are lines that I added to the VBPROJ file manually.

Legend:

1 - The AssemblyName works for the BUILD but not for the PUBLISH. Let me elaborate a little. If the Release-City1 configuration is selected, the Build works great and uses the AssemblyName and ApplicationIcon values from the Configuration settings that I added to the VBPROJ file. However, when you Publish the application, the Build works and then you get an error with the Publishing stating that it cannot find a folder. What appears to be happening is that the compiler uses the Configuration settings to build the application and stage its files, but when the Publish section starts, it ignores all of the values in the Configuration settings. Per the example above the Build creates a folder \bin\MyApp - City1.Publish and puts the necessary files in there then the Publish starts and is looking for a folder \bin\MyApp.Publish.

2 - The ApplicationIcon can be set in the appropriate PropertyGroup and it will work. (I forced this value into the VBPROJ file and it worked.)

3 - The PublishUrl value only works from the main PropertyGroup. It does not work from within the various configuration PropertyGroups.

4 - The ProductName value only works from the main PropertyGroup. It does not work from within the various configuration PropertyGroups. (This is the Product Name displayed on the ClickOnce Publish.htm page.)

5 - The PublisherName value only works from the main PropertyGroup. It does not work from within the various configuration PropertyGroups. (This is the name given to the folder in the Start -> All Programs menu)

In conclusion, from what I can garner about the process, it appears that the Build process will use the Configuration settings, but the Publish ignores them. This creates the error during the Publish.

Assumedly, the Publish portion of the Build process creates the Publish.htm and MyApp_ver_ver_ver_ver.application files, as those two files do not exist in the \bin\MyApp - City1.Publish folder.