TFS 2008 SP1 upgrade nightmare

I'm typically very pro-TFS on this blog, but I have to share some negative news. Mainly because it cost me, and my colleagues, an entire evening away from our families while we upgraded. Here are the issues and resolutions from our nightmare. It took us six hours in total. We're done now and I'm off to my home where I can lie awake all night and stew.

Issue #1:
Issue: .NET Framework 3.5 SP1 install failed when installing Visual Studio 2008 SP1

Resolution: Installed framework separately.


Issue #2:
Connecting to SQL Server Reporting Services. Please wait...
Invalid namespace
Value cannot be null.
Parameter name: uriString
Configuring SQL Server Reporting Services failed.

Resolution: Removed the Standby Key from the database.
C:\Program Files\Microsoft SQL Server\90\Tools\binn\rskeymgmt -l
C:\Program Files\Microsoft SQL Server\90\Tools\binn\rskeymgmt -r asdfasfasdf-8968686-asfa-asdfdsf-9as7dfasfasd



Issue #3:
Issue: Restart attempt failed.

IIS Admin Service or a service dependent on IIS Admin is not active. It most likely failed to start, which may mean that it's disabled.

03/26/09 19:55:52 DDSet_Status: Process returned 1062
03/26/09 19:55:52 DDSet_Error: The Commandline '"C:\WINDOWS\system32\iisreset.exe" /Restart' returned non-zero value: 1062.
MSI (s) (AC!14) [19:55:52:046]: Product: Microsoft Visual Studio 2008 Team Foundation Server - ENU -- Error 32000.The Commandline '"C:\WINDOWS\system32\iisreset.exe" /Restart' returned non-zero value: 1062.

Error 32000.The Commandline '"C:\WINDOWS\system32\iisreset.exe" /Restart' returned non-zero value: 1062.
03/26/09 19:55:52 DDSet_CARetVal: 32000

Resolution: The install person watched the IIS Admin service and 'W3SVC' service. When they went down, he manually brought them back up. We tried this five times manually and never got it right. We finally reverted to issuing the command "iisreset & issreset & issrest …" continually. Eventually we got the timing right and it went through. This is the one that really gets me upset. I've never seen such a thing.

Issue #4:
Issue: Team System Web Access 2008 uninstalls, but you have to delete the IIS Website and App Pool manually through the IIS Manager. You also have to purge d:\Program Files\Team System Web Access 2008 as the uninstall does not delete that either.

Resolution: Delete the IIS data and installed files manually.


Issue #5:
Issue: We ran into the issue with .NET 3.5 SP1 and Fully Qualified Domain Names. Here are some of the error messages we found in the Event Viewer.

Exception Message: TF30063: You are not authorized to access tfs.mycompany.com

TF53002: Unable to obtain registration data for application Build.

Detailed Message: TF53005: Unable to retrieve the Team Foundation Server installed UI culture.

Detailed Message: TF53002: Unable to obtain registration data for application VersionControl.

Detailed Message: TF30040: The database is not correctly configured. Contact your Team Foundation Server administrator.


Resolution: See here for resolution steps. The first one (i.e. recommended) didn't work for us. We had to do the second one, which worked. http://support.microsoft.com/kb/926642. A special thanks to Brian and Buck for posting this on their blogs.

Honestly, if it wasn't for what people post on blogs, we would have had to rolled back. A special thanks to everyone in the community who shares their experiences. Hopefully some of the issues and solutions I've given here help others.

UPDATE (3/37/09): One thing to remember is if you use a FQDN (e.g. tfs.mycompany.com) is you have to run ActivateAT [MachineName] (e.g. activateat MyTFSAppTier) on the primary app tier before running the install. We knew about this as someone else at the company told us so it wasn't an issue that we ran into, but could be one that you do if you fail to do this.

Cross Platform Builds with Team Build

For some reason I seem to constantly find myself working with cross platform solutions (e.g. .NET, J2EE, C). Recently I've been assigned to figure out how we can compile custom Apache HTTPd modules without re-inventing our current build solution (which uses Team Build). I've created a diagram that describes the proposed solution.

The basic idea is we use Team Build for all the peripheral build tasks, outside of compile. Compile has to be done on platform specific machines (e.g. Linux 64). So to compile, we simply use Ant's SSH tasks to 1) transfer code to the platform machines, 2) compile via make and then 3) transfer binaries back to the Team Build server from the platform machines so the build can continue. We use Teamprise's Build Extensions to call Ant. In addition, in previous lives, we've used Teamprise for #1 and we'll probably look at doing the same in the near future. Here is the diagram.