Tuesday, July 03, 2012

Sporadic Access Denied Messages with Team Explorer Everywhere Running on Linux Host

Hello again! I’ve since moved on from TFS stuff (working in our data center as a systems architect), but the other day I was brought back in to look at a few things. Fortunately enough, through I’ve forgotten nearly everything, I haven’t lost some good relationships with folks that are still very involved in TFS/TEE and was able t bounce off some ideas and boil this issues down.

Basically, there was a group on Linux that kept sporadically getting the following two errors when using the TEE Command Line client and doing "get" calls.  After thinking about it a bit over an afternoon cup of coffee, I kind of felt might be due to erroneously sending our calls out through the http proxy. These are internal calls so they shouldn’t be going out through the proxy.

An argument error occurred: The workspace 'surlyfurious-bdmac;domain\myaccount' was found in the local workspace cache, but it no longer exists on the server. It has been removed from the cache.

and

An error occurred: Access denied connecting to TFS server http://mytfs:8080/ (authenticating as domain\myaccount)

While in their profile they set -string:httpProxyEnabled=false, they were still getting these errors. This left us a bit dumfounded.

After talking with a good friend in MS who works on TEE (and Teamprise before that), he suggested we remove the http_profile environment setting on the host. We simple removed this value from the users .profile (.bashrc in their case) and then logged out and back in.

So far these errors have went away. We’ll let this bake for a few more days, but things are looking much better. My guess is that while we’re setting -string:httpProxyEnabled=false in our profile, the system wide http_proxy setting was overriding this. And since we didn’t have a no_proxy value set (we should have), everything was getting routed out.

Friday, March 18, 2011

TFS 2010 SP1 Upgrade

It took 8 hours, but we’ve finally got our TFS system upgraded to TFS 2010 SP1 from TFS 2008 SP1. I've been critical of them in the past, but with 2010, I would say MS greatly improved the upgrade process. I really like how they separated installing the bits from doing the migration. And when we did run into an issue, we were amazed that when we re-ran the install (after correcting our issue), the upgrade wizard started off where we ran into issues.

This all being said, as much as this experience was better than our TFS 2005, TFS 2008 and TFS 2008 SP1 upgrades we did run into issues - primarily ours. Here they are.

1) We had to install VS 2010 Ultimate on each build machine. I didn’t dig into this much, but apparently our testers use “Web Tests” during the smoke test phase of our builds. These “Web Tests” apparently require the VS 2010 Ultimate install. Again, this is not really my area of expertise so we went ahead with the request and dumped Ultimate on 30 build hosts.

2) We ran into an issue where our drive for logs (100gig) filled up. This did not happen the first two times we did test upgrades, but as luck would happen, when we tried to go live, this issue popped up. We paged the DBA to have our log files truncated. We were really nervous that we’d have to go through the entire migration again. However, much to our surprise, the upgrade started right where it left off. A wonderful surprise!!! Thanks Microsoft!!!

3) We had some problems with the encryption of our Reporting Services View State. A colleague of mine came across this link which states – “If your application is installed http://www.blogger.com/img/blank.gifin a Web farm, you need to change the validationKey from AutoGenerate,IsolateApps to a specific manually generated key value”. After making this change, our encryption problems were solved.

4) We had some sporadic issues getting people re-connected (mainly use error), but nothing significant.

5) Make sure your users get Visual Studio 2010 SP1. Some power users who branch from labels, found that the ability to branch on a label from a folder that was already branched, was removed. (Say that 10 times fast!) VS 2010 SP1 adds that ability back into the client.

6) Make sure any VS 2008 users get this. This is so they can log into the 2010 server from a 2008 client. Visual Studio Team System 2008 SP1

Again, over all this experience was pretty damn good. I give MS an A- for this upgrade.

Monday, February 28, 2011

Groovy Nature Locks Jar Files in Eclipse Project

For the past few months many of us have been struggling with an issue where we can’t Get Latest on Jar files stored in TFS when using TEE (or Teamprise). The issue is that TEE can’t overwrite the local file because the file is locked by Eclipse. (non-Windows users would not see this as Mac and Linux platforms use a more passive file locking strategy.) On the surface, this issue seemly looks like an issue with the TEE plug-in. However, in these file locking cases, we’ve found that other plug-ins are actually the culprit.

While we’ve been able to reproduce the issue commonly, we’ve finally got some conformation from codehaus.org that the Groovy plug-in many of you use will cause this issue. The root case is that they use the URLClassLoader from Sun/Oracle which won’t release locks on jar files when loaded to the classpath. Or something like that.

Here is a link to the issue we submitted. You can follow that for updates to the plug-in. Hopefully they have this fixed in an upcoming release. Until this time, the best way to work around the issue is to temporarily remove the Groovy Project Nature from your project, do Get Latest, then add the Groovy Project Nature back in.

Remove Groovy Nature = Right Click on Project | Groovy | Remove Groovy Nature

Add Groovy Nature = Right Click on Project | Configure | Convert to Groovy Project

Tuesday, February 22, 2011

Custom Response Header For Our TFS 2010 App Tier Farm

I’m curious to see if people think this is a good idea or not. In our TFS 2010 farm we have four application servers. In the process of testing load balancing and fail over, I was having trouble trying to figure out which box my requests were going to. Our load balancer sets a cookie so we persist to the same App Tier machine, but it’s an obfuscated value that no one seems to be able to provide any insight into how to un-obfuscate it. (And I’m not sure if devenv.exe is passing that cookie as we seem to be bouncing across all four App Tiers when doing things like “Get.” Not a big deal, but it would be nice if there was a way to persist.)

To help me out, I set a custom response header called “x-tfs-machine” that resolves the name of the App Tier machine. Via Fiddler or Wireshark, I know can inspect what App Tier I’m hitting. It seems to be working quite well for me.

While I’m just using this for testing, I’m thinking about leaving it on when we go live so we can see what server a user is hitting if they are running into issues.

Tuesday, February 15, 2011

Team Explorer Everywhere Options for Users Wanting a Standalone Client

We’ve had some users who have shied away from using Eclipse plug-ins and have become comfortable with Teamprise Explorer (aka Fat Client). When Team Explorer Everywhere (TEE) was launched, Microsoft discontinued delivering the Fat Client. Thus, those users have continued to use Teamprise. We’d like to get those users cut over so Brian has showed a pretty good work around that should work for these users. Here is what we sent to our teams:

If you’re used to using the Teamprise Explorer for development, here are some options you have when migrated to Team explorer Everywhere (TEE).

1) Switch to using the TEE plug-in in Eclipse.
2) Use the free Team Explorer plug-in to the free Visual Studio 2010 shell. (This only works for Windows users)
3) Per Brian Harry’s post, use the Eclipse Platform Runtime Binary standalone client.

If you decide to go with the last option, here are some general instructions for using the Eclipse Platform Runtime Binaries.

1) Download and install Eclipse Platform Runtime Binary- Install TEE
2) Window Show View Other Team Foundation Server Team Explorer
3) In the Team Explorer panal, select the Add Existing Team Project dialog button
4) Add your server and connection information
5) Select Team Projects you’re involved with
6) Double click on Source Control to get the friendly Source Control tree view
7) Get yourself a cup of hot coffee

Friday, December 17, 2010

Windows Process Activation Service – Error 183

This is not related to TFS, but I need a place to write this down. On one of our runtime machines, we had an issue where II 7 stopped running. It turns out that the Windows Process Activation Service (WAS) was down. When trying to restart WAS, we got an Error 183 – Unable to create file (or something like that). When trying to open Event Viewer or any other snap-ins to MMC, we got a “could not load snap-in error.”

Through hair pulling and swearing, we finally tracked down a machine.config change that one of my colleagues made. In the process of doing some tuning, the I haven’t reviewed the schema, but I’m guessing this is not valid as it does not make sense.

After removing the autoConfig, we were back up and running. Thanks to Carlo for pointing us in the right direction.

<processModel maxWorkerThreads="100" maxIoThreads="100" minWorkerThreads="50" memoryLimit="90"/>

<httpRuntime minFreeThreads="12"/>

<processModel autoConfig="true"/>

Monday, December 13, 2010

Windows "Loopback bug" when using a FQDN

We just got our new TFS 2010 hardware! Unfortunately, my smile was quickly brought to its knees by the "loopback bug" where you can access a IIS site (in our case it's a Share Point site running under IIS 7.5) via fully qualified domain name (FQDN) from a client machine, but can't use the FQDN on the server looping back to the server. When trying to use the FQDN locally on the server, we continually get the username/password prompt.

Here is the work around if you loose a day on this like I did. Option #1 worked for us. Thanks for the post at Server Fault for reminding me.

Monday, October 04, 2010

Spring: Ambiguous constructor argument types

This is not TFS related, but build related and this is my only public blog where I document build-like stuff for future reference if I was to ever need it.

In Spring for Java, if you’re going to use “names” for wiring up things like constructor arguments, the class files have to be built with javac –g, which is an entry-level debug mode for javac. This will compile the classes, keeping such things as variable names. If you don’t use javac –g, then Java takes the liberty to modify variable names. I’m guessing they do this for, among other things, keep the class sizes down.

Below is a poorly formatted, convoluted series of knowns and tests to come up with a hypothesis and eventual solution to Spring issue we were running into.

Knowns

1) The Spring configuration is using variable name for mapping constructer arguments in the spring configuration to the set of constructor parameters.

bean id="SetManager" class="com.mycompany.manager.SetManager" lazy-init="false"

constructor-arg name="courtSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg name="pubSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg name="jurSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

bean

2) When deploying the application out, the error we’re getting is this.

SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'SetManager' defined in class path resource [spring-manager.xml]: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.springframework.core.io.Resource]: Ambiguous constructor argument types - did you specify the correct bean references as constructor arguments?

at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:684)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:192)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:984)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:886)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:479)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:450)

3) When creating a WAR via Eclipse’s Export feature, the bindings are wired up fine. When creating the WAR via Ant (or using javac from the command line, which is what Ant is doing), we get the error. When diff-ing the WAR files, they are the same with one exception: The class files created via Eclipse are larger.

Hypothesis Given all of this, our hypothesis is that running the build via Ant, the method names are getting truncated (or modified) such that using the name attribute in Spring won’t work.

Tests

1) To test the use of indexes, instead of names we used something like this: “index="0"”. Using this got us around the initial issue, but ran us into a very similar problem loading another bean. At this point, we knew we were on to something. Or on something. Here are the Spring configuration snippets of using names verses indexes.

Names

bean id="SetManager" class="com.mycompany.manager.SetManager" lazy-init="false"

constructor-arg name="courtSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg name="pubSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg name="jurSetCSV" type="org.springframework.core.io.Resource" value="classpath:..."

bean

Indexes

bean id="SetManager" class="com.mycompany.manager.SetManager" lazy-init="false"

constructor-arg index="0" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg index="1" type="org.springframework.core.io.Resource" value="classpath:..."

constructor-arg index="2" type="org.springframework.core.io.Resource" value="classpath:..."

bean

We used JAD to decompile both classes. Bingo, the issue is highlighted! When using javac, the default settings truncate the variable names. If Spring is to use variable names for wiring up arguments or properties, this won’t work very well ;)

Ant

public SetManager(Resource resource, Resource resource1, Resource resource2)

{

loadCourtSetMaps(resource);

loadPub(resource1);

loadJur(resource2);

}

Eclipse

public CustomDigestCollectionSetManager(Resource courtSetCSV, Resource pubSetCSV, Resource jurnSetCSV)

{

loadCourtlineCollectionSetMaps(courtSetCSV);

loadPublicationCollectionSetMap(pubSetCSV);

loadJurisdictionCollectionSetMap(jurSetCSV);

}


Solution

1) What parameter change in Ant’s javac Task could we change to get the accurate parameter names?

a. Set debug = true. Turning debug on will make sure the class files are compiled with the same variable names (and other things) as what is in the source code. This will increase the class file size, but from what we’ve noticed so far, does not cause great performance degradation.

Friday, July 30, 2010

Why a Web.config change does not result in a new process id

A colleague asked me why he sees Application_Start events after modifying his Web.config but does not get a new process id for the worker process. While this article is addressed to IIS 5 and 6, I think the same principle must apply for IIS 7.

From what I can tell, a change to the Web.config will result in tear down of the Application Domain, but not the Application Manager. The Application Manager is what controls the worker process.

If anyone disagrees with this, let me know.

Thursday, July 22, 2010

“The Network Path Was Not Found” When Installing our .NET Node Agent

We ran into a problem installing some custom software we call the .NET Node Agent on machines that were in a new COMPANYQA Windows domain. After some troubleshooting, we found the issue to be related to how we figure out what domain the machine is in. We do this logic so we can insert the correct .NET Node Agent service account (COMPANYMGMT|COMPANY\svcNodeAgnet) into the machine’s local Administrators group. If a machine is in COMPANYMGMT, we need to add COMPANYMGMT\svcNodeAgnet. If the machine is in COMPANY, or now COMPANYQA, then we need to insert COMPANY\svcNodeAgnet. (We could have asked a human to make this distinction during install time, but asking a human to do anything around here causes more trouble than it’s worth.)

We use .NET’s System.DirectoryServices namespace to surf Active Directory (AD). We use the machine’s parent to grab Properties. When trying to get Properties from the parent (DirectoryEntry.Parent.Properties) we continually got the message “The Network Path Was Not Found.” After some in-depth troubleshooting, we found that we had to add companyqa.mycompany.com to the DNS suffix list. The only values were company.mycompany.com and so we couldn’t find the LDAP.companyqa.mycompany.com AD server.

In Windows 2008 Server, this setting is under Control Panel - Network Connections - Local Area Connections – Internet Protocol Version 4 (TCP/IPv4) - Properties – DNS – Append These DNS Suffixes. After we added companyqa.mycompany.com we were on our way.