Three or four months back we decided to move our build systems from a dedicated virtual machine (VM) farm to the company supported VM farm in the data center. The data center infrastructure had all the bells and whistles (support, backups, redundancy, etc.,).
Everything worked well with the exception of the Gets. Our Gets were taking between 5-10 times longer than in our dedicated farm. We thought it might be a hardware issue, but the new VMs had more memory, more processors, sat physically closer to our TFS server, etc. What the heck?
I spent a good 2-3 days running tests and engaging experts including Teamprise and Microsoft. (I also gained some gray hair.) No such luck getting it figured out.
Turns out my colleague ran into a guy who was having similar issues after migrating to the data center VM farm. Turns out, the data center VM farm was running a new version of our virus scanning software. The real-time scanning must have been checking every file. Once we turned that off, our Gets were significantly faster in the data center VM farm.
What I still don't understand is that the Gets were taking abnormally longer on JAR files (e.g. mac.jar) compared to other binary files (e.g. mac.msi). I'm thinking that since a Jar file could be setup as an executable (I think that is possible in Windows), the virus scanner was spending extra time on those. This is just a hunch.
So if you're running into performance issues with your TFS Gets, try disabling your virus scanner (just for testing of course) and see if that is the cause.
Tuesday, July 07, 2009
Slow "Get" issue resolved
Subscribe to:
Post Comments (Atom)
1 comment:
Interesting!
One possibility is that since JARs use a ZIP container, the virus scanner is uncompressing the JAR to virus scan the contents. While the same could (and probably should) be done with MSIs, the virus scanner may not handle them the same way.
Post a Comment