Implement Usage Analysis that Works easily (aka. Google Analytics)

Now this is post I’ve been meaning to write for about a year – there’s quite a few on my todo list. The good thing about this one being so long in the works is that the proposed method of usage analysis is thoroughly tested and works great

For (almost) all the SharePoint sites I’m more or less responsible for we always use the same small piece of code to enable Google Analytics (GA) that coupled with a few custom settings on the GA side of things makes it all very easy to get extremely powerful reporting for nothing. Most of the stuff here relate equally well to non-SharePoint sites.

So what is it?

1. A fairly simple web control to insert into your master page that includes support for SharePoint user groups
2. A rather complicated setup in Google Analytics

Why? (or: The value propositions of this solution are)

• The same code is deployed to all tiers in a farm and can indeed be shared between disparate farms
• Filters on GA splits up the usage data in whatever categories needed (e.g. dev, test and prod tiers)
• The actual tracking is done by the client browser, so
  • Even though your intranet servers have no internet connection you can still track your visitors as long as they do
  • You get detailed info about client capabilities that many other analytics tool (that crawls the server logs) miss
  • You only get info about page views not document or image upload/download and some other actions. To get that you need something to crawl the server logs (Nintex reporting would do the trick for SharePoint) L
  • Your data is not polluted by the crawler (it does not run JavaScript)
• You are not allowed to track individual users (though it is perfectly possible) due to the usage terms (section 7 Privacy) of GA
  • But I’m using some code to track what SharePoint groups the user is a member of, which is very useful. Even though I cannot identify individual users (and generally don’t need it) I can track how many of, say, editors in Sweden are using the site (segment by SharePoint group “editors” and use the map overlay report in GA)
• You get a very powerful front-end for free (I hate the inflated “enterprise grade” term)
  • Track visitors by almost anything you can imagine
  • Even track the SharePoint search engine (but not the results)
  • Combine every report with segments, so you can limit it to certain SharePoint user groups
  • Schedule reports and distribute to key-users (I love everything that automates some aspects of my job)
  • Finally remember that reports don’t equate insights you need some time to experiment before you get anything but fancy pie-charts out of it
  • The front-end is very powerful and useful but I won’t call it easy to use

I believe that GA is superior to most usage trackers (in particular the one built into SharePoint 2007 or 2010), however what’s missing is

1. No knowledge of what happens on the server side of things, i.e.
   1. No tracking of page speed (load time, generation time)
   2. No tracking of server load
   3. No tracking of resource (document, image, etc.) actions
   4. No personally identifiable information
2. Reliant on the client to provide the data
   1. JavaScript and cookies must be enabled
   2. Vulnerable to add blockers
   3. Internet connection must be available (that is actually not always a given even in this day – different countries, different cultures)

If you really need those things you can always use SharePoint’s usage analysis (at least the search usage analysis reports are very useful) or a third party tool like Nintex Reporting (others do exist).

As of today the features of neither Nintex Reporting nor the advances made with SharePoint 2010 replaces the value of GA. To get the best data you need both.

The Code

This is quite simply a web control that emits the required JavaScript to enable GA. I’ve used the latest version of the tracking scripts, but if/when Google changes it the control will need to be updated. As far as I know that has only happened once so far and they were kind enough to maintain backwards compatibility.

There is one twist; it will dump all the SharePoint user groups that the current user is a member of into the custom variable and ship it off to GA, which can then be used to segments the users. The field is called “User Defined” in GA. If you are not using this for SharePoint then remove or change that piece of the code.

Inject it anywhere into your masterpage using something like (remember a tagprefix for your assembly):

<WebParts:UsageAnalysis runat=”server” id=”tracking” UAKey=”UA-XXXXXX-1″ />

Download the code here (no need to inject 100 lines of code here).

GA setup

My primary goal is to be able to use the same GA control unchanged through all tiers, possibly through all farms. That is, I don’t want to change the tracking tag, because then I would need different master pages or some other logic in the web control, which defeats the purpose of staging environment just a little bit.

That means that all data goes into one big pile and I then use filters based on the host name to group them into different profiles.

Get the Account and Master Profile

To start it all, go to GA and get a new account and a master profile with a tracking tag (the term is “Web Property ID” in GA). You only need one.

(Disclaimer: Don’t expect this to be a point and click guide all the way through)

1. Create a new account with a default “master profile” for all data. In the Create New Website profile page choose “Add a profile for new domain” and use the URL for your production site. It does not matter what you use here, it is just a reference used later. Copy the tracking id (“U-XXXXXXX-1″) to the master page or web control above
2. As different domains goes into this profile we need to add the host name to the URI being tracked through a filter. Setup a filter for this profile like this (I usually apply this filter to ALL my profiles as I’m then able to easier spot errors in the filtering):

   

3. GA uses case sensitive URIs in some reports (or it shows in some, most don’t matter) therefore I like to convert them all to lowercase. Setup and attach a new filter like this (apply to all profiles):

   

Setup Profile for Dev, Test, QA, Prod etc.

Now you are ready to create a couple of additional profiles for whatever specialised reporting you might need. I usually have one for DEV, TEST, PROD tiers and possibly for subareas on PROD. There’s nothing here you couldn’t get from the master profile but it does make reporting a lot easier and makes it possible to limit/allow your business administrators to access reports for only their sites.

To create a new profile click the “Add new profile” button on the overview page:

You can see that I’ve created one for DEV, TEST and PROD. More will follow.

Now you need to create a bunch of filters that allows you to extract data from the Master Profile (that contains everything) into the specialised profiles. The trick is to create include or exclude filters based on the host header for each page hit.

The filters are based on regular expressions on the hostnames. I’ve been fond of regular expressions for many years – if you are not then GA has some syntax help. I generally find that it’s fairly easy to distinguish the test and dev environments. Prod is then everything excluding dev and test.

I often use the following filters:

Name	Type	Regular expression
Test env only	Include	^test\..* (Anything that starts with “test.”)
Dev env only	Include	.*dev.*|.*udv.*|^[a-zA-Z0-9]*$ (Anything that includes “dev” or “udv” anywhere in the hostname or any single word hostname without periods (e.g. “extranet” or “intranet”)
Exclude test	Exclude	^test\..*
Exclude dev	Exclude	.*dev.*|.*udv.*|^[a-zA-Z0-9]*$

Note that especially the dev filter will need to be adjusted to suit your individual needs.

For the prod profile I then use “Exclude test”, “Exclude dev”, “Append hostname” and “Request URI to lowercase” filters.

Note: It takes a little while for results to show in GA, usually hours and certainly less than a day.

Useful reports

Three simple reports that I find useful to show of GA to new people is usually

• The Content / Content Drilldown report that answer questions like how many visits is there on a given page and how many on the subpages (if looking at visits)
• The Visitors / Map Overlay report that correlates number of visitors to countries and present a nice colour coded map
• The Visitors / Browser Capabilities / Browsers that shows the browser statistics (and yes, marketing and developers are the ones that don’t use IE)

Segments

Segments are a sort of filtering that you can apply on top of (almost) any given report. That in itself is quite useful. I’ve used the function to be able to filter reports on what SharePoint user groups the visitors belong to (see the code linked above).

For instance the geographic distribution of visitors on one of my sites that is not part of the HQ

To set it up I click the segment button / “Create a new advanced segment”. Drag the “User defined Value” to the box below.

The web control dumps the list of SharePoint user groups into the “user defined value” with a semicolon as separator. It is therefore just a question of creating matching text filters possibly by combining a number of criteria (even regular expressionsJ).

Happy reporting!

 

What Local Admin Privilege Really Mean

This is a small post to explain common misunderstandings regarding local administrators rights that I encounter so often that I find I need somewhere to point for an explanation.

So what you can do with local admin rights is almost everything specifically:

• You can modify almost any file, except those with (very) special security set
  • But you do have the right to change that security
• Similarly you can modify almost all parts of the registry
  • And grant yourself access to remaining parts
• But you are not exempt from Group Policies set by the domain administrators for either the computer or your account. You may find that you cannot
  • Use the command line (but perhaps bat files?)
  • Not start registry editor (but the command line REG command works)
  • Not start any MMC snapin’s (sometimes it’s only authoring that is disabled)
  • Type addresses in the windows explorer bar
  • Etc..

  When in doubt you can run “GPResults.exe” (part of win xp) to get the list of policies applied.

  What can you do about it? You can remove the computer from the domain but that’s likely not an option. You do not have the option of removing the GPO’s but you probably have (or is able to grant) sufficient access to the registry to disable them temporarily. Each one will need to be disabled in different manners, e.g. to enable command prompt again run “REG add HKCU\Software\Policies\Microsoft\Windows\System /v DisableCMD /t REG_DWORD /d 0 /f”. Needless to say it is very awkward and time consuming – your changes will be overwritten by next GPO update – so you probably have to live with this.

• If your server host MOSS/WSS you can do pretty much everything as long as you are sure you are browsing that server. Known by most people this is a very useful little “backdoor” (by design) security feature

And there are so many details better left out here…

 

Heads-up: Your MOSS Web Services are not Secured

This is a little heads-up (and fix) of a security issue in MOSS web services for anonymous accessible sites.

I’ve tested a number of public MOSS sites and I only found one where I couldn’t access “…/_vti_bin/SiteData.asmx” (none mentioned none called out). Try it on your favorite MOSS site and you’ll most likely get the list of web method exposed by this webservice.

You can access all the other MOSS webservices as well.

This is not as big a security hole as you might think because the MOSS webservices will still respect the SharePoint permissions, so the anonymous evil internet hacker can probably not do anything that anonymous users are not allowed to. But he can probably access a whole lot more information than is available on your main site and are you really sure you secured it all properly? Do you really want people to access your webservices?

Specifically the SiteData.asmx webservice is bad story here. It’s used by the crawler to list all pages on the SharePoint site that have changed since last time it crawled. In other words it lists every page in your farm. It also lists secured and/or unpublished pages. You definitely do not want this. What about the title (not the actual content!) of your next stock announcement being available to the outside, before it’s published?

How to fix

I’ve messed a bit with the web.config file that is associated with the web services to disallow anonymous access by default. It’s just a few lines and then all web service calls requires authentication

The default web.config file lists 3 web services that are always allowed anonymous access; I suggest that you leave those unchanged.

If you have any additional web services that require anonymous access you can just add it to the list (after my additions).

So; change the web.config file in “C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI” on all your frontend servers to this:

[..]

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
    <system.web>
        <webServices>
            <protocols>
                <remove name="HttpGet" />
                <remove name="HttpPost" />
                <remove name="HttpPostLocalhost" />
                <add name="Documentation" />
            </protocols>
        </webServices>
        <customErrors mode="On"/>

	<!-- Søren Nielsen: 23-02-2009 disable anonymous access to webservices -->
        <authorization>
            <deny users="?"/>
        </authorization>
    </system.web>

    <location path="authentication.asmx">
        <system.web>
            <authorization>
                <allow users="*"/>
            </authorization>
        </system.web>
    </location>
    <location path="wsdisco.aspx">
        <system.web>
            <authorization>
                <allow users="*"/>
            </authorization>
        </system.web>
    </location>
    <location path="wswsdl.aspx">
        <system.web>
            <authorization>
                <allow users="*"/>
            </authorization>
        </system.web>
    </location>
</configuration>

[..]

Notes

• This is a farm wide setting so all your sites will be affected. Non anonymous sites are unchanged as this is similar to their default behaviour.
• You could copy this web.config file to all your servers (not just frontend) if you like. I don’t see any problems with this, however I’ve not tested it thoroughly enough to say for sure.
• If you add another frontend server be sure to deploy it there too.
• You could add this web.config file to a wsp file and deploy it through SharePoint, but in that case be careful when you retract it because SharePoint will then remove the file and no webservices will work across the entire farm.
• I suggest that you verify that the web.config file is still in place every time you apply a major upgrade/service pack

 

Automatic Setup of Blank Sites across Farm

This is another PowerShell automation blog post

What

Setup a blank site (that is: “Sorry site is down for maintenance”) on all the front end servers and stop all existing web applications with a minimum of work. Likewise take it down easily.

By taking it down I mean deleting it so there is no risk that “someone” (the ghost in the server) might activate it by accident.

One script running on one server will install the blank site to multiple front-end servers.

Why

When you need to take down your sites for maintenance the right thing to do is to setup some temporary site – otherwise your visitors might easily see a semi-functioning site or a huge ugly exception when you’re in the middle of something.

This is not a big task to do manually, however it is tedious if you have many front-end servers and/or many web applications.

How

This is a bunch of powershell scripts that is started by two batch files – one for installing the blank site and one for uninstalling. The tasks performed during installation/setup of blank site are:

For each web server (defined as arguments to the PowerShell script within the batch file):

1. Stop all running websites
2. Setup a new web site (named “Blank Site” on port 80, path “c:\inetpub\wwwroot\blanksite” – customize yourself)
3. Copy all the files (html, aspx, css, etc..) that goes into the blank site to the newly created directory

When you uninstall the blank site the same tasks are executed in reverse order. If there is more than one blank site on the server (i.e. you happened to run the _InstallBlankSite.bat file twice) they will all be deleted by the _UninstallBlankSite.bat script.

Notes

These scripts are used by me on my SharePoint farms; however it can be used on any type of (IIS) web server setup. I’ve tested it with IIS 6 but I believe it should work with IIS 5 and upwards.

You need to edit the two batch files with the correct servernames for your farm. You may add as many as you like (separate with spaces)

I recommend that you run the scripts on a server with PowerShell installed that is not an actual front-end server. I consider it fairly bad practice to install PowerShell on the front-end servers – I only install it on the backend CA/index server.

I use one of MS’ vbs scripts from IIS6 to do interact with the IIS. Another option is to use WMI, but only if you are either only accessing local server or using PowerShell 2.0. I’ve included that script in my package, so there is no requirement that the server executing the PowerShell scripts has an IIS server installed. Not sure if it’s actually legal, but I have a hard time seeing the harm done.

The uninstall script will start all stopped/paused websites regardless if they were actually running before you started the whole thing.

If you can’t get it to work at all, check your execution policy.

Download

The following scripts are included

_InstallBlankSite.bat: Main script to execute the dependent scripts. It will start powershell, and you need to add your server names here.

_UninstallBlankSite.bat: Main script to remove blank site. You need to add your servernames here.

InstallBlankSiteOnServers.ps1: Given a blank site and list of servers do the magic and call dependent scripts.

StopAllWebsites.ps1: Stop all running websites on a given server.

CreateBlankSite.ps1: Create a new website on server with the given local path (does not copy files).

UninstallBlankSiteOnServers.ps1: Remove a blank site from a list of servers. Assumes a local path to files.

StartAllWebSites.ps1: Start all stopped websites on a (remote) server.

DeleteBlankSite.ps1: Removes the blank site(s) from the IIS on the specified server (UninstallBlankSiteOnServers.ps1 deletes the files).

BlankSite: (Directory) the files that constitute your blank site. You’ll want to make it prettier but you get the point from what’s in there.

IISWeb.vbs: IIS script file for IIS6 from Microsoft that performs all IIS management tasks. It has not been altered in any way, it’s merely there for convenience. You’ll find it in system32 on your webservers.

The package can be found here

 

Fixing the Timer Service when everything breaks down

This is a pure troubleshooting post (that will save you days!) if you experience any of the following problems:

• You have trouble deploying solutions to a your farm (more than one node) where nothing happens the deployment never completes
  • You may be able to complete deployments by executing “stsadm –o execadmsvcjobs” on every server in the farm
• You experience frequent CPU spikes of 100% every few minutes on your frontend servers and they are all but unresponsive at those times
• You find one of the following lines in the ULS logs (the SharePoint logs)
  • OWSTimer.exe, w3wp.exe (Id: 888k) : “File system cache monitor encountered error, flushing in memory cache”
  • OWSTimer.exe (Id: 8xqx): “Exception in RefreshCache. Exception message :Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.”
  • OWSTimer.exe (5utx): “The timer service could not initialize its configuration, please check the configuration database. Will retry later.”
• “Some” of the administrative pages in the central administration occasionally fails with the error
  • “Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information”
• The timer and topology cache is never updated (see next section…)
• PSConfig (GUI wizard or not) fails to configure your server and the log file reveals the “LoaderExceptions” error above.

The above problems will probably affect all servers in the farm.

Note: The fusion log troubleshooting will be applicable to all .NET loader errors, not just SharePoint specific stuff.

First shot: Clear the Config Cache

A little known fact is that SharePoint maintains a disk cache (and memory) based configuration on every server that contains the topology information and timer job definitions. Go have a look at “C:\Documents and Settings\All Users\Application Data\Microsoft\SharePoint\Config\<guid>\”.

Sometimes old stuff can get stuck in there and you can kick start the timer by clearing it.

The procedure is simple (do it for every server):

1. Stop the Administration and Timer service
2. Delete all XML files in the directory (not the folder itself)
3. Open “Cache.ini” and write the number 1 instead of the existing number (you might want to make a note of it)
4. Start the services again
5. Wait for a minute or two and see if the folder starts to fill up with xml files. It is likely that it will contain less than before clearing it.
6. Check the cache.ini file. If it’s accessible and the number is considerable greater than 1 your cache has been properly initialized and chances are that your problems are now fixed. It didn’t fix my problem, so you may need to read on… (if you didn’t have the “888k” log entry mentioned above you probably have it now)

The above procedure is grabbed from tomblog’s excellent post (that will also help if you actually did delete the folder too)

[Updated] Or you can run this batch file. [/Updated]

If the procedure didn’t fix the problem you’ll notice that the xml files are updated (timestamp) every few minutes coinciding with the CPU spikes.

Second shot: Digging in (Using the fusion log)

The core problem in my case was that some .NET class/assembly could not be loaded as the message “Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information” strongly hints. It may be a little surprising how many times an assembly, completely unrelated to the task at hand (e.g. deployment), is actually loaded. To fix the problem all we have to do is identify the assembly and “make it available”.

Enabling Fusion Log

You need to debug the failed assembly bindings you need to look into the fusion log. Fusion is an ancient (.Net 1.0) codename for the GAC or assembly loader (I think).

To enable the log you need to add/change three keys in the registry:

HKLM\Software\Microsoft\Fusion\LogPath    REG_SZ    (path to local directory)

HKLM\Software\Microsoft\Fusion\LogFailues    DWORD     1

HKLM\Software\Microsoft\Fusion\EnableLog    DWORD    1

It is not strictly necessary to restart anything, but I recommend that you now restart your timer service in order for it to log any binding errors that occurs. Try starting it a couple of times with 10 minutes in between. That should reveal any binding error patterns.

(Refer to Brad Adams for more info)

Interpreting the Log

Finally we will use the “fuslogvw.exe” program that is part of the .NET SDK to view the actual logs. Your development machine will have this file; copy the executable to your server.

It is not a very good program. It gets the job done, but it’s hard to figure out, you can’t order errors by date you can’t resize the window, etc..

Hopefully your window will contain a lot less entries (in my case the offending entry is the highlighted one. It also failed with similar errors from the psconfig wizard and powershell).

Look for patterns using the timestamps. Did you get a group of binding errors a few minutes after you started the timer service? Or are there just some errors that look spurious? I’ll recommend skipping the core internal microsoft dll’s to begin with (msvcm80 is the c runtime library version 8 that .Net 2.0 uses).

So what goes into the log? Every assembly binding failure, which is (at least):

• If an assembly cannot be located (it will show you where it searched and any assembly redirects)
• If the dll load methods within the assembly throws an exception – e.g. a dependent assemly could not be found – it sadly looks exactly like the dll file could not be found (other exceptions are possible but unlikely for .NET people). The one way to distinguish is to look for the line “LOG: GAC Lookup was unsuccessful.” if it is not there then it was (probably) found in the GAC and a dependent assembly failed.

The last bullet means that if assembly A depends on B and B could not be found then both the binding for A and B fails (at the same time). To distinguish the two I’ll recommend that you look at the “Calling Assembly” in the bind log.

In my case:

1. The Nintex.Workflow dll failed to load and the calling assembly was Microsoft.SharePoint
2. The Nintex.Workflow.Charting.dll failed to load and the calling assembly was Nintex.Workflow. Aha, so the Nintex.Workflow.dll was actually found but failed to find the dependent Nintex.Workflow.Charting assembly.
3. Found the assembly on a development machine and copied it to the server. Retried. Added another missing assembly
4. And everything worked!

I should stress that neither the error type nor the troubleshooting is Nintex workflow specific.

The bind log failure from step 2 above was:

*** Assembly Binder Log Entry (1/13/2009 @ 4:34:15 PM) ***

The operation failed.

Bind result: hr = 0×80070002. The system cannot find the file specified.

Assembly manager loaded from: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll

Running under executable C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN\STSADM.EXE

— A detailed error log follows.

=== Pre-bind state information ===

LOG: User = ….

LOG: DisplayName = Nintex.Charting, Version=1.0.0.0, Culture=neutral, PublicKeyToken=913f6bae0ca5ae12

(Fully-specified)

LOG: Appbase = file:///C:/Program Files/Common Files/Microsoft Shared/web server extensions/12/BIN/

LOG: Initial PrivatePath = NULL

LOG: Dynamic Base = NULL

LOG: Cache Base = NULL

LOG: AppName = NULL

Calling assembly : Nintex.Workflow, Version=1.0.0.0, Culture=neutral, PublicKeyToken=913f6bae0ca5ae12.

===

LOG: This bind starts in default load context.

LOG: No application configuration file found.

LOG: Using machine configuration file from C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\config\machine.config.

LOG: Post-policy reference: Nintex.Charting, Version=1.0.0.0, Culture=neutral, PublicKeyToken=913f6bae0ca5ae12

LOG: GAC Lookup was unsuccessful.

LOG: Attempting download of new URL file:///C:/Program Files/Common Files/Microsoft Shared/web server extensions/12/BIN/Nintex.Charting.DLL.

LOG: Attempting download of new URL file:///C:/Program Files/Common Files/Microsoft Shared/web server extensions/12/BIN/Nintex.Charting/Nintex.Charting.DLL.

LOG: Attempting download of new URL file:///C:/Program Files/Common Files/Microsoft Shared/web server extensions/12/BIN/Nintex.Charting.EXE.

LOG: Attempting download of new URL file:///C:/Program Files/Common Files/Microsoft Shared/web server extensions/12/BIN/Nintex.Charting/Nintex.Charting.EXE.

LOG: All probing URLs attempted and failed.

Note: To grab a dll from the GAC on one machine you need to use a shell (cmd, bash or powershell) to go into the c:\windows\assembly\gac_msil\… folder structure and copy it.

Why did it go wrong in the First Place?

It’s always a useful exercise to figure out what went wrong and why instead of just fixing the error at hand.

The root cause of the error was that the dll’s in questions were distributed through a wsp solution file which was referenced in other code. When that particular solution was undeployed the dll’s were removed (from the GAC) and the new updated wsp file only contained the new versions. Suddenly something went missing

It can also happen if you use some auto wsp packaging procedures like vbs scripts or the ubiquitous WSPBuilder. What if one of the SharePoint system dll’s are marked as “copy local” on one (of the others’) local develop environment and automatically included in the wsp file? It will be deployed, no problems, but when undeployed everything stops as the SharePoint system dll just got removed from the GAC. Oops. Prevention is obviously to use a build server to make clean builds, to educate your people and to have the means to troubleshoot when it didn’t work out.

 

Automatic Configuration of Asp.NET Ajax Extension 1.0

I’m currently testing the world famous RadEditor for MOSS from Telerik and come across a rather annoying deployment issue. Don’t get me wrong the RadEditor is a fantastic product; however it relies on the Asp.NET Ajax Extension 1.0 from Microsoft that really is a royal pain to install and configure.

Installation of the Ajax extension is fairly straight forward, just run through the installer.

Configuring a given website (here it’s a SharePoint site with a lengthy web.config file to begin with) is a rather cumbersome task, described in detail at Mike Ammerlaan’s nice blog post. In total there are about 20 additional tags that need to be inserted into your web.config file in just the right places.

It’s a tedious and error prone process that I’m unwilling to go through with 10+ websites replicated in 2 different environments. Googling for an hour revealed no real solution; it seems that no one has properly automated the process.

I’ve used a small tool ConfigMerge with bit of powershell to do the magic. I can now just run a script that will deploy the configuration settings to all my web.config files at my SharePoint sites (have to run it on every server)

Read on…

Web.Config Modifications

I’ve gathered the required modifications in a xml file that resembles the structure of a “normal” web.config file, which are then merged into the real web.config files. The file is “ajax35.config” in the downloadable zip file.

Please note that the following applies to Asp.NET Ajax Extensions 1.0 for a .NET 3.5 web sites, slight variations are likely for 2.0 and 3.0 sites.

Auto Configuration

[Updated 23-12-2008]

The key component in the auto configuration is the ConfigMerge program found on CodeProject that will merge ajax config file with the existing web.config file. If the nodes are already there they will be updated, not duplicated.

ConfigMerge uses a small list of attributes to identify similar nodes which I’ve had to extend just a bit for our needs (look for my comment on the CodeProject page near the bottom). The point is that it is now idempotent – if you run it more than once, no changes are performed.
I’ve included both the new binary and the modified source in the zip package.

The usage of the ConfigMerge utility is:

ConfigMerge.exe ExistingConfigFile ChangesConfigFile OutputFile

However I prefer to use powershell to call it, and perform (necessary) backup of web.config.

Note: If you don’t like powershell stop reading now and just use the above line in your own bat files and you’ll be good.

Powershelling

To get started with powershell read my old post, though this time I’m actually not accessing the SharePoint API.

Usage for the EnableAjax.ps1 script is (from within powershell):

EnableAjax.ps1 WebConfigPath [AjaxConfigFile]

Alternatively from a batch file/cmd use:

Powershell –command EnableAjax.ps1 WebConfigPath [AjaxConfigFile]

WebConfigPath is the path where the script should search for web.config files. It’ll go recursively down from the specified directory. If you only want to work on a single web application then use the root dir for that web application, if you want to run on all SharePoint applications you can use something like “c:\inetpub\wwwroot\wss\virtualdirectories”.

AjaxConfigFile is the name of the config file you want to merge in. It defaults to “ajax35.config” in the same dir as EnableAjax.ps1.

When the script is about to fix a web.config file it performs a backup first named with the current time, e.g. “web_ajax_2008_12_21 21_06_09.backup”.

Finally here is the actual script (EnableAjax.ps1):

# Apply Asp.NET Ajax Extensions to a number of web.config files easily.

#

# 21/12-2008 Søren L. Nielsen (soerennielsen.wordpress.com)

#

$ajaxconfig = "./ajax35.config"

$path = "."

if( $args.length -eq 2 ){

    $path = $args[0]

    $ajaxconfig = $args[1]

}

elseif ( $args.length -eq 1 ){

    $path = $args[0]

}

else {

    Write "Usage: ./EnableAjax.ps1 [WebConfigPath [AjaxConfigFile]]"

    Write ""

    Write "WebConfigPath - Path where the script should look for web.config files"

    Write "                Script will search for web.config files recursively and "

    write "                modify all it finds."

    Write ""

    Write ("AjaxConfigFile - Default is " + $ajaxconfig + " set it to a xml config")

    Write "                 file with the same structure as a 'normal' web.config file"

    Write "                 but only containing the nodes that should be added/updated"

    Write "                 in the web config files."

    Write "                 The script is build for Asp.Net Ajax extensions it will however"

    Write "                 be useful in many settings, e.g. manipulating dev, QA, Prod "

    Write "                 environment settings."

    Exit

}

write ("Modifying web.configs found in " + $args[0])

foreach( $f in Get-ChildItem $args[0] -filter web.config -recurse ){

    $realname = $f.FullName

    $backup = $f.FullName.ToLower().Replace(".config", "_ajax_" +

[DateTime]::Now.ToString("yyyy_MM_dd HH_mm_ss") + ".backup"

    Write ("Updating " + $f.FullName )

    Write ("Backup file " + $backup )

    $f.MoveTo( $backup )

    ./ConfigMerge.exe $backup $ajaxconfig $realname

}

As the web.config is fairly critical for your application you should be thorough when you test this script, I’ll recommend using WinMerge when comparing before and after config files.

Or you could just trust me an know that I take absolutely no responsibilty for any harm that comes your way

[Updated 23-12-2009] Note: You need to run the script for every server, however if you use UNC paths as input to the script you can run it for every server from just one. I’ll recommend that you only install powershell on your backend servers not your frontends (if you’re on Server 2003).

Note 2: You need to rerun the script if you provision new web applications, or if you add a new server to the farm (on that server). The web.config modifications performed by SharePoint should not interfere with the script.

Download (Source, Scripts and Binaries)

Auto Asp Net Ajax Config

Final Notes

Please note that the files provided here only configures Asp.NET Ajax Extentions 1.0; it does install or configure the telerik RadEditor. It requires two more keys documented in the install guide (and a number of other small steps), that you can easily add through the SharePoint API (WebConfigModifications).

I should stress again that the version of ConfigMerge used here has been modified a bit. If you use the unmodified version you risk that some of the existing tags are “reused”/changed for the Ajax configuration and then SharePoint might well be in trouble.

 

PowerShell – Friend to every MOSS developer

Recently I’ve made the effort to integrate PowerShell with MOSS in my development environment. I recommend that every one of you do the same

Why?

Because you get the ability to jump right into your local farm and use the object model directly – no need to write small programs, and jump into the debugger, just to inspect if an SPItem’s folder object is null or not (note: It will be null for non-folder type items). What that means is:

• It’s scripting that understands .Net. If you’ve ever had to do “normal” scripting you will love this. Every .Net object is accessible and usually very fast to prototype against.
• You can very easily explore your farms stranger aspects and change what you don’t like, i.e. did you know that a number of lists are hidden and that you can easily make them visible by just flicking the switch?
• It is a super fast way to prototype and/or troubleshoot errors
• What is the current query for a given view?
• What does your content type really look like?
• What is really stored in a given items metadata? Was it null or was it an empty string? How should a lookup value look?
• As with all scripting you get pretty much unlimited power. You can chain operations as you like, as complex as needed. You can write small procedures that batch updates pretty much whatever you like – but please be careful with this…
• Objects can be manipulated through chaining, combine that with filtering/selection, and you got something that pretty much resembles Linq for SharePoint.

Why not? Because it’s hard. The learning curve is rather steep at first, which is why I’m posting some of my learnings and tips here. It will be much slower the first few times and it will be frustrating at times.

I’m currently using PowerShell only in my various dev and test environments, I do not do any batch scripting in my production environments. I simply don’t feel that confident (“with great power comes great responsibility” ) and besides it shouldn’t be necessary. Pretty much the same argument that restricts me from just accessing the content database directly and update the faulty content type on a couple of thousand documents though I know I can do it. What could possibly go wrong?

Either pessimism or experience…

Now I’ll not claim to be an expert PowerShell user, very far from it, and I will not create a full PS primer and explain everything to you. When you hit a problem google it and the help will surprisingly often be right there. How do I select the items where the creator is from my local domain? I used to not know, now I know that there is a “where-object” statement that will help me.

I suggest that you follow my simple steps below to get a crack at accessing the SharePoint OM and then head off to some of the PowerShell references at the bottom that will explain all the things I neglected (or simply don’t know).

Anyway, let’s get started with the first baby steps in PowerShell and then move on to the few scripts I created to make my life that much easier.

Getting Started

Install the damn thing. It’s easy, it’s a small download and it’s unfortunately not a portable application (my main complaint). Install from here.

PowerShell scripts use the “.ps1″ extension and be default PowerShell will not execute any script for you (what the hell?). To remedy that:

1. Start powershell
2. Type “Set-ExecutionPolicy RemoteSigned”

Now it will execute all local scripts and only remote scripts that have been signed. The long story is available by executing “get-help Set-ExecutionPolicy”.

Accessing MOSS

PowerShell knows .Net and as such is able to execute code to get you a SPFarm, SPSite or SPWeb in much the same way you do in C# (there are some syntax differences). The caveat here is that you’ll have to load/reference the SharePoint assembly into PowerShell first. It’s hard to remember how, but the correct statement is:

[Reflection.Assembly]::Load("Microsoft.SharePoint,
   Version=12.0.0.0,
   Culture=neutral,
   PublicKeyToken=71e9bce111e9429c")

The good news is that you don’t have to remember it, I’ll save it in the default profile so it’s always available, when I start PowerShell. Let’s also stuff a number of WSS specific methods in there while we’re at it.

What is the statement doing? It’s simply accessing the static Load method of the Reflection.Assembly object with the full assembly name of the Microsoft.SharePoint.dll, which is usually invoked for you by the .Net framework when you reference the assembly in your project. You can also use some of the other load methods, e.g. LoadWithPartialName, if you like that better.

Next I’ve stolen a script from Darrin Bishop that I heard speak at the SharePoint Conference in Seattle ‘08. You really need to read his post that goes with the scripts as he explains a lot of powershell details that I’ve neglected:

[Reflection.Assembly]::Load("Microsoft.SharePoint,
     Version=12.0.0.0,
     Culture=neutral,
     PublicKeyToken=71e9bce111e9429c")

function Get-LocalFarm{

  return [Microsoft.SharePoint.Administration.SPFarm]::Local

}

filter Get-WebService{

   $webServices = new-object
           Microsoft.SharePoint.Administration.SPWebServiceCollection($_)

   $webServices

}

filter Get-WebApplication

   $_.WebApplications

}

filter Get-SiteCollection {

   $_.Sites

}

filter Get-Web {

   $_.AllWebs

}

These few, but rather obscure for the unenlightened, allows you to access all of the SharePoint object model in a somewhat convenient way. To use it you need to chain the methods (using the script pipe operator “|”), e.g. to get all the SPWebs in your site execute:

get-localfarm |get-webservice | get-webapplication | get-sitecollection | get-web

It will return an array of SPWeb objects and dump all their properties. It will be slow (but speed is hardly an issue here) and it will be very long. You can also filter it to get only the webs that has anonymous access enabled and format the result with a table with title, url and the site template used:

Get-localfarm |get-webservice | get-webapplication | get-sitecollection | get-web | Where-Object{$_.allowAnoymousAccess -eq $true} | format-table title, url, WebTemplate

Whew! That’s some PowerShell statement to just throw out. Key notes: 1) Use Format-table to save you to look through hundreds of lines and only show what you want to see, 2) Use Where-Object {} to query and use “$_” to access the current object in the script block. I haven’t listed any of the options where-object supports, but be careful not to use “=” for equality. In our case it would enable anonymous access on all webs. SharePoint fortunately (in this case) use the pattern of calling Update() when you need to commit changes, which you obviously should not call in this case. As I didn’t store the SPWebs I actually cannot even do that.

PowerShell 101

One of the best things to learn in PS is to use variables to save you from typing the same over and over all the time and to enable you to change properties through later Update() calls.

To store all the SPWebs from the previous example:

$webs = get-localfarm |get-webservice | get-webapplication | get-sitecollection | get-web

If the statement returns more than one SPWeb $webs will be an array. To get the first SPWeb type:

$webs[0]

To get a list of all methods and properties that the SPWeb supports, type:

$webs[0] | get-members

To iterate over each object in the pipeline use:

$webs | foreach-object { $_.Lists} | Format-table Name, Url

This will go through all the webs and all their lists and finally output the name and URLs in a simple table.

Combine the foreach statement with the where statement and then you can solve pretty much any batch update task. One fairly easy and useful task would be to write a one-liner that will publish all draft documents in every pages library. You might want to learn a bit more PowerShell first though

Accessing MOSS 2

I created the following 4 functions that I continue to use every time I need to access my local farm:

function Find-Webs([string] $urlpattern) 

   Get-localfarm | get-webservice | get-webapplication |
         get-sitecollection |
         get-web  | Where-Object{$_.url -like "$urlpattern"}

}

function Get-Site([string] $url){
    $site = New-Object Microsoft.SharePoint.SPSite($url)

    return $site

}

function Get-Web([string] $url){

    $u = New-Object Uri($url)

    $site = Get-Site($url)

    $web = $site.OpenWeb($u.AbsolutePath)

    return $web

}

function Select-ItemFields($item) {

    #primitive formatting

    $item.Fields | ForEach-Object { new-object
       System.Collections.DictionaryEntry($_.title, $item[$_.Title] ) }

}

Find-Webs will locate any web that matches the url pattern, so to get any of my “news” webs (in the entire farm):

$news = Find-Webs(“*/news”)

Usually I use it with a specific enough url to get at only one specific site. It is a bit slow because it relies on the above methods from Darrin Bishop that iterates through all web apps/site collections/sites and it does require you to be local admin on your farm (which you probably are anyway).

Get-Site and Get-Web will obviously get a SPSite or SPWeb object given a full url. It is faster and almost as easy to use as Find-Webs.

Select-ItemFields is a small method that will list every metadata field of a given SPListItem. I simply got tired of writing

$item["field1"]

to read specific fields ($item holds the SPListItem). I found no really easy way to iterate over the values in the default indexer of a given object, so I just wrote the few lines required. I’m sure there is a PowerShell wizkid out there that will say, Idiot! Use the built-in in Select-XXX instead. Please speak up; I’ll like to know…

These four functions actually cover my MOSS PowerShell needs. I keep coming back to them to get the first hook into the OM and then perform the task at hand (often a small test) on a given list item, content type, field etc.

Your mileage may vary, so by all means add 10 more convenience methods.

Note: To test my method, just paste the content of the entire script file into your PowerShell window – you don’t need to create fancy scripts or setup profiles.

Making it Convenient

Finally I like stuff to be convenient and easy.

To that end I’ve stuffed all of the above scripting in my profile file, so it’s always loaded when I start PowerShell, in particular the Microsoft.SharePoint.dll is loaded (download my profile script file here. Note: Strip away the “.gif” extension – crappy wordpress restrictions).

To add it to your profile:

1. Go to your “My Documents” folder
2. If there is no directory named “WindowsPowerShell” create it and navigate to it
3. Create/edit a file named “profile.ps1″ and paste my script into it

That’s it! When you launch PowerShell the script should be loaded. To verify, look for the assembly loaded confirmation message, when you start PowerShell. And of course: Execute some of the functions!

One last tip: F7 is your friend. Try it.

References

Darrin Bishops original scripts. He has also written an additional blog post that will help you with filtering.

The basics are coveraged better than I could possibly do on webmat’s blog.

Deeper coverage, when you need to know what’s really going on: Learn PowerShell in Your Launch Break Day 1 and Day 2 (and just keep going…)

And finally, given a problem or a difficult task ask the experts

 

Audit your Content Type Hierarchy

Shortly after you deploy a SharePoint site you probably start to wonder how your carefully designed content types are actually being used. And abused. More importantly I want to know if any of my feature based content types have been disconnected (unghosted) from the XML source file.

To that end I looked into extracting and visualizing the content type hierarchy to get the overview – and I found that it was already built by some nice chaps at codeplex (can be found here)! The community works like never before

I changed it in a number of ways (in my mind improved it):

• The original only showed the content types declared at the site collection root, now I use a completely different way of finding inherited content types through the SPContentTypeUsage classes, to be able to show derived content types in sub sites as well
• Using the SPContentTypeUsage will also give you derived list content types, so they can be displayed as well. There might be a large number of lists, so you can switch the list content type display on and off. Default is off.

  Caveat: To figure out whether a given (list) content type is a direct descendent of another I have to open the actual list content type and check. Some lists are special (like _catalogs) and will be rendered slightly off in the hierarchy (hierarchy “missing” but the list content type location will be there)

• Added my very own detection of the ghosting/unghosting status of any given content type. There are three states: ghosted (connected to xml source), unghosted (disconnected from xml source) and “DB only” (created through the UI or OM, not from a feature)

At the end of the day pretty much all of the code has been changed, while keeping the layout. I’m still grateful for the very nice first step provided and I will post this code back to the codeplex site. Hopefully the guys there will accept the modified code.
[Updated April 14: Patch posted now]

Detecting Ghosted/Unghosted State

I did a lot of digging to figure out how to get at the ghosting status of a content type. You can have a look in the wss content database (table “ContentTypes”) to see if your content types are disconnected or not (column “definition” is null – see my other blog on this). There’s even a dedicated stored procedure that will tell you the ghosting state of a given content type (named “proc_IsContentTypeGhosted”).

I really did not want to directly call stored procedures on the database, so I had a good long hard look at the properties available through the SPContentType class (and many others through reflector) and came up with the following conditions for detecting the state:

• Is it a feature based content type, by checking “FeatureID != null”, if not mark it as “DB only”. Note that this is actually a private field, so I have to go the long way around and fetch it through reflection.
• If it is feature based on the version > 0 then it has been unghosted. The version number seem to work well for this. It will always be 0 for ghosted content types and be incremented by one every time somebody modifies (and thereby unghost it). According to the xml schema you can specify this field in your content type definition, but it will (fortunately) not make it to the database so the detection seems sound.

  If you make new versions of the xml file, the server won’t really notice (see post here).

Disclaimer: This algorithm works well for me, seems reasonable sensible, but there might be an edge case that I’m not aware of.

The Result

Here is what it looks like:

The page is rather long – you are seeing about half of it. It’s also slow, but that’s hardly a performance issue you should care to fix

The Code

You need to create a feature to deploy the page, the feature.xml I use:

<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Id="2D9921AE-D263-4e2b-B4F7-ABDD6223C8B0"
   Scope="Site"
   Title="My Administration Utilities (Site collection)"
   Description="My adminsitration tools at the site collection level">

   <ElementManifests>
      <ElementManifest Location="ContentTypeHierarchy.xml" />
   </ElementManifests>
</Feature>

The referenced ContentTypeHierarchy.xml file (will create a link in the Gallery section of site settings):

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<CustomAction Id="ContentTypeHierarchy"
   GroupId="Galleries"
   Location="Microsoft.SharePoint.SiteSettings"
   Sequence="0"
   Title="Site Content Types Hierarchy"
   Description="Displays a hierarchy of content types.">

   <UrlAction Url="_layouts/ContentTypeHierarchy.aspx"/>
</CustomAction>
</Elements>

And finally the magic page, ContentTypeHierarchy.aspx (deployed to the TEMPLATE\Layouts folder) (you can download it here, you’ll have to rename it back to aspx):

<%@ Page Language="C#" MasterPageFile="~/_layouts/application.master" %>

<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral,PublicKeyToken=71e9bce111e9429c" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<%@ Import Namespace="Microsoft.SharePoint.WebControls" %>
<%@ Import Namespace="System.Data.SqlClient" %>
<%@ Import Namespace="System.Reflection" %>
<%@ Register TagPrefix="wssuc" TagName="ToolBar" Src="~/_controltemplates/ToolBar.ascx" %>
<%@ Register TagPrefix="wssuc" TagName="ToolBarButton" Src="~/_controltemplates/ToolBarButton.ascx" %>
<asp:Content ID="Content2" runat="server" ContentPlaceHolderID="PlaceHolderPageTitleInTitleArea">
	Content Types Hierarchy
</asp:Content>
<asp:Content ID="Content3" runat="server" ContentPlaceHolderID="PlaceHolderPageDescription">
	This page shows all Site Content Types and their hierarchical relationships
</asp:Content>
<asp:Content ID="Content4" runat="server" ContentPlaceHolderID="PlaceHolderMain">

	<table border="0" width="100%" cellspacing="0" cellpadding="0">
		<tr>
			<td id="mngfieldToobar">
				<wssuc:ToolBar id="onetidMngFieldTB" runat="server">
					<template_buttons>
					  <wssuc:ToolBarButton runat="server" Text="<%$Resources:wss,multipages_createbutton_text%>" id="idAddField" ToolTip="<%$Resources:wss,mngctype_create_alt%>" NavigateUrl="ctypenew.aspx" ImageUrl="/_layouts/images/newitem.gif" AccessKey="C" />

						<table cellpadding="1" cellspacing="0" border="0">
							<tr>
								<td class="ms-toolbar" nowrap style="padding:0px">
									<asp:CheckBox runat="server" CssClass="ms-toolbar" Text="Show lists" id="cbList" AutoPostBack="true" />
								</td>
							</tr>
						</table>                

						<table cellpadding="1" cellspacing="0" border="0">
							<tr>
								<td class="ms-toolbar" nowrap style="padding: 0px">
									<asp:CheckBox runat="server" CssClass="ms-toolbar"  Text="Show descriptions" id="cbDesc" AutoPostBack="true" Checked="true" />
								</td>
							</tr>
						</table>
					</template_buttons>
				</wssuc:ToolBar>
			</td>
		</tr>
	</table>
	<asp:PlaceHolder ID="phHierarchy" runat="server" />
</asp:Content>

<script runat="server">
    Hashtable contentTypesShown = new Hashtable();

    protected override void OnLoad(EventArgs e)
    {
		base.OnLoad(e);

		try {
			EnsureChildControls();
			cbDesc.CheckedChanged += new EventHandler(CheckedChanged);
			cbList.CheckedChanged += new EventHandler(CheckedChanged);
			BuildHierarchy();
		}
		catch (Exception x) {
			OutputHtml(x.Message);
		}
	}

	void CheckedChanged(object sender, EventArgs e) {
		BuildHierarchy();
	}

	public void BuildHierarchy() {
		EnsureChildControls();
		phHierarchy.Controls.Clear();
        contentTypesShown.Clear();
		SPSite site = SPControl.GetContextSite(Context);
		SPContentTypeCollection types = site.RootWeb.ContentTypes;

		SPContentTypeId id = types[0].Id;
		OutputHtml("<table style='font-size:10pt' border='0'" + " cellpadding='2' width='100%'><tr><td><ol>");
		ShowContentType(types[0]);
		OutputHtml("</ol></td></tr></table>");
	}

	public void OutputHtml(string html) {
		Literal l = new Literal();
		l.Text = html;
		phHierarchy.Controls.Add(l);
	}

	public void ShowContentType(SPContentType currenttype) {
		try {
			OutputHtml("<li><a class='ms-topnav'" +
				" href=\"" + currenttype.Scope.TrimEnd('/') +
				"/_layouts/ManageContentType.aspx?ctype=" +
				currenttype.Id.ToString() + "\">" + currenttype.Name +
				"</a>" + ExtraInfo(currenttype) +
				(cbDesc.Checked ? "<span class='ms-webpartpagedescription'>" + currenttype.Description + "</span>" : "") +
				"</li>");

			SPContentTypeId id = currenttype.Id;

			OutputHtml("<ol>");

			System.Collections.Generic.IList<SPContentTypeUsage> usages = SPContentTypeUsage.GetUsages(currenttype);

			int listCount = 0;
            foreach (SPContentTypeUsage usage in usages)
            {
                if (!contentTypesShown.ContainsKey(usage.Url + "|||" + usage.Id))
                {
                    contentTypesShown.Add(usage.Url + "|||" + usage.Id, true);

                    if (!usage.IsUrlToList)
                    {
                        using (SPWeb subSite = SPControl.GetContextSite(Context).OpenWeb(usage.Url))
                        {
                            SPContentType subType = subSite.ContentTypes[usage.Id];
                            if (subType.Parent.Id == id && subType.Parent.Id != subType.Id)
                            {
                                ShowContentType(subType);
                            }
                        }
                    }
                    else if (usage.IsUrlToList && cbList.Checked)
                    {
                        //1. Find web + list
                        //2. Open content type
                        listCount++;

                        Match m = Regex.Match(usage.Url, "^(?<web>.*?/)(Lists/)?(?<list>[^/]+)$");
                        if (m.Success)
                        {

                            try
                            {

                                using (SPWeb subSite = SPControl.GetContextSite(Context).OpenWeb(m.Groups["web"].Value))
                                {
                                    SPContentType subType = subSite.Lists[m.Groups["list"].Value].ContentTypes[usage.Id];
                                    if (subType.Parent.Id == id && subType.Parent.Id != subType.Id)
                                    {
                                        OutputHtml("<li>(List) <a class='ms-topnav'" +
                                            " href=\"" + m.Groups["web"] + "_layouts/ManageContentType.aspx?ctype=" +
                                            usage.Id.ToString() + "&list=" + subType.ParentList.ID.ToString() + "\">" + subType.Name + "</a> " + usage.Url + "</li>");
                                    }
                                }
                            }
                            catch (Exception e)
                            {
                                //exceptions occur for some of the "special" list, e.g. catalogs, then just list the list 
                                OutputHtml("<li>(List) " + usage.Url + "</li>");
                            }
                        }
                    }
                }
            }

			OutputHtml("</ol>");
		}
		catch (Exception x) {
			OutputHtml("<b>Error:" + x.ToString() + "</b>");
		}

	}

	string ExtraInfo(SPContentType type) {
		string info = "";

		//Ghosted?
		// if the content type came from a feature and version > 0 then it's ghosted (as
		// far as I've been able to determine).
		// Version will not be greater than 0 for non-ghosted content types.
		// Alternative is to query DB directly (proc_IsContentTypeGhosted).

		//Need to find the featureId (if any) through reflection, since it's not a
		// public property)
		System.Reflection.PropertyInfo featureIdProp = type.GetType().GetProperty("FeatureId", BindingFlags.NonPublic | BindingFlags.Instance, null, typeof(Guid), Type.EmptyTypes, null);
		object featureId = featureIdProp.GetValue(type, null);

		if (type.Version > 0 && featureId != null && ((Guid)featureId) != Guid.Empty) {
			info += "Storage: <b>UNghosted (DB)</b>|";
		}
		else if (featureId != null && ((Guid)featureId) != Guid.Empty) {
			info += "Storage: Ghosted (Disk)|";
		}
		else {
			info += "Storage: DB only|";
		}

		if (type.ReadOnly) {
			info += "ReadOnly|";
		}
		if (type.Sealed) {
			info += "Sealed|";
		}
		info += "Scope:\"" + type.Scope + "\"|";

		info = info.TrimEnd('|').Replace("|", ", ");
		if (info.Length > 0) {
			info = " <i>(" + info + ")</i> ";
		}
		return info;
	}
</script>

Remember to reference the files in your solution manifest file and you should be good to go

Summary

It works and I’m very happy with it. It is an invaluable tool to debug all sorts of content type related problems. Big thanks to the guys at codeplex for letting me hit the ground running on this.

What’s not in it? Site columns. Didn’t find a good way for it and the need seems limited.

Finally understand that this is not really an audit in the normal SharePoint sense, so you should also consider to enable auditing on content type changes as well. Even when you have locked down security properly accidents do happen (only for your fellow admins of course).

 

MOSS does .NET 3.5 Surprisingly Well

I’m sure that a lot of you guys out there are smarter than me and laugh at me writing this.

But…

MOSS does .NET 3.5 without any worries! It works as good as you could possibly hope for – I expected it not to

My experience is hampering me here, as I remember all the problems we had with .NET 2.0 for WSSv2 and/or SPS 2003 (hint: One of the two works) that effectively resulted in us staying with version 1.1 far longer than anyone wanted (same thing happened with the shift from 1.0). The average developer usually want to switch to the v.next right around the beta 2 comes out (don’t we ever get smarter?).

I just wished somebody had written it plainly sooner so I wouldn’t have to go through the hoops myself and we could have started to use .NET 3.5 much sooner.

It is slightly curious how this works though. In the IIS management your SharePoint site (or any plain ASP.NET web application) that you want to use .NET 3.5 in should be set to “ASP.NET 2.0.xx” mode – there is no 3.0/3.5 mode. They really should re-label the option in IIS to something less confusing (I haven’t checked how it’s labeled in IIS7).

How can this be? It’s been a while since I did compiler design courses (I had the “tiger” book) but I can still give some educated guesses:

• The CLR is unchanged between 2.0, 3.0 and 3.5.
• Lambda expressions and Linq are handled at compile time so the actual MSIL code are unchanged along with some new libraries – in my younger days we used to call it “syntactic sugar”
• .NET already knows how to link to libraries of dihfferent versions, provided that they use the same version of the CLR (as far as I know there are three 1.0, 1.1 and 2.0 plus some beta versions in between). Your .NET 3.5 code will use the libraries associated with that version of .NET and the .NET 2.0 code will use its own libraries – the w3wp process that runs both the .NET 2.0 and 3.5 code (as the MOSS site will do) therefore loads both the 2.0 and 3.5 libraries as needed.

I apologize to everyone who already knew this and thinks it obviously should just work. You may be smarter or less experienced/pessimistic (is there a difference?)

Enjoy!

 

”Not enough storage” event log error

[Note: Updated Feb. 22 2008, solution at the bottom] 

I’m responsible for a couple of SharePoint 2007 (MOSS) farms where all SharePoint servers showed a number of annoying errors in the application event log.

Every minute the following three errors show up in the event log:

Event Type: Error
Event Source: Windows SharePoint Services 3
Event Category: Timer
Event ID: 6398
Date: 3/6/2007
Time: 11:47:58 AM
User: N/A
Computer:
Description:
The Execute method of job definition Microsoft.Office.Server.Administration.ApplicationServerAdministrationServiceJob (ID 371548ff-a05e-41f0-90da-6f2d25fbb483) threw an exception. More information is included below.

Not enough storage is available to process this command.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

And

Event Type: Error
Event Source: Office SharePoint Server
Event Category: Office Server Shared Services
Event ID: 7076
Date: 3/6/2007
Time: 11:47:58 AM
User: N/A
Computer:
Description:
An exception occurred while executing the Application Server Administration job.

Message: Not enough storage is available to process this command.
Techinal Support Details:
System.Runtime.InteropServices.COMException (0×80070008): Not enough storage is available to process this command.

Server stack trace:
at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
at System.DirectoryServices.DirectoryEntry.Bind()
at System.DirectoryServices.DirectoryEntry.get_IsContainer()
at System.DirectoryServices.DirectoryEntries.CheckIsContainer()
at System.DirectoryServices.DirectoryEntries.Find(String name, String schemaClassName)
at Microsoft.SharePoint.AdministrationOperation.Metabase.MetabaseObjectCollection`1.Find(String name)
at Microsoft.SharePoint.AdministrationOperation.Metabase.MetabaseObjectCollection`1.get_Item(String name)
at Microsoft.SharePoint.AdministrationOperation.SPProvisioningAssistant.ProvisionIisApplicationPool(String name, ApplicationPoolIdentityType identityType, String userName, SecureString password, TimeSpan idleTimeout, TimeSpan periodicRestartTime)
at Microsoft.SharePoint.AdministrationOperation.SPAdministrationOperation.DoProvisionIisApplicationPool(String name, Int32 identityType, String userName, String password, TimeSpan idleTimeout, TimeSpan periodicRestartTime)
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(RuntimeMethodHandle md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext)
[…continues…]

And

Event Type: Error
Event Source: Office SharePoint Server
Event Category: Office Server Shared Services
Event ID: 7076
Date: 3/6/2007
Time: 11:47:58 AM
User: N/A
Computer:
Description:
An exception occurred while executing the Application Server Administration job.

Message: Not enough storage is available to process this command.

Techinal Support Details:
System.Runtime.InteropServices.COMException (0×80070008): Not enough storage is available to process this command.


Server stack trace:
[…continues…]

Sometimes they are replaced with another three that has the same event id and source, but with the error message being “Old format or invalid type library” instead. The stack trace will differ a little bit.

Obviously it’s a timer job scheduled every minute that fails. The one in question is “Application Server Administration Service Timer Job”, which apparently is in charge of ensuring that the IIS application pools are in sync (or something like that). Nothing seems to be broken in the farm by the job failures.

And quite annoyingly: They all disappear when you reboot the server and will not reappear until after “a few days” in my case. I’m sure that the re-surface time will differ between systems.

I’ve read many proposed solutions for this error, including adding ram, disc etc., often claimed to work. I seriously doubt that any of these solutions actually work. My SharePoint servers are equipped with no less than 6gb of ram with default settings for all the application pools (I know that the ram is hardly utilized with these settings, but in my world it’s sometimes cheaper to go for a “standard server” where you only utilize 70% than one customized for your needs), plenty of disc etc.

As a side note you can also find references to this error in connection with SQL server 2005, the fix below possibly also work for that as well.

The solution turns out to be quite easy – the patch you are looking for is kb923028. It is an update for an error in the .NET 2.0 remoting subsystem, and has actually nothing to do with SharePoint at all. Reading the description it is quite hard to glean that it’ll solve your problem. MS support pointed me to it and it seems to work just fine.

Caveat: I’ve seen multiple versions of this file, the one I have working (until proven wrong) is “NDP20-KB923028-X86.exe” (1,936,224 bytes). I’ve tested another one with the same name with a filesize of 1,963,440 bytes that didn’t work.

The server has been chugging along for some time now (about a week) without the bug so let’s hope it’s all done.

That didn’t work…

Update: Finally a resolution

You need to look at hotfix KB946517, which will fix the problem. It is a private hotfix, so you’ll need to contact MS acquire it.

The servers have been running for about a week now, they are still ok and other people are also reporting success.

For the third time I’m confident that the problem has been solved – guess I don’t learn from experience