Tool for Deployment of SSP search settings

I recently had the dubious honor to transfer search settings from one SSP to another. Going through every managed property, content source, search scope etc. just wasn’t something I looked forward to. On top of that – in the near future I will have to do it again when we deploy another SharePoint site to production.

Searching the net I found a tool created by Sahil Malik that could create the managed properties for me (link), provided that you manually merged some xml dumps of crawled and managed properties. Thanks Sahil for that great start – I needed something more therefore this post.

I modified Sahils code to suit my additional needs. It took me two full days to complete and test the code and in the end I guess that about 30% of the code base is Sahils original code.

I now have a tool that can import/export content sources, crawled properties, managed properties and (shared) search scopes – and it works!

I designed the import procedures so that they create, or synchronize, the destination SSP search settings with the xml files given, but do not delete anything not in those files, i.e. it will synchronize/create all the managed properties in the input xml file but not tough the existing ones not mentioned in the input file.

Ok, here are the details for the various operation types. The order listed here is the order that they should be imported in a complete SSP import.

Content Sources

Type, name, schedules, start addresses etc. are all handled. As far as I know that is everything, I’ve not been able to test the Exchange and BDC content sources, but they should work.

If you are transferring settings between two servers you probably want to correct the search start addresses as they are likely wrong. I’ve not tried to do anything fancy with automatic recognition of the local farm address and the like as the risk of error is too great, I wanted to keep the focus on the SSP settings not the various sites and their access mappings etc. Sorry for that you can’t have everything.

There is an option to start – and wait – for a full crawl after the import (“-FullCrawl”). This will allow the indices to be built and crawled properties will automatically be added for the crawled content. This is the “normal” way to create crawled properties.

Currently the program will wait a maximum of two hours for the crawl to complete, it will probably be configurable in the future (if I need it).

Crawled Properties

It is possible to import as well as export these. I should stress that the import operation should be considered experimental.

Why would you want to import crawled properties? They are usually created by the crawler and are available for use in managed properties immediately afterwards. However if the content in question have not yet been created (e.g. you are deploying a site to a new farm) or if you don’t want to wait for a full crawl before you create the managed properties, you might want to import them.

I’m not really using this feature myself so I don’t consider my testing to be conclusive enough.

Managed Properties

The code to import and export managed properties is originally from Sahil Malik, though considerable redesigned and bug fixed. It is now possible to dump all managed properties from one site and import them to another – there is no need to extract the standard system managed properties from your own custom (you are welcome if you want to), all can be imported with no changes.

The import will fail if one of the managed properties maps to an unknown crawled property, then you might need to either schedule a full crawl to create the crawl properties or import them too.

The “remove excess mappings” option (“-RemoveExcessMappings”)can be used to delete mappings from existing managed properties to crawled when those properties exists in the input xml file with other mappings, i.e. using this option will ensure that the SSP managed properties are exactly the same as those in the xml file after the import.

Search Scopes

The shared search scopes (those defined in the SSP) are fully supported – settings and rules are all transferred. The import will prune the scope rules to match the import xml file.

The import will fail for scopes that use property rules if the managed properties used has not been defined or marked for use in scopes (the “allow this property to be used in scopes” switch. Import of the managed property includes this setting).

The option “-StartCompilation” starts a scope compilation after the import but not wait for completion (not much point in waiting for that).

The one thing is missing from the scope import is scope display groups. They are of used on sites to populate the search scope dropdown (and some of my own search webparts as well) and are quite important for the end user search experience. You will have to set those yourself as I limited the scope (sorry for the pun) of the program to the setting stored in the SSP. Should be fairly easy for a site collection administrator to enter them however. In a similar vein any site specific search scopes are not handled. I don’t use that feature at all so there’s no support. Perhaps a topic for future improvement.

How to use

Usage: sspc.exe -o <operation> <target type> <parameters>

Operation = Export|Import

Target type = ContentSources|CrawledProperties|ManagedProperties|SearchScopes

Parameters = -url <ssp url> -file <input/output file name> [-FullCrawl|-RemoveExcessMappings|-StartCompilation]

Note all arguments are case insensitive.

This post is quite long enough as is so if you want to see the exact xml format needed download the code and run the export.

Sample Export

SSPC.exe -o export ContentSources -url http://moss:7000/ssp/admin -file output_contentsources.xml

SSPC.exe -o export CrawlProperties -url http://moss:7000/ssp/admin -file output_crawlproperties.xml

SSPC.exe -o export ManagedProperties -url http://moss:7000/ssp/admin -file output_managedproperties.xml

SSPC.exe -o export SearchScopes -url http://moss:7000/ssp/admin -file output_searchscopes.xml

I created a batch file for a full export (excluding crawled properties):

“Export SSP settings.bat” http://moss:7000/ssp/admin

which will create the output files “output_contentsources.xml”, “output_managedproperties.xml” and “output_searchscopes.xml”.

Sample Import

SSPC.exe -o import ContentSources -fullcrawl -url http://moss:7002/ssp/admin -file input_contentsources.xml

SSPC.exe -o import CrawlProperties -url http://moss:7002/ssp/admin -file input_crawlproperties.xml

SSPC.exe -o import ManagedProperties -removeexcessmappings -url http://moss:7002/ssp/admin -file input_managedproperties.xml

SSPC.exe -o import SearchScopes -startcompilation -url http://moss:7002/ssp/admin -file input_searchscopes.xml

The corresponding batch import file:

“Import SSP settings.bat” http://moss:7002/ssp/admin

which assumes the presence of input files “output_contentsources.xml”, “output_managedproperties.xml” and “output_searchscopes.xml” generated above.

Code Design Notes

Sahil Malik named the program SSPC (supposedly short for “Shared Services Provider Property Creation”) and the corresponding project name on the codeplex site is SSSPPC (“Sharepoint Shared Services Search Provider Property Creation”). It’s a mess and now that I’ve expanded the scope of the program considerably the name is even more misleading now.

Just to avoid further confusion I’ve refrained from renaming the program.

Sahil Malik spent some time doing a proper code design for the initial version. I personally think that he did go a bit over the top (sorry Sahil), but I’ve nevertheless retained most of the basic design.

He split up the code in a number of layers (we all love that) where each layer is a different class-library project. I kept that design and therefore the download will contain a number of dll files as well as the actual exe file. Just keep them all in the same directory and all should be well.

Some comments:

  • I did not change the naming of the existing projects (i.e. they are all named “Winsmarts.*” though I did change a lot of the code) but the ones I added are named “Carlsberg.*”
  • I redesigned/recoded the managed property import section as I simply hate duplicated code and deleted the duplicated BO classes that were present in the old “MAMC project” (now moved to “Winsmarts.SSPC.ManagedProperies”).
  • The import code is now always present in the same project that performs the export.
  • The managed property import/export is now complete in the sense that it can now export and import everything including the system properties. No need to sort through it all and find the ones you are responsible for (though it might still be a good idea to sift through and ensure that old test data are removed)
  • I renamed a number of the classes as some of the BO objects were named as their SharePoint counterparts and the code was quite a bit harder to read than it needed to be.
  • Version number of all (sub) projects has been changed to 1.1.0.0.
  • Error handling is still pretty basic so you’ll get an exception with a stack trace in the console if anything is amiss

[Updated]

My code changes has now been merged into the main code base at the codeplex site. These changes breaks everything in the original code, so you will need to update xml and script files…

Future Improvements

This is the list of future improvements I’ve noted that might be added if I find the time and need for it.

  • [Updated: Done] The code could be cleaned up somewhat (there shouldn’t be any todo’s in released code)
  • Perhaps site scopes should be added
  • Scope display groups might be added (requires some connection from SSP to the sites)
  • It might make sense to add these commands to the list of operations supported by stsadm, which is fairly easy to do (see Andrew Connells excellent post for a sample)
  • [Updated: Done] I’m not too fond of the serialization classes – basically the same piece of code is copied four times with minimal changes. I always consider duplicated code as a bug

Downloads

[Updated]

The code has now been merged with the existing code base at codeplex, so head over there for the latest download.

Codeplex/SSSPPC

References

Sahil Maliks original post

The current Codeplex site

A couple of useful MS articles: Creating Content Sources to Crawl Business Data in SharePoint Server 2007 Enterprise Search and Creating and Exposing Search Scopes in SharePoint Server 2007 Enterprise Search

Advertisements

About Søren Nielsen
Long time SharePoint Consultant.

20 Responses to Tool for Deployment of SSP search settings

  1. shehan says:

    I too had the task of exporting the ssp search settings from one server to another and your tool really came in handy.

    I modified it a bit more to include export and import of crawl rules and authoritative pages.
    Also a found and fixed a small bug in the import of sp content sources….

    Let me know if you’re interested and i’ll mail you the code 🙂

  2. Excellent!

    Send me the stuff and I’ll update the codebase on codeplex.

  3. shehan says:

    Hey Soren –
    don’t have your email address. can you drop me a mail to digitalshehan At gmail.c0m and i’ll reply with the code.

    Thanks!

  4. Prashanth says:

    Soren,

    Can you please forward me/point to the updated code to export the crawl rules and authoritative pages ?

    Thanks,
    Prashanth

  5. jt says:

    Søren,

    I am having issue with Importing Crawled properties, I know in your posting you state that you haven’t tested this feature extensively but I was wondering if you’ve have any more input on the topic.

    The problem I’m finding that is if I run code similar to :

    category.CreateCrawledProperty(“100”, true, new Guid(“9a93244d-a7ad-4ff8-9b99-45ee4cc09af6″), 31 );

    I get the error message “”The propset 9a93244d-a7ad-4ff8-9b99-45ee4cc09af6 is not contained within this category Category 12”, while this crawled property did exist on the system I exported my crawled properties from it won’t let me dynamically create these properties on the new system.

    I do understand that if I run a full crawl on the new system then the crawled property will be discovered but the situation I’m running into that at the time my importing code is being run its not guarantied that the system has run a full crawl and I don’t want to wait for a full crawl to complete before finishing my import.

    Have you run into this situation? Any suggestions on why the CreateCrawledProperty isn’t working as I expected?

    Thanks,

    jt

  6. Hi Jt

    I’ve had the same requirement as you.

    It’s been a while since I used it last.

    But yes I did run into some problem with crawled properties and I fixed a bug in the related code.
    You are using version 1.1.1.1 from codeplex, right?

    It might be a bug I haven’t found… still waiting for code from shehan, I’ll try to poke him again 😉

  7. Ludvig says:

    Great work but how did it go with the exporting of rules? It’s not in the release right?

  8. Rao says:

    When we are importing content sources then it should import schedules also like full & incremental schedules.
    could you please explain me how can I achieve this?

  9. Søren Nielsen says:

    Rao: That should be the default behaviour. Just export/import the content sources and you should be fine (don’t need any of the other options for that).

    You can verify that the xml file contains the required scheduling information.

  10. Rao says:

    Thanks Soren.

    I am using SSSPPC 1[1].1.1.1, is it the latest version?

    If yes then please let me know that How can I export / import crawl schedules. If not then please let me know the latest version number.

    Thanks in advance.

    Regards
    Rao

  11. Rao says:

    Sorry Soren, actually it was typo in last post actually I was asking about import export crawl rules.

    I am using SSSPPC 1[1].1.1.1, is it the latest version?

    If yes then please let me know that How can I export / import crawl rules. If not then please let me know the latest version number.

    Thanks in advance.

    Regards
    Rao

  12. Sorry for late reply.

    You are correct there is no support for crawl rules 😦

    Neither me nor Sahil apparently needed that feature so it was never implemented. It shouldn’t be too hard to do though.
    Copy/paste the existing code structure for a new branch and define how to export/import crawl rules.

    You’ll have to do that yourself though 😉

    (Happy to add your code to the trunk)

  13. Sly W says:

    I notice that the PropSet GUID is exported explicitly. Is this not a problem when importing into a different instance of SharePoint?

    To be honest, I am nervous to attempt a migration of fields so long as the PropSet GUID is directly referenced.

    Now, if you can explain what the PropSet GUID is and why this is not a concern, I would be willing to shout from the tops of mountains how useful this utility truly is!

  14. Trevor says:

    In the sample usage the CrawlProperties should be CrawledProperties.
    Thanks for the tool. Has saved lots of time.

  15. great article, thanks for sharing.

  16. IsoJorma says:

    I am personally in the process of doing complete search settings replication from “known set” to a series of dev + test + QA + finally production servers.
    However, for sanity reasons, I tend to use powershell. I know. Lose my sanity trying to command sharepoint from there. What I mean is, to be able to “see” and understand completely what search settings are moved etc, and limiting and changing the code without vstudio and executables and worrying about context and so forth and so on.

    I have successfully done the search content source settings, BDC I prefer to import manually, SSO manually as well as the settings vary per environment.
    I am yet to do managed properties export + import but I will do so using powershell, hopefully the codeplex code will give me pretty much all of it.
    Then I only have display group and search scope settings, and those can be done easily using powershell.
    I also prefer to do all of these in separate scripts, which can of course be chained if desired. This allows complete freedom since it’s text files, to the “end user” using the scripts.
    Also, powershell 2.0 (which was never distributed continuing the ctp name but under the Microsoft name “windows management framework), allows nice write-to-xml and read from xml direct functionality.

    I am merely stating this as a comment, and I guess I will approach gary lapointe to add the scripts to his sets or something at some point.

    • Hi Jorma

      That sounds like a very good solution you are building.

      Since I wrote this post we’ve been using powershell extensively for a number of purposes and I also agree that scripting is definitely to be preferred over executabels. Old habits die hard somehow.

      We been focusing the powershell code on deployment so pretty much all of the new cmdlets available in SharePoint 2010 we made first using the API 😉

      Please share the result with the community when you get there.

      Kind regards Søren

  17. kitti says:

    Hi Soren this is indeed a wonderful tool thanks for sharing it is it possible to export only a subset of values instead of the whole set of values

    • Hi Kitti

      Not directly. All values are exported, however it is possible to modify the xml afterwards to remove some values. Or perhaps more to the point to only include the small subset that you are actually interested in.

      It works well for us to only maintain the few managed properties that we actually care about and only import those into the SSP.

      /Søren

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: