I recently had the dubious honor to transfer search settings from one SSP to another. Going through every managed property, content source, search scope etc. just wasn’t something I looked forward to. On top of that – in the near future I will have to do it again when we deploy another SharePoint site to production.
Searching the net I found a tool created by Sahil Malik that could create the managed properties for me (link), provided that you manually merged some xml dumps of crawled and managed properties. Thanks Sahil for that great start – I needed something more therefore this post.
I modified Sahils code to suit my additional needs. It took me two full days to complete and test the code and in the end I guess that about 30% of the code base is Sahils original code.
I now have a tool that can import/export content sources, crawled properties, managed properties and (shared) search scopes – and it works!
I designed the import procedures so that they create, or synchronize, the destination SSP search settings with the xml files given, but do not delete anything not in those files, i.e. it will synchronize/create all the managed properties in the input xml file but not tough the existing ones not mentioned in the input file.
Ok, here are the details for the various operation types. The order listed here is the order that they should be imported in a complete SSP import.
Content Sources
Type, name, schedules, start addresses etc. are all handled. As far as I know that is everything, I’ve not been able to test the Exchange and BDC content sources, but they should work.
If you are transferring settings between two servers you probably want to correct the search start addresses as they are likely wrong. I’ve not tried to do anything fancy with automatic recognition of the local farm address and the like as the risk of error is too great, I wanted to keep the focus on the SSP settings not the various sites and their access mappings etc. Sorry for that you can’t have everything.
There is an option to start – and wait – for a full crawl after the import (“-FullCrawl”). This will allow the indices to be built and crawled properties will automatically be added for the crawled content. This is the “normal” way to create crawled properties.
Currently the program will wait a maximum of two hours for the crawl to complete, it will probably be configurable in the future (if I need it).
Crawled Properties
It is possible to import as well as export these. I should stress that the import operation should be considered experimental.
Why would you want to import crawled properties? They are usually created by the crawler and are available for use in managed properties immediately afterwards. However if the content in question have not yet been created (e.g. you are deploying a site to a new farm) or if you don’t want to wait for a full crawl before you create the managed properties, you might want to import them.
I’m not really using this feature myself so I don’t consider my testing to be conclusive enough.
Managed Properties
The code to import and export managed properties is originally from Sahil Malik, though considerable redesigned and bug fixed. It is now possible to dump all managed properties from one site and import them to another – there is no need to extract the standard system managed properties from your own custom (you are welcome if you want to), all can be imported with no changes.
The import will fail if one of the managed properties maps to an unknown crawled property, then you might need to either schedule a full crawl to create the crawl properties or import them too.
The “remove excess mappings” option (“-RemoveExcessMappings”)can be used to delete mappings from existing managed properties to crawled when those properties exists in the input xml file with other mappings, i.e. using this option will ensure that the SSP managed properties are exactly the same as those in the xml file after the import.
Search Scopes
The shared search scopes (those defined in the SSP) are fully supported – settings and rules are all transferred. The import will prune the scope rules to match the import xml file.
The import will fail for scopes that use property rules if the managed properties used has not been defined or marked for use in scopes (the “allow this property to be used in scopes” switch. Import of the managed property includes this setting).
The option “-StartCompilation” starts a scope compilation after the import but not wait for completion (not much point in waiting for that).
The one thing is missing from the scope import is scope display groups. They are of used on sites to populate the search scope dropdown (and some of my own search webparts as well) and are quite important for the end user search experience. You will have to set those yourself as I limited the scope (sorry for the pun) of the program to the setting stored in the SSP. Should be fairly easy for a site collection administrator to enter them however. In a similar vein any site specific search scopes are not handled. I don’t use that feature at all so there’s no support. Perhaps a topic for future improvement.
How to use
Usage: sspc.exe -o <operation> <target type> <parameters>
Operation = Export|Import
Target type = ContentSources|CrawledProperties|ManagedProperties|SearchScopes
Parameters = -url <ssp url> -file <input/output file name> [-FullCrawl|-RemoveExcessMappings|-StartCompilation]
Note all arguments are case insensitive.
This post is quite long enough as is so if you want to see the exact xml format needed download the code and run the export.
Sample Export
SSPC.exe -o export ContentSources -url http://moss:7000/ssp/admin -file output_contentsources.xml
SSPC.exe -o export CrawlProperties -url http://moss:7000/ssp/admin -file output_crawlproperties.xml
SSPC.exe -o export ManagedProperties -url http://moss:7000/ssp/admin -file output_managedproperties.xml
SSPC.exe -o export SearchScopes -url http://moss:7000/ssp/admin -file output_searchscopes.xml
I created a batch file for a full export (excluding crawled properties):
“Export SSP settings.bat” http://moss:7000/ssp/admin
which will create the output files “output_contentsources.xml”, “output_managedproperties.xml” and “output_searchscopes.xml”.
Sample Import
SSPC.exe -o import ContentSources -fullcrawl -url http://moss:7002/ssp/admin -file input_contentsources.xml
SSPC.exe -o import CrawlProperties -url http://moss:7002/ssp/admin -file input_crawlproperties.xml
SSPC.exe -o import ManagedProperties -removeexcessmappings -url http://moss:7002/ssp/admin -file input_managedproperties.xml
SSPC.exe -o import SearchScopes -startcompilation -url http://moss:7002/ssp/admin -file input_searchscopes.xml
The corresponding batch import file:
“Import SSP settings.bat” http://moss:7002/ssp/admin
which assumes the presence of input files “output_contentsources.xml”, “output_managedproperties.xml” and “output_searchscopes.xml” generated above.
Code Design Notes
Sahil Malik named the program SSPC (supposedly short for “Shared Services Provider Property Creation”) and the corresponding project name on the codeplex site is SSSPPC (“Sharepoint Shared Services Search Provider Property Creation”). It’s a mess and now that I’ve expanded the scope of the program considerably the name is even more misleading now.
Just to avoid further confusion I’ve refrained from renaming the program.
Sahil Malik spent some time doing a proper code design for the initial version. I personally think that he did go a bit over the top (sorry Sahil), but I’ve nevertheless retained most of the basic design.
He split up the code in a number of layers (we all love that) where each layer is a different class-library project. I kept that design and therefore the download will contain a number of dll files as well as the actual exe file. Just keep them all in the same directory and all should be well.
Some comments:
- I did not change the naming of the existing projects (i.e. they are all named “Winsmarts.*” though I did change a lot of the code) but the ones I added are named “Carlsberg.*”
- I redesigned/recoded the managed property import section as I simply hate duplicated code and deleted the duplicated BO classes that were present in the old “MAMC project” (now moved to “Winsmarts.SSPC.ManagedProperies”).
- The import code is now always present in the same project that performs the export.
- The managed property import/export is now complete in the sense that it can now export and import everything including the system properties. No need to sort through it all and find the ones you are responsible for (though it might still be a good idea to sift through and ensure that old test data are removed)
- I renamed a number of the classes as some of the BO objects were named as their SharePoint counterparts and the code was quite a bit harder to read than it needed to be.
- Version number of all (sub) projects has been changed to 1.1.0.0.
- Error handling is still pretty basic so you’ll get an exception with a stack trace in the console if anything is amiss
[Updated]
My code changes has now been merged into the main code base at the codeplex site. These changes breaks everything in the original code, so you will need to update xml and script files…
Future Improvements
This is the list of future improvements I’ve noted that might be added if I find the time and need for it.
- [Updated: Done]
The code could be cleaned up somewhat (there shouldn’t be any todo’s in released code) - Perhaps site scopes should be added
- Scope display groups might be added (requires some connection from SSP to the sites)
- It might make sense to add these commands to the list of operations supported by stsadm, which is fairly easy to do (see Andrew Connells excellent post for a sample)
- [Updated: Done]
I’m not too fond of the serialization classes – basically the same piece of code is copied four times with minimal changes. I always consider duplicated code as a bug
Downloads
[Updated]
The code has now been merged with the existing code base at codeplex, so head over there for the latest download.
References
A couple of useful MS articles: Creating Content Sources to Crawl Business Data in SharePoint Server 2007 Enterprise Search and Creating and Exposing Search Scopes in SharePoint Server 2007 Enterprise Search
19 Feb 2008 at 7:09
I too had the task of exporting the ssp search settings from one server to another and your tool really came in handy.
I modified it a bit more to include export and import of crawl rules and authoritative pages.
Also a found and fixed a small bug in the import of sp content sources….
Let me know if you’re interested and i’ll mail you the code
19 Feb 2008 at 20:53
Excellent!
Send me the stuff and I’ll update the codebase on codeplex.
21 Feb 2008 at 12:13
Hey Soren –
don’t have your email address. can you drop me a mail to digitalshehan At gmail.c0m and i’ll reply with the code.
Thanks!
01 Mar 2008 at 8:19
Soren,
Can you please forward me/point to the updated code to export the crawl rules and authoritative pages ?
Thanks,
Prashanth
02 May 2008 at 19:30
Søren,
I am having issue with Importing Crawled properties, I know in your posting you state that you haven’t tested this feature extensively but I was wondering if you’ve have any more input on the topic.
The problem I’m finding that is if I run code similar to :
category.CreateCrawledProperty(“100″, true, new Guid(“9a93244d-a7ad-4ff8-9b99-45ee4cc09af6″), 31 );
I get the error message “”The propset 9a93244d-a7ad-4ff8-9b99-45ee4cc09af6 is not contained within this category Category 12″, while this crawled property did exist on the system I exported my crawled properties from it won’t let me dynamically create these properties on the new system.
I do understand that if I run a full crawl on the new system then the crawled property will be discovered but the situation I’m running into that at the time my importing code is being run its not guarantied that the system has run a full crawl and I don’t want to wait for a full crawl to complete before finishing my import.
Have you run into this situation? Any suggestions on why the CreateCrawledProperty isn’t working as I expected?
Thanks,
jt
05 May 2008 at 20:30
Hi Jt
I’ve had the same requirement as you.
It’s been a while since I used it last.
But yes I did run into some problem with crawled properties and I fixed a bug in the related code.
You are using version 1.1.1.1 from codeplex, right?
It might be a bug I haven’t found… still waiting for code from shehan, I’ll try to poke him again
24 Nov 2008 at 13:52
Great work but how did it go with the exporting of rules? It’s not in the release right?
15 Jan 2009 at 6:03
When we are importing content sources then it should import schedules also like full & incremental schedules.
could you please explain me how can I achieve this?
16 Jan 2009 at 12:43
Rao: That should be the default behaviour. Just export/import the content sources and you should be fine (don’t need any of the other options for that).
You can verify that the xml file contains the required scheduling information.
20 Jan 2009 at 9:49
Thanks Soren.
I am using SSSPPC 1[1].1.1.1, is it the latest version?
If yes then please let me know that How can I export / import crawl schedules. If not then please let me know the latest version number.
Thanks in advance.
Regards
Rao
22 Jan 2009 at 6:37
Sorry Soren, actually it was typo in last post actually I was asking about import export crawl rules.
I am using SSSPPC 1[1].1.1.1, is it the latest version?
If yes then please let me know that How can I export / import crawl rules. If not then please let me know the latest version number.
Thanks in advance.
Regards
Rao
02 Feb 2009 at 10:56
Sorry for late reply.
You are correct there is no support for crawl rules
Neither me nor Sahil apparently needed that feature so it was never implemented. It shouldn’t be too hard to do though.
Copy/paste the existing code structure for a new branch and define how to export/import crawl rules.
You’ll have to do that yourself though
(Happy to add your code to the trunk)
02 Sep 2009 at 5:15
I notice that the PropSet GUID is exported explicitly. Is this not a problem when importing into a different instance of SharePoint?
To be honest, I am nervous to attempt a migration of fields so long as the PropSet GUID is directly referenced.
Now, if you can explain what the PropSet GUID is and why this is not a concern, I would be willing to shout from the tops of mountains how useful this utility truly is!
03 Sep 2009 at 21:59
I don’t think you should worry.
It is not an ID of the actual crawled property (it is certainly not unique), it’s more of a category id. I believe these categories are equal across farms (at least all the system ones are).
I found a bit of description for it here: http://blogs.technet.com/anneste/archive/2008/11/20/mystery-solved-crawled-properties-in-sharepoint-part-1.aspx
whether that is enough to ease your mind is up to you. However if everything goes down the drain and my tool kills your search (it has never happened to me!) you can always just delete the SSP and create a new one (you can maintain mysites regardless). The backup plan is sound.
15 Sep 2009 at 23:49
In the sample usage the CrawlProperties should be CrawledProperties.
Thanks for the tool. Has saved lots of time.