The Long Path to Content Deployment
19 Jun 2007 56 Comments
Recently I’ve had the pleasure to use the content deployment feature of MOSS between some of our SharePoint environments.
It was not a smooth ride and I believe that I have now met every single obstacle and bug in that feature. This posting is the result of several months’ worth of frustration.
We got three environments Dev, Test and Prod and we want to use content deployment to move content from the production servers to the development and test servers, simply to ensure current test data.
As a side node, you might want to read the Microsoft brief “Team-based Development in Microsoft Office SharePoint Server 2007” for another good reason to make content deployment one of your skills.
Well, without beating around the bush anymore the following are the steps needed to do a content deployment (with some minor obvious exceptions) in order of appearance.
Setup path
Content deployment requires you to first setup a path and then one or more jobs for that path. Simply put the path is the server to server connection configuration, the job is a specification of what to do and when.
I try to use security best practices so all my both service users are just standard domain users, any special rights they might need are assigned by MOSS during the configuration wizard. In the same vein my farm administrator accounts are standard domain users that are assigned extra permissions only through SharePoint.
The first problem with creating a path is an access denied problem during the path setup page (/_admin/DeploymentPath.aspx), when you select the source web application and site collection. For the destination site collection you are required to provide an explicit user with sufficient rights.
Figure 1: The “Access denied” message
It seems that your logged in user needs to be able to read the IIS metabase when you select a web application. There are many ways to grant access, I choose to add my farm administrator to the IIS_WPG local security group on the server. In my opinion the SharePoint team forgot to impersonate the call to read the IIS metadata. Will hopefully be fixed in some future service pack.
The second problem (on the very same page) occurs if you connect to another farm using SSL (which you should!) – You get the exception “Requested registry access not allowed” when you submit the page.
After some tracing I’ve learned that the problem is that the server tries to store the SSL key for your destination server in the registry hive for the system user, which is the correct one, but apparently the SharePoint configuration wizard tightened the security on the keys in question.
To get around this: Grant your user (farm admin) membership to the local WSS_RESTRICTED_WPG security group and grant that group “full control” to HKEY_USERS\.DEFAULT\Software\Microsoft\SystemCertificates. You could also opt for granting your user direct access to the key.
To sum it up:
- Your logged in user should be a farm administrator
- Should be a member of either the local IIS_WPG or WSS_WPG groups
- Grant access to HKEY_USERS\.DEFAULT\Software\Microsoft\SystemCertificates
- Add user to WSS_RESTRICTED_WPG group
- Grant that group “full control” access to the key
That’s it! You should now be able to setup a path.
Setup Job
Next setup the job to your liking, choose whatever options you desire.
Run the job. If it succeeds then stop reading now and save a few minutes of your time. No? Carry on then…
Hotfix deployment
At this point I got the following exception:
User cannot be found. at Microsoft.SharePoint.SPUserCollection.GetByID(Int32 id) at Microsoft.SharePoint.SPWeb.get_Author() at Microsoft.SharePoint.Deployment.WebSerializer.GetDataFromObjectModel(Object obj, SerializationInfo info, StreamingContext context) at Microsoft.SharePoint.Deployment.DeploymentSerializationSurrogate.GetObjectData(Object obj, SerializationInfo info, StreamingContext context) at Microsoft.SharePoint.Deployment.XmlFormatter.SerializeObject(Object obj, ISerializationSurrogate surrogate, String elementName, Boolean bNeedEnvelope) at Microsoft.SharePoint.Deployment.XmlFormatter.Serialize(Stream serializationStream, Object topLevelObject) at Microsoft.SharePoint.Deployment.ObjectSerializer.Serialize(DeploymentObject deployObject, Stream serializationStream) at Microsoft.SharePoint.Deployment.SPExport.SerializeObjects() at Microsoft.SharePoint.Deployment.SPExport.Run()
Looking like this:
Figure 2: Deployment error – no creator/owner of site
The exception occurs fairly quickly during the preparation phase. It obviously indicates that a creator of a (sub) site it not to be found in the SharePoint user database.
In my case it happened because the farm was originally deployed using a site collection backup/restore from another AD domain, the creators of various sites would then be users in the original SharePoint farm which would be unknown in the new (which is now my source). I suppose you might see this error if you deleted some users as well.
There is nothing you can do about this error; Microsoft however, has a hotfix for this (which also solves a few other bugs). Hotfix number is 313183 and the knowledge base article you are trying to address is kb936867. At the moment of writing this is private hotfix that can only be obtained by contacting MS support. Sucks but there is a way at least.
The hotfix solves a total of 11 bugs including one more in relation to Content Deployment entitled “Violation of PRIMARY KEY” – it seems I avoided one snag after all.
Feature problems
Next error in line occurred right after the last during the export phase. I received the very informative exception:
Failed to compare two elements in the array.
at System.Collections.Generic.ArraySortHelper`1.QuickSort[TValue](T[] keys, TValue[] values, Int32 left, Int32 right, IComparer`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.QuickSort[TValue](T[] keys, TValue[] values, Int32 left, Int32 right, IComparer`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.QuickSort[TValue](T[] keys, TValue[] values, Int32 left, Int32 right, IComparer`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.Sort[TValue](T[] keys, TValue[] values, Int32 index, Int32 length, IComparer`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.Sort(T[] items, Int32 index, Int32 length, IComparer`1 comparer)
at System.Array.Sort[T](T[] array, Int32 index, Int32 length, IComparer`1 comparer)
at System.Collections.Generic.List`1.Sort(Int32 index, Int32 count, IComparer`1 comparer)
at System.Collections.Generic.List`1.Sort(IComparer`1 comparer)
at Microsoft.SharePoint.Deployment.WebSerializer.GetDataFromObjectModel(Object obj, SerializationInfo info, StreamingContext context)
at Microsoft.SharePoint.Deployment.DeploymentSerializationSurrogate.GetObjectData(Object obj, SerializationInfo info, StreamingContext context)
at Microsoft.SharePoint.Deployment.XmlFormatter.SerializeObject(Object obj, ISerializationSurrogate surrogate, String elementName, Boolean bNeedEnvelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Serialize(Stream serializationStream, Object topLevelObject)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Serialize(DeploymentObject deployObject, Stream serializationStream)
at Microsoft.SharePoint.Deployment.SPExport.SerializeObjects()
at Microsoft.SharePoint.Deployment.SPExport.Run()
*** Inner exception:
Object reference not set to an instance of an object.
at Microsoft.SharePoint.SPFeature.EnsureProperties()
at Microsoft.SharePoint.SPFeature.get_TimeActivated()
at Microsoft.SharePoint.Deployment.WebSerializer.ExportFeatureComparer.System.Collections.Generic.IComparer .Compare(ExportObject exportObject1, ExportObject exportObject2)
at System.Collections.Generic.ArraySortHelper`1.QuickSort[TValue](T[] keys, TValue[] values, Int32 left, Int32 right, IComparer`1 comparer)
Going to the source server I tried to pinpoint the error by running “stsadm –o export …” on the base site collection url (same error) and then the first level of sub sites (all exported fine).
That command is exactly the same as a content deployment just without the transfer and import part on the destination end (you are responsible for transfer and then use the import command).
The exception basically means that some of the features activated at the site collection level no longer exist on disk. Their feature definition files have probably been deleted. This can easily occur if you delete some features from your solution packs without deactivating/reactivating all features on deployment (and who cares to do that?).
If you know exactly what features are the problem (might be more than you know) I suppose you might be able to reinstall, deactivate and then uninstall to fix the problem. You might also be able to create dummy features with the correct ids and then try to install, deactivate and uninstall.
I did neither; code had to be written 😉
What you need to do is:
- Go recursively through your web application / site collection / root web / web
- Each of these “parent” objects have a Feature collection
- Each of these features in the feature collection should be examined
- If there is no feature.Definition (== null) then this is one of the faulty features.Simple remove it by executing parent.Features.Remove( id, true ). The force parameter is needed since the feature is not properly installed anymore, so you just remove it without any knowledge of deactivation event handlers etc.
I wrote an aspx page for this that is installed as a feature in the Central Administration site:
Figure 3: My page to list and possibly remove features from web app / site / web
Pretty cool 🙂
In due time, I will clean up the code and publish another article about this administration feature along with a few others that I’ve developed and since found indispensable.
Server Name Problem
Finally the export phase can be completed. Next problem is the transfer phase, which simply moves the exported files from the server assigned the task of performing the content deployment job to the destination server. In the path specification the location of the Central Administration site on the destination server is specified along with the destination web application and site collection.
The source server tries to export the file directly to the destination server handling content deployment jobs, which might or might not be the same server that’s running the Central Administration site. It does so by resolving the FQDN (Fully Qualified Domain Name) of the server, which might very likely be a problem to you.
If you deploy content between servers in separate network segments this won’t work out of the box, e.g. the source server can probably not find your destination server by the name “my_dest_server.my_domain”, which is only known within the immediate local AD domain of that server.
There’s no reason to think too deeply about these names – just try to do a content deployment and if it fails during the transport phase it’ll report “The remote upload Web request failed” along with the name it’s trying to resolve. A similar event log entry is also created:
Event Type: Error
Event Source: Windows SharePoint Services 3
Event Category: Timer
Event ID: 6398
Date: 6/12/2007
Time: 9:48:07 PM
User: N/A
Computer:
Description:
The Execute method of job definition Microsoft.SharePoint.Publishing.Administration.ContentDeploymentJobDefinition (ID 2f94ff2b-2aa1-498b-96ba-649c2e75ada7) threw an exception. More information is included below.
The remote name could not be resolved: ‘servername.dev.local’
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
The fix is simple:
- Open your host file, usually located at c:\windows\system32\drivers\etc\hosts
- Add a new line (don’t delete existing lines): “qqq.xxx.yyy.zzz servername.dev.local“. Be sure to use the correct ip address and name for your destination server. Remember that it should be an ip address that works from the source server (in some cases internal other external ip). You can probably get away with specifying the same ip address you used for the Central Administration site in that path definition – if you used a dns name then just ping the dns and grab the ip address.
- Save the file
- Retry your content deployment. No need to restart any services the fix is immediate.
Feature problems – part II
We’re now in the import phase and almost through. During the import SharePoint will check whether all the activated features at the source site collection (and sub sites) are also available at the destination site collection.
If not you’ll get an informative error, similar to: “Content deployment job ‘Remote import job for job with sourceID = 71bb6ada-762c-4e78-8bc3-2a105bbe5988’ failed.The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘Could not find Feature xxxxxx.’
The solution is obvious – install the same solutions/features at the destination web application as on the source.
Specified Name is already in use
If you get an error similar to “Content deployment job ‘Remote import job for job with sourceID = 71bb6ada-762c-4e78-8bc3-2a105bbe5988’ failed.The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘The specified name is already in use. A list, survey, discussion board, or document library cannot have the same name as another list, survey, discussion board, or document library in this Web site. Use your browser’s Back button, and type a new name.’”
You forgot to read the manual (ok, blogs) that specifies that the destination site collection should be a brand new blank site collection – in my case I tried to export to a newly created publishing site collection.
The Small Print
Finally your content deployment ought to be complete 🙂
If your site looks a bit strange it’s probably because the master page settings wasn’t copied, so you’ll have to assign the correct master page through the site settings. Might also be the case for the welcome page, though I haven’t confirmed it.
During the export every file of your site will be copied to the deployment package so if any files have gone missing you’ll get appropriate warnings during the export, but it’ll still complete.
This is actually a great way to detect if any aspx files have been deleted by accident, e.g. an AllItems.aspx page for a custom list might have gone missing if somebody changed the list definition (probably deployed within a feature).
Phew! At least it works for me now…
(Updated) Other bugs
This is a small list of other bugs I’ve heard/read about
1. (Thanks Harry!) The Cab-files are not always deleted and remain in the folder C:\WINDOWS\Temp\ContentDeployment. Regardless of the content deployment setting of the number of jobs to keep, they will not be deleted. Roll your own workaround and delete the files. You could schedule a job to delete everything older than 1 day (as I’m sure your job can complete in less time than that)
And of course: Move the directory location to a non-system drive
2. (Thanks Harry!) Specific sites within the site collection: I select only the language-variations but not the root. After Test Job or Run Now the root has been always added to the Scope (Fix 937208 apparently solves the problem)
3. Some characters are mangled after deployment, specifically “nbsp;” in html fields (probably a lot of others too). Update: This has been fixed with hotfix 938536 (private hotfix for now – sorry) 🙂
4. Missing disk space on destination server will be reported (on the source server) as
Failed to read package file. at Microsoft.SharePoint.Deployment.ImportDataFileManager.Uncompress(SPRequest request) at Microsoft.SharePoint.Deployment.SPImport.Run() *** Inner exception: Failure writing to target file at Microsoft.SharePoint.Library.SPRequest.ExtractFilesFromCabinet(String bstrTempDirectory, String bstrCabFileLocation) at Microsoft.SharePoint.Deployment.ImportDataFileManager.<>c__DisplayClass2.<Uncompress>b__0() at Microsoft.SharePoint.SPSecurity.CodeToRunElevatedWrapper(Object state) at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass4.<RunWithElevatedPrivileges>b__2() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode) at Microsoft.SharePoint.Deployment.ImportDataFileManager.Uncompress(SPRequest request)
5. If you choose to copy “all” security information between servers not in the same domain you might get the following error (copying role definitions only works fine)
A duplicate name “62c4fcbb-7ff7-4cc3-842e-17476b2e6219” was found. at Microsoft.SharePoint.SPFieldCollection.AddFieldAsXmlInternal(String schemaXml, Boolean addToDefaultView, SPAddFieldOptions op) at Microsoft.SharePoint.Deployment.ListSerializer.CreateOrUpdateField(SPList list, String fieldName, XmlNode fieldNode) at Microsoft.SharePoint.Deployment.ListSerializer.UpdateListFields(SPList list, Dictionary`2 listMetaData) at Microsoft.SharePoint.Deployment.ListSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector) at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject) at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope) at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader) at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects() at Microsoft.SharePoint.Deployment.SPImport.Run()
On hotfixes: Please note that hotfixes are cumulative and later numbers super seeds earlier ones – so the game is to get the highest number of all 😉
(Updated) Conclusion (sort of)
Some of you have asked (in the comments) whether or not it all worked in the end. It actually did work! 🙂 All of the above steps where enough to solve our immediate problems with the content deployment between our three environments.
Great! So do we use it? No.
We really really wanted to use this feature and all looked so well after our the latest hotfix deployment that we planned to use it for the existing publishing site as well as for a some additional upcoming site collections we’re developing.
One particular annoying feature (sorry for the pun) is breaking the content deployment which I have yet to find a solution for. The new sites are based on custom site definitions and a number of features that creates custom content types and site columns from a number of xml files. Some of these site columns are lookup columns that which cannot be created with xml files the same way as every other site column, because they need to refer to an existing list by list id (in the xml file) but the list will be assigned a dynamic id by the system when it’s created (by another feature that the site columns is dependent on).
To get around that problem a feature activation handled is executed that creates the lookup site columns using custom code that find the dynamic list id from the name. That code is roughly based on code found on codeplex (here). Some minor differences: I fixed some bugs with internal/display name mixup, reactivation problems and a missing “webid” (had to be dynamic as well) in the constructed field. The missing “webid” did cause content deployment to fail immediatly. For some reason I have yet to track down it now fails if the content type using one of the lookup site columns is in use, i.e. a list item of that type exists anywhere in the site collection. For lookup site columns created through the web interface there are no problems. Bummer.
The choice we’re facing (barring that I manage to solve the problem before long) is
1. Either: We can define the content type through xml files
2. Or: we can use content deployment and create content types manually through the web interface – they will be copied as part of the content deployment)
We chose the first option as content deployment at the current level of maturity seems too unstable. Furthermore we also decided to use only one strategy and therefore not use the content deployment for the first site collection that this article originally targetted (one where all content types where created through the web interface).
How do we do it now? We use “stsadm -o backup/restore” to deploy new versions of the site. It’s essentially a database backup with all the benefits and drawbacks of such. It’s very stable. Specifically history of all items are maintained, you get a messed up user database where you’ll (eventually) find users from all environments, you need to explicitly set new ownership (to a valid user that can be found in the relevant environments AD), you get the luxury of copied security permissions sets and groups (which I might still build a tool to import/export).
About Søren Nielsen
Thoughts on computing, SharePoint and all the rest 
Hi Søren,
my experiences with the Content Deployment:
1. The Cab-files are not always deleted and remain in the folder C:\WINDOWS\Temp\ContentDeployment.
2. Jobdefinition -> Scope -> Specific sites within the site collection: I select only the language-variations but not the root. After Test Job or Run Now the root has been allways added to the Scope.
Harry
I found an Answer for 2:
FIX 937208 solves the following problem:
– The whole site collection is deployed instead of only the subsite after you update the settings of an existing content deployment job
You update the settings of an existing content deployment job. The content deployment job is configured to deploy a subsite. However, when you run the content deployment job to deploy the subsite, the whole site collection is deployed instead of only the subsite.
For example, this issue occurs after you change the description of the existing content deployment job.
Hi Harry
Thank you for the input! I’ve updated the post a bit, and also included another bug I’ve seen – character mangling.
At least it seems like MS are churning out hotfixes every day at the moment… In a years time I’m sure most of our worries are gone 😉
Great stuff, thank you.
How about the error where it says a certain masterpage file needs to be checked out first.
The file “http://server:port/_catalogs/masterpage/masterpagesample.aspx” is not checked out. You must first check out this document before making changes.
Hi Johnny
I’ve never seen this error in relation to content deployment.
How did you manage to get that one?
I have seen something similar using SharePoint Designer (Ugh!) though. If I open up two sites and drag/drop files between them. It’s an easy way to deploy selected files, however I get the “not checked out” error for the source files. Dunno why it would like to check out those files, when they are the source.. pressing cancel will copy them just fine though (or more likely: they are already copied at this time and the designer probably just want to update a modified date, or something like that).
Hey Soren,
re: Feature problems – part II.
We have a bunch of InfoPath forms which show up as site features on my development server. I assumed, incorrectly, that those features would get automatically created at my content deployment target path. I get the “Could not find feature ” error for those features.
What’s the best practice for deploying those features to all the environments and keeping them in sync? If I change my InfoPath form, do I have to manually redeploy it as a feature from my development environment to both staging and production?
Thank you
About Feature problems.
This problem in my case was in clear Beta2 TR installation, which then had been updated to RTM version.
I resolved this problem by copying folders with features files from my Beta 2 TR backup.
List of features that haven’t been deactivated during update:
1. IPFSDocumentConversion Id=”ec25254b-58bb-4faf-958b-27ed98cbf0e0″ Scope=”WebApplication”
2. PremiumRootSiteStapling Id=” D1D81B84-40E1-4b7a-BD49-E55E7D24BD54″ Scope=”Farm”
3. PremiumRootSite Id=”15DD658A-9006-42e7-AB43-54554CF67AA5″ Scope=”Site”
Even if you do not have backup with Beta 2 TR 12 hive files you can recreate this files using information above and manually deactivate features.
Any way, thank you Soren for your post.
Alas, I am encountering “The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘The specified name is already in use. ” on a job that has been working fine for some time now.
Another bug I have found: CQWP that is configured to point to a specific list ,i.e., “/somelist” deploys to the target sites with an altered path of “/default.aspx/somelist”.
Content deployment is craptastic.
Hi Soren,
“About Master Page not checked out issue”.
The master page will end up deployed. One may have to delete the deployed version and then re-import.
Another related fix, is that digging into the “Failed” error message, you may notice object model calls into methods that try to fix broken links and caching policy stuff. That may be related to content types and inheritance of content types. Go all the way back to the parent content type and check all settings and configurations and then apply your update all the way down the chain of command. Plus make sure where there is a “Description” field, there is no entry for a URI.
In addition to that, cahing policies may be an issue there too. Check to see you have valid cahing policies and then View the properties on the file and make sure if it references a caching policy, that it is the right one.
l8tr
Hi George Durz
Content deployment actually deploys no features in itself, but it does check that every activated feature is installed at the destination (I guess they will then be activated upon deployment).
So your infopath feature will need to be deployed/installed separately.
Infopath forms are usually stored directly in a forms library and as such should be deployed with the content deployment. However if your forms library is deployed as a list feature that won’t work without that feature being deployed at destination too.
As to best practice for that? I’m a bit pressed there. I simply don’t know. I think it’s a trade off between easy deployment (i.e. stuff it into a feature / solution) and ease of changing/adding forms (i.e. connect directly to the running server and update form).
My experience with forms services are limited to problems with anonymous access. We’re currently trying to solve a problem with anonymous access to infopath forms (make “save” work for anonymous users), proof-of-concept works, but let’s see if it really works…
Hi Doug
I’ve seen that one too. 😦
I found that it happened under these circumstances (bear with me – from memory):
1. You did an initial content deployment with “add only new items” option in the job (can’t remember the exact wording)
2. You probably changed some content on the destination server (e.g. added a list etc.)
3. You try to do the content deployment again and it now fails.
I solved it by simply changing the job to deploy all items instead of only the changed ones.
Conceivably there might be situations where you’ll need to delete the destination site collection and do a new content deployment to a blank site. Whether or not that’s acceptable to you (certainly annoying!) I don’t know.
Sorgy: I’ll try to find the time during the weekend to publish my feature to find missing/faulty features on the source server that will cause the content deployment to fail.
Might make it 😉
I’m trying to setup content deployment between two farms and getting the dreaded bland message “The remote upload Web Request failed”. From what I have seen, the export and transporting phases goes ok (contentDeployment directory in both servers have transient files that are cleanup), but the import phase fails without any message in the logs. Anybody has has idea how to troubleshoot this?
Pablo: I would do two things:
1. Try to do a content deployment within the same farm first, i.e. create a new blank site collection as your destination somewhere on your source farm.
2. Try to do an import/export with stsadm. It’s basically the same operations but it enables you to get a proper log of what the system tried to do when it failed.
Be aware that import/export do not preserve item identity (afaik), so you won’t be able to repeat this process. Just use it as troubleshooting, not as the solution.
You can also specify the “-nofilecompression” flag to be able view the actual files it tries to import. Content of the xml files can be surprising (look in manifest.xml, it’s big).
How did this thing end? Did you get it running in a satisfactory way? I have a client – will you recommend making a setup like this, now that you have taken the long haul and maybe cleared the way?
I have almost the same environment as you, and I´m trying to do the deployment with the paths and jobs tool. Few minutes after the import starts I take the error
11:56 Access denied. at Microsoft.SharePoint.SPGlobal.HandleUnauthorizedAccessException(UnauthorizedAccessException ex) at Microsoft.SharePoint.Library.SPRequest.PatchUrl(String bstrUrl, String bstrWebRelFileUrl, String bstrWebRelOldUrl, String bstrWebRelNewUrl, Int32 grf) at Microsoft.SharePoint.Deployment.ImportObjectManager.FixLinkInFile(String fileUrl, Guid webId, String oldTargetUrl, String newTargetUrl) at Microsoft.SharePoint.Deployment.LinkSerializer.SetObjectData …. etc…
Do you have any idea about it?
From what i heard on the grapewine, Carlsberg ended up ditching the content deployment. Did you guys get it working in the end?
Hi Larissa
I haven’t seen that particular error before (and I’ve seen quite a few).
What hotfix are you working on, i.e. what version of the SharePoint binaries are runnning on your servers (go to an actual site -> All site settings, there should be a version number). I’m currently at 12.0.0.6065, with a bit of luck your error has already been fixed.
Barring that slim hope, I would do two thing:
1. Have a look at the SharePoint code through Reflector to the extent that it hasn’t been obfuscated (I haven’t the option to check whether that part is open or not at the moment). It often helps quite a bit.
2. Try to do a stsadm import/export as a test. It performs the same operation (very closely, some options are different so don’t go this route for production systems – specifically item identities are not maintained) and should result in the same error.
– Then export with the option of not compressing the files.
– Have a look at the manifest.xml file which contains metadata for all your entries in the site collection (it will be big).
– Try to locate the offending file/item and with a bit of luck you might spot something amiss (only if it’s possible to locate the file, otherwise it’s too big), and with even more luck you might be able to do something with the problem once you know what it’s trying to do
Sorry not to be able to give you anything more specific
I’ve used the method above succesfully in troubleshooting some problems with faulty lookup fields, where I could edit the manifest file and actually make the content deployment work that way. Obviously this is only something you do in dev to figure out what’s wrong…
Rasmus and Anders: I added another paragraph to the text with some comments on the current state.
We are setting up a content deployment job between two farms, each with a dedicated C.A./Index server. We made sure all custom solutions are deployed to ALL servers in both farms (very important) and we have done some successfull deployments. Now however, we get an error with custom field types:
Field type DKPostalCodeAndCity is not installed properly. . Go to the list settings page to delete this field.
We made sure that the definition for the field type is deployed to and present on all servers and have currently no idea on how to solve this issue. Does anyone have any ideas?
Pingback: Fixing Uninstalled Activated Features « Thoughts on computing, SharePoint and all the rest
Thank you so much for making your findings available to all! I’ve spent the last two days pulling my hair out only to finally stumble across this blog. It’s bookmarked now and I’m sure it’ll come in handy in the future!
Thanks,
John
Fixing Deployment error – no creator/owner of site. I recieved this same error and determined that true the user that the site was created with was in a different domain and when you create the sites SP tags the author of that site with the user that created the site. I found no way in the UI to change the author. So i wrote a web-part that when placed on the root site page will enumerate through all the sites in the collection and change the author to the current user. When tried deployment after that the site deployed. Seems to be the only way to fix it.
This issuse with the CQWP when set to point at specific list, the CQWP does not store the path to the list but the list’s GUID instead. There in lies the problem because when you deploy to a new site all the list are ‘created’ new and therefore have new GUIDs and so CQWP failure. You would think that this would have been thought of by MS but… So i try as much as possible to point to a site path and create content types of the specific ones that i want to return.
Thank you very much for this wonderful posting. it helped a lot.
Now i’m trying to deploy a very big site collection (1.3GB). i receive this error:
“Content deployment job ‘Full Deployment’ failed.The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘Export ran out of memory while compressing a very large file. To successfully export, turn compression off by specifying the -nofilecompression parameter.'”
Now i know that i have this option to go for the STSAdm.exe -o export. but the problem is that i don’t want it to be a one time deployment as i will need to schedule the synchronization betwwen the 2 machines.
Do you have any suggestion for my problem??
Pingback: Should You Depend on Content Deployment? « Thoughts on computing, SharePoint and all the rest
Fayza: I think you may be in trouble. It sounds an awful lot like the problems we had with the old “smigrate” tool for SPS2003.
Your manifest.xml file has simply grown too large to handle.
You can try to do an export without the compression just to learn how big it really is. If possible you might shave it down by deleting some items (a lot!).
The manifest.xml file contains all your listitems, site columns, etc. – basically everything that is not binary files.
If you’re in luck it might be fixed in SP1, that was just released.
very interesting, but I don’t agree with you
Idetrorce
Here’s why our company decided to give up and to develop our future version of Intranet in plain good ASP.NET 3.5:
http://imukai.spaces.live.com/blog
http://pro-thoughts.blogspot.com/2008/01/why-i-hate-ms-share
Any changes or, possibly, success with SP1?
Good question. Anyone using this in production after SP1?
Errors I’ve run into:
1) Timeout for job: the fix: in code you have to do: ContentDeploymentConfiguration.GetInstance().RemoteTimeout = 6000;
ContentDeploymentConfiguration.GetInstance().Update();
Why? The “timeout” happens if an item hasnt been imported in 600 seconds. Except if you have a bunch of cab files, the FIRST item doesnt get imported before 600 seconds. The import actually succeeds, but MOSS never tells you that.
-“The remote upload Web request failed.” In the event log: “The underlying connection was closed: An unexpected error occurred on a receive” For us this happened on cab files that were larger than ~16 meg. The solution: turn off the Windows Server 2003 Scalable Networking pack. Namely: EnableTCPChimney, EnableRSS, EnableTCPA.
-“There is no row at position 0.
at System.Data.RBTree`1.GetNodeByIndex(Int32 userIndex)
at System.Data.DataRowCollection.get_Item(Int32 index)
at
Microsoft.SharePoint.Deployment.ObjectHelper.RetrieveDataFromDatabase(Ex
po
rtObject exportObject)
at
Microsoft.SharePoint.Deployment.FolderObjectHelper.RetrieveData(ExportOb
je
ct exportObject)
at
Microsoft.SharePoint.Deployment.ExportObjectManager.GetObjectData(Export
Ob
ject exportObject)
at Microsoft.SharePoint.Deployment.ExportObjectManager.MoveNext()
at
Microsoft.SharePoint.Deployment.ExportObjectManager.ExportObjectEnumerat
or
.MoveNext()
at Microsoft.SharePoint.Deployment.SPExport.SerializeObjects()
at Microsoft.SharePoint.Deployment.SPExport.Run()”
Seems to happen as the result of an object that has lost some permissions.
-ContentDeploymentJob job = ContentDeploymentJob.GetInstance(“testjob”);
job.Path.EnableCompression = false;
This resulted in the following error: Content deployment job ‘testjob’ failed.The exception thrown was ‘System.IO.FileNotFoundException’ : ‘Could not find file ‘C:\WINDOWS\TEMP\ContentDeployment\b6c2d72c-7b38-46bf-abaa-8c5bd122b977\ExportedFiles.cab’.’
Presumably even though the API lets you specify not to use compression, it still assumes that you are.
-“Value cannot be null.
Parameter name: g
at System.Guid..ctor(String g)
at Microsoft.SharePoint.Deployment.FieldTemplateSerializer.FixLookupFieldSche
ma(XmlNode fieldNode, Guid parentWebId, Guid fieldId)
at Microsoft.SharePoint.Deployment.FieldTemplateSerializer.CreateField(SPWeb
web, SerializationInfoHelper infoHelper)
at Microsoft.SharePoint.Deployment.FieldTemplateSerializer.SetObjectData(Obje
ct obj, SerializationInfo info, StreamingContext context, ISurrogateSelector sel
ector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType,
Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type object
Type, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializat
ionStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serial
izationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReade
r xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()”
I believe this bug comes when a site column is of type Lookup. The manifest.xml does not include the appropriate GUID either for the web or the list that has the values for the lookup.
-Other’s that I can’t remember right now.
Hi there,
i have a big problem. On a customer project, which uses content deployment, all site columns and site content types are missing in the target web application after the deployment. the deployment itself works without errors and copies all the content (documents, list, structure etc.) fine, but the site columns and site content types are missing. The strange thing is, in our environment (which is normally the same) it works.. Can anyone help me ? its frustrating 😦
Thx
Thomas
I started to get the following error message during a content deployment using 2007. I can’t put a finger on what ‘memory’ it’s referring to:
Export ran out of memory while compressing a very large file
I’m sure that it is referring to the internal memory buffer used for the compression.
Try to do an export from the commandline and specify no compression. Look at the exported data. You’ll probably find one very large file, that’s causing the trouble.
If you are lucky you can identify the file on the portal and just delete it there (or zip or something else).
If not it’s the manifest.xml file that is huge and then you are out of luck. You simply have too many items on your site… 😦
Perhaps try to do content deployment on a smaller subset if possible.
Thanks….. let me do some testing…..
Hi:
When I try to do a deployment, I get the following two errors. Please help.
Error 1. A duplicate name “RightPara4” was found. at Microsoft.SharePoint.SPFieldCollection.AddFieldToWeb(String strXml, Boolean checkDisplayName) at Microsoft.SharePoint.SPFieldCollection.AddFieldAsXmlInternal(String schemaXml, Boolean addToDefaultView, SPAddFieldOptions op) at Microsoft.SharePoint.Deployment.FieldTemplateSerializer.CreateField(SPWeb web, SerializationInfoHelper infoHelper) at Microsoft.SharePoint.Deployment.FieldTemplateSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector) at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject) at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope) at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader) at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects() at Microsoft.SharePoint.Deployment.SPImport.Run()
Error 2: Content deployment job ‘Remote import job for job with sourceID = c29d3d46-25c5-4a92-9404-db87c2a7f4cb’ failed.The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘A duplicate name “RightPara4” was found.’
Hi,
Further to my above post, I would like to mention that I created a column “RightPara4″ ìn a content type which in turn was used in a layout. However, after creating the deployment path and job and on running the job, I get the above error. Can I get some help please?
Denny: Your destination site should be empty (that is, a blank team site) before you start deployment (the first time).
Did you create the column on the destination as well (through the UI or object model)? You shouldn’t.
If the source of the field is an xml file, you should deploy the relevant feature to the destination site as well (activation is probably not important).
If you are now “stuck in the middle” I guess you could delete the destination site and start a fresh content deployment to a new empty site.
Hi Soren:
Yes, my destination site was a blank team site. And, actually it did deploy perfectly the first time.
Then I made changes in my original site and tried to redeploy it and that’s when I got the error.
When all efforts failed, I created a new empty site and tried deploying – but this time it failed even the first time and every time with the same error.
So, now I am kind of sure that the problem is with the original site – but I am not able to find the duplicate name. The original works perfectly on its own. Its the deployment that does not.
The problem is solved. There were truly some duplicate columns in the source site. I wonder how the duplicates got created in the first place. Anyways, I deleted the duplicates and redeployed and it works fine now.
Another question. I have a two page test site. However, I find that the deployment takes about 45 mins. And, I observe that the Object Exported is 5000 plus. How can it be so high for such a small site? Can we control what get exported. I mean, are the various versions also getting deployed. If yes, can we stop it? And, only deploy the latest version of all objects.
Pingback: MOSS 2007 Content Deployment
I am getting Timed out exception when I ran a content deployment job.
Hi,
I had a big issue to do………
Actually I have 2 MOSS servers(separate installations), one is at Bharain and one is at Saudi, both are on 2 different domains. Now what I need is when I uploaded some content in Bharain server, after some time Saudi server also wanted to be updated automatically(Using Synchronization).
I am trying with central administration features following:
Central Administration–>Operations–>Content Deployment–>Content Deployment Paths and Jobs
But I am getting the following 2 error’s:
1)Export ran out of memory while compressing a very large file. To successfully export, turn compression off by specifying the -nofilecompression parameter. at Microsoft.SharePoint.Deployment.ExportDataFileManager.c__DisplayClass2.b__0() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.Deployment.ExportDataFileManager.Compress(SPRequest request) at Microsoft.SharePoint.Deployment.SPExport.Run()
2)Content deployment job ‘Home Deploy’ failed.The exception thrown was ‘Microsoft.SharePoint.SPException’ : ‘Export ran out of memory while compressing a very large file. To successfully export, turn compression off by specifying the -nofilecompression parameter.’
What might be the problem and how to resolve it? It’s very important and need to resolve as soon as possible, please concern with high priority and do needful Thank you……..
Please reply to naren378@gmail.com
Thanks,
Naresh Alapati
Naresh: Please see some of my other replies higher up – the best way to troubleshoot is to run an export from the command line (please do not do an import on that data, things will break) to see what it actually tries to export.
Use the no-compression parameter.
Hopefully you’ll learn a thing or two, e.g. what large file the system is complaining about (if it’s the manifest file, you might be in trouble).
Good Article.
Fayza – your comment about compression was from quite a while ago, but I discovered an STSADM command where you can disable file compression in a Content Deployment path:
stsadm -o editcontentdeploymentpath
One of the options you can pass in to this command is -enablecompression yes|no.
Thank you very much. This post was extremely usefull and safed me a lot of time.
I am facing an issue that is killing me for 4 long days.Still i couldnt able to solve that.I have wriiten 3 custom field controls where i have deployed its dll in bin folder.
Issue:
I am getting this error while doing Content Deployment from authoring server to production server.
The exception thrown was ‘System.Security.SecurityException’ : ‘Request for the permission of type ‘Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c’ failed.’
I am having a custom policy file where i have given all the permissions.
I did the following:
1.Deployed with Full trust,still the same error
2.Deployed with custom policy file,still the same error
3.Installed the dll in GAC(authoring server),the deployment job successfully exports from authoring server but at the time of importing in production server,i am getting the same error(in prod. server i insatlled the dll in GAC)
I am getting this error only for custom field controls dll.I am having a separate dll for webparts,that doesnt give me error.
Can anyone help me out to solve this issue asap.
Thanks in advance.
arun.k
Arun: The only thing I can think of is to make absolutely sure that the field control is registered and works on the production site, before you try the content deployment.
It should work on its own possibly just on some test site.
Then you can try to do content deployment.
My experiences with field controls are not good. They tend to be very fragile and once deployed you can most likely never get rid of them again. They have not failed for me with content deployment, but I’m not surprised that they might cause problems.
Hello Soren,
We are also having Content Deployment problems, the latest being the ‘Export ran out of memory while compressing a very large file’ error.
I have been able to do an export with compression turned off, and the resulting export file looks to be the right size, 104Meg. The Manifest.xml file is 30Meg, and one .dat file is 22Meg. Three other files are between 1 and 2M, and the other 2600 files are under 1M, most being very small.
Can I assume it’s the 22Meg .dat file which is causing the trouble? If so, how would I go about identifying which file it is on the portal? A second question, where exactly would I look for log information on this? Would it be part of the standard diagnostic logging?
Thanks for this hugely helpful blog, by the way.
Keith: A quick reply 😉
If you have search enabled you can go to the advanced search, select the size property, greater than operator, enter a byte size of 20mill and you should find it.
If not you could just open the file in notepad. You can probably glean what file type it is from that (header fields near the top). Probably some office document. Rename it to correct extension, open it and view what it is. I suppose that would give you a clue where to look for it.
I’m not sure there is a good log for this. You could try to do the stsadm export command. It creates an export log file that might help you too (location next to your export package).
Pingback: Issues with Sharepoint’s Content Deployment Temp File « Yiu San's Blog for Microsoft's Technology (Sharepoint, MOM, DPM, ISA, Exchange, etc)
Pingback: Name of the blog
Pingback: Name of the blog | MOSS 2007 Content Deployment