Content Migration using Web Service and Client Side Object Model

Most of our intranet sites have been moved from SharePoint 2007 to SharePoint 2013, like Team sites and MySite. The only one left is the our corporate portal site, which serves the entire company.


In MOSS 2007, we implemented many Farm solutions, such as, custom web parts, list event receivers, workflow and custom site template. Those are the bits that make upgrade challenging. For team sites, we re-complied the Farm solution (there is only one Farm solution for Team sites) and deployed to SharePoint 2013. Then take the content database to SharePoint 2010 first, then SharePoint 2013. However, for our corporate portal site, which is heavily customized, we don’t want to follow the same road-map since we try to avoid Farm solutions and convert everything to App Model. Business also want to split this one site collection to 3 new site collections in SharePoint 2013.


I developed a custom migration tool in a short period. It will allow site admin to migrate the MOSS 2007 site to SharePoint 2013 remotely as well as converting custom web parts to App parts and updating the links in the list items.


SharePoint 2007 SharePoint 2013

For MOSS 2007, I’m using Web Services like /_vti_bin/Webs.asmx, /_vti_bin/Lists.asmx,  /_vti_bin/Views.asmx and /_vti_bin/WebPartPages.asmx. There is also a custom web service developed due to the limitation of OOTB web service. For SharePoint 2013, Web Services are used as well as Client Side Object Model. Using this custom migration tool, site admins can move a site/list to anywhere they want.



Create Query Rule and Prompted Results SharePoint 2013

string siteUrl = "";
string resultSourceName = "Local SharePoint Results";
using (SPSite site = new SPSite(siteUrl))
  using(SPWeb web = site.OpenWeb("/en"))

    SPServiceContext context = SPServiceContext.GetContext(SPServiceApplicationProxyGroup.Default,

    SearchServiceApplicationProxy searchAppProxy = context.GetDefaultProxy(typeof(SearchServiceApplicationProxy))
                                       as SearchServiceApplicationProxy;

    SearchServiceApplicationInfo ssai = searchAppProxy.GetSearchServiceApplicationInfo();

    if (searchAppProxy != null)
      Guid searchAppId = searchAppProxy.GetSearchServiceApplicationInfo().SearchServiceApplicationId;
      // Get the application itself
      SearchServiceApplication searchApp = SearchService.Service.SearchApplications.GetValue<SearchServiceApplication>(searchAppId);

      QueryRuleManager queryRuleManager = new QueryRuleManager(searchApp);
      // SPWeb Level
      SearchObjectOwner searchOwner = new SearchObjectOwner(SearchObjectLevel.SPWeb, web);

      SearchObjectFilter searchFilter = new SearchObjectFilter(searchOwner);

      FederationManager federManager = new FederationManager(searchApp);

      Source resultSource = federManager.GetSourceByName(resultSourceName, searchOwner);

      QueryRuleCollection qrCollection = queryRuleManager.GetQueryRules(searchFilter);

      CreateQueryRule(queryRuleManager, qrCollection, searchFilter);

private void CreateQueryRule(QueryRuleManager queryRuleManager, QueryRuleCollection qrCollection, SearchObjectFilter searchFilter)
     QueryRule queryRule = qrCollection.CreateQueryRule("QueryRuleName", null, null, true);
     // Query Matches Keyword Exactly
     List<string> keyWords = new List<string>();
     KeywordCondition kwCondition = queryRule.QueryConditions.CreateKeywordCondition(keyWords, true);

     // Promopted Results
     Microsoft.Office.Server.Search.Query.Rules.BestBetCollection bestbets = queryRuleManager.GetBestBets(searchFilter);
     QueryAction queryAction = queryRule.CreateQueryAction(QueryActionType.AssignBestBet);
     Microsoft.Office.Server.Search.Query.Rules.BestBet bestBet = bestbets.CreateBestBet("Resutl1", null, "Description", true);
     bestBet = bestbets.CreateBestBet("Result2", new Uri(""), "Description", true);

     RegularExpressionCondition pnCondition = queryRule.QueryConditions.CreateRegularExpressionCondition("pattern", true);



Configure Site Navigation via CSOM

We would like to configure site navigation when a new site got created using CSOM. 
Here is the code I figured it out.
using(ClientContext ctx = new ClientContext(siteUrl)
   Web web = ctx.Web;
   TaxonomySession taxonomySession = TaxonomySession.GetTaxonomySession(ctx);
   WebNavigationSettings webNavSettings = new WebNavigationSettings(ctx, web);
   webNavSettings.CurrentNavigation.Source = StandardNavigationSource.PortalProvider;
   webNavSettings.CreateFriendlyUrlsForNewPages = false;

   // show only current site's navigation items in current navigation
   web.AllProperties["__InheritCurrentNavigation"] = "False";
   web.AllProperties["__NavigationOrderingMethod"] = "2";
   web.AllProperties["__NavigationShowSiblings"] = "True";
   web.AllProperties["__CurrentNavigationIncludeTypes"] = "3";
   web.AllProperties["__CurrentDynamicChildLimit"] = "100";

   // show subwebs and pages in navigation
   web.AllProperties["__IncludeSubSitesInNavigation"] = "True";
   web.AllProperties["__IncludePagesInNavigation"] = "True";



WCF Service Binding and Multiple Endpoints

  • webHttpBinding is the REST-style binding, where you basically just hit a URL and get back a truckload of XML or JSON from the web service
  • basicHttpBinding and wsHttpBinding are two SOAP-based bindings which is quite different from REST. SOAP has the advantage of having WSDL and XSD to describe the service, its methods, and the data being passed around in great detail (REST doesn’t have anything like that – yet). On the other hand, you can’t just browse to a wsHttpBinding endpoint with your browser and look at XML – you have to use a SOAP client, e.g. the WcfTestClient or your own app.

So your first decision must be: REST vs. SOAP (or you can expose both types of endpoints from your service – that’s possible, too).

Then, between basicHttpBinding and wsHttpBinding, there differences are as follows:

  • basicHttpBinding is the very basic binding – SOAP 1.1, not much in terms of security, not much else in terms of features – but compatible to just about any SOAP client out there –> great for interoperability, weak on features and security
  • wsHttpBinding is the full-blown binding, which supports a ton of WS-* features and standards – it has lots more security features, you can use sessionful connections, you can use reliable messaging, you can use transactional control – just a lot more stuff, but wsHttpBinding is also a lot *heavier” and adds a lot of overhead to your messages as they travel across the network

Example to expose two binding:

    <binding name="webHttpBindingWithJsonP" crossDomainScriptAccessEnabled="true">
      <security mode="Transport" />
    <binding name="wsHttpBindingConfig">
      <security mode="Transport">
        <transport clientCredentialType="None" />

    <behavior name="EndpBehavior">
      <webHttp />
    <behavior name="wsEndpBehavior">

    <behavior name="ServiceBehavior">
      <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true" />
      <serviceDebug includeExceptionDetailInFaults="true" />
    <behavior name="">
      <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true" />
      <serviceDebug includeExceptionDetailInFaults="false" />

  <service behaviorConfiguration="ServiceBehavior" name="Services.MyService">
    <endpoint name="soap" address="" behaviorConfiguration="WsEndpBehavior" binding="wsHttpBinding"
              bindingConfiguration="wsHttpsBindingConfig" contract="Services.IMyService" />
    <endpoint name="rest" address="/rest" behaviorConfiguration="EndpBehavior" binding="webHttpBinding
              bindingConfiguration="webHttpBindingWithJsonP" contract="Services.IMyService" />

The service interface definition:

I want to make this service can be consume through server-side (C#.Net) and client-side (jQuery & Cross Domain).

Cross Domain javascript will only work with “WebGet”.

public interface IMyService
  [WebGet(RequestFormat = WebMessageFormat.Json,ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Wrapped)]
  bool MyMethod(string msg);

Consume WCF Service

C#.Net with Proxy

WSHttpBinding binding = new WSHttpBinding(SecurityMode.Transport);

EndpointAddress endpoint = new EndpointAddress("");

ChannelFactory<IMyService> factory = new ChannelFactory<IMyService>(binding, endpoint);

IMyService proxy = factory.CreateChannel();

bool value = proxy.MyMethod("test");



C#.Net with Service Reference

When you reference the service on VS, it will gerneate Client Node in your web.config.

  <endpoint address=""
            binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_IMyService"
            contract="ServiceReference.IMyService" name="WSHttpBinding_IMyService" />

ServiceReference.MyServiceClient arsSvc = new ServiceReference.MyServiceClient(); 

jQuery + JSONP

         url: "",
         type: "GET",
         data: { msg: "test" },
         contentType: "application/json; charset=utf-8",
         dataType: "jsonp",
         crossDomain: true,
         success: function (data) { alert(data.MyMethodResult); },
         error: function (jqXHR, textStatus, errorThrown) {
           console.log("Fail " + textStatus + " - " + errorThrown);

SharePoint 2013 – Customize Followed Sites/Documents Left Navigation

Left Navigation (Quick Lanuch)

In personal site, there are several pages are hidden which cannot be viewed through /_layouts/15/sitemanager.aspx; such as /Social/Sites.aspx and /Social/FollowedContent.aspx.

These two pages have delegated SiteMapProvider defined on the page. (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\TEMPLATE\FEATURES\SocialDataStore\SocialDataStoreList)

Sites.aspx ==> MySiteSitesPageStaticProvider

which will only show “Followed Sites” on the left nav

<asp:SiteMapDataSource SiteMapProvider=”MySiteSitesPageStaticProvider” ShowStartingNode=”False”

id=”QuickLaunchSiteMap” StartingNodeUrl=”sid:1025″ runat=”server” />

FollowedContent.aspx ==> MySiteDocumentStaticProvider

which will only show “Followed Documents” on the left nav

<asp:SiteMapDataSource SiteMapProvider=”MySiteDocumentStaticProvider” ShowStartingNode=”False”

id=”QuickLaunchSiteMap” StartingNodeUrl=”sid:1025″ runat=”server” />

Our requirement is to make the left navigation show consistently. Here is our solution:

(1) Replace the static provider with MySiteHostQuickLaunchProvder, which will show all the quick launch items defined in MySiteHost.

<asp:SiteMapDataSource SiteMapProvider=”MySiteHostQuickLaunchProvider” ShowStartingNode=”False”

id=”QuickLaunchSiteMap” StartingNodeUrl=”sid:1025″ runat=”server” />

(2) Add MySiteQuickLaunch after V4QuickLaunchMenu, which will show all the quick launch items defined in personal site.

The result of PlaceHoderLeftNavBar will be :

<asp:Content ContentPlaceHolderId="PlaceHolderLeftNavBar" runat="server">
<a name="startNavigation"></a>
<div class="ms-core-sideNavBox-removeLeftMargin">
           runat="server" />
<div id="js-mysite-userquicklaunch">
<SharePoint:DelegateControl ID="DelegateControl6" runat="server" ControlId="MySiteUserQLDataSource">
<asp:SiteMapDataSource SiteMapProvider="SPNavigationProvider"
                       runat="server" />
<SharePoint:AspMenu id="MySiteUserQLMenu" runat="server"
EnableViewState="false" DataSourceId="MySiteUserQLSiteMap" UseSimpleRendering="true"
UseSeparateCss="false" Orientation="Vertical" StaticDisplayLevels="2"
MaximumDynamicDisplayLevels="0" SkipLinkText="" />

(3) Rename these pages – **.aspx and deploy it to replace the OOTB pages through CSOM or SSOM. Refer to my post for code example:


Overwrite the site page with custom page (Client and Server)- SharePoint 2013

In SharePoint 2013 MySite, there is a static search box in default.aspx. Since we need to apply our custom master page and make sure the search box will only show on the master page, we have to remove the static search box from default page.

Using Client Object Model

 List mpGallery = clientContext.Site.RootWeb.Lists.GetByTitle(folderName);
 FileCreationInformation fci = new FileCreationInformation();
 fci.Content = System.IO.File.ReadAllBytes(filePath);
 fci.Url = newUrl;
 fci.Overwrite = true;

 Microsoft.SharePoint.Client.File newfile = mpGallery.RootFolder.Files.Add(fci);

Using Server Object Model

// Overwrite Default page
SPFileCollection pages = web.RootFolder.Files;
byte[] files = GetPageFromLayout(@"\cibcmysite\default.aspx");
// Creates a file in the collection using the specified URL, a byte array that contains the contents 
of a file, and a Boolean value that specifies whether to overwrite any file that has the same name.
pages.Add("default.aspx", files, true);

// Get Custom default.aspx
public static byte[] GetPageFromLayout(string pageUrl)
       byte[] fileBinary = null;
       var path = SPUtility.GetVersionedGenericSetupPath(@"template\layouts" + pageUrl, 15);
       using (FileStream fs = System.IO.File.OpenRead(path))
            fileBinary = new byte[fs.Length];
            int numToRead = (int)fs.Length;
            int numRead = 0;
            while (numToRead > 0)
                 int n = fs.Read(fileBinary, numRead, numToRead);

                 if (n == 0)

                 numRead += n;
                 numToRead -= n;


        return fileBinary;


Upload User Profile Pictures Programmatically – SharePoint 2013


We are going to migration SharePoint 2007 User profile data to SharePoint 2013. Since we disabled the MySite in SP2007 and created our own custom list to stored user photos, there are only one profile picture per user.

This code will generate three thumbnails using sealed function in Microsoft.Office.Server.UserProfiles.UserProfilePhotos and update the PictureUrl property accordingly.

private void UploadProfileImages(string path, string url)
  using (SPSite site = new SPSite(url))
    SPServiceContext serverContext = SPServiceContext.GetContext(site);
    UserProfileManager userProfileManager = new UserProfileManager(serverContext);

    using (SPWeb web = site.OpenWeb())
      SPFolder subfolderForPictures = web.Folders["User Photos"].SubFolders["Profile Pictures"];
      // Get all image files in a folder
      DirectoryInfo dir = new DirectoryInfo(path);
      FileInfo[] Images = dir.GetFiles();
      foreach (FileInfo img in Images)
        string imageFilePath = path + "\\" + img.Name;
        string accountName = img.Name;

        // if User Profile exists then upload the images
        if (userProfileManager.UserExists(accountName))
          UploadPhoto(accountName, imageFilePath, subfolderForPictures);
          SetPictureUrl(accountName, subfolderForPictures, userProfileManager.GetUserProfile(accountName));
        {    Console.WriteLine("User {0} does not exist.", accountName);  }
      } // end of loop
} // end of function

private void UploadPhoto(string accountName, string imageFilePath, SPFolder subfolderForPictures)
  if (!File.Exists(imageFilePath) || Path.GetExtension(imageFilePath).Equals(".gif"))
  {  Console.WriteLine("File '{0}' does not exist or has invalid extension", imageFilePath);  }
    FileStream file = File.Open(imageFilePath, FileMode.Open);
    BinaryReader reader = new BinaryReader(file);

    if (subfolderForPictures != null)
      // try casting length (long) to int
      byte[] buffer = reader.ReadBytes((int)file.Length);

      int largeThumbnailSize = 300; 
      int mediumThumbnailSize = 72; 
      int smallThumbnailSize = 48; 

      using (MemoryStream stream = new MemoryStream(buffer))
        using (Bitmap bitmap = new Bitmap(stream, true))
          CreateThumbnail(bitmap, largeThumbnailSize, largeThumbnailSize, subfolderForPictures, accountName + "_LThumb.jpg");
          CreateThumbnail(bitmap, mediumThumbnailSize, mediumThumbnailSize, subfolderForPictures, accountName + "_MThumb.jpg");
          CreateThumbnail(bitmap, smallThumbnailSize, smallThumbnailSize, subfolderForPictures, accountName + "_SThumb.jpg");
    Console.WriteLine("Uploading image '{0}' for user '{1}'", imageFilePath, accountName);
} // end of function

/// Get sealed function to generate new thumbernails
public SPFile CreateThumbnail(Bitmap original, int idealWidth, int idealHeight, SPFolder folder, string fileName)
  SPFile file = null;

  Assembly userProfilesAssembly = typeof(UserProfile).Assembly;

  Type userProfilePhotosType = userProfilesAssembly.GetType("Microsoft.Office.Server.UserProfiles.UserProfilePhotos");
  MethodInfo [] mi_methods = userProfilePhotosType.GetMethods(BindingFlags.NonPublic | BindingFlags.Static);

  MethodInfo mi_CreateThumbnail = mi_methods[0];
  if (mi_CreateThumbnail != null)
    file = (SPFile)mi_CreateThumbnail.Invoke(null, new object[] { original, idealWidth, idealHeight, folder, fileName, null });

  return file;
} // end of function

/// Update User Profile Property
private void SetPictureUrl(string accountName, SPFolder subfolderForPictures, UserProfile userProfile)
  string account = accountName.Substring(0, accountName.IndexOf("_")) + "\\" + accountName.Substring(accountName.IndexOf("_") + 1);

  string pictureUrl = String.Format("{0}/{1}/{2}_MThumb.jpg", subfolderForPictures.ParentWeb.Site.Url, subfolderForPictures.Url, accountName);

  userProfile["PictureUrl"].Value = pictureUrl;
} // end of function