Migrate SharePoint List From On-Premises To SharePoint Online

Introduction 

 
Hi guys. Sometimes we want to migrate a list from SharePoint on-premises to SharePoint online without using any third-party tools, by retaining the ID, Created, Modified, CreatedBy and ModifiedBy column values along with their versions. This article will help you to accomplish the above requirement.
 
We will use the Sharepoint migration API PowerShell commands for migrating the list from on-premises to online. To learn more, visit this link.
 
Prerequisites
  • You should have an Azure account. If you don't have one, go and create the trial version for 30 days here
  • Source List
  • Destination List with the similar column as of source list 
These are the steps required to migrate an on-premises list online.
  • Create a storage account in Azure.
  • Export the on-premises list
  • Import the list on SharePoint online.

Create a storage account in azure

 
Log in to the Azure portal by using this URL: http://portal.azure.com/ with valid credentials. 
 
Go to Storage and Click on add create link.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Enter the following value in required fields and click on the review + create button.
 
Migrate SharePoint List From onpremises To SharePoint Online
Migrate SharePoint List From onpremises To SharePoint Online
 
Now click on the create button once the validation has passed.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Now wait for some time until the deployment completes.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Once the storage account is created, navigate to newly created storageaccount listmigrationapiarvind-> Access key, and copied the key and account name and save it somewhere else. The storage account is listmigrationapiarvind and the key will be required to upload the source list content here.
 
Migrate SharePoint List From onpremises To SharePoint Online
 

Export the on-premises list

 
To demonstrate the export and import using migration API, I created the list 'MigrationList' with versioning enabled on SharePoint on-premises site.
 
I had also created the list on the SharePoint online site using save as list template functionality with version enabled.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
A total of 5 items exist in the on-premises 'MigrationList' list.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
The version history of Item ID 1 is:
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Open the SharePoint PowerShell window and paste the following code:
 
 Export-SPWeb -Identity "Your Site Url" -ItemUrl "Lists/MigrationList" -Path "D:\Arvind\MigrationListContent" -NoFileCompression -IncludeVersions ALL
 
Migrate SharePoint List From onpremises To SharePoint Online
 
This command will export all the necessary files required for the SharePoint online site. The folder D:\Arvind\MigrationListContent will have the following files.
 
Migrate SharePoint List From onpremises To SharePoint Online
 

Import the list on SharePoint Online

 
We will import all the items of source list into SharePoint online using migration API. 
 
Firstly, we will convert the package into the required format for a SharePoint online site.
 
Note
I copied all the exported files to this path D:\Arvind\MigrateContent\MigartionList\sourceFile
 
Open SharePoint online management shell window on target site and paste this script. It will prompt for password, provide the password and press enter.
  1. $userName = "Your target site user name"  
  2. $cred = Get-Credential $userName  
  3. $sourceFilePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
  4. $sourcePackagePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
  5. $outPutPackagePath ="D:\Arvind\MigrateContent\MigartionList\outputPackage"  
  6. $targetWebUrl ="Your target webUrl"  
  7. $targetListName ="Lists/MigrationList"  
  8.   
  9. ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFilePath -SourcePackagePath $sourcePackagePath -OutputPackagePath $outPutPackagePath -TargetWebUrl $targetWebUrl -TargetListPath $targetListName -Credentials $cred -Verbose  
where,
  • SourceFilesPath : The content you want to migrate.
  • SourcePackagePath : The content you want to migrate.
  • OutPutPackagePath : Your tempory folder path.
  • TargetWebUrl : Your target web url
  • TargetListPath : Your targetList Name
  • Credentials: Valid Credentials of your target site.
Migrate SharePoint List From onpremises To SharePoint Online
 
Once the package has been converted successfully, it will look like:
 
Migrate SharePoint List From onpremises To SharePoint Online
 
After converting the source files into required output package files, upload the files to an Azure storage account with these scripts.
  1. #Upload the files and package to azure.  
  2. $fileContainerName ="migratelistcontainer" #Not Necessary  
  3. $packageContainerName ="migratelistpackage" #Not Necessary  
  4. $azureAccountName ="listmigrationapiarvind"  
  5. $azureAccountKey = "ozuKmwHT/5MDEeyiVz8FF0QpEh5sji1IlJmsH4UsfdYgn4gNBDx/I1+HLb0klQeT1wWquX6YfB5m/pJsi6Yqog=="  
  6. $azureQueueName = "migratelistqueue" #Not Required  
  7.   
  8. $azureLocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFilePath  -SourcePackagePath $outPutPackagePath -FileContainerName $fileContainerName -PackageContainerName $packageContainerName -AccountName $azureAccountName -AccountKey $azureAccountKey -Overwrite -AzureQueueName $azureQueueName  
Where:
  • SourceFilesPath: Path of Source file i.e. SharePoint on-premises export files path.
  • SourcePackagePath: Path of convert file i.e. Path of convert package.
  • FileContainerName: (Optional) It is an Azure storage container name.
  • PackageContainerName: (Optional) It is an Azure storage package container name.
  • AzureQueueName: (Optional) It is an Azure queue name.
  • AccountName: It is an Azure account name i.e. listmigrationapiarvind
  • AccountKey: It is an Azure account storage key access i.e. ozuKmwHT/5MDEeyiVz8FF0QpEh5sji1IlJmsH4UsfdYgn4gNBDx/I1+HLb0klQeT1wWquX6YfB5m/pJsi6Yqog==
Migrate SharePoint List From onpremises To SharePoint Online
 
It will upload the package into Azure storage container. To check it, log into the azure site with valid credentials, and navigate to your storage account(listmigrationapiarvind) Containers.
 
Migrate SharePoint List From onpremises To SharePoint Online
 
The migratelistpackage contains the following files:
 
Migrate SharePoint List From onpremises To SharePoint Online
 
The command $azureLocations | flis optional but it help us to check the following file uri i.e. FileContainerUri, PackageContainerUri, ReportingQueueUri and FileContainerUploadUri 
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Now submit the migration job with below command, so that it starts importing our on-premises list items into the targeted SharePoint online list.
 
Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azureLocations -Credentials $cred 
 
Where:
  • TargetWebUrl: Your target site Url i.e. Already created the variable in the above command which holds the target web URL value.
  • MigrationPackageAzureLocations: Above, set the variable value.
Migrate SharePoint List From onpremises To SharePoint Online
 
It is an optional command, but helps us to monitor the migration job status. 
 
Get-SPOMigrationJobStatus -TargetWebUrl $targetWebUrl -Credentials $cred
 
Migrate SharePoint List From onpremises To SharePoint Online
 
Once the migration job processing completes, verify the target list items and their version history.
 
Migrate SharePoint List From onpremises To SharePoint Online 
Migrate SharePoint List From onpremises To SharePoint Online
Note
You can check the job success or any job-related errors of submit a migration job on the Azure site queue.
 
To check the Azure queue, login to the azure site and navigate to your storage account (listmigrationapiarvind) then queue (migratelistqueue).
 
Migrate SharePoint List From onpremises To SharePoint Online
 
The whole script will look like:
  1. # Export the onpremises list content  
  2. Export-SPWeb -Identity "Your On Premises Site Url" -ItemUrl "Lists/MigrationList" -Path "D:\Arvind\MigrationListContent" -NoFileCompression -IncludeVersions ALL  
  3.   
  4. # create variable to hold value which requried to convert the source files  
  5. ########################################  
  6. # Convert the .dat files into proper packages.  
  7.   
  8. $userName = "Your targeted site user Name"  
  9. $cred = Get-Credential $userName  
  10. $sourceFilePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
  11. $sourcePackagePath = "D:\Arvind\MigrateContent\MigartionList\sourceFile"  
  12. $outPutPackagePath ="D:\Arvind\MigrateContent\MigartionList\outputPackage"  
  13. $targetWebUrl ="Your targeted site Url"  
  14. $targetListName ="Lists/MigrationList"  
  15.   
  16. # Convert the Source package into target site.  
  17.   
  18.  ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath $sourceFilePath -SourcePackagePath $sourcePackagePath -OutputPackagePath $outPutPackagePath -TargetWebUrl $targetWebUrl -TargetListPath $targetListName -Credentials $cred  
  19.   
  20. # Create azure variables and upload the packages on azure.  
  21. ####################################  
  22. #Upload the files and package to azure.  
  23. $fileContainerName ="migratelistcontainer" #Not Necessary  
  24. $packageContainerName ="migratelistpackage" #Not Necessary  
  25. $azureAccountName ="listmigrationapiarvind"  
  26. $azureAccountKey = "ozuKmwHT/5MDEeyiVz8FF0QpEh5sji1IlJmsH4UsfdYgn4gNBDx/I1+HLb0klQeT1wWquX6YfB5m/pJsi6Yqog=="  
  27. $azureQueueName = "migratelistqueue" #Not Required  
  28.   
  29. $azureLocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath $sourceFilePath  -SourcePackagePath $outPutPackagePath -FileContainerName $fileContainerName -PackageContainerName $packageContainerName -AccountName $azureAccountName -AccountKey $azureAccountKey -Overwrite -AzureQueueName $azureQueueName  
  30.   
  31. # Check the uploaded files Uri  
  32. $azureLocations | fl  
  33.   
  34. ############################################  
  35.   
  36. #Submit the job  
  37.   
  38. Submit-SPOMigrationJob -TargetWebUrl $targetWebUrl -MigrationPackageAzureLocations $azureLocations -Credentials $cred  
  39.   
  40. #Check the job status with this command  
  41. Get-SPOMigrationJobStatus -TargetWebUrl $targetWebUrl -Credentials $cred  
You can also download the script from the attachment.
 
I hope you liked this article.