Skip to main content

Project Server 2010 -> 2013 migration - Large databases -> Slow Convertto-SPProjectDatabase

The last couple of days i have tried to migrate a Project Server 2010 to Project Server 2013. The Project Server 2010 contained some quite large databases compared to what I normally encounter.
The shrinked database sizes was:
 - Archive:      75 GB
 - Draft:        40 GB
 - Published:    50 GB
 - Reporting:    20 GB
 - Content:       2 GB
As you properly already know the migration process contains the following steps:

 Mount-SPContentDatabase –Name ProjectServer_PWA –WebApplication http://projectserver/ -NoB2BSiteUpgrade  
 $wa = get-SPWebApplication http://projectserver  
 $wa.UseClaimsAuthentication = $true  
 $wa.Update()  
 (Get-SPWebApplication http://projectserver).migrateUsers($true)  
 Set-SPSite –Identity http://projectserver/TECHPWA -SecondaryOwnerAlias "CONTOSO\Administrator"  
 Upgrade-SPSite -Identity http://projectserver/TECHPWA -versionupgrade  
 Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver projectserver -ArchiveDbname TechPWA_Archive -DraftDbname TechPWA_Draft –PublishedDbname TechPWA_Published –ReportingDbname TechPWA_Reporting –ProjectServiceDbname TechPWA_ProjectWebApp  
 Mount-SPProjectDatabase –Name "ProjectServer_PWA_ProjectWebApp" –WebApplication http://projectserver  
 Upgrade-SPProjectDatabase -Name "ProjectServer_PWA_ProjectWebApp" -WebApplication http://projectserver  
 Mount-SPProjectWebInstance -DatabaseName "ProjectServer_PWA_ProjectWebApp" -SiteCollection " http://projectserver/TechPWA" -DatabaseServer "projectserver"  
 Upgrade-SPProjectWebInstance -Identity "http://projectserver/TechPWA"  
 Enable-SPFeature -Identity TechPWAsite -URL "http://projectserver/TechPWA"  
 Test-SPProjectWebInstance –Identity http://projectserver/TechPWA  

At the step where I consolidate the four Project Server 2010 databases (Convertto-SPProjectDatabase) I started to encounter some problems.

Firstly the new ProjectWebApp database started at the size of 5 MB and slowly increased with 5 MB every 30 sec until it reached 40 MB.
Now the size jumped to 54.000 MB and everything stalled. The database size was static, the memory did not increase, the CPU was idle and the Powershell just keep hanging. for this reason i tried to restart the Powershell command several times also on different servers, always the same result.
After around 1½ day the Convertto-SPProjectDatabase finally finished with an error:

 Convertto-SPProjectDatabase : INIT METADATA ERROR!  
 At least one mismatch found between SQL schema for view  
 MSP_EpmProject_UserView and reporting metadata. Schema column: Project Number.  
 Metadata column: NULL  
 Call to MSP_Epm_CheckIntrinsicAttributes failed for Projects  
 INIT METADATA ERROR.  
 At line:1 char:1  
 + Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver  
 ppm2013 ...  
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
 ~~~  
   + CategoryInfo     : InvalidOperation: (Microsoft.Offic...baseImpleme  
   ntor:ConvertToProjec...baseImplementor) [ConvertTo-SPProjectDatabase], Sql  
  Exception  
   + FullyQualifiedErrorId : Microsoft.Office.Project.Server.Cmdlet.PSCmdletC  
   onvertToProjectServiceDatabase  

I do not know the reason for the error (somthing with the schema for a custom field column not being correct). However, simply modify the stored procedure "MSP_Epm_CheckIntrinsicAttributes" to never return an error solved the problem.
So if you encounter this problem, simply execute the modified version of the Stored Procedure below:

 USE [ProjectServer_ProjectWebApp]/[ProjectServer_Reporting]
 GO  
 /****** Object: StoredProcedure [dbo].[MSP_Epm_CheckIntrinsicAttributes]  Script Date: 5/9/2015 4:02:14 PM ******/  
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 ALTER PROCEDURE [dbo].[MSP_Epm_CheckIntrinsicAttributes]  
   @EntityTypeUID       [uniqueidentifier],   
   @AttributeIsTimephased   [FLAG]          
 AS  
 BEGIN  
   DECLARE @TableName     [NAME]  
   DECLARE @UserViewName   [NAME]  
   DECLARE @OlapViewName   [NAME]  
   DECLARE @ErrorMessage   nvarchar(1000)  
   DECLARE @SchemaColumn   [NAME]  
   DECLARE @MetadataColumn  [NAME]  
   IF @AttributeIsTimephased = 0  
   BEGIN  
    SELECT @TableName = EntityNonTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   ELSE  
   BEGIN  
    SELECT @TableName = EntityTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   RETURN 0  
 LblError:  
   --PRINT @ErrorMessage  
   RETURN 0  
 END  

This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems.

Comments

  1. Great tutorial, Loved this blog. Thanks aand keep sharing this kind of stuffs here. A2hosting Review

    ReplyDelete
  2. Hi, thanks for your blog, i'm faced with this problem now and cant understand you write "This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems" But there are 4 bases draft, reporting, archive and published. Which one of them do you call ProjectWebApp in 2010?

    ReplyDelete

Post a Comment

Popular posts from this blog

Azure DevOps - Gantt Chart

It's been a while since my last post - in the past couple of weeks I have played around with some videos of topics I find interesting. One of these topics are a very cool way of displaying a Gantt Chart upon your Azure DevOps board's. Check it out here!

Sharepoint/Project Server App with javascript and Web Part

Getting Project Server code to run in a AppPart Today I tried to get project server CSOM/javascript code to run within an app part. This caused a lot of troubleshooting and guessing to get it to work. By simply adding the ps.js library to the app part page I kept getting an error. Library: <script type="text/javascript" src="/_layouts/15/ps.js"></script> Error: "executeordelayuntilscriptloaded is undefined" Then I tried to use ScriptLink to load the library but got the same error. <SharePoint:ScriptLink runat="server" Name="ps.js" Localizable="false" OnDemand="False" LoadAfterUI="True"></SharePoint:ScriptLink> The code I was trying to run was very simpel and worked fine if I added it to a normal aspx page.  $(document).ready(function () {     var projContext = PS.ProjectContext.get_current();       var projects = projContext.get_projects();     projContext.load(projects...

PowerShell results size unlimited/truncated - $FormatEnumerationLimit/Width

Ever experienced the problem where you run a Powershell command and you cannot see the whole result because the result is truncated. Problem: If you for example run the Test-SPsite command you might see something like the following: Site : SPSite Url=http://atlas/pwa Results : { SPSiteHealthResult Status=Passed RuleName="Conflicting Content Types" RuleId=befe203b-a8c0-48c2-b5f0-27c10f9e1622, SPSiteHealthResult Status=FailedWarning RuleName="Customized Files" RuleId=cd839b0d-9707-4950-8fac-f306cb920f6c, SPSiteHealthResult Status=Passed RuleName="Missing Galleries" RuleId=ee967197-ccbe-4c00-88e4-e6fab81145e1, SPSiteHealthResult Status=Passed RuleName="Missing Parent Content Types" RuleId=a9a6769f-7289-4b9f-ae7f-5db4b997d284, SPSiteHealthResult Status=FailedError RuleName="Missing Site Templates" RuleId=5258ccf5-e7d6-4df7-b8ae-12fcc0513ebd, ...