Skip to main content

Project Server 2010 -> 2013 migration - Large databases -> Slow Convertto-SPProjectDatabase

The last couple of days i have tried to migrate a Project Server 2010 to Project Server 2013. The Project Server 2010 contained some quite large databases compared to what I normally encounter.
The shrinked database sizes was:
 - Archive:      75 GB
 - Draft:        40 GB
 - Published:    50 GB
 - Reporting:    20 GB
 - Content:       2 GB
As you properly already know the migration process contains the following steps:

 Mount-SPContentDatabase –Name ProjectServer_PWA –WebApplication http://projectserver/ -NoB2BSiteUpgrade  
 $wa = get-SPWebApplication http://projectserver  
 $wa.UseClaimsAuthentication = $true  
 $wa.Update()  
 (Get-SPWebApplication http://projectserver).migrateUsers($true)  
 Set-SPSite –Identity http://projectserver/TECHPWA -SecondaryOwnerAlias "CONTOSO\Administrator"  
 Upgrade-SPSite -Identity http://projectserver/TECHPWA -versionupgrade  
 Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver projectserver -ArchiveDbname TechPWA_Archive -DraftDbname TechPWA_Draft –PublishedDbname TechPWA_Published –ReportingDbname TechPWA_Reporting –ProjectServiceDbname TechPWA_ProjectWebApp  
 Mount-SPProjectDatabase –Name "ProjectServer_PWA_ProjectWebApp" –WebApplication http://projectserver  
 Upgrade-SPProjectDatabase -Name "ProjectServer_PWA_ProjectWebApp" -WebApplication http://projectserver  
 Mount-SPProjectWebInstance -DatabaseName "ProjectServer_PWA_ProjectWebApp" -SiteCollection " http://projectserver/TechPWA" -DatabaseServer "projectserver"  
 Upgrade-SPProjectWebInstance -Identity "http://projectserver/TechPWA"  
 Enable-SPFeature -Identity TechPWAsite -URL "http://projectserver/TechPWA"  
 Test-SPProjectWebInstance –Identity http://projectserver/TechPWA  

At the step where I consolidate the four Project Server 2010 databases (Convertto-SPProjectDatabase) I started to encounter some problems.

Firstly the new ProjectWebApp database started at the size of 5 MB and slowly increased with 5 MB every 30 sec until it reached 40 MB.
Now the size jumped to 54.000 MB and everything stalled. The database size was static, the memory did not increase, the CPU was idle and the Powershell just keep hanging. for this reason i tried to restart the Powershell command several times also on different servers, always the same result.
After around 1½ day the Convertto-SPProjectDatabase finally finished with an error:

 Convertto-SPProjectDatabase : INIT METADATA ERROR!  
 At least one mismatch found between SQL schema for view  
 MSP_EpmProject_UserView and reporting metadata. Schema column: Project Number.  
 Metadata column: NULL  
 Call to MSP_Epm_CheckIntrinsicAttributes failed for Projects  
 INIT METADATA ERROR.  
 At line:1 char:1  
 + Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver  
 ppm2013 ...  
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
 ~~~  
   + CategoryInfo     : InvalidOperation: (Microsoft.Offic...baseImpleme  
   ntor:ConvertToProjec...baseImplementor) [ConvertTo-SPProjectDatabase], Sql  
  Exception  
   + FullyQualifiedErrorId : Microsoft.Office.Project.Server.Cmdlet.PSCmdletC  
   onvertToProjectServiceDatabase  

I do not know the reason for the error (somthing with the schema for a custom field column not being correct). However, simply modify the stored procedure "MSP_Epm_CheckIntrinsicAttributes" to never return an error solved the problem.
So if you encounter this problem, simply execute the modified version of the Stored Procedure below:

 USE [ProjectServer_ProjectWebApp]/[ProjectServer_Reporting]
 GO  
 /****** Object: StoredProcedure [dbo].[MSP_Epm_CheckIntrinsicAttributes]  Script Date: 5/9/2015 4:02:14 PM ******/  
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 ALTER PROCEDURE [dbo].[MSP_Epm_CheckIntrinsicAttributes]  
   @EntityTypeUID       [uniqueidentifier],   
   @AttributeIsTimephased   [FLAG]          
 AS  
 BEGIN  
   DECLARE @TableName     [NAME]  
   DECLARE @UserViewName   [NAME]  
   DECLARE @OlapViewName   [NAME]  
   DECLARE @ErrorMessage   nvarchar(1000)  
   DECLARE @SchemaColumn   [NAME]  
   DECLARE @MetadataColumn  [NAME]  
   IF @AttributeIsTimephased = 0  
   BEGIN  
    SELECT @TableName = EntityNonTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   ELSE  
   BEGIN  
    SELECT @TableName = EntityTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   RETURN 0  
 LblError:  
   --PRINT @ErrorMessage  
   RETURN 0  
 END  

This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems.

Comments

  1. Great tutorial, Loved this blog. Thanks aand keep sharing this kind of stuffs here. A2hosting Review

    ReplyDelete
  2. Hi, thanks for your blog, i'm faced with this problem now and cant understand you write "This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems" But there are 4 bases draft, reporting, archive and published. Which one of them do you call ProjectWebApp in 2010?

    ReplyDelete

Post a Comment

Popular posts from this blog

Sharepoint 2013 - Fullscreen mode/hide quick launch

If you want to hide the quick launch on a SharePoint page or web part page in 2013 you could of cause use CSS scripts as in 2010.
But as a new thing in 2013 the user have the possibility to click on full screen mode. so why not load your page in this mode and then the user can choose to exit the mode if he prefers.

Add a content editor webpart to your page and click on Edit HTML from the ribbon.
Add the following code:

<script type="text/javascript">
window.onload = function()
{
  SetFullScreenMode(true);
}
</script>

Install DLL in GAC - Windows 2008/2012 (Using Powershell, No GacUtil.exe)

If you want to install a DLL in the GAC and do not have the GACUtil.exe available. Powershell is properly the easiest way to procede.

Before Powershell you would properly just drag the DLL file into the C:\Windows\Assembly but this option is usually not available anymore.

Powershell - Add DLL to GAC So to install a DLL file in the GAC simply execute the below Powershell script. Remember you migth want to run the Powershell prompt as an administrator.
[System.Reflection.Assembly]::Load("System.EnterpriseServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")$publish=New-ObjectSystem.EnterpriseServices.Internal.Publish$publish.GacInstall("c:\temp\MyDllFile.dll")iisreset
The first line adds a reference to the assembly we need to be able to mange the GAC.
The second and third lines retrives the GAC object and publish a new DLL file to it.
The last line resets the Internet Information Services. This is only needed if your DLL file is used in a websi…

Reporting on Microsoft Planner or Microsoft Staffhub!

Reporting on Planner or StaffHub!


For a long time I have been waiting for the reporting API for Planner and Staffhub. Today I got tired of waiting and found another way to do it.

I manage to create a PowerBI report based on all the planner plans I have access to and the same method will work for Staffhub and properly many other services such as Microsoft Tasks, Microsoft Forms, etc. 


How?Easy, but requires some Azure, Flow and PowerBI knowledge. These are the steps to I used: 1.Create an Azure Blob to hold the data 2.Create a Flow to save Planner data in the Blob 3.Access the Blob from PowerBI Azure – Blob!So first, lets create a Blob. No, not that kind of Blob. Access the Azure portal - https://portal.azure.com/ Select New and search for “storage account” Fill out the form and click create. I kept all settings to their default Access the new storage and locate/copy the Storage Account Name and Access Key – you will need this for connections to Flow and PowerBI. Next let’s create a Blob Container an…