Requesting certificate for Microsoft Lync 2010 Pool with customized certificate template for Avaya RCC integration throws the error: “The certificate request with id is denied.”

Those who have come across my previous post about Microsoft Lync 2010 and Avaya RCC integration:

Lync Server 2010 & Avaya RCC Integration? Not yet as of June 7, 2011
http://terenceluk.blogspot.com/2011/06/lync-server-2010-avaya-rcc-integration.html

… may remember that I was working with a client who had an Avaya AES server that was more than 2 versions behind the current offered by Avaya which cause the new Lync deployment project I was in to come to a screeching halt.  Fast forward to almost a year later, we finally got approval for the new upgrade licenses and have thus upgraded the AES (Application Enablement Services) server to the latest version: r6-1-2-32-0.

I was extremely anxious to get back into the client’s office to retry getting the RCC going and also managed to find an Avaya document which provides step-by-step instructions:

Implementation notes on Integration of Avaya Aura Application Enablement ® Services with Microsoft® Lync® 2010 Server.
http://support.avaya.com/css/P8/documents/100144425

As straight forward as the instructions were, I did run into an issue from the beginning where I was required to request a certificate with the Lync certificate request tool.  The challenge here was that I had to use to the certificate request tool to request a certificate with a customized template that included Client Authentication (the regular Web Server template only included Server Authentication).  What I immediately found was that as soon as I tried to request the certificate for the Lync standard server, I would get the following error:

Request CertificateRequest-CSCertificate -New -Type Default,WebServicesInternal,WebServicesExternal -CA "svrcert02.contoso.internal\contoso Re" -Country BM -State "Hamilton" -City "Hamilton" -FriendlyName "svrlyncstd01" -KeySize 1024 -PrivateKeyExportable $False -Organization "contoso" -OU "IT" -DomainName "sip.contosore.com" -Template "Avaya AES" -Verbose -Report "C:\Users\tluk\AppData\Local\Temp\2\Request-CSCertificate-[2012_03_21][15_41_34].html"Creating new log file "C:\Users\tluk\AppData\Local\Temp\2\Request-CSCertificate-4ef9b765-da53-4c72-b955-0958f1e944eb.xml".Create a certificate request based on Lync Server configuration for this computer.Request with id  denied for use Default,WebServicesInternal,WebServicesExternalNo changes were made to the Central Management Store.Creating new log file "C:\Users\tluk\AppData\Local\Temp\2\Request-CSCertificate-[2012_03_21][15_41_34].html".Warning: Request-CSCertificate encountered errors. Consult the log file for a detailed analysis, and ensure all errors (1) and warnings (0) are addressed before continuing.Warning: Detailed results can be found at "C:\Users\tluk\AppData\Local\Temp\2\Request-CSCertificate-[2012_03_21][15_41_34].html".

The certificate request with id  is denied.

Those who have come across the The certificate request with id is denied. error would know that this error message usually includes number after the id but in my case, it didn’t.  Reviewing the Microsoft Lync Server 2010 Deployment Log gave me a bit more information as to why the request failed but solution to the problem wasn’t obvious:

Error: Denied by Policy Module 0x80094800, The request was for a certificate template that is not supported by the Active Directory Certificate Services policy: Avaya AES.

I’ve also gone ahead to review the certificate authority’s issued certificates and noticed that no certificate was issued after the error was thrown:

From here, I went ahead to review the settings of the template and noticed that the Minimum key size for the template was set to 2048 and not 1024 as the guide instructs you to use so I went ahead and changed it:

I also went ahead to compare the Web Server template with the Avaya AES custom template I created and other than a few extra properties, I didn’t notice anything out of the ordinary:

Reviewing the certificate authority’s event logs showed a warning message that was similar to the error I got from the Microsoft Lync Server 2010 Deployment Log:

To confirm that it was indeed the certificate template I was using which was the problem, I went ahead to try and use the Lync certificate tool to issue a certificate but to use the default Web Server template and noticed that the request would complete without errors.

After fiddling around with the encryption bit from 1024 to 2048 and other settings I could think of, I gave up and went directly to the certificate authority’s /certsrv webpage to try and request the certificate from there:

Note that the settings I used which is worth mentioning are:

CSP: Microsoft RSA SChannel Cryptographic Provider

Key Size: 2048

One I got the certificate to successfully issue, I went ahead to assign the certificate to the Lync pool, completed the PowerShell cmdlets to create the static routes, trusted application pools, etc and immediately got RCC going.

Note that I am aware that the document clearly states the following for the certificate:

On the Name and Security Settings screen, enter a friendly name and change the bit length by
selecting 1024 in the dropdown. Select Next.

Yet shows that the Key Size in the summary screen as 2048:

I’m not exactly sure what key size we’re supposed to use by the instructions appear to suggest it’s 1024 and not 2048.  The only concern I initially had was that the Avaya AES server does not support key sizes larger than 1024 but since RCC is working, I supposed it does.

Hope this helps anyone out there who may come across this problem.