Distribution and mail-enabled security group members no longer receive e-mails after transitioning / upgrading from Exchange 2003 to 2010

Ran into an interesting problem last month when I noticed that members belonging to distribution and mail-enabled security group no longer receive e-mails after transitioning their Exchange server from 2003 to 2010.

Problem

You’ve noticed that users who belong to distribution groups no longer receive emails sent to those groups so you open up the Message Tracking tool in the Exchange Management Console:

The delivery report shows the following:

Delivery Report for Group - Help Desk ‎(HelpDesk@contoso.bm)‎
Submitted
8/25/2011 2:51 PM contMBX01.contosonet.com
The message was submitted to contcas01.contosonet.com.
Pending
8/25/2011 2:51 PM contcas01.contosonet.com
The message has been queued on server 'contcas01.contosonet.com' since 8/25/2011 2:51:25 PM (UTC-04:00) Atlantic Time (Canada). The last attempt to send the message was at 8/25/2011 3:28:12 PM (UTC-04:00) Atlantic Time (Canada) and generated the error '451 4.4.0 Primary target IP address responded with: "421 4.2.1 Unable to connect." Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.'.

8/25/2011 3:36 PM contcas01.contosonet.com
Message delivery is taking longer than expected. There may be system delays. For more information, contact your helpdesk.

A quick look in the Queue Viewer shows the following details for the message that is stuck in the queue:

The last error reported for the message reads:

400 4.4.7 Message delayed

You open up the problematic distribution group’s properties, navigate to the Membership Approval tab and see that all of the radio button options are grayed out:

Solution

Andy Olson actually blogged about this strange behavior in one of his posts: http://blogs.pointbridge.com/Blogs/olson_andy/Pages/Post.aspx?_ID=13 and because he mentioned that some of his customers had issues if they didn’t upgrade the groups, I thought maybe the issue I was experiencing was related to these Exchange 2003 migrated groups as well.

Since the environment I was working on had quite a few distribution groups, I went ahead and used the cmdlet in PowerShell to upgrade them all at once:

[PS] C:\Windows\system32>Get-DistributionGroup -ResultSize Unlimited | Set-DistributionGroup –ForceUpGrade

You’ll notice that the screenshot above throws quite a few errors and warnings and the reason why is because the environment I was working on had distribution groups with alias’ that had spaces in them.

Notice in the screenshot below that the Exchange 2003 migrated distribution group on the right still referenced the old Exchange 2003 mailbox server:

Go ahead and remove it as shown below:

Once you’ve completed this for the problematic distribution groups, the members belonging to them should start receiving the queued up mail.  The following is the tracking I immediately performed after upgrading the distribution lists:

There is no harm to execute the cmdlet to upgrade the distribution groups again because you will be warned that no changes were made.  I recommend running it again just as a sanity check:

Hope this helps anyone out there who may experience this issue.

Problem with Exchange Server 2010’s default settings for the “Alias” attribute (Active Directory Attribute: “mailNickname”) when creating new mailboxes in bundles

I’m sure many Exchange Server 2010 administrators have encountered this problem and because I rarely handle day-to-day operations (I’m more of a projects consultant), last week would have the only the 3rd time since I’ve had to deal with this issue.  Since I’m sure I’m bound to encounter this again in the future, I thought it would be beneficial to write a blog post about this so I won’t have to review my old notes to figure out how to workaround the problem.

Problem

The E-mail Address Policy in your Exchange Server 2010 organization uses the option: Use alias to set a user’s primary SMTP address:

The problem with this type of E-mail Address Policy is if you create users in bulk as shown in the following screenshots, you end up with the alias set with the format: firstnameLastname so in the case of someone with the name such as John Smith, you will end up with the alias JohnSmith:

 

 

 

**Note how the Alias field is grayed out with the message: “An alias will be automatically generated for each mailbox.

Unfortunately, this setting can be edited if you were creating a single mailbox but not when you are creating multiple mailboxes.

 

This would not be a problem if you had the E-mail Address Policy in your Exchange Server 2010 organization set to the option: First name initial and last name (jsmith):

Solution

I’m a bit surprised that Microsoft never thought of allowing Exchange administrators to have the option of using the user’s login name for the SMTP address as name conflicts are extremely common so I went ahead and reached out to Microsoft about whether there’s a way to change the default format that Exchange Server 2010 uses to generate the alias but was told that it wasn’t possible.  The best way around this is to actually use a script to create the accounts and/or mailbox setting the proper alias.  The less efficient way but will provide you with a quick and dirty fix is to use the ADModify tool to correct the attribute after the mailbox have been created in bulk. I’ll write another blog post to demonstrate the use of a script but in the meantime, I will show how to use the ADModify tool to correct this.

Proceed with downloading the ADModify tool from here:

Launch the tool and select Modify Attributes:

Once the Modify Attributes console is loaded, proceed with selecting the appropriate values for:

1. Domain List
2. Domain Controller List
3. Domain Tree List

For #3, you should navigate to the OU where all or most of the accounts you’ve created are located.  Note that the ADModify tool is quite crude and you’re not able to expand the width of the window if you expand the side of the window so if your DN is long, you won’t be able to see the full string.  Once you have the OU selected, proceed with clicking on the green button with the white arrow pointing to the right to list the user accounts:

Within the window that opens, click on the Custom tab and fill in the:

1. Attribute Name with mailnickname

2. Attribute Value as ‘sAMAccountName’

Click on the Go! button when to set the attribute:

Notice how the alias is now set to the username of the user’s account:

Additional Recommendations

As the ADModify tool isn’t exactly easy to position and resize, what I recommend to make select users easier is to create a sub OU named Temp for the OU that you have created new accounts in, move the new user accounts into the Temp OU then navigate into that OU with the ADModify tool to modify the user attributes.  This makes it much easier to select all of the accounts that need to be modified.  The following is what this process looks like:

Hope this helps anyone looking for a quick way to change a group of users’ alias / mailNickname attribute to their login account names.

Updating a Cisco UCS C Series server with Host Based Upgrade shows CIMC and BIOS as *Unsupported

I recently had to update a new Cisco UCS C250 M1’s firmware to the latest version before it installing VMware ESXi and noticed that when I booted into the Host Based Upgrade package, it showed the CIMC and BIOS components as unsupported.

I usually prefer to use the Host Based Upgrade package because I know exactly what I’m upgrading the firmware to and it takes much less time than using Cisco UCS Server Configuration Utility and while I had the option of proceeding to update all the components anyways, I always prefer to choose the safest route for all production servers.

After thinking the situation through, I had a hunch that the reason why I’m seeing the *Unsupported warning was because the firmware on the server was just too old.  The firmware package I was using was the latest version available:

hostupgrade_v1_3_1d_4.iso

Yet the firmware version for components such as the CIMC was at version 1.0 so the next logical step I did was download an next older Host Based Upgrade package to see if it made a difference:

Long behold it did as it warned me that the existing firmware version was too old and the specific version of firmware that was needed:

So I proceeded to download the firmware that was specified:

… only to realize that it was not a bootable ISO:

This package was obviously not too useful since it required an operating system to be installed in order to upgrade the BIOS.  With that being said, I know that the CIMC web interface allowed upgrading the CIMC with a file so I went ahead to upgrade the CIMC:

After successfully upgrading the CIMC and loading the second oldest Host Based Upgrade package, I noticed that it no longer complained about both the CIMC and BIOS being unsupported:

I didn’t want to risk the possibility of having an unstable server due to an improper firmware upgrade so I went ahead and downloaded the Cisco UCS Server Configuration Utility 2.0 ISO:

As the CIMC did not have internet connectivity, I chose to use the From SCU boot media option to upgrade the firmware:

Since I received no warnings when I selected the option, I proceeded to upgrade the firmware:

Logging back onto the CIMC showed that the BIOS version was updated to 1.2.1b.0:

… so I proceeded to use the original Host Based Upgrade package to boot the server again:

Seeing how there were no warnings, I went ahead and upgraded all of the components:

 

The server has since had ESXi installed and memtest running for the past 3 days and while it probably needs to run memtest much longer to verify that all the memory is operating correctly (it has 96GB), everything looks good so far.

Cursor stuck within console window of a virtual machine through the vSphere client of an VMware ESXi 4.x server

I’ve been meaning to write a post about this annoying problem that I encounter every so often while working with the “open console” window through the vSphere Client on an ESXi 4.x server.  I’m sure others have experienced this at one point or another where you’ve been working with the console window of a virtual machine and for whatever reason you notice that even after you’ve gotten out of the context of the virtual machine console window, your mouse movements are restricted to the bracket of that window.  I’ve experienced this with and without VMware tools installed onto the virtual machine.

The resolution of this problem is actually quite simple and that is to use your keyboard to open up Windows Task Manager and end the vmware-vmrc.exe *32 process:

Though this might be difficult to do with your mouse cursor restricted to a bracket, what I rely on is the following keyboard hot keys:

CTRL + Shift + Esc <— Brings up Windows Task Manager

Tab <— Cycles through the tabs and fields

Space bar <— Executes the button you’ve cycled to with the tab key

------------------------------------------------------------------------------------------------------------------------------------------------

Hope this helps anyone who may be consistently experiencing this issue.

Unable to send e-mails from a Blackberry with event ID: 20209 warnings logged on the BES 4.x server’s application logs

I ran into an issue today that I have come across before but did not remember how to fix it and since I know I’m bound to run into this situation again, I thought it would be a good idea to blog it.

Problem

You have a user who has a Blackberry that is unable to send e-mail messages but is able to receive messages.  Logging onto the server and reviewing the application logs shows that several event ID: 20209 warnings are logged stating the following:

{UserFirstName UserLastName} DecryptDecompress() failed, Tag=159789, Error=604

Solution

The reason why this message would be logged is if there is a problem with the encryption key between the device and BES and to correct this problem, all you need to do is manually regenerate the encryption key on the user’s Blackberry by navigating to Options –> Security Options –> Information then scroll down to the Services section, highlight the Desktop line item, click and select Regenerate Encryption Key. This will force a regeneration of the encryption key so the user can send emails from their Blackberry again.