Emails to distribution groups do not get delivered while migrating from Exchange 2007 to 2013

I ran into a rather tough issue that got me stumped for a few hours and thought it’s worth blogging.  The problem is that all of a the distribution groups in an environment I’m migrating from Exchange 2007 to 2013 does not get delivered and eventually generates the following bounce back NDR:

Diagnostic information for administrators:
Generating server: exchange2013.contoso.local
Receiving server: exchange2007.contoso.local (192.168.9.2)
secondary@contoso.com
Remote Server at exchange2007.contoso.local (192.168.9.2) returned '400 4.4.7 Message delayed'
9/10/2014 2:54:59 PM - Remote Server at exchange2007.contoso.local (192.168.9.2) returned '441 4.4.1 Error encountered while communicating with primary target IP address: "Failed to connect. Winsock error code: 10061, Win32 error code: 10061." Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts. The last endpoint attempted was 192.168.9.2:25'
Original message headers:
Received: from exchange2013-2.contoso.local (10.10.7.33) by exchange2013.contoso.local
(10.10.7.32) with Microsoft SMTP Server (TLS) id 15.0.847.32; Wed, 10 Sep
2014 08:00:54 -0300
Received: from exchange2013-2.contoso.local ([fe80::e183:a641:f02:ceb1]) by
exchange2013-2.contoso.local ([fe80::e183:a641:f02:ceb1%16]) with mapi id
15.00.0847.030; Wed, 10 Sep 2014 08:00:54 -0300
Content-Type: application/ms-tnef; name="winmail.dat"
Content-Transfer-Encoding: binary
From: Kate Ross <kross@contoso.com>
To: secondary <secondary@contoso.com>
Subject: Wed. Cover/Calendar
Thread-Topic: Wed. Cover/Calendar
Thread-Index: Ac/M5l84/Y7M2h8hSKKr2Unpc4VGhA==
Importance: high
X-Priority: 1
Date: Wed, 10 Sep 2014 08:00:53 -0300
Message-ID: <b1b6f4f8ddd847579cafe082b03e3e85@exchange2013-2.contoso.local>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: <b1b6f4f8ddd847579cafe082b03e3e85@exchange2013-2.contoso.local>
MIME-Version: 1.0
X-Originating-IP: [10.51.1.102]
Return-Path: kross@contoso.com

The troubleshooting steps I’ve performed are:

Upgrading from Exchange 2007 to 2013

Upgrade the distribution groups from Exchange 2007 to 2013 by first listing the groups’ versions with the cmdlet:

Get-DistributionGroup | fl *version

Then executing:

Get-DistributionGroup | Set-DistributionGroup

Once the above cmdlet is executed, the…

Get-DistributionGroup | fl *version

… cmdlet should now display the version as:

ExchangeVersion : 0.10 (14.0.100.0)

Upgrading Distribution Groups to Universal

Upgrading the distribution groups did not appear to correct the issue so after a bit more troubleshooting, I discovered that all of the groups were still of Global type rather than Universal so I went ahead and upgraded them with:

Get-DistributionGroup | where { $_.Grouptype -Like "Global*" } | Set-Group -Universal

Note that if any of the distribution groups have members or are members of Global groups then the associated groups will also need to be upgraded to Universal.

Removing the homeMTA Attribute

After trying the troubleshooting steps above and still encountering the same messages stuck in the queue with the message:

Next Hop Domain: Exchange2007server.domain.local

Delivery Type: SMTP Relay to specified Exchange servers

Status: Retry

With the message details:

Identity: Exchange2013\12736\2598455214130

Subject: Testing123 - No need to reply

Internet Message ID: <ae09691dea2a4c2eb147da76d594518e@BHS-EXMBX-01.domain.local>

From Address: CCS-User@domain.bm

Status: Ready

Size (KB): 7

Message Source Name: SMTP:Default Exchange2013

Source IP: 10.10.7.32

SCL: -1

Date Received: 9/12/2014 10:14:43 PM

Expiration Time: 9/14/2014 10:14:43 PM

Last Error:

Queue ID: Exchange2013\12736

Recipients: staff@domain.bm;2;1;[{LRT=};{LED=};{FQDN=};{IP=}];0;CN=Exchange2007,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Domain,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=local;0

 

I then started combing through the difference between a newly created distribution group I created with the Exchange 2013 ECP and an existing distribution group that wasn’t working. Using the cmdlet:

Get-DistributionGroup "<distribution group name>" | fl

… did not not show any differences side by side so I went ahead and opened adsiedit comparing the attributes which was when I noticed that the new distribution group created with Exchange 2013 did not have a

… while an old distribution group did:

As soon as I cleared the attribute from the problematic distribution group, the messages were delivered.

Import-Module ServerManager

Add-WindowsFeature -Name "RSAT-AD-PowerShell" -IncludeAllSubFeature

Unfortunately, you would need to restart the server before you can import the Active Directory cmdlets with the following import command (I noticed that sometimes I wouldn’t need to run it yet the AD cmdlets would work):

Import-Module ActiveDirectory

Once you have access to the AD cmdlets, you can use the following cmdlet to clear the homeMTA attribute:

Set-ADGroup ‘<distributionGroupName>’ -Clear homeMTA

I originally thought that I could pipe the results from Get-DistributionGroup into Set-ADGroup but that doesn’t work so the way I worked around this was use:

Get-DistributionGroup | FL name > C:\DistroNames.txt

… to get the names of all of the distribution groups in the format:

Name: Distro1

Name: Distro2

… into a text file then opened it in notepad and used a search and replace to format it properly with the Set-ADGroup cmdlet to execute.  Not exactly the most elegant solution but it works.