The nominated Exchange Server for offline address book has been deleted

Recently I was installing a new Exchange 2010 SP2 server into an Exchange 2010 'Enterprise' Hosting Test Lab and came across this error:

I have never seen this error before, it interested my hugely as I didn't think this was actually possible to let happen. Anyway I thought, ok so let's clean the offending OAL up and then I can continue the install.

So I check the Exchange Management Console:

And I check the Exchange Management Shell:

Very strange.. I cannot see the offending OAL anywhere. So I can't delete it nor fix it to continue the setup.

So I fire up ADSIEdit:

Fantastic I can see it. The reason I can see it is because it obviously exists, Exchange wouldn't be complaining about it otherwise, but when looking in the configuration container like I am above I see every object that is held within there, whether it is considered corrupt by Exchange or not.

What appears to have happened is that because the OAL has an issue, that issue being the home server for it doesn't exist anymore, Exchange is not showing it within the EMC or Powershell and thus it isn't fixable from those management tools.

In this instance I knew this was a test OAL and not needed, so I simply deleted it via ADSIEdit (right click, delete).

So can I continue the install now?

You bet, the issue is resolved.


Finally if this is a production environment you could open the object and find the 'offLineABServer' attribute and modify it to a server that does exist in the Org. This should be enough to allow it to show in the EMC or Powershell, where you can then update it and force a rebuild. But if you don't want to do that the above method is safe to use but remember you will have to recreate the object and apply it to databases and/or users ensuring your provisioning engine will be ok with that and continue to reference the newly created OAL with the old.

I hope this helps anyone else that runs into this obscure issue.

Take care,

Oliver Moazzezi MVP - Exchange Server

Tenant Isolation in a Hosted Exchange Environment

With the removal of the /hosting edition of Exchange Server and the reintroduction and support of hosting using the /enterprise version of Exchange Server I thought it a good idea to cover some topics that help an IT Admin implement certain features for tenant isolation into the /enterprise edition of the product.

A feature a tenant may require is tenant or semi-tenant isolation. This is where the tenant cannot contact any other tenants in the Hosted Org. A customer may require total lockdown or require just a subset of users have this restriction in place.

Luckily with Transport Rules and some Powershell magic we can implement this feature. In a perfect world you will have a front end or control panel that can configure this magic for you, but if your control panel is one step behind, this is the backend steps you need to take to implement it.


Firstly you will need a Transport Rule to disallow the tenant or certain users within the tenant, to send outside of the Organization.

If you are using HMC, or have a framework that works along the lines of HMC, for example, an AllUsers and AllAdminUsers groups, then we can lock a tenant down with these security groups. If however only part of a tenant is to be isolated, then we will need to create another group, adding the required users to that group and then potentially hiding that group from the GAL so as not to mess up a nice clean address book view. So back on track..

..This is simply achieved with a transport rule like the following:

New-TransportRule -Name 'Tenant Isolation - Outside Org Test' -Comments 'Another Transport rule to block my tenant from sending outside of the org. Entirely a test.' -Conditions 'Microsoft.Exchange.MessagingPolicies.Rules.Tasks.FromMemberOfPredicate','Microsoft.Exchange.MessagingPolicies.Rules.Tasks.SentToScopePredicate' -Actions 'Microsoft.Exchange.MessagingPolicies.Rules.Tasks.DeleteMessageAction' -Enabled $true -Priority 'number'

This looks like so in the UI:

However on top of this you also need to provide a way of locking down messaging within the Org. As The Enterprise version of Exchange has no concept of 'tenant's, you need to build another Transport with an Exception.

The above rule then allows the tenant to recieve emails from the users within but stops communication to other tenants on the hosted platform.

We can then look to modify the rules further, maybe for example by modifying 'silently drop the message' to actually inform the user the message was indeed refused with a custom error.


So the above controls outbound messages, but we still need to be able to control who can send messages to the isolated tenant or the isolated users. We can again use Transport Rules, but again we would need to create two, one for inside the organization and another for outside. In this instance I prefer to perform the lock down on the user object itself, this creates less transport overhead on your messaging subsystem.

Finally, if your hosting framework cannot support message delivery restrictions then you can tie this into your tenants OU and selected users with a little powershell. This can be run as a scheduled task against the tenant as a ps script. The below is an example I have made and use but will need some modification.

#Allow users to only recieve messages from a specified DL
#Modify $NIS and $AllUsers as appropriate
#Test in Test Lab prior to running in Live
#Confirm $NIS and $AllUsers are valid targets with Get-DistributionGroup
#Replace $NIS with a 'No Internet Sending Security Group' created in Control Panel for the Tenant
#Replace $AllUsers with the HMC4.5/hosting framework AllUsers Group for the Tenant

Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010Get-DistributionGroupMember $NIS set-mailbox -AcceptMessagesOnlyFromDLMembers $AllUsers

The end result is as follows:

Isolated tenant/user sending out to the internet:

Isolated tenant/user sending to another tenant in the Hosted Exchange platform:

Another tenant or external user trying to send to the isolated tenant/user:

Take Care


Oliver Moazzezi MVP - Exchange Server

U-turn on Exchange 2010 SP1 /hosting mode guidance?

If you have been following Exchange 2010 options for hosters you will know that until recently you were pretty much pushed down the /hosting mode route. Well today I spotted this blog post by Ian Hameroff and Michael van Dijken, So, You Want to Host Exchange?, suggesting we don’t have to use /hosting if we want UM and some other functionality.

As with Exchange 2000, 2003 and 2007 we are able to make a standard deployment of Exchange 2010 SP1 work as a multi-tenant environment, Paul Roman covers it nicely here, The Hard Way - Hosting Environment Preparation.

My first Hosted Exchange solution was Exchange 2000 with a custom built automation engine, which was great, but I remember how much better the support was when HMC was introduced. With /hosting mode it became even simpler and although I desperately want UM in my solution today, I am not sure moving away from /hosting is the right direction.

What are your thoughts?

Daniel

Technorati Tags: Exchange,cloud,hosting

See me and Cobweb at the IP EXPO 20th-21st October 2010

Hi,

I will be representing Cobweb at the IP EXPO next week. Come and find our stand and talk to me about the Cloud, Exchange 2010 and what Exchange solutions we can provide to succesfully outsource your infrastructure.

See you there,

http://www.cobweb.com/tour.aspx

http://www.ipexpo.co.uk

Oliver Moazzezi

MVP - Exchange Server

Webinar: An Overview of Microsoft Hosted Exchange 2010

Hope you all enjoyed the latest Cobweb Webinar on Hosted Exchange 2010.

Join me for the next one in October. To see availability and dates, and of course to register please follow http://www.cobweb.com/events.aspx

I take a Q&A session at the end of every one, so feel free to prepare any questions before and during should I not answer them in the presentation.

Oliver Moazzezi

MVP - Exchange Server

Join me at Microsoft London Victoria for an overview of Exchange 2010

Registration is free, sign up here.

The event covers Hosted Exchange 2010, but also covers Hosted CRM4 and Hosted Sharepoint.

Oliver Moazzezi

MVP - Exchange Server

Cobweb Hosted Exchange 2010 Webinar

Please join me in this webinar on Hosted Exchange 2010.

For more information please click here:

http://www.cob-blog.com/cobweb/2010/02/what-is-hosted-exchange-2010-httpwwwcobwebcomonline-learningwebinar-what-is-he-2010aspx.html

See you there!



Oliver Moazzezi

MVP - Exchange Server

Hot on the heels of vBlock?

Cisco, VMWare and NetApp are due to annouce a collaboration of some sort come January 26th.

I would imagine this is an alternative to the vBlock cloud solution already offered by EMC, Cisco and VMWare, but sitting on NetApp storage.

More information here:

http://www.theregister.co.uk/2010/01/14/netapp_cisco_vmware/

Oliver Moazzezi

MVP - Exchange Server

Over 2000 Seats? I'm over here, i'm free and i'm available!!!!

Thinking of going to 'The Cloud' to save on your Exchange Server infrastructure spend?


Over 2000 seats?


I will design a custom solution for you taking into account your needs and my time will be on the house, zero, free - although you're welcome to buy me a beer :-)


If you are interested please contact me.


Oliver Moazzezi

MVP - Exchange Server

msExchMailboxFolderSet and HMC4.0 + HMC3.5 co-existence

If you are deploying HMC4.0 in co-existance with HMC3.5, and not a Greenfield install, then read on.

In co-existance when moving all POP, SMTP, IMAP and OWA urls(inc. Outlook Anyhwere/ RPC over HTTPs access) over to Exchange 2007 Hub Transport and Client Access Servers, be aware there is a bug if you have used OWA segmentation using the msExchMailboxFolderSet attribute.

More on Exchange 2003 OWA segmentation here:

http://support.microsoft.com/default.aspx/kb/833340

The issue is if you use the value '4294967295' (FFFFFFFF) to set 'All Features' for Premium/Gold/'your full access mailbox type here'.

'FFFFFFFF' is read as -1, and Exchange refuses to allow POP3 and IMAP connectivity on the mailbox. Further the Exchange Management Console also flags the 'msExchangeMailboxFolderSet' as corrupt when selecting a user under 'Recipient Configuration \ Mailbox'.

Setting the value to '131071', as mentioned in the article, fixes the issue. (edit: there was an issue with this kb which has now been fixed, so additional notation on 131071 and 1310071 removed).
The way to fix this is to set the attribute to (null), or 131071 as mentioned in KB833340.

Other values that actually do restrict OWA use appear to not be affected.

This will also effect any ordinary enviroment that is in transition from Exchange 2003 to 2007 and is using the msExchMailboxFolderSet attribute in this way.

Oliver Moazzezi

MVP - Exchange Server

Mail enabled Contacts in a Hosted Enviroment and the Offline Address Book.

Contacts in a Hosted Exchange environment can be tricky to implement succesfully, with 1) the way Exchange searches object attributes to create an Offline Address Book and 2) Active Directory not allowing 2 objects to have the same proxy address (which in all fairness is actually a great necessary check in the GUI to have – although this can be bypassed with LDAP manipulation! (ADSI too) – Note: having two objects with an identical proxyaddress will break delivery to that address and is considered attribute corruption of Active Directory).

So how does the Exchange 2003 System Attendant (using oabgen.dll) determine objects to be included for OAB generation? - It looks to see if the object has two attributes: a ‘proxyaddress’ and ‘mail’ attribute. It will further check to ensure the primary (SMTP in uppercase) ‘proxyaddress’ matches the mail attribute address.

So how does an Exchange Hoster get around 2 companies having the same contact of john@doe.com for example?

First let me explain the TargetAddress and ProxyAddress attributes on a mail enabled AD contact.

The TargetAddress is their actual email address, for example : bill@microsoft.com
The ProxyAddress is what RUS (if you use it – HMC disables all but Enterprise RUS (enabled for System Attendant operation)) stamps on the objects email addresses tab. RUS can of course be told to bypass objects by unchecking ‘Automatically update email addresses based on recipient policy’. You will find the primary proxyaddress will be the address of the contact, matching the targetaddress, and depending on RUS and Recipient Policy configuration it could well be stamped with further proxyaddresses.

So, john@doe.com – how can two customers have this contact in an HMC/Hosted Exchange environment?

The short answer is they can, but it cannot show up in the OAL. This is due to the Offline Address Book generation specifying proxyaddress attributes I mentioned earlier, rather than also considering targetaddress attributes.

99% of hosters won’t have this problem – and contacts will be generated with a proxy address (something HMC supports by default). However when you run into this problem it does cause customer grief.

One way of bypassing it is to give a bogus proxyaddress, for instance ‘HostedCompanyName.joe@bloggs.com’, where HostedCompanyName is the name of the Hosted Exchange customer.

This does work, but introduces other issues when a user outside the Org performs a ‘Reply All’. Take a look.

Here’s the properties of the contact from the GAL:

Here’s the contact from the AD, I have pulled the info from ADSIEdit:

You can see the highlighted proxyaddress and targetaddress attributes clearly:

When you send a message outside of the Org, and include the contact, if anyone that is also outside the Org does a 'Reply All', they will only see the incorrect proxyaddress and not the correct SMTP address of the contact, which is the targetaddress:

This of course will result in an NDR

The fix? Remove the proxy attribute altogether, removing the contact from OAB generation, or have the primary proxy address match the target address (standard Exchange2003/2007 behaviour) – but something that will cause mail flow issues when you get a customer with the same contact.


Oliver Moazzezi

MVP - Exchange Server

Hosted Exchange for the world

Large Hosted Exchange providers can get bitten by scheduled maintenance – it will always be in everyones contract, but what happens when a certain percentage of your customers are outside your timezone? – worse still, substantially.

To expand your Hosted mailboxes you have to reach further than your own country – and a lot of Hosted Exchange providers can say they host mailboxes for companies across both the Americas, Europe and the Middle East/Asias.

Intelligence has to be added to your provisioning portal – otherwise your Hong Kong users from Company A could be put on the same Exchange Server (not necessarily the same Mailstore or even Storage Group) as the rest of Company A’s users from Europe. And what’s worse? The rest of the users on the Exchange Server are based in Europe. How is the scheduled maintenance justified to the Hong Kong contigent when it's happening during their working day?

So how does scheduled maintenance come into effect here?

Working out of hours to GMT isn’t going to cut it for the users in Hong Kong as their day is still in full swing – this is where careful planning and design is required. The ideal answer is to carve up the World Map into set zones, so whether a single company is from Dubai, or a single company has offices in Dubai, Europe and the USA you do not affect their respective core working hours. This requires a lot of Dev work - as although HMC supports provisioning to multiple stores for a customer, it doesn’t have the intelligence of splitting users between ‘time zone Exchange Server farms’ based on their location for example. This is where in-house or outsourced Dev work is required.

Suffice to say our current Exchange 2003 solution doesn’t have this feature – we support the provisioning to multiple databases – even across multiple Exchange Servers – however there is not the intelligence that is required for a Hosted Exchange supplier to rule all time zones and keep customers that have offices in some or all, happy.

This might have to be a phase 2 or 3 step in most Hosters plans, but it is a much needed step to successfully achieve 100,000 mailboxes and beyond.


Oliver Moazzezi

MVP - Exchange Server

Hosted versus In-House

We recently came across an article that weighs up the pros and cons of each. I specifically wanted to address the questions for the Hosted Exchange provider.

The article is here:

http://theessentialexchange.com/blogs/michael/archive/2007/12/17/moving-from-in-house-exchange-to-hosted-exchange.aspx

The questions it poses are below; i've answered each one if taking the Hosted Exchange Solution provided by Cobweb.


1. Does the hosting environment allow multiple hosting clients to have contacts with the same e-mail address? (This question can be restated as: how does the hosting software deal with SMTP address collisions?)

The answer is yes _and_ no. Active Directory cannot support two objects with an identical proxyaddress, and unfortunately the OAL is built based on objects having this attribute. The solution is to remove the proxyaddress, giving the contact just it's targetaddress attribute. This allows the exact same contact to exist in multiple customers OU's, but will remove the contact from the OAL. We have been working with Microsoft on this issue, and a resolution to this is promised in the next version of Active Directory/Exchange.


2. Does the hosting environment allow you to share SMTP address space, either as a master or as a slave environment, with a hosted SMTP domain? (This question can be restated as: can you do a step-wise migration, or do you have to migrate all mailboxes at once?)

Yes we have supported this for around two years. We can share SMTP address space and either pass mail over VPN tunnels or over the Internet using SMTP over TLS. We also provide SMTP over the Internet for customers that are not concerned about potential internal mail being sent in clear text across the Internet. In all cases we suggest TLS/VPN solutions, which we manage with the customer and help setup.

3. Does the hosting environment support Deleted Item Retention? For how long? Does their deployment environment set the DumpsterAlwaysOn registry key for Outlook? (This question can be restated as: what happens when someone deletes something they didn't mean to!)

We support DIR for 14 days (two weeks), we also keep deleted mailboxes for 31 (effectively 1 calendar month), of course all mailboxes deleted after this time are still recoverable from our backups.

4. Does the hosting environment support Deleted Mailbox Retention? For how long? (Restatement: can I easily restore the mailbox if my company administrator deletes a mailbox by mistake?)

Answered above.

5. Does the hosting company do backups? How often and how long do they retain them? Can they do single mailbox recovery? (Restatement: if the hosting company has a "disaster" can they recover my mailboxes? Also, if the timeframe for Deleted Mailbox Retention has expired, can I recover the company president's mailbox from last month?)

Again partially answered above, we keep monthly backups for 7 years (yes 7 years). We can restore a mailbox to any given day in the past 4 week window - after that we keep one full backup per month.

6. Does the hosting environment support journaling? What are the data-retention options for the journal mailbox? Can I have an external interface to a journal solution?

Cobweb supports Journaling, we can Journal your mailboxes and send them to an external solution of your choosing (we have no control of this data - you ensure this provider can do the job), or we can Journal your mail ourselves. we use Zantaz EAS and support envelope journaling. We have default plans of 1, 2, 5 and 7 years. We can also provide custom retention policies. This is searchable using a built in Zantaz EAS plugin, which retrieves the archived mail from your own personal document store over SSL.

7. Does the hosting environment support catchall mailboxes? (This is simple a feature that some companies use. Others don't.)

We don't support this, we could but I can honestly say i've never had any customers require this

8. Does the hosting environment have a decent anti-spam solution? (More than the Outlook Junk Mail Filter!) Does the anti-spam solution support individual mailbox quarantines? If there is a false-positive, how can you get your file/message delivered?

We use MessageLabs as standard for all Hosted Exchange mailboxes. We also use Antigen for virus detection on the actual Exchange Servers themselves - supporting 4 AV engines.

9. Does the hosting environment allow you to truly white-label their services? (Restatement: can you have a custom OWA URL? Can you have a custom RPC/HTTP URL? When you connect to an SMTP virtual server, does it say YOUR domain name?)

Yes you sure can, although there is of course an extra cost associated with this.

10. Does the hosting environment allow you to have custom OWA themes? Does it support OWA segmentation

We support OWA segmentation, we base this around our own custom mailbox plans. We can support custom OWA themes but so far we have not had any customers require this.

11. Does the hosting environment support SPF and/or Sender-ID incoming? Does it require it outgoing? Can you decide or are you limited to their default?

MessageLabs support SPF, we don't use Sender-ID within the Exchange Org, we help customers setup their own SPF records.

12. Does the hosting environment support SSL for OWA? TLS for SMTP? Form-based authentication for OWA? Two-factor authentication for OWA and for Outlook?

SSL for OWA with FBA - Yes
SMTP over TLS - Yes
IMAPS - Yes
POP3S - Yes
RPC over HTTPS - Yes

We currently do not provide two factor authentication processes.

13. Does the hosting environment allow you to specify on a per-user basis who gets EAS (ActiveSync)? Blackberry services? Goodlink services?

Yes, which user gets what is entirely customisable via the customers Portal Administrators.

14. Does the hosting environment allow you to create custom address lists?

Currently no, this is something I want to bring into our Exchange 2007 offering. Support for 3 to 5 custom address lists is what I want to achieve.

15. Does the hosting environment allow you to force an Offline Address Book (OAB) update?

Yes, this is done simply by modifying a user in our Portal, we then automatically set instructions to rebuild your OAL.

16. How is disk space aggregated? Is each mailbox billed separately? Is the company/domain aggregated together? Can different mailboxes have different default allocations? Can you manage the limits? Can you get disk space reports? Can you create/manage a "Mailbox Manager" policy for your domain?

Whilst I cannot answer any billing questions, I can state mailbox size is highly configurable. Bought two mailboxes with the default of 200mb each for you and your secretary? Don't need that space for her? No problem, take space off her mailbox and assign it to yourself or your public folders.

17. What are the hard limits on mailboxes sizes?

We don't have any, we do warn (due to current limitations in certain administration tools and tasks) against going over 2GB.

18. Does the hosting environment run a gateway anti-virus solution? An information store anti-virus solution? A file-based anti-virus solution? If there is a false-positive, how can you get your file/message delivered?

MessageLabs for the gateway, Antigen on the servers. Customers get their own Spam Manager Portal to login and check any spam messages that have been quarantined.

19. Does the hosting environment support "Send As" permissions and "Send On Behalf Of" permissions? Can you manage this yourself?

We do support this yes, our existing Portal does not support this feature, our new Exchange 2007 Portal will support this.

20. Does the hosting environment support LDAP access to your address books?

No, however watch this space.

21. Do you have access to SMTP log files? Do you have access to message tracking log files?

SMTP protocol logging is turned on and off by Cobweb as/when there is any possible issue. In regards to access to Message Tracking, the answer is no. However this is something I want to incorporate into our Portal.

22. What is the maximum incoming message size? The maximum outgoing message size? Can you adjust it?

20MB, customers cannot adjust this currently no.

23. What is the maximum number of message recipients? Can you adjust it?

500, this is not configurable.

24. Does the hosting environment support public folders? How many? How big? Can you mail-enable public folders?

We support Public Folders yes. We also support mail enabling them.

25. Does the hosting environment support an interface to SharePoint services?

We current offer Sharepoint 2.0. We are launcing our new Sharepoint 3.0 service sometime over the Summer.

26. Does the hosting environment allow for external SMTP relays by IP address? What about by authorized users?

We support this yes.

27. Does the hosting environment allow for POP-3 or IMAP users to access Exchange mailboxes?

This is configurable by the customer within the Portal.

28. Does the hosting company offer a network Service Level Agreement (SLA)? Does the hosting company offer an Exchange SLA? Does the SLA have any teeth?

Check http://www.cobweb.com for our SLA, I believe currently it is 99.9%, which we meet.

Oliver

Oliver Moazzezi

MVP - Exchange Server