Hero Banner

Key Resources and Guides

Find key resources and guides that you can accelerate implementations

Reply
Level 3 Contributor

CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Bringing-forward from https://www.yammer.com/office365partners/?show_login=true#/Threads/show?threadId=145300852449280...  (Why the multiple communities here?  Which is the go-forward platform here, Yammer or microsoftpartnercommunity.com ?)

 

How to support CSP customers while protecting the security of their tenant? Critical for security-conscious customers, and/or those with high-security requirements. Permissions granted by delegated admin are too far-reaching, do not allow for fine-grained access, and even the ability to audit use is unclear or non-existent.

 

This is something I had first started looking at just over a year ago. It doesn't look like anything has improved since, but customers are understandably only becoming increasingly concerned with this.

 

Here is some feedback we just received earlier today from a current customer looking to move to CSP licensing:

 

> Even with a NDA with <direct CSP provider> I am not at all comfortable making a third party of a third party delegated admins for our environment and granting them the ability to access all our cloud data which is what the terms state we’d be agreeing to. ... I really like the benefits of CSP otherwise so if there’s some other way to make this work we’d probably still do it. I also have reservations about a vendor having these all-encompassing permissions. I understand it can be needed for some admin tasks and troubleshooting but I don’t like how much of a free for all Microsoft makes it.

 

(I do believe there is some confusion in the above, in that our direct CSP provider / reseller would not have delegated admin - but only access to contact, subscription, and billing details. However, even this would be good to have further detailed in a format that we can share with concerned customers.)

 

If I were in the customer's place, I would absolutely be asking the exact same questions, and imposing the same requirements - and have so in prior organizations.

 

I have several other customers I've been working with, who I have not, in good faith, been able to recommend putting into CSP and the required delegated admin for - given the concerns outlined here.

 

We currently have a few hundred customers enrolled in CSP - and have an opportunity to bring even more into the CSP program, but the concerns outlined here do not help with the effort.

 

See also: https://docs.microsoft.com/en-us/azure/cloud-solution-provider/customer-management/administration-delegation

 

A number of specific points and concerns:

 

1) The list of roles and permissions that can be applied through Partner Center are available at https://docs.microsoft.com/en-us/partner-center/permissions-overview . Of these, only "Admin agent", "Helpdesk Agent", and maybe "Sales agent" apply to the customer tenants. "Admin agent" is basically the equivalent of "Global Administrator" within a customer tenant, and "Helpdesk Agent" is effectively a "Helpdesk Administrator" (Password Administrator). The admin roles from a tenant perspective are documented at https://docs.microsoft.com/en-us/office365/admin/add-users/about-admin-roles . As an O365 admin in my own organization, I can delegate permissions to others through at least 262,144 different combinations of the 18 customized administrator roles currently available in my tenant. (This increases to 38 roles available and nearly 275 billion combinations when using the roles available in Azure AD.) As a partner with delegated admin to a customer, there are only 2: "Everything", or "Helpdesk Agent" (or none). See also: https://www.yammer.com/office365partners/#/Threads/show?threadId=119473804279808 and https://www.yammer.com/office365partners/#/Threads/show?threadId=804328021 .

 

2) Everything is "all or nothing". We also have well over 100 engineers in our organization (not counting back office staff, etc.). To assign either "Admin agent" or "Helpdesk Agent" to one of our staff, means that they have that same permission across a few hundred customers. There is no way to filter a staff member's access to only one customer, or ideally a group of customers. (While we do have some additional flexibility in granting access through guest accounts in Azure AD, these permissions do not carry over to / are not usable in O365.)

 

3) We could consider falling-back to the use of extended and manually-reviewed audit logging as a compensating control. However, there are many limitations here - including that all logging appears to be held within each individual customer tenant only - and for us to review delegated admin activity from our partner account as a whole, it appears that we'd need to query each and every one of our customer tenants. Further concerns are raised here at https://www.yammer.com/office365partners/#/Threads/show?threadId=62199238885376 (unanswered).

 

While I'm on the topic, the "pretty wicked security hole" documented at https://office365.uservoice.com/forums/264636-general/suggestions/33233917-powershell-mfa-for-csp-delegated-admin-privileges also still needs to be addressed, well over a year later now. Can someone from Microsoft please review and respond to this UserVoice entry? Everyone else - please vote...

 

What are other CSP partners doing in terms of security controls to meet customer needs and expectations as it relates to CSP licensing? (We do have our own internal security program that I am a part of - which includes even stronger MFA controls across our entire organization than we'd otherwise been able to have approved, partially because of the customer security concerns here.)

 

As per https://blogs.technet.microsoft.com/uspartner_ts2team/2016/04/30/removing-the-csp-as-delegated-admin-in-office-365-customer-tenants/ , it looks like the simplest answer may to just be to have needing customers remove the delegated admin permissions. However: 1) Is this still accurate and working, 3 years later? And 2) we'll need to determine what limitations, if any, may then exist that would prohibit us from providing CSP licensing in this model, even if we then are not able to directly assign new licenses to users, or otherwise provide direct support to the customer as per CSP terms (at least not without having other customer-provided access given to us).

 

Just looking at this again now, it looks like we *might* be able to put the "Admin agent" and "Helpdesk Agent" roles behind the new "Azure AD entitlement management". This is less than 2 months old, still in preview, and what Privileged Identity Management really should have been all along, in my opinion. It does require Azure AD Premium P2 (or its parent EMS E5 offfering).

13 REPLIES 13
Level 5 Contributor

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

I'd like to echo the points made by @ziesemer.  We've brought this up with our Microsoft account manager, but MSFT hasn't changed anything yet.  We recently learned that with the new CSP portal, engineers need "Admin Agent" to open Office 365 related support cases, which seems like way too much access (Helpdesk agent isn't enough).  Also, I very much agree with point 2 about engineers having access to all clients or none.  We want granular control of which clients an engineer has access.

Community Manager

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Good day @ziesemer & @Jinseng ,

 

Thank you for raising these concerns with the Community!

We have Azure AD team and other teams working on these known aspects in the background and I would advise meanwhile to address such issues during the Office Hours (2 live sessions available!) by registering here.

This allows you to stay on top of the latest updates.

Also you can consult this resource : https://docs.microsoft.com/en-us/partner-center/partner-security-requirements.

Finally keep an eye on the Security Guidance Community threads.

 

Wish you a great day!
Andra

Visitor 1

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Hi Mark,

 

did you get any answers regarding the delegated admin issue. I share your view around customers not wanting to grant full access to us and we have this issue right now. The problem is that if a customer removes delegate admin (or we uncheck the box "Include delegated administration privileges for Azure Active Directory and Office 365." in Partner Center on the 'Request a reseller relationship' page when we set them up) then we cannot provision them any licences or see what they have. This seems to contradict what Woody Walton says in the blog (link in your post). I'm not sure if this is just a fault with Partner Center at the moment but it seems having a transacting only (not delegate admin) relationship is not possible.

Highlighted
Microsoft

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Provisioning licenses or checking on licenses you have already provisioned for a customer is possible without having delegated admin permissions (also changing license count and suspending licenses is possible). So, if you experience issues with this this seems to be a technical issue where you can contact support (Just checked again in my test setup and I can manage licenses without having delegated admin - as expected).

 

Opening support tickets on behalf of the customer is not possible without delegated admin - so this might require that delegated admin is re-established once the customer asks for support and it is determined a ticket has to be raised at Microsoft support.

Visitor 1

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Sorry, but's that not just an acceptable solution or workaround. I'm surprised that the whole CSP ecosystem is not designed with security in mind. What's even worse, is that several of the activities an Admin Agent performs in the customers tenant, doesn't get logged and customer cannot see who/when an admin agent jumps to the tenant. 

There is also 2 twists about this problem:

1. Customers sometimes do remove DAP themself, either by mistake or by intention. Then the customer cannot create support tickets in Azure at all. In M365 it's still technical possible (there's a "New Microsoft service request" button in the bottom of the wizard). If a customer do that, the customer is technically using the ASfP/PremierSupportForPartners agreement for the partner, which is a violation of the CSP itself. 

 

2. Some customers implement Conditional Access rules, to block all access from non-corporate devices (AzureAD/HybridAD joined devices) - which makes sens . In that case, even if DAP is still in place, an Admin Agent cannot access the customer tenant and thereby cannot create support tickets. 

Customers who requires a zero-trust approach, cannot use the CSP model as it is now as there is no built-in functionality that allows access to the tenant without allowing any device. I'm quite surprised that the architecture design behind the CSP got approval from a security & compliance perspective, as it contradicts Microsoft's own security guidelines and recommendations.  

Microsoft

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

User activites are logged, the individual user account that is acting as admin agent is visible in the customer logs. What is not logged is when an admin agent just accesses a customer tenant without doing an activity (since there is no login happening, this can not be seen by the customer, though he could set a conditional access policies that can prevent this or force the Partner user to do MFA in his tenant again).

 

The customer can not create any support tickets himself, regardless if DAP is set or not. When DAP is removed the Partner can not do this anymore though.

It is correct when a customer opens a ticket in M365 admin portal, but he is not using the ASfP support contract. he will receive standard support same as every other customer. Even when he would use the ASfP contract there is no impact to the Partner since thre number of tickets is not limited. 

Only for Premier this could happen - but the TAM will immediately notice this.

 

When the customer wants to restrict access via conditional access he can do so, he can always set certain exclusions for the Partner admin agents and/or specific other security requirements they need to fulfill. 

 

If the customer asks a Cloud Solution Provider to be his managed service provider, but then has zero trust for the service provider of choice, indeed CSP is the wrong model for them. On the other hand, if the CSP partner is not able to show certain attestation that they have certain security implementations in place, they are not the right CSP for a customer that has those requirements.

CSP was designed for a scenario where the Partner is responsible for the management and has a vlaue-add service, not for a scenario where the customer just buys via a CSP, locks them out and only contacts them to raise a support ticket. If this is the intent of the customer, he shoud rather get a direct agreement with Microsoft.

Level 2 Contributor

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

@JanoschUlmer I don't agree

 

Many of our customers has very sensitive information and they not allow us to have any permissions on their resources.
Currentlly We are working with those customers on remote sessions or in their office, using the customer username

But, I understand that on the new “Azure Plan” program the partner must have admin rights on the customer environment to get rebate.
And if the customer remove the permission then the partner is losing the rebate.

By forcing this policy Microsoft is build a system that weaken security significantly
Microsoft own best practice recommendation is no more than 3 global admin, and no more than 3 owners for subscriptions
But when giving partner full access then it’s adding all partner technical users (few dozen) with permission equivalent to global admin and subscriptions owners.
Just imagen that hacker manage to compremise security of Microsoft CSP partner
He will have full access not to one company but to hundreds of companies at once with full access to everything (DB’s, Storage’s, VM’s, E-mails etc.)

Another concern
On Azure calculator to get CSP pricing option the user must have “Admin Agent” role.
So even salespeople (that need this options) must have an admin role

Level 2 Contributor

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

@JanoschUlmer I don't agree

 

Many of our customers has very sensitive information and they not allow us to have any permissions on their resources.
Currentlly We are working with those customers on remote sessions or in their office, using the customer username

But, I understand that on the new “Azure Plan” program the partner must have admin rights on the customer environment to get rebate.
And if the customer remove the permission then the partner is losing the rebate.

By forcing this policy Microsoft is build a system that weaken security significantly
Microsoft own best practice recommendation is no more than 3 global admin, and no more than 3 owners for subscriptions
But when giving partner full access then it’s adding all partner technical users (usully few dozen) with permission equivalent to global admin and subscriptions owners.

Microsoft

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

The CSP model indeed requires Partner to have strong security and control implemented on his end, since when a managed service provider is responsible for the enbd customers services it should of course not lower the security, but ideally increase it. This is also why the new Partner Agreement  requires Partner to have such controls in place that protects privacy and ensure securioty of customer data.

That is what CSP is all about, it is not just a licensing program, but a model for (Cloud) Solution Providers that take over the responsibility for service management, and naturally the "Solution Provider" needs some for of access on the environment. It might not be the right model for all customers of course.

 

Also, independent from what has been done reg. Azure Plan, the CSP partners needs to have some form of access to open support tickets on behalf of the customer - delegated admin permissions for O365 (which is equivalent of Global Admin), and in Azure there can be more fine grained roles, like Support Contributor.

 

I agree with you that just using the default permissions that are set (AOBO/Foreign Principal with Owner rights on subscription) are not sufficient for the demands of some customers, this is why there are additional options - like using an account that is created in the customer tenant, but linked to the Partner ID (PAL - Partner Admin Link). For your scenario this sounds like the right approach, in order to be able to open support tickets on behalf of the customer I would also think about using a guest account from your Partner tenant to the subscription.

https://docs.microsoft.com/en-us/partner-center/azure-plan-manage

Specifically take a look at the list of roles that can be used to get PEC

Beneath that there are further options that give the customer more control, like e.g. using MFA for those accounts where only the customer has access to the 2nd factor, use auditing, use Azure policies etc...

 

 

 

Level 2 Contributor

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

Yes, but you are ignoring the most important thing, the money

If partner don’t have admin on behalf (PAL is it’s not enough) then he is not getting any rebate or margin from Microsoft

In practice, he is losing money on every such customer

So partner biggest incentive is to set admin on behalf and not lose  ~20% rebait and margin money

On the customer side, if he removes partner permissions then the partner will cancel any discount the customer has on Azure

So by this policy Microsoft encourage both sides (partners and customers) to weaken security

Microsoft

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

That is not true, AOBO is one of three methods to get PEC. You can also use Lighthouse for solutions you deploy, and you can link an account from customer tenant, a guest account or a service princpal via PAL. And those account do also not need to have owner permissions, other roles work to.

This is all described in the before mentioned links: https://docs.microsoft.com/en-us/partner-center/azure-plan-manage

 

When customer removes AOBO, there is warning message displayed for the customer that this might impact the Partner relationship, and we also encourage Partner to think about setting up an alert for any of those changes, this can also be set if the specific account is changed/deleted via you have access: https://docs.microsoft.com/en-us/partner-center/azure-plan-manage#create-an-azure-monitor-alert  

So this does not weaken security if implemented the right way with least privilege in mind (e.g.the idea to hand over the 2nd factor MFA so only he can approve Partner sign in). It is also important Partner does have the necessary controls in house for basic security on his side. And to me this is the important resulting change - the need to have such controls, to be able to attest to the customer that you as service provider are able to provide secure delegated management, is much more important than before, at least to be able to hand over discounted prices because of course the customer should have questions if giving the Partner the keys to the kingdom is "secure enough" - this was important before, but now there is more impact.

Again, CSP is not just a licensing channel where it is only about margins for reselling service, CSP is, and was also before Azure Plan, a servicing model. I'm not ignoring anyhow that it is about the money, I'm just emphasizing the fact that it is not only about resale-driven revenue, it is also about revenue generated via end-to-end & ongoing servicing. I know that CSP was used by some only to be able to hand over some discounts, but this was never the idea, and the new Azure plans made this much more visible.

 

If the customer wants to lock a Partner out from certain areas completely, then you are correct that without PEC there will not be any discount for them - incentives are still payed though (Actually is 15% discount for Azure and ~6% incentives, not 20% overall).

 

However, I'm not agreeing with you that in practice Partner will lose money - it is a matter of discussing the impact with the customer. It would certainly not be a good idea to set up an agreement that forces you to give a fixed discount even when the customer locks you out, so the matter of PEC & the need to have "some" permissions need to be included in the agreements you have with the customer - and also impact on servicing fees when permissions are removed (since you might have incorporated those to some part into the margin). 

 

I do fully understand that those changes are fundamental and will leed to some tough discussions with customers and I appreciate the open & honest feedback on this. I also fully agree that it would be great to have imporovements in the way delegated admin permission can be scoped on Partner side when using AOBO, this is something I discussed a lot internally during the last months (but as a Partner consultant unfortunately I have no power to decide on this).

Level 2 Contributor

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

@JanoschUlmer Thank you for detail answers

Are you sure that PAL with user from the customer tenant is good enough for rebait and margin?

If so it can solve the problem

 

And Microsoft must accept that partners can give an excellent service without any permissions

Using only remote desktop sharing sessions (the same way as Microsoft support works)

Some customers will never give permission to partner.

Never mind the partner security level.

So for those customers CSP is currently not an options

Microsoft

Re: CSP and Delegated Admin security concerns: MFA is great, but what about the several outstanding Microsoft limitations?

@goldmine :

Yes, PAL will be sufficient, as documented here: https://docs.microsoft.com/en-us/partner-center/azure-plan-manage#link-your-partner-id-mpn-idto-your-credentials-for-managing-customers-azure-resources 

Take note of the description for "Directory or Guest user" which says:

Partner needs to associate the MPN ID to the user or service principal in the customer tenant. For more information - Link Partner ID.

 

And also the information reg. the proper roles (e.g. PAL for a user that only has read access won't work to get PEC).

 

 

I agree that by using Remote support some scenarions do work for reactive support, when it comes to proactively managing the environment (e.g. "Hey, we have found that you have some VMs mostly idle, can we discuss how to automate shutdown or change VM sizes?") this fails.

While it is true that Microsoft support will try to resolve issues with the customer/partner using remote support, Microsoft of course has certain access to the backend also (not necessarily the frontline support engineer, and never without an internal approval process - e.g. think about the obvious scenario that customer locks himself out when forgetting the admin password and he contacts support) - and this is where Microsoft provides a lot of proof & attestation in the trust center that while Microsoft has this access, respective controls are in place - and also legal documentation in the contracts that document proper handling of customer data and security controls. So by choosing a cloud provider, customer does grant the cloud provider always an implicit access to his environment - at least thise parts of it where he can't apply additional security concepts like e.g customer lockbox, HYOK encryption, confidential computing etc. - and therefore customer has put some trust in the cloud provider already. So when the customer is using cloud services, it is clearly a sign that they are not saying "we can't let a 3rd party access our systems ever", but "we can only give a 3rd party some limited access to our system when certain conditions are fulfilled".

I personally think that the relation between a managed service provider (=CSP) and a customer is similar in this regard, finally it is a matter of trust & control. Of course, earning the trust and explaining the controls can be a tough job :-)