Automate retention events through the RecordsManagement Graph API

It is possible for some time already to automate retention events through an API endpoint, however this is based on the Compliance API’s provided by Exchange.This works perfectly fine, but is until this very day authenticated still using Basic Authentication.
As you know, everything on Exchange is moving to OAuth. Strange enough there was no alternative for this API (even though it’s built all around compliancy).

Until now!

Before you continue
Everything in this blog post is based on the new RecordsManagement API which is, as of this writing, still in Preview. Please note that this API can still be subject to (fundamental) changes, as it is not released in General Availability yet. I will update this blog post as soon as new information comes available.

While investigating how I can make use of the RecordsManagement API, I noticed information is scattered all over the place. Therefor I’m writing this tutorial to guide you step-by-step through the process of utilizing the new RecordsManagement API provided by Microsoft Graph to automate retention events.

Because it is necessary to execute a couple of steps I’ve added a Table of Contents for your convenience.

Prerequisites

Before you can automate your retention events I’m considering the following in place:

• The use case of automating retention events is clear to you (drop me a note if you want to learn more)
• Next to that you’ve already set up retention in Microsoft PurView Compliance (labels, event types & policies)
• You are a Global Administrator of the Azure tenant were you’re performing these actions
• All licenses are sorted on the Azure tenant to make use of Microsoft PurView
• You know and understand concepts of API’s, specifically Microsoft Graph
• You are familiar with the Azure Portal, specifically Azure Active Directory
• Infrastructural configuration allows execution of the steps described

Create User Account

The RecordsManagement API only supports delegated permissions, which means we do not only need to create an App Registration to authenticate to the API, but also a user to perform these steps as. When authenticating to the API the App Registration will present itself as being a user, as is the nature of delegated permissions.

This means we need to create a User Account which we’ll use for API authentication. Please note that this username is reported when looking at ‘Created By’ on the created retention events.

Bring Your Own User
Obviously, if you want to use an already existing user you may skip these steps.

• Go to the Azure Portal, and ceate a new user called ‘Compliance Service User’ in the Azure Active Directory
• Reset the temporary password to a new – preferably strong – password
• There is no need to assign this account any licenses
• Collect the username and password somewhere secure (we’ll need it later)

Create App Registration

Next we need to create an App Registration. This will be used together with the user account we created before when authenticating through the delegated authentication flow.

• In the Azure Portal, go to ‘App Registrations’
• Click ‘New’ in the menu bar to create a new App Registration
• Provide a name for the new App Registration, e.g. “RecordsManagement API”
• Set the ‘Redirect URI’ to “Web” and add “https://localhost”
• Now click ‘Register’ to create your App Registration

When your app is created you are redirected to it’s blade. Make sure to store the following information on the same secure location as we did previously:

• Application (client) ID
• Directory (tenant) ID

Assign API Permissions

The App Registration will need permissions before it can successfully call the RecordsManagement API. The steps below describe how to add the permissions

• Select ‘API permissions’ in the side menu of the App Registration’s blade
• Click ‘+ Add a permission’ and select ‘Microsoft Graph’

• Next, choose ‘Delegated permissions’ and search for “recordsmanagement”
• Check the box next to ‘RecordsManagement.ReadWrite.All’
• Click ‘Add permissions’ to assign these permissions to your App Registration

Grant Admin Consent

When you’ve added the permissions they don’t automatically become active. You first need to provide consent. This can be done through the user, however in this example we’re providing consent as Global Administrator.

Types of consent
The difference between providing User Consent or Admin Consent is that, with providing User Consent you make the API permissions available for that user only. When granting Admin Consent you allow all users in the tenant to connect through this App Registration. It is a choice whether you want every user to provide consent separately, or once for the entire tenant. Please note that these permissions in itself do not provide access to the RecordsManagement API, thus providing Admin Consent does not directly enable your entire tenant to send authenticated requests.

• After adding the API permissions , click ‘√ Grant admin consent for {tenantName}‘
• Click ‘Yes’ to grant Admin Consent

When Admin Consent is provided correctly you see the status showing a green checkmark:

Create Secret

Now we’ve got the API permission set we’re at the last step of setting up the App Registration: create a secret. This is used for authenticating the App Registration to the RecordsManagement API.

• Select ‘Certificates & secrets’ in the side menu of the App Registration’s blade
• Click ‘+ New client secret’ on the ‘Client secrets (0)’ tab
• Provide a description for your key (this is for future reference)
• Select an expiration time (after this time you need to create a new key)
• Now click ‘Add’ to generate a new secret

The newly created client secret will show. Copy it to the same secure location where we stored our information in earlier steps. We can almost put it to use!

Don’t hit refresh!
Make sure to copy the client secret directly after it is created. When you leave the page the key won’t show anymore, and there is no way to retrieve it anymore!

Assign Azure AD Role

For the RecordsManagement API to successfully authenticate using the delegated permissions, you must add an Azure AD Role to both the App Registration and the User Account. This Azure AD Role will provide you access to Microsoft PurView Compliance, and needs to be present in both for delegated permissions to work.

• Go to ‘Azure Active Directory’ in the Azure Portal
• Select ‘Roles and administrators’ in the side menu

• Search for “compliance” in the ‘All roles’ blade
• Click on the rule containing ‘Compliance Administrator’

• Now click ‘+ Add assignments’
• Search for both your User Account and App Registration
• Select them to add them to the ‘Selected Items’ section

• Hit ‘Add’ to add the Azure AD Role to both
• You should now see the assignments being added for a ‘User’ and ‘ServicePrincipal’ (in other words: User Account and App Registration)

Screens may differ
For this example I’m using the default experience for assigning Azure AD Roles. If you are using Privileged Identity Management (PIM) the process is slightly different as you need to provide justification before assigning Azure AD Roles.

Test API in Postman

Now we’ve got our User Account and App Registration set up, and provided the permission, it is now time to test the RecordsManagement API. This is a two step process:

1. Request Access Token
2. Use Access Token to make an authenticated request to the RecordsMangement API

The following paragraphs provide details on each of these steps.

Get Access Token

First step is retrieving an Access Token. This will be used in the next step for authenticating the request to the RecordsManagement API.

• Create a new request in Postman
• Set the method to ‘Post’
• Enter the following request URL (change {tenantId} with the Directory (tenant) ID we securely stored earlier):

https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token

• Set the ‘Body’ to “x-www-form-urlencoded” and fill the details below using the details we stored securely along the way of this guide:

Key	Value
client_id	Application (client) ID
client_secret	Secret of the App Registration
grant_type	password
username	Username of the created User Account
password	Password of the created User Account
scope	RecordsManagement.ReadWrite.All

• Hit ‘Send’ to fire the request
• If everything went correctly you should see a body like below
• Copy the contents of the “access_token” attribute

Call Retention Events API

Next step is to actually call the RecordsManagement API. For this we need the “access_token” from the previous step to authenticate the request.

• Create another request in Postman
• Go the the ‘Authorization’ tab
• Set ‘Type’ to ‘Bearer token’ and paste the “access_token” in the ‘Token’ field

• Now go to the ‘Body tab
• Set the method to ‘Post’
• Fill in the following request URL:

https://graph.microsoft.com/beta/security/triggers/retentionEvents

• Set the body to ‘raw’ and set the type to ‘JSON’
• Use the following sample body, and set the values as desired
  • Change {assetId} to the Compliance Asset ID used in SharePoint to make the event look at the correct data, while leaving it prefixed with “ComplianceAssetId:”
  • Change {keywords} to the keywords you want to use to search through Exchange, separated by commas
  • Both ‘displayName’ and ‘description’ are strings, which you can fill for human reference
  • The ‘eventTriggerDateTime’ field is a string, but should contain a valid ISO 8601 datetime
  • Change {eventTypeId} to the ID of the retention event type, but leave the rest of ‘retentionEventType’ as is (otherwise it throws a “retentionEventType can not used for event creation” error)

{
  "displayName": "string",
  "description": "string",
  "eventQueries": [
    {
      "queryType": "files",
      "query": "ComplianceAssetId:{assetId}"
    },
    {
      "queryType": "messages",
      "query": "{keywords}"
    }
  ],
  "eventTriggerDateTime": "string(date)",
  "retentionEventType@odata.bind": "https://graph.microsoft.com/beta/security/triggerTypes/retentionEventTypes/{eventTypeId}"
}

For your information
In the example body above I’ve added both “files” and “messages” as value for the ‘eventQueries’ array, just for your reference. You can leave out any you don’t use, as long as you’re providing at least one query.

• Now hit ‘Send’ to execute the request
• You should see a ‘201 Created’ when the event is successfully created

Considerations

While writing this guide, providing the examples, I noted some considerations you might want to take into account:

• Normally I would aim for Application Permissions instead of Delegated Permissions, because you would not want to have a user in between, however as of this writing this is the only option (I’ll keep an eye out for the future)
• When talking security:
  • You can (and should) configure Multi-Factor Authentication (MFA) on the User Account, however the calling IP address should be whitelisted from enforced MFA for authentication to work properly
  • With that said: make sure you got a fixed IP address used for calling the API to raise security
  • You should make sure security is set properly to make sure this API cannot be abused (as it can have big consequences)

Next Steps

Setting it up in Postman, and seeing it work is great! However, this is only the start. Best practice would be to convert the steps above to an Azure API Management policy to provide an additional layer of security, before this API is consumed by users or integrations.

Please feel free to discuss with me on this topics through the comments below. Off course, you can also get in touch and leave me a message. Looking forward to IT!