Set up authentication for Azure Machine Learning resources and workflows

Source: Set up authentication for Azure Machine Learning resources and workflows

Learn how to set up authentication to your Azure Machine Learning workspace. Authentication to your Azure Machine Learning workspace is based on Azure Active Directory (Azure AD) for most things. In general, there are four authentication workflows that you can use when connecting to the workspace:

Interactive: You use your account in Azure Active Directory to either directly authenticate, or to get a token that is used for authentication. Interactive authentication is used during experimentation and iterative development. Interactive authentication enables you to control access to resources (such as a web service) on a per-user basis.
Service principal: You create a service principal account in Azure Active Directory, and use it to authenticate or get a token. A service principal is used when you need an automated process to authenticate to the service without requiring user interaction. For example, a continuous integration and deployment script that trains and tests a model every time the training code changes.
Azure CLI session: You use an active Azure CLI session to authenticate. Azure CLI authentication is used during experimentation and iterative development, or when you need an automated process to authenticate to the service using a pre-authenticated session. You can log in to Azure via the Azure CLI on your local workstation, without storing credentials in Python code or prompting the user to authenticate. Similarly, you can reuse the same scripts as part of continuous integration and deployment pipelines, while authenticating the Azure CLI with a service principal identity.
Managed identity: When using the Azure Machine Learning SDK on an Azure Virtual Machine, you can use a managed identity for Azure. This workflow allows the VM to connect to the workspace using the managed identity, without storing credentials in Python code or prompting the user to authenticate. Azure Machine Learning compute clusters can also be configured to use a managed identity to access the workspace when training models.

Regardless of the authentication workflow used, Azure role-based access control (Azure RBAC) is used to scope the level of access (authorization) allowed to the resources. For example, an admin or automation process might have access to create a compute instance, but not use it, while a data scientist could use it, but not delete or create it. For more information, see Manage access to Azure Machine Learning workspace.

Azure AD Conditional Access can be used to further control or restrict access to the workspace for each authentication workflow. For example, an admin can allow workspace access from managed devices only.

Azure Active Directory

All the authentication workflows for your workspace rely on Azure Active Directory. If you want users to authenticate using individual accounts, they must have accounts in your Azure AD. If you want to use service principals, they must exist in your Azure AD. Managed identities are also a feature of Azure AD.

For more on Azure AD, see What is Azure Active Directory authentication.

Once you've created the Azure AD accounts, see Manage access to Azure Machine Learning workspace for information on granting them access to the workspace and other operations in Azure Machine Learning.

Configure a service principal

To use a service principal (SP), you must first create the SP and grant it access to your workspace. As mentioned earlier, Azure role-based access control (Azure RBAC) is used to control access, so you must also decide what access to grant the SP.

IMPORTANT: When using a service principal, grant it the minimum access required for the task it is used for. For example, you would not grant a service principal owner or contributor access if all it is used for is reading the access token for a web deployment.

The reason for granting the least access is that a service principal uses a password to authenticate, and the password may be stored as part of an automation script. If the password is leaked, having the minimum access required for a specific tasks minimizes the malicious use of the SP.

The easiest way to create an SP and grant access to your workspace is by using the Azure CLI. To create a service principal and grant it access to your workspace, use the following steps:

NOTE: You must be an admin on the subscription to perform all of these steps.

Authenticate to your Azure subscription:
```
 az login
```
If the CLI can open your default browser, it will do so and load a sign-in page. Otherwise, you need to open a browser and follow the instructions on the command line. The instructions involve browsing to https://aka.ms/devicelogin and entering an authorization code.

If you have multiple Azure subscriptions, you can use the az account set -s <subscription name or ID> command to set the subscription. For more information, see Use multiple Azure subscriptions.

For other methods of authenticating, see Sign in with Azure CLI.

Create the service principal. In the following example, an SP named ml-auth is created:

 az ad sp create-for-rbac --sdk-auth --name ml-auth

The output will be a JSON similar to the following. Take note of the clientId, clientSecret, and tenantId fields, as you will need them for other steps in this article.

 {
     "clientId": "your-client-id",
     "clientSecret": "your-client-secret",
     "subscriptionId": "your-sub-id",
     "tenantId": "your-tenant-id",
     "activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
     "resourceManagerEndpointUrl": "https://management.azure.com",
     "activeDirectoryGraphResourceId": "https://graph.windows.net",
     "sqlManagementEndpointUrl": "https://management.core.windows.net:5555",
     "galleryEndpointUrl": "https://gallery.azure.com/",
     "managementEndpointUrl": "https://management.core.windows.net"
 }
  

Retrieve the details for the service principal by using the clientId value returned in the previous step:

 az ad sp show --id your-client-id

The following JSON is a simplified example of the output from the command. Take note of the objectId field, as you will need its value for the next step.

 {
     "accountEnabled": "True",
     "addIns": [],
     "appDisplayName": "ml-auth",
     ...
     ...
     ...
     "objectId": "your-sp-object-id",
     "objectType": "ServicePrincipal"
 }
  

Allow the SP to access your Azure Machine Learning workspace. You will need your workspace name, and its resource group name for the -w and -g parameters, respectively. For the --user parameter, use the objectId value from the previous step. The --role parameter allows you to set the access role for the service principal. In the following example, the SP is assigned to the owner role.

IMPORTANT: Owner access allows the service principal to do virtually any operation in your workspace. It is used in this document to demonstrate how to grant access; in a production environment Microsoft recommends granting the service principal the minimum access needed to perform the role you intend it for. For information on creating a custom role with the access needed for your scenario, see Manage access to Azure Machine Learning workspace.
```
 az ml workspace share -w your-workspace-name -g your-resource-group-name --user your-sp-object-id --role owner
```
This call does not produce any output on success.

Configure a managed identity

IMPORTANT: Managed identity is only supported when using the Azure Machine Learning SDK from an Azure Virtual Machine or with an Azure Machine Learning compute cluster. Using a managed identity with a compute cluster is currently in preview.

Managed identity with a VM

Enable a system-assigned managed identity for Azure resources on the VM.
From the Azure portal, select your workspace and then select Access Control (IAM).
Select Add, Add Role Assignment to open the Add role assignment page.
Assign the following role. For detailed steps, see Assign Azure roles using the Azure portal.

Setting Value

Role The role you want to assign.

Assign access to Managed Identity

Members The managed identity you created earlier

Setting	Value
Role	The role you want to assign.
Assign access to	Managed Identity
Members	The managed identity you created earlier

Managed identity with compute cluster

For more information, see Set up managed identity for compute cluster.

Use interactive authentication

Most examples in the documentation and samples use interactive authentication. For example, when using the SDK there are two function calls that will automatically prompt you with a UI-based authentication flow:

Calling the from_config() function will issue the prompt.
```
  from azureml.core import Workspace
  ws = Workspace.from_config()
```
The from_config() function looks for a JSON file containing your workspace connection information.

Using the Workspace constructor to provide subscription, resource group, and workspace information, will also prompt for interactive authentication.

  ws = Workspace(subscription_id="your-sub-id",
                resource_group="your-resource-group-id",
                workspace_name="your-workspace-name"
                )
  

TIP: If you have access to multiple tenants, you may need to import the class and explicitly define what tenant you are targeting. Calling the constructor for InteractiveLoginAuthentication will also prompt you to login similar to the calls above.
from azureml.core.authentication import InteractiveLoginAuthentication
interactive_auth = InteractiveLoginAuthentication(tenant_id="your-tenant-id")

When using the Azure CLI, the az login command is used to authenticate the CLI session. For more information, see Get started with Azure CLI.

TIP: If you are using the SDK from an environment where you have previously authenticated interactively using the Azure CLI, you can use the AzureCliAuthentication class to authenticate to the workspace using the credentials cached by the CLI:
from azureml.core.authentication import AzureCliAuthentication
cli_auth = AzureCliAuthentication()
ws = Workspace(subscription_id="your-sub-id",
               resource_group="your-resource-group-id",
               workspace_name="your-workspace-name",
               auth=cli_auth
               )

Use service principal authentication

To authenticate to your workspace from the SDK, using a service principal, use the ServicePrincipalAuthentication class constructor. Use the values you got when creating the service provider as the parameters. The tenant_id parameter maps to tenantId from above, service_principal_id maps to clientId, and service_principal_password maps to clientSecret.

from azureml.core.authentication import ServicePrincipalAuthentication

sp = ServicePrincipalAuthentication(tenant_id="your-tenant-id", # tenantID
                                    service_principal_id="your-client-id", # clientId
                                    service_principal_password="your-client-secret") # clientSecret
  

The sp variable now holds an authentication object that you use directly in the SDK. In general, it is a good idea to store the ids/secrets used above in environment variables as shown in the following code. Storing in environment variables prevents the information from being accidentally checked into a GitHub repo.

import os

sp = ServicePrincipalAuthentication(tenant_id=os.environ['AML_TENANT_ID'],
                                    service_principal_id=os.environ['AML_PRINCIPAL_ID'],
                                    service_principal_password=os.environ['AML_PRINCIPAL_PASS'])
  

For automated workflows that run in Python and use the SDK primarily, you can use this object as-is in most cases for your authentication. The following code authenticates to your workspace using the auth object you created.

from azureml.core import Workspace

ws = Workspace.get(name="ml-example",
                   auth=sp,
                   subscription_id="your-sub-id")
ws.get_details()
  

Use a service principal from the Azure CLI

You can use a service principal for Azure CLI commands. For more information, see Sign in using a service principal.

Use a service principal with the REST API (preview)

The service principal can also be used to authenticate to the Azure Machine Learning REST API (preview). You use the Azure Active Directory client credentials grant flow, which allow service-to-service calls for headless authentication in automated workflows. The examples are implemented with the ADAL library in both Python and Node.js, but you can also use any open-source library that supports OpenID Connect 1.0.

NOTE: MSAL.js is a newer library than ADAL, but you cannot do service-to-service authentication using client credentials with MSAL.js, since it is primarily a client-side library intended for interactive/UI authentication tied to a specific user. We recommend using ADAL as shown below to build automated workflows with the REST API.

Node.js

Use the following steps to generate an auth token using Node.js. In your environment, run npm install adal-node. Then, use your tenantId, clientId, and clientSecret from the service principal you created in the steps above as values for the matching variables in the following script.

const adal = require('adal-node').AuthenticationContext;

const authorityHostUrl = 'https://login.microsoftonline.com/';
const tenantId = 'your-tenant-id';
const authorityUrl = authorityHostUrl + tenantId;
const clientId = 'your-client-id';
const clientSecret = 'your-client-secret';
const resource = 'https://management.azure.com/';

const context = new adal(authorityUrl);

context.acquireTokenWithClientCredentials(
  resource,
  clientId,
  clientSecret,
  (err, tokenResponse) => {
    if (err) {
      console.log(`Token generation failed due to ${err}`);
    } else {
      console.dir(tokenResponse, { depth: null, colors: true });
    }
  }
);
  

The variable tokenResponse is an object that includes the token and associated metadata such as expiration time. Tokens are valid for 1 hour, and can be refreshed by running the same call again to retrieve a new token. The following snippet is a sample response.

{
    tokenType: 'Bearer',
    expiresIn: 3599,
    expiresOn: 2019-12-17T19:15:56.326Z,
    resource: 'https://management.azure.com/',
    accessToken: "random-oauth-token",
    isMRRT: true,
    _clientId: 'your-client-id',
    _authority: 'https://login.microsoftonline.com/your-tenant-id'
}
  

Use the accessToken property to fetch the auth token. See the REST API documentation for examples on how to use the token to make API calls.

Python

Use the following steps to generate an auth token using Python. In your environment, run pip install adal. Then, use your tenantId, clientId, and clientSecret from the service principal you created in the steps above as values for the appropriate variables in the following script.

from adal import AuthenticationContext

client_id = "your-client-id"
client_secret = "your-client-secret"
resource_url = "https://login.microsoftonline.com"
tenant_id = "your-tenant-id"
authority = "{}/{}".format(resource_url, tenant_id)

auth_context = AuthenticationContext(authority)
token_response = auth_context.acquire_token_with_client_credentials("https://management.azure.com/", client_id, client_secret)
print(token_response)
  

The variable token_response is a dictionary that includes the token and associated metadata such as expiration time. Tokens are valid for 1 hour, and can be refreshed by running the same call again to retrieve a new token. The following snippet is a sample response.

{
    'tokenType': 'Bearer',
    'expiresIn': 3599,
    'expiresOn': '2019-12-17 19:47:15.150205',
    'resource': 'https://management.azure.com/',
    'accessToken': 'random-oauth-token',
    'isMRRT': True,
    '_clientId': 'your-client-id',
    '_authority': 'https://login.microsoftonline.com/your-tenant-id'
}
  

Use token_response["accessToken"] to fetch the auth token. See the REST API documentation for examples on how to use the token to make API calls.

Java

In Java, retrieve the bearer token using a standard REST call:

String tenantId = "your-tenant-id";
String clientId = "your-client-id";
String clientSecret = "your-client-secret";
String resourceManagerUrl = "https://management.azure.com";

HttpRequest tokenAuthenticationRequest = tokenAuthenticationRequest(tenantId, clientId, clientSecret, resourceManagerUrl);

HttpClient client = HttpClient.newBuilder().build();
Gson gson = new Gson();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
if (response.statusCode == 200)
{
     body = gson.fromJson(body, AuthenticationBody.class);

    // ... etc ... 
}
// ... etc ...

static HttpRequest tokenAuthenticationRequest(String tenantId, String clientId, String clientSecret, String resourceManagerUrl){
    String authUrl = String.format("https://login.microsoftonline.com/%s/oauth2/token", tenantId);
    String clientIdParam = String.format("client_id=%s", clientId);
    String resourceParam = String.format("resource=%s", resourceManagerUrl);
    String clientSecretParam = String.format("client_secret=%s", clientSecret);

    String bodyString = String.format("grant_type=client_credentials&%s&%s&%s", clientIdParam, resourceParam, clientSecretParam);

    HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(authUrl))
            .POST(HttpRequest.BodyPublishers.ofString(bodyString))
            .build();
    return request;
}

class AuthenticationBody {
    String access_token;
    String token_type;
    int expires_in;
    String scope;
    String refresh_token;
    String id_token;
    
    AuthenticationBody() {}
}
  

The preceding code would have to handle exceptions and status codes other than 200 OK, but shows the pattern:

Use the client ID and secret to validate that your program should have access
Use your tenant ID to specify where login.microsoftonline.com should be looking
Use Azure Resource Manager as the source of the authorization token

Use managed identity authentication

To authenticate to the workspace from a VM or compute cluster that is configured with a managed identity, use the MsiAuthentication class. The following example demonstrates how to use this class to authenticate to a workspace:

from azureml.core.authentication import MsiAuthentication

msi_auth = MsiAuthentication()

ws = Workspace(subscription_id="your-sub-id",
                resource_group="your-resource-group-id",
                workspace_name="your-workspace-name",
                auth=msi_auth
                )
  

Use Conditional Access

As an administrator, you can enforce Azure AD Conditional Access policies for users signing in to the workspace. For example, you can require two-factor authentication, or allow sign in only from managed devices. To use Conditional Access for Azure Machine Learning workspaces specifically, assign the Conditional Access policy to Machine Learning Cloud app.