Configure seamless single sign-on with SQL analytics in Amazon SageMaker Unified Studio

Amazon SageMaker Unified Studio offers a unified expertise for utilizing knowledge, analytics, and AI capabilities. SageMaker Unified Studio now helps trusted id propagation (TIP) for SQL workloads, enabling fine-grained knowledge entry management based mostly on particular person consumer identities. Organizations can use this integration to handle knowledge permissions by way of AWS Lake Formation whereas utilizing their present single sign-on (SSO) infrastructure.

Organizations already utilizing Amazon Redshift with TIP can prolong their present Lake Formation permissions to SageMaker Unified Studio. Customers merely log in by way of SSO and entry their licensed knowledge utilizing the SQL editor, sustaining constant safety controls throughout their analytics surroundings.

This submit demonstrates the best way to configure SageMaker Unified Studio with SSO, arrange initiatives and consumer onboarding, and entry knowledge securely utilizing built-in analytics instruments.

Answer overview

For our use case, a retail company is planning to implement gross sales analytics to determine gross sales patterns and product classes which might be doing effectively. It will assist the gross sales crew enhance on gross sales planning with focused promotions and assist the finance crew plan budgeting with higher stock administration. The company shops a buyer desk in an Amazon Easy Storage Service (Amazon S3) knowledge lake and a store_sales desk in a Redshift cluster.

The company makes use of SageMaker Unified Studio because the UI, with customers onboarded from their id supplier (IdP) to AWS IAM Identification Middle with TIP. Amazon SageMaker Lakehouse centralizes knowledge from Amazon S3 and Amazon Redshift, and Lake Formation offers fine-grained entry management based mostly on consumer id. For our instance use case, we discover two totally different customers. The next desk summarizes their roles, the instruments they use, and their knowledge entry.

Person	Group	Instrument	Knowledge Entry
Ethan (Knowledge Analyst)	Gross sales	Amazon Athena for interactive SQL evaluation	Non-sensitive buyer knowledge (id, c_country, birth_year) and store_sales full desk entry
Frank (BI Analyst)	Finance	Amazon Redshift for reviews and visualization	US buyer knowledge (c_country='US')

The next diagram illustrates the answer structure.

SageMaker Unified Studio with IAM Identification Middle simplifies the consumer journey from authentication to knowledge evaluation. The workflow consists of the next steps:

1. Customers check in with organizational SSO credentials by way of their IdP and are redirected to SageMaker Unified Studio.
2. Customers configure IAM Identification Middle authentication for Amazon Redshift, linking id administration with knowledge entry.
3. Customers entry the question editor for Amazon Redshift or SageMaker Lakehouse, triggering IAM Identification Middle federation to generate session and entry tokens.
4. SageMaker Unified Studio retrieves consumer authorization particulars and group membership utilizing the session token.
5. Customers are authenticated as IAM Identification Middle customers, able to discover and analyze knowledge utilizing Amazon Redshift and Amazon Athena.

To implement our resolution, we stroll by way of the next high-level steps:

1. Arrange SageMaker Lakehouse assets.
2. Create a SageMaker Unified Studio area with SSO and TIP enabled.
3. Configure Amazon Redshift for TIP and validate entry.
4. Validate knowledge entry utilizing Amazon Athena.

Conditions

Earlier than you start implementing the answer, you have to have the next in place:

1. For those who don’t have an AWS account, you possibly can join for one.
2. We offer utility scripts to assist arrange varied sections of the submit. To make use of them:
1. Proper-click this hyperlink and save the utility scripts zip file.
2. Unzip the file to a terminal that has the AWS Command Line Interface (AWS CLI) configured. You can even use AWS CloudShell.
3. Run the scripts solely when prompted within the related sections.

Word: The utility scripts are configured for
us-east-1 area. For those who favor one other area, edit the area within the scripts earlier than working them.

3. To deploy the infrastructure, right-click this hyperlink and choose ‘Save Hyperlink As’ to reserve it as sagemaker-unified-studio-infrastructure.yaml. Then add the file when creating a brand new stack within the AWS CloudFormation console, which is able to create the next assets:
   1. An S3 bucket to carry the shopper knowledge used on this submit.
   2. An AWS Identification and Entry Administration (IAM) position referred to as DataTransferRole with permissions as outlined in Conditions for managing Amazon Redshift namespaces within the AWS Glue Knowledge Catalog.
   3. An IAM position referred to as IAMIDCRedshiftRole, which will likely be used later to arrange the IAM Identification Middle Redshift software.
   4. An IAM position referred to as LakeFormationRegistrationRole, following the directions in Necessities for roles used to register areas, and vital IAM insurance policies.
4. For those who don’t have a Lake Formation consumer, you possibly can create one. For this submit, we use an admin consumer. For directions, see Create a knowledge lake administrator.
5. If IAM Identification Middle will not be enabled, discuss with Enabling AWS IAM Identification Middle for directions to allow it.
   1. If you want to migrate present Redshift customers and teams, use the IAM Identification Middle Redshift migration utility.
   2. For a fast approach to take a look at the function and familiarize your self with the method, we offer a script to generate mock customers and teams. Run the setup-idc.sh script, which is offered in Step 2, to create take a look at customers and teams in IAM Identification Middle for demonstration functions.
6. Combine IAM Identification Middle with Lake Formation. For directions, see Connecting Lake Formation with IAM Identification Middle.
7. Register the S3 bucket as a knowledge lake location:
   1. On the Lake Formation console, select Knowledge lake areas within the navigation pane.
   2. Select Register location.
   3. For the position, use LakeFormationRegistrationRole.
8. Create an IAM Identification Middle Redshift software, as detailed in our earlier submit:
   1. On the Amazon Redshift console, select IAM Identification Middle connections within the navigation pane and select Create software.
   2. For each the show title and software title, enter redshift-idc-app.
   3. Set the IdP namespace to awsidc.
   4. Select IAMIDCRedshiftRole because the IAM position.
   5. Select Subsequent to create the appliance.
   6. Pay attention to the appliance Amazon Useful resource Title (ARN) to make use of in subsequent steps. The ARN format is arn:aws:sso:::software/ssoins-/apl-.
9. For those who don’t have present Redshift tables to work with, run the script setup-producer-redshift.sh, which is offered in Step 2, to create a producer namespace and workgroup, arrange a pattern gross sales database, and generate vital tables with take a look at knowledge.
10. The submit additionally makes use of simulated buyer knowledge saved within the AWS Glue Knowledge Catalog. To arrange this knowledge and configure the mandatory Lake Formation permissions, run the setup-glue-tables-and-access.sh script offered in Step 2.

Arrange SageMaker Lakehouse assets

On this part, we configure the foundational lakehouse assets required for SageMaker to entry and analyze knowledge throughout a number of storage programs. We’ll register the Redshift occasion to the AWS Glue Knowledge Catalog to make warehouse knowledge discoverable and set up Lake Formation permissions on lakehouse assets for consumer identities to make sure safe, ruled entry to each knowledge lake and knowledge warehouse assets from inside SageMaker environments.

Register Redshift occasion to the Knowledge Catalog

On this step, we use the store_sales knowledge, which we created earlier utilizing the setup-producer-redshift.sh script. You possibly can register complete clusters to the Knowledge Catalog and create catalogs managed by AWS Glue. To register a cluster to the Knowledge Catalog, full the next steps:

1. On the Lake Formation console, select Administrative roles and duties within the navigation pane.
2. Below Knowledge lake directors, select Add.
3. Select Learn-only administrator, then select AWSServiceRoleForRedshift.
4. On the Amazon Redshift console, open your namespace.
5. On the Actions dropdown menu, selected Register with AWS Glue Knowledge Catalog, then select Register.
   
6. Check in to the Lake Formation console as the information lake administrator and select Catalogs within the navigation pane.
7. Below Pending catalog invites, choose the namespace and settle for the invitation by selecting Approve and create catalog.
   
8. Present the title for the catalog as salescatalog.
9. Choose Entry this catalog from Apache Iceberg suitable engines, select DataTransferRole for the IAM position, then select Subsequent.
10. Select Add permissions and select the admin IAM position beneath IAM customers and roles.
11. Choose Tremendous consumer for catalog permissions and select Add.
12. Select Subsequent.
13. Select Create catalog.
    

Arrange Lake Formation permission on lakehouse assets for consumer identities

On this part, we configure Lake Formation permissions to allow safe entry to lakehouse assets for federated consumer identities. Lake Formation offers fine-grained entry management that works seamlessly with IAM Identification Middle, permitting you to handle permissions centrally whereas sustaining safety boundaries.

We’ll deal with granting database entry to IAM Identification Middle teams in Lake Formation and setting table-level permissions for federated Redshift catalog tables. These permissions type the safety basis for our federated question structure, enabling customers to seamlessly entry each S3 knowledge lake and Redshift knowledge warehouse assets by way of a unified interface.

Grant database entry to IAM Identification Middle teams in Lake Formation

After you share your Redshift catalog with the Knowledge Catalog and combine with Lake Formation, you have to grant acceptable database entry. Comply with these steps to arrange permissions in your knowledge lake assets for company identities:

1. On the Lake Formation console, beneath Permissions within the navigation pane, select Knowledge permissions.
2. Select Grant.
3. Choose Principals for Principal kind.
4. Below Principals, choose IAM Identification Middle and select Add.
5. Within the pop-up window, if that is your first time assigning customers and teams, select Get began.
6. Seek for and choose the IAM Identification Middle teams awssso-sales and awssso-finance.
7. Select Assign.
8. Below LF-Tags or catalog assets, select Named Knowledge Catalog assets.
   1. Select :salescatalog/dev for Catalogs.
   2. Select sales_schema for Database.
9. Below Database permissions, choose Describe.
10. Select Grant to use the permissions.

Grant table-level permissions for federated Redshift catalog tables

Full the next steps to grant desk permissions to the IAM Identification Middle teams:

1. On the Lake Formation console, beneath Permissions within the navigation pane, select Knowledge permissions.
2. Select Grant.
3. Choose Principals for Principal kind.
4. Below Principals, choose IAM Identification Middle and select Add.
5. Within the pop-up window, if that is your first time assigning customers and teams, select Get began.
6. Seek for and choose the IAM Identification Middle group awssso-sales.
7. Select Assign.
8. Below LF-Tags or catalog assets, select Named Knowledge Catalog assets.
   1. Select :salescatalog/dev for Catalogs.
   2. Select sales_schema for Database.
   3. Select store_sales for Desk.
9. Choose Choose and Describe for Desk permissions.
10. Select Grant to use the permissions.

Create a SageMaker Unified Studio area with SSO and TIP enabled

For directions to create a SageMaker Unified Studio area, discuss with Create an Amazon SageMaker Unified Studio area – fast setup. As a result of your IAM Identification Middle integration is already full, you possibly can specify an IAM Identification Middle consumer within the area configuration settings.

Allow TIP in SageMaker Unified Studio

Full the next steps to allow TIP in SageMaker Unified Studio:

1. On the SageMaker console, use the AWS Area selector within the prime navigation bar to decide on the suitable Area.
2. Select View domains and select the area’s title from the checklist.
3. On the area’s particulars web page, on the Venture profiles tab, select a venture profile, for instance, SQL analytics.
4. Choose SQL analytics and select Edit.
5. Within the Blueprint parameters part, choose enableTrustedIdentityPropagationPermissions and select Edit.
   
6. Replace the worth as true.
7. To implement authorization-based on TIP, the SageMaker Unified Studio admin could make this parameter non-editable.
8. Select Save.
   

Allow consumer entry for SageMaker Unified Studio area

Full the next steps to allow consumer entry for the SageMaker Unified Studio area:

1. Open the SageMaker console within the acceptable Area and select Domains within the navigation pane.
2. Select an present SageMaker Unified Studio area the place you need to add SSO consumer entry.
3. On the area’s particulars web page, on the Person administration tab, within the Customers part, select Add and Add SSO customers and teams.
4. Select the consumer (for this submit, we add the consumer Frank) from the dropdown checklist and select Add customers and teams.

Add venture members

SageMaker Unified Studio initiatives facilitate crew collaboration for various enterprise initiatives. Because the venture proprietor, Ethan now can add Frank as a crew member to allow their collaboration. So as to add members to an present venture, full the next steps:

1. Check in to the SageMaker Unified Studio console utilizing the SSO credentials of who owns the venture (for this submit, Ethan).
2. Select Choose a venture.
3. Select the venture you need to edit.
4. On the Venture overview web page, develop Actions and select Handle members.
5. Select Add members.
6. Enter the title of the consumer or group you need to add (for this submit, we add Frank).
7. Choose Contributor if you wish to add the venture member as a contributor.
8. (Optionally available) Repeat these steps so as to add extra venture members. You possibly can add as much as eight venture members at a time.
9. Select Add members.

Create a SQL analytics venture in Unified Studio

On this step, we federate into SageMaker Unified Studio and create a venture utilizing SQL analytics. Full the next steps:

1. Federate into SageMaker Unified Studio utilizing your IAM Identification Middle credentials:
   1. On the SageMaker console, select Domains within the navigation pane.
   2. Copy the SageMaker Unified Studio URL in your area and enter it into a brand new browser window.
   3. Select Check in with SSO.
   4. A browser pop-up will redirect you to your most popular IdP login web page, the place you enter your IdP credentials.
   5. If authentication if profitable, you can be redirected to SageMaker Unified Studio.
2. After logging in, select Create venture.
3. Enter a reputation in your venture. This venture title is ultimate and might’t be modified later.
4. (Optionally available) Enter an outline in your venture. You possibly can edit this later.
5. Select a venture profile. For this demo, we select the SQL analytics profile from the obtainable templates.
   
6. Go away the default values as they’re or modify them in line with your use case, then select Proceed.
   
7. Select Create venture to finalize the venture and initialize your SQL analytics workspace.
   

For extra detailed info and superior configurations, discuss with Create a venture.

Configure Amazon Redshift for TIP and validate entry

Run the setup-consumer-redshift.sh script (offered within the conditions). This script will create a brand new namespace and workgroup and add the required tags, which you’ll use later to combine with SageMaker Unified Studio compute.

In case you are creating the cluster manually, add one of many following tags to the Redshift cluster or workgroup that you just need to add to SageMaker Unified Studio:

• Possibility 1 – Add a tag to permit solely a particular SageMaker Unified Studio venture to entry it: AmazonDataZoneProject=
• Possibility 2 – Add a tag to permit all SageMaker Unified Studio initiatives on this account to entry it: for-use-with-all-datazone-projects=true

Create compute utilizing IAM Identification Middle authentication

After you arrange your venture, the subsequent step is to determine a compute useful resource connection on the SageMaker Unified Studio console. Comply with these steps so as to add both Amazon Redshift Serverless or a provisioned cluster to your venture surroundings:

1. Go to the Compute part of your venture in SageMaker Unified Studio.
2. On the Knowledge warehouse tab, select Add compute.
3. You possibly can create a brand new compute useful resource or select an present one. For this submit, we select Hook up with present compute assets, then select Subsequent.
4. Select the kind of compute useful resource you need to add, then select Subsequent. For this submit, we select Redshift Serverless.
5. Below Connection properties, present the JDBC URL or the compute you need to add, which is built-in with IAM Identification Middle. If the compute useful resource is in the identical account as your SageMaker Unified Studio venture, you possibly can choose the compute useful resource from the dropdown menu. In our instance, we use the buyer account that was simply provisioned.
6. Below Authentication, choose IAM Identification Middle.
7. For Title, enter the title of the Redshift Serverless or provisioned cluster you need to add.
8. For Description, enter an outline of the compute useful resource.
9. Select Add compute.
   

The SageMaker Unified Studio Venture Compute and Knowledge pages will now show info for that useful resource.

If every thing is configured accurately, your compute will likely be created utilizing IAM Identification Middle. As a result of your IdP credentials are already cached when you’re logged in to SageMaker Unified Studio, it makes use of the identical credentials and creates the compute.

Check knowledge entry utilizing Amazon Redshift

When Ethan logs in to SageMaker Unified Studio utilizing IAM Identification Middle authentication, he efficiently federates and might entry buyer knowledge from all nations however just for non-sensitive columns. Let’s connect with Amazon Redshift in SageMaker Unified Studio by following these steps:

1. Select Actions and select Open Question editor.
2. Select Redshift within the Knowledge explorer pane.
3. Run the shopper gross sales calculation question to look at that consumer Ethan (a knowledge analyst) can entry buyer knowledge from all nations however solely non-sensitive columns (id, birth_country, product_id):

   choose current_user, c.*, sum(s.sales_amount) as total_sales
   from "awsdatacatalog"."customerdb"."buyer" c
   be a part of "dev@salescatalog"."sales_schema"."store_sales" s 
   on c.id=s.id
   group by all;

   

You’ve efficiently configured Redshift to make use of IAM Identification Middle authentication in SageMaker Unified Studio.

Validate knowledge entry utilizing Amazon Athena

When Frank logs in to SageMaker Unified Studio utilizing IAM Identification Middle authentication, he efficiently federates and might entry buyer knowledge just for america. To question with Athena, full the next steps:

1. Select Actions and select Open Question editor.
2. Select Lakehouse within the Knowledge explorer pane.
3. Discover AwsDataCatalog, develop the database, select the respective desk, and on the choices menu (three dots), select Preview knowledge.

The next demonstration illustrates how consumer Frank, a BI analyst, can carry out SQL evaluation utilizing Athena. Attributable to row-level filtering applied by way of Lake Formation, Frank’s entry is restricted to buyer knowledge from america solely. Moreover, you possibly can observe that within the Knowledge explorer pane, Frank can solely view the customerdb database. The dev@salescatalog database will not be seen to Frank as a result of no entry has been granted to his respective group from Lake Formation.

The IAM Identification Middle authentication integration is full; you should use each Amazon Redshift and Athena by way of SageMaker Unified Studio in a simplified, all-in-one interface.Word that, on the time of writing, Athena doesn’t work with Redshift Managed Storage (RMS).

Clear up

Full the next steps to wash up the assets you created as a part of this submit:

1. Delete the information from the S3 bucket.
2. Delete the Knowledge Catalog objects.
3. Delete the Lake Formation assets and Athena account.
4. Delete the SageMaker Unified Studio venture and related area.
5. For those who created new Redshift cluster for testing this resolution, delete the cluster.

Conclusion

On this submit, we offered a complete information to enabling trusted id propagation inside SageMaker Unified Studio. We coated the setup of a SageMaker Unified Studio area with SSO, the creation of tailor-made initiatives, environment friendly consumer onboarding with acceptable permissions, and the administration of AWS Glue and Amazon Redshift managed catalog permissions utilizing Lake Formation. By way of sensible examples, we demonstrated the best way to use each Amazon Redshift and Athena inside SageMaker Unified Studio, showcasing safe knowledge entry and evaluation capabilities. This strategy helps organizations keep strict id controls whereas serving to knowledge scientists and analysts derive priceless insights from each knowledge lake and knowledge warehouse environments, supporting each safety and productiveness in machine studying workflows.

For extra info on this integration, discuss with Trusted id propagation.

---

Concerning the authors