Allow or disable ACL crawling safely in Amazon Q Enterprise

Amazon Q Enterprise just lately added help for directors to switch the default entry management listing (ACL) crawling function for knowledge supply connectors.

Amazon Q Enterprise is a totally managed, AI powered assistant with enterprise-grade safety and privateness options. It contains over 40 knowledge supply connectors that crawl and index paperwork. By default, Amazon Q Enterprise indexes ACL data hooked up to paperwork together with the paperwork themselves and makes use of this to filter chat responses primarily based on the person’s doc entry. With this new function, you may allow or disable ACL crawling as required by their enterprise use case.

This put up introduces the brand new ACL toggle function for Amazon Q Enterprise, which you should use to allow or disable ACL crawling. We’ll discover use instances for disabling ACLs and talk about methods to safely allow or disable ACL crawling.

Overview of entry management listing crawling

Amazon Q Enterprise knowledge supply connectors assist crawl varied knowledge sources to gather and index content material in Amazon Q Enterprise for quick discovery and retrieval when answering person queries. These knowledge sources usually comprise paperwork with completely different classifications resembling public, inner public, personal, and confidential. To offer fine-grained management over entry rights, you may connect ACLs to paperwork, permitting you to specify completely different ranges of entry for varied customers or teams. To confirm that Amazon Q Enterprise respects entry management insurance policies and that customers solely obtain responses for content material they’re approved to entry, the info supply connectors robotically crawl for entry permissions related to the content material, person identifiers, and teams.

The previous determine illustrates the Amazon Q Enterprise knowledge supply crawler with ACL crawling enabled. Because the connector retrieves content material from the info supply, it examines the related ACL and compiles an inventory of customers and teams with learn permissions for every doc. The connector additionally collects person identifiers, that are saved within the Amazon Q Enterprise person retailer for fast matching throughout question execution. Each the ACL and content material are optimized and saved within the Amazon Q Enterprise index storage, enabling safe and environment friendly retrieval when answering person queries. For extra data on the person retailer, see Understanding Amazon Q Enterprise Consumer Retailer.

When to disable ACL crawling?

ACL crawling builds a security-aware index that respects entry management insurance policies within the major knowledge supply. This course of helps keep knowledge privateness and entry management required for regulatory compliance, ensuring that delicate data isn’t inadvertently uncovered by means of person question outcomes. It gives a scalable mechanism to deal with giant quantities of content material whereas sustaining consistency between the precise entry controls on the info and what’s discoverable by means of search. Due to these benefits, ACL crawling is strongly beneficial for all knowledge sources. Nevertheless, there are some circumstances while you may have to disable it. The next are some explanation why you may disable ACL crawling.

Internally public content material

Organizations usually designate sure knowledge sources as internally public, together with HR insurance policies, IT data bases, and wiki pages. As an illustration, an organization may allocate a whole Microsoft SharePoint web site for insurance policies accessible to all staff, classifying it as internal-public. In such instances, crawling ACLs for permissions that embody all staff might be expensive and create pointless overhead. Turning off ACL crawling is likely to be advantageous in these eventualities.

Knowledge supply comprises irreconcilable identities

Amazon Q Enterprise requires all customers to authenticate with an enterprise-approved identification supplier (IdP). After profitable authentication, Amazon Q Enterprise makes use of the IdP-provided person identifier to match in opposition to the person identifier fetched from the info supply throughout ACL crawling. This course of validates person entry to content material earlier than retrieving it for question responses.

Nevertheless, due to legacy points resembling mergers and acquisitions, knowledge supply configuration limitations, or different constraints, the first person identifier from the IdP may differ from the one within the knowledge supply. This discrepancy can forestall Amazon Q Enterprise from retrieving related content material from the index and answering person queries successfully.

In such instances, it is likely to be essential to disable ACL crawling and use different choices. These embody implementing attribute filters or constructing devoted restricted functions with entry restricted to particular audiences and content material. For extra data on attribute filters, see Filtering chat responses utilizing doc attributes.

Use case-driven focused deployments

As a totally managed service, Amazon Q Enterprise might be rapidly deployed in a number of situations for scoped down focused use instances. Examples embody an HR bot in Slack or an AI assistant for buyer help brokers in a contact heart. As a result of these AI assistants is likely to be deployed for a restricted viewers, and the listed content material is likely to be usually obtainable to all customers with utility entry, ACL crawling might be turned off.

Be aware of warning

Amazon Q Enterprise can not implement entry controls if ACL crawling is disabled. When ACL crawling is disabled for an information supply, listed content material in that supply might be thought-about accessible to customers with entry to the Amazon Q Enterprise utility. Subsequently, disabling ACL crawling needs to be finished with warning and due diligence. The next are some beneficial finest practices:

• Notify knowledge supply content material house owners and directors of your intent to disable ACL crawling and procure their approval beforehand.
• If relevant, think about implementing different choices resembling attribute filtering to limit content material retrieval or deploying a scoped-down, use-case-driven deployment to a restricted viewers.
• Preserve a choice doc that clearly articulates the explanations for disabling ACL crawling, the scope of affected content material, and precautions taken to stop indexing of delicate data.

Be aware: As a precaution, you can’t disable ACL crawling for an current Amazon Q Enterprise knowledge supply that already has ACL crawling enabled. To disable ACL crawling, you will need to delete the info supply and recreate it. You may solely disable ACL crawling throughout the knowledge supply creation course of, and this requires an account administrator to grant permission for disabling ACL crawling when configuring the info supply.

Procedures for configuring ACL crawling

Amazon Q Enterprise ACL crawling helps shield your knowledge. Amazon Q Enterprise gives safeguards to assist directors and builders mitigate unintentionally disabling ACL crawling. On this part, we are going to cowl how one can enable or deny the ACL crawling disable function, discover procedures to allow or disable ACL crawling, clarify methods to monitor logs for ACL crawling configuration adjustments, and troubleshoot frequent points.

Personas for configuring ACL crawling

ACL crawling configuration usually entails a number of roles, relying in your organizational construction. To maximise safeguards, it’s beneficial that these roles are stuffed by completely different people. For quicker deployments, determine the required personnel inside your group earlier than beginning the undertaking and guarantee they collaborate to finish the configuration. Listed here are the frequent roles wanted for ACL crawling configuration:

1. AWS account administrator – An AWS account administrator is a person with full entry to AWS providers and the power to handle IAM sources and permissions within the account. They will create and handle organizations, enabling centralized administration of a number of AWS accounts.
2. Amazon Q Enterprise administrator – An Amazon Q Enterprise administrator is often a person or function accountable for managing and configuring the Amazon Q Enterprise service. Their duties embody creating and optimizing Amazon Q Enterprise indexes, establishing guardrails, and tuning relevance. Additionally they arrange and keep connections to varied knowledge sources that Amazon Q Enterprise will index, resembling Amazon Easy Storage Service (Amazon S3) buckets, SharePoint, Salesforce, and Confluence.

Stipulations for ACL crawling

• Amazon Q Enterprise utility.
• Amazon Q Enterprise knowledge supply connector that helps ACL crawling configuration.
• Knowledge supply authentication that meets the permissions required for crawling content material and ACLs.

Course of to disallow the choice to disable ACL crawling

By default, the choice to disable ACL crawling is enabled for an account. AWS account directors can disallow this function by establishing an account-level coverage. It’s beneficial to configure an express deny for manufacturing accounts by default. The next beneath exhibits the related actions in relation to the personas concerned within the configuration course of.

Directors can connect the IAM motion qbusiness:DisableAclOnDataSource to the Amazon Q Enterprise administrator person or function coverage to disclaim or enable the choice to disable ACL crawling. The instance IAM coverage code snippet that follows demonstrates methods to arrange an express deny.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
          "Effect": "Deny",
          "Action": [
                "qbusiness:DisableAclOnDataSource"
            ],
          "Useful resource": ["*"]
       }
    ]
}

Be aware that even when the choice to disable ACL crawling is denied, the person interface won’t grey out this selection. Nevertheless, in case you try and create an information supply with this selection disabled, it is going to fail the validation verify, and Amazon Q Enterprise won’t create the info supply.

Course of to disable ACL crawling for an information supply connector

Earlier than establishing an information supply connector with ACL crawling disabled in your Amazon Q Enterprise utility deployment, just be sure you haven’t any delicate content material within the knowledge supply or have applied controls to assist forestall unintended content material publicity. Confirm that the info supply connector helps the choice to disable ACL crawling. Notify data custodians, content material house owners, and knowledge supply directors of your intent to disable ACL crawling and procure their documented approvals, if essential. In case your account administrator has explicitly denied the choice to disable ACL crawling, request non permanent permission. After you will have secured all approvals and exceptions, create a brand new knowledge supply with ACL crawling disabled and sync the info. With ACL crawling disabled, Amazon Q Enterprise customers will be capable to uncover data and procure solutions from the listed paperwork by means of this connector. Notify the account administrator to revert the account coverage again to explicitly denying the disable ACL crawling choice. The method and interplay between completely different roles are proven within the following chart.

The next is an outline of the process to create an information supply with ACL crawling disabled utilizing AWS Console:

1. Navigate to the Amazon Q Enterprise console.
2. Choose the Amazon Q Enterprise utility that you simply wish to add an information supply connector to.
3. Select Add knowledge supply within the Knowledge sources part and choose the specified connector.
4. Replace the connector configuration data. See Connecting Amazon Q Enterprise knowledge sources for configuration particulars.
5. Within the Authorization part, select Disable ACLs and verify the acknowledgment to just accept the dangers of disabling ACL crawling.
6. Full the remaining connector configuration and select Save.
7. Sync the info supply.

Be aware: You can’t disable ACL crawling for an current knowledge supply connector that was created with ACL crawling enabled. It’s essential to create a brand new knowledge supply connector occasion with ACL disabled and delete the older occasion that has ACL crawling enabled.

Course of to allow ACL crawling for an information supply connector

Creating an information supply connector with ACL crawling enabled is beneficial and doesn’t require further enable itemizing from AWS account directors. To allow ACL crawling, you observe steps just like disabling ACLs as described within the earlier part. When configuring the info supply connector utilizing the console, select Allow ACLs within the Authorization part to create a connector with ACL crawling enabled. You can too allow ACL crawling at any time for an current knowledge supply connector that was created with this selection disabled. Sync the info supply connector for the ACL enforcement to take impact. Amazon Q Enterprise customers can solely question and procure solutions from paperwork to which they’ve entry within the authentic knowledge supply.

It’s vital to overview that the info supply administrator has arrange the required permissions correctly, ensuring that the crawler has permission to crawl for ACLs within the knowledge supply earlier than enabling ACL crawling. You will discover the required permissions within the prerequisite part of the connector in Connecting Amazon Q Enterprise knowledge sources. The next exhibits the method for establishing an information supply connector with ACL crawling enabled.

Logging and monitoring the ACL crawling configuration

Amazon Q Enterprise makes use of AWS CloudTrail for logging API calls associated to ACL crawling configuration. You may monitor the CloudTrail log for CreateDataSource and UpdateDataSource API calls to determine ACL crawling-related adjustments made to knowledge supply configuration. For an entire listing of Amazon Q Enterprise APIs which might be logged to CloudTrail, see Logging Amazon Q Enterprise API calls utilizing AWS CloudTrail.

Directors can configure Amazon CloudWatch alarms to generate automated alert notifications if ACL crawling is disabled for an information supply connector, permitting them to provoke corrective motion. For step-by-step directions on establishing CloudWatch alarms primarily based on CloudTrail occasions, see How do I exploit CloudWatch alarms to observe CloudTrail occasions.

The instance CloudWatch alarm code snippet that follows exhibits the filter sample for figuring out occasions associated to disabling ACL crawling in an information supply connector.

 ($.eventName = UpdateDataSource)
    )
    && ($.requestParameters.disableAclCrawl = true) 

Suggestions for troubleshooting

When configuring Amazon Q Enterprise knowledge supply connectors, you may often encounter points. The next are some frequent errors and their attainable resolutions.

Not approved to disable ACL crawling

When creating a brand new knowledge supply connector with ACL crawling disabled, you may see an error message stating not approved to carry out: qbusiness:DisableAclOnDataSource as proven within the following picture.

This error signifies that your administrator has explicitly denied the choice to disable ACL crawling to your AWS account. Contact your administrator to allow-list this motion to your account. For extra particulars, see the Course of to disable ACL crawling for an information supply connector part earlier on this put up.

Knowledge supply connection errors

Knowledge supply connectors may also fail to connect with your knowledge supply or crawl knowledge. In such instances, confirm that Amazon Q Enterprise can attain the info supply by means of the general public web or by means of a VPC personal community. See Connecting Amazon Q Enterprise knowledge sources to guarantee that your knowledge supply authentication has the permissions wanted to crawl content material and ACLs, if enabled.

Identification and ACL mismatch errors

Lastly, after efficiently syncing knowledge with ACL crawling enabled, some customers may nonetheless be unable to get solutions to queries, despite the fact that the related paperwork had been listed. This situation generally happens when the person lacks entry to the listed content material within the authentic knowledge supply, or when the person identification obtained from the info supply doesn’t match the sign-in identification. To troubleshoot such ACL mismatch points, look at the info supply sync report. For extra data, see Introducing document-level sync stories: Enhanced knowledge sync visibility in Amazon Q Enterprise.

Key issues and proposals

Given the influence that disabling ACL crawling can have on content material safety, think about these restrictions and finest practices when disabling ACL crawling in Amazon Q Enterprise knowledge supply connectors:

• ACL crawling enablement is a one-way management mechanism. After it’s enabled, you can’t disable it. This helps forestall unintentionally disabling ACL crawling in manufacturing environments.
• Hold ACL crawling enabled by default and disable it just for the subset of information supply connectors that require it.
• If essential, think about splitting the indexing of an information supply by establishing a number of knowledge supply connectors and limiting ACL crawling disablement to a smaller content material phase. Use the doc Inclusion and Exclusion function of information supply connectors to outline the indexing scope.
• When ACL crawling is disabled due to irreconcilable identities, think about different choices. These embody implementing attribute filters, proscribing entry to the Amazon Q Enterprise utility, and establishing guardrails.
• As a safety finest follow, AWS Organizations and account directors ought to add a service management coverage to explicitly deny the qbusiness:DisableAclOnDataSource permission for all accounts. Grant this permission solely when requested by an Amazon Q Enterprise administrator. After configuring an information supply connector with ACL crawling disabled, revert to an express deny. Use a ticketing system to keep up a report of exception approvals. For extra data, see .
• At present, disabling ACL crawling is accessible for restricted connectors, together with ServiceNow, Confluence, SharePoint, Jira, Google Drive, OneDrive, Salesforce, Zendesk, GitHub, MS Groups, and Slack. For the newest listing of connectors that help disabling ACL crawling, see Connecting Amazon Q Enterprise knowledge sources.

Clear up

To keep away from incurring further fees, be sure to delete any sources created on this put up.

1. To delete any knowledge supply created in Amazon Q Enterprise, observe the directions in Deleting an Amazon Q Enterprise knowledge supply connector to delete the identical.
2. To delete any Amazon Q Enterprise utility created, observe the directions in Deleting an utility.

Conclusion

Amazon Q Enterprise knowledge supply connector ACL crawling is an important function that helps organizations construct, handle, and scale safe AI assistants. It performs a vital function in imposing regulatory and compliance insurance policies and defending delicate content material. With the introduction of a self-service function to disable ACL crawling, Amazon Q Enterprise now gives you extra autonomy to decide on deployment choices that fit your group’s enterprise wants. To begin constructing safe AI assistants with Amazon Q Enterprise, discover the Getting began information.

In regards to the Authors

Rajesh Kumar Ravi, a Senior Options Architect at Amazon Internet Providers, makes a speciality of constructing generative AI options utilizing Amazon Q Enterprise, Amazon Bedrock, and Amazon Kendra. He helps companies worldwide implement these applied sciences to boost effectivity, innovation, and competitiveness. An completed expertise chief, Rajesh has expertise creating progressive AI merchandise, nurturing the builder group, and contributing to new concepts. Outdoors of labor, he enjoys strolling and brief mountaineering journeys.

Meenakshisundaram Thandavarayan works for AWS as an AI/ML Specialist. He has a ardour to design, create, and promote human-centered knowledge and analytics experiences. Meena focuses on creating sustainable programs that ship measurable, aggressive benefits for strategic prospects of AWS. Meena is a connector and design thinker and strives to drive enterprise to new methods of working by means of innovation, incubation, and democratization.

Amit Choudhary is a Product Supervisor for Amazon Q Enterprise connectors. He likes to construct merchandise that make it straightforward for patrons to make use of privacy-preserving applied sciences (PETs) resembling differential privateness

Keerthi Kumar Kallur is a Software program Improvement Engineer at AWS. He’s a part of the Amazon Q Enterprise workforce and labored on varied options with prospects. In his spare time, he likes to do outside actions resembling mountaineering and sports activities resembling volleyball.