Summarize name transcriptions securely with Amazon Transcribe and Amazon Bedrock Guardrails

Given the quantity of conferences, interviews, and buyer interactions in fashionable enterprise environments, audio recordings play a vital position in capturing precious data. Manually transcribing and summarizing these recordings is usually a time-consuming and tedious process. Fortuitously, developments in generative AI and automated speech recognition (ASR) have paved the way in which for automated options that may streamline this course of.

Customer support representatives obtain a excessive quantity of calls every day. Beforehand, calls have been recorded and manually reviewed later for compliance, rules, and firm insurance policies. Name recordings needed to be transcribed, summarized, after which redacted for private identifiable data (PII) earlier than analyzing calls, leading to delayed entry to insights.

Redacting PII is a crucial follow in safety for a number of causes. Sustaining the privateness and safety of people’ private data is just not solely a matter of moral duty, but in addition a authorized requirement. On this submit, we present you how you can use Amazon Transcribe to get close to real-time transcriptions of calls despatched to Amazon Bedrock for summarization and delicate knowledge redaction. We’ll stroll by means of an structure that makes use of AWS Step Features to orchestrate the method, offering seamless integration and environment friendly processing

Amazon Bedrock is a completely managed service that gives a selection of high-performing basis fashions (FMs) from main mannequin suppliers reminiscent of AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Mistral AI, and Amazon by means of a single API, together with a broad set of capabilities you’ll want to construct generative AI purposes with safety, privateness, and accountable AI. You should utilize  Amazon Bedrock Guardrails to redact delicate data reminiscent of PII discovered within the generated name transcription summaries. Clear, summarized transcripts are then despatched to analysts. This supplies faster entry to name traits whereas defending buyer privateness.

Resolution overview

The structure of this answer is designed to be scalable, environment friendly, and compliant with privateness rules. It contains the next key elements:

1. Recording – An audio file, reminiscent of a gathering or assist name, to be transcribed and summarized
2. Step Features workflow – Coordinates the transcription and summarization course of
3. Amazon Transcribe – Converts audio recordings into textual content
4. Amazon Bedrock – Summarizes the transcription and removes PII
5. Amazon SNS – Delivers the abstract to the designated recipient
6. Recipient – Receives the summarized, PII-redacted transcript

The next diagram exhibits the structure overflow –

The workflow orchestrated by Step Features is as follows:

1. An audio recording is offered as an enter to the Step Features workflow. This could possibly be achieved manually or mechanically relying on the particular use case and integration necessities.
2. The workflow invokes Amazon Transcribe, which converts the multi-speaker audio recording right into a textual, speaker-partition transcription. Amazon Transcribe makes use of superior speech recognition algorithms and machine studying (ML) fashions to precisely partition audio system and transcribe the audio, dealing with varied accents, background noise, and different challenges.
3. The transcription output from Amazon Transcribe is then handed to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock by means of AWS Lambda. This mannequin was chosen as a result of it has comparatively decrease latency and value than different fashions. The mannequin first summarizes the transcript in accordance with its abstract directions, after which the summarized output (the mannequin response) is evaluated by Amazon Bedrock Guardrails to redact PII. To be taught the way it blocks dangerous content material, seek advice from How Amazon Bedrock Guardrails works. The directions and transcript are each handed to the mannequin as context.
4. The output from Amazon Bedrock is saved in Amazon Easy Storage Service (Amazon S3) and despatched to the designated recipient utilizing Amazon Easy Notification Service (Amazon SNS). Amazon SNS helps varied supply channels, together with electronic mail, SMS, and cell push notifications, ensuring that the abstract reaches the supposed recipient in a well timed and dependable method

The recipient can then assessment the concise abstract, shortly greedy the important thing factors and insights from the unique audio recording. Moreover, delicate data has been redacted, sustaining privateness and compliance with related rules.

The next diagram exhibits the Step Features workflow –

Stipulations

Observe these steps earlier than beginning:

1. Amazon Bedrock customers must request entry to fashions earlier than they’re out there to be used. This can be a one-time motion. For this answer, you’ll want to allow entry to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock. For extra data, seek advice from Entry Amazon Bedrock basis fashions. Deployment, as described under, is at the moment supported solely within the US West (Oregon) us-west-2 AWS Area. Customers could discover different fashions if desired. You may want some customizations to deploy to various Areas with completely different mannequin availability (reminiscent of us-east-1, which hosts Anthropic’s Claude 3.5 Sonnet). Ensure you take into account mannequin high quality, velocity, and value tradeoffs earlier than selecting a mannequin.
2. Create a guardrail for PII redaction. Configure filters to dam or masks delicate data. This feature could be discovered on the Amazon Bedrock console on the Add delicate data filters web page when making a guardrail. To learn to configure filters for different use circumstances, seek advice from Take away PII from conversations through the use of delicate data filters.

Deploy answer assets

To deploy the answer, obtain an AWS CloudFormation template to mechanically provision the mandatory assets in your AWS account. The template units up the next elements:

• A Step Features workflow
• Lambda capabilities
• An SNS subject
• An S3 bucket
• AWS Key Administration Service (AWS KMS) keys for knowledge encryption and decryption

By utilizing this template, you’ll be able to shortly deploy the pattern answer with minimal guide configuration. The template requires the next parameters:

• E-mail handle used to ship abstract – The abstract shall be despatched to this handle. You will need to acknowledge the preliminary Amazon SNS affirmation electronic mail earlier than receiving further notifications.
• Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract
• Guardrail ID – That is the ID of your just lately created guardrail, which could be discovered on the Amazon Bedrock Guardrails console in Guardrail overview

The Abstract directions are learn into your Lambda operate as an surroundings variable.

 
# Use the offered directions to supply the abstract. Use a default if no intructions are offered.
SUMMARY_INSTRUCTIONS = os.getenv('SUMMARY_INSTRUCTIONS')
 
These are then used as a part of your payload to Anthropic’s Claude 3 Haiku mannequin. That is shared to offer you an understanding of how you can move the directions and textual content to the mannequin.
 
# Create the payload to supply to the Anthropic mannequin.
        user_message = {"position": "consumer", "content material": f"{SUMMARY_INSTRUCTIONS}{transcript}"}
        messages = [user_message]
response = generate_message(bedrock_client, 'anthropic.claude-3-haiku-20240307-v1:0"', "", messages, 1000)
 
The generate_message() operate incorporates the invocation to Amazon Bedrock with the guardrail ID and different related parameters.
 
def generate_message(bedrock_runtime, model_id, system_prompt, messages, max_tokens):
    physique = json.dumps(
        {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens,
            "system": system_prompt,
            "messages": messages
        }
    )
print(f'Invoking mannequin: {BEDROCK_MODEL_ID}')
 
    response = bedrock_runtime.invoke_model(
        physique=physique,
        modelId=BEDROCK_MODEL_ID,
        # contentType=contentType,
        guardrailIdentifier =BEDROCK_GUARDRAIL_ID,
        guardrailVersion ="1",
        hint ="ENABLED")
    response_body = json.masses(response.get('physique').learn())
    print(f'response: {response}')
    return response_body

Deploy the answer

After you deploy the assets utilizing AWS CloudFormation, full these steps:

1. Add a Lambda layer.

Though AWS Lambda usually updates the model of AWS Boto3 included, on the time of scripting this submit, it nonetheless supplies model 1.34.126. To make use of Amazon Bedrock Guardrails, you want model 1.34.90 or increased, for which we’ll add a Lambda layer that updates the Boto3. You possibly can comply with the official developer information on how you can add a Lambda layer.

There are other ways to create a Lambda layer. A easy methodology is to make use of the steps outlined in Packaging the layer content material, which references a pattern utility repo. It’s best to be capable of substitute requests==2.31.0 inside necessities.txt content material to boto3, which can set up the most recent out there model, then create the layer.

So as to add the layer to Lambda, ensure that the parameters laid out in Creating the layer match the deployed Lambda. That’s, you’ll want to replace compatible-architectures to x86_64.

2. Acknowledge the Amazon SNS electronic mail affirmation that you need to obtain a number of moments after creating the CloudFormation stack
3. On the AWS CloudFormation console, discover the stack you simply created
4. On the stack’s Outputs tab, search for the worth related to AssetBucketName. It should look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
5. On the Amazon S3 console, discover your S3 property bucket.

That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.

6. Add your recording to the recordings folder in Amazon S3

Importing recordings will mechanically set off the AWS Step Features state machine. For this instance, we use a pattern crew assembly recording from the pattern recording.

7. On the AWS Step Features console, discover the summary-generator state machine. Select the identify of the state machine run with the standing Working.

Right here, you’ll be able to watch the progress of the state machine because it processes the recording. After it reaches its Success state, you need to obtain an emailed abstract of the recording. Alternatively, you’ll be able to navigate to the S3 property bucket and consider the transcript there within the transcripts folder.

Increase the answer

Now that you’ve got a working answer, listed here are some potential concepts to customise the answer to your particular use circumstances:

• Attempt altering the method to suit your out there supply content material and desired outputs:
  • For conditions the place transcripts can be found, create an alternate AWS Step Features workflow to ingest current text-based or PDF-based transcriptions
  • As an alternative of utilizing Amazon SNS to inform recipients by means of electronic mail, you should utilize it to ship the output to a special endpoint, reminiscent of a crew collaboration web site or to the crew’s chat channel
• Attempt altering the abstract directions for the AWS CloudFormation stack parameter offered to Amazon Bedrock to provide outputs particular to your use case. The next are some examples:
  • When summarizing an organization’s earnings name, you might have the mannequin concentrate on potential promising alternatives, areas of concern, and issues that you need to proceed to observe
  • When you’re utilizing the mannequin to summarize a course lecture, it might determine upcoming assignments, summarize key ideas, listing info, and filter out small discuss from the recording
• For a similar recording, create completely different summaries for various audiences:
  • Engineers’ summaries concentrate on design choices, technical challenges, and upcoming deliverables
  • Venture managers’ summaries concentrate on timelines, prices, deliverables, and motion objects
  • Venture sponsors get a short replace on venture standing and escalations
  • For longer recordings, strive producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, you may need to alter the max_tokens_to_sample parameter to accommodate completely different content material lengths.

Clear up

Clear up the assets you created for this answer to keep away from incurring prices. You should utilize an AWS SDK, the AWS Command Line Interface (AWS CLI), or the console.

1. Delete Amazon Bedrock Guardrails and the Lambda layer you created
2. Delete the CloudFormation stack

To make use of the console, comply with these steps:

3. On the Amazon Bedrock console, within the navigation menu, choose Guardrails. Select your guardrail, then choose Delete.
4. On the AWS Lambda console, within the navigation menu, choose Layers. Select your layer, then choose Delete.
5. On the AWS CloudFormation console, within the navigation menu, choose Stacks. Select the stack you created, then choose Delete.

Deleting the stack gained’t delete the related S3 bucket. When you not require the recordings or transcripts, you’ll be able to delete the bucket individually. Amazon Transcribe is designed to mechanically delete transcription jobs after 90 days. Nevertheless, you’ll be able to decide to manually delete these jobs earlier than the 90-day retention interval expires.

Conclusion

As companies flip to knowledge as a basis for decision-making, being able to effectively extract insights from audio recordings is invaluable. By utilizing the facility of generative AI with Amazon Bedrock and Amazon Transcribe, your group can create concise summaries of audio recordings whereas sustaining privateness and compliance. The proposed structure demonstrates how AWS providers could be orchestrated utilizing AWS Step Features to streamline and automate complicated workflows, enabling organizations to concentrate on their core enterprise actions.

This answer not solely saves effort and time, but in addition makes certain that delicate data is redacted, mitigating potential dangers and selling compliance with knowledge safety rules. As organizations proceed to generate and course of giant volumes of audio knowledge, options like this can grow to be more and more necessary for gaining insights, making knowledgeable choices, and sustaining a aggressive edge.

Concerning the authors

Yash Yamsanwar is a Machine Studying Architect at Amazon Internet Providers (AWS). He’s accountable for designing high-performance, scalable machine studying infrastructure that optimizes the total lifecycle of machine studying fashions, from coaching to deployment. Yash collaborates carefully with ML analysis groups to push the boundaries of what’s potential with LLMs and different cutting-edge machine studying applied sciences.

Sawyer Hirt is a Options Architect at AWS, specializing in AI/ML and cloud architectures, with a ardour for serving to companies leverage cutting-edge applied sciences to beat complicated challenges. His experience lies in designing and optimizing ML workflows, enhancing system efficiency, and making superior AI options extra accessible and cost-effective, with a specific concentrate on Generative AI. Outdoors of labor, Sawyer enjoys touring, spending time with household, and staying present with the most recent developments in cloud computing and synthetic intelligence.