This content originally appeared on DEV Community and was authored by Julian Michel
Amazon Q Business, like almost all other AWS services, can be configured using the AWS Console. To deploy it in production environments, you probably want to use Infrastructure as Code to avoid manual configuration tasks. This blog post shows how to deploy Amazon Q Business using AWS CDK, including a Confluence integration as a data source. For some basic understanding, take a look at the previous blog post where I configured everything manually and also described the prerequisites such as AWS IAM Identity Center.
Currently, only L1 constructs based on CloudFormation resource types are supported. Using the L1 construct is more complicated. However, I implemented the CDK code based on my previous manual configuration, which was very helpful.
You can also find the full source code in my GitHub repo.
Q Business CloudFormation resource types overview
CloudFormation provides six resource types for deploying Q Business:
- AWS::QBusiness::Application - The main application
- AWS::QBusiness::Index - The index that contains the knowledge imported from the data sources
- AWS::QBusiness::Retriever - The retriever that reads data from the index for Q Business conversations
- AWS::QBusiness::WebExperience - The frontend application
- AWS::QBusiness::DataSource - Configuration of data sources
- AWS::QBusiness::Plugin - Q Business plugins (e.g. Jira write access) - not used in this example
Most resources require IAM roles. In this blog post, we will first create the IAM role for each resource and then create the Q-Business resource.
Getting started with the AWS CDK project
This project is based on a new AWS CDK TypeScript project. In my sample project, I have added configuration variables for the AWS IAM Identity Center ARN (identityCenterInstanceArn) and the URL of the Confluence data source (confluenceHostUrl). Feel free to add more/other configuration variables - this is sufficient for my sample project.
File bin/cdk-q-business-confluence.ts contains values for these configuration options.
new CdkQBusinessConfluenceStack(
app,
'CdkQBusinessConfluenceStack',
{
identityCenterInstanceArn:
'arn:aws:sso:::instance/ssoins-123467890abc',
confluenceHostUrl: 'https://my-domain.atlassian.net/',
},
);
These values are passed to the CDK Stack that is deployed (see filename lib/cdk-q-business-confluence-stack.ts).
interface QBusinessStackProps extends cdk.StackProps {
identityCenterInstanceArn: string;
confluenceHostUrl: string;
}
export class CdkQBusinessConfluenceStack extends cdk.Stack {
constructor(
scope: Construct,
id: string,
props: QBusinessStackProps,
) {
super(scope, id, props);
Confluence authentication credentials
Q Business reads data from Confluence to add it to the index. The credentials are stored in a Secret Manager Secret, which is created first. The encryption key is only used for the secret - feel free to use it to encrypt data in Q Business itself (my cluster is currently unencrypted as it only contains test data generated for this purpose).
The source code doesn't contain the actual secrets - they have to be maintained manually in the Secrets Manager in the AWS console. If you also want to deploy your secrets/passwords with AWS CDK, use the cdk-sops-secrets library, which supports this in a secure way.
const encryptionKey = new kms.Key(
this,
'QBusinessKey',
{
alias: 'QBusinessKey',
pendingWindow: cdk.Duration.days(7),
removalPolicy: cdk.RemovalPolicy.DESTROY, // for testing only
},
);
const secret = new secretsmanager.Secret(
this,
'Secret',
{
secretObjectValue: {
username: cdk.SecretValue.unsafePlainText(
'dummy value - please chanage manually after deployment',
),
hostUrl: cdk.SecretValue.unsafePlainText(
'dummy value - please chanage manually after deployment',
),
password: cdk.SecretValue.unsafePlainText(
'dummy value - please chanage manually after deployment',
),
},
encryptionKey,
},
);
IAM policy and role for the Q Business application
The Q Business application mainly requires some CloudWatch Logs permissions.
How do I determine which permissions are required? I first configured Q Business manually. It provides options to automatically generate all required IAM roles. Then I transferred the permissions to the AWS CDK code. I recommend doing this for all IAM roles used in Q Business as long as no L2 Construct is available in AWS CDK.
const applicationPolicy = new iam.ManagedPolicy(
this,
'ApplicationPolicy',
{
statements: [
new iam.PolicyStatement({
sid: 'AmazonQApplicationPutMetricDataPermission',
actions: ['cloudwatch:PutMetricData'],
resources: ['*'],
conditions: {
StringEquals: {
'cloudwatch:namespace': 'AWS/QBusiness',
},
},
}),
new iam.PolicyStatement({
sid: 'AmazonQApplicationDescribeLogGroupsPermission',
actions: ['logs:DescribeLogGroups'],
resources: ['*'],
}),
new iam.PolicyStatement({
sid: 'AmazonQApplicationCreateLogGroupPermission',
actions: ['logs:CreateLogGroup'],
resources: [
`arn:aws:logs:${this.region}:${this.account}:log-group:/aws/qbusiness/*`,
],
}),
new iam.PolicyStatement({
sid: 'AmazonQApplicationLogStreamPermission',
actions: [
'logs:DescribeLogStreams',
'logs:CreateLogStream',
'logs:PutLogEvents',
],
resources: [
`arn:aws:logs:${this.region}:${this.account}:log-group:/aws/qbusiness/*:log-stream:*`,
],
}),
],
},
);
const applicationRole = new iam.Role(
this,
'ApplicationRole',
{
assumedBy: new iam.ServicePrincipal(
'qbusiness.amazonaws.com',
{
conditions: {
StringEquals: {
'aws:SourceAccount': this.account,
},
ArnLike: {
'aws:SourceArn': `arn:aws:qbusiness:${this.region}:${this.account}:application/*`,
},
},
},
),
managedPolicies: [applicationPolicy],
},
);
Q Business application
The Q Business application itself has only a few parameters. Always check which additional parameters are supported, e.g. you need to specify an encryption key if you want to encrypt data in Q Business.
const application = new q.CfnApplication(
this,
'Application',
{
displayName: 'CDK_QBusiness',
identityCenterInstanceArn:
props.identityCenterInstanceArn,
roleArn: applicationRole.roleArn,
},
);
Q Business index
The Q Business index stores the knowledge from the data sources. In the example, a starter index type is configured with one capacity unit.
const index = new q.CfnIndex(this, 'Index', {
type: 'STARTER',
capacityConfiguration: {
units: 1,
},
applicationId: application.attrApplicationId,
displayName: 'Index',
});
Q Business retriever
The Q Business retriever is used to read the data from the index when users interact with the Q Business. You must specify the index type (in this example NATIVE_INDEX instead of using Amazon Kendra) and a reference to the index itself.
new q.CfnRetriever(this, 'Retriever', {
type: 'NATIVE_INDEX',
applicationId: application.attrApplicationId,
displayName: 'Retriever',
configuration: {
nativeIndexConfiguration: {
indexId: index.attrIndexId,
},
},
});
IAM policy and role for the Q Business web experience
The Q Business web experience is the frontend for Q Business. It requires permissions to interact with Q Business and to use and create Q Business apps.
const principal = new iam.ServicePrincipal(
'application.qbusiness.amazonaws.com',
{
conditions: {
StringEquals: {
'aws:SourceAccount': this.account,
},
ArnEquals: {
'aws:SourceArn': application.attrApplicationArn,
},
},
},
);
const webExperiencePolicy = new iam.ManagedPolicy(
this,
'WebExperiencePolicy',
{
statements: [
new iam.PolicyStatement({
sid: 'QBusinessConversationPermission',
actions: [
'qbusiness:Chat',
'qbusiness:ChatSync',
'qbusiness:ListMessages',
'qbusiness:ListConversations',
'qbusiness:DeleteConversation',
'qbusiness:PutFeedback',
'qbusiness:GetWebExperience',
'qbusiness:GetApplication',
'qbusiness:ListPlugins',
'qbusiness:GetChatControlsConfiguration',
],
resources: [application.attrApplicationArn],
}),
new iam.PolicyStatement({
sid: 'QBusinessQAppsPermissions',
actions: [
'qapps:CreateQApp',
'qapps:PredictProblemStatementFromConversation',
'qapps:PredictQAppFromProblemStatement',
'qapps:CopyQApp',
'qapps:GetQApp',
'qapps:ListQApps',
'qapps:UpdateQApp',
'qapps:DeleteQApp',
'qapps:AssociateQAppWithUser',
'qapps:DisassociateQAppFromUser',
'qapps:ImportDocumentToQApp',
'qapps:ImportDocumentToQAppSession',
'qapps:CreateLibraryItem',
'qapps:GetLibraryItem',
'qapps:UpdateLibraryItem',
'qapps:CreateLibraryItemReview',
'qapps:ListLibraryItems',
'qapps:CreateSubscriptionToken',
'qapps:StartQAppSession',
'qapps:StopQAppSession',
],
resources: [application.attrApplicationArn],
}),
],
},
);
const webExperienceRole = new iam.Role(
this,
'WebExperienceRole',
{
assumedBy: principal,
managedPolicies: [webExperiencePolicy],
},
);
webExperienceRole.assumeRolePolicy?.addStatements(
new iam.PolicyStatement({
actions: ['sts:SetContext'],
principals: [principal],
}),
);
Q Business web experience
In the web experience, specify the IAM role. No additional configuration is required if you use the defaults.
new q.CfnWebExperience(this, 'WebExperience', {
applicationId: application.attrApplicationId,
roleArn: webExperienceRole.roleArn,
});
IAM policy and role for the Q Business Confluence Data Source
The data source configurations specify which source systems will be integrated with Q Business. For example, it requires permissions to create and delete documents in order to ingest the necessary data into Q Business.
const confluenceDataSourcePolicy =
new iam.ManagedPolicy(
this,
'ConfluenceDataSourcePolicy',
{
statements: [
new iam.PolicyStatement({
sid: 'AllowsAmazonQToGetS3Objects',
actions: ['s3:GetObject'],
resources: ['arn:aws:s3:::bucket/*'],
conditions: {
StringEquals: {
'aws:ResourceAccount': this.account,
},
},
}),
new iam.PolicyStatement({
sid: 'AllowsAmazonQToGetSecret',
actions: ['secretsmanager:GetSecretValue'],
resources: [secret.secretArn],
}),
new iam.PolicyStatement({
sid: 'AllowsAmazonQToDecryptSecret',
actions: ['kms:Decrypt'],
resources: [encryptionKey.keyArn],
conditions: {
StringLike: {
'kms:ViaService': [
'secretsmanager.*.amazonaws.com',
],
},
},
}),
new iam.PolicyStatement({
sid: 'AllowsAmazonQToIngestDocuments',
actions: [
'qbusiness:BatchPutDocument',
'qbusiness:BatchDeleteDocument',
],
resources: [index.attrIndexArn],
}),
new iam.PolicyStatement({
sid: 'AllowsAmazonQToIngestPrincipalMapping',
actions: [
'qbusiness:PutGroup',
'qbusiness:CreateUser',
'qbusiness:DeleteGroup',
'qbusiness:UpdateUser',
'qbusiness:ListGroups',
],
resources: [
application.attrApplicationArn,
index.attrIndexArn,
`${index.attrIndexArn}/data-source/*`,
],
}),
],
},
);
const confluenceDataSourceRole = new iam.Role(
this,
'ConfluenceDataSourceRole',
{
assumedBy: new iam.ServicePrincipal(
'qbusiness.amazonaws.com',
{
conditions: {
StringEquals: {
'aws:SourceAccount': this.account,
},
ArnEquals: {
'aws:SourceArn':
application.attrApplicationArn,
},
},
},
),
managedPolicies: [confluenceDataSourcePolicy],
},
Q Business Confluence Data Source
The data source configuration must be repeated for each data source. It contains a display name, the role created earlier and a configuration (which is still empty).
new q.CfnDataSource(this, 'ConfluenceDataSource', {
applicationId: application.attrApplicationId,
displayName: 'ConfluenceDataSource',
indexId: index.attrIndexId,
configuration: {},
roleArn: confluenceDataSourceRole.roleArn,
});
In my opinion, the configuration attribute is the most complex part of the documentation. It contains the configuration options that are specific to each data source. This means that each data source has its own set of attributes. The attributes are described in a JSON Schema (see CloudFormation documentation).
To get the specific JSON Schema, go to the list of available connectors and select the source system. In chapter "Using the API" you will find the specific JSON Schema, such as the Confluence JSON Schema.
I started writing the configuration with the help of the JSON Schema Validator to be sure that I will match the schema. For the basic fields I managed to specify all the information. When I got to the more complex repositoryConfigurations section with the field mappings, I was looking for a simpler solution.
I decided to use the CloudFormation IaC (infrastructure as code generator) generator for the first time. It can generate CloudFormation code based on manually created resources. I assumed that it would output the entire configuration including the field mappings. Unfortunately, the scan didn't show any Q Business resources because Q Business is not yet supported in the IaC generator.
Since the IaC generator doesn't work for this scenario, I tested the AWS CLI. First, go to the AWS console and open the data source you want to use as a template. The URL will contain three GUIDs that need to be specified in the CLI command:
https://us-east-1.console.aws.amazon.com/amazonq/business/applications/60935d06-f061-4b67-a7ef-6a3e748ed97b/indices/cab1f65f-1666-431c-836a-2e9f8d993c35/datasources/81675834-6e45-4698-aa46-f788b9809e92/details
Paste the GUIDs into the get-data-source command and run the command.
aws qbusiness get-data-source \
--application-id 60935d06-f061-4b67-a7ef-6a3e748ed97b \
--index cab1f65f-1666-431c-836a-2e9f8d993c35 \
--data-source-id 81675834-6e45-4698-aa46-f788b9809e92
As a result, you get the full JSON configuration needed for the configuration (the screenshot only shows the first few lines).
Now I copied this configuration into the configuration attribute. I changed some of the values, such as a dynamic reference to the secret and the host URL. It looks complex, but it contains everything needed to deploy the data source.
new q.CfnDataSource(this, 'ConfluenceDataSource', {
applicationId: application.attrApplicationId,
displayName: 'ConfluenceDataSource',
indexId: index.attrIndexId,
configuration: {
secretArn: secret.secretArn,
syncMode: 'FORCED_FULL_CRAWL',
enableIdentityCrawler: true,
connectionConfiguration: {
repositoryEndpointMetadata: {
type: 'SAAS',
hostUrl: props.confluenceHostUrl,
authType: 'Basic',
},
},
repositoryConfigurations: {
space: {
fieldMappings: [
{
dataSourceFieldName: 'itemType',
indexFieldName: '_category',
indexFieldType: 'STRING',
},
{
dataSourceFieldName: 'url',
indexFieldName: '_source_uri',
indexFieldType: 'STRING',
},
],
},
page: {
fieldMappings: [
{
dataSourceFieldName: 'itemType',
indexFieldName: '_category',
indexFieldType: 'STRING',
},
{
dataSourceFieldName: 'url',
indexFieldName: '_source_uri',
indexFieldType: 'STRING',
},
{
dataSourceFieldName: 'author',
indexFieldName: '_authors',
indexFieldType: 'STRING_LIST',
},
{
dataSourceFieldName: 'createdDate',
indexFieldName: '_created_at',
dateFieldFormat: "yyyy-MM-dd'T'HH:mm:ss'Z'",
indexFieldType: 'DATE',
},
{
dataSourceFieldName: 'modifiedDate',
indexFieldName: '_last_updated_at',
dateFieldFormat: "yyyy-MM-dd'T'HH:mm:ss'Z'",
indexFieldType: 'DATE',
},
],
},
},
type: 'CONFLUENCEV2',
additionalProperties: {
isCrawlPageComment: false,
exclusionUrlPatterns: [],
inclusionFileTypePatterns: [],
isCrawlBlog: false,
blogTitleRegEX: [],
proxyPort: '',
inclusionUrlPatterns: [],
attachmentTitleRegEX: [],
includeSupportedFileType: false,
isCrawlPage: true,
fieldForUserId: 'uuid',
commentTitleRegEX: [],
exclusionSpaceKeyFilter: [],
isCrawlBlogAttachment: false,
isCrawlPersonalSpace: false,
exclusionFileTypePatterns: [],
isCrawlPageAttachment: false,
inclusionSpaceKeyFilter: [],
maxFileSizeInMegaBytes: '50',
proxyHost: '',
isCrawlArchivedSpace: false,
isCrawlArchivedPage: false,
isCrawlAcl: true,
pageTitleRegEX: [],
isCrawlBlogComment: false,
},
},
roleArn: confluenceDataSourceRole.roleArn,
});
At first, I only used the attributes where I specified a value. After getting this error in the AWS console, I specified all attributes, including empty values or false boolean values.
We couldn't sync the following data source: 'ConfluenceDataSource',
at start time Aug 10, 2024, 5:05 PM GMT+2. Confluence Connector error
code: CNF-5133 Error message: The isCrawlPersonalSpace value is
invalid. Reason: isCrawlPersonalSpace should be a boolean value
true or false.
The deployment
The CloudFormation stack deployed successfully - the CloudFormation deployment took 21 minutes (destroying it again took 25 minutes).
I had to start synchronizing the data source manually because I didn't configure periodic synchronization. It is also necessary to assign users manually, as there is no CloudFormation resource for this.
Summary
After understanding the entire configuration, I was able to successfully deploy Q Business using AWS CDK. As a best practice, I can recommend to do the configuration manually for the first time and then use the IAM roles and the data source configuration created in the AWS console as a template. Especially the data source configuration, which is described in a JSON schema, is much easier if you can copy the values from the AWS CLI command. There are some reasonable default values there, such as mapping the source fields, which are hard to define on your own.
This content originally appeared on DEV Community and was authored by Julian Michel
Julian Michel | Sciencx (2024-08-11T20:52:26+00:00) Deploy Amazon Q Business with AWS CDK – example and best practices. Retrieved from https://www.scien.cx/2024/08/11/deploy-amazon-q-business-with-aws-cdk-example-and-best-practices/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.