Skip to content

Commit e885d6e

Browse files
tejasgn1Tejas Ganesh Naik
tejasgn1
and
Tejas Ganesh Naik
authored
Syncing S3 and Knowledge Base with Premium Examples (#7597)
--------- Co-authored-by: Tejas Ganesh Naik <“tejasgn@amazon.com”>
1 parent e758b03 commit e885d6e

File tree

12 files changed

+240
-0
lines changed

12 files changed

+240
-0
lines changed

‎.github/workflows/sync-S3-KB.yml‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
name: syncS3andKB
2+
on:
3+
push:
4+
branches: ["main"]
5+
workflow_dispatch:
6+
inputs:
7+
sdk_name:
8+
description: 'SDK Name'
9+
required: true
10+
default: 'python'
11+
type: choice
12+
options:
13+
- javascriptv3
14+
- dotnetv4
15+
- javav2
16+
- rustv1
17+
- gov2
18+
- swift
19+
- python
20+
- ruby
21+
- php
22+
- cpp
23+
- kotlin
24+
25+
permissions:
26+
id-token: write
27+
28+
jobs:
29+
run_job_with_aws:
30+
runs-on: ubuntu-latest
31+
env:
32+
sdk_name: ${{ github.event.inputs.sdk_name || 'python' }}
33+
34+
steps:
35+
- name: Checkout
36+
uses: actions/checkout@v4
37+
38+
- name: Configure AWS Credentials
39+
uses: aws-actions/configure-aws-credentials@v5
40+
with:
41+
role-to-assume: ${{ secrets.AWS_ASSUME_ROLE }} # once merged, update trust policy of the role to point to main branch
42+
aws-region: us-west-2
43+
44+
- name: Set SDK and language mapping for S3
45+
run: |
46+
if [ "$sdk_name" == "javascriptv3" ]; then
47+
echo "S3_LANGUAGE=javascript" >> $GITHUB_ENV
48+
elif [ "$sdk_name" == "dotnetv4" ]; then
49+
echo "S3_LANGUAGE=dotnet" >> $GITHUB_ENV
50+
elif [ "$sdk_name" == "javav2" ]; then
51+
echo "S3_LANGUAGE=java" >> $GITHUB_ENV
52+
elif [ "$sdk_name" == "rustv1" ]; then
53+
echo "S3_LANGUAGE=rust" >> $GITHUB_ENV
54+
elif [ "$sdk_name" == "gov2" ]; then
55+
echo "S3_LANGUAGE=go" >> $GITHUB_ENV
56+
else
57+
echo "S3_LANGUAGE=$sdk_name" >> $GITHUB_ENV
58+
fi
59+
60+
- name: Extract and copy premium examples in temp. dir.
61+
run: |
62+
MARKDOWN_FILE="./$sdk_name/premium-ex.md"
63+
64+
if [ ! -f "$MARKDOWN_FILE" ]; then
65+
echo "Premium examples file not found: $MARKDOWN_FILE"
66+
exit 1
67+
fi
68+
69+
extract_paths() {
70+
local level="$1"
71+
local section_found=false
72+
73+
while IFS= read -r line; do
74+
if [[ "$line" =~ ^##[[:space:]]*${level}:[[:space:]]*$ ]]; then
75+
section_found=true
76+
continue
77+
elif [[ "$line" =~ ^##[[:space:]] ]] && [ "$section_found" = true ]; then
78+
break
79+
elif [ "$section_found" = true ] && [[ "$line" =~ ^/ ]]; then
80+
echo "$line"
81+
fi
82+
done < "$MARKDOWN_FILE"
83+
}
84+
85+
for level in "basics" "feature-scenario" "complex-feature-scenario"; do
86+
paths=$(extract_paths "$level")
87+
88+
if [ -n "$paths" ]; then
89+
mkdir -p "./extracted_snippets/$level"
90+
91+
while IFS= read -r path; do
92+
if [ -n "$path" ]; then
93+
source_path="./$sdk_name$path"
94+
if [ -e "$source_path" ]; then
95+
cp -r "$source_path" "./extracted_snippets/$level/"
96+
fi
97+
fi
98+
done <<< "$paths"
99+
fi
100+
done
101+
102+
- name: Upload/Sync to S3
103+
run: |
104+
for level in "basics" "feature-scenario" "complex-feature-scenario"; do
105+
if [ -d "./extracted_snippets/$level" ]; then
106+
aws s3 sync "./extracted_snippets/$level/" "s3://$S3_LANGUAGE-premium-bucket/$level/" --delete
107+
echo "Uploaded $level examples to S3"
108+
fi
109+
done
110+
111+
- name: Sync Knowledge Base Data Source
112+
run: |
113+
aws lambda invoke \
114+
--function-name KB_Updater \
115+
--payload "{\"language\":\"$S3_LANGUAGE\",\"region\":\"us-west-2\"}" \
116+
--cli-binary-format raw-in-base64-out \
117+
response.json
118+
119+
echo "Knowledge Base sync initiated"
120+
cat response.json

‎cpp/premium-ex.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
3+
## basics:
4+
/example_code/s3/s3_getting_started_scenario.cpp
5+
6+
## feature-scenario:
7+
/example_code/medical-imaging/imaging_set_and_frames_workflow
8+
9+
## complex-feature-scenario:
10+
/example_code/s3/s3_object_integrity_workflow

‎dotnetv4/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/ControlTower
6+
7+
## feature-scenario:
8+
/CloudWatch/Scenarios
9+
10+
## complex-feature-scenario:
11+
/S3/Scenarios/S3_CreatePresignedPost

‎gov2/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/s3/actions/bucket_basics.go
6+
7+
## feature-scenario:
8+
/s3/actions/bucket_basics.go
9+
10+
## complex-feature-scenario:
11+
/dynamodb/scenarios/scenario_movie_table.go

‎javascriptv3/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/example_code/s3/scenarios/basic.js
6+
7+
## feature-scenario:
8+
/example_code/s3/scenarios/object-locking
9+
10+
## complex-feature-scenario:
11+
/example_code/bedrock-runtime/scenarios/converse_tool_scenario/converse-tool-scenario.js

‎javav2/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/example_code/iotsitewise/src/main/java/com/example/iotsitewise/HelloSitewise.java
6+
7+
## feature-scenario:
8+
/example_code/s3/src/main/java/com/example/s3/lockscenario
9+
10+
## complex-feature-scenario:
11+
/example_code/bedrock-runtime/src/main/java/com/example/bedrockruntime/scenario

‎kotlin/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/services/iotfleetwise
6+
7+
## feature-scenario:
8+
/services/dynamodb/src/main/kotlin/com/kotlin/dynamodb/scenario/ScenarioPartiQ.kt
9+
10+
## complex-feature-scenario:
11+
/services/dynamodb/src/main/kotlin/com/kotlin/dynamodb/scenario/ScenarioPartiQLBatch.kt

‎php/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/example_code/s3/GettingStartedWithS3.php
6+
7+
## feature-scenario:
8+
9+
10+
## complex-feature-scenario:
11+
/applications/photo_asset_manager

‎python/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/example_code/controltower
6+
7+
## feature-scenario:
8+
/example_code/s3/scenarios/conditional_requests
9+
10+
## complex-feature-scenario:
11+
/example_code/medical-imaging/imaging_set_and_frames_workflow

‎ruby/premium-ex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
//Add paths for premium examples here which will be consumed by workflow to update KB with quality code
2+
//Don't forget to add new line at the end of this file
3+
4+
## basics:
5+
/example_code/s3/scenario_getting_started.rb
6+
7+
## feature-scenario:
8+
9+
10+
## complex-feature-scenario:
11+

0 commit comments

Comments
 (0)