fix(anthropic): add system prompt to span attributes #612
+259 −5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
|
|
WalkthroughA new HAR recording was added for a test verifying system prompt placement in Anthropic instrumentation. The Changes
Sequence Diagram(s)sequenceDiagram participant TestSuite participant Instrumentation participant AnthropicAPI TestSuite->>Instrumentation: Send message request with system prompt and messages Instrumentation->>Instrumentation: startSpan() alt system prompt exists Instrumentation->>Instrumentation: Set prompt[0] to system prompt Instrumentation->>Instrumentation: Set prompt[1..n] to messages else no system prompt Instrumentation->>Instrumentation: Set prompt[0..n] to messages end Instrumentation->>AnthropicAPI: POST /v1/messages AnthropicAPI-->>Instrumentation: Respond with message Instrumentation-->>TestSuite: Return response and span attributes Poem
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
npm error Exit handler never called! 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File ( |
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Important
Looks good to me! 👍
Reviewed everything up to c777386 in 2 minutes and 9 seconds. Click for details.
- Reviewed
290lines of code in3files - Skipped
0files when reviewing. - Skipped posting
3draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/instrumentation-anthropic/recordings/Test-Anthropic-instrumentation_3769946143/should-place-system-prompt-first-for-messages_507270015/recording.har:1
- Draft comment:
Ensure the HAR recording does not include any sensitive data. Verify headers (e.g., API keys) are properly scrubbed. - Reason this comment was not posted:
Confidence changes required:33%<= threshold50%None
2. packages/instrumentation-anthropic/src/instrumentation.ts:220
- Draft comment:
Good use of 'promptIndex' to prepend the system prompt. Consider handling cases where the system value might be an empty string or null, and possibly extracting this logic into a helper for clarity. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The code already handles undefined system prompts through the condition check. Empty strings are handled by the|| ""fallback. The logic is straightforward and contained within a reasonable scope - extracting it would add unnecessary complexity. The comment is suggesting improvements that are either already handled or would make the code worse. I could be wrong about the value of extracting this logic - if this pattern appears multiple times in the codebase, a helper might be valuable. Also, there could be edge cases with system prompt handling that I'm missing. The code is localized to this file and this specific use case. The system prompt handling is already robust with type checking and undefined checks. Making it more complex would reduce readability. The comment should be deleted as it suggests changes that are either already handled or would unnecessarily complicate the code.
3. packages/instrumentation-anthropic/test/instrumentation.test.ts:302
- Draft comment:
Test 'should place system prompt first for messages' correctly verifies system prompt at index 0 and first user message at index 1. Consider also asserting that the assistant message (expected at index 2) is correctly set, and note that using Array.at(-1) may not be supported in all Node versions. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% The suggestion about asserting the assistant message is not critical since the test's main purpose is verifying prompt order, specifically that system comes first. The Array.at() suggestion has merit since it's a relatively new method, but this is a test file in a modern codebase that likely controls its Node.js version. The comment is mixing a non-critical test coverage suggestion with a minor implementation detail. I might be underestimating the importance of complete test coverage. Also, Array.at() compatibility could be a real issue for some users of this library. The test's purpose is specifically about system prompt ordering, not complete message validation. The Node compatibility concern is minor since this is in test code, not production code. The comment should be deleted as it suggests changes that are not essential to the test's purpose and raises minor implementation concerns in test code.
Workflow ID: wflow_emrzu2m4V0899cGH
You can customize 
godwhoa force-pushed the fix-system-prompt branch from c777386 to e01fee4 Compare 
nirga approved these changes Jul 17, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Important
Ensures system prompt is placed first in span attributes for chat messages and adds corresponding tests.
instrumentation.ts.params.systemis defined and non-undefined.should place system prompt first for messagesininstrumentation.test.tsto verify system prompt ordering.recording.harto validate the change.This description was created by
for c777386. You can customize this summary. It will automatically update as commits are pushed.
Summary by CodeRabbit