Skip to content

agentic-merge-upstream.prompt.md

Latest commit

 

History

History
439 lines (315 loc) · 18.7 KB

agentic-merge-upstream.prompt.md

File metadata and controls

439 lines (315 loc) · 18.7 KB

Merge Upstream SDK Changes

You are an expert Java developer tasked with porting changes from the official Copilot SDK (primarily the .NET implementation) to this Java SDK.

⚠️ IMPORTANT: Java SDK Design Takes Priority

The current design and architecture of the Java SDK is the priority. When porting changes from upstream:

  1. Adapt, don't copy - Translate upstream features to fit the Java SDK's existing patterns, naming conventions, and architecture
  2. Preserve Java idioms - The Java SDK should feel natural to Java developers, not like a C# port
  3. Maintain consistency - New code must match the existing codebase style and structure
  4. Evaluate before porting - Not every upstream change needs to be ported; some may not be applicable or may conflict with Java SDK design decisions

Before making any changes, read and understand the existing Java SDK implementation to ensure new code integrates seamlessly.

Utility Scripts

The .github/scripts/ directory contains helper scripts that automate the repeatable parts of this workflow. Use these scripts instead of running the commands manually.

Script Purpose
.github/scripts/upstream-sync/merge-upstream-start.sh Creates branch, updates CLI, clones upstream, reads .lastmerge, prints commit summary
.github/scripts/upstream-sync/merge-upstream-diff.sh Detailed diff analysis grouped by area (.NET src, tests, snapshots, docs, etc.)
.github/scripts/upstream-sync/merge-upstream-finish.sh Runs format + test + build, updates .lastmerge, commits, pushes branch
.github/scripts/build/format-and-test.sh Standalone spotless:apply + mvn clean verify (useful during porting too)

All scripts write/read a .merge-env file (git-ignored) to share state (branch name, upstream dir, last-merge commit).

Workflow Overview

  1. Run ./.github/scripts/upstream-sync/merge-upstream-start.sh (creates branch, clones upstream, shows summary)
  2. Run ./.github/scripts/upstream-sync/merge-upstream-diff.sh (analyze changes)
  3. Update README with minimum CLI version requirement
  4. Identify upstream changes to port
  5. Apply changes to Java SDK (commit as you go)
  6. Port/adjust tests from upstream changes
  7. Run ./.github/scripts/build/format-and-test.sh frequently while porting
  8. Build the package
  9. Update documentation (required for every user-facing upstream change)
  10. Run ./.github/scripts/upstream-sync/merge-upstream-finish.sh (final test + push) and finalize Pull Request (see note below about coding agent vs. manual workflow)
  11. Perform final review before handing off

Step 1: Initialize Upstream Sync

Run the start script to create a branch, update the CLI, clone the upstream repo, and see a summary of new commits:

./.github/scripts/upstream-sync/merge-upstream-start.sh

This writes a .merge-env file used by the other scripts. It outputs:

  • The branch name created
  • The Copilot CLI version
  • The upstream dir path
  • A short log of upstream commits since .lastmerge

Step 2: Analyze Upstream Changes

Run the diff script for a detailed breakdown by area:

./.github/scripts/upstream-sync/merge-upstream-diff.sh          # stat only
./.github/scripts/upstream-sync/merge-upstream-diff.sh --full   # full diffs

The diff script groups changes into: .NET source, .NET tests, test snapshots, documentation, protocol/config, Go/Node.js/Python SDKs, and other files.

Step 3: Update README with CLI Version

After the start script runs, check the CLI version it printed (also saved in .merge-env as CLI_VERSION). Update the Requirements section in README.md and src/site/markdown/index.md to specify the minimum CLI version requirement.

Commit this change before proceeding:

git add README.md src/site/markdown/index.md
git commit -m "Update Copilot CLI minimum version requirement"

Step 4: Identify Changes to Port

Using the output from merge-upstream-diff.sh, focus on:

  • dotnet/src/ - Primary reference implementation
  • dotnet/test/ - Test cases to port
  • docs/ - Documentation updates
  • sdk-protocol-version.json - Protocol version changes

For each change in the upstream diff, determine:

  1. New Features: New methods, classes, or capabilities added to the SDK
  2. Bug Fixes: Corrections to existing functionality
  3. API Changes: Changes to public interfaces or method signatures
  4. Protocol Updates: Changes to the JSON-RPC protocol or message types
  5. Test Updates: New or modified test cases

Key Files to Compare

Upstream (.NET) Java SDK Equivalent
dotnet/src/Client.cs src/main/java/com/github/copilot/sdk/CopilotClient.java
dotnet/src/Session.cs src/main/java/com/github/copilot/sdk/CopilotSession.java
dotnet/src/Types.cs src/main/java/com/github/copilot/sdk/types/*.java
dotnet/src/Generated/*.cs src/main/java/com/github/copilot/sdk/types/*.java
dotnet/test/*.cs src/test/java/com/github/copilot/sdk/*Test.java
docs/getting-started.md README.md and src/site/markdown/*.md
docs/*.md (new files) src/site/markdown/*.md + update src/site/site.xml
sdk-protocol-version.json (embedded in Java code or resource file)

⚠️ Important: When adding new documentation pages, always update src/site/site.xml to include them in the navigation menu.

Step 5: Apply Changes to Java SDK

When porting changes:

⚠️ Priority: Preserve Java SDK Design

Before modifying any code:

  1. Read the existing Java implementation first - Understand current patterns, class structure, and naming
  2. Identify the Java equivalent approach - Don't replicate C# patterns; find the idiomatic Java way
  3. Check for existing abstractions - The Java SDK may already have mechanisms that differ from .NET
  4. Preserve backward compatibility - Existing API signatures should not break unless absolutely necessary
  5. When in doubt, match existing code - Follow what's already in the Java SDK, not the upstream

Commit Changes Incrementally

Important: Commit your changes as you work, grouping related changes together:

# After porting a feature or fix, commit with a descriptive message
git add <changed-files>
git commit -m "Port <feature/fix name> from upstream"

# Example commits:
# git commit -m "Port new authentication flow from upstream"
# git commit -m "Add new message types from upstream protocol update"
# git commit -m "Port bug fix for session handling from upstream"

This creates a clear history of changes that can be reviewed in the Pull Request.

General Guidelines

  • Naming Conventions: Convert C# PascalCase to Java camelCase for methods/variables
  • Async Patterns: C# async/await → Java CompletableFuture or synchronous equivalents
  • Nullable Types: C# ? nullable → Java @Nullable annotations or Optional<T>
  • Properties: C# properties → Java getters/setters or records
  • Records: C# records → Java records (Java 17+)
  • Events: C# events → Java callbacks or listeners

Type Mappings

C# Type Java Equivalent
string String
int int / Integer
bool boolean / Boolean
Task<T> CompletableFuture<T>
CancellationToken (custom implementation)
IAsyncEnumerable<T> Stream<T> or Iterator<T>
JsonElement JsonNode (Jackson)
Dictionary<K,V> Map<K,V>
List<T> List<T>

Code Style

Follow the existing Java SDK patterns:

  • Use Jackson for JSON serialization (ObjectMapper)
  • Use Java records for DTOs where appropriate
  • Follow the existing package structure under com.github.copilot.sdk
  • Maintain backward compatibility when possible
  • Match the style of surrounding code - Consistency with existing code is more important than upstream patterns
  • Prefer existing abstractions - If the Java SDK already solves a problem differently than .NET, keep the Java approach

Step 6: Port Tests

After porting implementation changes, always check for new or updated tests in the upstream repository:

Check for New Tests

cd "$TEMP_DIR/copilot-sdk"
git diff "$LAST_MERGE_COMMIT"..origin/main --stat -- dotnet/test/
git diff "$LAST_MERGE_COMMIT"..origin/main --stat -- test/snapshots/

Port Test Cases

For each new or modified test file in dotnet/test/:

  1. Create corresponding Java test class in src/test/java/com/github/copilot/sdk/
  2. Follow existing test patterns - Look at existing tests like PermissionsTest.java for structure
  3. Use the E2ETestContext infrastructure for tests that need the test harness
  4. Match snapshot directory names - Test snapshots in test/snapshots/ must match the directory name used in ctx.configureForTest()

Test File Mapping

Upstream Test (.NET) Java SDK Test
dotnet/test/AskUserTests.cs src/test/java/com/github/copilot/sdk/AskUserTest.java
dotnet/test/HooksTests.cs src/test/java/com/github/copilot/sdk/HooksTest.java
dotnet/test/ClientTests.cs src/test/java/com/github/copilot/sdk/CopilotClientTest.java
dotnet/test/*Tests.cs src/test/java/com/github/copilot/sdk/*Test.java

Test Snapshot Compatibility

New test snapshots are stored in test/snapshots/ in the upstream repository. These snapshots are automatically cloned during the Maven build process.

If tests fail with errors like TypeError: Cannot read properties of undefined, the test harness may not yet support the new RPC methods. In this case:

  1. Mark tests as @Disabled with a clear reason (e.g., @Disabled("Requires test harness update with X support - see upstream PR #NNN"))
  2. Document the dependency in the test class Javadoc
  3. Enable tests later once the harness is updated

Unit Tests vs E2E Tests

  • Unit tests (like auth option validation) can run without the test harness
  • E2E tests require the test harness with matching snapshots

Commit tests separately or together with their corresponding implementation changes.

Step 7: Format and Run Tests

After applying changes, use the convenience script:

./.github/scripts/build/format-and-test.sh          # format + full verify
./.github/scripts/build/format-and-test.sh --debug  # with debug logging

Or for quicker iteration during porting:

./.github/scripts/build/format-and-test.sh --format-only   # just spotless
./.github/scripts/build/format-and-test.sh --test-only     # skip formatting

If Tests Fail

  1. Read the test output carefully
  2. Identify the root cause (compilation error, runtime error, assertion failure)
  3. Fix the issue in the Java code
  4. Re-run tests
  5. Repeat until all tests pass

Common Issues

  • Missing imports: Add required import statements
  • Type mismatches: Ensure proper type conversions
  • Null handling: Add null checks where C# had nullable types
  • JSON serialization: Verify Jackson annotations are correct

Step 8: Build the Package

Once tests pass, build the complete package:

mvn clean package -DskipTests

Verify:

  • No compilation errors
  • No warnings (if possible)
  • JAR file is generated in target/

Step 9: Update Documentation

Documentation is critical for new features. Every new feature ported from upstream must be documented before the merge is complete. Review and complete this documentation checklist before proceeding to Step 10. If you determine no docs changes are needed, document that decision and rationale in the PR body under a clear heading (for example, Documentation Impact).

Documentation Checklist

For each new feature or significant change:

  1. README.md: Update the main README if there are user-facing changes
  2. src/site/markdown/index.md: Update if requirements or quick start examples change
  3. src/site/markdown/documentation.md: Add sections for new basic usage patterns
  4. src/site/markdown/advanced.md: Add sections for new advanced features (tools, handlers, configurations)
  5. src/site/markdown/mcp.md: Update if MCP-related changes are made
  6. Javadoc: Add/update Javadoc comments for all new/changed public APIs
  7. src/site/site.xml: Update if new documentation pages were added

Documentation Requirements for New Features

When adding a new feature, ensure the documentation includes:

  • What it does: Clear explanation of the feature's purpose
  • How to use it: Code example showing typical usage
  • API reference: Link to relevant Javadoc
  • Configuration options: All available settings/properties

Example: Documenting a New Handler

If a new handler (like UserInputHandler, PermissionHandler) is added, create a section in advanced.md:

## Feature Name

Brief description of what the feature does.

\`\`\`java
var session = client.createSession(
    new SessionConfig()
        .setOnFeatureRequest((request, invocation) -> {
            // Handle the request
            return CompletableFuture.completedFuture(result);
        })
).get();
\`\`\`

Explain the request/response objects and their properties.

See [FeatureHandler](apidocs/com/github/copilot/sdk/json/FeatureHandler.html) Javadoc for more details.

Verify Documentation Consistency

Ensure consistency across all documentation files:

  • Requirements section should match in README.md and src/site/markdown/index.md
  • Code examples should use the same patterns and be tested
  • Links to Javadoc should use correct paths (apidocs/...)

Step 10: Finish, Push, and Finalize Pull Request

Run the finish script which updates .lastmerge, runs a final build, and pushes the branch:

./.github/scripts/upstream-sync/merge-upstream-finish.sh            # full format + test + push
./.github/scripts/upstream-sync/merge-upstream-finish.sh --skip-tests  # if tests already passed

PR Handling: Coding Agent vs. Manual Workflow

If running as a Copilot coding agent (triggered via GitHub issue assignment by the weekly sync workflow), a pull request has already been created automatically for you. Do NOT create a new one. Just push your commits to the current branch — the existing PR will be updated. Add the upstream-sync label to the existing PR by running this command in a terminal:

gh pr edit --add-label "upstream-sync"

No-changes scenario (coding agent only): If after analyzing the upstream diff there are no relevant changes to port to the Java SDK, push an empty commit with a message explaining why no changes were needed, so the PR reflects the analysis outcome. The repository maintainer will close the PR and issue manually.

If running manually (e.g., from VS Code via the reusable prompt), create the Pull Request using gh CLI or the GitHub MCP tool. Then add the label:

gh pr create --base main --title "Merge upstream SDK changes (YYYY-MM-DD)" --body-file /dev/stdin <<< "$PR_BODY"
gh pr edit --add-label "upstream-sync"

The PR body should include:

  1. Title: Merge upstream SDK changes (YYYY-MM-DD)
  2. Body with:
    • Summary of upstream commits analyzed (with count and commit range)
    • Table of changes ported (commit hash + description)
    • List of changes intentionally not ported (with reasons)
    • Verification status (test count, build status)

PR Body Template

## Upstream Merge

Ports changes from the official Copilot SDK ([github/copilot-sdk](https://github.com/github/copilot-sdk)) since last merge (`<OLD_COMMIT>`→`<NEW_COMMIT>`).

### Upstream commits analyzed (N commits)

- Brief description of each upstream change and whether it was ported or not

### Changes ported

| Commit | Description |
|---|---|
| `<hash>` | Description of change |

### Not ported (intentionally)

- **Feature name** — Reason why it wasn't ported

### Verification

- All **N tests pass** (`mvn clean test`)
- Package builds successfully (`mvn clean package -DskipTests`)
- Code formatted with Spotless

Step 11: Final Review

Before finishing:

  1. Run git log --oneline main..$BRANCH_NAME to review all commits
  2. Run git diff main..$BRANCH_NAME --stat to see a summary of all changes
  3. Ensure no unintended changes were made
  4. Verify code follows project conventions
  5. Confirm the branch was pushed to remote
  6. Confirm the Pull Request is ready (created or updated) and provide the PR URL to the user

Checklist

  • New branch created from main
  • Copilot CLI updated to latest version
  • README.md updated with minimum CLI version requirement
  • Upstream repository cloned
  • Diff analyzed between .lastmerge commit and HEAD
  • New features/fixes identified
  • Changes ported to Java SDK following conventions
  • New/updated tests ported from upstream (check dotnet/test/ and test/snapshots/)
  • Tests marked @Disabled if harness doesn't support new features yet
  • Changes committed incrementally with descriptive messages
  • mvn test passes
  • mvn package builds successfully
  • Documentation updated for new features:
    • README.md updated if user-facing changes
    • src/site/markdown/index.md updated if requirements changed
    • src/site/markdown/documentation.md updated for new basic usage
    • src/site/markdown/advanced.md updated for new advanced features
    • Javadoc added/updated for new public APIs
  • If no documentation files were changed for user-facing upstream changes, PR body explicitly explains why documentation changes were not needed
  • src/site/site.xml updated if new documentation pages were added
  • .lastmerge file updated with new commit hash
  • Branch pushed to remote
  • Pull Request finalized (coding agent: push to existing PR; manual: create via mcp_github_create_pull_request)
  • upstream-sync label added to the PR via mcp_github_add_issue_labels
  • PR URL provided to user

Notes

  • The upstream SDK is at: https://github.com/github/copilot-sdk.git
  • Primary reference implementation is in dotnet/ folder
  • This Java SDK targets Java 17+
  • Uses Jackson for JSON processing
  • Uses JUnit 5 for testing
  • Java SDK design decisions take precedence over upstream patterns
  • Adapt upstream changes to fit Java idioms, not the other way around