ci(github): add container support for self-hosted runners#9
ci(github): add container support for self-hosted runners#9dipankardas011 wants to merge 42 commits intomainfrom
Conversation
3298b6c to
15eae35
Compare
Switch all workflows to use containers on self-hosted runners for improved consistency and isolation. Add system dependency installation steps and configure Docker and Git safe directory where needed. Update job dependencies and permissions for better security and workflow reliability. This enables better compatibility with shared self-hosted environments. chore(ci): use public-repo runner in all workflows Replace the custom self-hosted runner labels with [public-repo] in all GitHub Actions workflow files. This change standardizes the runner environment and improves compatibility for public repositories. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> fix the core volume mount problem Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> fix(ci): update wp-env test config and add --update flag - Remove "plugins" field from .wp-env.test.json to prevent broken mounts - Add dynamic host mapping for plugin in test config - Add --update flag to npm run wp-env:test start commands for fresh env Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com> ci: improve Docker mapping and plugin activation Update workflows to fix wp-env host mapping, add plugin activation steps, and ensure Docker environments are properly started and stopped. Switch PHPUnit commands to use tests-cli for accurate environment handling. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
15eae35 to
5e132e8
Compare
Refactor GitHub Actions workflows to standardize environment variable usage and port assignments for wp-env across build, E2E, and PHPUnit jobs. Removes dynamic COMPOSE_PROJECT_NAME generation and sets static WP_ENV_PORT and WP_ENV_TESTS_PORT values to avoid conflicts on shared self-hosted runners. Updates .wp-env.test.json to include testsPort. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
f8d1bec to
9289434
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
9289434 to
872e914
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Ensure wp-env uses a dedicated /tmp/wp-env directory for all jobs by setting WP_ENV_HOME and mounting it in the container. This prevents permission issues and data conflicts in CI. Updates all workflow steps to use the new environment variable. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
- Increase `timeout-minutes` for build, e2e, and phpunit workflows to 30 minutes for more reliable CI runs. - Set `timeout_minutes` to 15 for Docker environment startup steps. - Add `git config --global --add safe.directory '/tmp/wp-env/*'` to address Git safe directory issues in Docker. - Run E2E tests with `--debug` for better diagnostics. - No functional changes to plugin code. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Update CI workflow to use step outputs for job filtering instead of hardcoded values. This enables more flexible and accurate job execution based on file changes. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
When running in the Docker-out-of-Docker CI setup, the actions/checkout step creates the workspace files as the `root` user. However, wp-env dynamically configures its internal test containers to run as `wpuser` (UID 1001) to match the host user executing `wp-env start`. This caused a permission collision where the test runner (running as UID 1001) could not write test artifacts (like .phpunit.result.cache and code coverage XML/HTML reports) back to the workspace directory owned by root. This adds a step to `chown -R wpuser:wpuser .` across the build, e2e, and phpunit workflows immediately before starting wp-env. This ensures the containerized test runner has the necessary write privileges to generate and upload coverage reports successfully. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
857f1e8 to
ac888b5
Compare
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Add --add-host=host.docker.internal:host-gateway to container options in the reusable-e2e.yml workflow. Update WP_BASE_URL to use host.docker.internal for E2E tests, enabling proper connectivity between Playwright and the WordPress instance when running in Docker. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Use --network=host for the CI container to ensure Playwright shares the host Docker daemon's network. This prevents "ECONNREFUSED" errors when WordPress issues 302 redirects to "localhost", avoiding loopback traps inside the CI container. Also remove WP_BASE_URL env from test command. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Switch CI container from --network=host to --add-host and socat proxy for localhost:8889. This ensures Playwright and WordPress agree on "localhost" and prevents network isolation issues. Adds socat to dependencies and updates E2E test step to forward traffic, improving test stability. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Set Playwright to run in headless mode by default for more consistent and reliable test execution in CI and local environments. This avoids issues with UI rendering and ensures tests do not require a display server. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Removed the --debug flag from the E2E test command in the CI workflow to ensure tests run in standard mode. Updated Playwright config to spread baseConfig.use for better configuration inheritance. These changes improve consistency and maintainability of E2E test execution. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Updated CI workflow to use dynamic outputs from filter steps instead of hardcoded values. Enabled e2e job to trigger based on detection results and phpunit status. Removed outdated comments for clarity. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
justlevine
left a comment
There was a problem hiding this comment.
Not sure what here was authored with human intent, so I'm having some trouble reviewing it. Seems the CI is failing too....
.github/workflows/ci.yml
Outdated
| needs: [detect, phpunit] | ||
| if: needs.detect.result == 'success' && (needs.phpunit.result == 'success' || needs.phpunit.result == 'skipped') && (needs.detect.outputs.php == 'true' || needs.detect.outputs.e2e == 'true' || needs.detect.outputs.js == 'true' || needs.detect.outputs.css == 'true') |
There was a problem hiding this comment.
Were these changes intentional? Why do we need to wait for phpunit to complete?
There was a problem hiding this comment.
wait for phpunit to complete?
we need that becuase we are running on a self-hosted runner having same docker service, which means if 2 services run in parallel then ports can collide! and there are also complications with ensuring the docker compose serviceName and containerName so we need to make it serial execution to avoid complication.
yes fixing the unnecessary deps to streamline it.
There was a problem hiding this comment.
What happens if is skipped, will e2e still run or will it be skipped as well?
There was a problem hiding this comment.
Looks like it never runs. This made debugging super time wasteful tonight... is there a way to allow it to run after phpunit but regardless of if phpunit's status?
There was a problem hiding this comment.
Pull request overview
This PR migrates the repository’s GitHub Actions CI to run inside containers on self-hosted runners, aiming to improve isolation/consistency, while also adjusting wp-env and Playwright behavior to fit the new execution model.
Changes:
- Move reusable workflows and the main CI workflow to self-hosted runners with job-level containers, adding system dependency installation and Git safe-directory configuration.
- Update wp-env test configuration and CI orchestration (ports, tests environment, mappings) and adjust the PHPUnit npm script to run in the tests container.
- Disable Playwright’s automatic
webServerorchestration to avoid port conflicts when CI manages wp-env lifecycle externally.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
playwright.config.ts |
Disables Playwright webServer orchestration and forces headless usage. |
package.json |
Updates PHPUnit command to use tests-cli under the wp-env test config. |
.wp-env.test.json |
Enables tests environment and sets a dedicated tests port. |
.github/workflows/reusable-wp-playground-pr-preview.yml |
Runs preview job in a container on self-hosted runner; installs minimal deps; tightens permissions. |
.github/workflows/reusable-phpunit.yml |
Containerizes PHPUnit job; adds Docker/wp-env setup and cleanup; modifies permissions and orchestration. |
.github/workflows/reusable-phpstan.yml |
Containerizes PHPStan job; adds system deps and safe-directory config. |
.github/workflows/reusable-phpcs.yml |
Containerizes PHPCS job; adds system deps and safe-directory config. |
.github/workflows/reusable-lint-css-js.yml |
Containerizes JS/CSS linting using node:22; adds safe-directory config. |
.github/workflows/reusable-jest.yml |
Containerizes Jest using node:22; adds safe-directory config. |
.github/workflows/reusable-e2e.yml |
Containerizes Playwright E2E job; adds Docker/wp-env setup, Playwright install, socat forwarding, and cleanup. |
.github/workflows/reusable-build.yml |
Containerizes build job; adds Docker/wp-env setup, mapping fixups, and cleanup. |
.github/workflows/ci.yml |
Containerizes detect job, tightens permissions, serializes PHPUnit matrix, and reorders job dependencies (E2E after PHPUnit; build after E2E). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- Switch blueprint JSON mutation from Node.js to jq in the workflow, simplifying dependencies and improving readability. - Add jq to system dependencies in the workflow. - Refactor Playwright config to disable webServer orchestration by destructuring base config, preventing port conflicts and aligning with manual wp-env lifecycle management. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
- Use `ubuntu-latest` instead of custom container in CI workflow - Remove unnecessary system dependency and git safe directory steps - Use `sudo -E -u wpuser` for environment consistency in reusable workflows - Remove explicit port settings from `.wp-env.test.json` - Update `test:php` script to use `cli` instead of `tests-cli` - Improve workflow maintainability and reduce complexity Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Ensure PATH is preserved when running npm and npx commands as wpuser via sudo in GitHub Actions workflows. This prevents issues where npm, npx, or node may not be found due to a restricted environment PATH. Updates all relevant commands in reusable-build, reusable-e2e, and reusable-phpunit workflows. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
This reverts commit cafed50.
Ensure npm is available in PATH for the wpuser by updating .bashrc in build, e2e, and phpunit reusable workflows. This helps prevent issues with npm commands not being found during CI runs. Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
| - name: Activate plugin | ||
| run: | | ||
| sudo -u wpuser env "PATH=$PATH" "WP_ENV_HOME=$WP_ENV_HOME" npm run wp-env:test run cli -- wp plugin activate plugin-skeleton-d | ||
| sudo -u wpuser env "PATH=$PATH" "WP_ENV_HOME=$WP_ENV_HOME" npm run wp-env:test run tests-cli -- wp plugin activate plugin-skeleton-d |
There was a problem hiding this comment.
Is there a reason you added this Activate plugin step? The "mappings" in the *wp-env.test that you removed was intended to keep the plugin disabled, and how the plugin runners currently work.
|
@dipankardas011 is there a way to cache those install commands? Setting up docker, php, reinstalling npm deps... (not composer, since that's handled by our action) |
| contents: read | ||
|
|
||
| env: | ||
| WP_ENV_PORT: 8889 |
There was a problem hiding this comment.
I was able to get rid of the (hopefully) unnecessary WP_TESTS_ENV_PORT everywhere when I fixed the .tests.wp-env.json file . I'm pretty sure we can get rid of this one too but the CI is waaay to slow and I ran out of time to try. What do you think?
As a reminder, we're using wp-env 11.1. so there's 1 encironment in a JSON and it defaults to the prt defined in the JSON or the constant
|
@dipankardas011 I've restored the public gh runners to live alongside these private one, and started on some cleanup regarding the I ran out of time to finish testing because of how slow things are, so please review and finish up what else is needed. As you can see both PHPUnit and Playwright E2E tests are passing on the GH runners, so they should all be passing on the private runner side too. 🙇 Edit: and they are 🚀 So just playground and cleanup/optim left. |
What
Switch all workflows to use containers on self-hosted runners for improved
consistency and isolation. Add system dependency installation steps and
configure Docker and Git safe directory where needed. Update job dependencies
and permissions for better security and workflow reliability. This enables
better compatibility with shared self-hosted environments.